Presentation: Adding Math to Digital Talking Books

George Kerscher, DAISY Consortium

Dennis Leas, gh, LLC

Neil Soiffer, Design Science, Inc.

DAISY

Vision: All information that is published should be accessible:

• at the same time
• at no greater cost
• in a highly functional feature-rich format

Based on open, non-proprietary W3C standards such as SMIL, XHTML and XML.

1998

DAISY 2.0

2001

DAISY 2.02, the DTB format most commonly used worldwide today

2002

ANSI/NISO Z39.86 (DAISY 3, DAISY/NISO), Specifications for the Digital Talking Book

2005

Second version of ANSI/NISO Z39.86 released, coined the DAISY/NISO Standard

Exactly What is NIMAS?

It is...

• An application of the DTBook element set of the DAISY/NISO Standard
• XML
• A collection of digital files that provides an accessible representation of the printed book for blind, visually disabled, and print-disabled users

NIMAS is eloquently described on the CAST Web site as follows:

NIMAS guides the production and electronic distribution of digital versions of textbooks and other instructional materials so they can be more easily converted to accessible formats, including Braille and text-to-speech.

The key points about NIMAS are:

accessibility

limitations & barriers print poses will be eliminated

flexibility

different people have different reading needs

subset of DAISY

does not provide richness of DAISY, but does at least provide for consistant markup; full DAISY can be requested

• Navigation Center: provides the global navigation mechanisms
• DTBook: XML textual representation of the complete book-- NIMAS Points to this
• Compressed audio files, such as MP3, that allow up to 40 hours on a single CD
• Images of photos, flow charts, pie graphs, etc
• Synchronized Multimedia Integration Language (SMIL) files to coordinate text, audio, and images
• Resource file: helps reading systems present more information
• Bookmark file: for notes, bookmarks, audio notes, etc
• Package file: packing list of all the files

Math in DAISY

• DAISY/NISO and previous versions allowed only images with alt text
• DAISY/NISO allows for "modular extensions" (think toppings on a pizza)
• The first topping (math) was approved by the DAISY Board February 23, 2007

• Working group was formed in 2005
• MathML chosen because:
• • Can be converted to speech
  • Can be converted to braille math codes
  • Already lots of MathML tools
• First Public Draft: Summer, 2006
• Second Public Draft: December, 2006
• Final approval: February, 2007
• Public announcement: CSUN, 2007

Authoring Math in DAISY

• First extension to DAISY/NISO
• Extension needed to be valid DAISY/NISO
• Needed to decide upon & fallback behavior
• Ultimately decided on two levels: Advanced & Basic

Math and DAISY Players

Advanced

Players that directly read the MathML convert it for audio and/or display

Basic

Players that rely on supplied audio, alternative text, and/or image for the math

<p>
 <sent id="cn0004" smilref="nativemathml.smil#tcp0005">
   Following this dtbook paragraph is a mathml island.</sent>
 <sent id="cn0005" smilref="nativemathml.smil#tcp0006">
   Presentational markup is used.
 </sent>
</p>
<m:math xmlns:dtbook="http://www.daisy.org/z3986/2005/dtbook/"
  id="math0001" dtbook:smilref="nativemathml.smil#math0001"
  altimg="nativemathml0001.png" alttext="cube root of x">
  <m:mroot>
    <m:mi>x</m:mi>
    <m:mn>3</m:mn>
  </m:mroot>
</m:math>

<seq id="math0001" class="mathExt" end="DTBuserEscape;math-par.end">
  <par id="math-par">
    <text src="nativemathml.xml#math0001"
          type="http://www.w3.org/1998/Math/MathML"
          id="mml0001"/>
    <audio src="nativemathml0001.mp3"
           id="math-audio0001"
           clipBegin="00:00:01.539"
           clipEnd="00:00:12.082"/>
  </par>
<seq>

Now forget you ever saw this...

• Allow synchronization
• Hook up IDs between DTBook and SMIL files
• Generate all the other files
• Producers/Converts
  • APH — Book Wizard Producer, Studio Recorder
  • Dolphin — Publisher, EaseProducer (Word)
  • Phoneticom — Phoneticom DAISY Generator
  • Plextalk — Portable Recorder, Recording Studio Pro
  • DAISY Pipeline

An updated list of production software can be found at http://www.daisy.org/tools.

• Authoring documents with accessible math using Word/MathType
• • Think about the math, not the MathML
  • No special work required to make the math accessible
• XHTML is similar to DTBook file

• MathType--most widely used math editor for Microsoft Word
• WebEQ--Java-based MathML editor
• Scientific Notebook--WYSWYG doc editor with Math support
• Open Office--an open source office suite
• MathFlow for ArborText--a MathML editor and converter for ArborText's XML editor
• TtM, TeX4ht, LaTeX2HTML--conversion from TeX/LaTeX to HTML/MathML
• Mathematica, Maple, MathCAD, TI-89/92+--mathematical computation systems

An updated list of MathML software can be found at www.w3.org/Math/implementations.html

Actually a variety of players, but we can simplify to two species with the following characteristics:

• Audio Only - aka "hardware player"
  • Typically highly mobile system
  • Most common type
  • Recorded audio
  • More limited navigation
  • SMIL file centric
• Text Capable - aka "software player"
  • Typically desktop system
  • Text-to-speech capability
  • Fine-grain, word-level navigation
  • Extendable to "intelligent reading system"
  • DTBook file centric

From the viewpoint of MathML support, we now have three types of players

• MathML-unaware - Of course, all current players are MathML-unaware.
  • Audio Only
    • All Current DAISY/NISO 2005 players will play MathML-extended DTBs
    • SMIL file is essentially unchanged
    • Math rendered via recorded audio
  • Text capable
    • Current players WILL NOT render MathML properly
    • MathML-extended DTBs contain new tags in DTBook file
    • Current players must be updated to recognize MathML-extended DTBs
• Basic MathML Player
• Advanced MathML Player

All MathML-aware players must recognize the math tag in the DTBook file. However, player support for MathML can vary greatly. We recognize two major categories.

• Basic MathML player
  • Does not incorporated native understanding of MathML structure and semantics
  • Relies on fallbacks to provide an alternate rendering to the user
  • Math displayed as static image
  • Math read as static text
• Advanced MathML player
  • Native understanding of MathML structures and semantics
  • Able to render and navigate within the MathML content intelligently
  • Math image created dynamically
  • Reader able to adjust how math is read

• Easy to implement for player manufacturers
• Simple for player system
  • Doesn't require fast computer
  • Doesn't require much memory
• Math image and text produced when book is authored
• Similar in presentation to current players

• Math image is static and cannot be changed by Reader
• • Jaggy when zoomed
  • Doesn't change colors
• Reader cannot navigate to individual parts of math, since it is one unified image
• Math text is static and cannot be adapted to Reader's preferences
• Many of the same disadvantages we have now

• Math image produced when book is played
  • Image zooms smoothly
  • Image colors properly
• Math text produced when book is played, can be customized to Reader's preferences such as wording and language
• Preserves individual parts of math, permiting fine-grain navigation
• Highly adaptable to Reader's preferences, supports "dynamic overview"

• More difficult for player manufacturer to implement
• More processing required by player system
• • More processor resources - speed and memory
  • Additional software to recognize MathML
• May be more difficult to use for beginning math students

The gh PLAYER is a full-featured software DAISY player. Here is a demonstration of the upcoming version 2.2, which incorporates MathML recognition. Note that this is a pre-beta version.

• Basic MathML mode - completely implemented
• Advanced MathML mode - partial implementation

The MathML Extension supports a wide range of player behaviors. This provides a choice of price/performance to the Reader.

• As newer players expanded their capabilities, the same MathML-extended DTB provide a richer Reader experience.
• MathML-extended DTBs will retain their value to publishers
• MathML-extended DTBs will retain their value to Readers

MathML for everybody is better, but it is essential for persons with print disabilities. What can you do to bring this about?

• Demand that all legislation require MathML markup in the files used for accessibility.
• Demand that the DSS office makes MathML content available in your books.
• Demand from organizations that provide accessible books to use MathML in their DAISY books.
• Demand support for MathML in authoring tools that you purchase.
• Demand intelligent reading of MathML in the software you purchase.

Build the darn book with true MathML!