Image Description Manual
2. Underlying Research
3. What are Image Descriptions?
3.1. Types of Image Descriptions
3.2. Content Types Supported by Tobi for Image Descriptions
3.3. Attributes Pertinent to an Image Description
4. Creating or Modifying Image Descriptions
4.1. Navigation Pane (F8)
4.1.1. Selecting an Image
4.1.2. Displaying an Image
4.1.3. Adding Image Descriptions
4.2. Document Pane (F6)
4.2.1. Selecting an Image
4.2.2. Adding Image Descriptions
4.3. Editor Window (Ctrl+Shift+D)
4.3.1. Methods of Invoking Editor Window
4.3.2. Basic Editor Window
220.127.116.11. Adding a Textual Description
18.104.22.168. Modifying a Textual Description
22.214.171.124. Adding a Graphical Description
126.96.36.199. Removing a Graphical Description
188.8.131.52. Clearing the Text Entered for Tour in Graphical Description
4.3.3. Advanced Editor Window
184.108.40.206. Metadata Tab
220.127.116.11. Descriptions Tab
18.104.22.168.1. Encapsulated Information Tabs
22.214.171.124.2. Adding a Description
126.96.36.199.3. Editing a Description
188.8.131.52.4. Removing a Description
5. Exporting Image Descriptions
5.1. Details of the DIAGRAM Description Directory
5.2. Embedding Image Descriptions in DAISY/EPUB Publication
6. Quick start guide for creating image descriptions
7. Related resources
But comprehending a diagram or graphic can be challenging for visually impaired or people with cognitive and learning disabilities. However, a meticulously devised description can elucidate an image by shedding light on the layout and other details.
With this feature, we intend to provide a framework wherein the author can supply text and audio describing the image and actually make the image speak.
Various descriptions can be packaged together to address the diverse needs of audience. For example, descriptions can be long or concise, may vary on the level of simplicity, or can be alternative simplified or tactile images.
Tobi enables you to supply image descriptions in a full-text, full-audio DAISY 3 book or EPUB media overlays via its comprehensive functionality of creating and modifying image descriptions. It can also import existing full-text, full-audio DAISY book and DTBook.xml enabling insertion of image descriptions in both of them.
The DIAGRAM Content Model lays down the specifications for creating alternative verbal or pictorial depictions for the graphical content present in the publications, which can be referenced by the graphical elements in the host languages such as EPUB or HTML.
The author of the description should be fully aware of the needs of the target audience, as it is very essential for the description to be understandable by the people for whom it is created. The guidelines for appositely creating descriptions pertinent to the field of STEM (Science, Technology, Engineering, and Mathematics) can be found on http://ncam.wgbh.org/experience_learn/educational_media/stemdx/guidelines.
The following types of textual descriptions can be created in Tobi.
- Long description: A comprehensive verbal representation of the image exposing the constitution, structure, and significance of the image. This is the primary description of the image and thus cannot be omitted.
- Summary: A brief and concise version of the long description giving the central idea.
- Simplified Language: Description of the image in basic, easy to understand language.
- Annotation: Explanations that are relevant to the subject matter but cannot be included in the main description. These may include, but are not restricted to, clarification of some words used in the main description.
In addition to these textual descriptions, the following graphical descriptions may also be supplied.
- Tactile image: Alternative image transcribed according to the production method such as swell paper production or stereolithographic printing. This image should be fully formed for output rendering by end users.
- Simplified image: The modified version of the main image either removing the intricacies or providing extras such as labels for each component of the graphic. A high-contrast version of the image may also be provided in this category.
Textual descriptions, in entirety, will become a part of the DAISY or EPUB publication. At the same time, they will be exported to the DIAGRAM descriptions directory, as described in Details of the DIAGRAM Descriptions Directory. On the other hand, graphical descriptions will only be exported in the DIAGRAM descriptions directory.
- Graphical descriptions may also include a textual tour explaining how to explore the alternative image.
- Out of all the description types, only long description is mandatory. However, it is recommended to create at least one description for each description type.
- There can be more than one description for each description type such as tactile image descriptions varying on the production method or simplified language description differing on the level of simplicity.
The following table tries to explain the implication of a content type for both textual and graphical descriptions. If a content type is irrelevant for a description type, it is marked with “-“.
|Text||Depiction of the image in words||Text giving the tour of the image|
|Audio||Narration of the text supplied in the “text” category||-|
|Image||-||Alternative tactile or simplified image|
|Attribute||Properties that describe and provide references for the description||Properties that define and characterize the alternative image|
The following list describes what can be supplied in each content type.
- The content of text type may include the following:
- Plain text
- XML markup for block elements such as paragraph (<p>)
- XML markup for inline elements like line breaks (<br>)
- The content of audio type may either be human voice based or generated using synthetic voice.
- The alternative image can be in any of the following formats: JPEG, PNG, BMP, GIF, and SVG.
|Attribute name||Attribute value|
|xml:id||Unique ID for an image description that helps in distinguishing one description from another.|
|xml:lang||The language used by the author of the description such as “English (U.K.)”.|
|ref||Points to the ID of another description. For example, for a long description, this field may contain the unique ID of the corresponding annotation.|
|by||Relationship of the author of the description with the target audience such as teacher. This field may have the value “author”, if the author does not have any such direct relationship with the target audience.|
|role||The purpose of the image description. For example, “clarification”, if the annotation description is created to clarify; “swell paper production”, if tactile image description is transcribed for swell paper production.|
|src||Path of the image file for a graphical description.|
|srctype||Type of the source specified in the “src” attribute; for instance, “image/jpeg” for a JPEG image.|
For a full view of Tobi’s UI, refer to the tutorial User Interface of Tobi.Navigation pane, as the name suggests, is a panel to navigate to various elements of the document such as headings, pages, images, etc. It is positioned on extreme left and structured in four tabs representing different set of elements; each tab contains a sequential or hierarchical list of objects relating to that element. For example, the Descriptions tab shows the list of graphical elements of the document.
You can navigate among these tabs by pressing Ctrl+Tab. Mouse users can click on the arrows on the right and left side of the horizontal list of tabs, positioned at the bottom of the Navigation pane, to scroll through them and click on the desired tab when revealed.
For image descriptions, you must bring the Descriptions tab in view. It displays the list of all the available images in the document order. Each item of the list comprises of the image name, a preceding visual tick box indicating the presence of a description corresponding to it, and an accessible label stating whether the image is described or not. Please note that visual tick mark and “Described” statement will appear only if a long description has been created for the image.
- Traverse through the list sequentially by using Up and Down arrow keys and stop where the desired image name is encountered.
- Click on the image name directly, if it is visible. Otherwise, you can bring the desired image in view using the adjacent vertical scroll bar and click on the image when revealed.
Apropos of image descriptions, the Document pane also contains graphical elements for which image descriptions should be provided.
Please note that the Document pane is not synchronized with the Navigation pane. This means that traversing through the elements of any of them will have no effect on the other of the two.
- Mouse users can simply click on the image. If the desired image is not visible, you can bring it in view by using scroll bar on the right side of the Document pane.
- Keyboard users can use the shortcuts Ctrl+Period to move to the next phrase and Ctrl+Comma to move to the previous phrase.
You can switch between the basic and advanced view of Editor Window by using Switch to Basic Editor Window/advanced Editor Window button.
Some controls will be present in both basic and advanced modes of Editor Window and are listed below.
- Import link for DIAGRAM XML: Imports the contents from the XML file containing the descriptions pertaining to an image. But for the import operation to execute correctly, it is important for the file to be in conformity with the DIAGRAM Content Model.
- Export link for DIAGRAM XML: Exports all the descriptions of the current image to an external file. This function is independent of Tobi’s main export and thus does not affect the DAISY or EPUB publication.
- OK button: Commits the changes to the project and exits the Editor Window. Please note that committing the changes to the project is different from saving the project, which can be achieved by pressing Ctrl+S.
- Apply button: Commits the changes to the project without exiting the Editor Window.
- Cancel button: Exits the Editor Window discarding the additions or modifications made therein.
Please note that you can move among the controls of Editor Window, such as lists or buttons, by pressing Tab.Navigation pane or Document pane. After selecting an image, do any of the following:
- Click Edit->Add image description/Modify descriptions on the menu bar.
- Press Ctrl+Shift+D.
- Use the Add/Edit button of the Navigation pane.
The Text descriptions tab is arranged in three vertical groups; each group signifying a description type determined by its label. Every group comprises of an edit box displaying the text entered for the description and will be blank initially, an Edit button to invoke a dialog box to add or modify the textual content of the description, a Clear button to remove the description, and an Audio button to place the audio narration of the description.
The Alternative images tab follows a similar arrangement with two groups; each displaying the corresponding alternative image, an Open… button to browse for the file containing the image, and a Clear button to delete the image. It also enables you to enter a textual tour, for exploring the alternative image, via the Edit button.
An Edit in advanced mode button is provided under each group to facilitate the users in quick and easy insertion of audio and attributes for the description. Clicking on this button will have the following effect:
- The Editor Window will switch to the advanced mode.
- The respective description will be selected in the Descriptions summary list.
- The encapsulated information tabs, where you can place the audio or assign attributes, will display the contents of the selected description.
Like all other tabs, you can alternate between the tabs of the basic view using Ctrl+Tab or by clicking on the tab name.
- Bring the Text descriptions tab into view.
- Go to the long descriptions group.
- Click on the Edit button beneath the edit box that will be blank initially. A dialog box appears that lets you insert the text for the description.
- Enter the text for the description, which may be first created using document editing software like WordPad, etc. and then copied into the edit box of newly opened window. You may also type the text directly in the edit box.
- Click on the OK button.
Tobi will return to the Editor Window and display the text just entered by you in the edit box immediately below the group label, which is Long description in this case. Please note that you can also include inline, paragraph, and MathML markup in the text for the descriptions.
To insert audio or assign attributes, click on the Edit in advanced mode button, which will open the Editor window in advanced mode and display the contents of the respective description.
- Bring the Alternative images tab into view.
- Go to the tactile image group using Tab.
- Click on the Open… button. A dialog box appears that lets you browse for the file containing the tactile image.
- Select the desired file and click on the Open button to return to the Editor Window. Now the selected image will be displayed in the tactile image group.
- To add a tour for the image, click on the Edit button at the bottom of the group. A dialog box opens presenting an edit box, an OK button, and a Cancel button.
- Enter the text advising how to explore the tactile image such as “Moving from front to the top right corner…”
- Press OK to commit and return to the Editor Window.
The tactile image group of the Editor Window will start displaying the selected tactile image beneath the group label, and the textual tour will show in the edit box above the Edit button.
You can assign attributes to the description by clicking on the Edit in advanced mode button, which will open the Editor Window in the advanced mode and display the contents of the respective description.
The following table lists the controls presented under this tab and their functionality.
|Metadata properties||List Box||Lists all the properties defined for the image with their values.|
|Add||Button||Opens a dialog box that lets you select a property name using the “Name” field and specify its value using the “Value” field.|
|Edit||Button||Opens a dialog box enabling you to modify the name and/or the value of a metadata property.|
|Remove||Button||Deletes a property from the metadata of the image.|
Please note that Tobi allows you to create custom metadata properties also. This can be achieved by clicking the Add button and subsequently typing both the name and value for the custom property.
As there can be numerous descriptions for an image, it is a good practice to define a unique ID for each description that would be helpful in distinguishing it from other descriptions of the same image. The ID should be consequential in proper identification of the image. For example, “Tactile-Swell Paper” suggests that the description is a tactile image apposite for swell paper production. Please note that it is essential for the description of annotation type to have an ID, since annotations are referenced from other types of descriptions.
Initially, only the Add button is enabled that lets you add first description for the image. As soon as you add a description for the image, other controls will get activated that are described below.
- Descriptions summary list: A list of descriptions created for the image; each item of the list exhibits the unique ID of the description and its description type.
- Add button: Adds a new description corresponding to the image. During this operation, Tobi only asks for the description type and unique ID; other details should be provided in the encapsulated information tabs described below.
- Remove button: Removes the selected description from the project.
- Encapsulated information tabs: A set of tabs enabling you to supply matter for a description.
The Text tab enables you to enter text for your description, which should describe the image in the case of textual description and give a tour for exploring alternative image in the case of graphical description. It is of utmost importance for a textual description as it contains the textual representation of the image, which is imperative for a textual description to exist. In addition to plain text, it may also contain paragraph, inline, or MathML markup. It is comprised of an edit box exposing the contents of the textual information of a description, an Edit button facilitating insertion/modification of text, and a Clear button to remove the textual content of a description.
The Image tab, on the contrary, is vital for a graphical description. However, it is extraneous for textual descriptions such as long description, etc. It consists of an Open… button to select the file containing alternative tactile/simplified image and a Clear button to remove the specified file.
The Audio tab contains narration of text entered in Text tab, which may be human voice recording or generated using Text to speech software (TTS). It presents an Add/Edit button to place the audio or perform audio editing, a Play button to hear the playback, a slider indicating the percentage of audio played, and a Clear button to delete the audio content. It should be noted that currently this feature is not supported for the tour of graphical descriptions.
The Attribute tab is useful for describing or characterizing both textual and graphical descriptions. It is not mandatory, but recommended, to provide values for all the relevant attributes of a description. It displays a list of assigned attributes, an Add button to ascribe an attribute, an Edit button to modify the name and/or the value of an attribute, and a Remove button to remove an attribute.
You can reach the encapsulated information tabs using the keystroke - Tab. Thereafter, you can use Ctrl+Tab to switch from one tab to another.
A description can be initialized by following the steps listed below:
- Click on the Add button beneath the Descriptions summary list. A dialog box opens inquiring the type of description to be created.
- Select the respective description type from the combo box Description type, such as “d:long desc” for long description. If you want to create a description of custom type, simply type the name of the custom type.
- Click on the OK button. Another dialog appears, asking for a unique ID.
- Enter a meaningful unique ID.
- Click on the OK button.
Tobi creates a new description and adds an entry corresponding to it in the Descriptions summary list. When you return to the Editor Window by performing the last step listed above, the newly created description will be selected in the Descriptions summary list and the encapsulated information tabs will point to it.
Next, we should insert textual content or alternative image depending on whether it is a textual description or a graphical description. For a textual description, proceed as follows:
- In the Text tab of the encapsulated information tabs, click on the Edit button.
- A dialog box opens that lets you enter the textual matter for the description, which can be copied or typed directly in the space provided. Please note that apart from the plain text, markup such as MathML, line break and paragraph is also allowed.
- After entering and checking the contents, click on the OK button. The text just entered will appear in the edit box under the Text tab. A Clear button will also get activated that enables the users to remove the content any time.
- Next, you should add narration of the text just entered, which can be accomplished using the Audio tab. Switch to the Audio tab by pressing Ctrl+Tab or by simply clicking on it.
- To add audio, click on the Add/Edit button. Tobi opens the Audio Editor Window facilitating the insertion of audio content and editing it.
- You can perform text to speech conversion of the text entered in the Text tab by simply clicking on the Generate TTS button or by pressing Ctrl+G. The voice used for conversion will be the default voice of your system. However, you can alter the defaults by selecting desired voice in the Audio settings dialog, which can be brought in view by clicking Audio->Audio settings… on the menu bar. If you wish to provide human voice recording, click on the Record button to start recording directly and click on the Stop button when finished.
- For inspecting the correctness of the audio content, click on the Play button to hear the playback. A slider beneath it indicates the playback position in percentage. If there are some errors, they can be rectified using the audio editing functionality of Tobi described in the tutorial Audio Insertion and Editing. But if you wish to discard the narration completely, it can be achieved by using the Clear button underneath the Add/Edit button.
If you are working on a graphical description, the following steps will guide your way:
- Switch to the Image tab of encapsulated information tabs.
- Click on the Open… button, which will result in opening a dialog box enabling you to browse for the file containing alternative simplified or tactile image.
- Select the image file and click on the Open button. The image will be displayed under the tab label. The insertion of an image will also activate the Clear button enabling the deletion of the image.
- The next step is to add a textual tour for exploring the alternative image, which can be done using the Text tab. Switch to the Text tab using Ctrl+Tab repeatedly till you land on it.
- Click the Edit button therein. Consequently, a dialog box appears showing an edit box, and OK and Cancel buttons.
- Type or copy the text informing how to explore the alternative image; for instance “In the upper left corner of the tactile…”
- Click on the OK button.
Now that you have inserted the material for the constitution of the image description, you should assign attributes using the Attributes tab by following the instructions listed below.
- Switch to the Attributes tab.
- Click on the Add button. A dialog box opens asking for the name and value of the attribute.
- Select the attribute name from the combo box - Name.
- Enter the value in the edit box – Value.
- Click on the OK button.
The newly assigned attribute will start showing in the Attribute list. You may choose to remove it any time by selecting it from the list and then clicking on the Remove button. The name and/or value of the selected attribute can also be modified by using the Edit button.
Please note that the encapsulated information tabs are synchronized with the Descriptions summary list. This means that if you select a description in the Descriptions summary list, the encapsulated information tabs will start displaying its contents and attributes.
- It provides an option to embed textual image descriptions in full-text, full-audio DAISY book or EPUB media overlays.
- It exports both textual and graphical image descriptions to the DIAGRAM descriptions directory.
The DIAGRAM XML file is based on DAISY Authoring and Interchange Framework and contains the following items:
- Metadata properties defined for the image
- Textual content of textual descriptions
- Tour of graphical descriptions
- Attributes assigned to the descriptions
The audio content, containing the narration of a description, is copied to an audio file, in WAV or MP3 format, and there will be a distinct file for each description.
Then, the files containing the alternative images will be copied in the directory. Tobi will not alter the format of the image files and copy them as is.
In case of EPUB publications, an XHTML file will also be created. It will contain all the textual and graphical descriptions of the respective image.
But in the case of EPUB publication, Tobi embeds the descriptions in the following ways.
- The described image will be accompanied by its long description. The reading system or the screen reader will prompt the user to view the off-page description by pressing a set of keys. If the description contains audio, the HTML 5.0 browsers will also provide a player for playing it. Please note that the description will be invisible to the visual users.
- If the aria-describedby attribute is included, all the descriptions will be encompassed in an IFrame.
- If the aria-details attribute is included, a link will be provided to the page containing the descriptions corresponding to the image.
The Export dialog for EPUB provides checkboxes to include aria-describedby and aria-details attributes in the EPUB publication. But long descriptions are always attached to the described images.step by step instructions for creating image descriptions, which illustrate one method of creating image descriptions in Tobi.
Overview of the DIAGRAM Content Model: http://diagramcenter.org/standards-and-practices/content-model.html
Standards for adding long description http://diagramcenter.org/standards-and-practices/html-standards.html
DIAGRAM description feature schema: http://www.daisy.org/z3998/2012/auth/features/description/1.0/z3998-description.html
DAISY Authoring and Interchange Framework: http://www.daisy.org/z3998/2012/