Image Description Manual

Images are a crucial part of learning process. Whether it’s a histogram of statistics, demand and supply curve of economics, construction of geometry, map of a location, or portrait of history, an image helps in grasping and memorizing a concept in a convenient and effective way.

But comprehending a diagram or graphic can be challenging for visually impaired or people with cognitive and learning disabilities. However, a meticulously devised description can elucidate an image by shedding light on the layout and other details.

With this feature, we intend to provide a framework wherein the author can supply text and audio describing the image and actually make the image speak.

Various descriptions can be packaged together to address the diverse needs of audience. For example, descriptions can be long or concise, may vary on the level of simplicity, or can be alternative simplified or tactile images.

Please note that Tobi can import both existing full-text, full-audio DAISY book as well as DTBook.xml, enabling insertion of image descriptions in both of them.

Underlying Research

The image description workflow is based on the DIAGRAM Content Model, which is developed under the directions of DIAGRAM Center established by US Department of Education (Office of Special Education Programs).

The DIAGRAM Content Model lays down the specifications for creating alternative verbal or pictorial depictions for the graphical content present in the publications, which can be referenced by the graphical elements in the host languages such as EPUB or HTML.

What are image descriptions?

An image description is a textual, audio or graphical content portraying the image in a representation intelligible by the addressees. The description should be comprehensive as well as perceptible by the target audience. For example, carefully formulated long description and a corresponding tactile image can be useful for the people with visual impairment, whereas a simplified language description and simplified image can decipher the difficult concepts for slow learners.

The author of the description should be fully aware of the needs of the target audience, as it is very essential for the description to be understandable by the people for whom it is created. The guidelines for appositely creating descriptions pertinent to the field of STEM (Science, Technology, Engineering, and Mathematics) can be found on http://ncam.wgbh.org/experience_learn/educational_media/stemdx/guidelines.

Types of image descriptions

Image descriptions can be broadly classified in two categories, namely, textual and graphical. Textual descriptions explicate the layout and other details of the image in words (both typed and spoken), whereas the graphical descriptions are tactile or simplified depictions of the main image.

The following types of textual descriptions can be created in Tobi.

Long description: A comprehensive verbal representation of the image exposing the constitution, structure, and significance of the image. This is the primary description of the image and thus can not be omitted.
Summary: A brief and concise version of the long description giving the central idea.
Simplified Language: Description of the image in basic, easy to understand language.
Annotation: Explanations that are relevant to the subject matter but can not be included in the main description. These may include, but are not restricted to, clarification of some words used in the main description.

In addition to these textual descriptions, the following graphical descriptions may also be supplied.

Tactile image: Alternative image transcribed according to the production method such as swell paper production or stereolithographic printing. This image should be fully formed for output rendering by end users.
Simplified image: The modified version of the main image either removing the intricacies or providing extras such as labels for each component of the graphic. A high-contrast version of the image may also be provided in this category.

Textual descriptions, in entirety, will become a part of full text, full audio DAISY 3 book. At the same time, they will be exported to the DIAGRAM descriptions directory, as described in Files created by the export operation. On the other hand, graphical descriptions will only be exported in the DIAGRAM descriptions directory.

Notes:

Graphical descriptions may also include a textual tour explaining how to explore the alternative image.
Out of all the description types, only long description is mandatory. However, it is recommended to create at least one description for each description type.
There can be more than one description for each description type such as tactile image descriptions varying on the production method or simplified language description differing on the level of simplicity.

Content types supported by Tobi for image descriptions

Although you can equip your description with more than one type of content, the principal content type differs on what category the description falls in. For example, the content type “image” is vital for a tactile image description to exist. As a general rule, all the textual descriptions must have descriptive text accompanying them and all the graphical descriptions should contain an image.

The following table tries to explain the implication of a content type for both textual and graphical descriptions. If a content type is irrelevant for a description type, it is marked with “-“.

Content type	Textual	Graphical
Text	Depiction of the image in words	Text giving the tour of the image.
Audio	Narration of the text supplied in the “text” category	-
Image	-	Alternative tactile or simplified image
Attribute	Properties that describe and provide references for the description	Properties that define and characterize the alternative image

Notes:

The content of text type may include the following:

Plain text
XML markup for block elements such as paragraph (<p>)
XML markup for inline elements like line break (<br>)
MathML

The content of audio type may either be human voice based or generated using synthetic voice.
The alternative image can be in any of the following formats: JPEG, PNG, BMP, GIF, and SVG.

Attributes pertinent to an image description

Attributes of image descriptions describe and characterize them. This section lists some of the attributes that are relevant for an image description.

Attribute name	Attribute value
xml:id	Unique ID for an image description that helps in distinguishing one description from another.
xml:lang	The language used by the author of the description such as “English (U.K.)”.
ref	Points to the ID of another description. For example, for a long description, this field may contain the unique ID of the corresponding annotation.
by	Relationship of the author of the description with the target audience such as teacher. This field may have the value “author”, if the author does not have any such direct relationship with the target audience.
role	The purpose of the image description. For example, “clarification”, if the annotation description is created to clarify; “swell paper production”, if tactile image description is transcribed for swell paper production.
src	Path of the image file for a graphical description
srctype	Type of the source specified in the “src” attribute; for instance, “image/jpeg” for a JPEG image.

These attributes are context relevant. This means that the “src” attribute of the tactile image description will refer to the tactile image when used in the tactile image description, but it will refer to the path of the main image when used in the document.

Relevant GUI components

The process of creating image descriptions involves the following GUI components of Tobi:

Navigation pane (F8)
Document pane (F6)
Editor Window (Ctrl+Shift+D)

Navigation pane (F8)

Navigation pane, as the name suggests, is a panel to navigate to various elements of the document such as headings, pages, images, etc. It is structured in four tabs representing different set of elements; each tab contains a sequential or hierarchical list of objects relating to that element. For example, the Descriptions tab shows the list of graphical elements of the document.

You can navigate among these tabs by pressing Ctrl+Tab. Mouse users can click on the arrows on the right and left side of the horizontal list of tabs, positioned at the bottom of the Navigation pane, to scroll through them and click on the desired tab when revealed.

For image descriptions, you must bring the Descriptions tab in view. It displays the list of all the available images in the document order. Each item of the list comprises of the image name, a preceding visual tick box indicating the presence of a description corresponding to it, and an accessible label stating whether the image is described or not. Please note that visual tick mark and “Described” statement will appear only if a long description has been created for the image.

Selecting an image

You can select an image from the list of available images by one of the following methods:

Traverse through the list sequentially by using Up and Down arrow keys and stop where the desired image name is encountered.
Click on the image name directly, if it is visible. Otherwise, you can bring the desired image in view using the adjacent vertical scroll bar and click on the image when revealed.

Displaying an image

You can display the image by pressing Enter on the selected image or double-clicking on the image name. The image will be displayed in the Document pane.

Adding a description for the image

You can add descriptions to the selected image by either using the Add/Edit button beneath the list of images or by pressing Ctrl+Shift+D. This will open the Editor Window where all the actions pertaining to the creation, modification and removal of image descriptions can be performed.

Position

Navigation pane is positioned on the extreme left corner of the application window.

Document pane (F6)

Document pane exposes all the contents of the document such as text, XML markup, and graphical elements. It is composed of one main window displaying the entire document, and a Document structure trail that displays the predecessor hierarchy of the selected XML element.

Apropos of this, the Document pane also contains graphical elements for which image descriptions should be provided.

Please note that the Document pane is not synchronized with the Navigation pane. This means that traversing through the elements of any of them will have no effect on the other of the two.

Selecting an image

An image can be selected by both mouse and keyboard shortcuts. Mouse users can simply click on the image. If the desired image is not visible, you can bring it in view by using scroll bar on the right side of the Document pane. Keyboard users can use the shortcuts Ctrl+Period to move to the next phrase and Ctrl+Comma to move to the previous phrase.

Adding image descriptions

You can create or modify a description for the selected graphical element by clicking Edit->Add image description/modify descriptions on the menu bar or by pressing Ctrl+Shift+D.

Position

Document pane is towards the right side in the work area of Tobi.

Editor Window (Ctrl+Shift+D)

Editor Window is the dialog box where actual process of creating and editing image descriptions takes place. It can be used in both basic and advanced modes. In the basic mode, it provides a framework to create one description for each fundamental description type by entering the information essential for the respective type of description to exist. On the other hand, the advanced mode can be used to create any number of descriptions in their full fledged forms

You can switch between the basic and advanced view of Editor Window by using Switch to Basic Editor Window/advanced Editor Window button.

Some controls will be present in both basic and advanced modes of Editor Window and are listed below.

Import link for DIAGRAM XML: Imports the contents from the XML file containing the descriptions pertaining to an image. But for the import operation to execute correctly, it is important for the file to be in conformity with the DIAGRAM Content Model.
Export link for DIAGRAM XML: Exports all the descriptions of the current image to an external file.
OK button: Commits the changes to the project and exits the Editor Window. Note here that committing the changes to the project is different from saving the project, which can be achieved by pressing Ctrl+S.
Apply button: Commits the changes to the project without exiting the Editor Window.
Cancel button: Exits the Editor Window discarding the additions or modifications made therein.

Please note that you can move among the controls of Editor Window, such as lists or buttons, by pressing Tab.

Methods of invoking Editor Window

Editor Window can be invoked only if an image is selected, which can be accomplished using Navigation pane or Document pane. After selecting an image, do any of the following:

Click Edit->Add image description/Modify descriptions on the menu bar.
Press Ctrl+Shift+D.
Use the Add/Edit button of the Navigation pane.

Basic Editor Window

The basic view of Editor Window has two main tabs, namely, Text descriptions to create textual descriptions and Alternative images to create graphical descriptions. The Text descriptions tab provides functionality for placing text for fundamental textual descriptions, i.e., long description, summary, and simplified language. While the Alternative images tab enables you to insert a tactile and simplified version of the main image along with their tours. However, you can not add more than one description for any of these description types, nor you can add audio and attributes for any description using the basic view of Editor Window.

The Text descriptions tab is arranged in three vertical groups; each group signifying a description type determined by its label. Every group comprises of an edit box displaying the text entered for the description and will be blank initially, an Edit button to invoke a dialog box to add or modify the textual content of a description, a Clear button to remove the description, and an Audio button to place the audio narration of the description.

The Alternative images tab follows a similar arrangement with two groups; each displaying the corresponding alternative image, an Open… button to browse for the file containing the image, and a Clear button to delete the image. It also enables you to enter a textual tour, for exploring the alternative image, via the Edit button.

An Edit in advanced mode button is provided under each group to facilitate the users in quick and easy insertion of audio and attributes for the description. Clicking on this button will have the following effect:

The Editor Window will switch to the advanced mode.
The respective description will be selected in the Descriptions summary list.
The encapsulated information tabs, where you can place the audio or assign attributes, will display the contents of the selected description.

Like all other tabs, you can alternate between the tabs of the basic view using Ctrl+Tab or by clicking on the tab name.

Adding a textual description

The process of creating all the textual description is the same, so this section will exemplify it using one type of image description, that is, long description.

Bring the Text descriptions tab into view.
Go to the long descriptions group.
Click on the Edit button beneath the edit box that will be blank initially. A dialog box appears that lets you insert the text for the description.
Enter the text for the description, which may be first created using document editing software like WordPad, etc. and then copied into the edit box of newly opened window. You may also type the text directly in the edit box.
Click on the OK button.

Tobi will return to the Editor Window and display the text just entered by you in the edit box immediately below the group label, which is Long description in this case. Please note that you can also include inline, paragraph, and MathML markup in the text for the descriptions.

To insert audio or assign attributes, click on the Edit in advanced mode button, which will open the Editor window in advanced mode and display the contents of the respective description.

Modifying a textual description

You may modify the text any time using the same Edit button and the same successive dialog box used for creating the image description.

Removing a textual description

A textual description can be deleted completely from the project by clicking on the Clear button adjacent to the Edit button under the same group. This will result in removal of the entire content (text, audio, image and attributes) corresponding to the deleted description.

Adding a graphical description

Here the process of adding a tactile image description will be illustrated and the process of creating simplified image description will be along the same lines.

Bring the Alternative images tab into view.
Go to the tactile image group using Tab.
Click on the Open… button. A dialog box appears that lets you browse for the file containing the tactile image.
Select the desired file and click on the Open button to return to the Editor Window. Now the selected image will be displayed in the tactile image group.
To add a tour for the image, click on the Edit button at the bottom of the group. A dialog box opens presenting an edit box, an OK button, and a Cancel button.
Enter the text advising how to explore the tactile image such as “Moving from front to the top right corner…”
Press OK to commit and return to the Editor Window.

The tactile image group of the Editor Window will start displaying the selected tactile image beneath the group label and the textual tour in the edit box above the Edit button.

You can assign attributes to the description by clicking on the Edit in advanced mode button, which will open the Editor Window in the advanced mode and display the contents of the respective description.

Removing a graphical description

You can remove a graphical description specified for the image by clicking on the Clear button next to the Open… button.

Clearing the text entered for tour in graphical description

The text entered in the edit box pertaining to the tour of image can be erased by clicking on the Clear button next to the Edit button. In this case, the description will not be deleted, only the tour will be removed.

Advanced Editor Window

Advanced view of Editor Window encompasses the complete functionality for working on image descriptions. It covers the information concerning the metadata of the image and entire information contained in all the descriptions corresponding to an image. It has two tabs, namely, Metadata to edit the metadata properties of the main image and Descriptions to create and modify descriptions corresponding to the image.

Metadata tab

Using the Metadata tab, you can add, modify, or remove metadata properties of the image. If no property has been defined for the image, only the Add button will be enabled. As soon as you assign a property using the Add button, other controls will get activated.

The following table lists the controls presented under this tab and their functionality.

Control	Control type	Functionality
Metadata properties	List Box	Lists all the properties defined for the image with their values.
Add	Button	Opens a dialog box that lets you select a property name using the “Name” field and specify its value using the “Value” field.
Edit	Button	Opens a dialog box enabling you to modify the name and/or the value of a metadata property.
Remove	Button	Deletes a property from the metadata of the image.

Please note that Tobi allows you to create custom metadata properties also. This can be achieved using the Add button and subsequently entering the name and value for the custom property.

Descriptions tab

This is the place where you can create descriptions for an image in entirety. The descriptions can belong to any description type; even custom description types are allowed. Furthermore, you can create several descriptions belonging to one description type and place all relevant information within using a single view.

As there can be numerous descriptions for an image, it is a good practice to define a unique ID for each description that would be helpful in distinguishing it from other descriptions of the same image. The ID should be consequential in proper identification of the image. For example, “Tactile-SwellPaper” suggests that the description is a tactile image apposite for swell paper production. Please note that description of annotation type must have an ID since annotations are referenced from other types of descriptions.

Initially, only the Add button is enabled that lets you add first description for the image. As soon as you add a description for the image, other controls will get activated that are described below.

Descriptions summary list: A list of descriptions created for the image; each item of the list exhibits the unique ID of the description and its description type.
Add button: Adds a new description corresponding to the image. During this operation, Tobi only asks for the description type and unique ID; other details should be provided in the encapsulated information tabs described below.
Remove button: Removes the selected description from the project.
Encapsulated information tabs: A set of tabs enabling you to supply matter for a description.

Encapsulated information tabs

The Descriptions tab encapsulates a set of tabs, positioned on the right side of Description summary list, that enable you to provide entire information pertaining to a description. There are four tabs in this set, namely, Text, Audio, Image, and Attribute.

The Text tab enables you to enter text for your description, which should describe the image in case of textual description and give a tour for exploring alternative image in case of graphical description. It is of utmost importance for a textual description as it contains the textual representation of the image, imperative for a textual description to exist. In addition to plain text, it may also contain paragraph, inline, or MathML markup. It is comprised of an edit box exposing the contents of the textual information of a description, an Edit button facilitating insertion/modification of text, and a Clear button to remove the textual content of a description.

The Image tab, on the contrary, is vital for a graphical description. However, it is extraneous for textual descriptions such as long description, etc. It consists of an Open… button to select the file containing alternative tactile/simplified image and a Clear button to remove the specified file.

The Audio tab contains narration of text entered in Text tab, which may be human voice recording or generated using Text to speech software (TTS). It presents an Add/Edit button to place the audio or perform audio editing, a Play/Pause button to hear the playback or pause it, a slider indicating the percentage of audio played, and a Clear button to delete the audio content. It should be noted that currently this feature is not supported for the tour of graphical descriptions.

The Attribute tab is useful for describing or characterizing both textual and graphical descriptions. It is not mandatory, but recommended, to provide values for all the relevant attributes of a description. It displays a list of assigned attributes, an Add button to ascribe an attribute, an Edit button to modify the name and/or the value of an attribute, and a Remove button to remove an attribute.

You can reach the encapsulated information tabs using the keystroke - Tab. Thereafter, you can use Ctrl+Tab to switch from one tab to another.

Adding a description

In the advanced mode of Editor Window, the processes of adding textual descriptions and graphical descriptions follow the similar lines. First of all, a description should be initialized by entering the description type and allocating a unique ID. Thereafter, you can insert the text, audio, or alternative image forming the description. But before concluding that the description is complete, you must check the attributes section that should characterize the description for the readers; for example, the value of “role” attribute of tactile image description should notify the production method for which the description is created.

A description can be initialized by following the steps listed below:

Click on the Add button beneath the Descriptions summary list. A dialog box opens inquiring the type of description to be created.
Select the respective description type from the combo box Description type, such as “d:long desc” for long description. If you want to create a description of custom type, simply type the name of the custom type.
Click on the OK button. Another dialog appears, asking for a unique ID.
Enter a meaningful unique ID.
Click on the OK button.

Tobi creates a new description and adds an entry corresponding to it in the Descriptions summary list. When you return to the Editor Window by performing the last step listed above, the newly created description will be selected in the Descriptions summary list and the encapsulated information tabs will point to it.

Next, we should insert textual content or alternative image depending on whether it is a textual description or a graphical description. For a textual description, proceed as follows:

In the Text tab of the encapsulated information tabs, click on the Edit button.
A dialog box opens that lets you enter the textual matter for the description, which can be copied or typed directly in the space provided. Please note that apart from the plain text, MathML, inline, and paragraph markup is also allowed.
After entering and checking the contents, click on the OK button. The text just entered will appear in the edit box under the Text tab. A Clear button will also get activated that enables the users to remove the content any time.
Next, you should add narration of the text just entered, which can be accomplished using the Audio tab. Switch to the Audio tab by pressing Ctrl+Tab or by simply clicking on it.
To add audio, click on the Add/Edit button. Tobi opens the Audio Editor Window facilitating the insertion of audio content and editing it.
You can perform text to speech conversion of the text entered in the Text tab by simply clicking on the Generate TTS button or by pressing Ctrl+G. The voice used for conversion will be the default voice of your system. However, you can alter the defaults by selecting desired voice in the Audio settings… dialog, which can be brought in view by clicking Audio->Audio settings… on the menu bar. If you wish to provide human voice recording, click on the Record button to start recording directly and click on the Stop button when finished.
For inspecting the correctness of the audio content, click on the Play/Pause button to hear the playback. A slider beneath it indicates the playback position in percentage. If there are some errors, they can be rectified using the audio editing functionality of Tobi described in the Tobi User Manual. But if you wish to discard the narration completely, it can be achieved by using the Clear button underneath the Add/Edit button.

If you are working on a graphical description, the following steps will guide your way:

Switch to the Image tab of encapsulated information tabs.
Click on the Open… button. A dialog box opens that lets you browse for the file containing alternative simplified or tactile image.
Select the image file and click on the Open button. The image will be displayed under the tab label. The insertion of an image will also activate the Clear button enabling the deletion of the image.
The next step is to add a textual tour for exploring the alternative image, which can be done using the Text tab. Switch to the Text tab using Ctrl+Tab repeatedly till you land on it.
Click the Edit button therein. Consequently, a dialog box appears showing an edit box, and OK and Cancel buttons.
Type or copy the text informing how to explore the alternative image; for instance “In the upper left corner of the tactile…”
Click on the OK button.

Now that you have inserted the material for the constitution of the image description, you should ascribe attributes using the Attributes tab by following the instructions listed below.

Switch to the Attributes tab.
Click on the Add button. A dialog box opens asking for the name and value of the attribute.
Select the attribute name from the combo box - Name.
Enter the value in the edit box – Value.
Click on the OK button.

The newly assigned attribute will start showing in the Attribute list. You may choose to remove it any time by selecting it from the list and then clicking on the Remove button. The name and/or value of the selected attribute can also be modified by using the Edit button.

Editing a description

You can edit the contents and attributes of a description by selecting it from the Descriptions summary list, and then, performing modifications in the encapsulated information tabs. The Edit and Add/Edit buttons (under the tabs) enable you to edit the text, audio and attributes of the description. The alternative image can be changed by clicking the Open… button, under the Image tab, and selecting the new image file in the dialog box thus opened.

Please note that the encapsulated information tabs are synchronized with the Descriptions summary list. This means that if you select a description in the Descriptions summary list, the encapsulated information tabs will start displaying its contents and attributes.

Removing a description

A description can be completely removed from the project by clicking on the Remove button adjacent to the Add button below the Descriptions summary list. Consequently, all its contents and attributes will also be deleted.

Files created by the export operation

The export operation stores all the data relating to image descriptions in the subdirectories of the export directory. A separate directory is created for each image containing a DIAGRAM XML file, audio files, and alternative image files. The name of the directory indicates which image it refers to and conforms to the format “project name-image name_DIAGRAM_description”.

The DIAGRAM XML file is based on DAISY Authoring and Interchange Framework and contains the following items:

Metadata properties defined for the image
Textual content of textual descriptions
Tour of graphical descriptions
Attributes ascribed to the descriptions

The audio content, containing the narration of a description, is copied to an audio file, in WAV or MP3 format, and there will be a distinct file for each description.

Finally, the files containing the alternative images will be copied in the directory. Tobi will not alter the format of the image files and copy them as it is.

Quick start guide for creating image descriptions

New users of Tobi can go through the step by step instructions for creating image descriptions, which illustrate one method of creating image descriptions in Tobi and can be found at http://www.daisy.org/tobi/ImageDescriptionWorkflow