ZedDist Design Goals

From zedwiki

Jump to: navigation, search

This page summarizes the overarching design goals of the Z39.86-2010 Distribution Framework.

Contents

Background reading

Essential prior publications are:


Notations used on this page

Goal: describes the overarching nature of this goal

Effect: describes the intended effect on stakeholders

Risk: notes any inherent risks/adverse effects

Details: Link to a landing page that discusses the goal and its implementation options in detail

Directive: defines a concrete spec implementation directive, as distilled from landing pages etc

?Directive?: a draft directive, not yet accepted or understood by the WG

(Implementation) Notes: Used for additional info, or when the goal is such a no-brainer that no landing page exists



Features: Evolutionary Targets

Text Media - Better Support for Visual Styling

Goal: Enable reading device implementors to make use of Browser/CSS rendering engines.

Effect: correct rendering of text becomes a free commodity, thus reducing cost-of-implementation for reading devices.

Effect: sighted users can enjoy a visually appealing reading experience, which in some cases will have a direct impact of the accessibility of the material

Effect: commercial publishers can easily produce visually appealing publications, using mainstream formats and tools.

Effect: Closer alignment to ePub XHTML formats

Risk: inflexible/poor semantics. See #Text_Media_-_Flexible_Semantics/Vocabularies.

Details: ZedDist_Text_Grammars

Directive: Used Textual Grammars must be of a nature that enables them to be successfully served as text/html to the 4(?) major browser engines. By "successfully" we mean a valid HTML DOM which is not in "quirks" mode.

Directive: The chosen solution should result in web-level fidelity, not necessarily page-level fidelity.

Text Media - Flexible Semantics/Vocabularies

Goal: make use of text grammars that in terms of vocabulary can adapt to different content natures (see "Explicitly Support Materials Beyond the Book" in ZedNext_Home_Page#Highest_Level_Goals)

Effect: by not locking to a finite vocabulary, the semantics of the textual content can be refined and adapted to varying and new content natures.

Risk: breakage of rendering support. See #Text_Media_-_Better_Support_for_Visal_Styling.

Details: ZedDist_Text_Grammars

Directive: introduce a mechanism to dynamically "plug in" semantic vocabularies, and use those to decorate the given host grammar.

Implementation notes: one concept here is to do the annotations on the attribute axis as to allow the base/host grammar to remain (relatively, at least) static. An option here is to use the role attribute (also used in ZedAI)

Implementation notes: We are dependent on HTML5's distributed extensibility solution, which is not yet defined.

Text Media - Dynamic Extensibility with Generalized Fallback

Goal: allow for extension by introducing foreign grammars to create compound documents, with a reliable fallback mechanism that renders in the default grammar

Effect: XML content types (think MathML, MusicML, ChemML, etc) can be included in the host grammar as "XML Islands".

Effect: By using a generalized principle for inclusion and fallback, implementors need not specialize their implementation for individual island types.

Risk: Interoperability and rendering consistency risk, unless proper and straight-forward fallback mechanisms are put in place. See #Generic_Fallback_Mechanism

Details: ZedDist_Text_Grammars

?Directive?: support for a CDI-type compound document composition mechanism

Directive: support for a CDR-type compound document composition mechanism (i.e. <object> element)

Directive: assure that Text media grammars do not syntactically/legally prevent the future inclusion of producer-provided XBL2 (we will not as it seems be able to reference XBL2 explicitly, just make room for it).

Directive: assure that Text media grammars do not legally prevent the future use of CSS3

Directive: Both host grammars and included CDI or CDR fragments must remain in XML form.

Directive: Make use of built-in fallback for <object> for CDR as an alternative to generic fallback mechanism.

Implementation note: If we do CDI, this is currently in conflict with the goal of serving content as text/html within the HTML5 constraints.

Navigation extends to XML islands

Goal: Allow fine-grained navigation of any structure in a compound document (e.g. mathematical expressions)

Effect: Navigate components and sub-components

Risk:

Details:

Directive: Allow SMIL linking at a deeper level than the XML island root. Use nested structures in the SMIL rendering of the components (as a whole and as individual parts).

(Implementation) Notes:

Text Media - Varying Structural Rigidity

Goal: provide for less well-structured content to be allowed within the DTB fileset

Effect: content that previously has not been DAISY-fiable can now be brought into DAISY form, thereby increasing the amount of content available, and reducing cost of production.

Risk: Ill-structured content brings a risk of a confusing reading experience, and content that does not age well.

Details: ZedDist_Text_Grammars

Directive: Introduce the possibility to define options in terms of which schema(s) Textual Content should be valid to.



Audio Media - Extended Codec Support

Goal: Extend/Modify the supported audio Codecs as compared to Z39.86-2002 and 2005.

Effect: DTB file sets containing audio can be shipped in reduced size and with increased audio quality by selecting a voice-oriented codec

Details: ZedDist CodecRC

Directive: chosen Codecs must have free decoder implementations

Directive: chosen Codecs must support simple positioning (non-VBR)

Directive: chosen Codecs must be platform/vendor independent

Directive: chosen Codecs must be from non-proprietary standards

Directive: chosen Codecs must have fixed-point implementation

Directive: chosen Codecs must have a good level of acceptance in the market

Directive: balance should be found between flexibility and too many Codecs

Implementation notes: we have a longstanding issue regarding proprietary vs open Codecs. Compare the video Codec discussion in HTML5.

DTB Fileset - A Single Container

Goal: Define a container file type to wrap an entire DAISY presentation, with a standard mime-type recognized by web-browsers (e.g. application/daisy+zip), and a standard file extension for application search/filtering purposes.

Effect: Distribution of DAISY file sets becomes less error-laden, and more user friendly.

Details: Container Research Committee findings

Directive: Define abstract container, which details the layout and relation of content within the publication.

Directive: Define allowed physical containers, with consideration to backwards compatibility (e.g. EPUB players expecting ZIP) and alternate use cases (e.g. streaming.)

Directive: the container must have the capability to express metadata about itself that allows it to replace Zed2005 distinfo and satisfies the requirements for generalized fallback mechanism

Implementation note: are we able to break up a single publication across multiple container files within the OCF container spec? Or include multiple publications within a single container file?

Implementation note: Specify use of the OCF (or a compatible extension) as the abstract container (straw man), and for the physical container allow ZIP or TAR.

DTB Fileset - Multiple Textual Content Files

The XML of large textbooks (such as those from the NIMAC) may top 15MBs using z39.86 DTBook markup. And, we've seen some of these textbooks in the wild with over 10000 separate images. Unless the reader software has the ability to load the book via an XML stream, and only some images a time, performance with these books is sluggish on all but powerful hardware.

Goal: Allow multiple content files, with deterministic order.

Effect: Splitting the content file up into multiple fragments will alleviate these issues with large books. It will also make dynamic web distribution of DAISY content (e.g. chapter at a time) more feasible.

Effect: Multiple fragments can be used proactively to achieve a pedagogic effect (i.e. scoping the content to avoid distraction.)

Risk: If fragments are too large, (particularly low end) players may experience delay while rendering the next fragment; this may be a worse user experience than the current longer (but single) delay at the beginning of the book. However, this would be mitigated by moving to a more mainstream markup (e.g. HTML5), for which highly optimized rendering engines exist.

Risk: Search across book becomes more difficult for players to implement. (may not be a risk, given Compiled Search Index goal)

Directive: Ensure container, package, navigation and director components account for possibility of multiple fragments.

DTB Fileset - Compiled search index

Goal: Define a new filetype for the DTB fileset to contain an inverted search index of the content material.

Effect: Reading device implementers need not implement fuzzy search and index algorithms themselves.

Directive: Design this document type so that it is usable by other standards than just DAISY.

Directive: Document type should support stemming (alternative forms of the same word), synonyms, and ranking

DTB Fileset - Machine-readable semantics

Goal: Add the ability to decorate DAISY fileset members (NCX, SMIL primarily) with extensible machine-readable semantics

Effect: Implementing support for semantics-based behavioral adaption in reading devices becomes viable and economic. Smart(er) SMIL behavior adaption implementation cost is much reduced.

Effect: The longstanding problem in NCX that navLists (and maps...) does not have a machine interface in terms of the nature of the targets, is solved.

Directive: Add a mechanism, ideally identical to the one in #Text_Media_-_Flexible_Semantics.2FVocabularies that allows pluggable vocabularies for semantic annotations in SMIL and NCX.


Implementation notes:

  • The role attribute has been added to SMIL3 for this purpose. The same concept can be integrated into NCX
  • The Ontologies being defined in ZedAI are intended to be cross-compatible with ZedDist, and thus implies the usage of role (and thus RDF)

DTB Fileset - Ruby Support (i18n)

Goal: Add support for Ruby markup in those members of the presentation fileset that carry text (content files or NCX labels, for instance)

Effect: DAISY presentations become viable for use in locales where Ruby is required.

@@@TODO: Check the allowed values of 'dir' attribute in HTML5.

DTB Fileset - Unicode support (i18n)

Goal: Ensure support of most recently published Unicode standard, as allowed by XML specification.

Effect: Improved support for various scripts.

Directive: Make use of XML 5th edition.

DTB Fileset - Modularization of NCX

Goal: modularize the NCX to allow specialized adaptions of it to be used in varying contexts

Effect: In the Profiling scenario, different profiles can make use of varying sets of modules from the NCX grammar to suit the intended usage.

Effect: We see continued and effective standards adoption of the NCX; IDPF being a primary example.

Directive: Decompose the notion of the NCX document type into a set of modules and an associated set of composition rules

Directive: Assure that the modularized result allows composing NCX document types that allow maximal terseness and efficiency, to allow usage in resource-constrained contexts (minmizing memory footprint, physical document size, support for simple streaming API access).

Implementation notes: we shall give the IDPF group opportunity to review the drafts.


Update of SMIL to 3.0

Goal: Update the incorporated SMIL to SMIL 3.0 DAISY Profile, extending the features available.

Effect: DAISY's use of SMIL becomes compatible with off-the-shelf SMIL renderers.


Generalized Bookmarking and Annotations Support

@@@TODO: need input from KeithF here

Goal: Generalize the support for shared annotations and bookmarks. It should not be specific to DTBs but should also be usable by EPUB or others.

Effect: Users of DAISY, EPUB and other presentations become able to share their annotations and bookmarks between devices and with other users.

Effect: It becomes possible to commercialize sets of annotations. For example, image descriptions could be added to regular ebooks.

Directive: Revise, rename the current bookmark DTD to meet effects above.

Note: see ZedDist_Bookmarks_and_Annotations

Specific annotation features

Save and load annotations

Goal: User controls saving and simultaneous loading of one or more annotation sets.

Effect: Users can take notes and retrieve them later. Users can view other people's notes.

Risk:

Details:

Directive:

(Implementation) Notes: Seems rather similar to our form submission needs

Annotation input types

Goal: Allow all user-agent- or profile-supported input types as annotation media

Effect: users may annotate publications using video, audio, text, image, etc.

Risk:

Details:

Directive:

(Implementation) Notes:

Comment on annotations and submitted answers

Goal: Users can comment on other annotations

Effect: Teacher can respond to student answers. Supports collaborative annotation.

Risk:

Details:

Directive: Have two reference points per annotation:

1. the main content reference 2. (optional) the response target (if in response to an annotation or answer)

(Implementation) Notes:

Features: Revolutionary Targets

Profiles

Goal: Add support for creating DAISY Distribution Profiles, providing the ability for content to be focused on a particular use case and/or content type.

Effect: Less costly tools can be provided. A user agent or authoring tool may implement support only for one particular profile.

Effect: Specialized tools can be provided, allowing tool providers to niche-in on a certain user group, also allowing a more streamlined user experience.

Effect: Facilitates extension of Zed to new technologies and new media types without need for full-blown spec revision. Encourages outside partners to embrace the standard.

Risk: Proliferation of profiles could cause fragmentation of content and player interoperability.

Details: ZedDist_Profiles

Directive: Define a profile composition framework.

Directive: Define a set of profiles to ship with the standard release.

Directive: Define how new profiles are added to the standard (requiring spec revision or not).

Implementation notes: See also the interrelated Generic Fallback Mechanism.

Generic Profile Fallback Mechanism

Goal: Define a universal fallback mechanism that all distribution profiles must adhere to.

Effect: The advent of various distribution profiles does not affect device compatibility negatively. All devices are able to render all distributed exemplars, albeit sometimes with a reduced feature set exposed to the user.

Effect: Reading device implementors need not support per-profile-specialized fallback mechanisms, but need only have one generic implementation in place. This also means that devices can fall back even when encountering a profile that is completely unknown to them.

Risk: Identifying a baseline could exclude some vendors.

Directive: Define a fallback mechanism that uses a hierarchy of profiles. Note: this implies a requirement on container to support profiles to share media resources.

Implementation notes: The discussion has been about the ability to define multiple alternate filesets at the package file (OPF) level. See ZedDist_Profiles_Universal_Fallback

Video Support

Recommended read: ZedDist_Video_UseCases_Requirements

Note: "VID" stands for "Video In DAISY", and is used as a prefix to discriminate requirements via a hierarchical numbering scheme (based on categories).

Codecs

  • VID_1.1 Default Video Codec
    • Goal: Recommend baseline codec for video-in-DAISY, free of charge and not locked into a patent scheme, with sufficient (but not necessarily the best) rendering quality and authoring/playback capabilities (e.g. streaming, frame-based timing, etc.)
    • Effect: At least one interoperable solution is guaranteed to producers and end-users.
    • Risk: Heated discussions, vendor lobbying (see HTML5 debate).
    • Details: Liaise with ZedDist's audio-codec research group.
    • Directive: Enforce at least one suitable video codec that user-agent must implement.

Navigation

  • VID_2.1 Accurate Frame-based Video Positioning
    • Goal: Precise access to time points in video content enables selective rendering of potentially very short clips (begin-end range).
    • Effect: Lip-reading and sign-language users benefit from correct rendering of animated lips or signing.
    • Risk: N/A
    • Details: see W3C Media Fragments
    • Directive: Ensure that time codes (in SMIL) enable precise access to time points within video content.
  • VID_2.2 Accessible Labeling
    • Goal: Ensure that content is tagged in the publication with not only known semantics but also using multimedia labels.
    • Effect: Users can explore the publication and can configure the content rendering based on their needs and their understanding of the presentation.
    • Risk: N/A
    • Details: N/A
    • Directive: Extend the concept of Resource File to incorporate more multimedia types.

Playback

  • VID_3.1 Volume Control
    • Goal: Ensure that both the author and the user can control volume levels, not only at the presentation level but also for individual streams of aural information.
    • Effect: The overall auditory reading experience is harmonious, using similar loudness across all media channels.
    • Effect: Some users with special needs can adjust volume levels individually to remove distractions and achieve better focus.
    • Risk: N/A
    • Details: N/A
    • Directive: Offer authors a way to define default volume levels for media tracks/channels that require it, and to semantically mark content streams with accessible labels so that functionality can be exposed to users for customization (assuming that user-agents offer a feature to allow users to modify the default or authored values).
  • VID_3.2 Speed/Pitch Control
    • Goal: Ensure that a presentation can globally be slowed-down or sped-up, with pitch correction if possible (tempo change).
    • Effect: Users with special needs can configure their experience via the user-agent, to meet their reading needs.
    • Risk: Implementation complexity.
    • Details: N/A
    • Directive: Offer authors synchronization / timing mechanisms that are globally affected uniformly at presentation time: speed alteration shouldn't break the authored intent.
  • VID_3.3 Layout / Positioning Control
    • Goal: Ensure that the presentation can be defined by the author, and altered by the user if necessary.
    • Effect: Good authors know best how to display visual information, but some users with special needs may want to configure their reading experience via the user-agent.
    • Risk: N/A
    • Details: N/A
    • Directive: Offer authors layout and positioning mechanisms, using relative or absolute coordinates. This authored intent must be semantically tagged and labeled in an accessible manner so that the "meaning" of the medias channels affected by the layout is surfaced to the user-agent (which in turns enables customizations by the user).
  • VID_3.4 Pause/Resume Media Channel
    • Goal: Ensure that a stream can be paused for another one to take priority and play until done.
    • Effect: Users have enough time to consume particular sections of the presentation.
    • Risk: N/A
    • Details: SMIL PriorityClass too complex ? Is EndSync usable ?
    • Directive: Ensure that content can be locally paused and resumed based on the playback of other media channels.
  • VID_3.5 Channel / track selection
    • Goal: Ensure that the various streams of information available in the presentation can be turned on/off by the user, unless their are authored explicitly as primary/mandatory content.
    • Effect: Users can select the media channels they need, which avoids information overload and enables better focus.
    • Risk: N/A
    • Details: Is CustomTest an existing solution ? (what about NCX lists, do we need them here ?)
    • Directive: Offer authors a way to qualify channels of information in terms of their role.

Semantics

  • VID_4.1 Semantic Annotations
    • Goal: Offer solutions to allow authors to expose "meaning".
    • Effect: User-agents can expose functionality and customizations to the user based on the understanding of specific semantics roles.
    • Risk: N/A
    • Details: N/A
    • Directive: Ensure that content can be tagged with semantic annotations, inline or external to the documents in question.

Forms and tests

Document features

Relationship of forms to profiles

Goal: Forms support is a feature. Profile adoption of this feature can be customized to the goals of the profile.

Effect:

1. Form content type must obey DAISY profile fallback rules 2. Profile should define input MIME types (as well as output MIME types)

Risk:

Details:

Directive: Forms is a single atomic module/feature. However, it can be customized regarding supported input types and submission mechanism (although this spec will likely exclude specifiying a submission mechanism).

(Implementation) Notes:

Timing in tests

Goal: Allow per-question and per-exam time limits

Effect: High-stakes tests will be able to enforce time limits

Risk: User agents must be able to override this, otherwise not accessible

Details: Some tests might have no timing

Directive: Time containers should allow author-defined (and user-agent-overridable) time limits. In general the document flow should not have absolute durations.

(Implementation) Notes: For example, if question #2 starts after 5 minutes, this assumes that question #1 is given 5 minutes. The start time should be expressed as an event or as part of the natural document flow rather than with a strict clock value.

Forms and presentation flow

Goal: Input controls should integrate well into the presentation media flow. Form engine should not conflict with SMIL playback.

Effect: Go back and fill out a question again. Repeat individual parts of the question. Get form-driven feedback (hints, errors) in an accessible way (implies a link to SMIL). Pause the SMIL when a user input field is reached, and resume when the user is done filling out their input.

Risk:

Details:

Directive:

(Implementation) Notes: Use SMIL state.

Group contextual presentation components

Goal: Identify connected questions and answers

Effect: Refer to question in its entirety and determine what the question was for a particular response

Risk:

Details: Questions often include

1. presentation of data (graph or paragraph) 2. question 3. answer field

Need to group these three things

Directive:

(Implementation) Notes:

Security and Encryption

Goal: Prevent unauthorized access to tests and test question answers

Effect: No cheating!

Risk: Encryption must not prevent anyone from accessing the document in an accessible way

Details:

Directive:

(Implementation) Notes: Applies to both test documents, user submitted answers, and any transmitted data

Lock portions of exam

Goal: Some exams do not allow test-takers to review their answers once they've passed a certain point

Effect: Users cannot go change their answers if it is no longer supposed to be permitted

Risk: Possible conflicts with normal navigation model

Details:

Directive: Implement document-specified navigation restrictions

(Implementation) Notes: A client-server model would reduce complexity in the user agent.


Dynamic content flow

Goal: Dynamic content flow

Effect: Create adaptive tests where the question sequence depends on answers

Risk: Implementation complexity

Risk: Might allow for creation of confusing navigation models

Details:

Directive:

(Implementation) Notes: Use SMIL State. Or, use a client-server model and reduce the burden on the user agent.

Form validation

Goal: Support client-side form validation, including required fields

Effect: Set data types and mark fields as required or optional. More complex validation includes, for example, requiring more information from the user if they make a particular selection (e.g. "other")

Risk: Not to be used for response evaluation.

Details:

Directive:

(Implementation) Notes: Form validation must play nicely with fallbacks and user input modes.


Submissions and user input

User submissions

Goal: User agent should collect user input

Effect: Able to post or store user answers for analysis

Risk: Security risk

Details: Implementation depends on connectivity (web or local); specification of actual submission format intentionally left out

Details: User experience should not be interrupted by having to hit a submit button for each question. Incremental auto-saving strongly suggested in user agents.

Directive:

(Implementation) Notes:

Incremental submissions

Goal: Store answers along the way and finish work later

Effect: This gives us the ability to use DAISY for longer form-based content such as workbooks

Risk:

Details:

Directive: Submission format must be appendable, reviseable, and must accept being partially filled-out

(Implementation) Notes:

Loading submissions

Goal: User should be able to load a particular set of responses (complete or partial).

Effect: Fill out a form (say a tax form). Then start over and fill out again (say a tax form for someone else). Should be able to choose which set of responses to load the next time.

Risk: User agent complexity

Details: User agent first opens DAISY document and then chooses from a set of answer files (if more than one) or has the option to start from scratch.

Directive:

(Implementation) Notes:

Extensibility of submission format

Goal: The form should be open enough to allow submission data retrieval and aggregation in various ways

Effect: A teacher can request to see all students' answers for a given question

Risk: Scope creep

Details:

Directive: This is not to be specified as part of DAISY beyond saying that the submission format should be flexible

(Implementation) Notes: Maybe use XML. There is some argument for specifying the submission format as part of the DAISY spec.

Allowable types of input

Goal: Accommodate two-way accessibility for open response questions

Effect: For example, sign-language books could accept sign-language answers. Portable audio players could support audio input.

Risk: Not covered by existing form markup languages.

Details:

Directive: Support audio, video, image (drawing, photograph, etc), text as input types

(Implementation) Notes: Universal fallback could be a file upload

Required form controls

Goal: Support educational testing

Effect: Represent existing printed test materials in an accessible format

Risk:

Details:

Directive: Include the following form controls:

1. Multiple choice (single or multiple selection) 2. Open response (see above for different modalities of input controls) 3. Numeric range 4. Button

(Implementation) Notes:

Mathematical input

Goal: Allow users to input mathematical equations

Effect: Higher-level math testing

Risk: Input control complexity for end users as well as user agent implementors

Details:

Directive:

(Implementation) Notes: How do our input modalities work with this?


Input timestamps

Goal: Put timestamps on annotations and submitted answers

Effect: Good for sorting annotations (timeline navigation). Teachers can see how long a student spends on a question or section (time on task).

Risk: Potential for abuse in critical situations (due dates)

Details:

Directive: Harmonize with document metadata date/time representation

(Implementation) Notes:

Domain Targets

Standards Harmonization - EPUB

The IDPF Epub specification has adopted DAISY NCX and DAISY XML. The Z39.86 2002 and 2005 standards has adopted the IDPF Package File (OPF), but remain incompatible with generic Epub because of its continued use of SMIL in Text-Only DTB filesets.

Goal: Achieve full technical compatibility between Epub and DAISY Text-Only filesets.

Effect: End users, publishers and tool providers alike reap the benefits of full device and content compatibility.

Directive: Make the text-only profile of ZedNext fully compatible with EPUB 2.0 (or EPUB-Next, depending on timing)

Implementation notes: Another option is to have ZedNext not define any text-only profile at all, thus delegating completely to IDPF the provision of this format. There are both socio-technical and legal repercussions to this approach.

Retrieved from "http://www.daisy.org/zw/ZedDist_Design_Goals"
Personal tools