Project Pipeline 2.0

• Markus Gylling, CTO, DAISY Consortium
• Kenny Johar, Enterprise Architect, Vision Australia
• Romain Deltour, Software Architect, DAISY Consortium

Editor: George Kerscher
Last revised June 23, 2009

Executive Summary

The Pipeline team is currently pondering and discussing with potential partners the proposal of a Pipeline 2.0 project. This was submitted to the DC board in June 2009, for consideration with an official launch in mid 2010. This paper summarizes the core reasons for doing so, and outlines a two year timeline and cost estimation.

If eventually approved and funded, the project would serve as:

• a next-generation platform for automated transformation services within the DAISY community;
• a reference implementation of the DAISY 4 revision;
• a collection of transformation utilities for use in commercial as well as non-commercial application contexts.

Rationale

We believe that the need for automated transformation services in the DAISY community will keep on increasing rapidly over the coming decades.

It is therefore crucial, as part of a longer term service provision strategy, to assure that we build on technologies and standards that maximize our outreach and compatibility.

Adapting to and embracing emerging technologies

Since the original Pipeline application was created, new standards and technologies have emerged that fundamentally have the same functional scope as the Pipeline. (Primary examples include W3C XProc, XSLT 2.0 and OSGi ). [i]

Longer term, it does not make economic sense to maintain our own version of the Pipeline core engine, since standards (and off-the-shelf implementations of those standards) with the same scope are now available.

By building on existing standards and technologies, we would minimize our own maintenance burden, increase the interoperability with the heterogeneous community needs, and lower the framework learning curve.

By using standard protocols, we can potentially interface easier with mainstream publishers for example.

Focus on integration in existing production contexts

A Pipeline 2.0 project would also be an opportunity to assure that the integration of the Pipeline into existing production contexts is made as smooth and efficient as possible.

Here, Pipeline 2.0 would make direct use of the PipeOnline web application developed during 2008 and 2009 in collaboration with NLB, as well as the PipelineWS web service layer developed in collaboration with Vision Australia during 2009. [ii]

Reference implementation of the Z39.86-2010 (“DAISY 4”) revision.

Pipeline 2.0 would also function as a reference implementation of the Z39.86-2010 revision. In this regard, the project would provide a series of essential transformation services to make DAISY 4 XML files compatible with existing and emerging production systems.

Prerequisites

The following prerequisites for going further with a formal project proposal have been identified so far:

• Established partnerships. A core prerequisite is that we establish collaborative relationships (expressed as funding and/or human resources) with partners (DAISY members, friends, and/or external partners) before formal project onset. If this fails, we should deem interest in the project insufficient, and cancel it.
• Continued liberal licensing. To assure that the application as a whole, as well as individual components of it, can be adopted and evolved by DAISY Friends and other commercial ventures, the currently used liberal licensing scheme, or a carefully chosen equivalent, should be used.

Intended effects

The following list summarizes the intended effects of the project:

• The redesign of the core engine to use standard protocols and web-services, and the provision of a new set of user interfaces, simplifies integration into the heterogeneous infrastructural contexts that constitute the current DAISY community. We record increased uptake in both developed and developing countries.
• The increased use of established standards makes it easier for partners to get involved and contribute to the project. We successively record an increasing amount of partners and contributors.
• By reusing existing implementations of existing standards (most notably existing implementations of XProc) in the core engine, we record a reduction of both initial development and eventual maintenance cost. Developers can focus more on actual transformations rather than the engine that drives the transformations.
• The increased use of standardized protocols makes it easier to adopt the Pipeline in other application contexts. Coupled with a continued liberal licensing, this means that we record increased re-use in commercial applications.
• By providing a set of transformation utilities supporting DAISY 4 early in the lifespan of the standard, we lower the entry threshold and speed up the adoption rate of said standard.

Draft timeline

The following table shows the major envisioned milestones, assuming (which is all we can do at this point) a two year total project time. The project would focus on delivering a first usable release early on (within 6 months after project onset), followed by several successive releases during the project time-span.

June 2009	Project outline to Board
Late June/early July 2009	Initial community announcement for participation
August-October 2009	Technology evaluation, Functional Requirements, Architecture Modelling
November 2009	Proposed Charter and Formal call for support
April 2010	Charter approval by Board
May 2010 - June 2012	Development time-span with successive iterative releases

 

Resource Requirements

The following outlines the resource requirements for an assumed two year project time-span. Depending on the final approved project and its scope the costs would be higher or lower.

Note: the outline includes only formal contributions from the DC and partners, assigned at project onset, and thus exclude additional contributions made during the project time-span.

Resources in Entities	Year 1	Year 2
Human Resources
Project lead/System Architect	1.0 FTE	1.0 FTE
Administration	0.2 FTE	0.2 FTE
Software Developers	3.0 FTE	2.0 FTE
Technical Writing	0.3 FTE	0.3 FTE
Testing	0.3 FTE	0.3 FTE
Integration and Training	0.1 FTE	0.2 FTE
Communication and Marketing	0.1 FTE	0.2 FTE
Direct Costs
Face to Face Meetings	2	1
Software and Hardware	div	div
Integration and Training	div	div
Communication and Marketing	div	div
Miscellaneous	div	div

 

Based on these estimations, the total costs for the project will be around 850K USD that will be payable in 2010 (270K), 2011 (420K), and 2012 (160K).

Participant skill sets

Software developers participating in the project need to possess one or both of the following skill sets. Ability to communicate using English in spoken and written form is a general requirement.

XSLT / XML

• extensive knowledge of XML technologies in general, and XSLT 1.0 and/or 2.0 transformation development in particular
• familiarity with structure and design of common XML grammars (such as DocBook, XHTML and TEI)
• understanding of XSL modularization design patterns

OO / SOA / Web services

• extensive knowledge of object oriented software development and design patterns
• practical experience with development using relevant SOA and Web Service technologies
• ability to engage in SOA-oriented design, architecture and implementation efforts

 

---

Footnotes

1. A more detailed technical discussion of architectural rationale and approaches is available at:
   http://daisy-trac.cvsdude.com/pipeline/wiki/NextGeneration/Proposal.
   
2. We are asserting that a web-based approach to transformation services are the way of the future. While a Pipeline 2.0 project would provide desktop user interfaces (and especially “Pipeline Lite”-like interfaces for use in MS Word and Open Office for example), the intent would be to initially emphasize web services as the primary way to use the application.