List of forums • View recent posts • View recent comments • 
• Go to daisy.org
Where in the sources are the highlighting of text handled?
Hi folks,
I'm browsing the sources to find in which file the highlighting of the text while it is being read is handled.
Anyone here that knows where I should look?
/Andy
Hi Andy, The actual
Hi Andy,
The actual highlighting is done here:
http://daisy-trac.cvsdude.com/amis/browser/amisapp/trunk/amis/src/AmisGuiMFC2/src/gui/TextRenderBrain.cpp#L327
The synchronization of the highlighting with the audio is a slightly more complex process. AMIS relies on events from an internal SMIL playback engine to know when it should render text.
Marisa
highlighting
I'm actually googling the web to find suggestions on how to solve a nice little practical problem to get a good idea to work.
I want some text presented in a browser to be read by a voice and the text in the browser should be highlighted in synch with the voice.
Currently I've found out that my TTS-engine (Acapella currently) can produce word-synch events. Nice. But I think it is worthless in my idea since there are delayes in the network and the user might pause the voice even if I found out a javascript to actually highlight the text. By the way I convert the .wav output to .mp3 with an external program (Lame) to get fewer bytes sent over the network so the events are not in synch with the audio-data anyway.
So I was thinking that there must be a better way...
I've browsed some more hours and found that Adobes Flash format seems to have som synch feature. I've never worked with Flash and want some more alternatives before start working so I browsed some more hours and found something about SMIL and SAMI but haven't read much yet about how or if it could help me.
You or someone else might have some valueable information about that?
Any suggestions on how to get my idea working would be highly appreciated.
/Andy
Sounds interesting -- I think
Sounds interesting -- I think the highlighting can be done pretty simply with javascript. However, the server-side TTS generation sounds like it might be tricky regardless of the synchronization mechanism used. I have no idea what benefits Flash would offer. SMIL is for synchronizing external media objects with each other, and is generally not that useful when dealing with a single media type (in your case, a single source media, text, being rendered in different ways).
Does the TTS have to come from the server? Can you pre-generate as much as possible?
Marisa
The solution should work for
The solution should work for anyone browsing that webpage so the TTS has to be serverside. If someone has listen to the text once it will the mp3-file will be chached for faster access next time it is issued by anyone.
If I have a Javascript I have to pass some synchinfo + the mp3 itself. If the synchinfo comes in a separate stream I think that info is worhtless...
The mp3 will be streamed so I cannot pass the synchinfo first and then count the sounddata arriving at the browser. The synchdata has to be streamed too.
I found a karaoke demo done with SMIL http://tranthuong.free.fr/3TMAN/New/Demo/KaraokeDemos/Karaoke.html
and thought that SMIL would be my hero. But you are right, that solution might need a lot of steps not being possible to do on the fly.
http://freshmeat.net/projects/karaokeconsmil is an open source SMIL-karaoke project. Do you think there is any reason for me to spend time trying to use there sources to get my idea running?
/Andy
I've done a little bit with
I've done a little bit with javascript and SMIL (including media playback event-listening and text highlighting) --
http://code.google.com/p/marisa/source/browse/#hg/parrot
I was using the HTML 5 audio tag, which you might want to consider if you haven't looked at it yet. You can play audio directly in the browser while your javascript controls the synchronization and text highlighting.
If you want to continue off-forum, just email me at marisa.demeglio [at] gmail [dot] com .