Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added progress events to flutter_tts_web using onboundary #526

Merged
merged 5 commits into from
Nov 14, 2024

Conversation

Robert01101101
Copy link
Contributor

@Robert01101101 Robert01101101 commented Sep 9, 2024

Added progress events for Web by adding a listener to SpeechSynthesisUtterance's onboundary event. The fix / added feature is in order to add the missing support for progress events on the Web, I had reported an issue on that earlier today: Fixes #525

This is my first time contributing to an open-source project, I hope I adhered to the best practices 😅

Some notes on the changes:

  • The end index and current word are not provided by the onboundary event, so this solution uses a regex to iterate through the text, starting from the boundary start index, to find the next separator character. I'm not sure if the regex matches the logic used by the browser's TTS, but in my tests it did match.
  • The onboundary events might not exactly match other platform behaviors, for instance, there are separate events for new sentences, which this solution doesn't fire events for.
  • ttsState isn't being set, since it should be set by start, stop etc.

@Robert01101101
Copy link
Contributor Author

After running into some issues with this fix in my build, I want to add some additional notes. While my local debug build worked without issues, once I built for web, there were some limitations with the reliability of when onboundary events are fired.

From what I can tell:

  • Firing of onboundary events might be more reliable on some browsers (found no issues in Edge) than other browsers (Chrome had issues)
  • On affected browsers (Chrome), the issue seems to only appear when certain voices / languages are used. When the default language voice is used, it fires the events as expected, when certain other voices are used, it doesn't fire them at all.
  • Possibly related: https://stackoverflow.com/questions/51645956/speechsynthesisutterance-onboundary-event-not-firing-properly
    • One of the replies here mentions this suggestion for a fix: store the utterances in a variable with global context, but I'm not sure how to do that here, and if it would work. I tried assigning the utterance to a global variable, but that didn't seem to fix the limited support for onboundary on some browsers with some voices.

I'd be happy to do more investigation or work on this proposed change!

@dlutton dlutton self-requested a review November 11, 2024 07:03
Copy link
Owner

@dlutton dlutton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dlutton
Copy link
Owner

dlutton commented Nov 11, 2024

@Robert01101101 looks like there are some conflicts, please resolve when you have a moment so I can merge this in. Also any updates on your investigation?

@Robert01101101
Copy link
Contributor Author

@dlutton I've resolved the merge conflict.

Regarding the investigation, I believe onboundary events are not supported at the moment for non-default voices.

Status: Won't Fix (Obsolete)
The boundary event is not supported for all speech engines.
Native speech synthesis on Mac OS X, Windows, and Chrome OS all support boundary events at the word level.
Network-based speech synthesis, and Android speech synthesis don't currently support these. There's no immediate plans to fix those as the upstream APIs literally don't expose them at all.
There's a feature request filed with Google's speech time to provide word timing information for the speech synthesis API that Chrome uses, but that's not available at this time.

For my use case, I updated my app to default to the native voice in the web build and disabled the option to switch voice. With that, it's working consistently.

My suggestion would be to include a note in the Readme. If you'd like I can take care of that, but I'm not quite sure how to add it to this PR. Can I commit the changes to my branch, push it, and that would update the changes included in this PR? I resolved the merge conflict using GitHub's conflict resolve UI and locally tested code with the merge conflict fix I copy pasted.

@dlutton
Copy link
Owner

dlutton commented Nov 12, 2024

@Robert01101101 yes that would be great if you could update the README. You should be able to simply push to it.

@Robert01101101
Copy link
Contributor Author

@dlutton I ended up adding 3 more small commits:

  • I noticed I forgot to adapt to the new interop types, I updated the code for consistency and tested it to verify it still works.
  • I updated the Readme to list Web support for speech marks, and explain limitations.
  • I noticed the Readme features section had repetition of top level bullet points: 'Android, iOS, macOS, Web, & Windows' and 'Android, iOS, Web, Windows & macOS'. I merged the two.

Let me know if I can do anything else or if there are any issues with this PR. And thanks for creating this package, I find it very useful!

@dlutton dlutton merged commit 37e39a1 into dlutton:master Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Progress Handler not working on web
2 participants