XToys Updates - Jan 28
Added 2022-01-28 18:12:50 +0000 UTCVideo Chat
Well...counter to my previous post, it turns out I am actually setting up an SFU (Selective Forwarding Unit) for enabling multi-person video chat. It was going to be about the same amount of work to re-work the existing webrtc setup as to go the SFU route. And this way video chat will work way better and with more than ~4 people in the session.
After much struggles and testing of different solutions I finally have the server side code largely working how I want (leveraging the LiveKit.io software). I still need to work through all the client side code though so it'll probably be another few weeks.
Text-to-Speech
I added support for using Amazon and Microsoft's text-to-speech voices instead of the browser built-in ones. The Microsoft ones in particular sound really good!
Teases can also now be set up to have each section of dialog be associated with a specific voice. So you can create interactive stories with multiple different voices.
Android 11 Screen Off Issues
Not great news here. Since the XToys app is basically just a webview wrapping the website it has to deal with the limitations of webviews. One of which is that Android attempts to throttle background pages, and that includes stopping the pattern update commands when the screen is off.
The only supported ways of bypassing that limitation is to have the site be playing audio (which works as an ugly workaround), or to have an active webrtc connection. And moving the impacted section of code to be run outside of the webview isn't really feasible.
So I'm not sure yet how this will be fixed.
