Text-to-Speech ElevenLabs Plugin for TouchDesigner with Whisper, ChatGPT, and MediaPipe Integration
Added 2024-05-08 16:57:03 +0000 UTCIn this tutorial, you'll learn how to use the new ElevenLabs text-to-speech plugin for TouchDesigner. We'll also walk through a project creating a voice-activated 3D talking character by integrating Whisper for speech-to-text, ChatGPT for generating dialogue, ElevenLabs for text-to-speech, and MediaPipe for motion tracking to create an interactive experience.
Previous Tutorial on ChatGPT and Whisper: https://www.youtube.com/watch?v=34s2p9gzWhs
Noto the Talking Ball's Tutorial: https://www.youtube.com/watch?v=va6cxORsFMw
MediaPipe Plugin Tutorial: https://www.youtube.com/watch?v=Cx4Ellaj6kk
⚠️NOTE⚠️ If you're using the ElevenLabs.tox in your project you must also include the TDAsyncIO.tox
Comments
Hi Torin,Thank you very much for providing the “Whisper” plugin — it’s really appreciated. I’ve been trying to use it, but even after specifying the audio file path or clicking “Transcribe from file,” it doesn’t seem to run. The plugin just stays in the “loading” state. I can confirm that my API key is working correctly, since the GPT-5 plugin runs without any issues. Could you please let me know how I might resolve this problem? Thank you very much for your help!
Rui Bai
2025-10-15 01:47:22 +0000 UTCHey, I haven't run into this. Does it happen when nothing has been played for a long time?
Torin Blankensmith
2025-08-19 20:21:43 +0000 UTCHi! Occasionally the elevenlabs component produces loud grinding static noises. It's uncommon but can cause trouble. Have you encountered this issue?
i_am_paying_creators
2025-08-19 20:20:51 +0000 UTCactually, I think I need to follow you're older video on speech to text first.
prachilai
2025-03-29 13:28:29 +0000 UTCHey Torin, My whisper component is showing an error 'no such file or directory:" ' inspire of a valid subscription. I copied open API's key here too but didn't work.
prachilai
2025-03-29 13:17:07 +0000 UTCthanks for this tip, i definitely wasted tokens during the first encounter with it ^^ considering the second post below, do you think it could be done, and would it bring real improvements and valuable advantages doing it ?
virusvjvisuals
2024-06-13 19:39:52 +0000 UTCI m using a pack with CoquiTTS, XTTS2, whisper, (python env, with Cuda) alongside lmstudio in local server mode. it s totally free, so no tokens, nor accounts there. and I was wondering if it may by implemented in TD, like replacing elevenlabs api, and also the openai one.. so it could lead to a free use with no more boundaries ... problem is that i m clearly below the TD / python technical level to accomplish such a porting for now.. but the idea is there and i d like to know if it s even possible or of there are some limitations that i m not aware of atm ?
virusvjvisuals
2024-06-13 19:33:57 +0000 UTCI usually turn off the eleven labs component while I'm testing. Make sure to finetine the microphone activation. It seems like it's getting triggered too easy. Also you can always get another 10k tokens by signing up with another email
Torin Blankensmith
2024-06-13 19:26:23 +0000 UTCHey There , same thing , got stuck because of quota exceeding on the elevenlabs account (10k tokens are definitely not enough) the llm was continuously looping on its answers, and each time it formulates your entire question.. so huge amount of tokens were burnt in the process ^^
virusvjvisuals
2024-06-13 19:24:34 +0000 UTCI think their new API requires you to pre pay for the API tokens. Check on the payment section of Open AI
Torin Blankensmith
2024-05-17 14:35:53 +0000 UTCHi Torin! Thanks for your plugin. But do you know why the error keep running "You exceeded your current quota". I upgraded the plan and using ChatGPT 4.
Qianqian Zhou
2024-05-17 14:34:10 +0000 UTCThank you for getting the streaming response for eleven labs in this! I was trying to work on it but didn't have the time to dedicate to getting it knocked out!
7337 Productions
2024-05-13 23:59:22 +0000 UTC
