ChatGPT Plugin for TouchDesigner supports GPT-4o + Vision
Added 2024-05-13 21:11:50 +0000 UTCI'm very excited to launch this new plugin that supports OpenAI's latest GPT-4o model which you can read more about here.
This new plugin update also includes the ability to have GPT process images, which you can easily turn on and off with each new message via the 'Upload Image With Message' toggle. You can attach any TOP to the component and the second you press 'Generate Response' it will snag the current frame from the input TOP and pass it along to GPT along with your message.
Here's a link to my previous tutorial that shows how to get an API key and set up your project (new tutorial coming soon)
Here's another tutorial showing how to combine the plugin with other AI models to create a voice-activated talking 3D character that also follows the viewer's movement.
If you're using the vision feature don't forget to clear your chat history eventually, because it will keep the images encoded as text inside the component, which can take up a decent amount of space over time.
⚠️ Don't forget to include the TDAsyncIO.tox in your same project to allow the component to run⚠️
Comments
Is it possible to refer to an online chat with chatgpt. or export/import a chat into touchdesigenr?
Phil Jungschlaeger
2025-01-02 19:30:40 +0000 UTCJust uploaded :) https://www.patreon.com/posts/118027526
Torin Blankensmith
2024-12-15 18:11:44 +0000 UTCHi! Thank you for the incredible work. Just a question: would it be possible to use the 4o-mini model instead of the "full" 4o?
Filippo Mirandola
2024-12-14 17:44:32 +0000 UTCIt's working now. I think the issue is probably caused by the td version I used. Thanks
Viola Liang
2024-11-25 15:34:06 +0000 UTCThanks for your response! I’m currently working with the OpenAI.170.toe project file.
Viola Liang
2024-11-24 19:49:28 +0000 UTCOk sweet, which project file are you using?
Torin Blankensmith
2024-11-24 19:34:52 +0000 UTCHi Torin, I’m using macOS 15.1.1 and TouchDesigner version 2023.11220. I’m working with a .toe project file. The other parts of the project are working pretty well. I’d really appreciate your guidance on how to make the whisper component work. Thanks
Viola Liang
2024-11-24 19:08:14 +0000 UTCWhat OS are you on, and what version of TouchDesigner? Also are you using a .toe project file, or are you dragging the whisper.tox into your project?
Torin Blankensmith
2024-11-24 18:24:12 +0000 UTCGreat work on this tutorial. Super helpful! But I ran into an issue where the Whisper component doesn't seem to work for me. Every time I hit the Record Audio button, my TD crashes. Any ideas on what might be causing this? Thanks!
Viola Liang
2024-11-24 16:51:44 +0000 UTCI have been working on a project where I'm integrating ChatGPT into a custom application. While everything is functioning well, I've encountered a challenge. I'm trying to configure the model to respond in Arabic instead of English, but despite various approaches, the responses still come back in English. Could you please guide me on the best way to ensure that the model consistently responds in Arabic? Any specific parameters or prompt configurations that I should consider would be really helpful.
Elrashid Ahmed
2024-08-27 09:00:58 +0000 UTCI swear every time I'm looking to add a feature you beat me to it and do it better :) Was just looking at adding image context for chat interactions with the new 4o model.
7337 Productions
2024-05-13 23:56:22 +0000 UTC
