Exploring the Frontiers of Intelligence … by co-creating LMcouncil.ai
Added 2025-10-15 15:47:00 +0000 UTCIf Simple Bench tests common sense, what is the best test of practical wisdom? For me, it was finding out whether 2025 language models could build the app I have been meaning to make for two years. They can’t, no yet, but they can get very close with a determined human in the loop.
lmcouncil.ai is a combination of SmartGPT, Karpathy’s vision for an ‘LLM Council’ and my own touches. Compare model outputs, call votes, set leaders, roleplay a full group chat of characters, and on and on …
To be clear, even with 500 hours work and the best models you can’t yet cover all edge cases, or get bug-free perfection. And that’s with GPT-5-Codex best for diff patches, Claude 4.5 Sonnet/4.1 Opus for mobile features, Gemini Deep Think for backend, and even that recommended list changes every week (w/ Google CodeMender coming for extra security)
But I would love your help:
I need you to tell me what features you like, to work more on
You can shape the app in ways no future user could
You get an exclusive discount, at almost below API rates
Comments
Hey, thank you so much for asking! Will be adding custom credit blocks in the new year, at the moment best option is Max tier, for a little bit, then downgrade if needed in say Feb or March. I use most of Max tier every month so even that not enough!
Philip
2025-12-23 09:20:19 +0000 UTCHow do I pay for an additional subscription at the same tier when I've hit the token limit?
Anon Anon
2025-12-23 08:44:27 +0000 UTCThat's so great to hear! Anytime I need to do another 15 hour day a comment like this keeps me going, means a lot.
Philip
2025-12-02 09:49:30 +0000 UTCThis is brilliant. I’ve always wanted this! And so glad you wound up making it. I trust you to take this great places.
Ben B
2025-12-02 05:06:44 +0000 UTCyes, did that ages ago but never tested, good point
Philip
2025-11-20 10:24:31 +0000 UTCHey Philip, was wondering, could you make perhaps a set of 10 or something new questions to check if maybe, just maybe the simple bench questions have leaked? I mean, no need to go through the vigorous question testing with PhDs, just a few off the top questions to test, say GPT 5.1 and Gemini 3, to see if you get similar scores? Makes sense?
Strangest
2025-11-19 23:13:51 +0000 UTCI'm looking forward to giving this a go as the weather gets worse and I am spending more time indoors.
Jason Dowd
2025-11-08 01:07:56 +0000 UTCPlease add an option to "delete all" past conversations. Thank you!
Riley Thomson
2025-11-05 01:02:11 +0000 UTCQuick feedback after signing up with email just now. The confirmation email went to Spam in my protonmail inbox. Maybe something to look into if you want to make onboarding smoother.
clay-loop
2025-11-02 10:03:10 +0000 UTCThanks for asking! For the very first prompt the models will not be able to see each other's answers before they are generated, so you would set up the system prompts 'in roles' then kick things off with full context and what is required. FOR ALL SUBSEQUENT ROUNDs, the models see everything said by everyone else, so you can get debates/arguments/consensus etc. The only thing I don't have is like an ongoing debate that continues indefinitely until you stop it. A couple of people have asked for that but it would mean a lot of credits get burnt, but if this is exactly what you meant, do let me know. Last, if you meant you want one model to output the consensus point/summary/table of points said by everyone else, that is the 'leader' button at the bottom.
Philip
2025-10-26 10:02:34 +0000 UTCI have not been able to find out how to let the models interact with each other. I would appreciate using system instructions to create, let's say, research team members represented by different models to run virtual research lab meetings where I throw in a topic/question and the models engage with each other to come up with a consensus.
Michaela Liegertová
2025-10-26 09:56:23 +0000 UTCdone, just for you! Let me know what you think!
Philip
2025-10-24 19:53:50 +0000 UTCone small feature request - Remove background button makes background black. Let us pick a color please. I hate dark themes, yes I'm minority but I like to not damage my eyesight ;)
Sam
2025-10-24 10:19:59 +0000 UTCGreat idea! You mean background image or a viewing mode like 'columns' or 'dim/dark' mode that we currently have?
Philip
2025-10-20 10:59:12 +0000 UTCI love the initiative! Not related to the AI field, but could be useful to set a default visualization for a shared public council conversation when sharing it. Will keep testing the tool and provide more feedback!
Maico Bernal
2025-10-19 22:44:42 +0000 UTCSuper cool, congratulations on putting all of this together, Philip!
Dorian Iten
2025-10-17 23:38:53 +0000 UTCIs it the boldness of the colours, the brightness of the screen or both, which you most want options over?
Philip
2025-10-17 10:41:55 +0000 UTCVery cool! Not AI related but would love some simple themes to have basic colour styling done for me for the different boxes etc. The bold colours are killing my eyes hahaha
Kyle Behrend
2025-10-17 10:08:07 +0000 UTCOn 1, where would you like this placed? I am thinking a 'Submit Idea / Feedback button below FAQ/Hotkeys... On 2, if anyone runs out of credits on Max tier I would love to hear about it, I like what you said here in principle, like a Custom tier, will review Stripe docs.
Philip
2025-10-17 08:06:57 +0000 UTCThanks so much Michael. Do send along your ideas on UI, unless you meant a dedicated mobile app, in which case - working on it.
Philip
2025-10-17 08:02:49 +0000 UTCThanks Jim, feedback always appreciated
Philip
2025-10-17 08:02:17 +0000 UTCCool idea! A couple of suggestions. #1 Make user feedback possible *directly* within the app, so you lose the vast majority of useful ideas as people have them. You could have AI triage and categorize common ideas if you get too much volume. #2 Allow users to purchase token limits in whatever amounts they'd like above some base allocation, either as additional subscription amounts or one-time purchases.
Alexis Olson
2025-10-17 03:58:15 +0000 UTCLove your idea! This is great. It is something I'm doing manually, asking several LLMs for an answer to get a broad overview or to get several different opinions. And your "Smart GPT" idea with the Council leader, is something I guess the top AI development leaders are implementing right now to improve their AIs answers. It is probably one of the best ways of improving the answers from an LLMs. Let's say there is a 1 in a 100 chance of getting a genius answer from an LLM. If you ask it 100 times and then pick all the different ideas from all the answers, you increase the chance of getting a genius answer from almost 0 to almost 1. About your app, I would love it if you update the UI to look more modern. Right now, it doesn't look that great. I think many more people would use your app and trust it more if the UI looked better.
Michael Lulev
2025-10-16 20:55:12 +0000 UTCAwesome! I’ve really been needing this. Will try out and pass along. Nice job and great work
Jim Beaver
2025-10-16 16:21:37 +0000 UTCAlready added two features based on the comments in this thread...!
Philip
2025-10-16 14:39:39 +0000 UTCIf anything, motivates me more to make videos. More sign-ups, even on Free, = more videos on all topics. But I can't ask more of those on paid Patreon though, you already do enough to keep channel going indefinitely
Philip
2025-10-16 13:35:35 +0000 UTCWould love any tips from someone who has attempted the same, keep 'em coming
Philip
2025-10-16 13:34:20 +0000 UTCThanks so much Peter
Philip
2025-10-16 13:34:02 +0000 UTCThat is a really good point, will work on that, expect rapid progress
Philip
2025-10-16 13:32:11 +0000 UTCYes, though I trust you guys to test it out and give me feedback as to whether its Explained-worthy
Philip
2025-10-16 13:31:36 +0000 UTCWhere does the brightness come from, if we control the background/bubbles? You mean the button text?
Philip
2025-10-16 13:30:38 +0000 UTCSuper cool Phillip ! Guessing eventually you will do a demo video of this on your youtube channel?
Daniel A Barbatti
2025-10-15 23:09:25 +0000 UTCPlease add light mode I'm scared of the dark :(
Ásgeir Thor Johnson
2025-10-15 19:33:19 +0000 UTCAs a user I want to see how many credits each model uses so I can balance intelligence vs cost when I’m creating councils.
Bill Ray
2025-10-15 19:13:42 +0000 UTCThe system prompt editor being hidden behind the “roleplay” button feels weird. It would be cool to be able to configure the models (system prompt, tools, effort level, etc.) in an intuitive way. Maybe by clicking on the model names which currently is how you replace them. Maybe users could even save individual model configurations and then call these different configurations to their councils.
Bill Ray
2025-10-15 19:11:16 +0000 UTCIs there a way to let the council deliberate amongst themselves for a few turns, reporting back with a verdict? I realize this is similar to the council leader feature, but I think some ideas would benefit from questioning and reflection.
Bill Ray
2025-10-15 19:02:16 +0000 UTCWhere’s GPT-5 with search? I want a cracked research council!
Bill Ray
2025-10-15 18:59:03 +0000 UTCWow I could see myself daily driving something like this if it had the conveniences of the other platforms. At which point I would almost certainly need the max tier. I could also see myself using an LLM Council API/Python library to integrate these “consensus” functionalities into my own projects. I hope this project becomes profitable enough to hire someone/a team to maintain and expand it! Very cool! Looking forward to more AI Explained videos!
Bill Ray
2025-10-15 18:53:28 +0000 UTCVery cool!
Peter Seelman
2025-10-15 18:26:16 +0000 UTCHey Phillip, I released a very similar product last month called https://pipeliney.ai/ didn't have much success in getting take up though! I'd love to collab we have very similar ideas on how this all should work and what could potentially make a good product!
Scott Rowlandson
2025-10-15 18:17:44 +0000 UTCYes to both Shawn! I agree on the niche papers, and yes on the topic and roles and having them debate. You initiate and then can ask them to respond to each other
Philip
2025-10-15 16:59:20 +0000 UTCGreat work Phillip! I'll have to check out the app! Is it possible to give a bunch of AI models a topic and roles and have them debate each other? That would be a really cool interaction. Also, I'm looking forward to seeing you cover more niche and experimental AI research papers like back in the day (Mamba for instance). I think you're channel is at it's strongest when you cover AI research that no one else is talking about rather than just focusing on the major releases.
Shawn Rosofsky
2025-10-15 16:56:33 +0000 UTCThank you Clarissa!
Philip
2025-10-15 16:54:59 +0000 UTCLove the branching! Different goal but very cool also
Philip
2025-10-15 16:54:50 +0000 UTCAlternative: aiexplained@outlook.com!
Philip
2025-10-15 16:54:30 +0000 UTCGood luck with the app. I hope it will not distract you too much from the AI Explained and Insider. 🤞
Pavol Vaskovic
2025-10-15 16:39:53 +0000 UTC29:25 I can't DM you on Patreon 😔
Gilad
2025-10-15 16:31:38 +0000 UTCVery cool!
Gilad
2025-10-15 16:24:39 +0000 UTCVery nice Philip! Also, as you have built such an interface, what do you think about this one? Maybe it even provides inspiration. https://x.com/maxleedev/status/1962938776294195498
Manuel
2025-10-15 15:56:09 +0000 UTCAWSOME PHILIPP
Clarissa Röthig
2025-10-15 15:49:06 +0000 UTC
