0.2.98: koboldai (beta), cohere
Added 2024-10-25 15:11:20 +0000 UTC✨ support for local models via koboldai in beta status*
✨ added models from cohere**
✨ filter by recommended models
✨ added models claude-3-5-sonnet-20241022 and llama-3.2-90b-vision-preview
✨ ru: server selection for internal openai and anthropic proxy added to settings***
*could not fully test on my machine, used weak models with a context window of 2-4k (in the game, characters have a total system context of ~6k). therefore, i have no impressions here, but i believe that it will work on proper hardware
on models with small context windows and languages other than english, everything is quite bleak, each letter takes several times more characters
appealing for help to kobold experts: please recommend models in the comments
**added models from cohere, at first glance, gemini-flash level is not very powerful, fast, not expensive. they offer a free trial, which is nice
***kazakhstani server suddenly stopped working, or rather openai and anthropic stopped working in Kazakhstan, if i understand correctly. while deciding where to move, decided to proxy through where gemini is running, there the server works on a single thread, possibly will be more loaded than usual for some time
on the help page in the game, a kobold section with screenshots appeared, i will duplicate a small instruction in text:
windows, download and install
official repository - also has a solution for linux
how it works:
launch koboldai → load model → set chat mode → launch game
to check that everything is working correctly:
koboldai should open in the browser http://localhost:5000
after sending a message from the game → go to kobold → memory menu (top right corner) → system context should load in all its glory
