Microsoft releases GRIN MoE. With only 6.6B activate parameters, it achieves exceptionally good performance across a diverse set of tasks, particularly in coding and mathematics tasks. Level above GPT-3.5 and close to the first version of GPT-4. Using little RAM due to the low number of active parameters. Benchmark results slightly higher than Phi-3.5-MoE also from Microsoft.
Benchmarks:
MMLU: 79.4
GSM-8K: 90.4
Trained with synthetic dataset generated by RDXL Pony Anime
https://civitai.com/models/772320/rd-anime-lora
2024-09-18 14:58:32 +0000 UTC View PostDescription: A realistic model that does not insist on preserving the characteristics of the Pony model in exchange for greater realism. Very good for generating some characters in a realistic style.
2024-08-27 12:57:57 +0000 UTC View PostDownloads:
I'm almost done with the new version, I'm just waiting to finish some training that's in progress and I'll produce the new screenshots.
2024-08-19 18:10:51 +0000 UTC View PostDownloads:
10.2 and 10.3 were train...
2024-08-17 17:05:05 +0000 UTC View Post
