Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
appvoid 
posted an update 13 days ago
Post
7,632
New leaderboard coming to the small language model community!

The leaderboard measures cognitive skills, which is more challenging and beyond language modeling and instruction-following. This is a first step towards more everyday useful and general small models.

dotlabs/cognirank

i have a small model that i need advice on benchmarking. Im a solo guy thats been doing it by myself for 4-5 years now. custom datasets and even now i have a 2.2b multi seq omni latent recursive model in training on Digital Ocean Mi350x card.

·

Sure, let me know what kind of advice are you looking for.

Though, a general advice, when it comes about llms is to never trust one benchmark. Test as much as possible.