4 ms·
According to the readme here - https://huggingface.co/google/gemma-3n-E4B-it-litert-preview https://huggingface.co/google/gemma-3n-E4B-it-litert-preview E4B ha
by IceWreck 1y ago
According to the readme here - https://huggingface.co/google/gemma-3n-E4B-it-litert-preview https://huggingface.co/google/gemma-3n-E4B-it-litert-preview
E4B has a score of 44.4 in the Aider polyglot dashboard. Which means its on-par with gemini-2.5-flash (not the latest preview but the version used for the bench on aider's website), gpt4o and gpt4.5.
Thats sounds very good - imagine what a coding focused version of this could do if this is a "generic" embedded only model.
On the other hand - this does have a much lower score for livecodebench.
- dingnuts 1y ago[flagged]
- nolist_policy 1y agoHmm, the Aider polyglot benchmark has been removed from the huggingface readme. Also: > These models were evaluated at full precision (float32) For 4B effective parameters that's 16 GB ram.