fredmendoza

12 ms·

1.
by fredmendoza 5mo ago
and that 4x difference allows you to use CPUs and much cheaper hardware to achieve the same level of outcome... for free
2.
by fredmendoza 5mo ago
yes, with one line change. grab the second code block in the article, that's the test harness rigged up to send all 80 questions and both turns through whatever model you want. find MODEL_ID = "google/gemma-4-E2B-it" and
3.
by fredmendoza 5mo ago
love hearing this. and think about it, if the 2B is already doing this well on your mac mini, imagine what the 4B, 26B, or 31B can do on 32 gigs. with lower quantization you can fit pretty much any of them. if you want full precision you st
4.
by fredmendoza 5mo ago
thank you for actually reading it and getting it. the airplane mode test is hilarious, the model sitting on your phone insisting it can't run on a phone. that's amazing. and yes we think exactly the same way. like picture a small
5.
by fredmendoza 5mo ago
clever guess but no lol. used claude for the writeup. the proof isn't the prose, it's the tape and the code. run it on your machine, you'll have a free private agent custom to whatever you need. that's the proof of conce
6.
by fredmendoza 5mo ago
Fred, nice to meet you. The grading model had no idea what was being tested. We used separate accounts to compartmentalize. The Claude grader was guessing GPT-3.5 Turbo or GPT-4 by the end. On the coding block it consistently scored respons
7.
by fredmendoza 5mo ago
good callout, want to clarify. claude helped us set up the test harness. gemma took every question alone with zero help. the ~8.0 is all gemma. and you're right, opus is in a completely different league. we're not arguing otherwis
8.
by fredmendoza 5mo ago
really appreciate you reading the article. the benchmark data, grading, and error classes were all done by hand though. the ~8.0 is the raw model with zero tooling, and the guardrail projections are documented separately. and yeah gpt-3.5 i
9.
by fredmendoza 5mo ago
fair enough, here are the actual fixes from the codebase with the tape examples they target: arithmetic (Q119): benjamin buys 5 books at $20, 3 at $30, 2 at $45. model writes "$245" first line then self-corrects to $280. fix: mode
10.
by fredmendoza 5mo ago
appreciate the vouch but come on lol. we ran 80 questions, graded 160 turns by hand, documented 7 error classes, open sourced all the code, and put a live bot up for people to test. to write this post up took me hours. everyone is a critic
11.
by fredmendoza 5mo ago
you're honestly not that far off. the coding block on this model scored 8.44 with zero help. it caught a None-init TypeError on a code review question that most people would miss. one question asked for O(n) and it just went ahead and
12.
by fredmendoza 5mo ago
you're right, they are tools. that's kind of the point. PAL is a subprocess that runs a python expression. Z3 is a constraint solver. regex is regex. calling them "surgical" is just about when they fire, not what they ar
13.
CPUs Aren't Dead. Gemma2B Out Scored GPT-3.5 Turbo on Test That Made It Famous(seqpu.com)
100 points by fredmendoza 5mo ago|52 comments
14.
by fredmendoza 5mo ago
we found something interesting and wanted to share it with this community. we wanted to know how google's gemma 4 e2b-it — 2 billion parameters, bfloat16, apache 2.0 — stacks up against gpt-3.5 turbo. not in vibes. on the same test. mt
15.
We put all 4 Gemma 4 models in one Telegram bot. Try it and see how we built it(seqpu.com)
1 points by fredmendoza 5mo ago|1 comments
16.
by fredmendoza 5mo ago
Text it, send voice memos, send docs, send photos. Switch between the 2B and the 31B (ranked #3 worldwide) mid-conversation with a slash command. Each model runs its own script on its own hardware. The 2B doesn't burn A100 hours, the 3
17.
Every Gemma 4 model in one chat. Try it in 60 seconds and see how we built it(seqpu.com)
3 points by fredmendoza 5mo ago|1 comments
18.
by fredmendoza 5mo ago
We put all 4 Gemma 4 models in one Telegram bot. Text it, send voice memos, send docs, send photos. Switch between the 2B and the 31B (ranked #3 worldwide) mid-conversation with a slash command. Each model runs its own script on its own har
19.
Every new Gemma 4 model in 1 chat. Use it in 60 seconds and see how we built it(seqpu.com)
2 points by fredmendoza 5mo ago|0 comments
20.
by fredmendoza 6mo ago
honestly for most coding tasks a smaller model gets it done. qwen3-32b on an a100 80gb handles single-repo work just fine — and if you're not convinced, you can test the same prompt against a 405B on a 2×B200 with literally a few butto