12 ms·
A $500 RL fine-tune of a 9B open model beat frontier models on catalog review
- nl 2mo agoA fine-tune will generally outperform just about any other method on a closed domain, non generative problem. LLMs are great because they can handle open domain problems in part because they are generative.
- jgalt212 2mo agoRL is the gold standard and significantly beats self learning methods, but other than coding and computer refereed games it's cost prohibitive.
- smithdamian 2mo ago[flagged]
- whiplash451 2mo agoWell I surely hope so. That’s the whole point of fine tuning.
- JSR_FDED 2mo agoI like the 2x2 grid that describes when to fine-tune a model, when to use a frontier model, etc. From the article it’s not clear how the scorer grades every episode - was it a frontier model that assigned the grade? How does that continue to work as the model that is being fine-tuned becomes better at the task than the frontier model?
- himata4113 2mo agoWhat I really started to notice is that SOTA models are really good at putting themselves out of the job. We can see this already with GPT how luna can do 90% of what sol is used for. The only reason why china still bothers 'distilling' models is accurate training data generation, something that oai and anthropic had to spend years collecting while trying to dodge legal challenges. The more intelligent models get, the more people will offramp to cheaper solutions that get the job done. There's no real benefit to using a sota model when the accuracy is already 99% and I think that is the biggest danger to US labs.
- com2kid 2mo agoThe upper end is all about coding. If I have terra on extra high write code, Sol will find a plethora of bugs and rip the code apart. Anything else? Sure use a cheaper model.
- himata4113 2mo agoYou can have sol write code and terra will find a plethora of bugs and rip the code apart. In reality this is just the nature of advisory prompting and why advisor from omp.sh is such a great feature. They get caught as they're being written.
- majormajor 2mo agoSometimes that'll turn up real bugs, sometimes just overengineered designs, premature-optimization, and 1-in-a-million possibility "bugs". And sometimes it's not about the model, it's just about refining the search space. E.g. I've had Opus write tests and GPT 5.5 write the implementation passing all the tests. Then ask about that specific implementation and find some real corner cases. Add those to tests, etc. But the other fun trick that's been working better and better on the GPT-5.6 series is that even the lower-end models can find the things they didn't think of first when inspecting the already-written output. I think there's still a bit of hard-to-quantify "creativity" to the bigger models - especially when trying to untangle (a) is this edge case that the model built a complicated way to avoid real/worth worrying about and also (b) even if it is real, is there not a better way to mitigate it? But it might be confirmation bias, in a way that definitely didn't use to be true about GPT-5.3 for planning and Composer 2 for implementation, say.
- solarkraft 2mo ago> even the lower-end models can find the things they didn't think of first when inspecting the already-written output I use Deepseek V4 Pro for my hobby project (an OpenCode client). The economics will obviously be different at work. It’s worse than the GPTs, but indeed, if you focus it on reviewing its own code (essentially spending more reasoning and changing perspective), it’s also quite capable at improving its own approaches. The GPTs are better at general architecting, but I think a lot of the performance gains also came from my more careful prompting (“okay, this is a hard problem, let’s think this through …”) to make using the more expensive model worth it. I use them and they are worth it on subsidized rates, but not at API prices. Since my Codex sub ran out I sometimes miss the models, but it really hasn’t devastated me.
- nzeid 2mo agoI didn't read the Ramp article but this reads like a post hoc fallacy. Companies with 2x revenue have money to spend on AI. Companies with 1.15x revenue don't.
- Quinner 2mo agoAlternatively, AI-heavy companies are much more likely to be more agile growth-stage companies rather than large mature companies.
- sudo_cowsay 2mo agoWhat benchmark is it? Is it super niche?
- stldev 2mo agoThey built their own benchmark and then trained directly against its scoring function.. seems to be the rage, but nothing convincing from the article alone.
- _345 2mo ago"87.3% Share of the maximum achievable score our GRPO-trained 9B open-source model reached on catalog review, vs 76.9% for the best frontier configuration: a 13.5% relative improvement over the frontier, and 36% over its own untrained base (64.2%). The five frontier models, even with optimized prompts, plateaued within a tenth of a point of each other; the trained specialist cleared that ceiling." _______ This is hard for me to believe. I have a lot of skepticism that frontier models like GPT 5.5 that are likely 2T+ parameters in size only got about 12% more accurate than an untrained 9b parameter LLM.
- baq 2mo agoWhy? This is a very narrow task, it’d be surprising if the results were different actually; more interesting question would be how an even smaller model performs in the same finetune.
- heresalexandria 2mo agoThis continuous cycle of fine-tuned open models beating frontier on (often vaguely labeled/defined) benchmarks doesn't provide an accurate comparison to the expanding generalized capabilities of the SoTA, which makes them effectively meaningless. If we were to take these at face value, why is it that the frontier labs' models are making legitimate new discoveries (e.g. Erdős and Jacobian conjectures) and these models are not? To me, a better signal of capability would be similarly performing novel work at the same or better level, which they presently are not. I say this as someone who very much looks forward to open models being more capable, but to deny the gap is misguided hopeful hype.
- ChanderG 2mo agoWhy? Why is the premise that Fine-tuned models should be geared towards new discoveries? The point of Fine-tuning small models is for specific downstream tasks, which SOTA models can do, but at higher costs. It is purely an economic play, not an attempt at pushing boundaries of SOTA.
- heresalexandria 2mo agoI'm not suggesting that fine-tuned models don't have their place, all I'm saying is that the constant drumbeat of "cheap model X beats more expensive model Z" completely misses that the more expensive model is capable of doing more things at a higher level. If the appropriate qualifiers were added to say "cheap model X does better at test Y than expensive model Z when we fine tune X to take Y test of existing knowledge" then it would be a more accurate statement, but naturally less impressive.
- echelon 2mo ago> I'm not suggesting that fine-tuned models don't have their place, all I'm saying is that the constant drumbeat of "cheap model X beats more expensive model Z" completely misses that the more expensive model is capable of doing more things at a higher level. What if you have to do the task a billion times? Which model will you choose?
- 2mo ago
- croemer 2mo ago[dead]
- cmiles8 2mo agoThe point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does the economic picture that justified the massive infrastructure building that’s now broadly funded by a complex network of debt. This is what makes open weight models so threatening to them. The political and “it’s China” angle is mostly just a cover for the real reasons why they’re freaked out. The fact that models are now a pure commodity is bad enough for the big labs. If small open weight models become the norm the big labs are toast.
- ozgrakkurt 2mo agoAs someone who worked at multiple startups, I am pretty sure they get it but it doesn’t fit their goals. They want to moat where they don’t need to compete with other companies because they have something that other companies can’t have. In my opinion this is a short-sighted and greedy worldview. Haven’t seen it work personally. It is a different version of the month-to-month salary guy thinking he will be a billionaire and having that thrash “mindset”. The reality is that practically none of those companies will amount to anything and they would be better off weighing the usefulness aspect of their output more. Instead they are imagining they will be Google. Anthropic and openai ofc are the pinnacle of this greed culture and they correspond to FTX from the crypto trash hype so I don’t think they fit into the scale of sensibility. Coming from this perspective, it is pretty easy to see what they are.
- spongebobstoes 2mo agothe major labs want to advance science. current business use cases are a happy accident
- cmiles8 2mo agoThat party is over. They’re all on the clock now to show they can make money or the plug will be pulled.
- deleted 2mo ago[deleted]
- nothrowaways 2mo agoTldr: we don't know what we are doing like the rest of 99% AI teams.
- mips_avatar 2mo agoThe problem i've had with finetuning models is that most of the time better prompting beats finetuning
- tikotus 2mo agoBetter prompting doesn't improve response time or price!
- davidpapermill 2mo agoYes, it can.
- mips_avatar 2mo agoOk but a task that works fine on qwen 397b can be finetuned on qwen9b. But in every case so far when building the eval for evaluating the traces I’ve discovered a better prompt that closes the gap better than the finetuning.
- adityas02 2mo ago[flagged]
- brainless 2mo agoI want small models to win and I am constantly experimenting with them. I have never tried fine-tuning and do not have that kind of budget. My approach is to remove some of the burden from models and bring into the agent. Tool calling is an example - in some tasks RAG works really well, including coding agents where code, git log, Epics/Tasks, dependencies sources, etc. are all available in very structured manner. You can save many extra tool calls if you can run separate prompts and retrieve the source data needed for the actual work - rather its prompt. And I really want to focus on search - this is the key technology if we want to use RAG instead of fine-tuning. If we can present really contextual sources in the prompts using a hybrid search approach - you can see how easily we get better results - either decisions or summaries from even small models.
- receptopalak 2mo ago[flagged]
- KennyBlanken 2mo agoComparing the revenue of the top quartile of AI-using companies to the average of all non-AI-using companies is beyond intellectually dishonest.
- mpaepper 2mo agoThere seems to be no hold out data for test, so this is just overfitting?
- h_mirin 2mo agoEvery time I see this kind of story, two things bother me. First, I have watched the free improvement of frontier models surpass the gains from retraining, many times now. Squeezing more out of the models that already exist, or simply doing nothing and waiting, is a real strategy and it often pays better. The fair comparison is not against today's frontier but against whatever ships while you are still maintaining your fine-tune. Second, the $500 training bill is the cheapest line item in this story. The expensive parts are creating the data and maintaining the model afterwards. How many use cases can actually produce 177k scored episodes? Here they had to generate them synthetically from Amazon Berkeley Objects. To me, that dataset is the strongest evidence in the article of how hard fine-tuning is to apply: if the data existed naturally, nobody would need to manufacture it.
- trentor 2mo agoWhat do you mean with maintaining the model? I am puzzled.
- fnands 2mo agoData drift. The data you are doing real world inference on can start changing, meaning your model performance starts degrading, so the model needs to be retrained on new data (that you have to collect and prepare).
- Turskarama 2mo agoSure, but that apparently only costs $500. If you do that once a month then that's still basically free. Hell if you do it once a week it's still about as much as a single cheap employee.
- zem 2mo agoyou missed the parent comment's point that the $500 training run was merely the last step, generating the training data is likely to be significantly more expensive and time-consuming. might still pay for itself, but it's not a trivial "spend $500 every month" decision.
- madhu_ghalame 2mo ago[dead]
- luciana1u 2mo ago[flagged]
- interleave 2mo agoI agree very much. I'm getting the same take-away even more now that I'm using autoresearch as my main strategy. And I wonder: We've got all these amazing (programming) languages to define solutions; where are the languages to clearly define the problems?
- interleave 2mo agoAmazing! I've been in a similar autoresearch-y rabbit-hole lately with getting Apple's 3B Foundation Model to match Sonnet 4.6 on a very specific task. The result was: 90% parity achieved with a weird combination of a fine-tuned adapter + 1 deterministic step. I wrote the whole thing down here: https://alexisrondeau.me/tada/research/FMDiscovery/dashboard/report.html https://alexisrondeau.me/tada/research/FMDiscovery/dashboard... which includes the question, the answer, the 96 experiments and their lineage etc. etc.
- docheinestages 2mo ago> I wrote the whole thing down here I like the idea, but it looks really hard to read. Try reading it top to bottom without skipping.
- interleave 2mo agoOh, thank you for noticing. Is it a layout issue for you (it lays out better on desktop) or the language of the text or...? Let me know!
- docheinestages 2mo agoThe main issue is the amount of information I have to go through to understand what I'm looking at. It's one of the side effects of writing with AI. Because it sees all tokens at once, it doesn't realize a reader might have no clue at all. What I recommend is trying to reduce the entire thing in 1-2 paragraphs that you wrote yourself. Cleaning up unnecessary prose and tucking away data that not everyone might be immediately interested in.
- npn 2mo agofine tuning a small LLMs or even real small language models (like bert) is what the recommended way since the introduction of LLM. the benefit is pretty much obvious: faster to run, fully controlling the stack, better fit for the custom domain... but in practice not many people do the fine tuning, for pretty much a single reason: large language models API cost are still very cheap, fast enough, and get improvement all the times. what the point of spend time (and money) to fine tune a specified model, then just when you release it a newer gen generic model is released and beat it? but if someday the progress for LLM is slowed, or the price increased to the point calling api is not a viable approach any more, then surely the day of fine tuning and small models will come again.
- dimgl 2mo agoIt's also really hard. And you need a lot of data for the fine tuning to produce good results.
- npn 2mo agowell usually you can just generate the data using LLM. use 2 or 3 different frontier models from different providers, then compare the results and pick the consensus. yes it is not 100% correct like when you make it manually, but then again even human makes mistakes.
- johsole 2mo agoI think we're going to get to that point on the 'S' curve. I also think the moat for a lot of companies is going to be their process and data, self hosting tuned models could be increasingly viewed as a trade secret.
- hermitShell 2mo agoI have great interest in fine-tuning open models, and I'm looking for resources that HN folks can personally recommend. This article looks good and I've bookmarked it to read more thoroughly over time. I've gotten as far as running Nemotron-3-Nano 30b locally, and plan to target models around 30b - 120b parameters. Based on brief examination of the results I can get, I think these vanilla models are capable enough to add real value, but training could push them over the finish line for specialized tasks. What I really appreciate is that the author is thinking about the whole process, which is also my goal. Confirmation that others are identifying the same use case, and the same strategy for adding value using this technology. This is a long term project, so I plan to buy hardware to conduct the fine-tune. ..
- johsole 2mo agoWhat kind of hardware are you using and what is your token generation speed (tok/s)? Every time I've tried to run local models, even on my beefy mac book (128Gb Ram), I've been very disappointed at tok/s speed.
- hermitShell 2mo agoI've been using dual NVIDIA GPU's with Ollama. Even when I push to 30b models and more context window, Ollama manages RAM / VRAM very well. I never got OOM errors, just massive slowdown as the PCI-E bottleneck throttles the GPU's. I've had some large prompts take 30 minutes to process. But I should mention: I am trying to implement workflows and processes akin to CI/CD that run 24/7 in the background. These are not interactive use cases, so I don't care so much about tokens per second.
- hizyyo 2mo ago[flagged]
- podgorniy 2mo agoWTF is happening here? It's a clear marketing piece with clear bunch of bot commenting like the article is a real deal. Is this a regular the modern day HN experience?
- blagui 2mo agoA 9B beating frontiers" No! But nailing specialized tasks Yes.
- binarymax 2mo agoNo mention of training/test/holdout dataset splits anywhere in this article - so how am I to know whether or not this is some overfit result?
- bfeynman 2mo agois this an ad? | Harvey's legal agent beats GPT-5.5 and Claude Opus 4.8 on its own rubrics, and Intercom's Fin Apex resolves more support issues at lower cost. The rubric and cost argument here just casually ignores all of the other challenges and real business issues of evolving models over time
- teravor 2mo agoit helps when you first start by distilling (full distribution) domain specific workflows from a larger open model (kimi k3 would be perfect for this now, we used glm 5.2) and then apply your own private RL pipeline to the result. we did this to get a private 27B model for English -> SQL and the results were better than Fable, it lost the ability to explain however. beating frontier models on highly specialized domains with a tiny fraction of the cost is not difficult to achieve, even when you don't have your own RL pipeline such distillation can still have major savings (27B model will become almost as good as a 3T model on the task). it probably won't work if the task is so ambitious as to not be suited for a specialist.
- Chipshuffle 2mo agoHow difficult is it to post train a model like this. I'm still imagining it being really complicated for someone that has no expert knowledge in this field. Or are there already specialized materials or offers for this that make it easy?
- Wetime 2mo agohttps://huggingface.co/papers/2607.23806 https://huggingface.co/papers/2607.23806 this is a cheaper solution
- elfayoumi 2mo ago[flagged]
- crawfordmarch 2mo ago[flagged]
- happyf_oder 1mo agoi think this is the last piece of local win before training on cerebras era started.