7 ms·

Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

484 points by adam_rida 2mo ago

I’ve been building Echo (https://echo.tracerml.ai/ https://echo.tracerml.ai/), an experiment in making one AI system out of a pool of open-weight models rather than choosing a single model and using it for every task. It started with a simple experiment. I took a group of models, including GLM-5.2, Kimi K2.7 and others, and ran them on the same evaluations. Then I measured what would happen if, for each problem, you somehow knew in advance which models would be useful and how their outputs should be combined. That hypothetical system performed substantially better than any individual model in the pool. Of course, it is not something you can actually deploy because it relies on knowing which decisions were good after seeing the result. Echo is my attempt to recover some of that advantage without having that information in advance. For each request, Echo decides how much computation to allocate, which models should participate, and how their work should be combined. Some prompts may only need a relatively small amount of inference, while others benefit from multiple models working on different parts of the problem. One thing that surprised me while building it was how complementary the models are. A model that is clearly weaker overall can still be extremely useful on particular problems or as part of a combination. On my first evaluation mix, Echo consistently performed better than the best individual model in its pool. It also reached roughly the same aggregate result as Fable, which I used as one of the stronger comparison systems, at around one third of the inference cost. There are still some cases where Echo makes the wrong allocation or combination decision. I’m currently spending a lot of time understanding those failures, as well as testing whether the same approach holds up on coding and agentic tasks where measuring the quality of each decision becomes much harder. I built a chat interface (echo.tracerml.ai) and an OpenAI-compatible API (https://echo.tracerml.ai/docs/api https://echo.tracerml.ai/docs/api) so the system can be tested outside the evaluation setup. Here is a short/high level video on how it works: https://www.youtube.com/watch?v=lJFJSvOdXhg https://www.youtube.com/watch?v=lJFJSvOdXhg I wrote up the evaluation methodology, individual model results, costs and current limitations here: https://echo.tracerml.ai/eval https://echo.tracerml.ai/eval I would love for you to try it! Especially if you hit any weird failure cases or places where the allocation looks unintuitive.

IrfanD 2mo ago
[dead]
codekansas 2mo ago
Fable-level, yea, but can it run gstack?
tj800x 2mo ago
No single signin. Privacy policy allows training. No try it first without credit card. It's a good idea, but this looks premature.
- adam_rida 2mo ago
  You are right that the privacy wording was too broad. We are fixing it now so it states explicitly that Echo does not use customer prompts, files, chats, or outputs to train or fine-tune models. We are updating the matching Terms language at the same time. Echo also starts users with free credit and does not require a credit card to try it. The current signup flow did not make that clear enough, so we are fixing that presentation too. Thanks for calling both out.
- cynicalsecurity 2mo ago
  I assume trying without a credit card will bankrupt him immediately.
- seizethecheese 2mo ago
  If you want to try a similar idea that you can chat with immediately (free tier uses open weight models), I'm working on this: http://pellmell.ai http://pellmell.ai
- DatCodeMania 2mo ago
  That would open him up to lots of abuse
kamranjon 2mo ago
No benchmarks, no info on which models are used, ai generated video, just a signup page with nothing else. Anyhow, this kinda reminds me of that quote about architecture: "We replaced our monolith with micro services so that every outage could be more like a murder mystery."
- guessmyname 2mo ago
  > No benchmarks, no info on which models are used, […] The benchmarks are here → https://echo.tracerml.ai/eval/ https://echo.tracerml.ai/eval/ They are not good benchmarks but at least they exist.
  - seizethecheese 2mo ago
    I've been working on a similar project and I found that it's easy to replicate Fable results if you use saturated benchmarks. In my project, I wasted a huge amount of time trying to improve GPQA Diamond results above ~93% range. I realized my mistake when Fable dropped and made no improvement on this benchmark vs. Opus.
    - yorwba 2mo ago
      I wouldn't be surprised if ≈7% of GPQA Diamond questions simply have the wrong answer in the ground truth data, so that getting such a question correct is graded as an error. Most machine-learning benchmarks are rather badly validated.
      - seizethecheese 2mo ago
        Yep! I found this interesting article after banging my head against a wall for a long time: https://epoch.ai/gradient-updates/gpqa-diamond-whats-left https://epoch.ai/gradient-updates/gpqa-diamond-whats-left
- j45 2mo ago
  It's easy enough to copy and paste in a prompt, no? Eval tests while giving general indicators might not be similar for each use case.
deleted 2mo ago
[deleted]
- adam_rida 2mo ago
  Thanks for checking the individual rows. HumanEval+ is one small code slice, not the whole basis for the launch claim. The public evaluator currently contains 907 rows across seven benchmark families, and matched SWE-bench Verified and BigCodeBench runs are the next code evidence being added. You also found a real UI bug: the inspector should show both stored answers and currently does not in some rows. We are fixing that. On the row you reran: the page records a frozen matched run. It does not claim that Fable is incapable of solving that prompt on another run. We are adding repeated matched trials and making run count and variance explicit. Your rerun is exactly the kind of external check the row-level page is intended to make possible. The broader result remains: Echo is competitive with Fable across the evaluated task mix at materially lower measured inference cost. We are filling in the harder agentic-code evidence now rather than asking anyone to infer it from HumanEval+.
fneddy 2mo ago
That’s basically the same idea IBM advertises with Bob?
- seizethecheese 2mo ago
  This looks different, it's not a main agent delegating to subagents, it's a router.
jmaw 2mo ago
I think approaches like this have potential. Only time will tell. This reminds me of the mixture of experts taken by deepseek r2 (I think it was r2, at least), but less specific models I guess. I have often wondered how tools like GHCP choose the best model for the job when set to "auto".
Alifatisk 2mo ago
This reminds me on OpenRouters report that combining multiple different models gave comparable performance to Fable 5. I think this approach has lots of potential. Maybe OpenAi was ahead of its time with GPT-5 (it being a router to different models rather than just being one new model)
- seizethecheese 2mo ago
  You might be interested in a project that I'm working on, which is kind of like OpenRouter Fusion, but instead of waiting for all models to synthesize, we stream the best model immediately and background the rest. The background models then reconcile with an emoji reaction and optional reply. It gets similar results to Fusion and is a lot faster! There's a free version that leverages open weight models here: http://pellmell.ai http://pellmell.ai.
  - hmokiguess 2mo ago
    This is really cool, super fun!
  - NetOpWibby 2mo ago
    I like that I was able to test without an account and my prompt (What's the feasibility of a NetNavi IRL, along with a PErsonal Terminal? Create a document that outlines how to make this happen.) makes me think Pellmell is gonna be a research tool I'll continue to use.
- tintor 2mo ago
  OpenAI's router chooses between models of different sizes, which are still trained on roughly the same data. Its purpose is to reduce infra cost for OpenAI for simpler queries. No need to pay for GPT 5.6 Sol inference for "Hello" prompt.
j45 2mo ago
If you copy Perplexity, they let you have the first few rounds of chat for free to get you going before asking to sign up.
jambalaya8 2mo ago
Might want to rethink the name to avoid an Amazon issue.
wizche 2mo ago
how does this differs from OpenRouter fusion?
- cawingcrow 2mo ago
  how does this differs from Sakana Fugu?
  - deleted 2mo ago
    [deleted]
bnjemian 2mo ago
I don’t find the recent spate of blog posts and systems delegating and combining LLMs to get better performance particularly interesting. Especially given that anyone who’s taken an ML 101 course has learned about ensemble methods. While an LLM isn’t what you’d traditionally consider a weak learner, the theorems on learning systems clearly point to them being so in this context. The feigned surprise at combining them to yield better results seems disingenuous. Even so, the work to predict which models are best suited for which task, how to delegate, and how to combine their outputs is interesting, especially if you’re placing a cost minimization objective on it. That said, this isn’t too far off from what many AI labs are already doing.
- abernard1 2mo ago
  The entirety of "agents" and tool calls is a process of combining LLMs to get better results. Is it the same LLM in many cases? Yes. But it doesn't have to be. It's the natural move that happened after people realized you couldn't throw away half a century of AI research. Most of these focus on costs. But it is simply the case that the one-shot output did not scale for harder problems on workflows.
theneocorner 2mo ago
[dead]
ototot 2mo ago
Is this yet another Sakana Fugu / OpenRouter Fusion?
- kachnuv_ocasek 2mo ago
  Looks like it but with open-weights only.
- code0igx 2mo ago
  The architecture does convey the same.
yonatan8070 2mo ago
I'm not an expert on this, but this sounds a lot like a larger-scale MoE (Mixture of Experts) type of architecture. As I understand it, in an MoE model, you essentially have hundreds of smaller sub-models ("experts") that are good at different tasks, and for every generated token, a single "master" model chooses which ones are most relevant to participate, and you only activate them.
- janalsncm 2mo ago
  In MoE systems the routing decision is made per-token, not per prompt or task. It’s one of ML’s many confusing naming conventions. Even more confusingly, there are older pre-LLM MoE systems which ensemble and pool the predictions from multiple sub-components. For example in a random forest you could take the majority vote of the decision trees or the average of their numerical predictions. After that, we developed neural net architectures for predicting a single thing like whether the user will click on your ad. An MMoE is in the same family. And so now we are at massive MoE networks for LLMs which have similarities with MMoE in that the “decision” is about the very next token to predict.
  - lukan 2mo ago
    "In MoE systems the routing decision is made per-token, not per prompt or task." Have there been experiments with doing it per task? Like, "oh this is python project, use this model" "oh this is about writing fantasy, use this"?
    - janalsncm 2mo ago
      It’s a good idea. The results probably depend a lot on how close your task is to the benchmarks though. I think OpenAI already has (had?) a feature like this called “auto” mode for thinking.
bbstats 2mo ago
M-o-MoE
maxdo 2mo ago
such a scam, there is only one fable-like model, that somewhat behind, it cost half, not 3x. so from here you can stop reading.
- seizethecheese 2mo ago
  They're obviously not claiming this is a new model that is fable-like at 1/3. It's a router that saves money by only using Fable when necessary. I don't think you should be calling someone's post a "scam" without doing a minimum of research. (I'm not associated with the company, but very interested in this.)
janalsncm 2mo ago
Intuitively, your savings depend heavily on how hard the tasks are in the first place. If you have a base rate where 99% of your tasks can be routed to a cheap model, yeah, you can save a ton by not using Fable for that. So “1/3 the cost” really depends.
- seizethecheese 2mo ago
  Sometimes expensive models are cheaper on easier tasks because they use fewer tokens, too.
  - janalsncm 2mo ago
    Yeah apparently Opus and Sonnet are like that.
    - seizethecheese 2mo ago
      Yep! There are a lot of models like this: https://artificialanalysis.ai/models/capabilities/coding?cost-per-task=index-vs-cost-per-task https://artificialanalysis.ai/models/capabilities/coding?cos...
ninjahawk1 2mo ago
I’m very confused on what this is, my initial thought was “oh nice, open source router.” I go to the website…and it’s a sign up. I expected a repo. Otherwise how do I use it? As a SaaS? Yeah right. Oh well I guess at least the benchmarks are good…I find the benchmarks and many are either not present or are not what the title claims. My main question is how this has so many updoots from HN, probably the passerby not looking closer for sure. I mean no offense and I really do wish you best on this, but it seems like what we used to call back in the day, vaporware.
jacobgold 2mo ago
> Fable-level results at 1/3 the cost using open-weight models But we get ~$2500/mo worth of Fable credits for $200/mo on Anthropic pan? I'm still confused why people (who don't have to use API billing) are chasing open weight models based on cost.
- teruakohatu 2mo ago
  Because that is a short term solution, it won’t be offered forever. Large organisations have to purchase credits at $/tokens. Eventually everyone else will too.
  - sscaryterry 2mo ago
    This is what OpenAI and Anthropic are trying to make everyone believe. Most accountants will flinch at this (they already are). The $200 odd plans are already out of reach of many, many people. The attrition of customers if they were to get rid of these subscriptions plans would be untenable.
    - recursivegirth 2mo ago
      I think you are looking at it incorrectly. No business is buying individual accounts, because if they do, they open themselves up to considerable risk. The $200 plans are priced so that the power-users use them and then advocate about how great the product is. If you're buying a $200 plan, you're not doing it because of the price point but rather because of the amount of work it is doing for you.
      - trollbridge 2mo ago
        Lots of businesses are buying and using these plans. Basically every small business I interact with.
      - anonzzzies 2mo ago
        All companies we interact with have 200 plans.
dluan 2mo ago
So this is the dogpile.com of the askjeeves, alta vista, and lycos approach? Time is a flat circle?
- glasss 2mo ago
  Good ideas are usually still good across time and tools
  - tiffanyh 2mo ago
    Exactly High quality, fast & cheap (all 3 combined) - is a formula success. It’s just way easier said than done.
    - Avicebron 2mo ago
      There's a reason most pros will tell you pick two of the three.
  - onlyrealcuzzo 2mo ago
    Dogpile was only a good idea while Search Engines were mostly trash. You needed to search all of them to find something decent. That's roughly analogous to today. Ignoring cost, you'd be way better off asking all the LLMs to solve a problem (like coding) where you can verify the answer. So the question is, for things like that -> can a group of models perform better than frontier models, especially at a reasonable cost? Fable is not a great value, so unless you're trying to find answers to Erdos questions, you can probably do better on cost. You can probably typically ask 3 or 4 of the top Chinese models for an answer and get a response for the same Fable question... Given that Fable isn't that much better, it's not surprising you can do better for a large subset of problems.
    - Terretta 2mo ago
      > Dogpile was only a good idea while Search Engines were mostly trash. Precisely.
hmokiguess 2mo ago
"Backed by YCombinator" https://www.ycombinator.com/companies?query=tracerml https://www.ycombinator.com/companies?query=tracerml I don't see it?
- deleted 2mo ago
  [deleted]
- zachdotai 2mo ago
  Not all YC companies have launched publicly yet but I am a current YC founder and I can confirm they exist in the internal directory.
- dang 2mo ago
  You're right to be skeptical; I saw a totally fake case of this just yesterday. But in the present case, they're just a startup in the current batch.
  - hmokiguess 2mo ago
    Thank you! Hard to trust things these days.
meander_water 2mo ago
Seems similar to Openrouter Fusion - https://openrouter.ai/docs/guides/routing/routers/fusion-router https://openrouter.ai/docs/guides/routing/routers/fusion-rou...
- seizethecheese 2mo ago
  Fusion is a totally different approach, though similar in the sense that it leverages different models. Fusion generates many replies then synthesizes. This adds a ton of latency and cost, so it's going to be better only for cases where you're willing to wait a lot and pay a lot more. Routers (like this project) are a different thing, they can theoretically improve performance and cost at the same time without increasing latency much. I'm a bit skeptical though, since knowing which LLM is going to be better on a cost adjusted basis is hard (see https://artificialanalysis.ai/models/capabilities/coding?cost-per-task=index-vs-cost-per-task https://artificialanalysis.ai/models/capabilities/coding?cos..., where the cost per task vs. performance is not what you expect, for example comparing Qwen 3.7 Max to GPT Sol. A project I'm working on is aimed at improving performance without added latency but from a different angle. Instead of waiting for all replies for synthesis (like OpenRouter Fusion), it streams the "best" reply immediately (using a router to pick the best model) then synthesizes with emoji reactions and optional replies from the background models. It's free to use here with no login: http://pellmell.ai http://pellmell.ai
slashdave 2mo ago
Replace "Show HN:" with "Advertisement:" ?
indiantinker 2mo ago
I have been using this : https://magnitude.dev/ https://magnitude.dev/ for a while now. Is it something similar you are doing? I would love to have something that would connect to my codex, Claude, and opencode subscription rather than having to make a new subscription.
jmspring 2mo ago
So the word security or any topic related to it is mentioned and it flips to an older gen model? Fable is nearly useless now it you do anything around auth.
- dijit 2mo ago
  I can get it to write win32 unsafe rust code. I can't get it to review win32 unsafe rust code. Make it make sense.
  - jmspring 2mo ago
    <snark>need bigger check to guy in DC</snark>
purplecats 2mo ago
but u wouldnt get caching savings
- seizethecheese 2mo ago
  You definitely still would, but you need to pay the full input cost twice, so the equation really depends on how much first message vs repeat message matter.
  - retinaros 2mo ago
    Ttl 1 hour maybe. 5min? Never
fgoose180 2mo ago
enjoying how people are re-discovering ensemble methods.
cheema33 2mo ago
> Fable-level results at 1/3 the cost I am guessing this is not targeting those of us on the heavily subsidized $200/mo plans. Sure, these plans may be temporary, but none of us really know how temporary they are. Until then, 1/3rd of the published API pricing is not very appealing.
- killingtime74 2mo ago
  All enterprises users (people using them for work and not side projects) can't get the subsidized plans. I would say subsidized plans are a minority of usage?
  - nickthegreek 2mo ago
    The subsidized userbase is large enough that cheema is right to call this out this distinction for other readers.
  - rodrodrod 2mo ago
    People _can_ get subsidized plans for work: we use Claude Teams, $100/mo premium seat, which caps at 150 seats. Not enterprise tier, but fine for SMBs.
    - lnrd 2mo ago
      As long as you are fine with everything your team puts into Claude being retained by Anthropic. Afaik only enterprise API plans provide non data retention policies.
  - monk_grilla 2mo ago
    I imagine that most small to medium sized businesses are on either individual plans or Teams plans. The vast majority of firms do not need more than 150 seats, and API rates are not sustainable for most.
    - byzantinegene 2mo ago
      if this is true, the frontier labs are not able to justify their trillion dollar valuations, they are barely making anything on subsidized plans.
subygan 2mo ago
This does not really work well, if you don't know the complexity of the problem ahead of time and ensure all future conversations go to the same model. Else, you break the cache by doing a round robin of the same conversation across different models. Likely you'll end up paying more than what it would've cost with a cache aware system
- hahahaa 2mo ago
  Cache hits should be part of the strategy.
- lbriner 2mo ago
  You can use the Ralph Wiggum technique: https://ghuntley.com/ralph/ https://ghuntley.com/ralph/
alightsoul 2mo ago
There's so many people reimplementing Sakana fugu from its two ICLR papers but no open source version of it.
raver1975 2mo ago
I can't sign up: too many authentication attempts
- adam_rida 2mo ago
  looking at it now
islambaraka 2mo ago
What signals does Echo use to decide which models should participate before it has seen their outputs?
- spidercob 2mo ago
  [flagged]
qainsights 2mo ago
Can't sign up: `too many authentication attempts`
- adam_rida 2mo ago
  looking at it now
cantalopes 2mo ago
By the way, is it just me or opus 4.8 is much better at some programming tasks than fable? I've been really disappointed lately, i stopped using it evem though it's "premium on my subscription"
- meowface 2mo ago
  It's just you. Fable 5 is consistently better than Opus 4.8 at literally every single level, for me. (I always use both at xhigh, for reference.) I could go down a laundry list of various issues I have with Opus that I don't have with Fable. For me, Opus 4.8 < GPT-5.6 Sol < Fable 5. Plus Fable is way less annoying to talk to than Opus 4.8. Opus 4.8's writing style is absolutely insufferable. Fable has some of the same quirks but it's way less bad.
  - anonzzzies 2mo ago
    We see no difference between opus 4.8 and fable except that fable is slower. So +1 for not just GP. We use neither interactive, just via our own tooling so the ‘talk to’ doesn’t apply and we use it 24/7. We currently run 25% of tasks on both fable and opus and the rest only opus. The 25% are being code reviewed side by side and we do detect when fable switches to opus for ‘security concerns’ nonsense. Opus generally finishes sooner, results are similar quality.
Art9681 2mo ago
The "ensemble of models" is nothing new. It's just that it's not really a moat that can be monetized. In the end you're always paying for something. You can engineer an elaborate harness with multiple models but it is not going to particularly solve a novel problem that the frontier models can with the same level of efficiency. You're saving money and paying with time. You're going to pay with something one way or another no matter what. The frontier providers aren't dumb. They charge what they charge because they know this. If you think Fable is too expensive then the type of problems you are solving don't demand that level of capability. If you are working on something cutting edge, something truly novel, the cost of frontier AI is well worth its price. With all that being said. No one is going to complain if we can get the same capability at a lower cost. And I mean true parity. Not trading space for time.
ljlolel 2mo ago
we did it with 1/10 the cost with Prometheus-1.0 and further https://trustedrouter.com/blog/prometheus-2-new-draco-state-of-the-art https://trustedrouter.com/blog/prometheus-2-new-draco-state-...
moriwo-dev-ai 2mo ago
[flagged]
alexzhangai 2mo ago
[flagged]
adam_rida 2mo ago
thanks to everyone for taking the time to try Echo and share feedback, this is precisely why i wanted to launch early. i am going to try to address a couple of topics that came up often: - i'll keep publishing stronger evals, including more difficult coding and agentic benchmarks, to map out more precisely the differences with sota - the public eval dashboard will keep expanding and be updated (very open to more benchmark suggestions as well!) - some people found issues in the eval dashboard ui and the sign up flow, should be now all fixed in prod some important precisions as well: - NO credit card is required to try Echo - each acount includes 10$ of free credits to try on both the API and the chat on the approach itself: the idea i'm exploring is more broader than model routing, i'm looking at how to allocate inference efficiently across open-weight models, deciding not only which models to use, but also how much computation a request deserves and how intermediate work should be combined. ensembling by itself is not new. since random forests and probably even before in statistics/classic ml we knew that bringing multiple models together can outperform individual ones. the interesting problem for Echo is how to model and leverage this without paying the full ensemble cost at each request. while there are conceptual similarities with systems like Fusion or Fugu, the architecture and optimization objective are different. thanks again for all the thoughtful feedback.
- user_7832 2mo ago
  Small feedback: the "create password" requires a symbol too, which Google's password manager by default does not use. I'm fairly sure a double-digit-level alphanumeric jumble is sufficient to be a password (or at least, Google thinks so). Great idea nonetheless!
  - troupo 2mo ago
    > I'm fairly sure a double-digit-level alphanumeric jumble is sufficient to be a password This isn't required either. Of course there's an xlcd for that: https://xkcd.com/936/ https://xkcd.com/936/ Besides, --- start quote --- Using complexity requirements (that is, where staff can only use passwords that are suitably complex) is a poor defence against guessing attacks. It places an extra burden on users, many of whom will use predictable patterns (such as replacing the letter ‘o’ with a zero) to meet the required 'complexity' criteria. https://www.ncsc.gov.uk/collection/passwords/updating-your-approach https://www.ncsc.gov.uk/collection/passwords/updating-your-a... --- end quote ---
zhonglin 2mo ago
Sounds like another token router. Not sure what is the benifit compared with codex, I used 5.6 Sol, codex already route some of the task to luna not sol.
blobbers 2mo ago
I'm curious if there is measurable value in diversity of thought, and if there's diminishing returns on a single models thought pattern. For example, compute X tokens with model A, then feed those into model B, etc. to get chain of thought through a diverse set of mdoels rather than chain of thought through a heterogeneous chain. Humans seem to strongly believe echo chambers are bad. Are LLMs the same?
- adam_rida 2mo ago
  there are and this what you optimize for. ensemble learning has a long literature on this. you want models that have the most diverse pool of capabilities so they complement each other. in verifiable tasks or classification this is straightfroward but a bit tricker in open ended text gen or CoT. this is one of our core research question
  - blobbers 2mo ago
    My experience is mostly around either ensembles of weak learners that are bootstrapped to create a strong learner, or around boosted learners where you're training on residuals. With CoT you're sort of adding entropy to your answer, but you're also possibly getting a more thorough answer? I'm curious though if these training methods are convergent or are models actually different; just like how in the stock market people think they're "diversified" but the truth is their exposure is likely much more risk correlated than one might think. In certain situations, one right answer is better than a committee discussing the problem, but in others its sometimes nice to have some alternative methods of solving something. Fun project nonetheless. My approach to using multiple models has been less about CoT but more about time to first token, and how you can use a small model to start interacting with the user while in parallel the more complex model is building a larger more complex thought. My work on this was primarily for voice backed interfaces before the voice models became quite a lot faster.
    - adam_rida 2mo ago
      i think that’s exactly the core of the question. There is meaningful decorrelation across models in some domains, like language for instance. but i agree that it’s much less obvious for reasoning. I don’t think more models is necessarily better, and one of our research questions is also: when does adding another reasoning path actually add useful information, and when is a single strong path better than an ensemble?
springtimesun 2mo ago
I have been trying something like this with local only models. I think I’ve tried everything that will fit in 96GB alone and in concert with each other using real project data (Rails). I haven’t found much benefit to anything other than Qwen 3.6 27b dense feeding a Claude final pass. I really thought layering was going to work like the law of large numbers, but for my workload it just hasn’t.
- XCSme 2mo ago
  I have a spare 3090 that I want to use to off-load some tasks from Claude to a local model (probably Qwen 3.6 27b), any success with that? Is it good enough to follow some tasks, coding requirements or browser usage?
  - springtimesun 2mo ago
    Alone, Qwen can close basic bug tickets. Passing up and passing down have not really worked that well for me. In some cases passing Qwen’s output up to Claude can be actively harmful bc it can anchor the solution space to a bad approach. Passing down isn’t worth the time. By the time I have Claude plan something, most of the time it’s 1/3 or 1/4 more tokens to just have Claude do the implementation. Where it has worked is to have Qwen not just write the proposed solution, but also include all the files it consulted and why it did or didn’t consider them. The documentation is often longer than the patch, but it lets me prompt Claude differently. I tell Claude to challenge the implementation and find holes, edge cases and things not considered. Maybe it’s my imagination, but Claude kicks it into another gear when you tell it to evaluate the results of another model. Maybe this is a pure preference but I find Qwen to be bad at UI and these edits Claude will often let go. One thing I haven’t tried, but will at some point is to hand Qwen all my half assed ideas and things I can’t be bothered to spec out. Probably not to take the output, but to help me draft a more complete issue because it’s only costing me electricity.
mandarinclips 2mo ago
[dead]
thatxliner 2mo ago
How is this different from what that feature OpenRouter does?
- hahahaa 2mo ago
  Not much probably but I'd rather my agentic system route to model choice (happy to use OpenRouter as a catalog) than something in the cloud. I.e. pi.dev routes > call this and it decides based on today's algo. In other words routing should be in my source code not yours.
sudo_cowsay 2mo ago
Hi this is very cool. As a student (read: more dumb than adults), I find the eval page too loaded with words. Maybe more graphs could help?
runtime_lens 2mo ago
[flagged]
- deleted 2mo ago
  [deleted]
- yieldcrv 2mo ago
  there are so many branches of evolution here, I also see them converging to a "best model becomes a niche concept" too the supercycle is on device models, and one of those evolutions is models baked into chip die, and you just upgrade chipsets every few years instead so it's the hyperscalers that will take the L in that environment
- eunos 2mo ago
  Isnt that the idea from gemini cli?
wayknow 2mo ago
[flagged]
deleted 2mo ago
[deleted]
jdthedisciple 2mo ago
Grandiose claims that are hardly backed up at all. I find that very off-turning!
- elnatro 2mo ago
  Have you taken a look at https://echo.tracerml.ai/eval/ https://echo.tracerml.ai/eval/? It looks promising for sure!
  - jdthedisciple 2mo ago
    None of those are SWE. And still Fable beats it hands down 8-0 in one of them, and is at worst even in some others. Also it doesn't make logical sense: A router can save costs, yes, but not magically be "smarter" somehow. That's like selling "free energy".
- saberience 2mo ago
  Sounds very scam-like. Literally promising frontier-equal results but at 1/3 price, i.e. cheaper than Kimi K3? Doesn't offer any real benchmarks or explanation of how this magic trick is accomplished. No credible team or notable scientists behind it...
haris599 2mo ago
Can you simply ensemble the outputs of all N models for each token? I'm curious how that compares to explicitly routing each token to a different model.
Arshad-Talpur 2mo ago
The business use case of rather than going on one model choosing the best openweight model and reducing the cost seems fascinating, however the context memory, or auditability of what is happening behind would be more complicated, even within single model we have to spend tons of time to decode and understand llm behavior , add context layers and so on, having said that for GenAI executions this might be the direction. Wish you best for the project
- huflungdung 2mo ago
  [dead]
alex-moon 2mo ago
I started building something like this myself (codename HFG - "Heroku for Groq") anticipating what we are indeed now seeing in ChatGPT viz. enshittification. The intention was to aim it at consumers and non-technical users who want a chatbot to help them draft reports, synthesise papers and so on. You've got a bunch of hackers in the comments going "But how can I run my own evals?" I think the answer for them is: this isn't for you. Don't want to derail what you're trying to do with Echo in case I'm wide of the mark, but yeah even in that case, if you hadn't considered that use case for it, I reckon there will, probably inside six months, be a substantial market for non-technical users who are sick of seeing ads in a service they already pay a subscription for, and who don't care what the underlying model is - or, indeed, don't even understand the concept of an "underlying model" because they interface with AI as a product. You have already taken the HFG idea way further than I had even thought of yet, and I feel vindicated in seeing someone else do it. I wish you the very best!
SubiculumCode 2mo ago
Isn't the issue with the approach generally that it ends up costing more?
stevefan1999 2mo ago
So it is a mixture of models, then each model have a mixture of experts, making it mixture of mixture of experts...why don't we just make it simple and have a central expert router made up of random forest...all we need some kind of actor system to share the gradients of the experts...and ensemble it
zuzululu 2mo ago
1/3rd of fable ? not interested
fmx 2mo ago
A "Message Echo" textbox that makes it look like you can get a response to a prompt without logging in, only to redirect to the sign-up page. Such a classic dark pattern - and such a sure way to get me to leave your site immediately. I've literally taken one step on your website - the one your site design invited me to take - and immediately got tripped up. I'm not coming back.
- pizzao 2mo ago
  On the other hand, the creator would need to pay for those initial queries. This also invites for misues that sum up to large bill for the creator. I dont know, it's something I can understand as someone also building AI stuff
  - lukan 2mo ago
    Just don't use this dark pattern and rather be clear sign up is reqired, but comes with 10$ of free credits and no strings attached?
  - boesboes 2mo ago
    Just dont' put a fucking fake chat box there. not that hard.
  - idonotknowwhy 2mo ago
    Then the creator should have a sign up button, not a fake chatbox. This dark pattern is reminiscent of those online test sites in the 2000's where you spend 10 minutes filling out some quiz, then get prompted for an email address to see the results. https://chat.mistral.ai/chat https://chat.mistral.ai/chat <- let me chat and actually responded without signing up. MoonshotAI had this fake chat box dark pattern. So I signed up with Mistral instead.
  - 7bit 2mo ago
    Nobody asked it to be for free.
- barapa 2mo ago
  [flagged]
matchartier 2mo ago
[flagged]
mohammedmsgm 2mo ago
[flagged]
datadrivenangel 2mo ago
So this is basically overfitting to the benchmaxxed open models. Or more positively, quality routing based on skills. makes sense if it works!
motbus3 2mo ago
i have being using anthropic opus 4.8 and fable 5 for some time now. i also tried latest openai models. in short, all of them are token diarrhea. Annoyingly logorrheic. they output so much useless stuff that it comes to the point that it makes me thing if that's not on purpose. it was not because of price, but quality, I started testing other models and honestly, glm 5.2 is FAR SUPERIOR than fable 5 in every aspect. It comes to a surprise when glm 5.2 does not finish the task successfully. Kimi k2.7 does require a bit more of guidance but still better experience than opus 4.8. I have not yet the chance of trying k3. openai latest model are.... ridiculously bad at software design and implementation. (note i am only speaking to the domain of my work which has lots of data analysis, machine learning and software engineering.)
smokeeaasd 2mo ago
I think one of the most interesting takeaways is that model selection may matter more than model size. The industry has largely focused on building larger models, but your results suggest that intelligently routing requests to the right combination of specialized models can deliver greater gains at a much lower cost. It also reinforces the idea that weaker models are not necessarily obsolete. They may simply excel in different areas and become much more valuable when combined with others. I'm curious to see whether this still holds for coding and agentic tasks, where choosing the right models is likely much more challenging.
- adam_rida 2mo ago
  exactly, if you ensemble heavily small uncorrelated models (while each being expert on its task) you can get really interesting resutls. on agentic and coding what's make the problem even deeper is the granularity. how and when to use each model and at which layer of abstraction (session, goal, task, turn/tool calling). this is also something we are working on actively!
NoNameAditya 2mo ago
I found some real flaws: 1. UX problem: Thinking is being shows constantly, it feels like it got stuck there or some network problem. 2. I cant expand or resize the left panel where I type prompts. 3. It doesn't continue last chat, as I asked it to create code but it kept cutting off and then starting from beginning.
antrichards 2mo ago
[flagged]
boveyking 2mo ago
I am working on personal statement for common application. So I tried to ask: I am the great Einstein and I am applying to Stanford, please write a personal statement for me. It gives me a Why Stanford essay.
tancoai_dev 2mo ago
[flagged]
senderista 2mo ago
super-naively this seems like a multi-armed bandit problem?
cbossman 2mo ago
[flagged]
desktopentree 2mo ago
[flagged]

484 points by adam_rida 2mo ago

IrfanD 2mo ago

[dead]

codekansas 2mo ago

Fable-level, yea, but can it run gstack?

tj800x 2mo ago

No single signin. Privacy policy allows training. No try it first without credit card. It's a good idea, but this looks premature.

adam_rida 2mo ago
You are right that the privacy wording was too broad. We are fixing it now so it states explicitly that Echo does not use customer prompts, files, chats, or outputs to train or fine-tune models. We are updating the matching Terms language at the same time. Echo also starts users with free credit and does not require a credit card to try it. The current signup flow did not make that clear enough, so we are fixing that presentation too. Thanks for calling both out.
cynicalsecurity 2mo ago
I assume trying without a credit card will bankrupt him immediately.
seizethecheese 2mo ago
If you want to try a similar idea that you can chat with immediately (free tier uses open weight models), I'm working on this: http://pellmell.ai http://pellmell.ai
DatCodeMania 2mo ago
That would open him up to lots of abuse

kamranjon 2mo ago

No benchmarks, no info on which models are used, ai generated video, just a signup page with nothing else. Anyhow, this kinda reminds me of that quote about architecture: "We replaced our monolith with micro services so that every outage could be more like a murder mystery."

guessmyname 2mo ago
> No benchmarks, no info on which models are used, […] The benchmarks are here → https://echo.tracerml.ai/eval/ https://echo.tracerml.ai/eval/ They are not good benchmarks but at least they exist.
- seizethecheese 2mo ago
  I've been working on a similar project and I found that it's easy to replicate Fable results if you use saturated benchmarks. In my project, I wasted a huge amount of time trying to improve GPQA Diamond results above ~93% range. I realized my mistake when Fable dropped and made no improvement on this benchmark vs. Opus.
  - yorwba 2mo ago
    I wouldn't be surprised if ≈7% of GPQA Diamond questions simply have the wrong answer in the ground truth data, so that getting such a question correct is graded as an error. Most machine-learning benchmarks are rather badly validated.
    - seizethecheese 2mo ago
      Yep! I found this interesting article after banging my head against a wall for a long time: https://epoch.ai/gradient-updates/gpqa-diamond-whats-left https://epoch.ai/gradient-updates/gpqa-diamond-whats-left
j45 2mo ago
It's easy enough to copy and paste in a prompt, no? Eval tests while giving general indicators might not be similar for each use case.

deleted 2mo ago

[deleted]

adam_rida 2mo ago
Thanks for checking the individual rows. HumanEval+ is one small code slice, not the whole basis for the launch claim. The public evaluator currently contains 907 rows across seven benchmark families, and matched SWE-bench Verified and BigCodeBench runs are the next code evidence being added. You also found a real UI bug: the inspector should show both stored answers and currently does not in some rows. We are fixing that. On the row you reran: the page records a frozen matched run. It does not claim that Fable is incapable of solving that prompt on another run. We are adding repeated matched trials and making run count and variance explicit. Your rerun is exactly the kind of external check the row-level page is intended to make possible. The broader result remains: Echo is competitive with Fable across the evaluated task mix at materially lower measured inference cost. We are filling in the harder agentic-code evidence now rather than asking anyone to infer it from HumanEval+.

fneddy 2mo ago

That’s basically the same idea IBM advertises with Bob?

seizethecheese 2mo ago
This looks different, it's not a main agent delegating to subagents, it's a router.

jmaw 2mo ago

I think approaches like this have potential. Only time will tell. This reminds me of the mixture of experts taken by deepseek r2 (I think it was r2, at least), but less specific models I guess. I have often wondered how tools like GHCP choose the best model for the job when set to "auto".

Alifatisk 2mo ago

This reminds me on OpenRouters report that combining multiple different models gave comparable performance to Fable 5. I think this approach has lots of potential. Maybe OpenAi was ahead of its time with GPT-5 (it being a router to different models rather than just being one new model)

seizethecheese 2mo ago
You might be interested in a project that I'm working on, which is kind of like OpenRouter Fusion, but instead of waiting for all models to synthesize, we stream the best model immediately and background the rest. The background models then reconcile with an emoji reaction and optional reply. It gets similar results to Fusion and is a lot faster! There's a free version that leverages open weight models here: http://pellmell.ai http://pellmell.ai.
- hmokiguess 2mo ago
  This is really cool, super fun!
- NetOpWibby 2mo ago
  I like that I was able to test without an account and my prompt (What's the feasibility of a NetNavi IRL, along with a PErsonal Terminal? Create a document that outlines how to make this happen.) makes me think Pellmell is gonna be a research tool I'll continue to use.
tintor 2mo ago
OpenAI's router chooses between models of different sizes, which are still trained on roughly the same data. Its purpose is to reduce infra cost for OpenAI for simpler queries. No need to pay for GPT 5.6 Sol inference for "Hello" prompt.

j45 2mo ago

If you copy Perplexity, they let you have the first few rounds of chat for free to get you going before asking to sign up.

jambalaya8 2mo ago

Might want to rethink the name to avoid an Amazon issue.

wizche 2mo ago

how does this differs from OpenRouter fusion?

cawingcrow 2mo ago
how does this differs from Sakana Fugu?
- deleted 2mo ago
  [deleted]

bnjemian 2mo ago

I don’t find the recent spate of blog posts and systems delegating and combining LLMs to get better performance particularly interesting. Especially given that anyone who’s taken an ML 101 course has learned about ensemble methods. While an LLM isn’t what you’d traditionally consider a weak learner, the theorems on learning systems clearly point to them being so in this context. The feigned surprise at combining them to yield better results seems disingenuous. Even so, the work to predict which models are best suited for which task, how to delegate, and how to combine their outputs is interesting, especially if you’re placing a cost minimization objective on it. That said, this isn’t too far off from what many AI labs are already doing.

abernard1 2mo ago
The entirety of "agents" and tool calls is a process of combining LLMs to get better results. Is it the same LLM in many cases? Yes. But it doesn't have to be. It's the natural move that happened after people realized you couldn't throw away half a century of AI research. Most of these focus on costs. But it is simply the case that the one-shot output did not scale for harder problems on workflows.

theneocorner 2mo ago

[dead]

ototot 2mo ago

Is this yet another Sakana Fugu / OpenRouter Fusion?

kachnuv_ocasek 2mo ago
Looks like it but with open-weights only.
code0igx 2mo ago
The architecture does convey the same.

yonatan8070 2mo ago

I'm not an expert on this, but this sounds a lot like a larger-scale MoE (Mixture of Experts) type of architecture. As I understand it, in an MoE model, you essentially have hundreds of smaller sub-models ("experts") that are good at different tasks, and for every generated token, a single "master" model chooses which ones are most relevant to participate, and you only activate them.

janalsncm 2mo ago
In MoE systems the routing decision is made per-token, not per prompt or task. It’s one of ML’s many confusing naming conventions. Even more confusingly, there are older pre-LLM MoE systems which ensemble and pool the predictions from multiple sub-components. For example in a random forest you could take the majority vote of the decision trees or the average of their numerical predictions. After that, we developed neural net architectures for predicting a single thing like whether the user will click on your ad. An MMoE is in the same family. And so now we are at massive MoE networks for LLMs which have similarities with MMoE in that the “decision” is about the very next token to predict.
- lukan 2mo ago
  "In MoE systems the routing decision is made per-token, not per prompt or task." Have there been experiments with doing it per task? Like, "oh this is python project, use this model" "oh this is about writing fantasy, use this"?
  - janalsncm 2mo ago
    It’s a good idea. The results probably depend a lot on how close your task is to the benchmarks though. I think OpenAI already has (had?) a feature like this called “auto” mode for thinking.

bbstats 2mo ago

M-o-MoE

maxdo 2mo ago

such a scam, there is only one fable-like model, that somewhat behind, it cost half, not 3x. so from here you can stop reading.

seizethecheese 2mo ago
They're obviously not claiming this is a new model that is fable-like at 1/3. It's a router that saves money by only using Fable when necessary. I don't think you should be calling someone's post a "scam" without doing a minimum of research. (I'm not associated with the company, but very interested in this.)

janalsncm 2mo ago

Intuitively, your savings depend heavily on how hard the tasks are in the first place. If you have a base rate where 99% of your tasks can be routed to a cheap model, yeah, you can save a ton by not using Fable for that. So “1/3 the cost” really depends.

seizethecheese 2mo ago
Sometimes expensive models are cheaper on easier tasks because they use fewer tokens, too.
- janalsncm 2mo ago
  Yeah apparently Opus and Sonnet are like that.
  - seizethecheese 2mo ago
    Yep! There are a lot of models like this: https://artificialanalysis.ai/models/capabilities/coding?cost-per-task=index-vs-cost-per-task https://artificialanalysis.ai/models/capabilities/coding?cos...

ninjahawk1 2mo ago

I’m very confused on what this is, my initial thought was “oh nice, open source router.” I go to the website…and it’s a sign up. I expected a repo. Otherwise how do I use it? As a SaaS? Yeah right. Oh well I guess at least the benchmarks are good…I find the benchmarks and many are either not present or are not what the title claims. My main question is how this has so many updoots from HN, probably the passerby not looking closer for sure. I mean no offense and I really do wish you best on this, but it seems like what we used to call back in the day, vaporware.

jacobgold 2mo ago

> Fable-level results at 1/3 the cost using open-weight models But we get ~$2500/mo worth of Fable credits for $200/mo on Anthropic pan? I'm still confused why people (who don't have to use API billing) are chasing open weight models based on cost.

teruakohatu 2mo ago
Because that is a short term solution, it won’t be offered forever. Large organisations have to purchase credits at $/tokens. Eventually everyone else will too.
- sscaryterry 2mo ago
  This is what OpenAI and Anthropic are trying to make everyone believe. Most accountants will flinch at this (they already are). The $200 odd plans are already out of reach of many, many people. The attrition of customers if they were to get rid of these subscriptions plans would be untenable.
  - recursivegirth 2mo ago
    I think you are looking at it incorrectly. No business is buying individual accounts, because if they do, they open themselves up to considerable risk. The $200 plans are priced so that the power-users use them and then advocate about how great the product is. If you're buying a $200 plan, you're not doing it because of the price point but rather because of the amount of work it is doing for you.
    - trollbridge 2mo ago
      Lots of businesses are buying and using these plans. Basically every small business I interact with.
    - anonzzzies 2mo ago
      All companies we interact with have 200 plans.

dluan 2mo ago

So this is the dogpile.com of the askjeeves, alta vista, and lycos approach? Time is a flat circle?

glasss 2mo ago
Good ideas are usually still good across time and tools
- tiffanyh 2mo ago
  Exactly High quality, fast & cheap (all 3 combined) - is a formula success. It’s just way easier said than done.
  - Avicebron 2mo ago
    There's a reason most pros will tell you pick two of the three.
- onlyrealcuzzo 2mo ago
  Dogpile was only a good idea while Search Engines were mostly trash. You needed to search all of them to find something decent. That's roughly analogous to today. Ignoring cost, you'd be way better off asking all the LLMs to solve a problem (like coding) where you can verify the answer. So the question is, for things like that -> can a group of models perform better than frontier models, especially at a reasonable cost? Fable is not a great value, so unless you're trying to find answers to Erdos questions, you can probably do better on cost. You can probably typically ask 3 or 4 of the top Chinese models for an answer and get a response for the same Fable question... Given that Fable isn't that much better, it's not surprising you can do better for a large subset of problems.
  - Terretta 2mo ago
    > Dogpile was only a good idea while Search Engines were mostly trash. Precisely.

hmokiguess 2mo ago

"Backed by YCombinator" https://www.ycombinator.com/companies?query=tracerml https://www.ycombinator.com/companies?query=tracerml I don't see it?

deleted 2mo ago
[deleted]
zachdotai 2mo ago
Not all YC companies have launched publicly yet but I am a current YC founder and I can confirm they exist in the internal directory.
dang 2mo ago
You're right to be skeptical; I saw a totally fake case of this just yesterday. But in the present case, they're just a startup in the current batch.
- hmokiguess 2mo ago
  Thank you! Hard to trust things these days.

meander_water 2mo ago

Seems similar to Openrouter Fusion - https://openrouter.ai/docs/guides/routing/routers/fusion-router https://openrouter.ai/docs/guides/routing/routers/fusion-rou...

seizethecheese 2mo ago
Fusion is a totally different approach, though similar in the sense that it leverages different models. Fusion generates many replies then synthesizes. This adds a ton of latency and cost, so it's going to be better only for cases where you're willing to wait a lot and pay a lot more. Routers (like this project) are a different thing, they can theoretically improve performance and cost at the same time without increasing latency much. I'm a bit skeptical though, since knowing which LLM is going to be better on a cost adjusted basis is hard (see https://artificialanalysis.ai/models/capabilities/coding?cost-per-task=index-vs-cost-per-task https://artificialanalysis.ai/models/capabilities/coding?cos..., where the cost per task vs. performance is not what you expect, for example comparing Qwen 3.7 Max to GPT Sol. A project I'm working on is aimed at improving performance without added latency but from a different angle. Instead of waiting for all replies for synthesis (like OpenRouter Fusion), it streams the "best" reply immediately (using a router to pick the best model) then synthesizes with emoji reactions and optional replies from the background models. It's free to use here with no login: http://pellmell.ai http://pellmell.ai

slashdave 2mo ago

Replace "Show HN:" with "Advertisement:" ?

indiantinker 2mo ago

I have been using this : https://magnitude.dev/ https://magnitude.dev/ for a while now. Is it something similar you are doing? I would love to have something that would connect to my codex, Claude, and opencode subscription rather than having to make a new subscription.

jmspring 2mo ago

So the word security or any topic related to it is mentioned and it flips to an older gen model? Fable is nearly useless now it you do anything around auth.

dijit 2mo ago
I can get it to write win32 unsafe rust code. I can't get it to review win32 unsafe rust code. Make it make sense.
- jmspring 2mo ago
  <snark>need bigger check to guy in DC</snark>

purplecats 2mo ago

but u wouldnt get caching savings

seizethecheese 2mo ago
You definitely still would, but you need to pay the full input cost twice, so the equation really depends on how much first message vs repeat message matter.
- retinaros 2mo ago
  Ttl 1 hour maybe. 5min? Never

fgoose180 2mo ago

enjoying how people are re-discovering ensemble methods.

cheema33 2mo ago

> Fable-level results at 1/3 the cost I am guessing this is not targeting those of us on the heavily subsidized $200/mo plans. Sure, these plans may be temporary, but none of us really know how temporary they are. Until then, 1/3rd of the published API pricing is not very appealing.

killingtime74 2mo ago
All enterprises users (people using them for work and not side projects) can't get the subsidized plans. I would say subsidized plans are a minority of usage?
- nickthegreek 2mo ago
  The subsidized userbase is large enough that cheema is right to call this out this distinction for other readers.
- rodrodrod 2mo ago
  People _can_ get subsidized plans for work: we use Claude Teams, $100/mo premium seat, which caps at 150 seats. Not enterprise tier, but fine for SMBs.
  - lnrd 2mo ago
    As long as you are fine with everything your team puts into Claude being retained by Anthropic. Afaik only enterprise API plans provide non data retention policies.
- monk_grilla 2mo ago
  I imagine that most small to medium sized businesses are on either individual plans or Teams plans. The vast majority of firms do not need more than 150 seats, and API rates are not sustainable for most.
  - byzantinegene 2mo ago
    if this is true, the frontier labs are not able to justify their trillion dollar valuations, they are barely making anything on subsidized plans.

subygan 2mo ago

This does not really work well, if you don't know the complexity of the problem ahead of time and ensure all future conversations go to the same model. Else, you break the cache by doing a round robin of the same conversation across different models. Likely you'll end up paying more than what it would've cost with a cache aware system

hahahaa 2mo ago
Cache hits should be part of the strategy.
lbriner 2mo ago
You can use the Ralph Wiggum technique: https://ghuntley.com/ralph/ https://ghuntley.com/ralph/

alightsoul 2mo ago

There's so many people reimplementing Sakana fugu from its two ICLR papers but no open source version of it.

raver1975 2mo ago

I can't sign up: too many authentication attempts

adam_rida 2mo ago
looking at it now

islambaraka 2mo ago

What signals does Echo use to decide which models should participate before it has seen their outputs?

spidercob 2mo ago
[flagged]

qainsights 2mo ago

Can't sign up: `too many authentication attempts`

adam_rida 2mo ago
looking at it now

cantalopes 2mo ago

By the way, is it just me or opus 4.8 is much better at some programming tasks than fable? I've been really disappointed lately, i stopped using it evem though it's "premium on my subscription"

meowface 2mo ago
It's just you. Fable 5 is consistently better than Opus 4.8 at literally every single level, for me. (I always use both at xhigh, for reference.) I could go down a laundry list of various issues I have with Opus that I don't have with Fable. For me, Opus 4.8 < GPT-5.6 Sol < Fable 5. Plus Fable is way less annoying to talk to than Opus 4.8. Opus 4.8's writing style is absolutely insufferable. Fable has some of the same quirks but it's way less bad.
- anonzzzies 2mo ago
  We see no difference between opus 4.8 and fable except that fable is slower. So +1 for not just GP. We use neither interactive, just via our own tooling so the ‘talk to’ doesn’t apply and we use it 24/7. We currently run 25% of tasks on both fable and opus and the rest only opus. The 25% are being code reviewed side by side and we do detect when fable switches to opus for ‘security concerns’ nonsense. Opus generally finishes sooner, results are similar quality.

Art9681 2mo ago

The "ensemble of models" is nothing new. It's just that it's not really a moat that can be monetized. In the end you're always paying for something. You can engineer an elaborate harness with multiple models but it is not going to particularly solve a novel problem that the frontier models can with the same level of efficiency. You're saving money and paying with time. You're going to pay with something one way or another no matter what. The frontier providers aren't dumb. They charge what they charge because they know this. If you think Fable is too expensive then the type of problems you are solving don't demand that level of capability. If you are working on something cutting edge, something truly novel, the cost of frontier AI is well worth its price. With all that being said. No one is going to complain if we can get the same capability at a lower cost. And I mean true parity. Not trading space for time.

ljlolel 2mo ago

we did it with 1/10 the cost with Prometheus-1.0 and further https://trustedrouter.com/blog/prometheus-2-new-draco-state-of-the-art https://trustedrouter.com/blog/prometheus-2-new-draco-state-...

moriwo-dev-ai 2mo ago

[flagged]

alexzhangai 2mo ago

[flagged]

adam_rida 2mo ago

thanks to everyone for taking the time to try Echo and share feedback, this is precisely why i wanted to launch early. i am going to try to address a couple of topics that came up often: - i'll keep publishing stronger evals, including more difficult coding and agentic benchmarks, to map out more precisely the differences with sota - the public eval dashboard will keep expanding and be updated (very open to more benchmark suggestions as well!) - some people found issues in the eval dashboard ui and the sign up flow, should be now all fixed in prod some important precisions as well: - NO credit card is required to try Echo - each acount includes 10$ of free credits to try on both the API and the chat on the approach itself: the idea i'm exploring is more broader than model routing, i'm looking at how to allocate inference efficiently across open-weight models, deciding not only which models to use, but also how much computation a request deserves and how intermediate work should be combined. ensembling by itself is not new. since random forests and probably even before in statistics/classic ml we knew that bringing multiple models together can outperform individual ones. the interesting problem for Echo is how to model and leverage this without paying the full ensemble cost at each request. while there are conceptual similarities with systems like Fusion or Fugu, the architecture and optimization objective are different. thanks again for all the thoughtful feedback.

user_7832 2mo ago
Small feedback: the "create password" requires a symbol too, which Google's password manager by default does not use. I'm fairly sure a double-digit-level alphanumeric jumble is sufficient to be a password (or at least, Google thinks so). Great idea nonetheless!
- troupo 2mo ago
  > I'm fairly sure a double-digit-level alphanumeric jumble is sufficient to be a password This isn't required either. Of course there's an xlcd for that: https://xkcd.com/936/ https://xkcd.com/936/ Besides, --- start quote --- Using complexity requirements (that is, where staff can only use passwords that are suitably complex) is a poor defence against guessing attacks. It places an extra burden on users, many of whom will use predictable patterns (such as replacing the letter ‘o’ with a zero) to meet the required 'complexity' criteria. https://www.ncsc.gov.uk/collection/passwords/updating-your-approach https://www.ncsc.gov.uk/collection/passwords/updating-your-a... --- end quote ---

zhonglin 2mo ago

Sounds like another token router. Not sure what is the benifit compared with codex, I used 5.6 Sol, codex already route some of the task to luna not sol.

blobbers 2mo ago

I'm curious if there is measurable value in diversity of thought, and if there's diminishing returns on a single models thought pattern. For example, compute X tokens with model A, then feed those into model B, etc. to get chain of thought through a diverse set of mdoels rather than chain of thought through a heterogeneous chain. Humans seem to strongly believe echo chambers are bad. Are LLMs the same?

adam_rida 2mo ago
there are and this what you optimize for. ensemble learning has a long literature on this. you want models that have the most diverse pool of capabilities so they complement each other. in verifiable tasks or classification this is straightfroward but a bit tricker in open ended text gen or CoT. this is one of our core research question
- blobbers 2mo ago
  My experience is mostly around either ensembles of weak learners that are bootstrapped to create a strong learner, or around boosted learners where you're training on residuals. With CoT you're sort of adding entropy to your answer, but you're also possibly getting a more thorough answer? I'm curious though if these training methods are convergent or are models actually different; just like how in the stock market people think they're "diversified" but the truth is their exposure is likely much more risk correlated than one might think. In certain situations, one right answer is better than a committee discussing the problem, but in others its sometimes nice to have some alternative methods of solving something. Fun project nonetheless. My approach to using multiple models has been less about CoT but more about time to first token, and how you can use a small model to start interacting with the user while in parallel the more complex model is building a larger more complex thought. My work on this was primarily for voice backed interfaces before the voice models became quite a lot faster.
  - adam_rida 2mo ago
    i think that’s exactly the core of the question. There is meaningful decorrelation across models in some domains, like language for instance. but i agree that it’s much less obvious for reasoning. I don’t think more models is necessarily better, and one of our research questions is also: when does adding another reasoning path actually add useful information, and when is a single strong path better than an ensemble?

springtimesun 2mo ago

I have been trying something like this with local only models. I think I’ve tried everything that will fit in 96GB alone and in concert with each other using real project data (Rails). I haven’t found much benefit to anything other than Qwen 3.6 27b dense feeding a Claude final pass. I really thought layering was going to work like the law of large numbers, but for my workload it just hasn’t.

XCSme 2mo ago
I have a spare 3090 that I want to use to off-load some tasks from Claude to a local model (probably Qwen 3.6 27b), any success with that? Is it good enough to follow some tasks, coding requirements or browser usage?
- springtimesun 2mo ago
  Alone, Qwen can close basic bug tickets. Passing up and passing down have not really worked that well for me. In some cases passing Qwen’s output up to Claude can be actively harmful bc it can anchor the solution space to a bad approach. Passing down isn’t worth the time. By the time I have Claude plan something, most of the time it’s 1/3 or 1/4 more tokens to just have Claude do the implementation. Where it has worked is to have Qwen not just write the proposed solution, but also include all the files it consulted and why it did or didn’t consider them. The documentation is often longer than the patch, but it lets me prompt Claude differently. I tell Claude to challenge the implementation and find holes, edge cases and things not considered. Maybe it’s my imagination, but Claude kicks it into another gear when you tell it to evaluate the results of another model. Maybe this is a pure preference but I find Qwen to be bad at UI and these edits Claude will often let go. One thing I haven’t tried, but will at some point is to hand Qwen all my half assed ideas and things I can’t be bothered to spec out. Probably not to take the output, but to help me draft a more complete issue because it’s only costing me electricity.

mandarinclips 2mo ago

[dead]

thatxliner 2mo ago

How is this different from what that feature OpenRouter does?

hahahaa 2mo ago
Not much probably but I'd rather my agentic system route to model choice (happy to use OpenRouter as a catalog) than something in the cloud. I.e. pi.dev routes > call this and it decides based on today's algo. In other words routing should be in my source code not yours.

sudo_cowsay 2mo ago

Hi this is very cool. As a student (read: more dumb than adults), I find the eval page too loaded with words. Maybe more graphs could help?

runtime_lens 2mo ago

[flagged]

deleted 2mo ago
[deleted]
yieldcrv 2mo ago
there are so many branches of evolution here, I also see them converging to a "best model becomes a niche concept" too the supercycle is on device models, and one of those evolutions is models baked into chip die, and you just upgrade chipsets every few years instead so it's the hyperscalers that will take the L in that environment
eunos 2mo ago
Isnt that the idea from gemini cli?

wayknow 2mo ago

[flagged]

deleted 2mo ago

[deleted]

jdthedisciple 2mo ago

Grandiose claims that are hardly backed up at all. I find that very off-turning!

elnatro 2mo ago
Have you taken a look at https://echo.tracerml.ai/eval/ https://echo.tracerml.ai/eval/? It looks promising for sure!
- jdthedisciple 2mo ago
  None of those are SWE. And still Fable beats it hands down 8-0 in one of them, and is at worst even in some others. Also it doesn't make logical sense: A router can save costs, yes, but not magically be "smarter" somehow. That's like selling "free energy".
saberience 2mo ago
Sounds very scam-like. Literally promising frontier-equal results but at 1/3 price, i.e. cheaper than Kimi K3? Doesn't offer any real benchmarks or explanation of how this magic trick is accomplished. No credible team or notable scientists behind it...

haris599 2mo ago

Can you simply ensemble the outputs of all N models for each token? I'm curious how that compares to explicitly routing each token to a different model.

Arshad-Talpur 2mo ago

The business use case of rather than going on one model choosing the best openweight model and reducing the cost seems fascinating, however the context memory, or auditability of what is happening behind would be more complicated, even within single model we have to spend tons of time to decode and understand llm behavior , add context layers and so on, having said that for GenAI executions this might be the direction. Wish you best for the project

huflungdung 2mo ago
[dead]

alex-moon 2mo ago

I started building something like this myself (codename HFG - "Heroku for Groq") anticipating what we are indeed now seeing in ChatGPT viz. enshittification. The intention was to aim it at consumers and non-technical users who want a chatbot to help them draft reports, synthesise papers and so on. You've got a bunch of hackers in the comments going "But how can I run my own evals?" I think the answer for them is: this isn't for you. Don't want to derail what you're trying to do with Echo in case I'm wide of the mark, but yeah even in that case, if you hadn't considered that use case for it, I reckon there will, probably inside six months, be a substantial market for non-technical users who are sick of seeing ads in a service they already pay a subscription for, and who don't care what the underlying model is - or, indeed, don't even understand the concept of an "underlying model" because they interface with AI as a product. You have already taken the HFG idea way further than I had even thought of yet, and I feel vindicated in seeing someone else do it. I wish you the very best!

SubiculumCode 2mo ago

Isn't the issue with the approach generally that it ends up costing more?

stevefan1999 2mo ago

So it is a mixture of models, then each model have a mixture of experts, making it mixture of mixture of experts...why don't we just make it simple and have a central expert router made up of random forest...all we need some kind of actor system to share the gradients of the experts...and ensemble it

zuzululu 2mo ago

1/3rd of fable ? not interested

fmx 2mo ago

A "Message Echo" textbox that makes it look like you can get a response to a prompt without logging in, only to redirect to the sign-up page. Such a classic dark pattern - and such a sure way to get me to leave your site immediately. I've literally taken one step on your website - the one your site design invited me to take - and immediately got tripped up. I'm not coming back.

pizzao 2mo ago
On the other hand, the creator would need to pay for those initial queries. This also invites for misues that sum up to large bill for the creator. I dont know, it's something I can understand as someone also building AI stuff
- lukan 2mo ago
  Just don't use this dark pattern and rather be clear sign up is reqired, but comes with 10$ of free credits and no strings attached?
- boesboes 2mo ago
  Just dont' put a fucking fake chat box there. not that hard.
- idonotknowwhy 2mo ago
  Then the creator should have a sign up button, not a fake chatbox. This dark pattern is reminiscent of those online test sites in the 2000's where you spend 10 minutes filling out some quiz, then get prompted for an email address to see the results. https://chat.mistral.ai/chat https://chat.mistral.ai/chat <- let me chat and actually responded without signing up. MoonshotAI had this fake chat box dark pattern. So I signed up with Mistral instead.
- 7bit 2mo ago
  Nobody asked it to be for free.
barapa 2mo ago
[flagged]

matchartier 2mo ago

[flagged]

mohammedmsgm 2mo ago

[flagged]

datadrivenangel 2mo ago

So this is basically overfitting to the benchmaxxed open models. Or more positively, quality routing based on skills. makes sense if it works!

motbus3 2mo ago

i have being using anthropic opus 4.8 and fable 5 for some time now. i also tried latest openai models. in short, all of them are token diarrhea. Annoyingly logorrheic. they output so much useless stuff that it comes to the point that it makes me thing if that's not on purpose. it was not because of price, but quality, I started testing other models and honestly, glm 5.2 is FAR SUPERIOR than fable 5 in every aspect. It comes to a surprise when glm 5.2 does not finish the task successfully. Kimi k2.7 does require a bit more of guidance but still better experience than opus 4.8. I have not yet the chance of trying k3. openai latest model are.... ridiculously bad at software design and implementation. (note i am only speaking to the domain of my work which has lots of data analysis, machine learning and software engineering.)

smokeeaasd 2mo ago

I think one of the most interesting takeaways is that model selection may matter more than model size. The industry has largely focused on building larger models, but your results suggest that intelligently routing requests to the right combination of specialized models can deliver greater gains at a much lower cost. It also reinforces the idea that weaker models are not necessarily obsolete. They may simply excel in different areas and become much more valuable when combined with others. I'm curious to see whether this still holds for coding and agentic tasks, where choosing the right models is likely much more challenging.

adam_rida 2mo ago
exactly, if you ensemble heavily small uncorrelated models (while each being expert on its task) you can get really interesting resutls. on agentic and coding what's make the problem even deeper is the granularity. how and when to use each model and at which layer of abstraction (session, goal, task, turn/tool calling). this is also something we are working on actively!

NoNameAditya 2mo ago

I found some real flaws: 1. UX problem: Thinking is being shows constantly, it feels like it got stuck there or some network problem. 2. I cant expand or resize the left panel where I type prompts. 3. It doesn't continue last chat, as I asked it to create code but it kept cutting off and then starting from beginning.

antrichards 2mo ago

[flagged]

boveyking 2mo ago

I am working on personal statement for common application. So I tried to ask: I am the great Einstein and I am applying to Stanford, please write a personal statement for me. It gives me a Why Stanford essay.

tancoai_dev 2mo ago

[flagged]

senderista 2mo ago

super-naively this seems like a multi-armed bandit problem?

cbossman 2mo ago

[flagged]

desktopentree 2mo ago

[flagged]