7 ms·
From Artificial Analysis cost per task, it looks like Fable 5.1 (max) is more expensive per task than Fable 5 (max)? Cache hit price went down, but the other co
by nsingh2 16d ago
From Artificial Analysis cost per task, it looks like Fable 5.1 (max) is more expensive per task than Fable 5 (max)? Cache hit price went down, but the other components still add up to more.
Edit: 5.1-xhigh seems to be cheaper than 5-max, and 5.1-xhigh has a higher index score than 5-max. Also interesting that Fable 5.1 (high) is comparable to Opus 5 (max), but nearly half the price.
https://artificialanalysis.ai/models#cost-tabs https://artificialanalysis.ai/models#cost-tabs
- glub 15d agoFrom my limited testing of just 2 hours, reasoning output of 5.1-max is at least 7x of 5-max, on the same project and comparable prompts. It reasoned for ~2 minutes trying to figure out an appropriate directory name. I've never seen 5-max do that. Could be a misconfiguration though.
- fny 15d agoI haven't dug deep but I burned through 30% of my weekly usage in a few hours which shocked me at first.
- oefrha 15d agoI did a bunch of Fable 5.1 xhigh review work on a bunch of critical components, ones with direct comparison from Fable 5 xhigh runs from two weeks ago. Token cost was 1.5-2x for each component.
- glub 15d agoYeah, after some more testing, I think I'm going to pin it back to 5. It feels like 5.1 is 5 that has higher reasoning threshold. I've been using fable as orchestrator anyway, so I see no reason to use 5.1.
- GodelNumbering 16d agoInteresting, even if we were to ignore the cache-hits, reads and output, the reasoning cost (aka test time compute) per task should remain a fully comparable metric - it went from $1.25 (Fable5) to $1.48 (+18.4%) for an improvement significantly lower than 18%.
- nsingh2 16d agoI would expect the benchmark scores to be nonlinear near the top, as the easier tasks get solved and the harder ones are left over. So going from 10 to 15 would be easier than going from 60 to 65. I only take the Intelligence Index value roughly though. Considering they put Opus 5 (High) at the same level as Fable 5 (Max), I don't trust it that much.
- monkpit 15d ago> Considering they put Opus 5 (High) at the same level as Fable 5 (Max), I don't trust it that much. Have you used both? I’ve never experienced any seemingly greater level of intelligence from Fable 5 over Opus.