16 ms·
60% Fable cost cut by converting code to images and having the model OCR it
- dimitropoulos 3mo agothere's also a DeepSeek whitepaper on this technique https://www.seangoedecke.com/text-tokens-as-image-tokens https://www.seangoedecke.com/text-tokens-as-image-tokens
- chickensong 3mo agoBinary compression unpacked by OCR? This is the stuff of nightmares. So cursed, and yet...
- genxy 3mo agoThis seems like a pricing hack that burns resources, that when the loophole gets closed the price of OCR will have to rise?
- ricardobeat 3mo agoIt’s not a loophole, it just happens that encoding information as optical tokens is much more efficient than text.
- guardiangod 3mo agoTruly a picture is worth a thousand words.
- TZubiri 3mo agoOf course it isn't A text encoding uses 8bits per character on average, tokenization further compresses that An image font would be 25 bits if 5x5, and most fonts are 12 pixels high Of course it isn't efficient, this is a pricing inefficiency and a hack to exploit it (even the author describes it as an exploit)
- legel 3mo agoYou are wrong. Text tokens are high-dimensional vectors, not 8 bits per character. Every token has a deep embedding, e.g. 1024 float values per text token. DeepSeek-OCR proved 10x+ compression from visual embedding of text, which was a groundbreaking result. [1] Very cool to see OP's project hacking on this principle. It's still not lossless, as noted in the github, but is a promising research direction. [1] https://github.com/deepseek-ai/DeepSeek-OCR/blob/main/DeepSeek_OCR_paper.pdf https://github.com/deepseek-ai/DeepSeek-OCR/blob/main/DeepSe...
- deburo 3mo agoA token is probably not a single char, and an image is probably decomposed into tokens as well (and god knows how many tokens an image is decomposed into) which probably map to similar float-hungry vectors. Your counterargument could use a bit more flesh. And we're talking about images of texts, not images that represent complex imagery such as a very detailed scene or what have you.
- Groxx 3mo agoI kinda wonder if it's extracting usable context from 2D proximity between lines? Normal text input wouldn't have that kind of information (though it could, and it's arguably just a lookahead/behind of N characters on average).
- TZubiri 3mo ago>Text tokens are high-dimensional vectors, You are conflating tokens with embeddings. Tokens fit in a single word, modern gpt uses a vocabulary with 200k possible values, which would fit into 18 bits. Have a good one
- gamblor956 3mo agoPeople really need to read their cites and not just the summaries. The paper notes two things: 1) While the compression ratio for visual text is better than it is for regular text, but the absolute space required is still higher for the images. OPs were talking about the space required, not the ratio. 2) The results of the OCR must still be fed into a text-based LLM for linguistic processing. Otherwise, all you have achieved is turning an image into a bunch of text.
- geor9e 3mo agoStep back and think about it another way - ask which scenario is more likely: Some random person discovered a 60% across the board gain in all LLMs, using an extremely simple trick that none of the labs noticed in all these years. That trick being to rasterize 8bit characters into 8x8 pixels in a big image. 60% in a market worth trillions of dollars. or Anthropic's marketing team arbitrarily prices tokens to drive growth, according to vibes and feelings, and didn't think they needed to price images on par with text in their rush to burn cash & drive growth. Some folks take advantage of the trick during the first few days of the model's availability before Anthopic corrects their pricing, to align more proportionally with actual compute costs.
- vineyardmike 3mo ago> Some random person discovered a 60% across the board gain in all LLMs, using an extremely simple trick that none of the labs noticed in all these years of multi-trillion dollar growth DeepSeek published a pretty well circulated paper on exactly this many months ago. It just hasn’t been attempted and shared publicly, asa retrofit, AFAIK. Also, it’s no free lunch, the readme indicates that this “use images” hack is lossy and reduces success rates alongside the reduced cost. Most labs would focus on success increases regardless of price.
- geor9e 3mo agoIf the trick were genuinely useful, and was well circulated months ago, the resource-starved inference providers would have squeezed this trick dry already, instead of wasting 60% of their tokens, waiting for users to implement it themselves in 5 minutes of effort.
- solenoid0937 3mo ago[flagged]
- Klathmon 3mo agoThat's like saying quantization isn't real because the frontier labs aren't using it in their production inference. This is a lossy process, it produces worse results. It might be worth it for some situations, but applying it to everything would just be making your SOTA model worse
- deleted 3mo ago[deleted]
- DaiPlusPlus 3mo ago> encoding information as optical tokens Educate me: what is an "optical token" when dealing with LLMs?
- jrm4 3mo agoAnyone else laugh out loud when they read this? Like, okay so NO, that's entirely impossible. What's really going on?
- Salgat 3mo agoThat's not what is happening. Claude isn't charging for the tokens it generates from the OCR on its side, but it's still processing the same number of tokens as if you had sent the text, just with the extra step of OCR on Claude's side. This is 100% a loophole that's burning extra resources.
- ricardobeat 3mo agoThere is no OCR, in the traditional sense, involved.
- samrus 3mo agoNot really. They arent actually using more resources this way either. This might be a fundamental inefficiency thats being removed It kinda makes sense too. Because while people do read code word by word, we often "glance over" it and do roughly pattern recognition on it to know what it does. Only homing in on something when we need to answer a specific question. I think humans kinda naturally do this exploit anyway
- aabhay 3mo agoAhhh my eyes the vibe coded readme
- mpalmer 3mo agoWhat, you don't like your caveats to be honest?
- sebmellen 3mo agoIt’s so painful to read the LLM-compressed explanations. I can’t exactly identify what it is, but it’s an immediate tell and literally requires twice the effort to comprehend. For example: > Honest caveat, visible in the clip: the pxpipe arm answered the count first and needed one follow-up nudge to also print the ledger balance in the requested one-line format; the plain arm followed the format on the first try. Legibility is solved on Fable — single-reply format compliance is the remaining rough edge. If I reread this four times, I can sort of interpolate what happened, but it’s mostly pointless and confusing information. In my experience all models do this to an extent, but Claude seems to be the worst at this. GPT 5.5 is a bit more terse but seems to compress more valuable information.
- hashmap 3mo agothe foamy hedging makes me ill and hurts my eyes
- deleted 3mo ago[deleted]
- quantummagic 3mo agoHere's one rewrite that would have helped: To be fair, as you can see in the clip, the two models handled the prompt slightly differently. The pxpipe variant gave the right count initially but needed a quick follow-up to output the ledger balance in a single line. The standard model, on the other hand, nailed the formatting on its first try. We've completely solved readability here on Fable; our only real hurdle left is getting the models to follow formatting constraints perfectly on the very first reply. Of course, this was just rewritten by another LLM.
- lpellis 3mo agoI tried the same thing last year (with openai models), back then it worked to reduce prompt tokens, but you needed way more completion tokens, ultimately more expensive (and slower) https://pagewatch.ai/blog/post/llm-text-as-image-tokens/ https://pagewatch.ai/blog/post/llm-text-as-image-tokens/
- nickpeterson 3mo agoa pictures worth a thousand tokens
- aabhay 3mo agoIn Gemini at least, if you look at how they process PDFs, they do an OCR and then feed the text + image to the model, without charging you for the text tokens (I believe). So my guess is that Claude’s backend is doing the same — so this hack is probably more of a loophole in token accounting that might get closed if Claude is doing what Gemini does
- hn_throwaway_99 3mo agoThis is really fascinating to me. I was reading this article and originally agreed with you, "I mean, under the covers it's got to be converting to text tokens at some point, so there is no way it's actually cheaper for Claude itself to execute." But then there is a comment below talking about how DeepSeek was able to get a huge improvement in compression by using visual tokens, https://news.ycombinator.com/item?id=48777848 https://news.ycombinator.com/item?id=48777848. I don't fully understand all of the underlying technical details so I am still fundamentally baffled about how going the OCR route could actually result in overall electricity/computational savings.
- DANmode 3mo agoIt wouldn’t, they’re subsidizing it for training. Edit: didn’t realize this occurred on local models(!!), this is smarter https://news.ycombinator.com/item?id=48779884 https://news.ycombinator.com/item?id=48779884
- NooneAtAll3 3mo agocan't explain with subsidies a model you host yourself (like deepseek)
- measurablefunc 3mo agoThen you are paying for the electricity. It's not physically possible to do more computation & not use more energy b/c every arithmetic operation requires a minimum amount of energy so more operations = more energy.
- dippogriff 3mo agoI want to see more text-free foundation models
- odo1242 3mo agoIf the model has to be able to output text it needs to be able to read text tho
- puppycodes 3mo agoThat is hilarious and an amazing find.
- __hugues 3mo agoseems really dumb and like it would need to violate basic information theory to work? input tokens are cheaper than output tokens. seems like it would maybe reduce input tokens at the expense of many more output tokens if you're actually triggering OCR via thinking?
- sachamorard 3mo agoIt's far from being a foolish idea, and it seems to me that the project correctly documents its own limitations. It's rare to see a README list so precisely where the tool falls short. Then, yes, input tokens are cheaper than output. But when it comes to find ways to get reduce costs, you have to explore all the options.
- himata4113 3mo agoRelated: https://blog.can.ac/2026/06/10/snapcompact/ https://blog.can.ac/2026/06/10/snapcompact/
- deleted 3mo ago[deleted]
- AIorNot 3mo agoI cant get past that LLM intense slop text in the Github repo
- yogthos 3mo agoIsn't this basically what DeepSeek came up with https://github.com/deepseek-ai/DeepSeek-OCR https://github.com/deepseek-ai/DeepSeek-OCR
- anigbrowl 3mo agoI'm sorry, but this is retarded. It works, and it's clever, but but it's clearly a workaround for a pricing failure. Much like the bounty on poisonous snakes leading to people taking up snake-breeding, this just exploits and promotes waste. I think ultimately blame falls on Anthropic for the poor pricing system the enables such arbitrage. But I'm also disgusted by the inevitable tide of people exploiting this until its fixed, and creating an entirely unnecessary extra tide of digital junk.
- electrotype 3mo agoWhat about: "Read this document online : [URL]" and you add your text/context to an online document? Would that reduce the number of tokens used too?
- mrbnprck 3mo agoDocuments are processed as tokens as well, unless its bitmap is ocr'd. Images tho are natively compatible with Multi-Modal LLMs, so theres no image->text translation layer in between. It's that the unit of cost is different (e.g. "visual token" vs text token)
- electrotype 3mo agoI see. I was thinking that it might be different if the document wasn't provided by you directly, but instead if the LLM fetched it itself online.
- cs702 3mo agoReminds me of caveman: https://news.ycombinator.com/item?id=47647455 https://news.ycombinator.com/item?id=47647455
- tru3_power 3mo agoThis probably works with PDF parsing as well I’m sure, even if it’s just from not having to parse pdf format alone.
- OSaMaBiNLoGiN 3mo agoSaw a Tweet a while ago from someone (maybe Carmack, maybe Geohot, maybe Karpathy?) wondering if images were just the better option. Since then I've been using images with very simply worded prompts whenever I'm informing an agent of what is happening. Sometimes no text in the prompt at all. It has been very very effective. That being said, this isn't really what Karpathy was talking about. But it got me thinking a bit, and that got me to a much nicer workflow.
- cindyllm 3mo ago[dead]
- felipelalli 3mo agoNo words.
- colwont 3mo ago[dead]
- tomyow 3mo ago[flagged]
- tomdow 3mo ago[flagged]
- tomfow 3mo ago[flagged]
- wigster 3mo agoa picture paints a thousand words
- shinryuu 3mo agoInteresting approach, though that readme really needs a rewrite by a human...
- brumar 3mo agoTangentially related: I don't think OCR is the right term and I am generally vocal about that. But seeing this unquestioned here, I am wondering if I am the one who is wrong here. Is it ok to call this OCR? To me ocr means text in the end, not visual tokens.
- parsimo2010 3mo agoOCR means optical character recognition. The terms do not require a direct transcription, but that is mostly what OCR meant in the past. If you’re using an LLM’s vision capability to pass in text and the LLM actually understands it, then I would say that it recognized the characters, hence OCR seems okay to use.
- brumar 3mo agoBut characters only exist when we ask the model about this and it does it best to do this projection if asked. Vision model are richer than that. It "understands" visually a document. If it was only about characters, then there will be no way it beats the traditional pipelines of image->text->extractions or obtain the kind of results we see in this article. Vision models are more than characters recognition and OCR term don't do it justice IMHO.
- devmor 3mo agoSo if I OCR a document, edit it, and print it, OCR didn't happen?
- TurdF3rguson 3mo agoIt's not. OCR is not what the vision model is doing here. We're used to using OCR as a verb but it's more accurate to say the model "visioned" it. Also, some models still do OCR and it's usually way more expensive that way.
- g42gregory 3mo agoI think Oh-My-Pi (OMP.sh) uses images for context compactificaton. OMP is built on top of Pi coding agent.
- lstroud 3mo agoAre we really re-discovering that compressed binary formats are more efficient data representations?
- solid_fuel 3mo agoNo? The image is still getting run through OCR and turned back into text before being fed into the LLM. There is no efficiency gain here, rather we have learned that Anthropic is applying a discount to text fed in via OCR.
- qingcharles 3mo agoI don't think this is what is happening, IMO. The models can genuinely "read" the text off the images, but usually at a less-than-perfect ratio, and it uses less tokens for the model on visual input than it does actually using OCR to convert them into text and then sending that in. I do not think there is any intermediate stage where they are applying a free OCR in this situation. (I realize that happens in some situations)
- npn 3mo agoit is funny because nobody ever bother points out that they overcharge you for text input token price. sure it was pretty resource intensity a few years before, but with turbo quant, sparse attention and various techniques, plus the advancing of hardware (dedicated prefill machine, memory pool for kv caching) the cost should be drastically reduced, and yet they still keep the same cost formula. I can't help but laugh whenever someone proudly share how many billion input tokens they spent in their code sections and how much they saved with the subscription, meanwhile it is pretty much just electricity cost for the providers.
- troglodytetrain 3mo agoSo, just be careful with this, it very likely is switching to other less capable model hence the cost reduction. So looks like Fable but isn’t. So you are doing extra work when you could just switch the model back to opus 4.8 instead.
- TokenLens 3mo ago[flagged]
- Fno44 2mo ago[flagged]