7 ms·

I've got a scan from a book that I OCR with new releases. Ligatures, critical sigla, Fraktur letterforms, subscripts, superscripts, etc. Nothing special about

I've got a scan from a book that I OCR with new releases. Ligatures, critical sigla, Fraktur letterforms, subscripts, superscripts, etc. Nothing special about this model for overly-detailed work like mine. It's been a while since I last tested (and discontinued my subscription), but the "pro" models from OpenAI dominate. Not surprising, given the price difference, but it would be nice if an OCR-specific model could perform better. It's worth mentioning that even the highest-end models do a pretty poor job with intricate text like mine.

petcat 1mo ago
> the "pro" models from OpenAI dominate. Not surprising considering the price difference, but it would ne nice if an OCR-specific model could do better. I haven't been impressed with any of Mistral's models. They obviously realized that they couldn't compete at the frontier so they decided to go for smaller focused models but even those have not been that good.
- booi 1mo ago
  This is what I found as well. We moved away from Cursor but I was looking for a model that would help with FIM (fill-in-middle) multiline autocompletion and people were recommending Mistral's Codestral. We gave it a shot and it was lackluster at best.. Even Google's Gemini did a significantly better job than Codestral. Ultimately Opus-class models got good enough and I don't do much manual coding anymore.
rtaylorgarlock 1mo ago
Yet: how is pricing? Evaluating contents and routing appropriately isn't a new challenge in OCR, one of the oldest fields of applications in ML. Thus, how do the smaller open models perform in tandem with relatively pricy $/pg models & APIs? Your use case is remarkably rare relative to the volume and price sensitivity of enterprise data warehouse ops.
- deleted 1mo ago
  [deleted]
- giancarlostoro 1mo ago
  The big difference is traditional OCR used basic pattern matching to find text, whereas models like Mistral OCR (and GPT, etc) use computer vision instead and deep learning to parse text, math equations, and apparently in some cases extract images too. I'd love to see some advancements in traditional OCR based on ideas and concepts we've learned from newer "OCR-like" models since traditional OCR is drastically cheaper.
kmitz 1mo ago
I got the opposite experience very recently : tried to OCR a bunch of handwritten emails addresses with chatGPT and I had to make so many corrections that I gave up. Whereas Mistral nailed it on first pass.
- usef- 1mo ago
  Which chatgpt model?
  - kmitz 1mo ago
    I tested different models of the 5.6 family with different settings, although I couldn't exactly remember precisely which ones.
kergonath 1mo ago
I have been quite happy with Mistral OCR for the documents I needed to process (typeset, but old, with questionable scan quality, sometimes elaborate typesetting or, much worse, typewriter-and-handwriting approximations of it). I do not test every new model when they are released, but I did a review shortly after Mistral OCR 3 was released and it was a very good compromise: cheap, fast, and good results without further processing. I found generalist models to be way too much faf to get them to avoid unnecessary modifications to the text and report accurate bounding boxes for figures and tables. That said, models have sometimes surprising weaknesses and a model could be terrible overall but magically work for one type of document.
giancarlostoro 1mo ago
So is Mistral OCR the best one? Have any other OCR models caught some of what you describe? I've been kind of interested in how "OCR" type models work compared to old school OCR.
- bugglebeetle 1mo ago
  The datalab models are the best ones.
- ComputerPerson 1mo ago
  My use case isn't in the realm of old-school OCR, so it's not a good comparison, but anyway: As another user pointed out, it's surprisingly random (task-specific). Llama Scout outperformed Gemini Flash 2.5 on a benchmark I built at the time. I didn't include an OCR models. Mistral might indeed be the best OCR-specific model for my task, now that you ask. Funny. It's so bad at my work that I didn't register it might be the best in its category. This is just based on vibes from my single scan.
josu 1mo ago
Whats the best open OCR at the moment?
- deleted 1mo ago
  [deleted]
- dcan 1mo ago
  I’ve had good experiences with MinerU and PaddleOCR-VL
SyneRyder 1mo ago
While I haven't tried OpenAI for OCR, I've put my small scale OCR work through both Claude and Mistral OCR. Claude is absolutely better - even in OCR work I did last week and compared with Mistral OCR 4.0. Mistral's one advantage is that Anthropic now flags OCR, because they don't allow anything that could be considered "reproduction", even of work for which you own the copyright. So my new workflow is Mistral OCR for the actual OCR, followed by a proofreading pass by Claude (which is allowed). Claude is obviously more expensive, but it caught entirely hallucinated sentences created by Mistral OCR 4.0, so I was glad for the backup check.
- Oras 1mo ago
  Same company that OCRed millions of books, the irony. I feel Anthropic is destroying itself with all these restriction. They got away because their models were the best for coding, but that is not an advantage anymore as OpenAI and other open source are already better.
  - usef- 1mo ago
    They were criticised/sued early on when people could reproduce copyright things. I dont think in this case it's something they'd prefer to do?
    - SyneRyder 1mo ago
      Yep, my understanding is that many guardrails like this are actually the result of government legislation (eg the Fable bans) or terms of settling copyright lawsuits over reproducing copyrighted text and lyrics. I mentioned it in a sibling reply, but here's Anthropic's support document about not using Claude to reproduce content verbatim that already exists, regardless of copyright. https://privacy.claude.com/en/articles/10023638-why-am-i-receiving-an-output-blocked-by-content-filtering-policy-error https://privacy.claude.com/en/articles/10023638-why-am-i-rec...
      - deleted 1mo ago
        [deleted]
razemio 1mo ago
I made a benchmark for handwriting recognition for a project while keeping line breaks and errors (grammar, spelling). Sonnet absolutely dominates it since a good half a year. 5.6 did not change that for me. This should also translate to better ocr.
- ComputerPerson 1mo ago
  You're the second person to mention handwriting. I think it might be handwriting-specific; perhaps Anthropic has a better corpus for this. I really wouldn't know, though. Anthropic models barf out copyright issues for my use case, so I'm unable even to benchmark them. It's a common problem when you're scanning public domain books. Mine are reference texts often cited.
  - razemio 1mo ago
    Ah okay. Ofc handwriting is not copyrighted. Makes sense, that we do not have these issues.
raverbashing 1mo ago
Do all the models give you the bounding boxes, block labels as this one (allegedly) do?
- ComputerPerson 1mo ago
  It's very common. PaddleOCR is enough to get extremely well-done bounding boxes, and it runs very fast on a $150.00 GPU. There's always room for improvement, though. I suspect a tool will emerge for highly detailed OCR that implements a nested bounding-box-based multi-scale approach, effectively OCRing small sections at a time and then gradually compiling them by expanding the surface area using the bounding boxes. I've thought a lot about implementing it anyway. edit: I see you're asking about the block labels. Leaving the comment in case someone finds it interesting.

petcat 1mo ago

> the "pro" models from OpenAI dominate. Not surprising considering the price difference, but it would ne nice if an OCR-specific model could do better. I haven't been impressed with any of Mistral's models. They obviously realized that they couldn't compete at the frontier so they decided to go for smaller focused models but even those have not been that good.

booi 1mo ago
This is what I found as well. We moved away from Cursor but I was looking for a model that would help with FIM (fill-in-middle) multiline autocompletion and people were recommending Mistral's Codestral. We gave it a shot and it was lackluster at best.. Even Google's Gemini did a significantly better job than Codestral. Ultimately Opus-class models got good enough and I don't do much manual coding anymore.

rtaylorgarlock 1mo ago

Yet: how is pricing? Evaluating contents and routing appropriately isn't a new challenge in OCR, one of the oldest fields of applications in ML. Thus, how do the smaller open models perform in tandem with relatively pricy $/pg models & APIs? Your use case is remarkably rare relative to the volume and price sensitivity of enterprise data warehouse ops.

deleted 1mo ago
[deleted]
giancarlostoro 1mo ago
The big difference is traditional OCR used basic pattern matching to find text, whereas models like Mistral OCR (and GPT, etc) use computer vision instead and deep learning to parse text, math equations, and apparently in some cases extract images too. I'd love to see some advancements in traditional OCR based on ideas and concepts we've learned from newer "OCR-like" models since traditional OCR is drastically cheaper.

kmitz 1mo ago

I got the opposite experience very recently : tried to OCR a bunch of handwritten emails addresses with chatGPT and I had to make so many corrections that I gave up. Whereas Mistral nailed it on first pass.

usef- 1mo ago
Which chatgpt model?
- kmitz 1mo ago
  I tested different models of the 5.6 family with different settings, although I couldn't exactly remember precisely which ones.

kergonath 1mo ago

I have been quite happy with Mistral OCR for the documents I needed to process (typeset, but old, with questionable scan quality, sometimes elaborate typesetting or, much worse, typewriter-and-handwriting approximations of it). I do not test every new model when they are released, but I did a review shortly after Mistral OCR 3 was released and it was a very good compromise: cheap, fast, and good results without further processing. I found generalist models to be way too much faf to get them to avoid unnecessary modifications to the text and report accurate bounding boxes for figures and tables. That said, models have sometimes surprising weaknesses and a model could be terrible overall but magically work for one type of document.

giancarlostoro 1mo ago

So is Mistral OCR the best one? Have any other OCR models caught some of what you describe? I've been kind of interested in how "OCR" type models work compared to old school OCR.

bugglebeetle 1mo ago
The datalab models are the best ones.
ComputerPerson 1mo ago
My use case isn't in the realm of old-school OCR, so it's not a good comparison, but anyway: As another user pointed out, it's surprisingly random (task-specific). Llama Scout outperformed Gemini Flash 2.5 on a benchmark I built at the time. I didn't include an OCR models. Mistral might indeed be the best OCR-specific model for my task, now that you ask. Funny. It's so bad at my work that I didn't register it might be the best in its category. This is just based on vibes from my single scan.

josu 1mo ago

Whats the best open OCR at the moment?

deleted 1mo ago
[deleted]
dcan 1mo ago
I’ve had good experiences with MinerU and PaddleOCR-VL

SyneRyder 1mo ago

While I haven't tried OpenAI for OCR, I've put my small scale OCR work through both Claude and Mistral OCR. Claude is absolutely better - even in OCR work I did last week and compared with Mistral OCR 4.0. Mistral's one advantage is that Anthropic now flags OCR, because they don't allow anything that could be considered "reproduction", even of work for which you own the copyright. So my new workflow is Mistral OCR for the actual OCR, followed by a proofreading pass by Claude (which is allowed). Claude is obviously more expensive, but it caught entirely hallucinated sentences created by Mistral OCR 4.0, so I was glad for the backup check.

Oras 1mo ago
Same company that OCRed millions of books, the irony. I feel Anthropic is destroying itself with all these restriction. They got away because their models were the best for coding, but that is not an advantage anymore as OpenAI and other open source are already better.
- usef- 1mo ago
  They were criticised/sued early on when people could reproduce copyright things. I dont think in this case it's something they'd prefer to do?
  - SyneRyder 1mo ago
    Yep, my understanding is that many guardrails like this are actually the result of government legislation (eg the Fable bans) or terms of settling copyright lawsuits over reproducing copyrighted text and lyrics. I mentioned it in a sibling reply, but here's Anthropic's support document about not using Claude to reproduce content verbatim that already exists, regardless of copyright. https://privacy.claude.com/en/articles/10023638-why-am-i-receiving-an-output-blocked-by-content-filtering-policy-error https://privacy.claude.com/en/articles/10023638-why-am-i-rec...
    - deleted 1mo ago
      [deleted]

razemio 1mo ago

I made a benchmark for handwriting recognition for a project while keeping line breaks and errors (grammar, spelling). Sonnet absolutely dominates it since a good half a year. 5.6 did not change that for me. This should also translate to better ocr.

ComputerPerson 1mo ago
You're the second person to mention handwriting. I think it might be handwriting-specific; perhaps Anthropic has a better corpus for this. I really wouldn't know, though. Anthropic models barf out copyright issues for my use case, so I'm unable even to benchmark them. It's a common problem when you're scanning public domain books. Mine are reference texts often cited.
- razemio 1mo ago
  Ah okay. Ofc handwriting is not copyrighted. Makes sense, that we do not have these issues.

raverbashing 1mo ago

Do all the models give you the bounding boxes, block labels as this one (allegedly) do?

ComputerPerson 1mo ago
It's very common. PaddleOCR is enough to get extremely well-done bounding boxes, and it runs very fast on a $150.00 GPU. There's always room for improvement, though. I suspect a tool will emerge for highly detailed OCR that implements a nested bounding-box-based multi-scale approach, effectively OCRing small sections at a time and then gradually compiling them by expanding the surface area using the bounding boxes. I've thought a lot about implementing it anyway. edit: I see you're asking about the block labels. Leaving the comment in case someone finds it interesting.