5 ms·

You might be able to drop the PDF backend since they're close to getting HTML running well: https://news.ycombinator.com/item?id=38713215 https://news.ycombinat

You might be able to drop the PDF backend since they're close to getting HTML running well: https://news.ycombinator.com/item?id=38713215 https://news.ycombinator.com/item?id=38713215 Using that might be easier than a multi-modal approach. Bonus points for: * Multiple papers at once * Comparing PDF and HTML output with the LLM as input for it correcting similar converter code

evanhu_ 3y ago
Definitely I'll move to the LaTeX source code instead of a PDF backend since that allows better support for non textual data that gets poorly scraped by GROBID. That is a really cool development I didn't know about, also theres https://ar5iv.labs.arxiv.org/ https://ar5iv.labs.arxiv.org/ which already has most arXiv papers as HTML documents. I chose GROBID because they not only parse the PDF but organize the text into logical sections for me (Intro, abstract, references) which I didn't want to manually do with heuristics that I'd have to devise.
- Mandelmus 3y ago
  Yes, using the LaTeX source code (or HTML, once that becomes reliable and widely used) should be much more robust than PDF parsing.
- skeptrune 3y ago
  That is a very good resource to save
- Koaisu 3y ago
  Maybe you could use: https://github.com/facebookresearch/nougat/tree/main https://github.com/facebookresearch/nougat/tree/main or https://github.com/VikParuchuri/marker https://github.com/VikParuchuri/marker Both are tools to convert pdfs into Latex or Markup with latex formulas. Maybe that helps
  - acqq 3y ago
    Reading the motivation for the second: "Relying on autoregressive forward passes to generate text is slow and prone to hallucination/repetition. From the nougat paper: We observed [repetition] in 1.5% of pages in the test set, but the frequency increases for out-of-domain documents. In my anecdotal testing, repetitions happen on 5%+ of out-of-domain (non-arXiv) pages." When these are fed in the next levels as inputs, isn't it even less surprising to get even more hallucinations/repetitions?

evanhu_ 3y ago

Definitely I'll move to the LaTeX source code instead of a PDF backend since that allows better support for non textual data that gets poorly scraped by GROBID. That is a really cool development I didn't know about, also theres https://ar5iv.labs.arxiv.org/ https://ar5iv.labs.arxiv.org/ which already has most arXiv papers as HTML documents. I chose GROBID because they not only parse the PDF but organize the text into logical sections for me (Intro, abstract, references) which I didn't want to manually do with heuristics that I'd have to devise.

Mandelmus 3y ago
Yes, using the LaTeX source code (or HTML, once that becomes reliable and widely used) should be much more robust than PDF parsing.
skeptrune 3y ago
That is a very good resource to save
Koaisu 3y ago
Maybe you could use: https://github.com/facebookresearch/nougat/tree/main https://github.com/facebookresearch/nougat/tree/main or https://github.com/VikParuchuri/marker https://github.com/VikParuchuri/marker Both are tools to convert pdfs into Latex or Markup with latex formulas. Maybe that helps
- acqq 3y ago
  Reading the motivation for the second: "Relying on autoregressive forward passes to generate text is slow and prone to hallucination/repetition. From the nougat paper: We observed [repetition] in 1.5% of pages in the test set, but the frequency increases for out-of-domain documents. In my anecdotal testing, repetitions happen on 5%+ of out-of-domain (non-arXiv) pages." When these are fed in the next levels as inputs, isn't it even less surprising to get even more hallucinations/repetitions?