8 ms·

A Python tool to parse PDF statements from Poste Italiane

58 points by genbs 1y ago

deleted 1y ago
[deleted]
Tox46 1y ago
Loving the subtitle in the md
- brightbeige 1y ago
  > agli sventurati che hanno un conto postale translates to: “to the unfortunate ones who have a postal account”
denysvitali 1y ago
I would love to have something more generic (and tried to build it already), but parsing tables and bank statements even from digital PDFs (as in, those that really have tables and not a picture) is still very difficult. Especially when the bank changes layouts from one month to another. I would love to be proven wrong, but everything I have tried so far is... subpar. Nowadays there's probably a solution based on LLMs, but I don't trust them with this kind of data
- vdm 1y ago
  https://github.com/qyhou/curated-table-structure-recognition https://github.com/qyhou/curated-table-structure-recognition
- jgalt212 1y ago
  > Nowadays there's probably a solution based on LLMs, but I don't trust them with this kind of data In practice, the flow from my perspective looks like LLM parser -> normalizer -> validator. So you only save one step (parser), and given the unique stochastic nature of the LLM output, the normalizer and validator can be trickier to write than one used for an old-fashioned rules-based parser. But each situation is different, or YMMV.
- dimitri-vs 1y ago
  Have you tried datalab-to/marker with the "Use LLM" option? They have a playground you can test it out on https://www.datalab.to/playground https://www.datalab.to/playground but I use their local CLI option: https://github.com/datalab-to/marker https://github.com/datalab-to/marker I just tried it on a fairly ugly TD Bank statement PDF I have and the markdown of the whole PDF (tables and all) is very accurate. Here is the config I use: marker_single --format_lines --use_llm --llm_service marker.services.gemini.GoogleGeminiService --gemini_model_name gemini-2.5-flash --disable_image_extraction --output_format markdown --output_dir "$OutDir" ` "$In" You might be able to tell the LLM to directly output the data in CSV format - granted it will still be in a .md file - using the `--block_correction_prompt` which apparently is "useful for custom formatting or logic that you want to apply to the output"
  - denysvitali 1y ago
    > Nowadays there's probably a solution based on LLMs, but I don't trust them with this kind of data If it works with a small model I can run locally, I might think of this approach, otherwise I'll skip
liendolucas 1y ago
Offtopic: Stay away from Poste Italiane at all costs. The worst bank I have ever dealt with in my entire life. I'm glad that I don't have to deal with them anymore. Terrible service and eternal waitings on their branches. They are extremeley incompetent.
- genbs 1y ago
  it was a mistake in my youth
- nkjoep 1y ago
  In fact, it's not even a bank.
  - koakuma-chan 1y ago
    A bit off topic, but how do you guys use banks to buy expensive things? I always run into issues, like I couldn't pay for my dentist because my card had a 300$ tap limit (I had to come back the next day and pay the rest lol), and there's all sort of limits, like I don't think I can spent more than a couple thousand dollars a day even if I physically bring my card and insert it or whatever.
    - vladvasiliu 1y ago
      All the banks I know have some limit over a sliding 30 day window. Some allow for this limit to be adjusted. But the minimum amount I've had was 1000 €. Contactless is limited to 50 € per transaction. Going above requires inserting the card and entering the pin.
    - HelloNurse 1y ago
      Currently, payments from your account to another account (you need the other party's IBAN) cost about 1.50€, probably less if you find a less greedy bank, and are executed immediately. Retro options include cheques or (better) cashier's cheques.
    - piltdownman 1y ago
pietroppeter 1y ago
For the curious on how it works (not mentioned in the readme), it uses pymupdf and a precise mapping of all information in area coordinates, as such the document layout is hard coded. When layout changes this breaks but layout changes on this sort of documents do not happen often (I think). Also code is very clean and it serms straightforward to fix. This kind of code is maybe something that can be generated from an LLM/agent? (It would be easy to write checks) Besides the practical value for those who might need it, I think it is possibly interesting for others to look at this approach. Neat project, thanks for sharing!
- genbs 1y ago
  Thanks for reporting and for the feedback
simonebrunozzi 1y ago
Subtitled: "agli sventurati che hanno un conto postale" ("(dedicated to) the unlucky ones that have a bank account with the Italian postal service") The usual, amazing irony of us Italians. Love it.
- genbs 1y ago
  :P
amadeuspagel 1y ago
My brother made a similar tool for trade republic: https://kontoauszug.jonathanpagel.com/ https://kontoauszug.jonathanpagel.com/
rcastellotti 1y ago
POPOPO POPOPO POPO POPOOOO come on fellow italians, sing with me!
- genbs 1y ago
  :D

58 points by genbs 1y ago

deleted 1y ago

[deleted]

Tox46 1y ago

Loving the subtitle in the md

brightbeige 1y ago
> agli sventurati che hanno un conto postale translates to: “to the unfortunate ones who have a postal account”

denysvitali 1y ago

I would love to have something more generic (and tried to build it already), but parsing tables and bank statements even from digital PDFs (as in, those that really have tables and not a picture) is still very difficult. Especially when the bank changes layouts from one month to another. I would love to be proven wrong, but everything I have tried so far is... subpar. Nowadays there's probably a solution based on LLMs, but I don't trust them with this kind of data

vdm 1y ago
https://github.com/qyhou/curated-table-structure-recognition https://github.com/qyhou/curated-table-structure-recognition
jgalt212 1y ago
> Nowadays there's probably a solution based on LLMs, but I don't trust them with this kind of data In practice, the flow from my perspective looks like LLM parser -> normalizer -> validator. So you only save one step (parser), and given the unique stochastic nature of the LLM output, the normalizer and validator can be trickier to write than one used for an old-fashioned rules-based parser. But each situation is different, or YMMV.
dimitri-vs 1y ago
Have you tried datalab-to/marker with the "Use LLM" option? They have a playground you can test it out on https://www.datalab.to/playground https://www.datalab.to/playground but I use their local CLI option: https://github.com/datalab-to/marker https://github.com/datalab-to/marker I just tried it on a fairly ugly TD Bank statement PDF I have and the markdown of the whole PDF (tables and all) is very accurate. Here is the config I use: marker_single --format_lines --use_llm --llm_service marker.services.gemini.GoogleGeminiService --gemini_model_name gemini-2.5-flash --disable_image_extraction --output_format markdown --output_dir "$OutDir" ` "$In" You might be able to tell the LLM to directly output the data in CSV format - granted it will still be in a .md file - using the `--block_correction_prompt` which apparently is "useful for custom formatting or logic that you want to apply to the output"
- denysvitali 1y ago
  > Nowadays there's probably a solution based on LLMs, but I don't trust them with this kind of data If it works with a small model I can run locally, I might think of this approach, otherwise I'll skip

liendolucas 1y ago

Offtopic: Stay away from Poste Italiane at all costs. The worst bank I have ever dealt with in my entire life. I'm glad that I don't have to deal with them anymore. Terrible service and eternal waitings on their branches. They are extremeley incompetent.

genbs 1y ago
it was a mistake in my youth
nkjoep 1y ago
In fact, it's not even a bank.
- koakuma-chan 1y ago
  A bit off topic, but how do you guys use banks to buy expensive things? I always run into issues, like I couldn't pay for my dentist because my card had a 300$ tap limit (I had to come back the next day and pay the rest lol), and there's all sort of limits, like I don't think I can spent more than a couple thousand dollars a day even if I physically bring my card and insert it or whatever.
  - vladvasiliu 1y ago
    All the banks I know have some limit over a sliding 30 day window. Some allow for this limit to be adjusted. But the minimum amount I've had was 1000 €. Contactless is limited to 50 € per transaction. Going above requires inserting the card and entering the pin.
  - HelloNurse 1y ago
    Currently, payments from your account to another account (you need the other party's IBAN) cost about 1.50€, probably less if you find a less greedy bank, and are executed immediately. Retro options include cheques or (better) cashier's cheques.
  - piltdownman 1y ago

pietroppeter 1y ago

For the curious on how it works (not mentioned in the readme), it uses pymupdf and a precise mapping of all information in area coordinates, as such the document layout is hard coded. When layout changes this breaks but layout changes on this sort of documents do not happen often (I think). Also code is very clean and it serms straightforward to fix. This kind of code is maybe something that can be generated from an LLM/agent? (It would be easy to write checks) Besides the practical value for those who might need it, I think it is possibly interesting for others to look at this approach. Neat project, thanks for sharing!

genbs 1y ago
Thanks for reporting and for the feedback

simonebrunozzi 1y ago

Subtitled: "agli sventurati che hanno un conto postale" ("(dedicated to) the unlucky ones that have a bank account with the Italian postal service") The usual, amazing irony of us Italians. Love it.

genbs 1y ago
:P

amadeuspagel 1y ago

My brother made a similar tool for trade republic: https://kontoauszug.jonathanpagel.com/ https://kontoauszug.jonathanpagel.com/

rcastellotti 1y ago

POPOPO POPOPO POPO POPOOOO come on fellow italians, sing with me!

genbs 1y ago
:D