I did exactly this for a school project about a year ago: https://github.com/bcuccioli/neural-ocr https://github.com/bcuccioli/neural-ocr There's a paper in t

I did exactly this for a school project about a year ago: https://github.com/bcuccioli/neural-ocr https://github.com/bcuccioli/neural-ocr There's a paper in there that explains the design of the system and my results, which weren't great, probably due to the small size of training data.