OCR, and what Tesseract does

August 2, 202611 min read

Say we have tons of files (paper copies, screenshots, receipts, car license plates, pictures of documents) all taken, and we want to process and store them, say use them for some analysis. It's going to be impossible to go through them one by one and get that data into a computer.

It might be structured (eg. paper reports), semi structured, or not structured at all. It might be clean printouts or human writing. The problem just gets exponential. And it's been a problem we've had for decades solved with OCR.

Optical Character Recognition is a process that changes pictures of text into real words that a computer can read, search, and change. In simple terms, we're "recognizing text", and it's four problems stacked together.

Typical OCR process

1. Cleanup

A scanner or a phone camera hands you a photograph of a page: a grid of pixels in millions of shades of grey, almost none of which say anything about what the letters are. Cleanup strips that back to a page that is straight, large enough to read, and reduced to two colours.

Binarization decides for every pixel whether it's ink or paper. Picking one brightness threshold for the whole page works on a clean flatbed scan and falls apart the moment lighting is uneven: photograph a page with a shadow across one corner and a global threshold turns that corner solid black. Adaptive thresholding picks a threshold per neighbourhood instead, so the shadowed corner gets judged against its own surroundings.

Deskew rotates the page so text lines run horizontally. Everything downstream assumes horizontal, so a three degree tilt on a wide page puts the right end of a line dozens of pixels below the left end, and a row of pixels stops corresponding to a row of text. The usual way to measure the angle is a projection profile: rotate through candidate angles, count ink pixels in every row, and take the angle where the variance of those counts peaks. Straight lines of text make the counts spiky, tilted text smears them flat.

Resolution. DPI is the number of pixels captured per inch of paper. At 300 DPI a lowercase letter is 20 to 30 pixels tall, which is enough detail to tell an e from a c. At 150 it's half that and those two collapse into the same smudge. Upscaling afterwards recovers nothing, because you're inventing pixels rather than reading them.

2. Layout analysis

Before anything can read a word, something has to work out where the words are. Layout analysis recovers the page's structure: which marks form a line, which lines form a paragraph, which paragraphs sit in the same column, and which ink isn't text at all. You'll also see it called page segmentation.

It starts with connected components. A connected component is a set of dark pixels that touch one another: pick any ink pixel, flood fill outward, and what you've collected is one component. Usually that's a single character, sometimes a fragment (a lowercase i is two, the stem and the dot), sometimes several characters at once if the print bled.

From there the engine builds upward. Components whose vertical positions line up become a line. Gaps wider than normal letter spacing split a line into words. Lines at consistent spacing become a block. Blocks separated by a wide gutter become columns. It also has to rule out ink that isn't text: logos, photographs, the rule under a heading, the gridlines of a table.

All of this is geometry. Positions, sizes, gaps. Nothing here knows what any of the words say, which is why layout analysis can be confidently wrong about a table while the recognition after it is perfect.

3. Recognition

This is the stage people mean when they say OCR. A picture of a line of text goes in, the characters come out.

The hard part isn't telling an A from a B. It's that you don't know where one character ends and the next begins. Letters touch in ordinary print, and rn is pixel-for-pixel almost identical to m. Splitting a word into characters and identifying those characters turn out to be the same problem, so you can't do either one first.

The old answer was to guess the splits, classify each piece, and backtrack when the result came out as nonsense. The modern answer is to stop splitting: hand the model a whole line, let it read left to right, and let the boundaries fall out of recognition rather than precede it.

Either way a dictionary gets the last word. When the pixels are genuinely ambiguous between rnedication and medication, the language model breaks the tie toward the real word. Helpful on prose, harmful on serial numbers and product codes, where the right answer was never a word.

4. Reading order

By now the engine knows what every block on the page says. It still has to decide which order to emit them in, and the pixels carry no instruction about that.

For a single column of prose it's top to bottom and nobody thinks about it. Everything else is a judgment call. Newspaper columns read down, not across. A form has labels and values sitting side by side. Headers and footers belong nowhere in the flow. Tables are the genuinely ambiguous case, because down the columns and across the rows are both defensible depending on what you want out of it.

Tesseract's own grouping shows how this goes wrong. It hands back block_num, par_num and line_num for every word, so the obvious move is to group by those and print them in order. But it segments a table into one block per column, so block order walks the page column-first: every label, then every value, then every entry in the third column. A row's label and its value end up many lines apart, and if you're extracting fields that pairing was the entire point.

The fix is to ignore the engine's grouping and rebuild lines from word geometry: group words into horizontal bands by the vertical centre of their bounding box, sort left to right within a band, and size the band relative to the median glyph height so it adapts to the scan's resolution. Tables then read as rows. A genuine two-column layout will interleave under that rule, which is the tradeoff you accept.

When this stage fails the output still looks like fluent text. It's fluent text in the wrong order, which is much harder to notice than obvious garbage.

Which parts are machine learning

Only one of the four. Cleanup is classical image processing and layout analysis in Tesseract is classical geometry, neither with a model in it. Reading order is heuristic. Recognition is a trained neural network, and has been since 2018.

The managed cloud services have since replaced layout analysis and reading order with learned models too, which is much of why they handle forms better. But the short version holds: the recognition stage is a trained model and the processing around it is classical. That's more useful than either "OCR is AI" or "OCR is just pattern matching".

What an OCR engine gives back

A good OCR engine doesn't hand you a wall of text. It hands you a table, one row per word, with a bounding box (left, top, width, height) and a confidence score from 0 to 100. The plain-text version is something you assemble from that table, and so is anything that highlights a source page later. Keep the boxes. Once you flatten to a string you can't get them back.

The confidence score is worth understanding before you lean on it. It's a per word number the recognizer derives from its own output, not a calibrated probability, and it will happily report a high number on garbage that came out word-shaped. Rows for the structural levels (page, block, paragraph, line) carry no text and a confidence of -1, so filter those out before averaging anything.

What it's good for is routing. Below a threshold a field goes to a human, above it goes straight through, and that threshold is the dial that sets your straight-through processing rate. Moving it is a cost decision more than a technical one. What it isn't good for is reporting accuracy: a mean confidence across a page reads like an accuracy score and isn't one. Only a labeled test set tells you how often you were right.

Build your own, or use an existing solution?

Managed options like Amazon Textract and Google Document AI cost money per page and need a network, but they handle forms and tables far better than anything you'll assemble in a weekend.

If you're building, Tesseract is an open source engine that covers stages 2 and 3. It's free, it runs offline, and it's been around since the 80s. Stages 1 and 4 are still yours.

Running both is reasonable: Tesseract locally so development never depends on a key or a connection, and a managed service in the deployed path where accuracy earns its cost. Put them behind one interface and the rest of your pipeline never learns which one it got.

What the managed services add

Textract's API splits along exactly the line this article keeps drawing. DetectDocumentText is OCR and nothing else, the same job Tesseract does. AnalyzeDocument is the layer above it: pairing form keys with their values, rebuilding tables as cells and rows, answering plain-English questions about the page, and labelling titles, headers and signature blocks.

The forms and tables part isn't natural language processing, which surprises people. It's models trained over geometry, text and pixels together, where a word's position on the page is an input exactly like the word itself. The research line is LayoutLM, DocFormer and Donut, and the shared idea is that a document is a 2D object, so flattening it to a string throws away half the signal. It's also why piping OCR text straight into a language model underperforms on forms. By then, the fact that a value sat immediately to the right of its label is gone.

Language processing proper is a separate service. On AWS that's Comprehend, which does entity extraction and code linking on text that has already been read.

How Tesseract Works

Modern Tesseract (v4 and up) is a line recognizer, not a character recognizer. For each text line it:

  1. Normalizes the line to a fixed pixel height.
  2. Slices it into a left-to-right sequence of thin vertical strips.
  3. Feeds that sequence into a bidirectional LSTM, which emits a probability distribution over characters for every strip.
  4. Decodes with CTC, the same trick speech recognition uses. It exists to solve "I don't know how many strips make up this letter."
  5. Nudges the result toward real words with a dictionary.

How it worked before 2018

Tesseract started at HP Labs in 1984, was open sourced in 2005, and Google picked up development the year after. Everything up to version 3 ran an engine with no neural network in it anywhere, and it's still sitting in the binary.

That engine went word by word rather than line by line: trace the outline of each blob of ink, split blobs that look like two characters stuck together, match each candidate character's outline features against stored prototypes, then pick the best word string with help from a dictionary. The interesting part is that it ran twice. Words recognized confidently on the first pass were fed to an adaptive classifier as training examples, so the engine tuned itself to the particular fonts on the page in front of it, then re-read the words it had struggled with. A small amount of learning, per document, at read time.

Neural networks were reading text well before 2018, just not in general purpose open source OCR. LeCun's convolutional networks were reading handwritten digits on American cheques through the mid 90s. What version 4 changed was the scope: one engine, arbitrary printed text, running on a laptop, for free.

Things that will still bite you

Handwriting is a different problem. Tesseract is trained on printed text and needs a different class of model to read cursive.

300 DPI is the comfortable target. Upscaling a 150 DPI scan recovers less than you'd hope, because the information was never captured in the first place.

Anything that isn't a word fights the language model: codes, IDs, license plates. Restricting the character set helps.

Record which engine and version read each page. A corpus built across an upgrade is otherwise unexplainable when the numbers move.

Sources / further reading