Lab notes
What we tried to make phone photos easier to read, and what we learned
For a few days we worked on one question: when you photograph a page with your phone, how do we turn it into the clearest scan we can? We tried a lot. Some of it is in ScanKit now, some of it made things worse, and one idea we put aside on purpose. Here is all of it, in plain words.
How we measured
"Looks sharper" is easy to fool yourself with, so we used a stricter test. We photographed a busy printed form with a phone, typed out the text that is really on it, and made five more versions of the photo: smaller and squashed, dim and noisy, tilted, with a shadow across it, and blurred. After every change we ran each result through an open-source text reader (OCR) and counted the letters it got wrong. Fewer misread letters meant a better change. Anything smaller than the reader's own wobble, about 0.4%, we counted as no change at all.
What worked (now in the app)
Doing the steps in the right order
The clean-up is a chain of steps: even out the paper, set the black and white points, sharpen, join dot-matrix print, then darken thin strokes. We tried them in many orders. Evening out the paper has to come first (without it, thousands more letters were misread), setting the tones beats sharpening first, and darkening thin strokes only helps at the end. With the best order, misread letters dropped from 21.7% to 18.7% across the readable versions, and the clean-up got about three times faster.
Straightening curved lines
Pages that bow or curl in a photo have curved lines of text. ScanKit now follows each line across the page, fits one smooth bend to all of them and flattens it. It leaves pages alone when the lines are already straight. Moving every pixel once instead of three times (perspective, tilt, then curve) gave the same letters and a little more speed.
Joining dot-matrix print
Old printers make letters out of dots, and a photo can turn them into loose specks. ScanKit now spots print made of evenly spaced dots of one size and closes the small gaps between them. It leaves the empty spaces inside and between letters open.
A better black-and-white cut-off
Black & white has to decide, pixel by pixel, what is ink and what is paper. We tested 19 ways of making that call. The ones that use one cut-off for the whole page fall apart under a shadow. The method we already used, which judges each spot against its neighbourhood, stayed. Giving it a larger neighbourhood helped on every version of the photo: misread letters went from 21.9% to 20.9%, and faint strokes lost to white went from 8.4% to 5.9%.
Smaller black-and-white PDFs
Black & white pages now go into the PDF without photo compression, which suits pure ink and paper much better. On our test page that was about a third of the size (184 KB instead of 640 KB), and just as readable.
Faster, in the background
The clean-up now runs in the background. A quick half-size preview shows up in about half a second while the full page is finished alongside it. While you look at one photo, the next one is already being prepared.
A "Crisp text" look, as an option
Some people want text in solid black with the pictures and logos left in grey, so we added Crisp text. On low-resolution photos the text reader misread a few more digits with it than with Magic color, so it is an option, not the default.
What did not work
Thinning heavy letters
Some phone scans look cleaner because heavy print is made thinner. We tried it. The letters did look lighter, but the text reader misread more on all three real photos we tested, so it is off.
Rebuilding letter edges
We had a step that redrew the edges of letters as a sharp ramp from ink to paper. The text did look crisper, but the step cost letters in every position we put it in the chain. It is out of the default clean-up. Removing noise on photos that were not noisy also cost a little, so it is out of the default as well.
Working at a bigger size
Enlarging the page more before cleaning it (3,000 to 4,000 pixels instead of 2,000) did not help on a small phone photo and was up to 2.4 times slower. Working smaller (1,600) clearly hurt. 2,000 pixels stays.
AI upscaling
We ran the page through an AI model that "imagines" missing detail. It invented letters that were not there, and the text reader read some of them with full confidence. Even a careful mix that only kept the AI's version where it read better cut misread letters only from 120 to 112, at about a minute and a half per page. Not worth it, and inventing letters is not something we want on your documents.
Tried, and put aside
Fixing misread words
We read the page's text, checked each word against an English word list, and corrected a word only when one real word was a clear winner. Numbers and names were never changed. On its own text that worked: fewer misread letters and no line made worse. We then went a step further and retyped the corrected words into the picture in the page's own typeface, and on clear print they blended in well. But retyping puts letters on the page that the camera never saw, and a wrong word drawn crisply looks more trustworthy than a blurry right one. For now we have put it aside rather than ship something that changes what your document says.
Everything above runs in your browser, on your device. Your photos are not uploaded to clean them up.
Questions, or a photo that still comes out hard to read? Write to [email protected].