DIY archivists push budget Nikons to 902,000 clicks to save 1,800 rare books

throwaway2037

Blog post about the project: https://ibteda.org/journey/ Digital library: https://archive.org/details/ibteda?tab=collection

mk_stjames

The approach noted in this article is similar to an experience I just had. I recently had some old hardware manuals I wanted to digitize, which were pages in 3-ring binders. I started thinking about scanning them but did not have an auto-loading, xerox style scanner at home. I looked around for free software that could do this 'google books style' as they did here in this article - take a photo of each page and automatically find the corners, dewarp, white balance, etc. and export to PDF. I actually didn't find a premade solution that looked like what I wanted. So I wrote my own, with Python an a simple tk GUI. I tried a lot Open CV messing about to do automatic page corner detection and whatnot, but found everything I tried unreliable. Sure, the corners may autodetect about 95% of the time, but if 5% of the pages fail and get cropped wrong or dewarped weird, I noted the time it took to go back and fix those errors eclipsed the time saved by the entire rest of the automatic process. The fastest method I found was a human in the loop semi-auto program: it loaded in all the images from SD card, and had the user manually click the 4-corners of the page as shown in the image. Upon the 4th point being clicked, the page would auto-de-perspective and apply the last saved default white balance curve, thresholding, and posterizing (color binning) / grayscaling / scaling to fixed width. For subsequent images in the camera roll this meant settings didn't have to usually be changed on a per-image basis. Then just a spacebar to add the image to the saved temp queue and at the end an export to PDF. This approach had me digitizing about 600 pages in an hour I estimate. Yes, it was a lot of clicking for the manual part.... but it allowed me to proof every page as it flew by and this caught a few photographic errors that I wouldn't have caught if it was fully automated, and not a single page was left mis-shapen or unreadable.