Appears to be necessary to disable each state of the pipeline that is
inactive, not just initial and terminal stages of an inactive segment.
If nothing else this makes what is going on more explicit.
pdftoppm in recent versions (0.26.4,5) seems to be incapable of
producing valid TIFFs, so have it dump a .pnm file and let ImageMagick
figure out how to convert it to TIFF. This is not ideal, but at least
it works.
convert .pnm -deskew <...> .pnm seems to have a bug that produces an
invalid .pnm file which later causes tesseract (specifically,
leptonica) to choke (using 3.02/1.71 as versions, respectively). Will
change pipeline to use tiffs internally since they are less stupid.
Put TESS_CFG_FILES last because it is optional and can be blank. If
omitted it breaks the sequence of subsequent parameters. Also cleanup
text output in this new mode.
If a page contains font data, the script would abort, unless -f was given,
in which case it would use pdftoppm to rasterize the font into a bitmap
and then attempt to OCR it. -f is almost certainly not what users want
unless they want to debug OCR or something.
If a PDF already has fonts it either was OCR'd already, or it is
a composite file containing, for example, some scanned documents appended
to a text report. In the latter case, this -s option provides OCR on
pages that don't have it without changing those that do, and if a PDF
was completely OCRed it will be converted to PDF/A. In batch jobs with
a mix of OCR and non-OCR the implicit conversion to PDF/A is also useful.
Leptonica does not interpret those extensions correctly. However, when
asked to produce a .pnm file, it will produce the expected .pbm/pgm/ppm
file depending on the input. So ask it to produce a .pnm and then
adjust the extension.
And add a test case.
A few design notes:
Leptonica's deskew is far superior to ImageMagick's convert -deskew command --
around 30-40x faster. Subjectively the output appears to this contributor to
be of higher quality as well. The difference is the algorithm: ImageMagick
uses the complex Hough transform to find the skew angle, while Leptonica uses
the simpler method, Postl's variance of differential line sums -- conceptually, shear the image and check for straight horizontal. In this case
simplicity wins. Finding the skew angle is the bulk of the work.
Leptonica's author explains the advantages of his approach here:
http://www.leptonica.com/skew-measurement.html
Leptonica is the low-level library that Tesseract depends on. Hence, this
project already depends on Leptonica. Leptonica can read and write most
common image file types on its own.
Unfortunately its error handling is poor: it seldom returns any meaningful
error codes. The best it manages is writing messages to stderr, which in
the context of a verbose script is just confusing since the error's source
is not indicated. The problem is compounded by Tesseract's use of Leptonica,
which will produce exactly the same errors in some cases. So we trap stderr
between calls to Leptonica and parse it for a few different types of error
message.
leptonica.py is Python 2/3 compatible and set up to provide access to other
Leptonica functions as needed. Of particular interest are its orientation
detection (including flip and rotation errors) which it does by comparing
text ascenders to descenders.
There is a PyPI "pylepthonica" package, however it is out of date by a few
years, and it implements all of Leptonica with Python wrappers -- so it is
massive, with one .py file at 2.5 MB. This module is loosely inspired by
pyleptonica but more modern, up to date, and contains only limited
functionality.
mktemp: consider both FreeBSD/OSX and Linux OS having incompatible
syntax
From now on temporary files are saved in the folder specified by the
environment variable $TMPDIR
self.width remains undefined unless hOCR finds text. It might not, if
a page contains only an image for example.
Full error message is:
AttributeError: ‘hocrTransform’ object has no attribute ‘width’
Jim Barlow
fritz-hh
Martin Ettl
Andreas Christ
Mansour Behabadi