v13.2.0 release notes

Closes #868

.github/workflows/build.yml

@@ -194,8 +194,7 @@ jobs:
       - name: Install system packages
         run: |
           choco install --yes --no-progress --pre tesseract
-          choco install --yes --no-progress ghostscript
-          choco install --yes --no-progress pngquant
+          choco install --yes --no-progress --ignore-checksums ghostscript pngquant
 
       - name: Install Python packages
         run: |

.readthedocs.yaml

@@ -14,7 +14,7 @@ formats:
 
 # Optionally set the version of Python and requirements required to build your docs
 python:
-  version: 3.7
+  version: "3.7"
   install:
     - method: pip
       path: .

docs/advanced.rst

@@ -236,12 +236,6 @@ PDF.js viewer.
 
 This works in all versions of Tesseract.
 
-The ``tesseract`` renderer
---------------------------
-
-The ``tesseract`` renderer was removed. OCRmyPDF's new approach to text
-layer grafting makes it functionally equivalent to ``sandwich``.
-
 Return code policy
 ==================
 

docs/api.rst

@@ -68,6 +68,14 @@ OCRmyPDF, use processes.
     not take at least one of these steps, process semantics will prevent
     OCRmyPDF from working correctly.
 
+.. warning::
+
+    On macOS with Python 3.7, you must call
+    :func:`multiprocessing.set_start_method("spawn")`. Without this, multiprocessing
+    will be unstable. From the command line, OCRmyPDF does this automatically,
+    but as an API user you must do this. See Python bpo-33725 for details.
+    Python 3.8+ also resolve this automatically.
+
 Logging
 -------
 

docs/contributing.rst

@@ -46,17 +46,6 @@ Style guide: Is it OCRmyPDF or ocrmypdf?
 
 The program/project is OCRmyPDF and the name of the executable or library is ocrmypdf.
 
-Known ports/packagers
-=====================
-
-OCRmyPDF has been ported to many platforms already. If you are interesting in
-porting to a new platform, check with
-`Repology <https://repology.org/projects/?search=ocrmypdf>`__ to see the status
-of that platform.
-
-Packager maintainers, please ensure that the command line completion scripts in
-``misc/`` are installed.
-
 Copyright and license
 =====================
 

docs/cookbook.rst

@@ -104,6 +104,9 @@ This produces a file named "output.pdf" and a companion text file named
     because of options like ``--skip-big`` or ``--tesseract-timeout``, those pages
     will not be in the sidecar.
 
+    If you don't want to generate the output PDF, use ``--output-type=none`` to
+    avoid generating one. Set the output filename to ``-`` (i.e. redirect to stdout).
+
     To extract all text from a PDF, whether generated from OCR or otherwise,
     use a program like Poppler's ``pdftotext`` or ``pdfgrep``.
 

docs/index.rst

@@ -40,6 +40,7 @@ image processing and OCR to existing PDFs.
    plugins
    apiref
    contributing
+   maintainers
 
 Indices and tables
 ==================

docs/installation.rst

@@ -211,69 +211,6 @@ user's ``PATH`` to check for the user's Python packages.
 
 To add JBIG2 encoding, see :ref:`jbig2`.
 
-Ubuntu 16.04 LTS
-----------------
-
-No package is available for Ubuntu 16.04. OCRmyPDF 8.0 and newer require
-Python 3.7. Ubuntu 16.04 ships Python 3.5, but you can install Python
-3.7 on it. Or, you can skip Python 3.7 and install OCRmyPDF 7.x or older
-- for that procedure, please see the installation documentation for the
-version of OCRmyPDF you plan to use.
-
-**Install system packages for OCRmyPDF**
-
-.. code-block:: bash
-
-    sudo apt-get update
-    sudo apt-get install -y software-properties-common python-software-properties
-    sudo add-apt-repository -y \
-        ppa:jonathonf/python-3.7 \
-        ppa:alex-p/tesseract-ocr
-    sudo apt-get update
-    sudo apt-get install -y \
-        ghostscript \
-        libexempi3 \
-        libffi6 \
-        pngquant \
-        python3.7 \
-        qpdf \
-        tesseract-ocr \
-        unpaper
-
-This will install a Python 3.7 binary at ``/usr/bin/python3.7``
-alongside the system's Python 3.5. Do not remove the system Python. This
-will also install Tesseract 4.0 from a PPA, since the version available
-in Ubuntu 16.04 is too old for OCRmyPDF.
-
-Now install pip for Python 3.7. This will install the Python 3.7 version
-of ``pip`` at ``/usr/local/bin/pip``.
-
-.. code-block:: bash
-
-    curl https://bootstrap.pypa.io/get-pip.py | sudo python3.7
-
-**Install OCRmyPDF**
-
-OCRmyPDF requires the locale to be set for UTF-8. **On some minimal
-Ubuntu installations**, such as the Ubuntu 16.04 Docker images it may be
-necessary to set the locale.
-
-.. code-block:: bash
-
-    # Optional: Only need to set these if they are not already set
-    export LC_ALL=C.UTF-8
-    export LANG=C.UTF-8
-
-Now install OCRmyPDF for the current user, and ensure that the ``PATH``
-environment variable contains ``$HOME/.local/bin``.
-
-.. code-block:: bash
-
-    export PATH=$HOME/.local/bin:$PATH
-    pip3.7 install --user ocrmypdf
-
-To add JBIG2 encoding, see :ref:`jbig2`.
-
 Arch Linux (AUR)
 ----------------
 

docs/maintainers.rst

@@ -0,0 +1,60 @@
+================
+Maintainer notes
+================
+
+This is for those who package OCRmyPDF for downstream use.  (Thank you 
+for your hard work.)
+
+Known ports/packagers
+=====================
+
+OCRmyPDF has been ported to many platforms already. If you are interesting in
+porting to a new platform, check with
+`Repology <https://repology.org/projects/?search=ocrmypdf>`__ to see the status
+of that platform.
+
+Make sure you can package pikepdf
+---------------------------------
+
+pikepdf, created by the same author, is a mixed Python and C++14 package with
+much stiffer build requirements. If you want to use OCRmyPDF on some novel platform
+or distribution, first make sure you can package pikepdf.
+
+Non-Python dependencies
+-----------------------
+
+Note that we have non-Python dependencies. In particular, OCRmyPDF requires
+Ghostscript and Tesseract OCR to be installed and needs to be able to locate their
+binaries on the system PATH. On Windows, OCRmyPDF will also check the registry 
+for their locations.
+
+Tesseract OCR relies on SIMD for performance and only has proper support for this
+on ARM and x86_64. Performance may be poor on other processor architectures.
+
+Versioning scheme
+-----------------
+
+OCRmyPDF uses setuptools-scm for versioning, which derives the version from
+Git as a single source of truth. This may be unsuitable for some distributions, e.g.
+to indicate that your distribution modifies OCRmyPDF in some way.
+
+You can patch the ``__version__`` variable in ``src/ocrmypdf/_version.py`` if 
+necessary.
+
+OCRmyPDF uses setuptools-scm-git-archive to ensure that tarballs downloaded from
+GitHub contain version information. Unfortunately, these tarballs are not always
+deterministic. See this 
+`issue <https://github.com/jbarlow83/OCRmyPDF/issues/841#issuecomment-936562696>`_.
+
+jbig2enc
+--------
+
+OCRmyPDF will use jbig2enc, a JBIG2 encoder, if one can be found. Some distributions
+have shied away from packaging JBIG2 because it contains patented algorithms, but
+all patents have expired since 2017. If possible, consider packaging it too to
+improve OCRmyPDF's compression.
+
+Command line completions
+------------------------
+
+Please ensure that command line completions are installed.

docs/release_notes.rst

@@ -14,17 +14,49 @@ wish to use some of its features for working with PDFs.
 
 .. note::
 
-   Python 3.6 reaches end of life on December 23, 2021. We have already ended support 
+   Python 3.6 reaches end of life on December 23, 2021. We have already ended support
    for Python 3.6 but might release fixes for critical issues if necessary before that
    date.
 
+v13.2.0
+=======
+
+-  Removed all runtime uses of distutils since it is deprecated in standard library. We
+   previous used ``distutils.version`` to examine version numbers of dependencies
+   at run time, and now use ``packaging.version`` for this. This is a new
+   dependency.
+-  Fixed an error message advising the user that Ghostscript was not installed being
+   suppressed when this condition actually happens.
+-  Fixed an issue with incorrect page number and totals being displayed in the progress
+   bar. This was purely a display/presentation issue. :issue:`876`.
+
+v13.1.1
+=======
+
+-  Fixed issue with attempting to deskew a blank page on Tesseract 5. :issue:`868`.
+
+v13.1.0
+=======
+
+-  Changed to using Python concurrent.futures-based parallel execution instead of
+   pools, since futures have now exceed pools in features.
+-  If a child worker is terminated (perhaps by the operating system or the user
+   killing it in a task manager), the parallel task will fail an error message.
+   Previously, the main ocrmypdf process would "hang" indefinitely, waiting for the
+   child to report.
+-  Added new argument ``--tesseract-thresholding`` to provide control over Tesseract 5's
+   threshold parameter.
+-  Documentation updates and changes. Better documentation for ``--output-type none``,
+   added a few releases ago. Removed some obsolete documentation.
+-  Improved bash completions - thanks to @FPille.
+
 v13.0.0
 =======
 
 **Breaking changes**
 
 -  The deprecated module ``ocrmypdf.leptonica`` has been removed.
--  We no longer depend on Leptonica (``liblept``) or CFFI (``libffi``, 
+-  We no longer depend on Leptonica (``liblept``) or CFFI (``libffi``,
    ``python3-cffi``). (Note that Tesseract still requires Leptonica; OCRmyPDF no longer
    directly uses this library.)
 -  The argument ``--remove-background`` is temporarily disabled while we search for an
@@ -48,7 +80,7 @@ v13.0.0
 
 **Fixes**
 
--  Fixed an issue where OCRmyPDF failed to find Ghostscript on Windows even when 
+-  Fixed an issue where OCRmyPDF failed to find Ghostscript on Windows even when
    installed, and would exit with an error.
 -  By removing Leptonica, we fixed all issues related to Leptonica on Apple
    Silicon or Leptonica failing to import on Windows.
@@ -56,7 +88,7 @@ v13.0.0
 v12.7.2
 =======
 
--  Fixed "invalid version number" error for Tesseract packaging with nonstandard 
+-  Fixed "invalid version number" error for Tesseract packaging with nonstandard
    version "5.0.0-rc1.20211030".
 -  Fixed use of deprecated ``importlib.resources.read_binary``.
 -  Replace some uses of string paths with ``pathlib.Path``.

misc/completion/ocrmypdf.bash

@@ -1,6 +1,6 @@
 # ocrmypdf completion                                     -*- shell-script -*-
 
-# Copyright 2019 Frank Pille
+# Copyright 2019, 2021 Frank Pille
 # Copyright 2020 Alex Willner
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
@@ -23,95 +23,285 @@
 
 set -o errexit
 
-_ocrmypdf()
+__ocrmypdf_arguments()
 {
-    local cur prev cword words split
+    local arguments="--help                   (show help message)
+--language               (language(s) of the file to be OCRed)
+--image-dpi              (assume this DPI if input image DPI is unknown)
+--output-type            (select PDF output options)
+--sidecar                (write OCR to text file)
+--version                (print program version and exit)
+--jobs                   (how many worker processes to use)
+--quiet                  (suppress INFO messages)
+--verbose                (set verbosity level)
+--title                  (set metadata)
+--author                 (set metadata)
+--subject                (set metadata)
+--keywords               (set metadata)
+--rotate-pages           (rotate pages to correct orientation)
+--remove-background      (attempt to remove background from pages)
+--deskew                 (fix small horizontal alignment skew)
+--clean                  (clean document images before OCR)
+--clean-final            (clean document images and keep result)
+--unpaper-args           (a quoted string of arguments to pass to unpaper)
+--oversample             (oversample images to this DPI)
+--remove-vectors         (don\'t send vector objects to OCR)
+--threshold              (threshold images before OCR)
+--force-ocr              (OCR documents that already have printable text)
+--skip-text              (skip OCR on any pages that already contain text)
+--redo-ocr               (redo OCR on any pages that seem to have OCR already)
+--skip-big               (skip OCR on pages larger than this many MPixels)
+--optimize               (select optimization level)
+--jpeg-quality           (JPEG quality [0..100])
+--png-quality            (PNG quality [0..100])
+--jbig2-lossy            (enable lossy JBIG2 (see docs))
+--pages                  (apply OCR to only the specified pages)
+--max-image-mpixels      (image decompression bomb threshold)
+--pdf-renderer           (select PDF renderer options)
+--rotate-pages-threshold (page rotation confidence)
+--pdfa-image-compression (set PDF/A image compression options)
+--fast-web-view          (if file size if above this amount in MB linearize PDF)
+--plugin                 (name of plugin to import)
+--keep-temporary-files   (keep temporary files (debug)
+--tesseract-config       (set custom tesseract config file)
+--tesseract-pagesegmode  (set tesseract --psm)
+--tesseract-oem          (set tesseract --oem)
+--tesseract-thresholding (set tesseract image thresholding)
+--tesseract-timeout      (maximum number of seconds to wait for OCR)
+--user-words             (specify location of user words file)
+--user-patterns          (specify location of user patterns file)
+--no-progress-bar        (disable the progress bar)
+"
 
-    # Homebrew on Macs have version 1.3 of bash-completion which doesn't include - see #502
-    if declare -F _init_completions >/dev/null 2>&1; then
-        _init_completion -s || return
-    else
-        COMPREPLY=()
-        _get_comp_words_by_ref cur prev words cword
+    COMPREPLY=( $( compgen -W "$arguments" -- "$cur") )
+
+    # Remove description if only one completion exists
+    if [[ ${#COMPREPLY[*]} -eq 1 ]]; then
+        COMPREPLY=( ${COMPREPLY[0]%% *} )
     fi
+}
 
-    if [[ $cur == -* ]]; then
-        COMPREPLY=( $( compgen -W '--language --image-dpi --output-type
-            --sidecar --version --jobs --quiet --verbose --title --author
-            --subject --keywords --rotate-pages --deskew
-            --clean --clean-final --unpaper-args --oversample --remove-vectors
-            --force-ocr --skip-text --redo-ocr
-            --skip-big --jpeg-quality --png-quality --jbig2-lossy
-            --max-image-mpixels --tesseract-config --tesseract-pagesegmode
-            --help --tesseract-oem --pdf-renderer --tesseract-timeout
-            --rotate-pages-threshold --pdfa-image-compression --user-words
-            --user-patterns --keep-temporary-files --output-type
-            --no-progress-bar --pages --fast-web-view' \
-            --  "$cur" ) )
-        return
-    else
-        _filedir
-        return
+__ocrmypdf_output-type()
+{
+    local choices="pdfa   (output a PDF/A (default))
+pdf    (output a standard PDF)
+pdfa-1 (output a PDF/A-1b)
+pdfa-2 (output a PDF/A-2b)
+pdfa-3 (output a PDF/A-3b)
+none   (do not produce an output PDF (for example, if you only care about --sidecar))"
+
+    COMPREPLY=( $( compgen -W "$choices" -- "$cur") )
+
+    # Remove description if only one completion exists
+    if [[ ${#COMPREPLY[*]} -eq 1 ]]; then
+        COMPREPLY=( ${COMPREPLY[0]%% *} )
     fi
+}
 
+__ocrmypdf_verbose()
+{
+    local choices="0  (standard output messages)
+1  (troubleshooting output messages)
+2  (debugging output messages)"
+
+    COMPREPLY=( $( compgen -W "$choices" -- "$cur") )
+
+    # Remove description if only one completion exists
+    if [[ ${#COMPREPLY[*]} -eq 1 ]]; then
+        COMPREPLY=( ${COMPREPLY[0]%% *} )
+    fi
+}
+
+__ocrmypdf_optimize()
+{
+    local choices="0  (do not optimize)
+1  (do safe, lossless optimizations (default))
+2  (do some lossy optimizations)
+3  (do aggressive lossy optimizations (including lossy JBIG2))"
+
+    COMPREPLY=( $( compgen -W "$choices" -- "$cur") )
+
+    # Remove description if only one completion exists
+    if [[ ${#COMPREPLY[*]} -eq 1 ]]; then
+        COMPREPLY=( ${COMPREPLY[0]%% *} )
+    fi
+}
+
+__ocrmypdf_pdf-renderer()
+{
+    local choices="auto      (auto select PDF renderer)
+hocr      (use hOCR renderer)
+hocrdebug (uses hOCR renderer in debug mode, showing recognized text)
+sandwich  (use sandwich renderer)"
+
+    COMPREPLY=( $( compgen -W "$choices" -- "$cur") )
+
+    # Remove description if only one completion exists
+    if [[ ${#COMPREPLY[*]} -eq 1 ]]; then
+        COMPREPLY=( ${COMPREPLY[0]%% *} )
+    fi
+}
+
+__ocrmypdf_pdfa-image-compression()
+{
+    local choices="auto     (let Ghostscript decide how to compress images)
+jpeg     (convert color and grayscale images to JPEG)
+lossless (convert color and grayscale images to lossless (PNG))"
+
+    COMPREPLY=( $( compgen -W "$choices" -- "$cur") )
+
+    # Remove description if only one completion exists
+    if [[ ${#COMPREPLY[*]} -eq 1 ]]; then
+        COMPREPLY=( ${COMPREPLY[0]%% *} )
+    fi
+}
+
+__ocrmypdf_tesseract-pagesegmode()
+{
+    local choices="0  (orientation and script detection (OSD) only)
+1  (automatic page segmentation with OSD)
+2  (automatic page segmentation, but no OSD, or OCR)
+3  (fully automatic page segmentation, but no OSD (default))
+4  (assume a single column of text of variable sizes)
+5  (assume a single uniform block of vertically aligned text)
+6  (assume a single uniform block of text)
+7  (treat the image as a single text line)
+8  (treat the image as a single word)
+9  (treat the image as a single word in a circle)
+10 (treat the image as a single character)
+11 (sparse text - find as much text as possible in no particular order)
+12 (sparse text with OSD)
+13 (raw line - treat the image as a single text line)"
+
+    COMPREPLY=( $( compgen -W "$choices" -- "$cur") )
+
+    # Remove description if only one completion exists
+    if [[ ${#COMPREPLY[*]} -eq 1 ]]; then
+        COMPREPLY=( ${COMPREPLY[0]%% *} )
+    fi
+}
+
+__ocrmypdf_tesseract-oem()
+{
+    local choices="0 (legacy engine only)
+1 (neural nets LSTM engine only)
+2 (legacy + LSTM engines)
+3 (default, based on what is available)"
+
+    COMPREPLY=( $( compgen -W "$choices" -- "$cur") )
+
+    # Remove description if only one completion exists
+    if [[ ${#COMPREPLY[*]} -eq 1 ]]; then
+        COMPREPLY=( ${COMPREPLY[0]%% *} )
+    fi
+}
+
+__ocrmypdf_tesseract-thresholding()
+{
+    local choices="auto          (let OCRmyPDF pick thresholding - current always uses otsu)
+otsu          (use hOCR renderer)
+adaptive-otsu (use adaptive Otsu thresholding)
+sauvola       (use Sauvola thresholding)"
+
+    COMPREPLY=( $( compgen -W "$choices" -- "$cur") )
+    # Remove description if only one completion exists
+    if [[ ${#COMPREPLY[*]} -eq 1 ]]; then
+        COMPREPLY=( ${COMPREPLY[0]%% *} )
+    fi
+}
+
+
+__ocrmypdf_check_previous()
+{
     case $prev in
-        --version|-h|--help)
-            return
-            ;;
-        --user-words|--user-patterns|--tesseract-config)
-            _filedir
-            return
-            ;;
-        --output-type)
-            COMPREPLY=( $( compgen -W 'pdfa pdf pdfa-1 pdfa-2 pdfa-3' -- \
-                "$cur" ) )
-            return
-            ;;
-        --pdf-renderer)
-            COMPREPLY=( $( compgen -W 'auto hocr sandwich' -- "$cur" ) )
-            return
-            ;;
-        --pdfa-image-compression)
-            COMPREPLY=( $( compgen -W 'auto jpeg lossless' -- "$cur" ) )
-            return
-            ;;
-        -O|--optimize|--tesseract-oem)
-            COMPREPLY=( $( compgen -W '{0..3}' -- "$cur" ) )
-            return
-            ;;
-        --jpeg-quality|--png-quality)
-            COMPREPLY=( $( compgen -W '{0..100}' -- "$cur" ) )
-            return
+        -h|--help|--version)
+            return 0
             ;;
         -l|--language)
             COMPREPLY=$( command tesseract --list-langs 2>/dev/null )
             COMPREPLY=( $( compgen -W '${COMPREPLY[@]##*:}' -- "$cur" ) )
-            return
+            return 0
             ;;
-        --image-dpi|--oversample|--skip-big|--max-image-mpixels|\
-        --tesseract-timeout|--rotate-pages-threshold)
-            COMPREPLY=( $( compgen -P "$cur" -W '{0..9}' ) )
-            return
+        --output-type)
+            __ocrmypdf_output-type
+            return 0
             ;;
         -j|--jobs)
             COMPREPLY=( $( compgen -W '{1..'$( _ncpus )'}' -- "$cur" ) )
-            return
+            return 0
             ;;
         -v|--verbose)
-            COMPREPLY=( $( compgen -W '{0..2}' -- "$cur" ) ) # max level ?
-            return
+            __ocrmypdf_verbose
+            return 0
+            ;;
+        -O|--optimize)
+            __ocrmypdf_optimize
+            return 0
+            ;;
+        --pdf-renderer)
+            __ocrmypdf_pdf-renderer
+            return 0
+            ;;
+        --pdfa-image-compression)
+            __ocrmypdf_pdfa-image-compression
+            return 0
             ;;
         --tesseract-pagesegmode)
-            COMPREPLY=( $( compgen -W '{1..13}' -- "$cur" ) )
-            return
+            __ocrmypdf_tesseract-pagesegmode
+            return 0
             ;;
-        --sidecar|--title|--author|--subject|--keywords|--unpaper-args|--pages|--fast-web-view)
+        --tesseract-oem)
+            __ocrmypdf_tesseract-oem
+            return 0
+            ;;
+        --tesseract-thresholding)
+            __ocrmypdf_tesseract-thresholding
+            return 0
+            ;;
+
+        --title|--author|--subject|--keywords|--unpaper-args|--pages|--plugin|\
+        --jpeg-quality|--png-quality|--image-dpi|--oversample|--skip-big|--max-image-mpixels|\
+        --tesseract-timeout|--rotate-pages-threshold|--fast-web-view)
             # argument required but no completions available
-            return
+            return 0
+            ;;
+        --tesseract-config|--user-words|--user-patterns|--sidecar)
+            _filedir
+            return 0
             ;;
     esac
 
-    $split && return
+    return 1
+}
+
+_ocrmypdf()
+{
+    local OLDIFS="$IFS"
+    local IFS=$'\n'
+
+    local cur prev
+
+    # Homebrew on Macs have version 1.3 of bash-completion which doesn't include - see #502
+    if declare -F _init_completion >/dev/null 2>&1; then
+      _init_completion  || return
+    else
+        COMPREPLY=()
+        _get_comp_words_by_ref cur prev
+    fi
+
+    if __ocrmypdf_check_previous -ne 0; then
+        return
+    fi
+
+    if [[ "$cur" == -* ]]; then
+        __ocrmypdf_arguments
+    else
+        _filedir
+    fi
+
+    IFS="$OLDIFS"
+
+    return
 } &&
 complete -F _ocrmypdf ocrmypdf
 

misc/completion/ocrmypdf.fish

@@ -105,20 +105,20 @@ complete -c ocrmypdf -x -l pages -d "apply OCR to only the specified pages"
 complete -c ocrmypdf -x -l tesseract-config -d "set custom tesseract config file"
 
 function __fish_ocrmypdf_tesseract_pagesegmode
-  echo -e "0\t"(_ "orientation and script detection (OSD) only")
-  echo -e "1\t"(_ "automatic page segmentation with OSD")
-  echo -e "2\t"(_ "automatic page segmentation, but no OSD, or OCR")
-  echo -e "3\t"(_ "fully automatic page segmentation, but no OSD (default)")
-  echo -e "4\t"(_ "assume a single column of text of variable sizes")
-  echo -e "5\t"(_ "assume a single uniform block of vertically aligned text")
-  echo -e "6\t"(_ "assume a single uniform block of text")
-  echo -e "7\t"(_ "treat the image as a single text line")
-  echo -e "8\t"(_ "treat the image as a single word")
-  echo -e "9\t"(_ "treat the image as a single word in a circle")
-  echo -e "10\t"(_ "treat the image as a single character")
-  echo -e "11\t"(_ "sparse text - find as much text as possible in no particular order")
-  echo -e "12\t"(_ "sparse text with OSD")
-  echo -e "13\t"(_ "raw line - treat the image as a single text line")
+    echo -e "0\t"(_ "orientation and script detection (OSD) only")
+    echo -e "1\t"(_ "automatic page segmentation with OSD")
+    echo -e "2\t"(_ "automatic page segmentation, but no OSD, or OCR")
+    echo -e "3\t"(_ "fully automatic page segmentation, but no OSD (default)")
+    echo -e "4\t"(_ "assume a single column of text of variable sizes")
+    echo -e "5\t"(_ "assume a single uniform block of vertically aligned text")
+    echo -e "6\t"(_ "assume a single uniform block of text")
+    echo -e "7\t"(_ "treat the image as a single text line")
+    echo -e "8\t"(_ "treat the image as a single word")
+    echo -e "9\t"(_ "treat the image as a single word in a circle")
+    echo -e "10\t"(_ "treat the image as a single character")
+    echo -e "11\t"(_ "sparse text - find as much text as possible in no particular order")
+    echo -e "12\t"(_ "sparse text with OSD")
+    echo -e "13\t"(_ "raw line - treat the image as a single text line")
 end
 complete -c ocrmypdf -x -l tesseract-pagesegmode -a '(__fish_ocrmypdf_tesseract_pagesegmode)' -d "set tesseract --psm"
 
@@ -129,6 +129,15 @@ function __fish_ocrmypdf_tesseract_oem
     echo -e "3\t"(_ "default, based on what is available")
 end
 complete -c ocrmypdf -x -l tesseract-oem -a '(__fish_ocrmypdf_tesseract_oem)' -d "set tesseract --oem"
+
+function __fish_ocrmypdf_tesseract_thresholding
+    echo -e "auto\t"(_ "let OCRmyPDF pick thresholding (current always uses otsu)")
+    echo -e "otsu\t"(_ "legacy Otsu thresholding")
+    echo -e "adaptive-otsu\t"(_ "use adaptive Otsu thresholding")
+    echo -e "sauvola\t"(_ "use Sauvola thresholding")
+end
+complete -c ocrmypdf -x -l tesseract-thresholding -a '(__fish_ocrmypdf_tesseract_thresholding)' -d "set tesseract thresholding method (needs Tesseract 5.x)"
+
 complete -c ocrmypdf -x -l tesseract-timeout -d "maximum number of seconds to wait for OCR"
 complete -c ocrmypdf -x -l rotate-pages-threshold -d "page rotation confidence"
 

pyproject.toml

@@ -12,7 +12,7 @@ version_scheme = "post-release"
 
 [tool.black]
 line-length = 88
-target-version = ["py36", "py37", "py38"]
+target-version = ["py37", "py38"]
 skip-string-normalization = true
 include = '\.pyi?$'
 exclude = '''
@@ -44,24 +44,38 @@ source = ["src/ocrmypdf"]
 [tool.coverage.report]
 # Regexes for lines to exclude from consideration
 exclude_lines = [
-    # Have to re-enable the standard pragma
-    "pragma: no cover",
-
-    # Don't complain if tests don't hit defensive assertion code:
-    "raise AssertionError",
-    "raise NotImplementedError",
-
-    # Don't complain if non-runnable code isn't run:
-    "if 0:",
-	  "if False:",
-    "if __name__ == .__main__.:",
-    "if TYPE_CHECKING:"
+  # Have to re-enable the standard pragma
+  "pragma: no cover",
+  # Don't complain if tests don't hit defensive assertion code:
+  "raise AssertionError",
+  "raise NotImplementedError",
+  # Don't complain if non-runnable code isn't run:
+  "if 0:",
+  "if False:",
+  "if __name__ == .__main__.:",
+  "if TYPE_CHECKING:"
 ]
 
 [tool.isort]
 profile = "black"
 known_first_party = "ocrmypdf"
-known_third_party = ["PIL", "flask", "img2pdf", "ocrmypdf", "pdfminer", "pikepdf", "pkg_resources", "pluggy", "pytest", "reportlab", "setuptools", "sphinx_rtd_theme", "tqdm", "watchdog", "werkzeug"]
+known_third_party = [
+  "PIL",
+  "flask",
+  "img2pdf",
+  "ocrmypdf",
+  "pdfminer",
+  "pikepdf",
+  "pkg_resources",
+  "pluggy",
+  "pytest",
+  "reportlab",
+  "setuptools",
+  "sphinx_rtd_theme",
+  "tqdm",
+  "watchdog",
+  "werkzeug"
+]
 
 [tool.pytest.ini_options]
 minversion = "6.0"
@@ -75,6 +89,13 @@ filterwarnings = ["ignore:.*XMLParser.*:DeprecationWarning"]
 
 [[tool.mypy.overrides]]
 module = [
-  'pluggy', 'tqdm', 'coloredlogs', 'img2pdf', 'pdfminer.*', 'reportlab.*'
+  'pluggy',
+  'tqdm',
+  'coloredlogs',
+  'img2pdf',
+  'pdfminer.*',
+  'reportlab.*',
+  'fitz',
+  'libxmp.utils'
 ]
 ignore_missing_imports = true

setup.cfg

@@ -48,6 +48,7 @@ install_requires =
     Pillow>=8.2.0
     coloredlogs>=14.0  # strictly optional
     img2pdf>=0.3.0,<0.5  # pure Python
+    packaging>=20
     pdfminer.six!=20200720,>=20191110,<=20211012
     pikepdf>=4.0.0
     pluggy>=0.13.0,<2

src/ocrmypdf/_exec/ghostscript.py

@@ -97,6 +97,7 @@ def rasterize_pdf(
             '-dSAFER',
             '-dBATCH',
             '-dNOPAUSE',
+            '-dInterpolateControl=-1',
             f'-sDEVICE={raster_device}',
             f'-dFirstPage={pageno}',
             f'-dLastPage={pageno}',

src/ocrmypdf/_exec/tesseract.py

@@ -9,13 +9,13 @@
 
 import logging
 import re
-from distutils.version import StrictVersion
 from math import pi
 from os import fspath
 from pathlib import Path
 from subprocess import PIPE, STDOUT, CalledProcessError, TimeoutExpired
 from typing import Dict, Iterator, List, Optional
 
+from packaging.version import Version
 from PIL import Image
 
 from ocrmypdf.exceptions import (
@@ -46,6 +46,13 @@ HOCR_TEMPLATE = """<?xml version="1.0" encoding="UTF-8"?>
 </html>
 """
 
+TESSERACT_THRESHOLDING_METHODS: Dict[str, int] = {
+    'auto': 0,
+    'otsu': 0,
+    'adaptive-otsu': 1,
+    'sauvola': 2,
+}
+
 
 class TesseractLoggerAdapter(logging.LoggerAdapter):
     def process(self, msg, kwargs):
@@ -53,25 +60,54 @@ class TesseractLoggerAdapter(logging.LoggerAdapter):
         return '[tesseract] %s' % (msg), kwargs
 
 
-class TesseractVersion(StrictVersion):
-
-    version_re = re.compile(
-        r'''
-        ^(\d+) \. (\d+) (\. (\d+))?  # groups: 1/major, 2/minor, 3/[skip], 4/patch
-        [-]?  # optional hyphen separator
-        (?: ((?:alpha|beta|rc|dev)\d*)? [.\-\ ]? (\d+)? )?  # 5/prerelease, 6/prerelease_num
-        (?:(?:-\d+)?-g[0-9a-f]+)?  # untagged git version
-        $
-        ''',
-        re.VERBOSE | re.ASCII,
+TESSERACT_VERSION_PATTERN = r"""
+    v?
+    (?:
+        (?:(?P<epoch>[0-9]+)!)?                           # epoch
+        (?P<release>[0-9]+(?:\.[0-9]+)*)                  # release segment
+        (?P<pre>                                          # pre-release
+            [-_\.]?
+            (?P<pre_l>(a|b|c|rc|alpha|beta|pre|preview))
+            [-_\.]?
+            (?P<pre_n>[0-9]+)?
+        )?
+        (?P<post>                                         # post release
+            (?:-(?P<post_n1>[0-9]+))
+            |
+            (?:
+                [-_\.]?
+                (?P<post_l>post|rev|r)
+                [-_\.]?
+                (?P<post_n2>[0-9]+)?
+            )
+        )?
+        (?P<dev>                                          # dev release
+            [-_\.]?
+            (?P<dev_l>dev)
+            [-_\.]?
+            (?P<dev_n>[0-9]+)?
+        )?
+        (?P<date>
+            [-_\.]
+            (?:20[0-9][0-9] [0-1][0-9] [0-3][0-9])       # yyyy mm dd
+        )?
+        (?P<gitcount>
+            [-_\.]?
+            [0-9]+
+        )?
+        (?P<gitcommit>
+            [-_\.]?
+            g[0-9a-f]{2,10}
+        )?
     )
+    (?:\+(?P<local>[a-z0-9]+(?:[-_\.][a-z0-9]+)*))?       # local version
+"""
 
-    def parse(self, vstring):
-        try:
-            super().parse(vstring)
-        except TypeError as e:
-            if 'int() argument must be a string' in str(e):
-                super().parse(vstring + '-0')
+
+class TesseractVersion(Version):
+    _regex = re.compile(
+        r"^\s*" + TESSERACT_VERSION_PATTERN + r"\s*$", re.VERBOSE | re.IGNORECASE
+    )
 
 
 def version() -> str:
@@ -87,6 +123,11 @@ def has_user_words():
     return version() >= '4.1'
 
 
+def has_thresholding():
+    """Does Tesseract have -c thresholding method capability?"""
+    return version() >= '5.0'
+
+
 def get_languages():
     def lang_error(output):
         msg = (
@@ -188,7 +229,9 @@ def get_deskew(
     except CalledProcessError as e:
         tesseract_log_output(e.stdout)
         tesseract_log_output(e.stderr)
-        if b'Empty page!!' in e.output:  # Not enough info for a skew angle
+        if b'Empty page!!' in e.output or (
+            e.output == b'' and e.returncode == 1
+        ):  # Not enough info for a skew angle - Tess 4 and 5 return different errors
             return 0.0
 
         raise SubprocessOutputError() from e
@@ -265,9 +308,11 @@ def generate_hocr(
     tessconfig: List[str],
     timeout: float,
     pagesegmode: int,
+    thresholding: int,
     user_words,
     user_patterns,
 ):
+    """Generate a hOCR file, which must be converted to PDF."""
     prefix = output_hocr.with_suffix('')
 
     args_tesseract = tess_base_args(languages, engine_mode)
@@ -275,6 +320,9 @@ def generate_hocr(
     if pagesegmode is not None:
         args_tesseract.extend(['--psm', str(pagesegmode)])
 
+    if thresholding != 0 and has_thresholding():
+        args_tesseract.extend(['-c', f'thresholding_method={thresholding}'])
+
     if user_words:
         args_tesseract.extend(['--user-words', user_words])
 
@@ -326,20 +374,15 @@ def generate_pdf(
     tessconfig: List[str],
     timeout: float,
     pagesegmode: int,
+    thresholding: int,
     user_words,
     user_patterns,
 ):
-    """Use Tesseract to render a PDF.
+    """Generate a PDF using Tesseract's internal PDF generator.
 
-    input_file -- image to analyze
-    output_pdf -- file to generate
-    output_text -- OCR text file
-    languages -- list of languages to consider
-    engine_mode -- engine mode argument for tess v4
-    tessconfig -- tesseract configuration
-    timeout -- timeout (seconds)
+    We specifically a text-only PDF which is more suitable for combining with
+    the input page.
     """
-
     args_tesseract = tess_base_args(languages, engine_mode)
 
     if pagesegmode is not None:
@@ -347,6 +390,9 @@ def generate_pdf(
 
     args_tesseract.extend(['-c', 'textonly_pdf=1'])
 
+    if thresholding != 0 and has_thresholding():
+        args_tesseract.extend(['-c', f'thresholding_method={thresholding}'])
+
     if user_words:
         args_tesseract.extend(['--user-words', user_words])
 

src/ocrmypdf/_pipeline.py

@@ -511,10 +511,6 @@ def create_ocr_image(image: Path, page_context: PageContext):
     output_file = page_context.get_path('ocr.png')
     options = page_context.options
     with Image.open(image) as im:
-        white = ImageColor.getcolor('#ffffff', im.mode)
-        # pink = ImageColor.getcolor('#ff0080', im.mode)
-        draw = ImageDraw.ImageDraw(im)
-
         log.debug('resolution %r', im.info['dpi'])
 
         if not options.force_ocr:
@@ -524,6 +520,7 @@ def create_ocr_image(image: Path, page_context: PageContext):
             if options.redo_ocr:
                 mask = True  # Mask visible text, but not invisible text
 
+            draw = ImageDraw.ImageDraw(im)
             for textarea in page_context.pageinfo.get_textareas(
                 visible=mask, corrupt=None
             ):
@@ -539,9 +536,8 @@ def create_ocr_image(image: Path, page_context: PageContext):
                     im.height - bbox[1] * xyscale[1],
                 )
                 log.debug('blanking %r', pixcoords)
-                draw.rectangle(pixcoords, fill=white)
-                # draw.rectangle(pixcoords, outline=pink)
-        del draw
+                draw.rectangle(pixcoords, fill='white')
+                # draw.rectangle(pixcoords, outline='pink')
 
         filter_im = page_context.plugin_manager.hook.filter_ocr_image(
             page=page_context, image=im

src/ocrmypdf/_plugin_manager.py

@@ -11,7 +11,7 @@ import importlib.util
 import pkgutil
 import sys
 from pathlib import Path
-from typing import List, Tuple, Union
+from typing import List, Sequence, Tuple, Union
 
 import pluggy
 
@@ -110,7 +110,7 @@ def get_plugin_manager(plugins: List[Union[str, Path]], builtins=True):
 
 
 def get_parser_options_plugins(
-    args,
+    args: Sequence[str],
 ) -> Tuple[argparse.ArgumentParser, argparse.Namespace, pluggy.PluginManager]:
     pre_options, _unused = plugins_only_parser.parse_known_args(args=args)
     plugin_manager = get_plugin_manager(pre_options.plugins)

src/ocrmypdf/_sync.py

@@ -5,15 +5,18 @@
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
 
+import argparse
 import logging
 import logging.handlers
 import os
 import sys
 import threading
+from concurrent.futures.process import BrokenProcessPool
+from concurrent.futures.thread import BrokenThreadPool
 from functools import partial
 from pathlib import Path
 from tempfile import mkdtemp
-from typing import List, NamedTuple, Optional, Tuple
+from typing import List, NamedTuple, Optional, Tuple, cast
 
 import PIL
 
@@ -46,7 +49,7 @@ from ocrmypdf._pipeline import (
     triage,
     validate_pdfinfo_options,
 )
-from ocrmypdf._plugin_manager import get_plugin_manager
+from ocrmypdf._plugin_manager import OcrmypdfPluginManager, get_plugin_manager
 from ocrmypdf._validation import (
     check_requested_output_file,
     create_input_file,
@@ -165,7 +168,7 @@ def make_intermediate_images(
     return ocr_image, preprocess_out
 
 
-def exec_page_sync(page_context: PageContext):
+def exec_page_sync(page_context: PageContext) -> PageResult:
     options = page_context.options
     tls.pageno = page_context.pageno + 1
 
@@ -223,7 +226,7 @@ def exec_page_sync(page_context: PageContext):
     )
 
 
-def post_process(pdf_file, context: PdfContext, executor: Executor):
+def post_process(pdf_file: Path, context: PdfContext, executor: Executor) -> Path:
     pdf_out = pdf_file
     if context.options.output_type.startswith('pdfa'):
         ps_stub_out = generate_postscript_stub(context)
@@ -233,7 +236,7 @@ def post_process(pdf_file, context: PdfContext, executor: Executor):
     return optimize_pdf(pdf_out, context, executor)
 
 
-def worker_init(max_pixels: int):
+def worker_init(max_pixels: int) -> None:
     # In Windows, child process will not inherit our change to this value in
     # the parent process, so ensure workers get it set. Not needed when running
     # threaded, but harmless to set again.
@@ -241,7 +244,7 @@ def worker_init(max_pixels: int):
     pikepdf_enable_mmap()
 
 
-def exec_concurrent(context: PdfContext, executor: Executor):
+def exec_concurrent(context: PdfContext, executor: Executor) -> None:
     """Execute the pipeline concurrently"""
 
     # Run exec_page_sync on every page context
@@ -302,7 +305,9 @@ def exec_concurrent(context: PdfContext, executor: Executor):
         copy_final(pdf, options.output_file, context)
 
 
-def configure_debug_logging(log_filename: Path, prefix: str = ''):
+def configure_debug_logging(
+    log_filename: Path, prefix: str = ''
+) -> logging.FileHandler:
     """
     Create a debug log file at a specified location.
 
@@ -321,7 +326,12 @@ def configure_debug_logging(log_filename: Path, prefix: str = ''):
     return log_file_handler
 
 
-def run_pipeline(options, *, plugin_manager, api=False):
+def run_pipeline(
+    options: argparse.Namespace,
+    *,
+    plugin_manager: Optional[OcrmypdfPluginManager],
+    api: bool = False,
+) -> ExitCode:
     # Any changes to options will not take effect for options that are already
     # bound to function parameters in the pipeline. (For example
     # options.input_file, options.pdf_renderer are already bound.)
@@ -380,7 +390,7 @@ def run_pipeline(options, *, plugin_manager, api=False):
             hasattr(options.output_file, 'writable') and options.output_file.writable()
         ):
             log.info("Output written to stream")
-        elif samefile(options.output_file, os.devnull):
+        elif samefile(options.output_file, Path(os.devnull)):
             pass  # Say nothing when sending to dev null
         else:
             if options.output_type.startswith('pdfa'):
@@ -407,6 +417,7 @@ def run_pipeline(options, *, plugin_manager, api=False):
             log.error("KeyboardInterrupt")
         return ExitCode.ctrl_c
     except (ExitCodeException if not api else NeverRaise) as e:
+        e = cast(ExitCodeException, e)
         if options.verbose >= 1:
             log.exception("ExitCodeException")
         elif str(e):
@@ -421,6 +432,17 @@ def run_pipeline(options, *, plugin_manager, api=False):
             "image pixel limit."
         )
         return ExitCode.other_error
+    except (
+        BrokenProcessPool if not api else NeverRaise,
+        BrokenThreadPool if not api else NeverRaise,
+    ) as e:
+        log.exception(
+            "A worker process was terminated unexpectedly. This is known to occur if "
+            "processing your file takes all available swap space and RAM. It may "
+            "help to try again with a smaller number of jobs, using the --jobs "
+            "argument."
+        )
+        return ExitCode.child_process_error
     except (Exception if not api else NeverRaise):  # pylint: disable=broad-except
         log.exception("An exception occurred while executing the pipeline")
         return ExitCode.other_error

src/ocrmypdf/_validation.py

@@ -78,11 +78,11 @@ def check_options_output(options):
             "`--pdf-renderer auto` (the default) to avoid this issue."
         )
 
-    if options.output_type == 'none' and options.output_file != os.devnull:
+    if options.output_type == 'none' and options.output_file not in (os.devnull, '-'):
         raise BadArgsError(
-            "Since you specified `--pdf-renderer none`, the output file "
+            "Since you specified `--output-type none`, the output file "
             f"{options.output_file} cannot be produced. Set the output file to "
-            f"{os.devnull} to suppress this message."
+            f"`-` to suppress this message."
         )
 
     lossless_reconstruction = False

src/ocrmypdf/api.py

@@ -242,6 +242,7 @@ def ocr(  # pylint: disable=unused-argument
     tesseract_config: Iterable[str] = None,
     tesseract_pagesegmode: int = None,
     tesseract_oem: int = None,
+    tesseract_thresholding: int = None,
     pdf_renderer=None,
     tesseract_timeout: float = None,
     rotate_pages_threshold: float = None,

src/ocrmypdf/builtin_plugins/concurrency.py

@@ -19,6 +19,7 @@ import queue
 import signal
 import sys
 import threading
+from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor, as_completed
 from contextlib import suppress
 from multiprocessing.pool import Pool, ThreadPool
 from typing import Callable, Iterable, Type, Union
@@ -30,7 +31,7 @@ from ocrmypdf._logging import TqdmConsole
 from ocrmypdf.exceptions import InputFileError
 from ocrmypdf.helpers import remove_all_log_handlers
 
-ProcessPool = Pool
+FuturesExecutorClass = Union[Type[ThreadPoolExecutor], Type[ProcessPoolExecutor]]
 Queue = Union[multiprocessing.Queue, queue.Queue]
 UserInit = Callable[[], None]
 WorkerInit = Callable[[Queue, UserInit, int], None]
@@ -110,11 +111,11 @@ class StandardExecutor(Executor):
     ):
         if use_threads:
             log_queue: Queue = queue.Queue(-1)
-            pool_class: Type[Pool] = ThreadPool
+            executor_class: FuturesExecutorClass = ThreadPoolExecutor
             initializer: WorkerInit = thread_init
         else:
             log_queue = multiprocessing.Queue(-1)
-            pool_class = ProcessPool
+            executor_class = ProcessPoolExecutor
             initializer = process_init
 
         # Regardless of whether we use_threads for worker processes, the log_listener
@@ -123,39 +124,35 @@ class StandardExecutor(Executor):
         listener = threading.Thread(target=log_listener, args=(log_queue,))
         listener.start()
 
-        with self.pbar_class(**tqdm_kwargs) as pbar:
-            pool = pool_class(
-                processes=max_workers,
-                initializer=initializer,
-                initargs=(log_queue, worker_initializer, logging.getLogger("").level),
-            )
+        with self.pbar_class(**tqdm_kwargs) as pbar, executor_class(
+            max_workers=max_workers,
+            initializer=initializer,
+            initargs=(log_queue, worker_initializer, logging.getLogger("").level),
+        ) as executor:
+            futures = [executor.submit(task, args) for args in task_arguments]
             try:
-                results = pool.imap_unordered(task, task_arguments)
-                for result in results:
-                    if task_finished:
-                        task_finished(result, pbar)
-                    else:
-                        pbar.update()
+                for future in as_completed(futures):
+                    result = future.result()
+                    task_finished(result, pbar)
             except KeyboardInterrupt:
                 # Terminate pool so we exit instantly
-                pool.terminate()
-                # Don't try listener.join() here, will deadlock
+                executor.shutdown(wait=False, cancel_futures=True)
                 raise
             except Exception:
                 if not os.environ.get("PYTEST_CURRENT_TEST", ""):
-                    # Unless inside pytest, exit immediately because no one wants
-                    # to wait for child processes to finalize results that will be
-                    # thrown away. Inside pytest, we want child processes to exit
-                    # cleanly so that they output an error messages or coverage data
-                    # we need from them.
-                    pool.terminate()
+                    # Normally we shutdown without waiting for other child workers
+                    # on error, because there is no point in waiting for them. Their
+                    # results will be discard. But if the condition above is True,
+                    # then we are running in pytest, and we want everything to exit
+                    # as cleanly as possible so that we get good error messages.
+                    executor.shutdown(wait=False, cancel_futures=True)
                 raise
             finally:
                 # Terminate log listener
                 log_queue.put_nowait(None)
-                pool.close()
-                pool.join()
 
+        # When the above succeeds, wait for the listener thread to exit. (If
+        # an exception occurs, we don't try to join, in case it deadlocks.)
         listener.join()
 
 

src/ocrmypdf/builtin_plugins/ghostscript.py

@@ -18,13 +18,13 @@ log = logging.getLogger(__name__)
 
 @hookimpl
 def check_options(options):
-    gs_version = ghostscript.version()
     check_external_program(
         program='gs',
         package='ghostscript',
-        version_checker=gs_version,
+        version_checker=ghostscript.version,
         need_version='9.15',  # limited by Travis CI / Ubuntu 14.04 backports
     )
+    gs_version = ghostscript.version()
     if gs_version in ('9.24', '9.51'):
         raise MissingDependencyError(
             f"Ghostscript {gs_version} contains serious regressions and is not "

src/ocrmypdf/builtin_plugins/tesseract_ocr.py

@@ -10,7 +10,7 @@ import os
 
 from ocrmypdf import hookimpl
 from ocrmypdf._exec import tesseract
-from ocrmypdf.cli import numeric
+from ocrmypdf.cli import numeric, str_to_int
 from ocrmypdf.helpers import clamp
 from ocrmypdf.pluginspec import OcrEngine
 from ocrmypdf.subprocess import check_external_program
@@ -43,13 +43,27 @@ def add_options(parser):
         metavar='MODE',
         choices=range(0, 4),
         help=(
-            "Set Tesseract 4.0 OCR engine mode: "
+            "Set Tesseract 4.0+ OCR engine mode: "
             "0 - original Tesseract only; "
             "1 - neural nets LSTM only; "
             "2 - Tesseract + LSTM; "
             "3 - default."
         ),
     )
+    tess.add_argument(
+        '--tesseract-thresholding',
+        action='store',
+        type=str_to_int(tesseract.TESSERACT_THRESHOLDING_METHODS),
+        default='auto',
+        metavar='METHOD',
+        help=(
+            "Set Tesseract 5.0+ input image thresholding mode. This may improve OCR "
+            "results on low quality images or those that contain high constrast color. "
+            "legacy-otsu is the Tesseract default; adaptive-otsu is an improved Otsu "
+            "algorithm with improved sort for background color changes; sauvola is "
+            "based on local standard deviation."
+        ),
+    )
     tess.add_argument(
         '--tesseract-timeout',
         default=180.0,
@@ -89,8 +103,14 @@ def check_options(options):
 
     if not tesseract.has_user_words() and (options.user_words or options.user_patterns):
         log.warning(
-            "Tesseract 4.0 ignores --user-words and --user-patterns, so these "
-            "arguments have no effect."
+            "Tesseract 4.0 (which you have installed) ignores --user-words and "
+            "--user-patterns, so these arguments have no effect."
+        )
+    if not tesseract.has_thresholding() and options.tesseract_thresholding != 0:
+        log.warning(
+            "The installed version of Tesseract does not support changes to its "
+            "thresholding method. The --tesseract-threshold argument will be "
+            "ignored."
         )
     if options.tesseract_pagesegmode in (0, 2):
         log.warning(
@@ -162,6 +182,7 @@ class TesseractOcrEngine(OcrEngine):
             tessconfig=options.tesseract_config,
             timeout=options.tesseract_timeout,
             pagesegmode=options.tesseract_pagesegmode,
+            thresholding=options.tesseract_thresholding,
             user_words=options.user_words,
             user_patterns=options.user_patterns,
         )
@@ -177,6 +198,7 @@ class TesseractOcrEngine(OcrEngine):
             tessconfig=options.tesseract_config,
             timeout=options.tesseract_timeout,
             pagesegmode=options.tesseract_pagesegmode,
+            thresholding=options.tesseract_thresholding,
             user_words=options.user_words,
             user_patterns=options.user_patterns,
         )

src/ocrmypdf/cli.py

@@ -6,12 +6,12 @@
 
 
 import argparse
-from typing import Any, Callable, Optional, TypeVar
+from typing import Any, Callable, Mapping, Optional, TypeVar
 
 from ocrmypdf._version import PROGRAM_NAME as _PROGRAM_NAME
 from ocrmypdf._version import __version__ as _VERSION
 
-T = TypeVar('T')
+T = TypeVar('T', int, float)
 
 
 def numeric(
@@ -21,17 +21,32 @@ def numeric(
     min_ = basetype(min_) if min_ is not None else None
     max_ = basetype(max_) if max_ is not None else None
 
-    def _numeric(string):
-        value = basetype(string)
+    def _numeric(s: str) -> T:
+        value = basetype(s)
         if (min_ is not None and value < min_) or (max_ is not None and value > max_):
-            msg = f"{string!r} not in valid range {(min_, max_)!r}"
-            raise argparse.ArgumentTypeError(msg)
+            raise argparse.ArgumentTypeError(
+                f"{s!r} not in valid range {(min_, max_)!r}"
+            )
         return value
 
     _numeric.__name__ = basetype.__name__
     return _numeric
 
 
+def str_to_int(mapping: Mapping[str, int]):
+    """Accept text on command line and convert to integer."""
+
+    def _str_to_int(s: str) -> int:
+        try:
+            return mapping[s]
+        except KeyError:
+            raise argparse.ArgumentTypeError(
+                f"{s!r} must be one of: {', '.join(mapping.keys())}"
+            )
+
+    return _str_to_int
+
+
 class ArgumentParser(argparse.ArgumentParser):
     """Override parser's default behavior of calling sys.exit()
 
@@ -404,7 +419,7 @@ Online documentation is located at:
         metavar='MPixels',
         help="Set maximum number of pixels to unpack before treating an image as a "
         "decompression bomb",
-        default=128.0,
+        default=250.0,
     )
     advanced.add_argument(
         '--pdf-renderer',

src/ocrmypdf/optimize.py

@@ -411,7 +411,7 @@ def _optimize_jpeg(args: Tuple[Xref, Path, Path, int]) -> Tuple[Xref, Optional[P
     if opt_jpg.stat().st_size > in_jpg.stat().st_size:
         log.debug("xref %s, jpeg, made larger - skip", xref)
         opt_jpg.unlink()
-        opt_jpg = None
+        return xref, None
     return xref, opt_jpg
 
 

src/ocrmypdf/subprocess/__init__.py

@@ -13,12 +13,13 @@ import re
 import sys
 from collections.abc import Mapping
 from contextlib import suppress
-from distutils.version import LooseVersion, Version
 from functools import lru_cache
 from subprocess import PIPE, STDOUT, CalledProcessError, CompletedProcess, Popen
 from subprocess import run as subprocess_run
 from typing import Callable, Optional, Type, Union
 
+from packaging.version import Version
+
 from ocrmypdf.exceptions import MissingDependencyError
 
 # pylint: disable=logging-format-interpolation
@@ -266,11 +267,11 @@ def check_external_program(
     *,
     program: str,
     package: str,
-    version_checker: Union[str, Callable],
+    version_checker: Callable,
     need_version: str,
     required_for: Optional[str] = None,
     recommended=False,
-    version_parser: Type[Version] = LooseVersion,
+    version_parser: Type[Version] = Version,
 ):
     """Check for required version of external program and raise exception if not.
 
@@ -291,7 +292,7 @@ def check_external_program(
     try:
         if callable(version_checker):
             found_version = version_checker()
-        else:
+        else:  # deprecated
             found_version = version_checker
     except (CalledProcessError, FileNotFoundError, MissingDependencyError):
         _error_missing_program(program, package, required_for, recommended)

src/ocrmypdf/subprocess/_windows.py

@@ -8,7 +8,6 @@ import logging
 import os
 import shutil
 import sys
-from distutils.version import LooseVersion
 from itertools import chain
 from pathlib import Path
 from typing import Any, Callable, Iterable, Iterator, Set, Tuple, TypeVar
@@ -23,6 +22,17 @@ log = logging.getLogger(__name__)
 T = TypeVar('T')
 
 
+def ghostscript_version_key(s: str) -> Tuple[int, int, int]:
+    """Compare Ghostscript version numbers."""
+    try:
+        release = [int(elem) for elem in s.split('.', maxsplit=3)]
+        while len(release) < 3:
+            release.append(0)
+        return (release[0], release[1], release[2])
+    except ValueError:
+        return (0, 0, 0)
+
+
 def registry_enum(
     key: winreg.HKEYType, enum_fn: Callable[[winreg.HKEYType, int], T]
 ) -> Iterator[T]:
@@ -51,7 +61,9 @@ def registry_path_ghostscript(env=None) -> Iterator[Path]:
         with winreg.OpenKey(
             winreg.HKEY_LOCAL_MACHINE, r"SOFTWARE\Artifex\GPL Ghostscript"
         ) as k:
-            latest_gs = max(registry_subkeys(k), key=LooseVersion)
+            latest_gs = max(
+                registry_subkeys(k), key=ghostscript_version_key, default=(0, 0, 0)
+            )
         with winreg.OpenKey(
             winreg.HKEY_LOCAL_MACHINE, fr"SOFTWARE\Artifex\GPL Ghostscript\{latest_gs}"
         ) as k:

tests/cache/manifest.jsonl

@@ -1,77 +1,84 @@
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000003_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000003_pp_rm_bg.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000005_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000005_pp_rm_bg.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000004_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000004_pp_rm_bg.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000005_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000005_pp_rm_bg.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_pp_rm_bg.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000004_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000004_pp_rm_bg.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_pp_rm_bg.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000006_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000006_pp_rm_bg.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000006_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000006_pp_rm_bg.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/graph_ocred.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000004_ocr.png__000004_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000004_ocr.png", "$TMPDIR/000004_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000004_ocr.png__000004_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000004_ocr.png", "$TMPDIR/000004_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000004_ocr.png__000004_ocr_tess__pdf__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000004_ocr.png", "$TMPDIR/000004_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_tess__pdf__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000005_ocr.png__000005_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000005_ocr.png", "$TMPDIR/000005_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/skew.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000005_ocr.png__000005_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000005_ocr.png", "$TMPDIR/000005_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/ccitt.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000006_ocr.png__000006_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000006_ocr.png", "$TMPDIR/000006_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000005_ocr.png__000005_ocr_tess__pdf__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000005_ocr.png", "$TMPDIR/000005_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000006_ocr.png__000006_ocr_tess__pdf__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000006_ocr.png", "$TMPDIR/000006_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000006_ocr.png__000006_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000006_ocr.png", "$TMPDIR/000006_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/ccitt.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/skew.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__7__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/skew.pdf", "args": ["-l", "eng", "--psm", "7", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__7__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/skew.pdf", "args": ["-l", "eng", "--psm", "7", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_rasterize.png__stdout", "sourcefile": "resources/lichtenstein.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_rasterize.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/lichtenstein.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000002_ocr.png__000002_ocr_tess__pdf__txt", "sourcefile": "resources/3small.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000002_ocr.png", "$TMPDIR/000002_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_rasterize.png__stdout", "sourcefile": "resources/palette.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_rasterize.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/3small.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/lichtenstein.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_tess__pdf__txt", "sourcefile": "resources/3small.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_rasterize.png__stdout", "sourcefile": "resources/jbig2.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_rasterize.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/palette.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_rasterize.png__stdout", "sourcefile": "resources/ccitt.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_rasterize.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/palette.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/jbig2.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/aspect.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/aspect.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/ccitt.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/ccitt.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000002_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000002_rasterize_preview.jpg", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000004_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000004_rasterize_preview.jpg", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000001_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000001_rasterize_preview.jpg", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000003_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000003_rasterize_preview.jpg", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000001_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000001_rasterize_preview.jpg", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000003_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000003_rasterize_preview.jpg", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000004_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000004_rasterize_preview.jpg", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000002_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000002_rasterize_preview.jpg", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/jbig2.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/jbig2.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000002_ocr.png__000002_ocr_tess__pdf__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000002_ocr.png", "$TMPDIR/000002_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000004_ocr.png__000004_ocr_hocr__hocr__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "$TMPDIR/000004_ocr.png", "$TMPDIR/000004_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_tess__pdf__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000004_ocr.png__000004_ocr_tess__pdf__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000004_ocr.png", "$TMPDIR/000004_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_hocr__hocr__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000002_ocr.png__000002_ocr_hocr__hocr__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "$TMPDIR/000002_ocr.png", "$TMPDIR/000002_ocr_hocr", "hocr", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/2400dpi.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/poster.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000001_rasterize_preview.jpg__stdout", "sourcefile": "resources/poster.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000001_rasterize_preview.jpg", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000003_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000003_rasterize.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000005_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000005_rasterize.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000004_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000004_rasterize.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000005_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000005_rasterize.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000004_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000004_rasterize.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_rasterize.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_rasterize.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000006_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000006_rasterize.png", "stdout"]}
-{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "platform": "Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000006_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000006_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000003_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000003_pp_rm_bg.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000005_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000005_pp_rm_bg.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000004_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000004_pp_rm_bg.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000005_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000005_pp_rm_bg.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_pp_rm_bg.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000004_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000004_pp_rm_bg.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_pp_rm_bg.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000006_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000006_pp_rm_bg.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000006_pp_rm_bg.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000006_pp_rm_bg.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/graph_ocred.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000004_ocr.png__000004_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000004_ocr.png", "$TMPDIR/000004_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000004_ocr.png__000004_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000004_ocr.png", "$TMPDIR/000004_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000004_ocr.png__000004_ocr_tess__pdf__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000004_ocr.png", "$TMPDIR/000004_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_tess__pdf__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000005_ocr.png__000005_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000005_ocr.png", "$TMPDIR/000005_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/skew.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000005_ocr.png__000005_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000005_ocr.png", "$TMPDIR/000005_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/ccitt.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000006_ocr.png__000006_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000006_ocr.png", "$TMPDIR/000006_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000005_ocr.png__000005_ocr_tess__pdf__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000005_ocr.png", "$TMPDIR/000005_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000006_ocr.png__000006_ocr_tess__pdf__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000006_ocr.png", "$TMPDIR/000006_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000006_ocr.png__000006_ocr_hocr__hocr__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "$TMPDIR/000006_ocr.png", "$TMPDIR/000006_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/ccitt.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/skew.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__7__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/skew.pdf", "args": ["-l", "eng", "--psm", "7", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__7__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/skew.pdf", "args": ["-l", "eng", "--psm", "7", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_rasterize.png__stdout", "sourcefile": "resources/lichtenstein.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/lichtenstein.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000002_ocr.png__000002_ocr_tess__pdf__txt", "sourcefile": "resources/3small.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000002_ocr.png", "$TMPDIR/000002_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_rasterize.png__stdout", "sourcefile": "resources/palette.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/3small.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/lichtenstein.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_tess__pdf__txt", "sourcefile": "resources/3small.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_rasterize.png__stdout", "sourcefile": "resources/jbig2.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/palette.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_rasterize.png__stdout", "sourcefile": "resources/ccitt.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/palette.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/jbig2.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/aspect.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/aspect.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/ccitt.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/ccitt.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000002_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000002_rasterize_preview.jpg", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000004_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000004_rasterize_preview.jpg", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000001_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000001_rasterize_preview.jpg", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000003_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000003_rasterize_preview.jpg", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000001_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000001_rasterize_preview.jpg", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000003_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000003_rasterize_preview.jpg", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000004_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000004_rasterize_preview.jpg", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000002_rasterize_preview.jpg__stdout", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000002_rasterize_preview.jpg", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/jbig2.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/jbig2.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000002_ocr.png__000002_ocr_tess__pdf__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000002_ocr.png", "$TMPDIR/000002_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000004_ocr.png__000004_ocr_hocr__hocr__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "$TMPDIR/000004_ocr.png", "$TMPDIR/000004_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_tess__pdf__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000004_ocr.png__000004_ocr_tess__pdf__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000004_ocr.png", "$TMPDIR/000004_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_hocr__hocr__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000002_ocr.png__000002_ocr_hocr__hocr__txt", "sourcefile": "resources/cardinal.pdf", "args": ["-l", "eng", "$TMPDIR/000002_ocr.png", "$TMPDIR/000002_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/2400dpi.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/poster.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__osd__--psm__0__000001_rasterize_preview.jpg__stdout", "sourcefile": "resources/poster.pdf", "args": ["-l", "osd", "--psm", "0", "$TMPDIR/000001_rasterize_preview.jpg", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000003_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000003_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000005_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000005_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000004_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000004_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000005_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000005_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000004_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000004_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000001_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000001_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000006_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000006_rasterize.png", "stdout"]}
+{"tesseract_version": "5.0.0-beta-20210916-12-g19cc9", "system": "Linux", "python": "3.8.10", "argv_slug": "__-l__eng__--psm__2__000006_rasterize.png__stdout", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "--psm", "2", "$TMPDIR/000006_rasterize.png", "stdout"]}
+{"tesseract_version": "4.1.1", "system": "Linux", "python": "3.9.5", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/trivial.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "4.1.1", "system": "Linux", "python": "3.9.5", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/trivial.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "4.1.1", "system": "Linux", "python": "3.9.5", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/trivial.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "4.1.1", "system": "Linux", "python": "3.9.5", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/trivial.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "4.1.1", "system": "Linux", "python": "3.9.5", "argv_slug": "__-l__eng__--oem__1__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/trivial.pdf", "args": ["-l", "eng", "--oem", "1", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0", "system": "Linux", "python": "3.9.5", "argv_slug": "__-l__eng__thresholding_method=1__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/trivial.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "-c", "thresholding_method=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.0.0", "system": "Linux", "python": "3.9.5", "argv_slug": "__-l__eng__thresholding_method=2__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/trivial.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "-c", "thresholding_method=2", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}

tests/cache/trivial/__-l__eng__--oem__1__000001_ocr.png__000001_ocr_tess__pdf__txt/stderr.bin

@@ -0,0 +1 @@
+Tesseract Open Source OCR Engine v4.1.1 with Leptonica

tests/cache/trivial/__-l__eng__--oem__1__000001_ocr.png__000001_ocr_tess__pdf__txt/txt.bin

@@ -0,0 +1 @@
+

tests/cache/trivial/__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt/stderr.bin

@@ -0,0 +1 @@
+Tesseract Open Source OCR Engine v4.1.1 with Leptonica

tests/cache/trivial/__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt/txt.bin

@@ -0,0 +1 @@
+

tests/conftest.py

@@ -9,17 +9,15 @@ import os
 import platform
 import sys
 from pathlib import Path
-from subprocess import PIPE, run
+from subprocess import PIPE, CompletedProcess, run
+from typing import List
 
 import pytest
 
 from ocrmypdf import api, pdfinfo
 from ocrmypdf._exec import unpaper
 from ocrmypdf._plugin_manager import get_parser_options_plugins
-
-if sys.version_info < (3, 5):
-    print("Requires Python 3.5+")
-    sys.exit(1)
+from ocrmypdf.exceptions import ExitCode
 
 
 def is_linux():
@@ -46,36 +44,35 @@ def have_unpaper():
 
 TESTS_ROOT = Path(__file__).parent.resolve()
 PROJECT_ROOT = TESTS_ROOT
-OCRMYPDF = [sys.executable, '-m', 'ocrmypdf']
 
 
 @pytest.fixture
-def resources():
+def resources() -> Path:
     return Path(TESTS_ROOT) / 'resources'
 
 
 @pytest.fixture
-def ocrmypdf_exec():
-    return OCRMYPDF
+def ocrmypdf_exec() -> List[str]:
+    return [sys.executable, '-m', 'ocrmypdf']
 
 
 @pytest.fixture(scope="function")
-def outdir(tmp_path):
+def outdir(tmp_path) -> Path:
     return tmp_path
 
 
 @pytest.fixture(scope="function")
-def outpdf(tmp_path):
+def outpdf(tmp_path) -> Path:
     return tmp_path / 'out.pdf'
 
 
 @pytest.fixture(scope="function")
-def outtxt(tmp_path):
+def outtxt(tmp_path) -> Path:
     return tmp_path / 'out.txt'
 
 
 @pytest.fixture(scope="function")
-def no_outpdf(tmp_path):
+def no_outpdf(tmp_path) -> Path:
     """This just documents the fact that a test is not expected to produce
     output. Unfortunately an assertion failure inside a test fixture produces
     an error rather than a test failure, so no testing is done. It's up to
@@ -83,13 +80,13 @@ def no_outpdf(tmp_path):
     return tmp_path / 'no_output.pdf'
 
 
-def check_ocrmypdf(input_file, output_file, *args):
-    """Run ocrmypdf and confirmed that a valid file was created"""
-    args = [str(input_file), str(output_file)] + [
+def check_ocrmypdf(input_file: Path, output_file: Path, *args) -> Path:
+    """Run ocrmypdf and confirm that a valid plausible PDF was created."""
+    api_args = [str(input_file), str(output_file)] + [
         str(arg) for arg in args if arg is not None
     ]
 
-    _parser, options, plugin_manager = get_parser_options_plugins(args=args)
+    _parser, options, plugin_manager = get_parser_options_plugins(args=api_args)
     api.check_options(options, plugin_manager)
     result = api.run_pipeline(options, plugin_manager=plugin_manager, api=True)
 
@@ -100,44 +97,53 @@ def check_ocrmypdf(input_file, output_file, *args):
     return output_file
 
 
-def run_ocrmypdf_api(input_file, output_file, *args):
-    """Run ocrmypdf via API and let caller deal with results
+def run_ocrmypdf_api(input_file: Path, output_file: Path, *args) -> ExitCode:
+    """Run ocrmypdf via its API in-process, and let test deal with results.
 
-    Does not currently have a way to manipulate the PATH except for Tesseract.
+    This simulates calling the command line interface in a subprocess, but
+    is easier for debuggers and code coverage to follow.
+
+    Any exception raised will be trapped and converted to an exit code.
+    The return code must always be checked or the test may declare a failure
+    to be pass.
     """
 
-    args = [str(input_file), str(output_file)] + [
+    api_args = [str(input_file), str(output_file)] + [
         str(arg) for arg in args if arg is not None
     ]
-    _parser, options, plugin_manager = get_parser_options_plugins(args=args)
+    _parser, options, plugin_manager = get_parser_options_plugins(args=api_args)
 
     api.check_options(options, plugin_manager)
     return api.run_pipeline(options, plugin_manager=None, api=False)
 
 
-def run_ocrmypdf(input_file, output_file, *args, text=True):
-    "Run ocrmypdf and let caller deal with results"
+def run_ocrmypdf(
+    input_file: Path, output_file: Path, *args, text: bool = True
+) -> CompletedProcess:
+    """Run ocrmypdf in a subprocess and let test deal with results.
+
+    If an exception is thrown this fact will be returned as part of the result
+    text and return code rather than exception objects.
+    """
 
     p_args = (
-        OCRMYPDF
+        [sys.executable, '-m', 'ocrmypdf']
         + [str(arg) for arg in args if arg is not None]
         + [str(input_file), str(output_file)]
     )
 
-    env = os.environ.copy()
     p = run(
         p_args,
         stdout=PIPE,
         stderr=PIPE,
         text=text,
-        env=env,
         check=False,
     )
     # print(p.stderr)
-    return p, p.stdout, p.stderr
+    return p
 
 
-def first_page_dimensions(pdf):
+def first_page_dimensions(pdf: Path):
     info = pdfinfo.PdfInfo(pdf)
     page0 = info[0]
     return (page0.width_inches, page0.height_inches)

tests/plugins/tesseract_cache.py

@@ -158,7 +158,7 @@ def cached_run(options, run_args, **run_kwargs):
 
     manifest = {}
     manifest['tesseract_version'] = TesseractOcrEngine.version().replace('\n', ' ')
-    manifest['platform'] = platform.platform()
+    manifest['system'] = platform.system()
     manifest['python'] = platform.python_version()
     manifest['argv_slug'] = cache_folder.name
     manifest['sourcefile'] = str(Path(source_file).relative_to(TESTS_ROOT))

tests/plugins/tesseract_simulate_oom_killer.py

@@ -0,0 +1,83 @@
+# © 2021 James R. Barlow: github.com/jbarlow83
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so, subject to
+# the following conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+# CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+# type: ignore
+
+"""Tesseract no-op plugin that simulates the OOM killer on page 4.
+
+OCRmyPDF can use a lot of memory, even that it might trigger the
+OOM killer on Linux or similar features on other platforms. We want to
+ensure we fail with an error rather than deadlock in such cases.
+
+Page 4 was chosen because of this number's association with bad luck
+in many East Asian cultures.
+"""
+
+import os
+import signal
+import sys
+from pathlib import Path
+
+from ocrmypdf import hookimpl
+
+# Ugly hack that let us use the NoopOcrEngine without setting up packaging for our
+# tests.
+# This hack also requires us to set type: ignore
+parent_file = Path(__file__).with_name('tesseract_noop.py')
+parent = compile(parent_file.read_text(), parent_file, mode='exec')
+exec(parent)
+NoopOcrEngine = locals()['NoopOcrEngine']
+
+
+class Page4Engine(NoopOcrEngine):
+    def __str__(self):
+        return f"NO-OP Page 4 {NoopOcrEngine.version()}"
+
+    @staticmethod
+    def generate_hocr(input_file: Path, output_hocr, output_text, options):
+        if input_file.stem.startswith('000004'):
+            # Suicide
+            os.kill(os.getpid(), signal.SIGKILL)
+        else:
+            return NoopOcrEngine.generate_hocr(
+                input_file, output_hocr, output_text, options
+            )
+
+    @staticmethod
+    def generate_pdf(input_file, output_pdf, output_text, options):
+        if input_file.stem.startswith('000004'):
+            # Suicide
+            os.kill(os.getpid(), signal.SIGKILL)
+        else:
+            return NoopOcrEngine.generate_pdf(
+                input_file, output_pdf, output_text, options
+            )
+
+
+@hookimpl
+def check_options(options):
+    if options.use_threads:
+        raise ValueError("I'm not compatible with use_threads")
+
+
+@hookimpl
+def get_ocr_engine():
+    return Page4Engine()

tests/test_concurrency.py

@@ -0,0 +1,25 @@
+# © 2021 James R. Barlow: github.com/jbarlow83
+#
+# This Source Code Form is subject to the terms of the Mozilla Public
+# License, v. 2.0. If a copy of the MPL was not distributed with this
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+
+import os
+
+import pytest
+
+from ocrmypdf import ExitCode
+
+from .conftest import run_ocrmypdf_api
+
+
+@pytest.mark.skipif(os.name == 'nt', reason="Windows doesn't have SIGKILL")
+def test_simulate_oom_killer(resources, no_outpdf):
+    exitcode = run_ocrmypdf_api(
+        resources / 'multipage.pdf',
+        no_outpdf,
+        '--force-ocr',
+        '--plugin',
+        'tests/plugins/tesseract_simulate_oom_killer.py',
+    )
+    assert exitcode == ExitCode.child_process_error

tests/test_ghostscript.py

@@ -76,7 +76,7 @@ def test_rasterize_rotated(francais, outdir, caplog):
 
 
 def test_gs_render_failure(resources, outpdf):
-    p, _out, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'blank.pdf',
         outpdf,
         '--plugin',
@@ -84,12 +84,12 @@ def test_gs_render_failure(resources, outpdf):
         '--plugin',
         'tests/plugins/gs_render_failure.py',
     )
-    assert 'Casper is not a friendly ghost' in err
+    assert 'Casper is not a friendly ghost' in p.stderr
     assert p.returncode == ExitCode.child_process_error
 
 
 def test_gs_raster_failure(resources, outpdf):
-    p, _out, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'francais.pdf',
         outpdf,
         '--plugin',
@@ -97,12 +97,12 @@ def test_gs_raster_failure(resources, outpdf):
         '--plugin',
         'tests/plugins/gs_raster_failure.py',
     )
-    assert 'Ghost story archive not found' in err
+    assert 'Ghost story archive not found' in p.stderr
     assert p.returncode == ExitCode.child_process_error
 
 
 def test_ghostscript_pdfa_failure(resources, outpdf):
-    p, _out, _err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'francais.pdf',
         outpdf,
         '--plugin',

tests/test_main.py

@@ -203,7 +203,7 @@ def test_force_ocr_on_pdf_with_no_images(resources, no_outpdf):
     # As a correctness test, make sure that --force-ocr on a PDF with no
     # content still triggers tesseract. If tesseract crashes, then it was
     # called.
-    p, _, _ = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'blank.pdf',
         no_outpdf,
         '--force-ocr',
@@ -241,7 +241,7 @@ def test_german(resources, outdir):
 
 
 def test_klingon(resources, outpdf):
-    p, _, _ = run_ocrmypdf(resources / 'francais.pdf', outpdf, '-l', 'klz')
+    p = run_ocrmypdf(resources / 'francais.pdf', outpdf, '-l', 'klz')
     assert p.returncode == ExitCode.missing_dependency
 
 
@@ -316,9 +316,45 @@ def test_pagesegmode(renderer, resources, outpdf):
     )
 
 
+def test_tesseract_oem(resources, outpdf):
+    check_ocrmypdf(
+        resources / 'trivial.pdf',
+        outpdf,
+        '--tesseract-oem',
+        '1',
+        '--plugin',
+        'tests/plugins/tesseract_cache.py',
+    )
+
+
+@pytest.mark.parametrize('value', ['auto', 'otsu', 'adaptive-otsu', 'sauvola'])
+def test_tesseract_thresholding(value, resources, outpdf):
+    check_ocrmypdf(
+        resources / 'trivial.pdf',
+        outpdf,
+        '--tesseract-thresholding',
+        value,
+        '--plugin',
+        'tests/plugins/tesseract_cache.py',
+    )
+
+
+@pytest.mark.parametrize('value', ['abcxyz'])
+def test_tesseract_thresholding_invalid(value, resources, no_outpdf):
+    with pytest.raises(SystemExit, match='2'):
+        run_ocrmypdf_api(
+            resources / 'trivial.pdf',
+            no_outpdf,
+            '--tesseract-thresholding',
+            value,
+            '--plugin',
+            'tests/plugins/tesseract_cache.py',
+        )
+
+
 @pytest.mark.parametrize('renderer', RENDERERS)
 def test_tesseract_crash(renderer, resources, no_outpdf):
-    p, _, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'ccitt.pdf',
         no_outpdf,
         '-v',
@@ -330,11 +366,11 @@ def test_tesseract_crash(renderer, resources, no_outpdf):
     )
     assert p.returncode == ExitCode.child_process_error
     assert not no_outpdf.exists()
-    assert "SubprocessOutputError" in err
+    assert "SubprocessOutputError" in p.stderr
 
 
 def test_tesseract_crash_autorotate(resources, no_outpdf):
-    p, out, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'ccitt.pdf',
         no_outpdf,
         '-r',
@@ -343,9 +379,9 @@ def test_tesseract_crash_autorotate(resources, no_outpdf):
     )
     assert p.returncode == ExitCode.child_process_error
     assert not no_outpdf.exists()
-    assert "uncaught exception" in err
-    print(out)
-    print(err)
+    assert "uncaught exception" in p.stderr
+    print(p.stdout)
+    print(p.stderr)
 
 
 @pytest.mark.parametrize('renderer', RENDERERS)
@@ -365,7 +401,7 @@ def test_tesseract_image_too_big(renderer, resources, outpdf):
 
 
 def test_algo4(resources, outpdf):
-    p, _, _ = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'encrypted_algo4.pdf',
         outpdf,
         '--plugin',
@@ -435,7 +471,7 @@ def test_destination_not_writable(resources, outdir):
     protected_file = outdir / 'protected.pdf'
     protected_file.touch()
     protected_file.chmod(0o400)  # Read-only
-    p, _out, _err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'jbig2.pdf',
         protected_file,
         '--plugin',
@@ -476,7 +512,7 @@ THIS FILE IS INVALID
 '''
         )
 
-    p, _out, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'ccitt.pdf',
         outdir / 'out.pdf',
         '--pdf-renderer',
@@ -485,8 +521,8 @@ THIS FILE IS INVALID
         cfg_file,
     )
     assert (
-        "parameter not found" in err.lower()
-        or "error occurred while parsing" in err.lower()
+        "parameter not found" in p.stderr.lower()
+        or "error occurred while parsing" in p.stderr.lower()
     ), "No error message"
     assert p.returncode == ExitCode.invalid_config
 
@@ -745,15 +781,13 @@ def test_pdfa_n(pdfa_level, resources, outpdf):
 
 
 def test_decompression_bomb_error(resources, outpdf):
-    p, _out, err = run_ocrmypdf(resources / 'hugemono.pdf', outpdf)
-    assert 'decompression bomb' in err and '--max-image-mpixels' in err
+    p = run_ocrmypdf(resources / 'hugemono.pdf', outpdf)
+    assert 'decompression bomb' in p.stderr and '--max-image-mpixels' in p.stderr
 
 
 @pytest.mark.slow
 def test_decompression_bomb_succeeds(resources, outpdf):
-    p, _out, err = run_ocrmypdf(
-        resources / 'hugemono.pdf', outpdf, '--max-image-mpixels', '2000'
-    )
+    p = run_ocrmypdf(resources / 'hugemono.pdf', outpdf, '--max-image-mpixels', '2000')
     assert p.returncode == 0
 
 
@@ -782,7 +816,7 @@ def test_text_curves(resources, outpdf):
 
 
 def test_output_is_dir(resources, outdir):
-    p, _out, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'trivial.pdf',
         outdir,
         '--force-ocr',
@@ -790,28 +824,28 @@ def test_output_is_dir(resources, outdir):
         'tests/plugins/tesseract_noop.py',
     )
     assert p.returncode == ExitCode.file_access_error
-    assert 'is not a writable file' in err
+    assert 'is not a writable file' in p.stderr
 
 
 @pytest.mark.skipif(os.name == 'nt', reason="symlink needs admin permissions")
 def test_output_is_symlink(resources, outdir):
     sym = Path(outdir / 'this_is_a_symlink')
     sym.symlink_to(outdir / 'out.pdf')
-    p, _out, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'trivial.pdf',
         sym,
         '--force-ocr',
         '--plugin',
         'tests/plugins/tesseract_noop.py',
     )
-    assert p.returncode == ExitCode.ok, err
+    assert p.returncode == ExitCode.ok, p.stderr
     assert (outdir / 'out.pdf').stat().st_size > 0, 'target file not created'
 
 
 def test_livecycle(resources, no_outpdf):
-    p, _, err = run_ocrmypdf(resources / 'livecycle.pdf', no_outpdf)
+    p = run_ocrmypdf(resources / 'livecycle.pdf', no_outpdf)
 
-    assert p.returncode == ExitCode.input_file, err
+    assert p.returncode == ExitCode.input_file, p.stderr
 
 
 def test_version_check():
@@ -868,7 +902,7 @@ def test_image_dpi_not_image(caplog, resources, outpdf):
 
 
 def test_outputtype_none_bad_setup(resources, outpdf):
-    p, _out, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'trivial.pdf',
         outpdf,
         '--output-type=none',
@@ -876,13 +910,13 @@ def test_outputtype_none_bad_setup(resources, outpdf):
         'tests/plugins/tesseract_noop.py',
     )
     assert p.returncode == ExitCode.bad_args
-    assert 'Set the output file to' in err
+    assert 'Set the output file to' in p.stderr
 
 
 def test_outputtype_none(resources, outtxt):
-    p, _out, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'trivial.pdf',
-        os.devnull,
+        '-',
         '--output-type=none',
         '--sidecar',
         outtxt,
@@ -890,4 +924,3 @@ def test_outputtype_none(resources, outtxt):
         'tests/plugins/tesseract_noop.py',
     )
     assert p.returncode == ExitCode.ok
-    assert outtxt.exists()

tests/test_metadata.py

@@ -62,7 +62,7 @@ def test_override_metadata(output_type, resources, outpdf):
     german = 'Du siehst den Wald vor lauter Bäumen nicht.'
     chinese = '孔子'
 
-    p, _out, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         input_file,
         outpdf,
         '--title',
@@ -75,7 +75,7 @@ def test_override_metadata(output_type, resources, outpdf):
         'tests/plugins/tesseract_noop.py',
     )
 
-    assert p.returncode == ExitCode.ok, err
+    assert p.returncode == ExitCode.ok, p.stderr
 
     before = pikepdf.open(input_file)
     after = pikepdf.open(outpdf)
@@ -99,7 +99,7 @@ def test_high_unicode(resources, no_outpdf):
     input_file = resources / 'c02-22.pdf'
     high_unicode = 'U+1030C is: 𐌌'
 
-    p, _out, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         input_file,
         no_outpdf,
         '--subject',
@@ -110,7 +110,7 @@ def test_high_unicode(resources, no_outpdf):
         'tests/plugins/tesseract_noop.py',
     )
 
-    assert p.returncode == ExitCode.bad_args, err
+    assert p.returncode == ExitCode.bad_args, p.stderr
 
 
 @pytest.mark.skipif(not fitz, reason="test uses fitz")

tests/test_optimize.py

@@ -51,7 +51,7 @@ def test_mono_not_inverted(resources, outdir):
     )
 
     with Image.open(fspath(outdir / 'im.png')) as im:
-        assert im.getpixel((0, 0)) == 255, "Expected white background"
+        assert im.getpixel((0, 0)) > 240, "Expected white background"
 
 
 @needs_pngquant

tests/test_preprocessing.py

@@ -40,6 +40,11 @@ def test_deskew(resources, outdir):
     assert -0.5 < skew_angle < 0.5, "Deskewing failed"
 
 
+def test_deskew_blank_page(resources, outpdf):
+    # Tesseract doesn't like blank pages - make sure we can get through
+    check_ocrmypdf(resources / 'blank.pdf', outpdf, '--deskew')
+
+
 @pytest.mark.xfail(reason="remove background disabled")
 def test_remove_background(resources, outdir):
     # Ensure the input image does not contain pure white/black

tests/test_rotation.py

@@ -242,7 +242,7 @@ def test_rotate_page_level(image_angle, page_angle, resources, outdir):
     test = make_rotate_test('test', image_angle, page_angle)
     out = test.with_suffix('.out.pdf')
 
-    p, _, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         test,
         out,
         '-O0',
@@ -251,7 +251,7 @@ def test_rotate_page_level(image_angle, page_angle, resources, outdir):
         '0.001',
         text=False,
     )
-    err = err.decode('utf-8', errors='replace')
+    err = p.stderr.decode('utf-8', errors='replace')
     assert p.returncode == 0, err
 
     assert compare_images_monochrome(outdir, reference, 1, out, 1) > 0.2

tests/test_stdio.py

@@ -60,10 +60,10 @@ def test_stdout(ocrmypdf_exec, resources, outpdf):
 @pytest.mark.skipif(os.name == 'nt', reason="invalid test on Windows")
 def test_bad_locale(monkeypatch):
     monkeypatch.setenv('LC_ALL', 'C')
-    p, out, err = run_ocrmypdf('a', 'b')
-    assert out == '', "stdout not clean"
+    p = run_ocrmypdf('a', 'b')
+    assert p.stdout == '', "stdout not clean"
     assert p.returncode != 0
-    assert 'configured to use ASCII as encoding' in err, "should whine"
+    assert 'configured to use ASCII as encoding' in p.stderr, "should whine"
 
 
 @pytest.mark.xfail(
@@ -74,7 +74,7 @@ def test_dev_null(resources):
     if 'COV_CORE_DATAFILE' in os.environ:
         pytest.skip(msg="Coverage uses stdout")
 
-    p, out, _err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / 'trivial.pdf',
         os.devnull,
         '--force-ocr',
@@ -82,4 +82,4 @@ def test_dev_null(resources):
         'tests/plugins/tesseract_noop.py',
     )
     assert p.returncode == 0, "could not send output to /dev/null"
-    assert len(out) == 0, "wrote to stdout"
+    assert len(p.stdout) == 0, "wrote to stdout"

tests/test_tesseract.py

@@ -82,6 +82,7 @@ def test_image_too_large_hocr(monkeypatch, resources, outdir):
         tessconfig=[],
         timeout=180.0,
         pagesegmode=None,
+        thresholding=0,
         user_words=None,
         user_patterns=None,
     )
@@ -102,6 +103,7 @@ def test_image_too_large_pdf(monkeypatch, resources, outdir):
         tessconfig=[],
         timeout=180.0,
         pagesegmode=None,
+        thresholding=0,
         user_words=None,
         user_patterns=None,
     )

tests/test_unpaper.py

@@ -71,7 +71,7 @@ def test_unpaper_args_valid(resources, outpdf):
 
 @pytest.mark.skipif(not have_unpaper(), reason="requires unpaper")
 def test_unpaper_args_invalid_filename(resources, outpdf):
-    p, _out, err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / "skew.pdf",
         outpdf,
         "-c",
@@ -80,13 +80,13 @@ def test_unpaper_args_invalid_filename(resources, outpdf):
         '--plugin',
         'tests/plugins/tesseract_noop.py',
     )
-    assert "No filenames allowed" in err
+    assert "No filenames allowed" in p.stderr
     assert p.returncode == ExitCode.bad_args
 
 
 @pytest.mark.skipif(not have_unpaper(), reason="requires unpaper")
 def test_unpaper_args_invalid(resources, outpdf):
-    p, _out, _err = run_ocrmypdf(
+    p = run_ocrmypdf(
         resources / "skew.pdf",
         outpdf,
         "-c",

tests/test_validation.py

@@ -103,13 +103,19 @@ def test_user_words(caplog):
         opts = make_opts(user_words='foo')
         plugin_manager = get_plugin_manager(opts.plugins)
         vd._check_options(opts, plugin_manager, set())
-        assert '4.0 ignores --user-words' in caplog.text
+        assert (
+            'Tesseract 4.0 (which you have installed) ignores --user-words'
+            in caplog.text
+        )
     caplog.clear()
     with patch('ocrmypdf._exec.tesseract.has_user_words', return_value=True):
         opts = make_opts(user_patterns='foo')
         plugin_manager = get_plugin_manager(opts.plugins)
         vd._check_options(opts, plugin_manager, set())
-        assert '4.0 ignores --user-words' not in caplog.text
+        assert (
+            'Tesseract 4.0 (which you have installed) ignores --user-words'
+            not in caplog.text
+        )
 
 
 def test_pillow_options():