v12.0.2 release notes

Closes Error when trying --remove-background on pdf #769

docs/docker.rst

@@ -103,16 +103,41 @@ Adding languages to the Docker image
 By default the Docker image includes English, German, Simplified Chinese,
 French, Portuguese and Spanish, the most popular languages for OCRmyPDF
 users based on feedback. You may add other languages by creating a new
-Dockerfile based on the public one:
+Dockerfile based on the public one.
 
 .. code-block:: dockerfile
 
    FROM jbarlow83/ocrmypdf
 
-   # Add French
-   RUN apt install tesseract-ocr-fra
+   # Example: add Italian
+   RUN apt install tesseract-ocr-ita
 
-You can also copy training data to ``/usr/share/tesseract-ocr/<tesseract version>/tessdata``.
+To install language packs (training data) such as the 
+`tessdata_best <https://github.com/tesseract-ocr/tessdata_best>`_ suite or
+custom data, you first need to determine the version of Tesseract data files, which
+may differ from the Tesseract program version. Use this command to determine the data
+file version:
+
+.. code-block:: bash
+
+   docker run -i --rm --entrypoint /bin/ls jbarlow83/ocrmypdf /usr/share/tesseract-ocr
+
+As of 2021, the data file version is probably ``4.00``.
+
+You can then add new data with either a Dockerfile:
+
+.. code-block:: dockerfile
+
+   FROM jbarlow83/ocrmypdf
+
+   # Example: add a tessdata_best file
+   COPY chi_tra_vert.traineddata /usr/share/tesseract-ocr/<data version>/tessdata/
+
+Alternately, you can copy training data into a Docker container as follows:
+
+.. code-block:: bash
+
+   docker cp mycustomtraining.traineddata name_of_container:/usr/share/tesseract-ocr/<tesseract version>/tessdata/
 
 Executing the test suite
 ========================

docs/languages.rst

@@ -12,9 +12,9 @@ languages <https://github.com/tesseract-ocr/tesseract/blob/master/doc/tesseract.
 Languages are identified by standardized three-letter codes (called ISO 639-2 Alpha-3).
 Tesseract's documentation also lists the three-letter code for your language.
 Some are anglicized, e.g. Spanish is ``spa`` rather than ``esp``, while others
-are not, e.g. German is ``deu``.
+are not, e.g. German is ``deu`` and French is ``fra``.
 
-After you have installed a language pack, you can use it ``ocrmypdf -l <language>``,
+After you have installed a language pack, you can use it with ``ocrmypdf -l <language>``,
 for example ``ocrmypdf -l spa``. For multilingual documents, you can specify
 all languages to be expected, e.g. ``ocrmypdf -l eng+fra`` for English and French.
 English is assumed by default unless other language(s) are specified.
@@ -35,8 +35,8 @@ Debian and Ubuntu users
 
 You can then pass the ``-l LANG`` argument to OCRmyPDF to give a hint as
 to what languages it should search for. Multiple languages can be
-requested using either ``-l eng+fre`` (English and French) or
-``-l eng -l fre``.
+requested using either ``-l eng+fra`` (English and French) or
+``-l eng -l fra``.
 
 Fedora users
 ============
@@ -51,8 +51,8 @@ Fedora users
 
 You can then pass the ``-l LANG`` argument to OCRmyPDF to give a hint as
 to what languages it should search for. Multiple languages can be
-requested using either ``-l eng+fre`` (English and French) or
-``-l eng -l fre``.
+requested using either ``-l eng+fra`` (English and French) or
+``-l eng -l fra``.
 
 macOS users
 ===========

docs/release_notes.rst

@@ -12,6 +12,20 @@ may be unreliable. Use the API to depend on precise behavior.
 The public API may be useful in scripts that launch OCRmyPDF processes or that
 wish to use some of its features for working with PDFs.
 
+
+v12.0.2
+=======
+
+-  Fix exception thrown when using ``--remove-background`` on files containing small
+   images (#769).
+-  Improve documentation for description of adding language packs to the Docker image
+   and corrected name of French language pack.  
+
+v12.0.1
+=======
+
+-  Fix "invalid version number" for untagged tesseract versions (#770).
+
 v12.0.0
 =======
 

src/ocrmypdf/_exec/tesseract.py

@@ -61,6 +61,7 @@ class TesseractVersion(StrictVersion):
         ^(\d+) \. (\d+) (\. (\d+))?  # groups: 1/major, 2/minor, 3/[skip], 4/patch
         [-]?  # optional hyphen separator
         (?:(alpha|beta|rc|dev)[.\-\ ]?(\d+)?)?  # 5/prerelease, 6/prerelease_num
+        (?:-(\d+)-g[0-9a-f]+)?  # untagged git version
         $
         ''',
         re.VERBOSE | re.ASCII,

src/ocrmypdf/leptonica.py

@@ -192,11 +192,17 @@ class _LeptonicaErrorTrap_Queue:
         if 'Error' in output:
             if 'image file not found' in output:
                 raise FileNotFoundError()
-            if 'pixWrite: stream not opened' in output:
+            elif 'pixWrite: stream not opened' in output:
                 raise LeptonicaIOError()
-            if 'index not valid' in output:
+            elif 'index not valid' in output:
                 raise IndexError()
-            raise LeptonicaError(output)
+            elif 'pixGetInvBackgroundMap: w and h must be >= 5' in output:
+                logger.warning(
+                    "Leptonica attempted to remove background from a low resolution - "
+                    "you may want to review in a PDF viewer"
+                )
+            else:
+                raise LeptonicaError(output)
         return False
 
 
@@ -656,6 +662,9 @@ class Pix(LeptonicaObject):
         bg_val=200,
         smooth_kernel=(2, 1),
     ):
+        if self.width < tile_size[0] or self.height < tile_size[1]:
+            logger.info("Skipped pixMaskedThreshOnBackgroundNorm on small image")
+            return self
         # Background norm doesn't work on color mapped Pix, so remove colormap
         target_pix = self.remove_colormap(lept.REMOVE_CMAP_BASED_ON_SRC)
         with _LeptonicaErrorTrap():

tests/test_validation.py

@@ -237,6 +237,13 @@ def test_version_comparison():
         need_version='4.0.0',
         version_parser=TesseractVersion,
     )
+    vd.check_external_program(
+        program="tesseract",
+        package="tesseract",
+        version_checker=lambda: '4.1.1-rc2-25-g9707',
+        need_version='4.0.0',
+        version_parser=TesseractVersion,
+    )
     with pytest.raises(MissingDependencyError):
         vd.check_external_program(
             program="dummy_fails",