Skip testing oom killer on Python 3.12

Need to investigate further if there's a safe way to do this test.

.github/ISSUE_TEMPLATE/1-bug-report-general.yml

@@ -1,4 +1,4 @@
-name: General issues
+name: Installation, packaging, dependencies
 description: Installation, packages, dependencies, "nothing works", test suite failures...
 title: "[Bug]: "
 labels: ["bug", "triage"]
@@ -9,6 +9,10 @@ body:
     attributes:
       value: |
         Thanks for taking the time to fill out this bug report!
+
+        If your issue involves using OCRmyPDF on specific file(s) and not getting
+        good results, this is the *wrong* issue template. Please use the recommended
+        template to ensure we have enough information to help.
   - type: textarea
     id: what-happened
     attributes:

.github/ISSUE_TEMPLATE/2-problem-with-specific-file.yml

@@ -32,7 +32,20 @@ body:
     attributes:
       label: Files
       description: Please attach the input and output files, or any screenshots that may be helpful.
-      placeholder: Drag and drop files here
+      placeholder: |
+        Drag and drop files here.
+
+        If you cannot provide a test file, we probably won't be able to help with the issue.
+        PDF is a complex file format, and there may be technical details in the PDF that are
+        causing the issue. There's really no substitute for a test file.
+
+        We understand files may contain personal or sensitive information. Here are some options:
+        - Try reproducing the issue with a file from the test suite. (See tests/resources)
+        - Try to create another file in the same way as your private file.
+        - Encrypt the file to OCRmyPDF's private GPG key.
+        - Use ``qpdf --json yourfile.pdf`` to produce a JSON representation of your file that
+          omits personal information.
+
   - type: dropdown
     id: packaging-system
     attributes:

.github/workflows/build.yml

@@ -225,8 +225,9 @@ jobs:
           python -m pip install --upgrade pip wheel build
           python -m build --sdist --wheel
 
-      - uses: actions/upload-artifact@v3
+      - uses: actions/upload-artifact@v4
         with:
+          name: artifact
           path: |
             ./dist/*.whl
             ./dist/*.tar.gz
@@ -240,7 +241,7 @@ jobs:
       id-token: write  # mandatory for PyPI publishing
     if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/v')
     steps:
-      - uses: actions/download-artifact@v3
+      - uses: actions/download-artifact@v4
         with:
           name: artifact
           path: dist
@@ -257,7 +258,7 @@ jobs:
       # Required to create a release
       contents: write
     steps:
-      - uses: actions/download-artifact@v3
+      - uses: actions/download-artifact@v4
         with:
           name: artifact
           path: dist

docs/installation.rst

@@ -404,7 +404,7 @@ the following when running in an Administrator command prompt):
 * ``choco install --pre tesseract``
 * ``choco install pngquant`` (optional)
 
-Either set of commands will install the required software. At the mmoment there is no
+Either set of commands will install the required software. At the moment there is no
 single command to install Windows.
 
 You may then use ``pip`` to install ocrmypdf. (This can performed by a user or
@@ -681,4 +681,4 @@ can still install and use OCRmyPDF. A warning message will appear.
 In practice, OCRmyPDF may need more than 32-bit memory space to run when
 large documents are processed, so there are practical limitations to what
 users can accomplish with it. Still, for the common use case of an 32-bit
-ARM NAS or Raspberry Pi processing small documents, it should work.
+ARM NAS or Raspberry Pi processing small documents, it should work.

docs/languages.rst

@@ -18,16 +18,26 @@ Tesseract's documentation also lists the three-letter code for your language.
 Some are anglicized, e.g. Spanish is ``spa`` rather than ``esp``, while others
 are not, e.g. German is ``deu`` and French is ``fra``.
 
+Language packs (strictly speaking, Tesseract "traineddata" files) generally correspond
+to the language in question, but different language packs are used in certain
+situations. For German, the "Fraktur" language pack can assist with reading older
+materials in the Fraktur typeface family (``deu_frak``). Some communities have changed
+their script from Cyrillic to Latin; the Cyrillic version of Uzbek is available
+as ``uzb_cyrl`` and the Latin version is ``uzb``.
+
 After you have installed a language pack, you can use it with ``ocrmypdf -l <language>``,
 for example ``ocrmypdf -l spa``. For multilingual documents, you can specify
 all languages to be expected, e.g. ``ocrmypdf -l eng+fra`` for English and French.
 English is assumed by default unless other language(s) are specified.
 
 For Linux users, you can often find packages that provide language
-packs:
+packs.
 
-Debian and Ubuntu users
-=======================
+Platform install steps
+======================
+
+Debian and Ubuntu (apt)
+-----------------------
 
 .. code-block:: bash
 
@@ -42,8 +52,8 @@ to what languages it should search for. Multiple languages can be
 requested using either ``-l eng+fra`` (English and French) or
 ``-l eng -l fra``.
 
-Fedora users
-============
+Fedora
+------
 
 .. code-block:: bash
 
@@ -58,8 +68,8 @@ to what languages it should search for. Multiple languages can be
 requested using either ``-l eng+fra`` (English and French) or
 ``-l eng -l fra``.
 
-Gentoo users
-============
+Gentoo
+------
 
 On Gentoo the package ``app-text/tessdata_fast``, which ``app-text/tesseract`` depends on, handles Tesseract languages.
 It accepts USE flags to select what languages should be installed, these can be set in ``/etc/portage/package.use``.
@@ -85,23 +95,31 @@ to what languages it should search for. Multiple languages can be
 requested using either ``-l eng+fra`` (English and French) or
 ``-l eng -l fra``.
 
-macOS users
-===========
+macOS
+-----
 
 You can install additional language packs by
 :ref:`installing Tesseract using Homebrew with all language packs <macos-all-languages>`.
 
-Docker users
-============
+Docker
+------
 
 Users of the OCRmyPDF Docker image should install language packs into a
 derived Docker image as
 :ref:`described in that section <docker-lang-packs>`.
 
-Windows users
-=============
+Windows
+-------
 
 The Tesseract installer provided by Chocolatey currently includes only English language.
 To install other languages, download the respective language pack (``.traineddata`` file)
 from https://github.com/tesseract-ocr/tessdata/ and place it in
 ``C:\\Program Files\\Tesseract-OCR\\tessdata`` (or wherever Tesseract OCR is installed).
+
+Custom language packs
+=====================
+
+If you have fine-tuned or trained Tesseract and generated custom trained data, you can
+copy your ``customlang.traineddata`` file into your Tesseract "tessdata" folder, and
+then use the ``-l customlang`` argument to tell OCRmyPDF to pass that language on to
+Tesseract.

docs/release_notes.rst

@@ -30,8 +30,30 @@ OCRmyPDF typically supports the three most recent Python versions.
 
 .. |OCRmyPDF PyPI| image:: https://img.shields.io/pypi/v/ocrmypdf.svg
 
-v16.0.0rc1
-==========
+
+v16.0.3
+=======
+
+-  Changed minimum required Ghostscript to 9.54, to support users of RHEL 9 and its
+   derivatives, since that is the latest version available there.
+-  Removed warning message about CVE-2023-43115, on the assumption that most
+   distributions have backported the patch by now.
+
+v16.0.2
+=======
+
+-  Temporarily changed PDF text renderer back to sandwich by default to address
+   regressions in macOS Preview.
+
+v16.0.1
+=======
+
+-  Fixed text rendering issue with new hOCR text renderer - extraneous byte order
+   marks.
+-  Tightened dependencies.
+
+v16.0.0
+=======
 
 -  Added OCR text renderer, combined the best ideas of Tesseract's PDF
    generator and the older hOCR transformer renderer. The result is a hopefully

pyproject.toml

@@ -14,11 +14,11 @@ requires-python = ">=3.10"
 dependencies = [
   "Pillow>=10.0.1",
   "deprecation>=2.1.0",
-  "img2pdf>=0.4.4",
+  "img2pdf>=0.5",
   "packaging>=20",
   "pdfminer.six>=20220319",
-  "pikepdf>=8.8.0",
-  "pluggy>=0.13.0",
+  "pikepdf>=8.10.1",
+  "pluggy>=1",
   "rich>=13",
 ]
 authors = [{ name = "James R. Barlow", email = "james@purplerock.ca" }]

src/ocrmypdf/builtin_plugins/ghostscript.py

@@ -54,7 +54,7 @@ def check_options(options):
         program='gs',
         package='ghostscript',
         version_checker=ghostscript.version,
-        need_version='9.55',  # Ubuntu 22.04's version
+        need_version='9.54',  # RHEL 9's version; Ubuntu 22.04 has 9.55
     )
     gs_version = ghostscript.version()
     if gs_version in BLACKLISTED_GS_VERSIONS:
@@ -62,14 +62,6 @@ def check_options(options):
             f"Ghostscript {gs_version} contains serious regressions and is not "
             "supported. Please upgrade to a newer version."
         )
-    if gs_version < Version('10.02.0'):
-        log.warning(
-            f"The installed version of Ghostscript {gs_version}, contains a remote "
-            "code execution security vulnerability. Please upgrade to a newer "
-            "version. For details see CVE-2023-43115. The issue is not known to "
-            "affect OCRmyPDF or processing PDFs with Ghostscript, but upgrading "
-            "Ghostscript is recommended."
-        )
 
     if options.output_type == 'pdfa':
         options.output_type = 'pdfa-2'

src/ocrmypdf/builtin_plugins/tesseract_ocr.py

@@ -146,7 +146,7 @@ def check_options(options):
 
     # Decide on what renderer to use
     if options.pdf_renderer == 'auto':
-        options.pdf_renderer = 'hocr'
+        options.pdf_renderer = 'sandwich'
 
     if not tesseract.has_thresholding() and options.tesseract_thresholding != 0:
         log.warning(

src/ocrmypdf/hocrtransform/_hocr.py

@@ -106,10 +106,10 @@ class HocrTransform:
         self.render_options = DebugRenderOptions(
             render_baseline=debug,
             render_triangle=debug,
-            render_line_bbox=debug,
+            render_line_bbox=False,
             render_word_bbox=debug,
-            render_paragraph_bbox=debug,
-            render_space_bbox=debug,
+            render_paragraph_bbox=False,
+            render_space_bbox=False,
         )
 
     def _get_element_text(self, element: Element):
@@ -365,7 +365,7 @@ class HocrTransform:
         if font_width > 0:
             text.text_transform(Matrix(1, 0, 0, 1, box.llx, 0))
             text.horiz_scale(100 * box.width / font_width)
-            text.show(elemtxt)
+            text.show(elemtxt.encode('utf-16be'))
 
         # Get coordinates of the next word (if there is one)
         hocr_next_box = (
@@ -389,7 +389,7 @@ class HocrTransform:
         space_width = self._font.text_width(' ', fontsize)
         if space_width > 0:
             text.horiz_scale(100 * space_box.width / space_width)
-            text.show(' ')
+            text.show(' '.encode('utf-16be'))
 
     def _debug_draw_paragraph_boxes(self, canvas: Canvas, color=CYAN):
         """Draw boxes around paragraphs in the document."""

tests/test_concurrency.py

@@ -4,6 +4,7 @@
 from __future__ import annotations
 
 import os
+import platform
 
 import pytest
 
@@ -13,6 +14,9 @@ from .conftest import run_ocrmypdf_api
 
 
 @pytest.mark.skipif(os.name == 'nt', reason="Windows doesn't have SIGKILL")
+@pytest.mark.skipif(
+    platform.python_version_tuple() >= ('3', '12'), reason="can deadlock due to fork"
+)
 def test_simulate_oom_killer(multipage, no_outpdf):
     exitcode = run_ocrmypdf_api(
         multipage,