v16.4.2 release notes

ghostscript: change input filename order for 10.03.1
Ghostscript now expects the pdfa.ps file to precede other files. Fixes #1359.
2024-07-22 15:02:53 -07:00 · 2024-07-22 14:56:03 -07:00 · 2024-07-22 14:54:40 -07:00 · 2024-07-18 06:28:07 -07:00 · 2024-07-16 13:36:48 -07:00 · 2024-07-15 13:28:36 -07:00
116 changed files with 1392 additions and 396 deletions
@@ -64,6 +64,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
  img2pdf \
  libsm6 libxext6 libxrender-dev \
  pngquant \
+  python-is-python3 \
  tesseract-ocr \
  tesseract-ocr-chi-sim \
  tesseract-ocr-deu \
@@ -1,7 +1,7 @@
 # SPDX-FileCopyrightText: 2023 James R. Barlow
 # SPDX-License-Identifier: MPL-2.0

-FROM alpine:3.18 as base
+FROM alpine:3.19.1 as base

 ENV LANG=C.UTF-8
 ENV TZ=UTC
@@ -1,7 +1,7 @@
-name: General issues
+name: Installation, packaging, dependencies
 description: Installation, packages, dependencies, "nothing works", test suite failures...
 title: "[Bug]: "
-labels: ["bug", "triage"]
+labels: ["triage"]
 assignees:
  - jbarlow83
 body:
@@ -9,6 +9,10 @@ body:
    attributes:
      value: |
        Thanks for taking the time to fill out this bug report!
+
+        If your issue involves using OCRmyPDF on specific file(s) and not getting
+        good results, this is the *wrong* issue template. Please use the recommended
+        template to ensure we have enough information to help.
  - type: textarea
    id: what-happened
    attributes:
@@ -20,7 +24,7 @@ body:
  - type: dropdown
    id: packaging-system
    attributes:
-      label: Where are you installing from?
+      label: Where are you installing/running from?
      multiple: true
      options:
        - PyPI (pip, poetry, pipx, etc.)
@@ -33,6 +37,11 @@ body:
        - source build
    validations:
      required: true
+  - type: input
+    id: version
+    attributes:
+      label: OCRmyPDF version
+      description: Paste "ocrmypdf --version" here
  - type: dropdown
    id: operating-system
    attributes:
@@ -43,6 +52,18 @@ body:
        - Windows
        - macOS
        - BSD
+  - type: input
+    id: os_version
+    attributes:
+      label: Operating system details and version
+  - type: checkboxes
+    attributes:
+      label: Simple sanity checks
+      description: Select all that apply
+      options:
+        - label: Operating system is currently supported by its vendor (not end of life)
+        - label: Python version is compatible with OCRmyPDF
+        - label: This issue is not about a specific input file
  - type: textarea
    id: logs
    attributes:
@@ -1,7 +1,7 @@
 name: Problem with specific file
 description: Something went wrong while trying to OCR a specific file
 title: "[Bug]: "
-labels: ["bug", "triage"]
+labels: ["triage"]
 assignees:
  - jbarlow83
 body:
@@ -21,7 +21,7 @@ body:
    id: reproduce
    attributes:
      label: Steps to reproduce
-      description: Please include steps to reproduce
+      description: Please include steps to reproduce.
      value: |
        1. Run ocrmypdf -v1 ...arguments... input.pdf output.pdf
        2. Open output.pdf
@@ -31,8 +31,21 @@ body:
    id: files
    attributes:
      label: Files
-      description: Please attach the input and output files, or any screenshots that may be helpful.
-      placeholder: Drag and drop files here
+      description: |
+        Please attach the input and output files, or any screenshots that may be helpful.
+
+        If you cannot provide a test file, we probably won't be able to help with the issue.
+        PDF is a complex file format, and there may be technical details in the PDF that are
+        causing the issue. There's really no substitute for a test file.
+
+        We understand files may contain personal or sensitive information. Here are some options:
+        - Try reproducing the issue with a file from the OCRmyPDF test suite. (See tests/resources)
+        - Try to create another file in the same way as your private file.
+        - Encrypt the file to OCRmyPDF's private GPG key, and then zip the GPG file.
+        - Use ``qpdf --json yourfile.pdf`` to produce a JSON representation of your file that
+          omits personal information.
+      placeholder: |
+        Drag and drop files here.
  - type: dropdown
    id: packaging-system
    attributes:
@@ -0,0 +1,83 @@
+name: Problem with third party app that uses OCRmyPDF
+description: |
+  For PDF generation issues with third party software such as Paperless-ngx that
+  uses OCRmyPDF to perform OCR or generate PDFs.
+title: "[3rdparty]: "
+labels: ["triage"]
+assignees:
+  - jbarlow83
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for taking the time to describe this issue with a particular file
+        and third party app.
+
+        If you are comfortable using OCRmyPDF, please trying to install OCRmyPDF,
+        run it on your file, and see if it works. It's easier for everyone
+        if you can confirm that the issue occurs with OCRmyPDF and not with
+        the third party app.
+  - type: checkboxes
+    attributes:
+      label: Simple sanity checks
+      description: Select all that apply
+      options:
+        - label: This is an issue with an app that uses OCRmyPDF for OCR
+        - label: I am using a recent version of the third party app
+        - label: I will include a file that reproduces the issuse
+  - type: input
+    id: thirdparty-app-name-version
+    attributes:
+      label: Third party app name and version
+      description: e.g. Paperless-ngx 2.9.0
+  - type: textarea
+    id: what-happened
+    attributes:
+      label: Describe the bug
+      description: A clear and concise description of what the bug is.
+      placeholder: Tell us what you see!
+    validations:
+      required: true
+  - type: textarea
+    id: reproduce
+    attributes:
+      label: Steps to reproduce
+      description: Please include steps to reproduce.
+      value: |
+        1. Import attached file into Paperless-ngx
+        2. Trigger OCR
+        3. Check log file
+        4. ...
+      render: plain text
+  - type: textarea
+    id: files
+    attributes:
+      label: Files
+      description: |
+        Please attach the input and output files, or any screenshots that may be helpful.
+
+        If you cannot provide a test file, we probably won't be able to help with the issue.
+        PDF is a complex file format, and there may be technical details in the PDF that are
+        causing the issue. There's really no substitute for a test file.
+
+        We understand files may contain personal or sensitive information. Here are some options:
+        - Try reproducing the issue with a file from the test suite. (See tests/resources)
+        - Try to create another file in the same way as your private file.
+        - Encrypt the file to OCRmyPDF's private GPG key, and then zip the GPG file.
+        - Use ``qpdf --json yourfile.pdf`` to produce a JSON representation of your file that
+          omits personal information.
+      placeholder: |
+        Drag and drop files here.
+  - type: input
+    id: version
+    attributes:
+      label: OCRmyPDF version
+      description: Paste "ocrmypdf --version" here
+      placeholder: ocrmypdf --version
+  - type: textarea
+    id: logs
+    attributes:
+      label: Relevant log output
+      description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
+      placeholder: Run OCRmyPDF with verbosity `-v1` to get more detailed logging output.
+      render: plain text
@@ -32,8 +32,8 @@ jobs:
          - os: ubuntu-latest
            python: "3.12"
            tesseract5: true
-          # - os: ubuntu-latest
-          #   python: "pypy3.10"
+          - os: ubuntu-latest
+            python: "pypy3.10"

    env:
      OS: ${{ matrix.os }}
@@ -61,6 +61,7 @@ jobs:
          sudo apt-get install -y --no-install-recommends \
            curl \
            ghostscript \
+            jbig2dec \
            img2pdf \
            libexempi8 \
            libffi-dev \
@@ -100,17 +101,20 @@ jobs:
          python -m pytest --cov-report xml --cov=ocrmypdf --cov=tests/ -n0 tests/

      - name: Upload coverage to Codecov
-        uses: codecov/codecov-action@v3
+        uses: codecov/codecov-action@v4
+        env:
+          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
        with:
          files: ./coverage.xml
          env_vars: OS,PYTHON

+
  test_macos:
    name: Test macOS
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
-        os: [macos-latest]
+        os: [macos-latest, macos-13]  # macos-latest is arm64, macos-13 is x86_64
        python: ["3.10", "3.11", "3.12"]

    env:
@@ -131,7 +135,6 @@ jobs:
            ghostscript \
            jbig2enc \
            openjpeg \
-            openssl \
            pngquant \
            tesseract

@@ -158,7 +161,9 @@ jobs:
          python -m pytest --cov-report xml --cov=ocrmypdf --cov=tests/ -n0 tests/

      - name: Upload coverage to Codecov
-        uses: codecov/codecov-action@v3
+        uses: codecov/codecov-action@v4
+        env:
+          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
        with:
          files: ./coverage.xml
          env_vars: OS,PYTHON
@@ -189,7 +194,7 @@ jobs:
      - name: Install system packages
        run: |
          choco install --yes --no-progress --pre tesseract
-          choco install --yes --no-progress --ignore-checksums ghostscript
+          choco install --yes --no-progress --ignore-checksums ghostscript --version 9.56.1

      - name: Install Python packages
        run: |
@@ -201,7 +206,9 @@ jobs:
          python -m pytest --cov-report xml --cov=ocrmypdf --cov=tests/ -n0 tests/

      - name: Upload coverage to Codecov
-        uses: codecov/codecov-action@v3
+        uses: codecov/codecov-action@v4
+        env:
+          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
        with:
          files: ./coverage.xml
          env_vars: OS,PYTHON
@@ -225,8 +232,9 @@ jobs:
          python -m pip install --upgrade pip wheel build
          python -m build --sdist --wheel

-      - uses: actions/upload-artifact@v3
+      - uses: actions/upload-artifact@v4
        with:
+          name: artifact
          path: |
            ./dist/*.whl
            ./dist/*.tar.gz
@@ -240,7 +248,7 @@ jobs:
      id-token: write  # mandatory for PyPI publishing
    if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/v')
    steps:
-      - uses: actions/download-artifact@v3
+      - uses: actions/download-artifact@v4
        with:
          name: artifact
          path: dist
@@ -257,7 +265,7 @@ jobs:
      # Required to create a release
      contents: write
    steps:
-      - uses: actions/download-artifact@v3
+      - uses: actions/download-artifact@v4
        with:
          name: artifact
          path: dist
@@ -351,9 +359,6 @@ jobs:
          username: jbarlow83
          password: ${{ secrets.DOCKERHUB_TOKEN }}

-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v3
-
      - name: Set up Docker Buildx
        id: buildx
        uses: docker/setup-buildx-action@v3
@@ -365,6 +370,6 @@ jobs:
        run: |
          docker buildx build \
            --push \
-            --platform linux/amd64  \
+            --platform linux/amd64,linux/arm64  \
            --tag "${DOCKER_REPOSITORY}/${DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_TAG}" \
            --file .docker/Dockerfile.alpine .
@@ -70,6 +70,7 @@ Linux, Windows, macOS and FreeBSD are supported. Docker images are also availabl
 | Windows Subsystem for Linux   | ``apt install ocrmypdf``      |
 | Fedora                        | ``dnf install ocrmypdf``      |
 | macOS (Homebrew)              | ``brew install ocrmypdf``     |
+| macOS (MacPorts)              | ``port install ocrmypdf``     |
 | macOS (nix)                   | ``nix-env -i ocrmypdf``       |
 | LinuxBrew                     | ``brew install ocrmypdf``     |
 | FreeBSD                       | ``pkg install py-ocrmypdf``   |
@@ -118,15 +118,16 @@ OCR for huge images
 -------------------

 Tesseract has internal limits on the size
-of images it will process. If you issue
-``--tesseract-downsample-large-images``, OCRmyPDF will downsample images
-to fit Tesseract limits. (The limits are usually entered only for scanned
-images of oversized media, such as large maps or blueprints exceeding
-110 cm or 43 inches in either dimension, and at high DPI.)
+of images it will process. By default,
+``--tesseract-downsample-large-images`` is enabled, and OCRmyPDF will
+downsample images to fit Tesseract limits. (The limits are usually encountered
+only for scanned images of oversized media, such as large maps or blueprints exceeding
+110 cm or 43 inches in either dimension, and at high DPI.) This feature can disabled
+using ``--no-tesseract-downsample-large-images``.

 ``--tesseract-downsample-above Npixels`` adjusts the threshold at which images
 will be downsampled. By default, only images that exceed any of Tesseract's
-internal limits are downsampled.
+internal limits are downsampled (32767 pixels on either dimension).

 You will also need to set ``--tesseract-timeout`` high enough to allow
 for processing.
@@ -227,6 +228,59 @@ then run ocrmypdf as follows (along with any other desired arguments):
   Some combinations of control parameters will break Tesseract or break
   assumptions that OCRmyPDF makes about Tesseract's output.

+Changing page segmentation mode
+-------------------------------
+
+The directive ``--tesseract-pagesegmode Nmode`` forwards the desired page segmentation
+mode to Tesseract OCR. The default is 3.
+
+Page segmentation can improve OCR results when you know that a PDF ought to be
+analyzed a particular way, such as PDFs whose pages contain only a single line of 
+text. For the vast majority of users, changing the page segmentation mode will only
+make things worse.
+
+As of June 2024, the Tesseract page segmentation modes are:
+
+-----+----------------------------------------------------------------------------------+
+| ID  | Description                                                                      |
+=====+==================================================================================+
+|  0  | Orientation and script detection (OSD) only.                                      |
+-----+----------------------------------------------------------------------------------+
+|  1  | Automatic page segmentation with OSD.                                             |
+-----+----------------------------------------------------------------------------------+
+|  2  | Automatic page segmentation, but no OSD, or OCR. (not implemented)               |
+-----+----------------------------------------------------------------------------------+
+|  3  | Fully automatic page segmentation, but no OSD. (Default)                         |
+-----+----------------------------------------------------------------------------------+
+|  4  | Assume a single column of text of variable sizes.                                 |
+-----+----------------------------------------------------------------------------------+
+|  5  | Assume a single uniform block of vertically aligned text.                         |
+-----+----------------------------------------------------------------------------------+
+|  6  | Assume a single uniform block of text.                                            |
+-----+----------------------------------------------------------------------------------+
+|  7  | Treat the image as a single text line.                                            |
+-----+----------------------------------------------------------------------------------+
+|  8  | Treat the image as a single word.                                                 |
+-----+----------------------------------------------------------------------------------+
+|  9  | Treat the image as a single word in a circle.                                     |
+-----+----------------------------------------------------------------------------------+
+| 10  | Treat the image as a single character.                                            |
+-----+----------------------------------------------------------------------------------+
+| 11  | Sparse text. Find as much text as possible in no particular order.                |
+-----+----------------------------------------------------------------------------------+
+| 12  | Sparse text with OSD.                                                            |
+-----+----------------------------------------------------------------------------------+
+| 13  | Raw line. Treat the image as a single text line, bypassing hacks that are        |
+|     | Tesseract-specific.                                                               |
+-----+----------------------------------------------------------------------------------+
+
+Modes 0, 1, 2, and 12 (all of those that enable orientation and script detection) 
+are not compatible with OCRmyPDF, which performs OSD in a separate step from OCR.
+Their use may interfere with ``--rotate-pages`` and other features.
+
+It is currently not possible to use advanced Tesseract OCR features, such as creating
+OCR information, when using Tesseract through OCRmyPDF.
+
 Changing the PDF renderer
 =========================

@@ -112,7 +112,9 @@ OCRmyPDF is strict about not writing to standard output so that
 users can safely use it in a pipeline and produce a valid output
 file. A caller application will have to ensure it does not write to
 standard output either, if it wants to be compatible with this
-behavior and support piping to a file.
+behavior and support piping to a file. Another benefit of running
+OCRmyPDF in a child process, as recommended above, is that it will
+not interfere with the parent process's standard output.

 Exceptions
 ----------
@@ -187,7 +187,7 @@ might remove desirable content, especially from poor quality scans.
   background from grayscale or color images. Monochrome images are
   ignored. This should not be used on documents that contain color
   photos as it may remove them.
-  ``--deskew`` will correct pages were scanned at a skewed angle by
+-  ``--deskew`` will correct pages that were scanned at a skewed angle by
   rotating them back into place.
 -  ``--clean`` uses
   `unpaper <https://www.flameeyes.eu/projects/unpaper>`__ to clean up
@@ -245,6 +245,20 @@ if all you want to is to apply image processing or PDF/A conversion.
    the case. Use ``--tesseract-non-ocr-timeout`` to control the timeout
    for non-OCR operations, if needed.

+Remove all text or OCR from my PDF
+----------------------------------
+
+This is getting ridiculous, but OCRmyPDF can complete strip all textual
+information from a PDF and reconstruct it as a "bag of images" PDF.
+
+.. code-block::
+
+    ocrmypdf --tesseract-timeout 0 --force-ocr input.pdf output.pdf
+
+Why would you want to do this? Perhaps you have a PDF where OCR
+fails to produce useful results, and just want to get rid of all OCR information.
+This command also removes OCR generated by third party tools.
+
 Optimize images without performing OCR
 --------------------------------------

@@ -393,4 +407,4 @@ any digital signatures will be invalidated.
 OCRmyPDF cannot open documents that are encrypted with a digital certificate.

 Versions of OCRmyPDF prior to 14.4.0 would invalidate existing digital signatures
-without warning.
+without warning.
@@ -42,7 +42,7 @@ execute the image:
     - Architecture
     - Description
   * - ``jbarlow83/ocrmypdf-alpine``
-     - x86_64 only
+     - x86_64 and arm64
     - Recommended image, based on Alpine Linux.
   * - ``jbarlow83/ocrmypdf-ubuntu``
     - x86_64 and arm64
@@ -81,7 +81,7 @@ To start a Docker container (instance of the image):

 .. code-block:: bash

-   docker tag jbarlow83/ocrmypdf ocrmypdf
+   docker tag jbarlow83/ocrmypdf-alpine ocrmypdf
   docker run --rm -i ocrmypdf (... all other arguments here...) - -

 For convenience, create a shell alias to hide the Docker command. It is
@@ -23,7 +23,9 @@ These platforms have one-liner installs:
 +-------------------------------+-----------------------------------------+
 | Fedora                        | ``dnf install ocrmypdf tesseract-osd``  |
 +-------------------------------+-----------------------------------------+
-| macOS                         | ``brew install ocrmypdf``               |
+| macOS (Homebrew)              | ``brew install ocrmypdf``               |
+-------------------------------+-----------------------------------------+
+| macOS (MacPorts)              | ``port install ocrmypdf``               |
 +-------------------------------+-----------------------------------------+
 | LinuxBrew                     | ``brew install ocrmypdf``               |
 +-------------------------------+-----------------------------------------+
@@ -99,12 +101,12 @@ For full details on version availability for your platform, check the
 Fedora
 ------

-.. |fedora-37| image:: https://repology.org/badge/version-for-repo/fedora_37/ocrmypdf.svg
-    :alt: Fedora 37
-
 .. |fedora-38| image:: https://repology.org/badge/version-for-repo/fedora_38/ocrmypdf.svg
    :alt: Fedora 38

+.. |fedora-39| image:: https://repology.org/badge/version-for-repo/fedora_39/ocrmypdf.svg
+    :alt: Fedora 39
+
 .. |fedora-rawhide| image:: https://repology.org/badge/version-for-repo/fedora_rawhide/ocrmypdf.svg
    :alt: Fedore Rawhide

@@ -113,7 +115,7 @@ Fedora
 +-----------------------------------------------+
 | |latest|                                      |
 +-----------------------------------------------+
-| |fedora-37| |fedora-38| |fedora-rawhide|      |
+| |fedora-38| |fedora-39| |fedora-rawhide|      |
 +-----------------------------------------------+

 Users of Fedora may simply
@@ -123,7 +125,7 @@ Users of Fedora may simply
    dnf install ocrmypdf tesseract-osd

 For full details on version availability, check the `Fedora Package
-Tracker <https://apps.fedoraproject.org/packages/ocrmypdf>`__.
+Tracker <https://packages.fedoraproject.org/pkgs/ocrmypdf/ocrmypdf/>`__.

 If the version available for your platform is out of date, you could opt
 to install the latest version from source. See `Installing HEAD revision
@@ -135,10 +137,36 @@ from sources <#installing-head-revision-from-sources>`__.
   issues. OCRmyPDF works fine without it but will produce larger output
   files. If you build jbig2enc from source, ocrmypdf 7.0.0 and later
   will automatically detect it on the ``PATH``. To add JBIG2 encoding,
-   see `Installing the JBIG2 encoder <jbig2>`__.
+   see :ref:`Installing the JBIG2 encoder <jbig2>`.

 .. _ubuntu-lts-latest:

+RHEL 9
+------
+
+Prepare the environment by getting Python 3.11:
+
+.. code-block:: bash
+
+    dnf install python3.11 python3.11-pip
+
+Then, follow `Requirements for pip and HEAD install <#requirements-for-pip-and-head-install>`__ to instal dependencies:
+
+.. code-block:: bash
+
+    dnf install ghostscript tesseract
+
+and build ocrmypdf in virtual environment:
+
+.. code-block:: bash
+
+    python3.11 -m venv .venv
+
+To add JBIG2 encoding, see :ref:`Installing the JBIG2 encoder <jbig2>`.
+
+Note Fedora packages for language data haven't been branched for RHEL/EPEL, but you can get traineddata files directly from `tesseract
+<https://github.com/tesseract-ocr/tessdata/>`__ and place them in ``/usr/share/tesseract/tessdata``.
+
 Installing the latest version on Ubuntu 22.04 LTS
 -------------------------------------------------

@@ -162,37 +190,13 @@ To add JBIG2 encoding, see :ref:`jbig2`.
 Ubuntu 20.04 LTS
 ----------------

-Ubuntu 20.04 includes ocrmypdf 9.6.0 - you can install that with ``apt``. To
-install a more recent version, uninstall the system-provided version of
-ocrmypdf, and install the following dependencies:
+Ubuntu 20.04 includes ocrmypdf 9.6.0 - you can install that with ``apt``. The
+most convenient way to install recent OCRmyPDF on older Ubuntu is to use
+Homebrew on Linux (Linuxbrew).

 .. code-block:: bash

-    sudo apt-get -y remove ocrmypdf  # remove system ocrmypdf, if installed
-    sudo apt-get -y update
-    sudo apt-get -y install \
-        ghostscript \
-        icc-profiles-free \
-        libxml2 \
-        pngquant \
-        python3-pip \
-        tesseract-ocr \
-        zlib1g
-
-To install ocrmypdf for the system:
-
-.. code-block:: bash
-
-    pip3 install ocrmypdf
-
-To install for the current user only:
-
-.. code-block:: bash
-
-    export PATH=$HOME/.local/bin:$PATH
-    pip3 install --user ocrmypdf
-
-To add JBIG2 encoding, see :ref:`jbig2`.
+    brew install ocrmypdf

 Arch Linux (AUR)
 ----------------
@@ -215,12 +219,12 @@ you are using a VM image, such as `the official Vagrant image
 be completed for you.

 Next you should install the `base-devel package group
-<https://www.archlinux.org/groups/x86_64/base-devel/>`__. This includes the
+<https://archlinux.org/packages/core/any/base-devel/>`__. This includes the
 standard tooling needed to build packages, such as a compiler and binary tools.

 .. code-block:: bash

-   sudo pacman -S base-devel
+   sudo pacman -S --needed base-devel

 Now you are ready to install the OCRmyPDF package.

@@ -258,7 +262,7 @@ page.
    <https://aur.archlinux.org/packages/jbig2enc-git/>`__ and may be installed
    using the same series of steps as for the installation OCRmyPDF AUR
    package. Alternatively, it may be built manually from source following the
-    instructions in `Installing the JBIG2 encoder <jbig2>`__.  If JBIG2 is
+    instructions in :ref:`Installing the JBIG2 encoder <jbig2>`.  If JBIG2 is
    installed, OCRmyPDF 7.0.0 and later will automatically detect it.

 Alpine Linux
@@ -323,6 +327,22 @@ languages you can optionally install them all:

    brew install tesseract-lang  # Optional: Install all language packs

+MacPorts
+--------
+
+.. image:: https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fports.macports.org%2Fapi%2Fv1%2Fports%2Focrmypdf%2F%3Fformat%3Djson&query=version&label=MacPorts
+   :alt: Macports Version Information
+   :target: https://ports.macports.org/port/ocrmypdf
+
+OCRmyPDF is includes in MacPorts:
+
+.. code-block:: bash
+
+    sudo port install ocrmypdf
+
+Note that while this will install tesseract you will need to install
+the appropriate tesseract `language ports <https://ports.macports.org/search/?selected_facets=categories_exact%3Atextproc&installed_file=&q=tesseract&name=on>`__. 
+
 Manual installation on macOS
 ----------------------------

@@ -404,7 +424,7 @@ the following when running in an Administrator command prompt):
 * ``choco install --pre tesseract``
 * ``choco install pngquant`` (optional)

-Either set of commands will install the required software. At the mmoment there is no
+Either set of commands will install the required software. At the moment there is no
 single command to install Windows.

 You may then use ``pip`` to install ocrmypdf. (This can performed by a user or
@@ -540,9 +560,25 @@ try:

    pip install --user ocrmypdf

+(If the message appears ``Requirement already satisfied: ocrmypdf in...``,
+you will need to use ``pip install --user --upgrade ocrmypdf``.)
+
 You should then be able to run ``ocrmypdf --version`` and see that the
 latest version was located.

+Installing with pipx
+====================
+
+Some users may prefer pipx. As with the method above, you will need to
+satisfy all non-Python dependencies. Then if pipx is installed, you
+can use
+
+.. code-block:: bash
+
+    pipx run ocrmypdf
+
+(If not installed, pipx will install first.)
+
 Requirements for pip and HEAD install
 -------------------------------------

@@ -553,7 +589,7 @@ manager. ``pip`` cannot provide them.
 The following versions are required:

 -  Python 3.10 or newer
-  Ghostscript 9.55 or newer
+-  Ghostscript 9.54 or newer
 -  Tesseract 4.1.1 or newer
 -  jbig2enc 0.29 or newer
 -  pngquant 2.5 or newer
@@ -605,7 +641,7 @@ environment:
    pip install git+https://github.com/ocrmypdf/OCRmyPDF.git

 Or, to install in `development
-mode <https://pythonhosted.org/setuptools/setuptools.html#development-mode>`__,
+mode <https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#working-in-development-mode>`__,
 allowing customization of OCRmyPDF, use the ``-e`` flag:

 .. code-block:: bash
@@ -645,7 +681,7 @@ To install all of the development and test requirements:
 .. code-block:: bash

    git clone -b main https://github.com/ocrmypdf/OCRmyPDF.git
-    python -m .venv
+    python -m venv .venv
    source .venv/bin/activate
    cd OCRmyPDF
    pip install -e .[test]
@@ -671,7 +707,7 @@ To manually install the ``fish`` completion, copy
 Note on 32-bit support
 ======================

-Many Python libraries no longer 32-bit binary wheels for Linux. This
+Many Python libraries no longer provide 32-bit binary wheels for Linux. This
 includes many of the libraries that OCRmyPDF depends on, such as
 Pillow. The easiest way to express this to end users is to say we don't
 support 32-bit Linux.
@@ -681,4 +717,4 @@ can still install and use OCRmyPDF. A warning message will appear.
 In practice, OCRmyPDF may need more than 32-bit memory space to run when
 large documents are processed, so there are practical limitations to what
 users can accomplish with it. Still, for the common use case of an 32-bit
-ARM NAS or Raspberry Pi processing small documents, it should work.
+ARM NAS or Raspberry Pi processing small documents, it should work.
@@ -41,7 +41,15 @@ For all other platforms, you would need to build the JBIG2 encoder from source:

 Dependencies include libtoolize and libleptonica, which on Ubuntu systems
 are packaged as libtool and libleptonica-dev. On Fedora (35) they are packaged
-as libtool and leptonica-devel.
+as libtool and leptonica-devel. For this to work, please make sure to install
+``autotools``, ``automake``, ``libtool`` and ``leptonica`` first if not already
+installed.
+
+.. code-block:: bash
+
+    [sudo] apt install autotools-dev automake libtool libleptonica-dev
+..
+

 Lossy mode JBIG2
 ================
@@ -18,16 +18,26 @@ Tesseract's documentation also lists the three-letter code for your language.
 Some are anglicized, e.g. Spanish is ``spa`` rather than ``esp``, while others
 are not, e.g. German is ``deu`` and French is ``fra``.

+Language packs (strictly speaking, Tesseract "traineddata" files) generally correspond
+to the language in question, but different language packs are used in certain
+situations. For German, the "Fraktur" language pack can assist with reading older
+materials in the Fraktur typeface family (``deu_frak``). Some communities have changed
+their script from Cyrillic to Latin; the Cyrillic version of Uzbek is available
+as ``uzb_cyrl`` and the Latin version is ``uzb``.
+
 After you have installed a language pack, you can use it with ``ocrmypdf -l <language>``,
 for example ``ocrmypdf -l spa``. For multilingual documents, you can specify
 all languages to be expected, e.g. ``ocrmypdf -l eng+fra`` for English and French.
 English is assumed by default unless other language(s) are specified.

 For Linux users, you can often find packages that provide language
-packs:
+packs.

-Debian and Ubuntu users
-=======================
+Platform install steps
+======================
+
+Debian and Ubuntu (apt)
+-----------------------

 .. code-block:: bash

@@ -42,8 +52,8 @@ to what languages it should search for. Multiple languages can be
 requested using either ``-l eng+fra`` (English and French) or
 ``-l eng -l fra``.

-Fedora users
-============
+Fedora
+------

 .. code-block:: bash

@@ -58,8 +68,24 @@ to what languages it should search for. Multiple languages can be
 requested using either ``-l eng+fra`` (English and French) or
 ``-l eng -l fra``.

-Gentoo users
-============
+Archlinux
+------
+
+.. code-block:: bash
+
+   # Display a list of all Tesseract language packs
+   pacman -Ss tesseract-data
+
+   # Install German language pack
+   pacman -S tesseract-data-deu
+
+You can then pass the ``-l LANG`` argument to OCRmyPDF to give a hint as
+to what languages it should search for. Multiple languages can be
+requested using either ``-l eng+fra`` (English and French) or
+``-l eng -l fra``.
+
+Gentoo
+------

 On Gentoo the package ``app-text/tessdata_fast``, which ``app-text/tesseract`` depends on, handles Tesseract languages.
 It accepts USE flags to select what languages should be installed, these can be set in ``/etc/portage/package.use``.
@@ -85,23 +111,31 @@ to what languages it should search for. Multiple languages can be
 requested using either ``-l eng+fra`` (English and French) or
 ``-l eng -l fra``.

-macOS users
-===========
+macOS
+-----

 You can install additional language packs by
 :ref:`installing Tesseract using Homebrew with all language packs <macos-all-languages>`.

-Docker users
-============
+Docker
+------

 Users of the OCRmyPDF Docker image should install language packs into a
 derived Docker image as
 :ref:`described in that section <docker-lang-packs>`.

-Windows users
-=============
+Windows
+-------

 The Tesseract installer provided by Chocolatey currently includes only English language.
 To install other languages, download the respective language pack (``.traineddata`` file)
 from https://github.com/tesseract-ocr/tessdata/ and place it in
 ``C:\\Program Files\\Tesseract-OCR\\tessdata`` (or wherever Tesseract OCR is installed).
+
+Custom language packs
+=====================
+
+If you have fine-tuned or trained Tesseract and generated custom trained data, you can
+copy your ``customlang.traineddata`` file into your Tesseract "tessdata" folder, and
+then use the ``-l customlang`` argument to tell OCRmyPDF to pass that language on to
+Tesseract.
@@ -64,4 +64,13 @@ installation documentation.

 If you maintain a Linux distribution that supports 32-bit x86 or ARM, OCRmyPDF
 should continue to work as long as all of its dependencies continue to be
-available in 32-bit form. Please note we do not test on 32-bit platforms.
+available in 32-bit form. Please note we do not test on 32-bit platforms.
+
+HEIF/HEIC
+---------
+
+OCRmyPDF defaults to installing the pi-heif PyPI package, which supports converting
+HEIF (High Efficiency Image File Format) images to PDF from the command line.
+If your distribution does not have this library available, you can exclude it and
+OCRmyPDF will gracefully degrade automatically, losing only support for this
+feature.
@@ -30,8 +30,135 @@ OCRmyPDF typically supports the three most recent Python versions.

 .. |OCRmyPDF PyPI| image:: https://img.shields.io/pypi/v/ocrmypdf.svg

-v16.0.0rc1
-==========
+v16.4.2
+=======
+
+-  Fixed order of filenames passed to Ghostscript for PDF/A generation. :issue:`1359`
+-  Suppressed missing jbig2dec warning message. :issue:`1358`
+-  Fixed calculation of image size when soft mask dimensions don't match image
+   dimension. :issue:`1351`
+-  Several fixes to documentation. Thanks to users Iris and JoKalliauer
+   who contributed these changes.
+-  Fixed error on processing PDFs that are missing certain image metadata. :issue:`1315`
+
+v16.4.1
+=======
+
+-  Fixed calculation of image printed area (used in finding weighted DPI for OCR).
+   :issue:`1334`
+-  Fixed "NotImplementedError: not sure how to get colorspace" error
+   messages in logs which simply records a failure to optimize images with
+   print production colorspaces. :issue:`1315`
+
+v16.4.0
+=======
+
+-  Selecting the ``osd`` and ``equ`` pseudo-languages with ``-l/--language`` now
+   exits with an error when using Tesseract OCR, because these are not
+   regular Tesseract languages but implementation details implemented.
+   Using them can cause Tesseract to crash.
+-  The hOCR renderer is more tolerant of extra whitespace in input files.
+-  watcher.py now changes the output file extension to .pdf when the input is not
+   .pdf.
+-  Improved handling of PDFs that contain circularly referenced Form XObjects.
+   :issue:`1321`
+-  Fixed Alpine Docker image for ARM64, which was not building correctly.
+-  Docker images now use pikepdf 9.0.0.
+-  Prevent use of Tesseract OCR 5.4.0, a version with known regressions.
+-  Disabled progressbar for "Linearizing" when ``--no-progress-bar`` set.
+-  Fixed some tests that warn about missing JBIG2 decoding via pikepdf, by
+   installing the necessary libraries during tests.
+
+v16.3.1
+=======
+
+-  Fixed a test suite failure with Ghostscript 10.03.0+. :issue:`1316`
+-  Fixed an issue with the presentation of the "OCR" progress bar. :issue:`1313`
+
+v16.3.0
+=======
+
+-  Fixed progress bar not displaying for Ghostscript PDF/A conversion. :issue:`1313`
+-  Added progress bar for linearization. :issue:`1313`
+-  If `--rotate-pages-threshold` issued without `--rotate-pages` we now exit with
+   an error since the user likely intended to use `--rotate-pages`. :issue:`1309`
+-  If Tesseract hOCR gives an invalid line box, print an error message instead of
+   exiting with an error. :issue:`1312`
+
+v16.2.0
+=======
+
+-  Fixed issue 'NoneType' object has no attribute 'get' when optimizing certain PDFs.
+   :issue:`1293,1271`
+-  Switched formatting from black to ruff.
+-  Added support for sending sidecar output to io.BytesIO.
+-  Added support for converting HEIF/HEIC images (the native image of iPhones and
+   some other devices) to PDFs, when the appropriate pi-hief library is installed.
+   This library is marked as a dependency, but maintainers may opt out if needed.
+-  We now default to downsampling large images that would exceed Tesseract's internal
+   limits, but only if it cause processing to fail. Previously, this behavior only
+   occurred if specifically requested on command line. It can still be configured
+   and disabled. See the --tesseract command line options.
+-  Added Macports install instructions. Thanks @akierig.
+-  Improved logging output when an unexpected error occurs while trying to obtain
+   the version of a third party program.
+
+v16.1.2
+=======
+
+-  Fixed test suite failure when using Ghostscript 10.3.
+-  Other minor corrections.
+
+v16.1.1
+=======
+
+-  Fixed PyPy 3.10 support.
+
+v16.1.0
+=======
+
+-  Improved hOCR renderer is now default for left to right languages.
+-  Improved handling of rotated pages. Previously, OCR text might be missing for
+   pages that were rotated with a /Rotate tag on the page entry.
+-  Improved handling of cropped pages. Previously, in some cases a page with a
+   crop box would not have its OCR applied correctly and misalignment between
+   OCR text and visible text coudl occur.
+-  Documentation improvements, especially installation instructions for less
+   common platforms.
+
+v16.0.4
+=======
+
+-  Fixed some issues for left-to-right text with the new hOCR renderer. It is still
+   not default yet but will be made so soon. Right-to-left text is still in progress.
+-  Added an error to prevent use of several versions of Ghostscript that seem
+   corrupt existing text in input PDFs. Newly generated OCR is not affected.
+   For best results, use Ghostscript 10.02.1 or newer, which contains the fix
+   for the issue.
+
+v16.0.3
+=======
+
+-  Changed minimum required Ghostscript to 9.54, to support users of RHEL 9 and its
+   derivatives, since that is the latest version available there.
+-  Removed warning message about CVE-2023-43115, on the assumption that most
+   distributions have backported the patch by now.
+
+v16.0.2
+=======
+
+-  Temporarily changed PDF text renderer back to sandwich by default to address
+   regressions in macOS Preview.
+
+v16.0.1
+=======
+
+-  Fixed text rendering issue with new hOCR text renderer - extraneous byte order
+   marks.
+-  Tightened dependencies.
+
+v16.0.0
+=======

 -  Added OCR text renderer, combined the best ideas of Tesseract's PDF
   generator and the older hOCR transformer renderer. The result is a hopefully
@@ -1,5 +1,6 @@
 #!/usr/bin/env python3
 # SPDX-FileCopyrightText: 2016 findingorder <https://github.com/findingorder>
+# SPDX-FileCopyrightText: 2024 nilsro <https://github.com/nilsro>
 # SPDX-License-Identifier: MIT

 """Example of using ocrmypdf as a library in a script.
@@ -13,7 +14,11 @@ You should edit this script to meet your needs.

 from __future__ import annotations

+import filecmp
 import logging
+import os
+import posixpath
+import shutil
 import sys
 from pathlib import Path

@@ -22,32 +27,65 @@ import ocrmypdf
 # pylint: disable=logging-format-interpolation
 # pylint: disable=logging-not-lazy

+
+def filecompare(a, b):
+    try:
+        return filecmp.cmp(a, b, shallow=True)
+    except FileNotFoundError:
+        return False
+
+
 script_dir = Path(__file__).parent
+# set archive_dir to a path for backup original documents. Leave empty if not required.
+archive_dir = "/pdfbak"

 if len(sys.argv) > 1:
    start_dir = Path(sys.argv[1])
 else:
-    start_dir = Path('.')
+    start_dir = Path(".")

 if len(sys.argv) > 2:
    log_file = Path(sys.argv[2])
 else:
-    log_file = script_dir.with_name('ocr-tree.log')
+    log_file = script_dir.with_name("ocr-tree.log")

 logging.basicConfig(
    level=logging.INFO,
-    format='%(asctime)s %(message)s',
+    format="%(asctime)s %(message)s",
    filename=log_file,
-    filemode='a',
+    filemode="a",
 )

+logging.info(f"Start directory {start_dir}")
+
 ocrmypdf.configure_logging(ocrmypdf.Verbosity.default)

-for filename in start_dir.glob("**/*.py"):
+for filename in start_dir.glob("**/*.pdf"):
    logging.info(f"Processing {filename}")
-    result = ocrmypdf.ocr(filename, filename, deskew=True)
-    if result == ocrmypdf.ExitCode.already_done_ocr:
-        logging.error("Skipped document because it already contained text")
-    elif result == ocrmypdf.ExitCode.ok:
+    if ocrmypdf.pdfa.file_claims_pdfa(filename)["pass"]:
+        logging.info("Skipped document because it already contained text")
+    else:
+        archive_filename = archive_dir + str(filename)
+        if len(archive_dir) > 0 and not filecompare(filename, archive_filename):
+            logging.info(f"Archiving document to {archive_filename}")
+            try:
+                shutil.copy2(filename, posixpath.dirname(archive_filename))
+            except OSError:
+                os.makedirs(posixpath.dirname(archive_filename))
+                shutil.copy2(filename, posixpath.dirname(archive_filename))
+        try:
+            result = ocrmypdf.ocr(filename, filename, deskew=True)
+            logging.info(result)
+        except ocrmypdf.exceptions.EncryptedPdfError:
+            logging.info("Skipped document because it is encrypted")
+        except ocrmypdf.exceptions.PriorOcrFoundError:
+            logging.info("Skipped document because it already contained text")
+        except ocrmypdf.exceptions.DigitalSignatureError:
+            logging.info("Skipped document because it has a digital signature")
+        except ocrmypdf.exceptions.TaggedPDFError:
+            logging.info(
+                "Skipped document because it does not need ocr as it is tagged"
+            )
+        except Exception:
+            logging.error("Unhandled error occured")
        logging.info("OCR complete")
-    logging.info(result)
@@ -53,9 +53,10 @@ for dir_name, _subdirs, file_list in os.walk(start_dir):
        ]
        logging.info(cmd)
        full_path_ocr = os.path.join(dir_name, filename_ocr)
-        with open(filename, 'rb') as input_file, open(
-            full_path_ocr, 'wb'
-        ) as output_file:
+        with (
+            open(filename, 'rb') as input_file,
+            open(full_path_ocr, 'wb') as output_file,
+        ):
            proc = subprocess.run(
                cmd,
                stdin=input_file,
@@ -7,7 +7,6 @@

 # Do not enable annotations!
 # https://github.com/tiangolo/typer/discussions/598
-# from __future__ import annotations

 import json
 import logging
@@ -47,15 +46,18 @@ class LoggingLevelEnum(str, Enum):
    CRITICAL = "CRITICAL"


-def get_output_dir(root: Path, basename: str, output_dir_year_month: bool) -> Path:
+def get_output_path(root: Path, basename: str, output_dir_year_month: bool) -> Path:
+    assert '/' not in basename, "basename must not contain '/'"
    if output_dir_year_month:
        today = datetime.today()
        output_directory_year_month = root / str(today.year) / f'{today.month:02d}'
        if not output_directory_year_month.exists():
            output_directory_year_month.mkdir(parents=True, exist_ok=True)
-        output_path = Path(output_directory_year_month) / basename
+        output_path = Path(output_directory_year_month) / Path(basename).with_suffix(
+            '.pdf'
+        )
    else:
-        output_path = root / basename
+        output_path = root / Path(basename).with_suffix('.pdf')
    return output_path


@@ -99,7 +101,7 @@ def execute_ocrmypdf(
    retries_loading_file: int,
    output_dir_year_month: bool,
 ):
-    output_path = get_output_dir(output_dir, file_path.name, output_dir_year_month)
+    output_path = get_output_path(output_dir, file_path.name, output_dir_year_month)

    log.info("-" * 20)
    log.info(f'New file: {file_path}. Waiting until fully written...')
@@ -131,7 +133,7 @@ def execute_ocrmypdf(


 class HandleObserverEvent(PatternMatchingEventHandler):
-    def __init__(
+    def __init__(  # noqa: D107
        self,
        patterns=None,
        ignore_patterns=None,
@@ -191,7 +193,7 @@ def main(
        bool,
        typer.Option(
            envvar='OCR_OUTPUT_DIRECTORY_YEAR_MONTH',
-            help='Create a subdirectory in the output directory for each year and month',
+            help='Create a subdirectory in the output directory for each year/month',
        ),
    ] = False,
    on_success_delete: Annotated[
@@ -12,13 +12,14 @@ readme = "README.md"
 license = { text = "MPL-2.0" }
 requires-python = ">=3.10"
 dependencies = [
-  "Pillow>=10.0.1",
  "deprecation>=2.1.0",
-  "img2pdf>=0.4.4",
+  "img2pdf>=0.5",
  "packaging>=20",
  "pdfminer.six>=20220319",
-  "pikepdf>=8.8.0",
-  "pluggy>=0.13.0",
+  "pi-heif",                # Heif image format - maintainers: if this is removed, it will NOT break
+  "pikepdf>=8.10.1",
+  "Pillow>=10.0.1",
+  "pluggy>=1",
  "rich>=13",
 ]
 authors = [{ name = "James R. Barlow", email = "james@purplerock.ca" }]
@@ -60,7 +61,7 @@ test = [
  "types-Pillow",
  "types-humanfriendly",
 ]
-watcher = ["watchdog>=1.0.2", "typer[all]", "python-dotenv"]
+watcher = ["watchdog>=1.0.2", "typer-slim[standard]", "python-dotenv"]
 webservice = ["Flask>=2.0.1"]

 [project.scripts]
@@ -78,29 +79,6 @@ namespaces = false
 [tool.distutils.bdist_wheel]
 python-tag = "py310"

-[tool.black]
-line-length = 88
-target-version = ["py310", "py311", "py312"]
-skip-string-normalization = true
-include = '\.pyi?$'
-exclude = '''
-/(
-    \.eggs
-  | \.git
-  | \.hg
-  | \.mypy_cache
-  | \.tox
-  | \.venv
-  | _build
-  | buck-out
-  | build
-  | dist
-  | docs
-  | misc
-  | \.egg-info
-)/
-'''
-
 [tool.coverage.run]
 branch = true
 parallel = true
@@ -131,7 +109,11 @@ norecursedirs = ["lib", ".pc", ".git", "venv", "output", "cache", "resources"]
 testpaths = ["tests"]
 addopts = "-n auto"
 markers = ["slow"]
-filterwarnings = ["ignore:.*XMLParser.*:DeprecationWarning"]
+filterwarnings = [
+  "ignore:.*XMLParser.*:DeprecationWarning",
+  "ignore:.*ast.NameConstant.*:DeprecationWarning:reportlab",
+  "ignore:.*distutils.*:DeprecationWarning:libxmp",
+]

 [tool.mypy]

@@ -149,7 +131,10 @@ module = [
 ignore_missing_imports = true

 [tool.ruff]
-select = [
+target-version = "py310"
+
+[tool.ruff.lint]
+"select" = [
  "D",    # pydocstyle
  "E",    # pycodestyle
  "W",    # pycodestyle
@@ -157,17 +142,18 @@ select = [
  "I001", # isort
  "UP",   # pyupgrade
 ]
-target-version = "py310"

-[tool.ruff.isort]
+[tool.ruff.lint.isort]
 known-first-party = ["ocrmypdf"]
-required-imports = ["from __future__ import annotations"]

-[tool.ruff.pydocstyle]
+[tool.ruff.lint.pydocstyle]
 convention = "google"

-[tool.ruff.per-file-ignores]
+[tool.ruff.lint.per-file-ignores]
 "docs/conf.py" = ["D100", "D101", "D105"]
 "tests/*.py" = ["D100", "D101", "D102", "D103", "D105"]
 "misc/*.py" = ["D103", "D101", "D102"]
 "src/ocrmypdf/builtin_plugins/*.py" = ["D103", "D102", "D105"]
+
+[tool.ruff.format]
+quote-style = "preserve"
@@ -18,8 +18,8 @@ architectures: [amd64]

 environment:
  TESSDATA_PREFIX: $SNAP/usr/share/tesseract-ocr/4.00/tessdata
-  GS_LIB: $SNAP/usr/share/ghostscript/9.55/Resource/Init
-  GS_FONTPATH: $SNAP/usr/share/ghostscript/9.55/Resource/Font
+  GS_LIB: $SNAP/usr/share/ghostscript/9.55.0/Resource/Init
+  GS_FONTPATH: $SNAP/usr/share/ghostscript/9.55.0/Resource/Font
  LD_LIBRARY_PATH: $SNAP/usr/lib/x86_64-linux-gnu

 apps:
@@ -7,8 +7,8 @@ from __future__ import annotations

 import threading
 from abc import ABC, abstractmethod
-from collections.abc import Iterable
-from typing import Any, Callable, TypeVar
+from collections.abc import Callable, Iterable
+from typing import Any, TypeVar

 from ocrmypdf._progressbar import NullProgressBar, ProgressBar

@@ -0,0 +1,8 @@
+# SPDX-FileCopyrightText: 2024 James R. Barlow
+# SPDX-License-Identifier: MPL-2.0
+
+# Enforce English hegemony
+DEFAULT_LANGUAGE = 'eng'
+
+# Default rotation threshold
+DEFAULT_ROTATE_PAGES_THRESHOLD = 14.0
@@ -177,6 +177,17 @@ class GhostscriptFollower:
        self.progressbar_class = progressbar_class
        self.progressbar = None

+    def __enter__(self):
+        # We can't actually set up the progressbar here, because we don't know
+        # how many pages there are until the first __call__() happens. So we
+        # do it in __call__().
+        return self
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        if self.progressbar:
+            return self.progressbar.__exit__(exc_type, exc_value, traceback)
+        return False
+
    def __call__(self, line):
        if not self.progressbar_class:
            return
@@ -187,7 +198,8 @@ class GhostscriptFollower:
                self.progressbar = self.progressbar_class(
                    total=self.count, desc="PDF/A conversion", unit='page'
                )
-                return
+                # Now that we know the count, we can set up the progressbar.
+                self.progressbar.__enter__()
        else:
            if self.re_page.match(line.strip()):
                self.progressbar.update()
@@ -265,7 +277,10 @@ def generate_pdfa(
    )
    args_gs.extend(fspath(s) for s in pdf_pages)  # Stringify Path objs
    try:
-        with Path(output_file).open('wb') as output:
+        with (
+            Path(output_file).open('wb') as output,
+            GhostscriptFollower(progressbar_class) as pbar,
+        ):
            p = run_polling_stderr(
                args_gs,
                stdout=output,
@@ -274,7 +289,7 @@ def generate_pdfa(
                text=True,
                encoding='utf-8',
                errors='replace',
-                callback=GhostscriptFollower(progressbar_class),
+                callback=pbar,
            )
    except CalledProcessError as e:
        # Ghostscript does not change return code when it fails to create
@@ -220,7 +220,8 @@ def get_deskew(

 def tesseract_log_output(stream: bytes) -> None:
    tlog = TesseractLoggerAdapter(
-        log, extra=log.extra if hasattr(log, 'extra') else None  # type: ignore
+        log,
+        extra=log.extra if hasattr(log, 'extra') else None,  # type: ignore
    )

    if not stream:
@@ -8,19 +8,17 @@ from __future__ import annotations
 import logging
 import os
 import shlex
-import sys
 from collections.abc import Iterator
 from contextlib import contextmanager
 from decimal import Decimal
 from pathlib import Path
 from subprocess import PIPE, STDOUT
 from tempfile import TemporaryDirectory
-from typing import Union

 from packaging.version import Version
 from PIL import Image

-from ocrmypdf.exceptions import MissingDependencyError, SubprocessOutputError
+from ocrmypdf.exceptions import SubprocessOutputError
 from ocrmypdf.subprocess import get_version, run

 # unpaper documentation:
@@ -29,7 +27,7 @@ from ocrmypdf.subprocess import get_version, run

 UNPAPER_IMAGE_PIXEL_LIMIT = 256 * 1024 * 1024

-DecFloat = Union[Decimal, float]
+DecFloat = Decimal | float

 log = logging.getLogger(__name__)

@@ -289,9 +289,9 @@ class OcrGrafter:

            # Translate the text so it is centered at (0, 0), rotate it there, adjust
            # for a size different between initial and text PDF, then untranslate, and
-            # finally move the lower left corner to match the mediabox. All transforms
-            # must be premultiplied so they are applied in reverse order here.
-            ctm = corner @ untranslate @ scale @ rotate @ translate
+            # finally move the lower left corner to match the mediabox.
+            ctm = translate @ rotate @ scale @ untranslate @ corner
+            log.debug("Grafting with ctm %r", ctm)

            base_resources = _ensure_dictionary(base_page.obj, Name.Resources)
            base_xobjs = _ensure_dictionary(base_resources, Name.XObject)
@@ -311,9 +311,16 @@ class OcrGrafter:

            if strip_old_text:
                strip_invisible_text(self.pdf_base, base_page)
-
+            base_page.contents_coalesce()
+            if self.render_mode == RenderMode.ON_TOP:
+                # Add q/Q to ensure content we append is drawn correctly
+                # Strictly speaking this needs to trace the whole q/Q stack in case
+                # stack is not balanced.
+                original = base_page.Contents.read_bytes()
+                base_page.Contents.write(b'q\n' + original + b'\nQ\n')
            base_page.contents_add(
-                new_text_layer, prepend=self.render_mode == RenderMode.ON_TOP
+                new_text_layer, prepend=self.render_mode == RenderMode.UNDERNEATH
            )
+            base_page.contents_coalesce()

            _update_resources(obj=base_page.obj, font=font, font_key=font_key)
@@ -93,8 +93,8 @@ class PageContext:
        state = self.__dict__.copy()

        state['options'] = copy(self.options)
-        if not isinstance(state['options'].input_file, (str, bytes, os.PathLike)):
+        if not isinstance(state['options'].input_file, str | bytes | os.PathLike):
            state['options'].input_file = 'stream'
-        if not isinstance(state['options'].output_file, (str, bytes, os.PathLike)):
+        if not isinstance(state['options'].output_file, str | bytes | os.PathLike):
            state['options'].output_file = 'stream'
        return state
@@ -153,22 +153,52 @@ def _set_language(pdf: Pdf, languages: list[str]):
    pdf.Root.Lang = iso639_2


+class MetadataProgress:
+    def __init__(self, progressbar_class, enable: bool = True):
+        self.progressbar_class = progressbar_class
+        self.progressbar = self.progressbar_class(
+            total=100, desc="Linearizing", unit='%', disable=not enable
+        )
+
+    def __enter__(self):
+        self.progressbar.__enter__()
+        return self
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        return self.progressbar.__exit__(exc_type, exc_value, traceback)
+
+    def __call__(self, percent: int):
+        if not self.progressbar_class:
+            return
+        self.progressbar.update(completed=percent)
+
+
 def metadata_fixup(
    working_file: Path, context: PdfContext, pdf_save_settings: dict[str, Any]
 ) -> Path:
-    """Fix certain metadata fields after Ghostscript PDF/A conversion.
+    """Fix certain metadata fields whether PDF or PDF/A.
+
+    Override some of Ghostscript's metadata choices.

    Also report on metadata in the input file that was not retained during
-    PDF/A conversion.
+    conversion.
    """
    output_file = context.get_path('metafix.pdf')
    options = context.options

-    with Pdf.open(context.origin) as original, Pdf.open(working_file) as pdf:
+    pbar_class = context.plugin_manager.hook.get_progressbar_class()
+    with (
+        Pdf.open(context.origin) as original,
+        Pdf.open(working_file) as pdf,
+        MetadataProgress(pbar_class, options.progress_bar) as pbar,
+    ):
        docinfo = get_docinfo(original, context)
-        with original.open_metadata(
-            set_pikepdf_as_editor=False, update_docinfo=False, strict=False
-        ) as meta_original, pdf.open_metadata() as meta_pdf:
+        with (
+            original.open_metadata(
+                set_pikepdf_as_editor=False, update_docinfo=False, strict=False
+            ) as meta_original,
+            pdf.open_metadata() as meta_pdf,
+        ):
            meta_pdf.load_from_docinfo(
                docinfo, delete_missing=False, raise_failure=False
            )
@@ -179,6 +209,6 @@ def metadata_fixup(
            report_on_metadata(options, meta_missing)

        _set_language(pdf, options.languages)
-        pdf.save(output_file, **pdf_save_settings)
+        pdf.save(output_file, progress=pbar, **pdf_save_settings)

    return output_file
@@ -12,8 +12,9 @@ import re
 import sys
 from collections.abc import Iterable, Iterator, Sequence
 from contextlib import suppress
+from io import BytesIO
 from pathlib import Path
-from shutil import copyfileobj, copystat
+from shutil import copyfileobj
 from typing import Any, BinaryIO, TypeVar, cast

 import img2pdf
@@ -34,17 +35,29 @@ from ocrmypdf.exceptions import (
    UnsupportedImageFormatError,
 )
 from ocrmypdf.helpers import IMG2PDF_KWARGS, Resolution, safe_symlink
-from ocrmypdf.hocrtransform import HocrTransform
+from ocrmypdf.hocrtransform import DebugRenderOptions, HocrTransform
+from ocrmypdf.hocrtransform._font import Courier
 from ocrmypdf.pdfa import generate_pdfa_ps
 from ocrmypdf.pdfinfo import Colorspace, Encoding, PageInfo, PdfInfo
 from ocrmypdf.pluginspec import OrientationConfidence

+try:
+    from pi_heif import register_heif_opener
+except ImportError:
+
+    def register_heif_opener():
+        pass
+
+
 T = TypeVar("T")
 log = logging.getLogger(__name__)

 VECTOR_PAGE_DPI = 400


+register_heif_opener()
+
+
 def triage_image_file(input_file: Path, output_file: Path, options) -> None:
    """Triage the input image file.

@@ -129,7 +142,7 @@ def _pdf_guess_version(input_file: Path, search_window=1024) -> str:
    """
    with open(input_file, 'rb') as f:
        signature = f.read(search_window)
-    m = re.search(br'%PDF-(\d\.\d)', signature)
+    m = re.search(rb'%PDF-(\d\.\d)', signature)
    if m:
        return m.group(1).decode('ascii')
    return ''
@@ -462,7 +475,7 @@ def calculate_raster_dpi(page_context: PageContext):
    page_dpi = get_page_square_dpi(page_context, image_dpi)
    if dpi_profile and dpi_profile.average_to_max_dpi_ratio < 0.8:
        log.warning(
-            "Weight average image DPI is %0.1f, max DPI is %0.1f. "
+            "Weighted average image DPI is %0.1f, max DPI is %0.1f. "
            "The discrepancy may indicate a high detail region on this page, "
            "but could also indicate a problem with the input PDF file. "
            "Page image will be rendered at %0.1f DPI.",
@@ -712,19 +725,29 @@ def create_pdf_page_from_image(
    pageinfo = page_context.pageinfo
    pagesize = 72.0 * float(pageinfo.width_inches), 72.0 * float(pageinfo.height_inches)
    effective_rotation = (pageinfo.rotation - orientation_correction) % 360
-    if effective_rotation % 180 == 90:
+    swap_axis = effective_rotation % 180 == 90
+    if swap_axis:
        pagesize = pagesize[1], pagesize[0]

-    # This create a single page PDF
-    with open(image, 'rb') as imfile, open(output_file, 'wb') as pdf:
+    # Create a new single page PDF to hold
+    bio = BytesIO()
+    with open(image, 'rb') as imfile:
        log.debug('convert')

        layout_fun = img2pdf.get_layout_fun(pagesize)
        img2pdf.convert(
-            imfile, layout_fun=layout_fun, outputstream=pdf, **IMG2PDF_KWARGS
+            imfile,
+            layout_fun=layout_fun,
+            outputstream=bio,
+            engine=img2pdf.Engine.pikepdf,
+            rotation=img2pdf.Rotation.ifvalid,
        )
        log.debug('convert done')

+    # img2pdf does not generate boxes correctly, so we fix them
+    bio.seek(0)
+    fix_pagepdf_boxes(bio, output_file, page_context, swap_axis=swap_axis)
+
    output_file = page_context.plugin_manager.hook.filter_pdf_page(
        page=page_context, image_filename=image, output_pdf=output_file
    )
@@ -741,15 +764,27 @@ def render_hocr_page(hocr: Path, page_context: PageContext) -> Path:
        return output_file

    dpi = get_page_square_dpi(page_context, calculate_image_dpi(page_context))
-    debug_mode = options.pdf_renderer == 'hocrdebug'
-
+    debug_kwargs = {}
+    if options.pdf_renderer == 'hocrdebug':
+        debug_kwargs = dict(
+            debug_render_options=DebugRenderOptions(
+                render_baseline=True,
+                render_triangle=True,
+                render_line_bbox=False,
+                render_word_bbox=True,
+                render_paragraph_bbox=False,
+                render_space_bbox=False,
+            ),
+            font=Courier(),
+        )
    HocrTransform(
        hocr_filename=hocr,
-        dpi=dpi.to_scalar(),  # square
-        debug=debug_mode,
+        dpi=dpi.to_scalar(),
+        **debug_kwargs,  # square
    ).to_pdf(
        out_filename=output_file,
        image_filename=None,
+        invisible_text=True if not debug_kwargs else False,
    )
    return output_file

@@ -769,7 +804,57 @@ def ocr_engine_textonly_pdf(
        output_text=output_text,
        options=options,
    )
-    return (output_pdf, output_text)
+    return output_pdf, output_text
+
+
+def _offset_rect(rect: tuple[float, float, float, float], offset: tuple[float, float]):
+    """Offset a rectangle by a given amount."""
+    return (
+        rect[0] + offset[0],
+        rect[1] + offset[1],
+        rect[2] + offset[0],
+        rect[3] + offset[1],
+    )
+
+
+def fix_pagepdf_boxes(
+    infile: Path | BinaryIO,
+    out_file: Path,
+    page_context: PageContext,
+    swap_axis: bool = False,
+) -> Path:
+    """Fix the bounding boxes in a single page PDF.
+
+    The single page PDF is created with a normal MediaBox with its lower left corner
+    at (0, 0). infile is the single page PDF. page_context.mediabox has the original
+    file's mediabox, which may have a different origin. We needto adjust the other
+    boxes in the single page PDF to match the effect they had on the original page.
+
+    When correcting page rotation, we create a single page PDF that is correctly
+    rotated instead of an incorrectly rotated and then setting page.Rotate on it.
+    If rotation is either 90 or 270 degrees, then this function can be called
+    with swap_axis to swap the X and Y coordinates of all the boxes.
+
+    We are not concerned with solving degenerate cases where the boxes overlap or
+    or express invalid rectangles. We merely pass the boxes, producing a
+    transformation equivalent to the change made by constructing a new page image.
+    """
+    with pikepdf.open(infile) as pdf:
+        for page in pdf.pages:
+            # page.BleedBox = page_context.pageinfo.bleedbox
+            # page.ArtBox = page_context.pageinfo.artbox
+            mediabox = page_context.pageinfo.mediabox
+            offset = mediabox[0], mediabox[1]
+            cropbox = _offset_rect(page_context.pageinfo.cropbox, offset)
+            trimbox = _offset_rect(page_context.pageinfo.trimbox, offset)
+
+            if swap_axis:
+                cropbox = cropbox[1], cropbox[0], cropbox[3], cropbox[2]
+                trimbox = trimbox[1], trimbox[0], trimbox[3], trimbox[2]
+            page.CropBox = cropbox
+            page.TrimBox = trimbox
+        pdf.save(out_file)
+    return pdf


 def generate_postscript_stub(context: PdfContext) -> Path:
@@ -11,13 +11,13 @@ import os
 import shutil
 import sys
 import threading
-from collections.abc import Sequence
+from collections.abc import Callable, Sequence
 from concurrent.futures.process import BrokenProcessPool
 from concurrent.futures.thread import BrokenThreadPool
 from contextlib import contextmanager
 from dataclasses import dataclass
 from pathlib import Path
-from typing import Callable, NamedTuple, cast
+from typing import NamedTuple, cast

 import PIL

@@ -4,7 +4,6 @@

 """Implements the concurrent and page synchronous parts of the pipeline."""

-
 from __future__ import annotations

 import argparse
@@ -4,7 +4,6 @@

 """Implements the concurrent and page synchronous parts of the pipeline."""

-
 from __future__ import annotations

 import argparse
@@ -80,7 +79,6 @@ def _exec_page_sync(page_context: PageContext) -> PageResult:
        page_context
    )
    ocr_out, text_out = _image_to_ocr_text(page_context, ocr_image_out)
-
    return PageResult(
        pageno=page_context.pageno,
        pdf_page_from_image=pdf_page_from_image_out,
@@ -105,14 +103,14 @@ def exec_concurrent(context: PdfContext, executor: Executor) -> Sequence[str]:
        try:
            set_thread_pageno(result.pageno + 1)
            sidecars[result.pageno] = result.text
-            pbar.update()
+            pbar.update(0.5)
            ocrgraft.graft_page(
                pageno=result.pageno,
                image=result.pdf_page_from_image,
                textpdf=result.ocr,
                autorotate_correction=result.orientation_correction,
            )
-            pbar.update()
+            pbar.update(0.5)
        finally:
            set_thread_pageno(None)

@@ -120,10 +118,9 @@ def exec_concurrent(context: PdfContext, executor: Executor) -> Sequence[str]:
        use_threads=options.use_threads,
        max_workers=max_workers,
        progress_kwargs=dict(
-            total=(2 * len(context.pdfinfo)),
+            total=len(context.pdfinfo),
            desc='OCR' if options.tesseract_timeout > 0 else 'Image processing',
            unit='page',
-            unit_scale=0.5,
            disable=not options.progress_bar,
        ),
        worker_initializer=partial(worker_init, PIL.Image.MAX_IMAGE_PIXELS),
@@ -156,12 +153,13 @@ def _run_pipeline(
    options: argparse.Namespace,
    plugin_manager: OcrmypdfPluginManager,
 ) -> ExitCode:
-    with manage_work_folder(
-        work_folder=Path(mkdtemp(prefix="ocrmypdf.io.")),
-        retain=options.keep_temporary_files,
-        print_location=options.keep_temporary_files,
-    ) as work_folder, manage_debug_log_handler(
-        options=options, work_folder=work_folder
+    with (
+        manage_work_folder(
+            work_folder=Path(mkdtemp(prefix="ocrmypdf.io.")),
+            retain=options.keep_temporary_files,
+            print_location=options.keep_temporary_files,
+        ) as work_folder,
+        manage_debug_log_handler(options=options, work_folder=work_folder),
    ):
        executor = setup_pipeline(options, plugin_manager)
        check_requested_output_file(options)
@@ -4,7 +4,6 @@

 """Implements the concurrent and page synchronous parts of the pipeline."""

-
 from __future__ import annotations

 import argparse
@@ -66,7 +66,7 @@ class ProgressBar(Protocol):
    def __exit__(self, *args):
        """Exit a progress bar context."""

-    def update(self, n=1):
+    def update(self, n=1, *, completed=None):
        """Update the progress bar by an increment.

        For use within a progress bar context.
@@ -85,7 +85,7 @@ class NullProgressBar:
    def __exit__(self, exc_type, exc_value, traceback):
        return False

-    def update(self, _arg=None):
+    def update(self, _arg=None, *, completed=None):
        return


@@ -103,6 +103,7 @@ class RichProgressBar:
        disable: bool = False,
        **kwargs,
    ):
+        self._entered = False
        self.progress = Progress(
            TextColumn(
                "[progress.description]{task.description}",
@@ -130,6 +131,7 @@ class RichProgressBar:

    def __enter__(self):
        self.progress.start()
+        self._entered = True
        return self

    def __exit__(self, exc_type, exc_value, traceback):
@@ -137,6 +139,10 @@ class RichProgressBar:
        self.progress.stop()
        return False

-    def update(self, value=None):
-        advance = self.unit_scale if value is None else value
-        self.progress.update(self.progress_bar, advance=advance)
+    def update(self, n=1, *, completed=None):
+        assert self._entered, "Progress bar must be entered before updating"
+        if completed is None:
+            advance = self.unit_scale if n is None else n
+            self.progress.update(self.progress_bar, advance=advance)
+        else:
+            self.progress.update(self.progress_bar, completed=completed)
@@ -20,6 +20,7 @@ import pikepdf
 import PIL
 from pluggy import PluginManager

+from ocrmypdf._defaults import DEFAULT_LANGUAGE, DEFAULT_ROTATE_PAGES_THRESHOLD
 from ocrmypdf._exec import unpaper
 from ocrmypdf.exceptions import (
    BadArgsError,
@@ -30,17 +31,9 @@ from ocrmypdf.exceptions import (
 from ocrmypdf.helpers import is_file_writable, monotonic, safe_symlink
 from ocrmypdf.subprocess import check_external_program

-# -------------
-# External dependencies
-
-DEFAULT_LANGUAGE = 'eng'  # Enforce English hegemony
-
 log = logging.getLogger(__name__)


-# --------
-
-
 def check_platform() -> None:
    if sys.maxsize <= 2**32:  # pragma: no cover
        log.warning(
@@ -61,6 +54,7 @@ def check_options_languages(
            log.debug("No language specified; assuming --language %s", DEFAULT_LANGUAGE)
    if not ocr_engine_languages:
        return
+
    missing_languages = set(options.languages) - set(ocr_engine_languages)
    if missing_languages:
        lang_text = '\n'.join(lang for lang in missing_languages)
@@ -130,6 +124,11 @@ def check_options_preprocessing(options: Namespace) -> None:
        options.clean = True
    if options.unpaper_args and not options.clean:
        raise BadArgsError("--clean is required for --unpaper-args")
+    if (
+        options.rotate_pages_threshold != DEFAULT_ROTATE_PAGES_THRESHOLD
+        and not options.rotate_pages
+    ):
+        raise BadArgsError("--rotate-pages is required for --rotate-pages-threshold")
    if options.clean:
        check_external_program(
            program='unpaper',
@@ -14,7 +14,7 @@ from collections.abc import Iterable, Sequence
 from enum import IntEnum
 from io import IOBase
 from pathlib import Path
-from typing import AnyStr, BinaryIO, Union
+from typing import AnyStr, BinaryIO
 from warnings import warn

 import pluggy
@@ -28,8 +28,8 @@ from ocrmypdf._validation import check_options
 from ocrmypdf.cli import ArgumentParser, get_parser
 from ocrmypdf.helpers import is_iterable_notstr

-StrPath = Union[Path, AnyStr]
-PathOrIO = Union[BinaryIO, StrPath]
+StrPath = Path | AnyStr
+PathOrIO = BinaryIO | StrPath

 # Installing plugins affects the global state of the Python interpreter,
 # so we need to use a lock to prevent multiple threads from installing
@@ -169,7 +169,7 @@ def _kwargs_to_cmdline(

        # We have a parameter
        cmdline.append(f"--{cmd_style_arg}")
-        if isinstance(val, (int, float)):
+        if isinstance(val, int | float):
            cmdline.append(str(val))
        elif isinstance(val, str):
            cmdline.append(val)
@@ -201,14 +201,17 @@ def create_options(
        defer_kwargs={'progress_bar', 'plugins', 'parser', 'input_file', 'output_file'},
        **kwargs,
    )
-    if isinstance(input_file, (BinaryIO, IOBase)):
+    if isinstance(input_file, BinaryIO | IOBase):
        cmdline.append('stream://input_file')
    else:
        cmdline.append(os.fspath(input_file))
-    if isinstance(output_file, (BinaryIO, IOBase)):
+    if isinstance(output_file, BinaryIO | IOBase):
        cmdline.append('stream://output_file')
    else:
        cmdline.append(os.fspath(output_file))
+    if 'sidecar' in kwargs and isinstance(kwargs['sidecar'], BinaryIO | IOBase):
+        cmdline.append('--sidecar')
+        cmdline.append('stream://sidecar')

    parser.enable_api_mode()
    options = parser.parse_args(cmdline)
@@ -219,6 +222,8 @@ def create_options(
        options.input_file = input_file
    if options.output_file == 'stream://output_file':
        options.output_file = output_file
+    if options.sidecar == 'stream://sidecar':
+        options.sidecar = kwargs['sidecar']

    return options

@@ -230,7 +235,7 @@ def ocr(  # noqa: D417
    language: Iterable[str] | None = None,
    image_dpi: int | None = None,
    output_type: str | None = None,
-    sidecar: StrPath | None = None,
+    sidecar: PathOrIO | None = None,
    jobs: int | None = None,
    use_threads: bool | None = None,
    title: str | None = None,
@@ -343,7 +348,7 @@ def ocr(  # noqa: D417

    if not plugins:
        plugins = []
-    elif isinstance(plugins, (str, Path)):
+    elif isinstance(plugins, str | Path):
        plugins = [plugins]
    else:
        plugins = list(plugins)
@@ -12,10 +12,10 @@ import queue
 import signal
 import sys
 import threading
-from collections.abc import Iterable
+from collections.abc import Callable, Iterable
 from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor, as_completed
 from contextlib import suppress
-from typing import Callable, Union
+from typing import Union

 from rich.console import Console as RichConsole

@@ -25,8 +25,10 @@ from ocrmypdf._progressbar import RichProgressBar
 from ocrmypdf.exceptions import InputFileError
 from ocrmypdf.helpers import remove_all_log_handlers

-FuturesExecutorClass = Union[type[ThreadPoolExecutor], type[ProcessPoolExecutor]]
-Queue = Union[multiprocessing.Queue, queue.Queue]
+FuturesExecutorClass = Union[  # noqa: UP007
+    type[ThreadPoolExecutor], type[ProcessPoolExecutor]
+]
+Queue = Union[multiprocessing.Queue, queue.Queue]  # noqa: UP007
 UserInit = Callable[[], None]
 WorkerInit = Callable[[Queue, UserInit, int], None]

@@ -128,11 +130,14 @@ class StandardExecutor(Executor):
        listener = threading.Thread(target=log_listener, args=(log_queue,))
        listener.start()

-        with self.pbar_class(**progress_kwargs) as pbar, executor_class(
-            max_workers=max_workers,
-            initializer=initializer,
-            initargs=(log_queue, worker_initializer, logging.getLogger("").level),
-        ) as executor:
+        with (
+            self.pbar_class(**progress_kwargs) as pbar,
+            executor_class(
+                max_workers=max_workers,
+                initializer=initializer,
+                initargs=(log_queue, worker_initializer, logging.getLogger("").level),
+            ) as executor,
+        ):
            futures = [executor.submit(task, *args) for args in task_arguments]
            try:
                for future in as_completed(futures):
@@ -8,7 +8,5 @@ from ocrmypdf import hookimpl


@hookimpl
-def filter_pdf_page(
-    page, image_filename, output_pdf
-):  # pylint: disable=unused-argument
+def filter_pdf_page(page, image_filename, output_pdf):  # pylint: disable=unused-argument
    return output_pdf
@@ -17,7 +17,7 @@ log = logging.getLogger(__name__)

 # Currently all blacklisted versions are lower than 9.55, so none need to
 # be added here. If a future version is blacklisted, add it here.
-BLACKLISTED_GS_VERSIONS: frozenset[str] = frozenset()
+BLACKLISTED_GS_VERSIONS: frozenset[Version] = frozenset()


@hookimpl
@@ -54,7 +54,7 @@ def check_options(options):
        program='gs',
        package='ghostscript',
        version_checker=ghostscript.version,
-        need_version='9.55',  # Ubuntu 22.04's version
+        need_version='9.54',  # RHEL 9's version; Ubuntu 22.04 has 9.55
    )
    gs_version = ghostscript.version()
    if gs_version in BLACKLISTED_GS_VERSIONS:
@@ -62,13 +62,16 @@ def check_options(options):
            f"Ghostscript {gs_version} contains serious regressions and is not "
            "supported. Please upgrade to a newer version."
        )
-    if gs_version < Version('10.02.0'):
-        log.warning(
-            f"The installed version of Ghostscript {gs_version}, contains a remote "
-            "code execution security vulnerability. Please upgrade to a newer "
-            "version. For details see CVE-2023-43115. The issue is not known to "
-            "affect OCRmyPDF or processing PDFs with Ghostscript, but upgrading "
-            "Ghostscript is recommended."
+    if Version('10.0.0') <= gs_version < Version('10.02.1') and (
+        options.skip_text or options.redo_ocr
+    ):
+        raise MissingDependencyError(
+            f"Ghostscript 10.0.0 through 10.02.0 (your version: {gs_version}) "
+            "contain serious regressions that corrupt PDFs with existing text, "
+            "such as those processed using --skip-text or --redo-ocr. "
+            "Please upgrade to a "
+            "newer version, or use --output-type pdf to avoid Ghostscript, or "
+            "use --force-ocr to discard existing text."
        )

    if options.output_type == 'pdfa':
@@ -126,7 +129,7 @@ def generate_pdfa(
 ):
    """Generate a PDF/A from the list of PDF pages and PDF/A metadata."""
    ghostscript.generate_pdfa(
-        pdf_pages=[*pdf_pages, pdfmark],
+        pdf_pages=[pdfmark, *pdf_pages],
        output_file=output_file,
        compression=context.options.pdfa_image_compression,
        color_conversion_strategy=context.options.color_conversion_strategy,
@@ -2,9 +2,9 @@
 # SPDX-License-Identifier: MPL-2.0
 """Built-in plugin to implement OCR using Tesseract."""

-
 from __future__ import annotations

+import argparse
 import logging
 import os

@@ -14,6 +14,7 @@ from ocrmypdf import hookimpl
 from ocrmypdf._exec import tesseract
 from ocrmypdf._jobcontext import PageContext
 from ocrmypdf.cli import numeric, str_to_int
+from ocrmypdf.exceptions import BadArgsError, MissingDependencyError
 from ocrmypdf.helpers import clamp
 from ocrmypdf.imageops import calculate_downsample, downsample_image
 from ocrmypdf.pluginspec import OcrEngine
@@ -94,7 +95,8 @@ def add_options(parser):
    )
    tess.add_argument(
        '--tesseract-downsample-large-images',
-        action='store_true',
+        action=argparse.BooleanOptionalAction,
+        default=True,
        help=(
            "Downsample large images before OCR. Tesseract has an upper limit on the "
            "size images it will support. If this argument is given, OCRmyPDF will "
@@ -143,10 +145,20 @@ def check_options(options):
        need_version='4.1.1',  # Ubuntu 22.04 version (also 20.04)
        version_parser=tesseract.TesseractVersion,
    )
+    tess_version = tesseract.version()
+    if tess_version == tesseract.TesseractVersion('5.4.0'):
+        raise MissingDependencyError(
+            "Tesseract 5.4.0 is not supported due to regressions in this version. "
+            "Please upgrade to a newer or supported older version."
+        )

    # Decide on what renderer to use
    if options.pdf_renderer == 'auto':
-        options.pdf_renderer = 'hocr'
+        if {'ara', 'heb', 'fas', 'per'} & set(options.languages):
+            log.info("Using sandwich renderer since there is an RTL language")
+            options.pdf_renderer = 'sandwich'
+        else:
+            options.pdf_renderer = 'hocr'

    if not tesseract.has_thresholding() and options.tesseract_thresholding != 0:
        log.warning(
@@ -159,6 +171,14 @@ def check_options(options):
            "The --tesseract-pagesegmode argument you select will disable OCR. "
            "This may cause processing to fail."
        )
+    DENIED_LANGUAGES = {'equ', 'osd'}
+    if DENIED_LANGUAGES & set(options.languages):
+        raise BadArgsError(
+            "The following languages for Tesseract's internal use and should not "
+            "be issued explicitly: "
+            f"{', '.join(DENIED_LANGUAGES & set(options.languages))}\n"
+            "Remove them from the -l/--language argument."
+        )


@hookimpl
@@ -216,7 +236,7 @@ class TesseractOcrEngine(OcrEngine):

    @staticmethod
    def creator_tag(options):
-        tag = '-PDF' if options.pdf_renderer == 'sandwich' else 'hOCR'
+        tag = '-PDF' if options.pdf_renderer == 'sandwich' else '-hOCR'
        return f"Tesseract OCR{tag} {TesseractOcrEngine.version()}"

    def __str__(self):
@@ -6,9 +6,10 @@
 from __future__ import annotations

 import argparse
-from collections.abc import Mapping
-from typing import Any, Callable, TypeVar
+from collections.abc import Callable, Mapping
+from typing import Any, TypeVar

+from ocrmypdf._defaults import DEFAULT_ROTATE_PAGES_THRESHOLD
 from ocrmypdf._version import PROGRAM_NAME as _PROGRAM_NAME
 from ocrmypdf._version import __version__ as _VERSION

@@ -390,7 +391,7 @@ Online documentation is located at:
        action='store',
        type=numeric(float, 0),
        metavar='MPixels',
-        help="Set maximum number of pixels to unpack before treating an image as a "
+        help="Set maximum number of megapixels to unpack before treating an image as a "
        "decompression bomb",
        default=250.0,
    )
@@ -403,7 +404,7 @@ Online documentation is located at:
    )
    advanced.add_argument(
        '--rotate-pages-threshold',
-        default=14.0,
+        default=DEFAULT_ROTATE_PAGES_THRESHOLD,
        type=numeric(float, 0, 1000),
        metavar='CONFIDENCE',
        help="Only rotate pages when confidence is above this value (arbitrary "
@@ -20,13 +20,12 @@ from __future__ import annotations
 import logging
 import logging.handlers
 import signal
-from collections.abc import Iterable, Iterator
+from collections.abc import Callable, Iterable, Iterator
 from contextlib import suppress
 from enum import Enum, auto
 from itertools import islice, repeat, takewhile, zip_longest
 from multiprocessing import Pipe, Process
 from multiprocessing.connection import Connection, wait
-from typing import Callable

 from ocrmypdf import Executor, hookimpl
 from ocrmypdf._concurrent import NullProgressBar
@@ -10,7 +10,7 @@ import multiprocessing
 import os
 import shutil
 import warnings
-from collections.abc import Iterable, Sequence
+from collections.abc import Callable, Iterable, Sequence
 from contextlib import suppress
 from decimal import Decimal
 from io import StringIO
@@ -19,13 +19,13 @@ from pathlib import Path
 from statistics import harmonic_mean
 from typing import (
    Any,
-    Callable,
    Generic,
    TypeVar,
 )

 import img2pdf
 import pikepdf
+from deprecation import deprecated

 log = logging.getLogger(__name__)

@@ -136,6 +136,7 @@ class Resolution(Generic[T]):
        return self._isclose(self.x, other.x) and self._isclose(self.y, other.y)


+@deprecated(deprecated_in='15.4.0')
 class NeverRaise(Exception):
    """An exception that is never raised."""

@@ -267,7 +268,9 @@ def check_pdf(input_file: Path) -> bool:
        return False
    else:
        with pdf:
-            messages = pdf.check()
+            with warnings.catch_warnings():
+                warnings.filterwarnings('ignore', message=r'pikepdf.*JBIG2.*')
+                messages = pdf.check()
            success = True
            for msg in messages:
                if 'error' in msg.lower():
@@ -18,7 +18,12 @@ from pikepdf.canvas import Font
 log = logging.getLogger(__name__)


-class GlyphlessFont(Font):
+class EncodableFont(Font):
+    def text_encode(self, text: str) -> bytes:
+        raise NotImplementedError()
+
+
+class GlyphlessFont(EncodableFont):
    CID_TO_GID_DATA = zlib.compress(b"\x00\x01" * 65536)
    GLYPHLESS_FONT_NAME = 'pdf.ttf'
    GLYPHLESS_FONT = (package_files('ocrmypdf.data') / GLYPHLESS_FONT_NAME).read_bytes()
@@ -27,11 +32,14 @@ class GlyphlessFont(Font):
    def __init__(self):
        pass

-    def text_width(self, text: str, fontsize: float) -> int:
+    def text_width(self, text: str, fontsize: float) -> float:
        """Estimate the width of a text string when rendered with the given font."""
        # NFKC: split ligatures, combine diacritics
        return len(unicodedata.normalize("NFKC", text)) * (fontsize / self.CHAR_ASPECT)

+    def text_encode(self, text: str) -> bytes:
+        return text.encode('utf-16be')
+
    def register(self, pdf: Pdf):
        """Register the glyphless font.

@@ -110,3 +118,24 @@ class GlyphlessFont(Font):
        font_descriptor.FontFile2 = pdf.make_stream(self.GLYPHLESS_FONT)
        cid_font_type2.FontDescriptor = font_descriptor
        return basefont
+
+
+class Courier(EncodableFont):
+    """Courier font."""
+
+    def text_width(self, text: str, fontsize: float) -> float:
+        """Estimate the width of a text string when rendered with the given font."""
+        return len(text) * fontsize
+
+    def text_encode(self, text: str) -> bytes:
+        return text.encode('pdfdoc', errors='ignore')
+
+    def register(self, pdf: Pdf) -> Dictionary:
+        """Register the font."""
+        return pdf.make_indirect(
+            Dictionary(
+                BaseFont=Name.Courier,
+                Type=Name.Font,
+                Subtype=Name.Type1,
+            )
+        )
@@ -27,11 +27,11 @@ from pikepdf.canvas import (
    MAGENTA,
    RED,
    Canvas,
-    Font,
    Text,
    TextDirection,
 )

+from ocrmypdf.hocrtransform._font import EncodableFont as Font
 from ocrmypdf.hocrtransform._font import GlyphlessFont

 log = logging.getLogger(__name__)
@@ -45,12 +45,12 @@ Element = ElementTree.Element
 class DebugRenderOptions:
    """A class for managing rendering options."""

-    render_paragraph_bbox: bool
-    render_baseline: bool
-    render_triangle: bool
-    render_line_bbox: bool
-    render_word_bbox: bool
-    render_space_bbox: bool
+    render_paragraph_bbox: bool = False
+    render_baseline: bool = False
+    render_triangle: bool = False
+    render_line_bbox: bool = False
+    render_word_bbox: bool = False
+    render_space_bbox: bool = False


 class HocrTransformError(Exception):
@@ -61,15 +61,25 @@ class HocrTransform:
    """A class for converting documents from the hOCR format.

    For details of the hOCR format, see:
-    http://kba.cloud/hocr-spec/.
+    http://kba.github.io/hocr-spec/1.2/.
    """

-    box_pattern = re.compile(r'bbox (\d+) (\d+) (\d+) (\d+)')
+    box_pattern = re.compile(
+        r'''
+        bbox \s+
+        (\d+) \s+   # left: uint
+        (\d+) \s+   # top: uint
+        (\d+) \s+   # right: uint
+        (\d+)       # bottom: uint
+        ''',
+        re.VERBOSE,
+    )
    baseline_pattern = re.compile(
        r'''
        baseline \s+
        ([\-\+]?\d*\.?\d*) \s+  # +/- decimal float
-        ([\-\+]?\d+)            # +/- int''',
+        ([\-\+]?\d+)            # +/- int
+        ''',
        re.VERBOSE,
    )

@@ -81,8 +91,21 @@ class HocrTransform:
        debug: bool = False,
        fontname: Name = Name("/f-0-0"),
        font: Font = GlyphlessFont(),
+        debug_render_options: DebugRenderOptions | None = None,
    ):
        """Initialize the HocrTransform object."""
+        if debug:
+            log.warning("Use debug_render_options instead", DeprecationWarning)
+            self.render_options = DebugRenderOptions(
+                render_baseline=debug,
+                render_triangle=debug,
+                render_line_bbox=False,
+                render_word_bbox=debug,
+                render_paragraph_bbox=False,
+                render_space_bbox=False,
+            )
+        else:
+            self.render_options = debug_render_options or DebugRenderOptions()
        self.dpi = dpi
        self.hocr = ElementTree.parse(os.fspath(hocr_filename))
        self._fontname = fontname
@@ -103,24 +126,13 @@ class HocrTransform:
            self.height = (coords.ury - coords.lly) / (self.dpi / INCH)
            # Stop after first div that has page coordinates
            break
-        self.render_options = DebugRenderOptions(
-            render_baseline=debug,
-            render_triangle=debug,
-            render_line_bbox=debug,
-            render_word_bbox=debug,
-            render_paragraph_bbox=debug,
-            render_space_bbox=debug,
-        )

-    def _get_element_text(self, element: Element):
+    def _get_element_text(self, element: Element) -> str:
        """Return the textual content of the element and its children."""
-        text = ''
-        if element.text is not None:
-            text += element.text
+        text = element.text if element.text is not None else ''
        for child in element:
            text += self._get_element_text(child)
-        if element.tail is not None:
-            text += element.tail
+        text += element.tail if element.tail is not None else ''
        return text

    @classmethod
@@ -215,12 +227,13 @@ class HocrTransform:
            if not found_lines:
                # Tesseract did not report any lines (just words)
                root = self.hocr.find(self._child_xpath('div', 'ocr_page'))
+                direction = self._get_text_direction(root)
                self._do_line(
                    canvas,
                    root,
                    "ocrx_word",
                    invisible_text,
-                    TextDirection.LTR,
+                    direction,
                    True,
                )
        # put the image on the page, scaled to fill the page
@@ -280,7 +293,13 @@ class HocrTransform:
        line_box = self.element_coordinates(line)
        if not line_box:
            return
-        assert line_box.ury > line_box.lly  # lly is top, ury is bottom
+        if line_box.ury <= line_box.lly:
+            log.error(
+                "line box is invalid so we cannot render it: box=%s text=%s",
+                line_box,
+                self._get_element_text(line),
+            )
+            return

        self._debug_draw_line_bbox(canvas, line_box)

@@ -311,8 +330,7 @@ class HocrTransform:
            line_box_height = abs(line_box.height) / cos(angle)
            fontsize = line_box_height + intercept
            text.font(self._fontname, fontsize)
-            if invisible_text or True:
-                text.render_mode(3)  # Invisible (indicates OCR text)
+            text.render_mode(3 if invisible_text else 0)

            self._debug_draw_baseline(
                canvas, line_matrix.inverse().transform(line_box), 0
@@ -362,10 +380,15 @@ class HocrTransform:
        self._debug_draw_word_bbox(canvas, box)

        # If this word is 0 units wide, our best bet seems to be to suppress this text
+        if text_direction == TextDirection.RTL:
+            log.info("RTL: %s", elemtxt)
        if font_width > 0:
-            text.text_transform(Matrix(1, 0, 0, 1, box.llx, 0))
+            if text_direction == TextDirection.LTR:
+                text.text_transform(Matrix(1, 0, 0, -1, box.llx, 0))
+            elif text_direction == TextDirection.RTL:
+                text.text_transform(Matrix(-1, 0, 0, -1, box.llx + box.width, 0))
            text.horiz_scale(100 * box.width / font_width)
-            text.show(elemtxt)
+            text.show(self._font.text_encode(elemtxt))

        # Get coordinates of the next word (if there is one)
        hocr_next_box = (
@@ -385,11 +408,16 @@ class HocrTransform:
        elif text_direction == TextDirection.RTL:
            space_box = Rectangle(next_box.urx, box.lly, box.llx, next_box.ury)
        self._debug_draw_space_bbox(canvas, space_box)
-        text.text_transform(Matrix(1, 0, 0, 1, space_box.llx, 0))
        space_width = self._font.text_width(' ', fontsize)
        if space_width > 0:
+            if text_direction == TextDirection.LTR:
+                text.text_transform(Matrix(1, 0, 0, -1, space_box.llx, 0))
+            elif text_direction == TextDirection.RTL:
+                text.text_transform(
+                    Matrix(-1, 0, 0, -1, space_box.llx + space_box.width, 0)
+                )
            text.horiz_scale(100 * space_box.width / space_width)
-            text.show(' ')
+            text.show(self._font.text_encode(' '))

    def _debug_draw_paragraph_boxes(self, canvas: Canvas, color=CYAN):
        """Draw boxes around paragraphs in the document."""
@@ -7,12 +7,12 @@ Derived from
 https://www.loc.gov/standards/iso639-2/ascii_8bits.html
 """

-
 from typing import NamedTuple


 class ISOCodeData(NamedTuple):
    """Data for a single ISO 639 code."""
+
    alt: str
    alpha_2: str
    english: str
@@ -168,8 +168,10 @@ ISO_639_3 = {
    'chu': ISOCodeData(
        '',
        'cu',
-        ('Church Slavic; Old Slavonic; Church Slavonic;'
-         ' Old Bulgarian; Old Church Slavonic'),
+        (
+            'Church Slavic; Old Slavonic; Church Slavonic;'
+            ' Old Bulgarian; Old Church Slavonic'
+        ),
        "slavon d'église; vieux slave; slavon liturgique; vieux bulgare",
    ),
    'chv': ISOCodeData('', 'cv', 'Chuvash', 'tchouvache'),
@@ -3,7 +3,6 @@

 """Post-processing image optimization of OCR PDFs."""

-
 from __future__ import annotations

 import logging
@@ -11,11 +10,10 @@ import sys
 import tempfile
 import threading
 from collections import defaultdict
-from collections.abc import Iterator, MutableSet, Sequence
+from collections.abc import Callable, Iterator, MutableSet, Sequence
 from os import fspath
 from pathlib import Path
-from typing import Any, Callable, NamedTuple, NewType
-from warnings import warn
+from typing import Any, NamedTuple, NewType
 from zlib import compress

 import img2pdf
@@ -30,6 +28,7 @@ from pikepdf import (
    Stream,
    UnsupportedImageTypeError,
 )
+from pikepdf.models.image import HifiPrintImageNotTranscodableError
 from PIL import Image

 from ocrmypdf._concurrent import Executor, SerialExecutor
@@ -91,6 +90,7 @@ def extract_image_filter(
        if (
            len(pim.filter_decodeparms) == 2
            and first_filtdp[0] == Name.FlateDecode
+            and first_filtdp[1] is not None
            and first_filtdp[1].get(Name.Predictor, 1) == 1
            and second_filtdp[0] == Name.DCTDecode
            and not second_filtdp[1]
@@ -155,6 +155,13 @@ def extract_image_jbig2(
                with imgname.open('wb') as f:
                    ext = pim.extract_to(stream=f)
                imgname.rename(imgname.with_suffix(ext))
+            except NotImplementedError as e:
+                if '/Decode' in str(e):
+                    log.debug(
+                        f"xref {xref}: skipping image with unsupported Decode table"
+                    )
+                    return None
+                raise
            except UnsupportedImageTypeError:
                return None
            finally:
@@ -194,7 +201,7 @@ def extract_image_generic(
            with imgname.open('wb') as f:
                ext = pim.extract_to(stream=f)
            imgname.rename(imgname.with_suffix(ext))
-        except UnsupportedImageTypeError:
+        except (UnsupportedImageTypeError, HifiPrintImageNotTranscodableError):
            return None
        return XrefExt(xref, ext)
    elif (
@@ -250,6 +257,9 @@ def _find_image_xrefs_container(
    for _imname, image in dict(xobjs).items():
        if image.objgen[1] != 0:
            continue  # Ignore images in an incremental PDF
+        xref = Xref(image.objgen[0])
+        if xref in include_xrefs or xref in exclude_xrefs:
+            continue  # Already processed
        if Name.Subtype in image and image.Subtype == Name.Form:
            # Recurse into Form XObjects
            log.debug(f"Recursing into Form XObject {_imname} in page {pageno}")
@@ -263,7 +273,6 @@ def _find_image_xrefs_container(
                depth + 1,
            )
            continue
-        xref = Xref(image.objgen[0])
        if Name.SMask in image:
            # Ignore soft masks
            smask_xref = Xref(image.SMask.objgen[0])
@@ -90,7 +90,7 @@ def generate_pdfa_ps(target_filename: Path, icc: str = 'sRGB'):
        icc: ICC identifier such as 'sRGB'
    References:
        Adobe PDFMARK Reference:
-        https://www.adobe.com/content/dam/acom/en/devnet/acrobat/pdfs/pdfmark_reference.pdf
+        https://opensource.adobe.com/dc-acrobat-sdk-docs/library/pdfmark/
    """
    if icc != 'sRGB':
        raise NotImplementedError("Only supporting sRGB")
@@ -10,9 +10,8 @@ import atexit
 import logging
 import re
 import statistics
-import sys
 from collections import defaultdict
-from collections.abc import Container, Iterable, Iterator, Mapping, Sequence
+from collections.abc import Callable, Container, Iterable, Iterator, Mapping, Sequence
 from contextlib import contextmanager
 from decimal import Decimal
 from enum import Enum, auto
@@ -20,7 +19,7 @@ from functools import partial
 from math import hypot, inf, isclose
 from os import PathLike
 from pathlib import Path
-from typing import Callable, NamedTuple
+from typing import NamedTuple
 from warnings import warn

 from pdfminer.layout import LTPage, LTTextBox
@@ -240,7 +239,13 @@ def _interpret_contents(contentstream: Object, initial_shorthand=UNIT_SQUARE):
                # to do. Just pretend nothing happened, keep calm and carry on.
                warn("PDF graphics stack underflowed - PDF may be malformed")
        elif operator == 'cm':
-            ctm = Matrix(operands) @ ctm
+            try:
+                ctm = Matrix(operands) @ ctm
+            except ValueError:
+                raise InputFileError(
+                    "PDF content stream is corrupt - this PDF is malformed. "
+                    "Use a PDF editor that is capable of visually inspecting the PDF."
+                )
        elif operator == 'Do':
            image_name = operands[0]
            settings = XobjectSettings(
@@ -364,8 +369,18 @@ class ImageInfo:
            pim = PdfImage(pdfimage)
        else:
            raise ValueError("Either pdfimage or inline must be set")
-        self._width = pim.width
-        self._height = pim.height
+        if pim.obj.get(Name.SMask, None) is not None:
+            # SMask is pretty much an alpha channel, but in PDF it's possible
+            # for channel to have different dimensions than the image
+            # itself. Some PDF writers use this to create a grayscale stencil
+            # mask. For our purposes, the effective size is the size of the
+            # larger component (image or smask).
+            smask = pim.obj[Name.SMask]
+            self._width = max(smask.get(Name.Width, 0), pim.width)
+            self._height = max(smask.get(Name.Height, 0), pim.height)
+        else:
+            self._width = pim.width
+            self._height = pim.height

        # If /ImageMask is true, then this image is a stencil mask
        # (Images that draw with this stencil mask will have a reference to
@@ -486,7 +501,7 @@ class ImageInfo:
        """Physical area of the image in square inches."""
        if not self.renderable:
            return 0.0
-        return float(self.width * self.dpi.x * self.height * self.dpi.y)
+        return float((self.width / self.dpi.x) * (self.height / self.dpi.y))

    def __repr__(self):
        """Return a string representation of the image."""
@@ -568,7 +583,7 @@ def _find_form_xobject_images(pdf: Pdf, container: Object, contentsinfo: Content
    xobjs = resources[Name.XObject].as_dict()
    for xobj in xobjs:
        candidate = xobjs[xobj]
-        if candidate is None or candidate[Name.Subtype] != Name.Form:
+        if candidate is None or candidate.get(Name.Subtype) != Name.Form:
            continue

        form_xobject = candidate
@@ -854,6 +869,12 @@ class PageInfo:
        width_pt = mediabox[2] - mediabox[0]
        height_pt = mediabox[3] - mediabox[1]

+        # self._artbox = [float(d) for d in page.artbox.as_list()]
+        # self._bleedbox = [float(d) for d in page.bleedbox.as_list()]
+        self._cropbox = [float(d) for d in page.cropbox.as_list()]
+        self._mediabox = [float(d) for d in page.mediabox.as_list()]
+        self._trimbox = [float(d) for d in page.trimbox.as_list()]
+
        check_this_page = pageno in check_pages

        if check_this_page and detailed_analysis:
@@ -970,6 +991,21 @@ class PageInfo:
        else:
            raise ValueError("rotation must be a cardinal angle")

+    @property
+    def cropbox(self) -> FloatRect:
+        """Return cropbox of page in PDF coordinates."""
+        return self._cropbox
+
+    @property
+    def mediabox(self) -> FloatRect:
+        """Return mediabox of page in PDF coordinates."""
+        return self._mediabox
+
+    @property
+    def trimbox(self) -> FloatRect:
+        """Return trimbox of page in PDF coordinates."""
+        return self._trimbox
+
    @property
    def images(self) -> list[ImageInfo]:
        """Return images."""
@@ -1039,18 +1075,12 @@ class PageInfo:

        weights = [area / total_drawn_area for area in image_areas]
        # Calculate harmonic mean of DPIs weighted by area
-        if sys.version_info >= (3, 10):
-            weighted_dpi = statistics.harmonic_mean(image_dpis, weights)
-        else:
-            weighted_dpi = sum(weights) / sum(
-                weight / dpi for weight, dpi in zip(weights, image_dpis)
-            )
+        weighted_dpi = statistics.harmonic_mean(image_dpis, weights)
        max_dpi = max(image_dpis)
        dpi_average_max_ratio = weighted_dpi / max_dpi

        arg_max_dpi = image_dpis.index(max_dpi)
        max_area_ratio = image_areas[arg_max_dpi] / total_drawn_area
-
        return PageResolutionProfile(
            weighted_dpi,
            max_dpi,
@@ -1155,7 +1185,7 @@ class PdfInfo:
    @property
    def filename(self) -> str | Path:
        """Return filename of PDF."""
-        if not isinstance(self._infile, (str, Path)):
+        if not isinstance(self._infile, str | Path):
            raise NotImplementedError("can't get filename from stream")
        return self._infile

@@ -5,12 +5,12 @@
 from __future__ import annotations

 import re
-from collections.abc import Mapping
+from collections.abc import Iterator, Mapping
 from contextlib import contextmanager
 from math import copysign
 from os import PathLike
 from pathlib import Path
-from typing import Any, Iterator
+from typing import Any
 from unittest.mock import patch

 import pdfminer
@@ -3,7 +3,6 @@

 """Utilities to measure OCR quality."""

-
 from __future__ import annotations

 import re
@@ -8,12 +8,11 @@ import logging
 import os
 import re
 import sys
-from collections.abc import Mapping, Sequence
+from collections.abc import Callable, Mapping, Sequence
 from contextlib import suppress
 from pathlib import Path
 from subprocess import PIPE, STDOUT, CalledProcessError, CompletedProcess, Popen
 from subprocess import run as subprocess_run
-from typing import Callable, Union

 from packaging.version import Version

@@ -23,7 +22,7 @@ from ocrmypdf.exceptions import MissingDependencyError

 log = logging.getLogger(__name__)

-Args = Sequence[Union[Path, str]]
+Args = Sequence[Path | str]
 OsEnviron = os._Environ  # pylint: disable=protected-access


@@ -172,6 +171,7 @@ def get_version(
        ) from e
    except CalledProcessError as e:
        if e.returncode != 0:
+            log.exception(e)
            raise MissingDependencyError(
                f"Ran program '{program}' but it exited with an error:\n{e.output}"
            ) from e
@@ -9,18 +9,13 @@ import os
 import re
 import shutil
 import sys
-from collections.abc import Iterable, Iterator
+from collections.abc import Callable, Iterable, Iterator
 from itertools import chain
 from pathlib import Path
-from typing import Any, Callable, TypeVar
+from typing import Any, TypeAlias, TypeVar

 from packaging.version import InvalidVersion, Version

-if sys.version_info >= (3, 10):
-    from typing import TypeAlias
-else:
-    from typing_extensions import TypeAlias  # pragma: no cover
-
 if sys.platform == 'win32':
    # mypy understands 'if sys.platform' better than try/except ModuleNotFoundError
    import winreg  # pylint: disable=import-error
@@ -84,7 +79,7 @@ def registry_path_ghostscript(env=None) -> Iterator[Path]:
                registry_subkeys(k), key=ghostscript_version_key, default=(0, 0, 0)
            )
        with winreg.OpenKey(
-            winreg.HKEY_LOCAL_MACHINE, fr"SOFTWARE\Artifex\GPL Ghostscript\{latest_gs}"
+            winreg.HKEY_LOCAL_MACHINE, rf"SOFTWARE\Artifex\GPL Ghostscript\{latest_gs}"
        ) as k:
            for _, gs_path, _ in registry_values(k):
                yield Path(gs_path) / 'bin'
@@ -0,0 +1,24 @@
+i a la Waterman
+
+4 ons linzen
+
+3 liter water
+
+3 uien
+
+bloem, boter
+
+2 kopjes melk
+
+laurier, kruidnagel, kerrie, zout
+
+De linzgen wassen en in-l liter kokend wa-
+ter 1 dag laten weken, 2 liter water bij
+de linzen voegen, zonder het water waarin
+ze geweekt zijn af te gieten, De helft van
+de uien bakken met laurier en Kruidnagel.
+Alle uien, kerrie en zgout bij de linzen
+voegen, Alles aan de kook brengen, Van de
+bloem met boter en melk een papje maken en
+verder afmaken met de soep, Als de linzen
+gfgaar Zijn is de soep klaar.
@@ -0,0 +1,15 @@
+Tarnose
+
+Bokale oa
+
+Lehuntze
+
+Mugerre
+
+Milafranga Komunikabideak
+
+BAIONA  zeiteninsiie —
+
+7 Trenbideak -----
+
+t\ Basusarri — spmeans:20141004 ae: . _ ~
@@ -0,0 +1 @@
+Covfefe is a perfectly cromulent word.
@@ -0,0 +1,24 @@
+Linzensoep a la Waterman
+
+4 ons linzen
+
+3 liter water
+
+3 uien
+
+bloem, boter
+
+2 kopjes melk
+
+laurier, kruidnagel, kerrie, zout
+
+De linzgen wassen en in-l liter kokend wa-
+ter 1 dag laten weken, 2 liter water bij
+de linzen voegen, zonder het water waarin
+ze geweekt zijn af te gieten, De helft van
+de uien bakken met laurier en Kruidnagel.
+Alle uien, kerrie en zgout bij de linzen
+voegen, Alles aan de kook brengen, Van de
+bloem met boter en melk een papje maken en
+verder afmaken met de soep, Als de linzen
+gfgaar Zijn is de soep klaar.
@@ -0,0 +1,12 @@
+Portez ce vieux whisky au juge
+blond qui fume sur son Ile
+interieure, a cöte de l'alcöve
+ovoide, oU les büches se
+consument dans l'ätre, ce qui
+ui permet de penser & la
+caenogenese de |'etre dont il
+est question dans la cause
+ambigu& entendue a MoY, dans
+un capharnaüm qui, pense-t-il,
+diminue ca et la la qualit& de son
+ceuvre.
@@ -0,0 +1,22 @@
+600
+500 .
+400 _ EB fp ys
+300 —f— EN / ~/ Y
+Tg ANA a3
+100 - nee -eemiae
+0-+—-* a ee ee ee eee
+S @ s > oO © ve S ) @ + & & “A 5 oo
+Se ° Ps as ge eS € x ro NS Po e? & s AS
+a a © FF SF HY HK SK BM ee sO
+e < Na ‘ : > cy xs eS @ Ww a) Oo
+ee S&S FF SF HK S e © 4
+~ & e
+3 x
+
+—¢—Support
+
+—H— No vote[note 1]
+ir Oppose
+—=—Net[note 2]
+
+re Percentage [note 3]
@@ -71,3 +71,15 @@
 {"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.6", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/poster.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
 {"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.6", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/multipage.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
 {"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.6", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_hocr__hocr__txt", "sourcefile": "resources/poster.pdf", "args": ["-l", "eng", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_hocr", "hocr", "txt"]}
+{"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.7", "argv_slug": "__-l__deu__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/francais.pdf", "args": ["-l", "deu", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.7", "argv_slug": "__-l__eng__--oem__1__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/trivial.pdf", "args": ["-l", "eng", "--oem", "1", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.7", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/trivial.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.7", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/2400dpi.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.7", "argv_slug": "__-l__eng__thresholding_method=1__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/trivial.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "-c", "thresholding_method=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.7", "argv_slug": "__-l__eng__thresholding_method=2__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/trivial.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "-c", "thresholding_method=2", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.7", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/graph_ocred.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.7", "argv_slug": "__-l__eng__000002_ocr.png__000002_ocr_tess__pdf__txt", "sourcefile": "resources/3small.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000002_ocr.png", "$TMPDIR/000002_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.7", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/3small.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.7", "argv_slug": "__-l__eng__000003_ocr.png__000003_ocr_tess__pdf__txt", "sourcefile": "resources/3small.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000003_ocr.png", "$TMPDIR/000003_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.7", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/poster.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
+{"tesseract_version": "5.3.2", "system": "Linux", "python": "3.11.7", "argv_slug": "__-l__eng__000001_ocr.png__000001_ocr_tess__pdf__txt", "sourcefile": "resources/poster.pdf", "args": ["-l", "eng", "-c", "textonly_pdf=1", "$TMPDIR/000001_ocr.png", "$TMPDIR/000001_ocr_tess", "pdf", "txt"]}
@@ -0,0 +1,118 @@
+The LinnSequencer
+32 Track MIDI Sequence Recorder
+
+The LinnSequencer is a state-of-the-art composition and performance tool for the professional musician. It is
+extremely powerful, yet amazingly simple to learn and use. It’s many remarkable features include:
+
+© Operation is similar to multi-track tape recorder with PLAY, STOP, RECORD, FAST
+FORWARD, REWIND, and LOCATE controls.
+
+e Each of the 100 sequences contains 32 simultaneous, polyphonic tracks. Each track may
+be assigned to one of 16 MIDI channels. Simultaneously plays up to 16 polyphonic
+
+synthesizers!
+
+© Ultra-fast 3!” disk drive stores complex songs in seconds and holds over 110,000 notes
+
+per disk!
+
+¢ One or all tracks may be TRANSPOSED at the touch of a key.
+e Exclusive real-time ERASE function makes editing FAST.
+e Exclusive REPEAT function automatically repeats any held notes at a pre-selected
+
+rhythmic value.
+
+¢ TIMING CORRECTION works during playback and operates without ‘chopping’ notes.
+
+¢ Optional SMPTE time code synchronization.
+
+¢ Optional remote control.
+
+Recording a Sequence
+
+To record a sequence, simply press RECORD and PLAY,
+then play your MIDI keyboard in time to the Sequencer’s
+click track. When the sequence loops back around to bar 1,
+you’ ll hear what you played—only all timing errors will be
+corrected! (Timing correction may be adjusted or defeated).
+Any additional notes played will be added into the track
+—existing notes are not erased while recording!
+
+FAST FORWARD, REWIND, and LOCATE controls
+may be used at any time to quickly access any location in
+your sequence for spot-recording. To overdub a new part,
+select a different track and start recording—while you
+record, the first track will play in perfect sync (unless you
+MUTE it, or SOLO another track). In this way, up to 32
+tracks may be overdubbed! All MIDI effects are recorded
+including pitch bend, modulation, velocity, aftertouch,
+sustain pedal, and program changes!
+
+Editing
+
+To erase a wrong note, simply hold ERASE and press
+the note to be erased just before it plays in the sequence—
+when played back, it will be gone. Notes may also be
+added, erased, or changed using the SINGLE STEP func-
+tion. To overdub notes at specific points within a sequence,
+
+Additional Features
+
+simply use LOCATE, FAST FORWARD, or REWIND to
+find the desired bar number, then start recording.
+
+The INSERT/COPY function allows you to move bars
+from one location to another—in the same sequence or a
+different one. For example, you might insert a copy of the
+first verse between the second chorus and the bridge.
+DELETE BARS operates the same way to remove
+unwanted sections.
+
+Creating a Song
+
+One way to create a song is to record each track all the
+way through (up to 999 bars). Another way is to record
+each basic section (verse, chorus, etc.) in individual
+sequences, then use the CREATE SONG function to “chain”
+them together. CREATE SONG will then automatically
+copy all the parts into a new sequence. If desired, you can
+even set the last few bars to repeat infinitely, for a fadeout.
+
+Composition Without Compromise
+
+The technology you use should never be so complex that
+it interferes with the creative process. That’s precisely why
+the LinnSequencer is designed to let you compose, record
+and edit while devoting your undivided attention to your
+music. See your Linn dealer today for a demonstration!
+
+¢ Simple, easy to learn operation—the 32 character LCD display clearly guides you through all operations. If needed, the
+
+HELP button displays additional explanations.
+
+® Non-destructive recording—existing notes are not erased while recording.
+© Two FOOTSWITCH INPUTS may be assigned to remotely control many of the commonly used functions, including
+
+ERASE, REPEAT, PLAY/STOP, or LOCATE.
+
+¢ Two TRIGGER OUTPUTS may be programmed to output pulses at any selected note value.
+
+@ Will sync to standard LinnDrum or Linn 9000 sync tone.
+
+© Utilizes ultra high-speed, 8 MHz 80186 16 bit computer internally for FAST operation.
+¢ TEMPO may be specified in BEATS-PER-MINUTE or FRAMES-PER-BEAT at 24, 25, or 30 frames per second,
+
+(even drop frame!)
+
+¢ TEMPO may be entered numerically, adjustable in tenths of a Beat-Per-Minute increments, or by tapping quarter notes
+
+on the TAP TEMPO button.
+
+¢ TEMPO CHANGES may be programmed into a sequence, with smooth transitions if desired.
+e Any TIME SIGNATURE may be used, and may be changed within a song.
+
+linn
+Linn Electronics, Inc.
+
+18720 Oxnard Street, Tarzana, CA 91356
+(818) 708-8131 TELEX #298949 LINN UR
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
James R. Barlow	fbaad570c7	v16.4.2 release notes	2024-07-22 15:02:53 -07:00
James R. Barlow	f974e3b3c1	ghostscript: change input filename order for 10.03.1 Ghostscript now expects the pdfa.ps file to precede other files. Fixes #1359.	2024-07-22 14:56:03 -07:00
James R. Barlow	46b49cc176	Suppress missing jbig2dec warning message Windows users can't resolve it easily.	2024-07-22 14:54:40 -07:00
Johannes KalliauerandGitHub	5256e74d0c	Update installation.rst "python -m venv .venv" (#1355 )	2024-07-18 06:28:07 -07:00
James R. Barlow	621d6a0b89	Fix image size calculation when SMask dimensions do not match image Closes [Bug]: Ghostscript rasterizing failed #1351	2024-07-16 13:36:48 -07:00
IrisandGitHub	08be7c8bbe	update arch base-devel install command (#1354 ) the '--needed' flag only installs the package if it isn't installed, otherwise it would reinstall it if already installed.	2024-07-15 13:28:36 -07:00
James R. Barlow	980a5472b6	Fix test failures due to `4dde378`	2024-07-09 15:54:53 -07:00
James R. Barlow	51c618e357	Merge branch 'main' of github.com:ocrmypdf/OCRmyPDF	2024-07-09 14:46:58 -07:00
James R. Barlow	4dde3786c2	Fix KeyError '/Subtype' Fixes #1350	2024-07-09 14:46:47 -07:00
James Barlow	d544342602	Merge branch 'main' of https://github.com/ocrmypdf/OCRmyPDF	2024-07-04 22:59:33 -07:00
James R. Barlow	fac91fca2a	v16.4.1 release notes	2024-06-30 00:11:27 -07:00
James R. Barlow	6edf756849	optimize: trap Hifi..Error Fixes [Bug]: NotImplementedError: not sure how to get colorspace #1315	2024-06-30 00:08:51 -07:00
James R. Barlow	4fb1bb4de6	pipeline: fix typo in message	2024-06-30 00:08:31 -07:00
James Barlow	6a8eb7daaa	docs: page seg mode	2024-06-26 01:16:31 -07:00
James R. Barlow	0544d06c3d	Fix calculation of image printed area (used in finding weighted DPI for OCR) Fixes #1334	2024-06-21 15:13:51 -07:00
James R. Barlow	34c285c9ac	v16.4.0 release notes (3)	2024-06-17 14:40:22 -07:00
James R. Barlow	2f53b27651	Disable progbar for linearizing when --no-progress-bar set Fixes #1332	2024-06-14 14:18:33 -07:00
James R. Barlow	772677746b	Update issue templates to improve data collection for 3rd party apps	2024-06-13 15:25:50 -07:00
James R. Barlow	f0bad87ea6	Restore choco since winget isn't supported (still)	2024-06-13 00:40:05 -07:00
James R. Barlow	44e71f8c14	Attempt to deal with jbig2dec warnings	2024-06-13 00:27:33 -07:00
James R. Barlow	964b30ca26	v16.4.0 release notes (2)	2024-06-11 16:55:33 -07:00
James R. Barlow	214a333e2d	Block Tesseract 5.4.0	2024-06-11 14:44:54 -07:00
James R. Barlow	ec6401ab57	Merge branch 'pr/helkaluin/1300'	2024-06-09 15:34:19 -07:00
James R. Barlow	cbc5e8ce8d	Revert "Delete and de-list snap because it no longer works" This reverts commit `3a721e6578`.	2024-06-09 15:32:34 -07:00
James R. Barlow	a1c4cfe8f1	Merge branch 'main' of github.com:ocrmypdf/OCRmyPDF	2024-06-08 01:37:19 -07:00
James R. Barlow	3a721e6578	Delete and de-list snap because it no longer works	2024-06-08 01:24:47 -07:00
Omid RahaandGitHub	e6b716cdde	Update docker.rst (#1327 )	2024-06-08 01:20:51 -07:00
James R. Barlow	02c39998b8	Note that alpine is now available for arm	2024-06-08 01:20:25 -07:00
James R. Barlow	0774bc7f14	v16.4.0 release notes	2024-06-01 02:01:11 -07:00
James R. Barlow	c6a98b3d0b	Merge branch 'feature/alpine-arm'	2024-06-01 01:55:26 -07:00
James R. Barlow	981bbf1105	optimize: add a recursion guard to avoid chasing cyclic form xobjects	2024-06-01 01:52:21 -07:00
James R. Barlow	2b0c6cfd40	v16.4.0 release ntoes	2024-06-01 00:26:07 -07:00
James R. Barlow	59f6bc8306	More Tesseract-specific language checks to its plugin	2024-06-01 00:15:50 -07:00
James R. Barlow	653c4ffb45	hocr: accept multiple spaces in bounding boxes Fixes #1322	2024-05-31 16:22:27 -07:00
James R. Barlow	d947ca258e	Prevent issuing equ and osd as languages	2024-05-25 01:17:57 -07:00
James R. Barlow	d5ff7f7db9	batch: fix issues flagged by ruff	2024-05-21 01:52:57 -07:00
James R. Barlow	579cef3649	watcher: Ensure output files are .pdf	2024-05-21 01:51:30 -07:00
James R. Barlow	cb2f090c60	v16.3.1 release notes	2024-05-21 01:39:30 -07:00
James R. Barlow	f3d6387bca	Fix "OCR" progress bar not matching actual progress	2024-05-21 01:35:14 -07:00
James R. Barlow	abf9729c61	Semfree test: accept pdfa conversion failed as a valid return code Fixes #1316	2024-05-21 01:26:11 -07:00
James R. Barlow	442e9c9f0d	Add missing codecov token where missed & drop unneeded brew openssl	2024-05-19 01:07:38 -07:00
James R. Barlow	397fad249d	v16.3.0 release notes	2024-05-19 00:50:59 -07:00
James R. Barlow	9a3c5a3f7c	Add progressbar for metadata_fixup Might take time for big files. Pdf.open() potentially is expensive as well, but QPDF doesn't give us progress feedback for that. Closes Show progress during postprocessing #1313	2024-05-19 00:46:50 -07:00
James R. Barlow	950c700274	Fix Ghostscript PDF/A progressbar not displaying	2024-05-19 00:44:21 -07:00
James R. Barlow	26432c38a9	Raise exception if rotate pages threshold adjusted without --rotate-pages Fixes Make usage of --rotate-pages-threshold clearer #1309	2024-05-18 23:49:27 -07:00
James R. Barlow	28be50136c	hocr: If a line box's coords are invalid, log and error and don't render Addresses [Bug]: Crash on multiple .pdf files #1312 Not actually a fix, but at least it will get us better diagnostics. Appears old Tesseract 4.x generates bad line boxes at times.	2024-05-18 23:32:18 -07:00
James R. Barlow	0c62f2de5d	Issue template: check for EOL OS	2024-05-17 19:51:15 -07:00
James R. Barlow	5caf654f22	Add new codecov token	2024-05-11 01:03:41 -07:00
James R. Barlow	205593445e	Change test to run on macos x64 and arm64	2024-05-11 00:13:08 -07:00
James R. Barlow	f25fb8c63a	Merge branch 'main' of github.com:ocrmypdf/OCRmyPDF	2024-05-08 00:39:27 -07:00
James R. Barlow	99c78650b6	Add better error message for PDFs with invalid CTMs Closes #1303	2024-05-07 14:00:30 -07:00
helkaluin	69355886a8	Fix wrong env var for GS path in Snap	2024-04-26 16:45:04 +08:00
Ahmed AbdouandGitHub	08e89e2dbe	Adding language install docs for archlinux (#1296 ) Adding language install docs for archlinux	2024-04-24 14:46:05 -07:00
James R. Barlow	0e013df161	v16.2.0 release notes	2024-04-16 00:37:03 -07:00
James R. Barlow	9ba4e3ab46	Log unusual exceptions when trying to obtain a version Fixes #1262	2024-04-07 14:39:08 -07:00
James R. Barlow	5fdcb7602b	Make downsampling large images that Tesseract would otherwise error on into default behavior Fixes #1281	2024-04-07 13:43:20 -07:00
James R. Barlow	b4db1b741f	optimize: fix handling of [/FlateDecode none] - type images Closes #1271	2024-04-07 01:44:08 -07:00
James R. Barlow	7a8cc21e31	Add support for sidecar output to io.BytesIO Closes #1252	2024-04-07 01:38:55 -07:00
James R. Barlow	0674829d8f	Remove tool.black config	2024-04-07 00:36:52 -07:00
James R. Barlow	315aa0474b	Merge branch 'main' of github.com:ocrmypdf/OCRmyPDF	2024-04-07 00:34:51 -07:00
Ben BeasleyandGitHub	df3451e779	Update the typer[all] dependency to typer-slim[standard] (#1287 ) In 0.12.1, Typer was significantly reorganized. - `typer-slim` is the library (for `import typer`) - `typer-slim[standard]` adds optional dependencies (currently `rich` and `shellingham`, basically equivalent to the old `typer[all]`) - `typer-cli` is the `typer` command-line tool - `typer` is now basically a metapackage that brings in all of the above, and it no longer has an `all` extra Pip will warn about this and proceed, ``` WARNING: typer 0.12.1 does not provide the extra 'all' ``` but there are other tools that will fail hard when asked to resolve a (now) nonexistent extra. Since this project doesn’t need the `typer` command-line tool, it looks like changing the dependency to `typer-slim[standard]` is the best way forward. See https://typer.tiangolo.com/release-notes/#0121 and tiangolo/typer#785 for further discussion and details.	2024-04-07 00:34:34 -07:00
akierigandGitHub	3ba42802d1	added Macports install information (#1286 )	2024-04-07 00:33:57 -07:00
James R. Barlow	d6342cb8c2	Add heif/heic input image support	2024-04-07 00:33:13 -07:00
James R. Barlow	065bddbc6c	Reformat with ruff format	2024-04-07 00:25:32 -07:00
James R. Barlow	067f429dde	Merge branch 'main' of github.com:ocrmypdf/OCRmyPDF	2024-03-26 15:34:00 -07:00
Daniel LovegroveandGitHub	6895c2d70f	Fix Broken Documentation Links (#1275 ) * Update URL for PDFMARK documentation For reference, here is a link to the old PDF: https://web.archive.org/web/20190806035303/https://www.adobe.com/content/dam/acom/en/devnet/acrobat/pdfs/pdfmark_reference.pdf It appears Adobe converted the PDF into a webpage-based document, the wording seems to almost identical b/w the PDF and the website. * Fix cross-references to JBIG2 page * Fix links for Fedora + Arch + HEAD revision install Fedora 39 has been released, and the package tracker no longer includes a release overview for Fedora 37 hence why it was removed here.	2024-03-22 14:38:52 -07:00
James R. Barlow	686481982a	Fix naming of hOCR rendered files	2024-03-22 13:27:20 -07:00
James R. Barlow	a9e1d19b78	v16.1.2 release notes	2024-03-20 12:56:13 -07:00
James R. Barlow	f95aa63718	Merge branch 'main' of github.com:ocrmypdf/OCRmyPDF	2024-03-20 12:26:02 -07:00
James Barlow	855de287b2	Fix test suite failure with Ghostscript >= 10.3 Ghostscript is more picky about a specific case with SMask that cannot be converted to PDF/A Details here https://github.com/ArtifexSoftware/ghostpdl/commit/4dcfae36bb4dcbc4ef3b5e5afc98bcde0d6b9ddc	2024-03-19 17:20:33 -07:00
NilsRoandGitHub	feeb9f213f	batch example: added archive, small corrections and optimizations (#1277 ) * Added archive, small corrections Added a function to archive originals and avoid calling ocrmypdf if they are still is PDF/A. * Added Copyright	2024-03-18 13:22:24 -07:00
Emiel MolenaarandGitHub	e7eb8fa805	Update Dockerfile.alpine (#1268 ) Use Alpine 3.19 as base image to ensure we get GhostScript 10.2.1 to eliminate serious regressions that corrupt PDFs with existing text.	2024-03-13 14:49:42 -07:00
James R. Barlow	8a747f005a	pixels -> megapixels Fixes #1265	2024-02-29 15:31:07 -08:00
James R. Barlow	16ab4a8b4e	Fix error message about missing Python exec Message is unable to start container process: exec: "python": executable file not found in $PATH: unknown. Closes #1260	2024-02-21 23:54:41 -08:00
James R. Barlow	8d30cff4ef	Undo future annotations from watcher.py till Typer fixes its issue Fixes #1258	2024-02-20 19:14:39 -08:00
James R. Barlow	59d5b0d1bd	v16.1.1 release notes	2024-02-15 16:56:25 -08:00
James R. Barlow	9ec0745ab8	Try pypy3.10	2024-02-14 14:25:13 -08:00
James R. Barlow	3a3635f7f9	Python 3.10 cleanup, manual fixes	2024-02-14 12:48:17 -08:00
James R. Barlow	6a746a1cbb	ruff linting/Python 3.10 cleanup	2024-02-14 12:41:51 -08:00
James R. Barlow	906c130f96	Update rust toml settings	2024-02-14 12:32:26 -08:00
James R. Barlow	4a78458821	v16.1.0 release notes	2024-02-12 01:46:21 -08:00
James R. Barlow	fddf3ce2f4	Clarify warnings filter	2024-02-12 01:43:47 -08:00
James R. Barlow	353b34e695	Merge branch 'feature/pageboxes'	2024-02-12 01:41:56 -08:00
James R. Barlow	7d63355c3c	Use hocr renderer for LTR languages	2024-02-12 01:41:41 -08:00
James R. Barlow	42ff7fc842	Fix handling of pages that are restored to correct orientation with /Rotate Appears inversion of CTM was incorrect, introduced in commit `9898904`	2024-02-12 01:32:26 -08:00
James R. Barlow	26470fe16a	Suppress reportlab deprecation warning	2024-02-12 01:17:08 -08:00
James R. Barlow	3b9d4b7f0a	Attempt to deal with oddball mediaboxes	2024-02-11 15:34:54 -08:00
James R. Barlow	11f53fe9a9	First cut at propagating page boxes This would fix the immediate issue, but does not address an offset mediabox.	2024-02-11 15:34:54 -08:00
James R. Barlow	123c0c766f	Mention pipx, install --user --upgrade Closes #1249	2024-02-08 09:42:00 -08:00
James R. Barlow	6a9be2142e	Advise Homebrew on Linux for Ubuntu 20.04	2024-02-07 19:52:50 -08:00
James R. Barlow	0bc350f55e	Merge branch 'main' of github.com:ocrmypdf/OCRmyPDF	2024-02-06 01:28:10 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	7a6edf62ba	Bump codecov/codecov-action from 3 to 4 (#1247 ) Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 3 to 4. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/codecov/codecov-action/compare/v3...v4) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-02-05 03:55:13 -08:00
James R. Barlow	07b6f06f11	optimize: log images with unclear decode tables	2024-02-01 15:42:40 -08:00
nisbet-hubbardandGitHub	2005f622bb	Update gs dependency & instructions for RHEL (#1228 ) * Update gs depedency to 9.54 * Add instructions for RHEL * Update instruction for using python3.11 on RHEL	2024-01-24 10:07:58 -08:00
James R. Barlow	cca04fd799	Document ability to remove all OCR	2024-01-23 23:35:43 -08:00
MaxiandGitHub	75bf8e4ba2	Add autotools automake libtool and leptonica requirements (#1230 ) These tools are required for a successful install of JBIG2 on Ubuntu	2024-01-09 11:58:17 -08:00
Anish SapkotaandGitHub	daabb5b100	Minor english correction in Docs (#1229 )	2024-01-08 10:18:26 -08:00
James R. Barlow	035ebea72f	Change Windows build to use older Ghostscript	2024-01-07 01:56:38 -08:00
James R. Barlow	a499956462	v16.0.4 release notes	2024-01-07 01:39:06 -08:00
James R. Barlow	74d2a156c4	Update cache	2024-01-07 01:35:05 -08:00
James R. Barlow	f87fc7b12d	gs: check for text-corrupting versions	2024-01-07 01:33:49 -08:00
James R. Barlow	602f5632cb	hocr: fix unbound local	2024-01-07 01:33:36 -08:00
James R. Barlow	9fbbcf7599	Merge branch 'feature/hocrdebug'	2024-01-07 01:22:54 -08:00
James R. Barlow	9498f01f59	hocr: Pass text direction for standalone words	2024-01-06 16:21:21 -08:00
James R. Barlow	2c59aca5a1	Fix render_mode ON_TOP when previous content stream does not restore CTM	2024-01-06 16:20:55 -08:00
James R. Barlow	51301d69c9	Change default graft render mode back to underneath It was mistakenly changed to on top at some point	2024-01-06 16:20:30 -08:00
James R. Barlow	7e608fd1df	Use Courier as debug font	2024-01-06 14:24:25 -08:00
James R. Barlow	ecc79315df	Adjust issue template files description	2024-01-04 00:27:06 -08:00
James R. Barlow	14365d10b8	Skip testing oom killer on Python 3.12 Need to investigate further if there's a safe way to do this test.	2024-01-02 16:28:22 -08:00
James R. Barlow	5e5320020f	v16.0.3 release notes	2024-01-02 02:49:54 -08:00
James R. Barlow	103c3e0cd6	Drop Ghostscript CVE warning and adjust version down to 9.54 Most distros have probably now backported the CVE fix. Dropped version to 9.54 so RHEL 9 can use recent ocrmypdf.	2024-01-01 13:02:57 -08:00
Anthony NabilandGitHub	7a1c89edd9	fixed a spelling mistake (#1222 )	2023-12-30 15:53:22 -08:00
James R. Barlow	a5ff3d2f42	Update instructions about custom language packages Closes #1218	2023-12-27 16:46:10 -08:00
James R. Barlow	b71d16dd96	Draw RTL text with reversed matrix instead of reversing characters	2023-12-24 01:34:03 -08:00
James R. Barlow	fd593eb5e9	Reversing character order for RTL helps output	2023-12-24 01:24:44 -08:00
James R. Barlow	a0b98abb94	Improve and externalize debug rendering settings	2023-12-24 00:10:09 -08:00
James R. Barlow	18353e1e94	Also fix space rendering	2023-12-24 00:09:47 -08:00
James R. Barlow	9adcad84da	Overhaul debug rendering to support possible use of Helvetica for testing	2023-12-23 23:47:35 -08:00
James R. Barlow	f2714586d8	Fix upside down glyphless font	2023-12-23 23:44:37 -08:00
James R. Barlow	0b6fb62967	Update issue template	2023-12-23 16:21:17 -08:00
James R. Barlow	1db8b0b943	Temporarily revert default renderer to sandwich while investigating regressions in macOS Preview	2023-12-23 16:19:44 -08:00
James R. Barlow	f38aebb3d5	Update issue template	2023-12-23 15:46:43 -08:00
James R. Barlow	7162c36d37	Name artifacts explicitly	2023-12-20 13:53:31 -08:00
James R. Barlow	f4d4ea46c8	Update artifact actions	2023-12-20 12:44:43 -08:00
James R. Barlow	2fd1a0f178	v16.0.1 release notes	2023-12-20 12:33:41 -08:00
James R. Barlow	73ed33a086	Tighten dependencies	2023-12-20 12:33:18 -08:00
James R. Barlow	e6095a9949	Fix text rendering issue with new hOCR text renderer	2023-12-20 12:26:06 -08:00
James R. Barlow	16f05af401	Fix release notes - drop rc from version	2023-12-18 20:08:45 -08:00