Compare commits

...
29 Commits
Author SHA1 Message Date
James R. Barlow 21fb6c82ca v13.6.2 release notes 2022-07-25 23:48:22 -07:00
James R. Barlow 27f7b9f255 Fix missing TypeAlias on <3.10 2022-07-25 16:32:55 -07:00
James R. Barlow 6f31a92ffb _windows: Avoid messy next()/StopIteration in generator 2022-07-23 15:40:05 -07:00
James R. Barlow da2276788c Fix _windows module typing 2022-07-23 15:32:14 -07:00
James R. Barlow dc6f1a266a Modernize type annotations 2022-07-23 00:39:24 -07:00
James R. Barlow 9c8ddd853d Typing adjustments 2022-07-23 00:07:50 -07:00
James R. Barlow 014d0302f2 Pre-commit: update 2022-07-22 23:49:51 -07:00
James R. Barlow 65568b3dbc Merge commit '05e2b6698dacce898abc356222124c7a1609f569' 2022-07-18 14:12:01 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
05e2b6698d build(deps): bump docker/setup-buildx-action from 1 to 2 (#994)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 1 to 2.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-18 14:10:45 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2f8e0f7d95 build(deps): bump actions/download-artifact from 2 to 3 (#995)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-18 14:10:33 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
7e7553fc6b build(deps): bump actions/setup-python from 2 to 4 (#996)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-18 14:10:19 -07:00
James R. BarlowandH. Felix Wittmann 6b425aaebe Add shim for cancel_futures in older Pythons
Thanks @hfwittmann

Closes #993

Co-authored-by: H. Felix Wittmann <hfwittmann@users.noreply.github.com>
2022-07-17 16:02:45 -07:00
James R. Barlow 725af43bc3 docs: fix badges for debian 2022-07-12 14:56:00 -07:00
James R. Barlow 5c60309609 Merge remote-tracking branch 'origin/master' 2022-07-12 02:23:17 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
7d5cd55909 build(deps): bump codecov/codecov-action from 1 to 3 (#990)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1 to 3.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1...v3)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-12 02:23:02 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
ec4a06fad2 build(deps): bump docker/login-action from 1 to 2 (#988)
Bumps [docker/login-action](https://github.com/docker/login-action) from 1 to 2.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-12 02:22:53 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
48c6e2318e build(deps): bump docker/setup-qemu-action from 1 to 2 (#987)
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 1 to 2.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-12 02:22:38 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2eafa5e070 build(deps): bump actions/upload-artifact from 2 to 3 (#989)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-12 02:21:27 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
6aa04d7569 build(deps): bump actions/checkout from 2 to 3 (#991)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-12 02:21:12 -07:00
James R. Barlow b9bffa97ba Fix ABCMeta typing 2022-07-12 02:20:22 -07:00
James R. Barlow 777ba99ccc v13.6.1 release notes 2022-07-12 02:11:26 -07:00
James R. Barlow 001b3324f1 Require setuptools-scm 7.0.5 to ensure sdists work alright 2022-07-12 02:09:37 -07:00
James R. Barlow b1f2d257e2 Typing improvements 2022-07-09 02:24:19 -07:00
James R. Barlow 24a08e5170 hocrtransform: suppress deprecation warning from importing reportlab 2022-07-07 02:09:47 -07:00
James R. Barlow adf97fd82c pipeline: eliminate useless test 2022-07-07 02:08:14 -07:00
James R. Barlow a60ea72517 docs: improve remarks about lossy JBIG2 and lossy image transformations 2022-07-04 23:01:20 -07:00
James R. Barlow 59f967cdcd Activate GHA dependabot 2022-07-04 22:43:19 -07:00
James R. Barlow 6e439ee89e Modernize setuptools usage and setuptools_scm 2022-07-04 02:26:14 -07:00
James R. Barlow da38e1b035 docs: adding missing plugins 2022-07-04 02:16:20 -07:00
96 changed files with 484 additions and 261 deletions
+3
View File
@@ -1 +1,4 @@
node: $Format:%H$
node-date: $Format:%cI$
describe-name: $Format:%(describe:tags=true)$
ref-names: $Format:%D$ ref-names: $Format:%D$
+11
View File
@@ -0,0 +1,11 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
- package-ecosystem: "github-actions" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "weekly"
+17 -17
View File
@@ -41,11 +41,11 @@ jobs:
PYTHON: ${{ matrix.python }} PYTHON: ${{ matrix.python }}
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v3
with: with:
fetch-depth: "0" # 0=all, needed for setuptools-scm to resolve version tags fetch-depth: "0" # 0=all, needed for setuptools-scm to resolve version tags
- uses: actions/setup-python@v2 - uses: actions/setup-python@v4
name: Install Python name: Install Python
with: with:
python-version: ${{ matrix.python }} python-version: ${{ matrix.python }}
@@ -111,7 +111,7 @@ jobs:
python -m pytest --cov-report xml --cov=ocrmypdf --cov=tests/ -n0 tests/ python -m pytest --cov-report xml --cov=ocrmypdf --cov=tests/ -n0 tests/
- name: Upload coverage to Codecov - name: Upload coverage to Codecov
uses: codecov/codecov-action@v1 uses: codecov/codecov-action@v3
with: with:
files: ./coverage.xml files: ./coverage.xml
env_vars: OS,PYTHON env_vars: OS,PYTHON
@@ -129,11 +129,11 @@ jobs:
PYTHON: ${{ matrix.python }} PYTHON: ${{ matrix.python }}
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v3
with: with:
fetch-depth: "0" # 0=all, needed for setuptools-scm to resolve version tags fetch-depth: "0" # 0=all, needed for setuptools-scm to resolve version tags
- uses: actions/setup-python@v2 - uses: actions/setup-python@v4
name: Install Python name: Install Python
with: with:
python-version: ${{ matrix.python }} python-version: ${{ matrix.python }}
@@ -166,7 +166,7 @@ jobs:
python -m pytest --cov-report xml --cov=ocrmypdf --cov=tests/ -n0 tests/ python -m pytest --cov-report xml --cov=ocrmypdf --cov=tests/ -n0 tests/
- name: Upload coverage to Codecov - name: Upload coverage to Codecov
uses: codecov/codecov-action@v1 uses: codecov/codecov-action@v3
with: with:
files: ./coverage.xml files: ./coverage.xml
env_vars: OS,PYTHON env_vars: OS,PYTHON
@@ -184,11 +184,11 @@ jobs:
PYTHON: ${{ matrix.python }} PYTHON: ${{ matrix.python }}
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v3
with: with:
fetch-depth: "0" # 0=all, needed for setuptools-scm to resolve version tags fetch-depth: "0" # 0=all, needed for setuptools-scm to resolve version tags
- uses: actions/setup-python@v2 - uses: actions/setup-python@v4
name: Install Python name: Install Python
with: with:
python-version: ${{ matrix.python }} python-version: ${{ matrix.python }}
@@ -208,7 +208,7 @@ jobs:
python -m pytest --cov-report xml --cov=ocrmypdf --cov=tests/ -n0 tests/ python -m pytest --cov-report xml --cov=ocrmypdf --cov=tests/ -n0 tests/
- name: Upload coverage to Codecov - name: Upload coverage to Codecov
uses: codecov/codecov-action@v1 uses: codecov/codecov-action@v3
with: with:
files: ./coverage.xml files: ./coverage.xml
env_vars: OS,PYTHON env_vars: OS,PYTHON
@@ -217,11 +217,11 @@ jobs:
name: Build sdist and wheels name: Build sdist and wheels
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v3
with: with:
fetch-depth: "0" # 0=all, needed for setuptools-scm to resolve version tags fetch-depth: "0" # 0=all, needed for setuptools-scm to resolve version tags
- uses: actions/setup-python@v2 - uses: actions/setup-python@v4
name: Install Python name: Install Python
with: with:
python-version: "3.7" python-version: "3.7"
@@ -232,7 +232,7 @@ jobs:
python setup.py sdist python setup.py sdist
python setup.py bdist_wheel python setup.py bdist_wheel
- uses: actions/upload-artifact@v2 - uses: actions/upload-artifact@v3
with: with:
path: | path: |
./dist/*.whl ./dist/*.whl
@@ -244,7 +244,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/v') if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/v')
steps: steps:
- uses: actions/download-artifact@v2 - uses: actions/download-artifact@v3
with: with:
name: artifact name: artifact
path: dist path: dist
@@ -274,22 +274,22 @@ jobs:
- name: Set image name - name: Set image name
run: echo "DOCKER_IMAGE_NAME=ocrmypdf" >> $GITHUB_ENV run: echo "DOCKER_IMAGE_NAME=ocrmypdf" >> $GITHUB_ENV
- uses: actions/checkout@v2 - uses: actions/checkout@v3
with: with:
fetch-depth: "0" # 0=all, needed for setuptools-scm to resolve version tags fetch-depth: "0" # 0=all, needed for setuptools-scm to resolve version tags
- name: Login to Docker Hub - name: Login to Docker Hub
uses: docker/login-action@v1 uses: docker/login-action@v2
with: with:
username: jbarlow83 username: jbarlow83
password: ${{ secrets.DOCKERHUB_TOKEN }} password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up QEMU - name: Set up QEMU
uses: docker/setup-qemu-action@v1 uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx - name: Set up Docker Buildx
id: buildx id: buildx
uses: docker/setup-buildx-action@v1 uses: docker/setup-buildx-action@v2
- name: Print image tag - name: Print image tag
run: echo "Building image ${DOCKER_REPOSITORY}/${DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_TAG}" run: echo "Building image ${DOCKER_REPOSITORY}/${DOCKER_IMAGE_NAME}:${DOCKER_IMAGE_TAG}"
+6 -6
View File
@@ -1,6 +1,6 @@
repos: repos:
- repo: https://github.com/pre-commit/pre-commit-hooks - repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.2.0 rev: v4.3.0
hooks: hooks:
- id: check-case-conflict - id: check-case-conflict
- id: check-merge-conflict - id: check-merge-conflict
@@ -11,23 +11,23 @@ repos:
rev: 5.10.1 rev: 5.10.1
hooks: hooks:
- id: isort - id: isort
args: ["--profile", "black"] args: ["--profile", "black", "-a", "from __future__ import annotations"]
- repo: https://github.com/psf/black - repo: https://github.com/psf/black
rev: 22.3.0 rev: 22.6.0
hooks: hooks:
- id: black - id: black
language_version: python language_version: python
- repo: https://github.com/asottile/setup-cfg-fmt - repo: https://github.com/asottile/setup-cfg-fmt
rev: v1.20.1 rev: v1.20.2
hooks: hooks:
- id: setup-cfg-fmt - id: setup-cfg-fmt
- repo: https://github.com/asottile/pyupgrade - repo: https://github.com/asottile/pyupgrade
rev: v2.32.1 rev: v2.37.2
hooks: hooks:
- id: pyupgrade - id: pyupgrade
args: ["--py37-plus"] args: ["--py37-plus"]
- repo: https://github.com/pre-commit/mirrors-mypy - repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.950 rev: v0.971
hooks: hooks:
- id: mypy - id: mypy
additional_dependencies: additional_dependencies:
+3 -1
View File
@@ -223,7 +223,9 @@ The ``hocr`` renderer
The ``hocr`` renderer works with older versions of Tesseract. The image The ``hocr`` renderer works with older versions of Tesseract. The image
layer is copied from the original PDF page if possible, avoiding layer is copied from the original PDF page if possible, avoiding
potentially lossy transcoding or loss of other PDF information. If potentially lossy transcoding or loss of other PDF information. If
preprocessing is specified, then the image layer is a new PDF. preprocessing is specified, then the image layer is a new PDF. (You may
need to disable PDF/A conversion nad optimization to eliminate all
lossy transformations.)
Unlike ``sandwich`` this renderer is implemented within OCRmyPDF; anyone Unlike ``sandwich`` this renderer is implemented within OCRmyPDF; anyone
looking to customize how OCR is presented should look here. A major looking to customize how OCR is presented should look here. A major
+8
View File
@@ -340,6 +340,9 @@ levels in the GCC compiler.
- Enables lossless optimizations, such as transcoding images to more - Enables lossless optimizations, such as transcoding images to more
efficient formats. Also compress other uncompressed objects in the efficient formats. Also compress other uncompressed objects in the
PDF and enables the more efficient "object streams" within the PDF. PDF and enables the more efficient "object streams" within the PDF.
(If ``--jbig2-lossy`` is issued, then lossy JBIG2 optimization is used.
The decision to use lossy JBIG2 is separate from standard optimization
settings.)
* - ``--optimize 2`` * - ``--optimize 2``
- All of the above, and enables lossy optimizations and color quantization. - All of the above, and enables lossy optimizations and color quantization.
* - ``--optimize 3`` * - ``--optimize 3``
@@ -360,3 +363,8 @@ fo a PDF.
ocrmypdf --optimize 3 in.pdf out.pdf # Make it small ocrmypdf --optimize 3 in.pdf out.pdf # Make it small
Some users may consider enabling lossy JBIG2. See: :ref:`jbig2-lossy`. Some users may consider enabling lossy JBIG2. See: :ref:`jbig2-lossy`.
.. note::
Image processing and PDF/A conversion can also introduce lossy transformations
to your PDF images, even when ``--optimize 1`` is in use.
+5 -5
View File
@@ -43,11 +43,11 @@ Installing on Linux
Debian and Ubuntu 18.04 or newer Debian and Ubuntu 18.04 or newer
-------------------------------- --------------------------------
.. |deb-stable| image:: https://repology.org/badge/version-for-repo/debian_stable/ocrmypdf.svg .. |deb-11| image:: https://repology.org/badge/version-for-repo/debian_11/ocrmypdf.svg
:alt: Debian stable :alt: Debian 11
.. |deb-testing| image:: https://repology.org/badge/version-for-repo/debian_testing/ocrmypdf.svg .. |deb-12| image:: https://repology.org/badge/version-for-repo/debian_12/ocrmypdf.svg
:alt: Debian testing :alt: Debian 12
.. |deb-unstable| image:: https://repology.org/badge/version-for-repo/debian_unstable/ocrmypdf.svg .. |deb-unstable| image:: https://repology.org/badge/version-for-repo/debian_unstable/ocrmypdf.svg
:alt: Debian unstable :alt: Debian unstable
@@ -66,7 +66,7 @@ Debian and Ubuntu 18.04 or newer
+-----------------------------------------------+ +-----------------------------------------------+
| |latest| | | |latest| |
+-----------------------------------------------+ +-----------------------------------------------+
| |deb-stable| |deb-testing| |deb-unstable| | | |deb-11| |deb-12| |deb-unstable| |
+-----------------------------------------------+ +-----------------------------------------------+
| |ubu-1804| |ubu-2004| |ubu-2204| | | |ubu-1804| |ubu-2004| |ubu-2204| |
+-----------------------------------------------+ +-----------------------------------------------+
+13 -1
View File
@@ -162,6 +162,11 @@ Examples
management system. management system.
Suppressing or overriding other plugins
---------------------------------------
.. autofunction:: ocrmypdf.pluginspec.initialize
Custom command line arguments Custom command line arguments
----------------------------- -----------------------------
@@ -172,7 +177,7 @@ Custom command line arguments
Execution and progress reporting Execution and progress reporting
-------------------------------- --------------------------------
.. autoclass: ocrmypdf.pluginspec.Executor .. autoclass:: ocrmypdf.pluginspec.Executor
:members: :members:
.. autofunction:: ocrmypdf.pluginspec.get_logging_console .. autofunction:: ocrmypdf.pluginspec.get_logging_console
@@ -216,3 +221,10 @@ PDF/A production
---------------- ----------------
.. autofunction:: ocrmypdf.pluginspec.generate_pdfa .. autofunction:: ocrmypdf.pluginspec.generate_pdfa
PDF optimization
----------------
.. autofunction:: ocrmypdf.pluginspec.optimize_pdf
.. autofunction:: ocrmypdf.pluginspec.is_optimization_enabled
+15
View File
@@ -24,6 +24,21 @@ tagged yet.
.. |OCRmyPDF PyPI| image:: https://img.shields.io/pypi/v/ocrmypdf.svg .. |OCRmyPDF PyPI| image:: https://img.shields.io/pypi/v/ocrmypdf.svg
v13.6.2
=======
- Added a shim to prevent an "error during error handling" for Python 3.7 and 3.8.
- Modernized some type annotations.
- Improved annotations on our _windows module to help IDEs and mypy figure out what
we're doing.
v13.6.1
=======
- Require setuptools-scm 7.0.5 to avoid possible issues with source distributions in
earlier versions of setuptools-scm.
- Suppress a spurious warning, improve tests, improve typing and other miscellany.
v13.6.0 v13.6.0
======= =======
+2 -1
View File
@@ -19,8 +19,9 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE. # SOFTWARE.
# This script must be edited to meet your needs. from __future__ import annotations
# This script must be edited to meet your needs.
import logging import logging
import os import os
import sys import sys
+2
View File
@@ -37,6 +37,8 @@ To use this as an API:
) )
""" """
from __future__ import annotations
import logging import logging
from PIL import Image from PIL import Image
+2 -1
View File
@@ -19,8 +19,9 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE. # SOFTWARE.
# This script must be edited to meet your needs. from __future__ import annotations
# This script must be edited to meet your needs.
import logging import logging
import os import os
import shutil import shutil
+2
View File
@@ -20,6 +20,8 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE. # SOFTWARE.
from __future__ import annotations
import json import json
import logging import logging
import os import os
+2
View File
@@ -24,6 +24,8 @@ to emphasize that SaaS deployments should make sure they comply with
Ghostscript's license as well as OCRmyPDF's. Ghostscript's license as well as OCRmyPDF's.
""" """
from __future__ import annotations
import os import os
import shlex import shlex
from subprocess import PIPE, run from subprocess import PIPE, run
+5 -6
View File
@@ -1,14 +1,12 @@
[build-system] [build-system]
requires = [ requires = [
"setuptools >= 30.3.0", "setuptools >= 52",
"wheel", "setuptools_scm[toml] >= 7.0.5",
"setuptools_scm[toml] >= 3.4", "wheel"
"setuptools_scm_git_archive"
] ]
build-backend = "setuptools.build_meta" build-backend = "setuptools.build_meta"
[tool.setuptools_scm] [tool.setuptools_scm]
version_scheme = "post-release"
[tool.black] [tool.black]
line-length = 88 line-length = 88
@@ -96,7 +94,8 @@ module = [
'pdfminer.*', 'pdfminer.*',
'reportlab.*', 'reportlab.*',
'fitz', 'fitz',
'libxmp.utils' 'libxmp.utils',
'importlib_metadata'
] ]
ignore_missing_imports = true ignore_missing_imports = true
+3
View File
@@ -56,6 +56,7 @@ install_requires =
tqdm>=4 tqdm>=4
importlib-metadata>=4;python_version<'3.8' # until Python 3.8 importlib-metadata>=4;python_version<'3.8' # until Python 3.8
importlib-resources>=5;python_version<'3.9' # until Python 3.9 importlib-resources>=5;python_version<'3.9' # until Python 3.9
typing-extensions>=4;python_version<'3.10'
python_requires = >=3.7 python_requires = >=3.7
include_package_data = True include_package_data = True
package_dir = package_dir =
@@ -86,6 +87,8 @@ test =
pytest-cov>=2.11.1 pytest-cov>=2.11.1
pytest-xdist>=2.2.0 pytest-xdist>=2.2.0
python-xmp-toolkit==2.0.1 # also requires apt-get install libexempi3 python-xmp-toolkit==2.0.1 # also requires apt-get install libexempi3
types-Pillow
types-humanfriendly
watcher = watcher =
watchdog>=1.0.2 watchdog>=1.0.2
webservice = webservice =
+3 -8
View File
@@ -6,13 +6,8 @@
"""setup.py to support older setuptools and pip.""" """setup.py to support older setuptools and pip."""
from __future__ import annotations
from setuptools import setup from setuptools import setup
# Minimal setup to support older setuptools/setuptools_scm setup()
setup(
setup_requires=[ # can be removed whenever we can drop pip 9 support
'setuptools_scm', # so that version will work
'setuptools_scm_git_archive', # enable version from github tarballs
],
use_scm_version={'version_scheme': 'post-release'},
)
+2
View File
@@ -6,6 +6,8 @@
"""Adds OCR layer to PDFs.""" """Adds OCR layer to PDFs."""
from __future__ import annotations
from pluggy import HookimplMarker as _HookimplMarker from pluggy import HookimplMarker as _HookimplMarker
from ocrmypdf import helpers, hocrtransform, pdfa, pdfinfo from ocrmypdf import helpers, hocrtransform, pdfa, pdfinfo
+2
View File
@@ -7,6 +7,8 @@
"""ocrmypdf command line entrypoint.""" """ocrmypdf command line entrypoint."""
from __future__ import annotations
import logging import logging
import os import os
import signal import signal
+7 -5
View File
@@ -6,9 +6,11 @@
"""OCRmyPDF concurrency abstractions.""" """OCRmyPDF concurrency abstractions."""
from __future__ import annotations
import threading import threading
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from typing import Callable, Iterable, Optional from typing import Callable, Iterable
def _task_noop(*_args, **_kwargs): def _task_noop(*_args, **_kwargs):
@@ -47,10 +49,10 @@ class Executor(ABC):
use_threads: bool, use_threads: bool,
max_workers: int, max_workers: int,
tqdm_kwargs: dict, tqdm_kwargs: dict,
worker_initializer: Optional[Callable] = None, worker_initializer: Callable | None = None,
task: Optional[Callable] = None, task: Callable | None = None,
task_arguments: Optional[Iterable] = None, task_arguments: Iterable | None = None,
task_finished: Optional[Callable] = None, task_finished: Callable | None = None,
) -> None: ) -> None:
""" """
Set up parallel execution and progress reporting. Set up parallel execution and progress reporting.
+2
View File
@@ -6,3 +6,5 @@
"""Manage third party executables""" """Manage third party executables"""
from __future__ import annotations
+4 -3
View File
@@ -7,6 +7,8 @@
"""Interface to Ghostscript executable""" """Interface to Ghostscript executable"""
from __future__ import annotations
import logging import logging
import os import os
import re import re
@@ -15,7 +17,6 @@ from io import BytesIO
from os import fspath from os import fspath
from pathlib import Path from pathlib import Path
from subprocess import PIPE, CalledProcessError from subprocess import PIPE, CalledProcessError
from typing import Optional
from PIL import Image, UnidentifiedImageError from PIL import Image, UnidentifiedImageError
@@ -77,8 +78,8 @@ def rasterize_pdf(
raster_device: str, raster_device: str,
raster_dpi: Resolution, raster_dpi: Resolution,
pageno: int = 1, pageno: int = 1,
page_dpi: Optional[Resolution] = None, page_dpi: Resolution | None = None,
rotation: Optional[int] = None, rotation: int | None = None,
filter_vector: bool = False, filter_vector: bool = False,
): ):
"""Rasterize one page of a PDF at resolution raster_dpi in canvas units.""" """Rasterize one page of a PDF at resolution raster_dpi in canvas units."""
+2
View File
@@ -7,6 +7,8 @@
"""Interface to jbig2 executable""" """Interface to jbig2 executable"""
from __future__ import annotations
from subprocess import PIPE from subprocess import PIPE
from ocrmypdf.exceptions import MissingDependencyError from ocrmypdf.exceptions import MissingDependencyError
+2
View File
@@ -7,6 +7,8 @@
"""Interface to pngquant executable""" """Interface to pngquant executable"""
from __future__ import annotations
from contextlib import contextmanager from contextlib import contextmanager
from io import BytesIO from io import BytesIO
from pathlib import Path from pathlib import Path
+11 -10
View File
@@ -7,13 +7,14 @@
"""Interface to Tesseract executable""" """Interface to Tesseract executable"""
from __future__ import annotations
import logging import logging
import re import re
from math import pi from math import pi
from os import fspath from os import fspath
from pathlib import Path from pathlib import Path
from subprocess import PIPE, STDOUT, CalledProcessError, TimeoutExpired from subprocess import PIPE, STDOUT, CalledProcessError, TimeoutExpired
from typing import Dict, List, Optional
from packaging.version import Version from packaging.version import Version
from PIL import Image from PIL import Image
@@ -46,7 +47,7 @@ HOCR_TEMPLATE = """<?xml version="1.0" encoding="UTF-8"?>
</html> </html>
""" """
TESSERACT_THRESHOLDING_METHODS: Dict[str, int] = { TESSERACT_THRESHOLDING_METHODS: dict[str, int] = {
'auto': 0, 'auto': 0,
'otsu': 0, 'otsu': 0,
'adaptive-otsu': 1, 'adaptive-otsu': 1,
@@ -162,7 +163,7 @@ def get_languages():
return {lang.strip() for lang in rest} return {lang.strip() for lang in rest}
def tess_base_args(langs: List[str], engine_mode: Optional[int]) -> List[str]: def tess_base_args(langs: list[str], engine_mode: int | None) -> list[str]:
args = ['tesseract'] args = ['tesseract']
if langs: if langs:
args.extend(['-l', '+'.join(langs)]) args.extend(['-l', '+'.join(langs)])
@@ -171,7 +172,7 @@ def tess_base_args(langs: List[str], engine_mode: Optional[int]) -> List[str]:
return args return args
def _parse_tesseract_output(binary_output: bytes) -> Dict[str, str]: def _parse_tesseract_output(binary_output: bytes) -> dict[str, str]:
def gen(): def gen():
for line in binary_output.decode().splitlines(): for line in binary_output.decode().splitlines():
line = line.strip() line = line.strip()
@@ -183,7 +184,7 @@ def _parse_tesseract_output(binary_output: bytes) -> Dict[str, str]:
def get_orientation( def get_orientation(
input_file: Path, engine_mode: Optional[int], timeout: float input_file: Path, engine_mode: int | None, timeout: float
) -> OrientationConfidence: ) -> OrientationConfidence:
args_tesseract = tess_base_args(['osd'], engine_mode) + [ args_tesseract = tess_base_args(['osd'], engine_mode) + [
'--psm', '--psm',
@@ -215,7 +216,7 @@ def get_orientation(
def get_deskew( def get_deskew(
input_file: Path, languages: List[str], engine_mode: Optional[int], timeout: float input_file: Path, languages: list[str], engine_mode: int | None, timeout: float
) -> float: ) -> float:
"""Gets angle to deskew this page, in degrees.""" """Gets angle to deskew this page, in degrees."""
args_tesseract = tess_base_args(languages, engine_mode) + [ args_tesseract = tess_base_args(languages, engine_mode) + [
@@ -306,9 +307,9 @@ def generate_hocr(
input_file: Path, input_file: Path,
output_hocr: Path, output_hocr: Path,
output_text: Path, output_text: Path,
languages: List[str], languages: list[str],
engine_mode: int, engine_mode: int,
tessconfig: List[str], tessconfig: list[str],
timeout: float, timeout: float,
pagesegmode: int, pagesegmode: int,
thresholding: int, thresholding: int,
@@ -372,9 +373,9 @@ def generate_pdf(
input_file: Path, input_file: Path,
output_pdf: Path, output_pdf: Path,
output_text: Path, output_text: Path,
languages: List[str], languages: list[str],
engine_mode: int, engine_mode: int,
tessconfig: List[str], tessconfig: list[str],
timeout: float, timeout: float,
pagesegmode: int, pagesegmode: int,
thresholding: int, thresholding: int,
+8 -7
View File
@@ -5,9 +5,10 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
# unpaper documentation: # unpaper documentation:
# https://github.com/Flameeyes/unpaper/blob/master/doc/basic-concepts.md # https://github.com/Flameeyes/unpaper/blob/master/doc/basic-concepts.md
"""Interface to unpaper executable""" """Interface to unpaper executable"""
import logging import logging
@@ -18,7 +19,7 @@ from contextlib import contextmanager
from decimal import Decimal from decimal import Decimal
from pathlib import Path from pathlib import Path
from subprocess import PIPE, STDOUT from subprocess import PIPE, STDOUT
from typing import Iterator, List, Optional, Tuple, Union from typing import Iterator, Union
from PIL import Image from PIL import Image
@@ -75,7 +76,7 @@ def version() -> str:
SUPPORTED_MODES = {'1', 'L', 'RGB'} SUPPORTED_MODES = {'1', 'L', 'RGB'}
def _convert_image(im: Image.Image) -> Tuple[Image.Image, bool]: def _convert_image(im: Image.Image) -> tuple[Image.Image, bool]:
im_modified = False im_modified = False
if im.mode not in SUPPORTED_MODES: if im.mode not in SUPPORTED_MODES:
@@ -99,7 +100,7 @@ def _convert_image(im: Image.Image) -> Tuple[Image.Image, bool]:
@contextmanager @contextmanager
def _setup_unpaper_io(input_file: Path) -> Iterator[Tuple[Path, Path, Path]]: def _setup_unpaper_io(input_file: Path) -> Iterator[tuple[Path, Path, Path]]:
with Image.open(input_file) as im: with Image.open(input_file) as im:
if im.width * im.height >= UNPAPER_IMAGE_PIXEL_LIMIT: if im.width * im.height >= UNPAPER_IMAGE_PIXEL_LIMIT:
raise UnpaperImageTooLargeError(w=im.width, h=im.height) raise UnpaperImageTooLargeError(w=im.width, h=im.height)
@@ -121,7 +122,7 @@ def _setup_unpaper_io(input_file: Path) -> Iterator[Tuple[Path, Path, Path]]:
def run_unpaper( def run_unpaper(
input_file: Path, output_file: Path, *, dpi: DecFloat, mode_args: List[str] input_file: Path, output_file: Path, *, dpi: DecFloat, mode_args: list[str]
) -> None: ) -> None:
args_unpaper = ['unpaper', '-v', '--dpi', str(round(dpi, 6))] + mode_args args_unpaper = ['unpaper', '-v', '--dpi', str(round(dpi, 6))] + mode_args
@@ -154,7 +155,7 @@ def run_unpaper(
) from e ) from e
def validate_custom_args(args: str) -> List[str]: def validate_custom_args(args: str) -> list[str]:
unpaper_args = shlex.split(args) unpaper_args = shlex.split(args)
if any(('/' in arg or arg == '.' or arg == '..') for arg in unpaper_args): if any(('/' in arg or arg == '.' or arg == '..') for arg in unpaper_args):
raise ValueError('No filenames allowed in --unpaper-args') raise ValueError('No filenames allowed in --unpaper-args')
@@ -166,7 +167,7 @@ def clean(
output_file: Path, output_file: Path,
*, *,
dpi: DecFloat, dpi: DecFloat,
unpaper_args: Optional[List[str]] = None, unpaper_args: list[str] | None = None,
) -> Path: ) -> Path:
default_args = [ default_args = [
'--layout', '--layout',
+4 -3
View File
@@ -6,10 +6,11 @@
"""For grafting text-only PDF pages onto freeform PDF pages.""" """For grafting text-only PDF pages onto freeform PDF pages."""
from __future__ import annotations
import logging import logging
from contextlib import suppress from contextlib import suppress
from pathlib import Path from pathlib import Path
from typing import Optional
from pikepdf import ( from pikepdf import (
Dictionary, Dictionary,
@@ -103,8 +104,8 @@ class OcrGrafter:
self, self,
*, *,
pageno: int, pageno: int,
image: Optional[Path], image: Path | None,
textpdf: Optional[Path], textpdf: Path | None,
autorotate_correction: int, autorotate_correction: int,
): ):
if textpdf and not self.font: if textpdf and not self.font:
+3 -1
View File
@@ -6,6 +6,8 @@
"""Defines context objects that are passed to child processes/threads.""" """Defines context objects that are passed to child processes/threads."""
from __future__ import annotations
import os import os
import shutil import shutil
import sys import sys
@@ -50,7 +52,7 @@ class PdfContext:
""" """
return self.work_folder / name return self.work_folder / name
def get_page_contexts(self) -> Iterator['PageContext']: def get_page_contexts(self) -> Iterator[PageContext]:
"""Get all ``PageContext`` for this PDF.""" """Get all ``PageContext`` for this PDF."""
npages = len(self.pdfinfo) npages = len(self.pdfinfo)
for n in range(npages): for n in range(npages):
+2
View File
@@ -6,6 +6,8 @@
"""Logging support classes.""" """Logging support classes."""
from __future__ import annotations
import logging import logging
from contextlib import suppress from contextlib import suppress
+15 -14
View File
@@ -6,6 +6,8 @@
"""OCRmyPDF page processing pipeline functions.""" """OCRmyPDF page processing pipeline functions."""
from __future__ import annotations
import logging import logging
import os import os
import re import re
@@ -14,7 +16,7 @@ from contextlib import suppress
from datetime import datetime, timezone from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
from shutil import copyfileobj from shutil import copyfileobj
from typing import Dict, Iterable, Optional from typing import Iterable
import img2pdf import img2pdf
import pikepdf import pikepdf
@@ -41,7 +43,7 @@ from ocrmypdf.pdfinfo import Colorspace, Encoding, PdfInfo
# Remove this workaround when we require Pillow >= 10 # Remove this workaround when we require Pillow >= 10
try: try:
BICUBIC = Image.Resampling.BICUBIC # type: ignore BICUBIC = Image.Resampling.BICUBIC # type: ignore
except AttributeError: except AttributeError: # pragma: no cover
# Pillow 9 shim # Pillow 9 shim
BICUBIC = Image.BICUBIC # type: ignore BICUBIC = Image.BICUBIC # type: ignore
@@ -494,7 +496,7 @@ def preprocess_deskew(input_file: Path, page_context: PageContext):
deskewed = im.rotate( deskewed = im.rotate(
deskew_angle_degrees, deskew_angle_degrees,
resample=BICUBIC, resample=BICUBIC,
fillcolor=ImageColor.getcolor('white', mode=im.mode), fillcolor=ImageColor.getcolor('white', mode=im.mode), # type: ignore
) )
deskewed.save(output_file, dpi=dpi) deskewed.save(output_file, dpi=dpi)
@@ -665,7 +667,7 @@ def ocr_engine_textonly_pdf(input_image: Path, page_context: PageContext):
return (output_pdf, output_text) return (output_pdf, output_text)
def get_docinfo(base_pdf: pikepdf.Pdf, context: PdfContext) -> Dict[str, str]: def get_docinfo(base_pdf: pikepdf.Pdf, context: PdfContext) -> dict[str, str]:
options = context.options options = context.options
def from_document_info(key): def from_document_info(key):
@@ -679,15 +681,14 @@ def get_docinfo(base_pdf: pikepdf.Pdf, context: PdfContext) -> Dict[str, str]:
k: from_document_info(k) k: from_document_info(k)
for k in ('/Title', '/Author', '/Keywords', '/Subject', '/CreationDate') for k in ('/Title', '/Author', '/Keywords', '/Subject', '/CreationDate')
} }
if options is not None: if options.title:
if options.title: pdfmark['/Title'] = options.title
pdfmark['/Title'] = options.title if options.author:
if options.author: pdfmark['/Author'] = options.author
pdfmark['/Author'] = options.author if options.keywords:
if options.keywords: pdfmark['/Keywords'] = options.keywords
pdfmark['/Keywords'] = options.keywords if options.subject:
if options.subject: pdfmark['/Subject'] = options.subject
pdfmark['/Subject'] = options.subject
creator_tag = context.plugin_manager.hook.get_ocr_engine().creator_tag(options) creator_tag = context.plugin_manager.hook.get_ocr_engine().creator_tag(options)
@@ -867,7 +868,7 @@ def enumerate_compress_ranges(iterable):
yield (skipped_from, index), None yield (skipped_from, index), None
def merge_sidecars(txt_files: Iterable[Optional[Path]], context: PdfContext): def merge_sidecars(txt_files: Iterable[Path | None], context: PdfContext):
output_file = context.get_path('sidecar.txt') output_file = context.get_path('sidecar.txt')
with open(output_file, 'w', encoding="utf-8") as stream: with open(output_file, 'w', encoding="utf-8") as stream:
for (from_, to_), txt_file in enumerate_compress_ranges(txt_files): for (from_, to_), txt_file in enumerate_compress_ranges(txt_files):
+6 -4
View File
@@ -6,13 +6,15 @@
"""Plugin manager using pluggy.""" """Plugin manager using pluggy."""
from __future__ import annotations
import argparse import argparse
import importlib import importlib
import importlib.util import importlib.util
import pkgutil import pkgutil
import sys import sys
from pathlib import Path from pathlib import Path
from typing import List, Sequence, Tuple, Union from typing import Sequence
import pluggy import pluggy
@@ -34,7 +36,7 @@ class OcrmypdfPluginManager(pluggy.PluginManager):
def __init__( def __init__(
self, self,
*args, *args,
plugins: List[Union[str, Path]], plugins: list[str | Path],
builtins: bool = True, builtins: bool = True,
**kwargs, **kwargs,
): ):
@@ -101,7 +103,7 @@ class OcrmypdfPluginManager(pluggy.PluginManager):
self.register(module) self.register(module)
def get_plugin_manager(plugins: List[Union[str, Path]], builtins=True): def get_plugin_manager(plugins: list[str | Path], builtins=True):
return OcrmypdfPluginManager( return OcrmypdfPluginManager(
project_name='ocrmypdf', project_name='ocrmypdf',
plugins=plugins, plugins=plugins,
@@ -111,7 +113,7 @@ def get_plugin_manager(plugins: List[Union[str, Path]], builtins=True):
def get_parser_options_plugins( def get_parser_options_plugins(
args: Sequence[str], args: Sequence[str],
) -> Tuple[argparse.ArgumentParser, argparse.Namespace, pluggy.PluginManager]: ) -> tuple[argparse.ArgumentParser, argparse.Namespace, pluggy.PluginManager]:
pre_options, _unused = plugins_only_parser.parse_known_args(args=args) pre_options, _unused = plugins_only_parser.parse_known_args(args=args)
plugin_manager = get_plugin_manager(pre_options.plugins) plugin_manager = get_plugin_manager(pre_options.plugins)
+11 -9
View File
@@ -7,6 +7,8 @@
"""Implements the concurrent and page synchronous parts of the pipeline.""" """Implements the concurrent and page synchronous parts of the pipeline."""
from __future__ import annotations
import argparse import argparse
import logging import logging
import logging.handlers import logging.handlers
@@ -18,7 +20,7 @@ from concurrent.futures.thread import BrokenThreadPool
from functools import partial from functools import partial
from pathlib import Path from pathlib import Path
from tempfile import mkdtemp from tempfile import mkdtemp
from typing import List, NamedTuple, Optional, Sequence, Tuple, cast from typing import NamedTuple, Sequence, cast
import PIL import PIL
@@ -74,9 +76,9 @@ class PageResult(NamedTuple):
"""Result when a page is finished processing.""" """Result when a page is finished processing."""
pageno: int pageno: int
pdf_page_from_image: Optional[Path] pdf_page_from_image: Path | None
ocr: Optional[Path] ocr: Path | None
text: Optional[Path] text: Path | None
orientation_correction: int orientation_correction: int
@@ -115,7 +117,7 @@ def preprocess(
def make_intermediate_images( def make_intermediate_images(
page_context: PageContext, orientation_correction: int page_context: PageContext, orientation_correction: int
) -> Tuple[Path, Optional[Path]]: ) -> tuple[Path, Path | None]:
options = page_context.options options = page_context.options
ocr_image = preprocess_out = None ocr_image = preprocess_out = None
@@ -232,7 +234,7 @@ def exec_page_sync(page_context: PageContext) -> PageResult:
def post_process( def post_process(
pdf_file: Path, context: PdfContext, executor: Executor pdf_file: Path, context: PdfContext, executor: Executor
) -> Tuple[Path, Sequence[str]]: ) -> tuple[Path, Sequence[str]]:
pdf_out = pdf_file pdf_out = pdf_file
if context.options.output_type.startswith('pdfa'): if context.options.output_type.startswith('pdfa'):
ps_stub_out = generate_postscript_stub(context) ps_stub_out = generate_postscript_stub(context)
@@ -259,7 +261,7 @@ def exec_concurrent(context: PdfContext, executor: Executor) -> Sequence[str]:
if max_workers > 1: if max_workers > 1:
log.info("Start processing %d pages concurrently", max_workers) log.info("Start processing %d pages concurrently", max_workers)
sidecars: List[Optional[Path]] = [None] * len(context.pdfinfo) sidecars: list[Path | None] = [None] * len(context.pdfinfo)
ocrgraft = OcrGrafter(context) ocrgraft = OcrGrafter(context)
def update_page(result: PageResult, pbar): def update_page(result: PageResult, pbar):
@@ -302,7 +304,7 @@ def exec_concurrent(context: PdfContext, executor: Executor) -> Sequence[str]:
# Merge layers to one single pdf # Merge layers to one single pdf
pdf = ocrgraft.finalize() pdf = ocrgraft.finalize()
messages: List[str] = [] messages: Sequence[str] = []
if options.output_type != 'none': if options.output_type != 'none':
# PDF/A and metadata # PDF/A and metadata
log.info("Postprocessing...") log.info("Postprocessing...")
@@ -337,7 +339,7 @@ def configure_debug_logging(
def run_pipeline( def run_pipeline(
options: argparse.Namespace, options: argparse.Namespace,
*, *,
plugin_manager: Optional[OcrmypdfPluginManager], plugin_manager: OcrmypdfPluginManager | None,
api: bool = False, api: bool = False,
) -> ExitCode: ) -> ExitCode:
# Any changes to options will not take effect for options that are already # Any changes to options will not take effect for options that are already
+7 -5
View File
@@ -7,6 +7,8 @@
"""Validate a work order from API or command line.""" """Validate a work order from API or command line."""
from __future__ import annotations
import locale import locale
import logging import logging
import os import os
@@ -14,7 +16,7 @@ import sys
import unicodedata import unicodedata
from pathlib import Path from pathlib import Path
from shutil import copyfileobj from shutil import copyfileobj
from typing import List, Optional, Sequence, Set, Tuple from typing import Sequence
import pikepdf import pikepdf
import PIL import PIL
@@ -144,8 +146,8 @@ def check_options_preprocessing(options):
raise BadArgsError("--unpaper-args: " + str(e)) from e raise BadArgsError("--unpaper-args: " + str(e)) from e
def _pages_from_ranges(ranges: str) -> Set[int]: def _pages_from_ranges(ranges: str) -> set[int]:
pages: List[int] = [] pages: list[int] = []
page_groups = ranges.replace(' ', '').split(',') page_groups = ranges.replace(' ', '').split(',')
for group in page_groups: for group in page_groups:
if not group: if not group:
@@ -241,7 +243,7 @@ def check_options(options, plugin_manager):
_check_options(options, plugin_manager, ocr_engine_languages) _check_options(options, plugin_manager, ocr_engine_languages)
def create_input_file(options, work_folder: Path) -> Tuple[Path, str]: def create_input_file(options, work_folder: Path) -> tuple[Path, str]:
if options.input_file == '-': if options.input_file == '-':
# stdin # stdin
log.info('reading file from standard input') log.info('reading file from standard input')
@@ -297,7 +299,7 @@ def report_output_file_size(
options, options,
input_file: Path, input_file: Path,
output_file: Path, output_file: Path,
optimize_messages: Optional[Sequence[str]] = None, optimize_messages: Sequence[str] | None = None,
file_overhead: int = 4000, file_overhead: int = 4000,
page_overhead: int = 3000, page_overhead: int = 3000,
): ):
+4 -2
View File
@@ -9,10 +9,12 @@
OCRmyPDF uses setuptools_scm to derive version from git tags. OCRmyPDF uses setuptools_scm to derive version from git tags.
""" """
from __future__ import annotations
try: try:
from importlib_metadata import version as _package_version
except ImportError:
from importlib.metadata import version as _package_version from importlib.metadata import version as _package_version
except ImportError:
from importlib_metadata import version as _package_version # type: ignore
PROGRAM_NAME = 'ocrmypdf' PROGRAM_NAME = 'ocrmypdf'
+4 -2
View File
@@ -6,6 +6,8 @@
"""Functions for using ocrmypdf as an API.""" """Functions for using ocrmypdf as an API."""
from __future__ import annotations
import logging import logging
import os import os
import sys import sys
@@ -13,7 +15,7 @@ import threading
from enum import IntEnum from enum import IntEnum
from io import IOBase from io import IOBase
from pathlib import Path from pathlib import Path
from typing import AnyStr, BinaryIO, Iterable, Optional, Union from typing import AnyStr, BinaryIO, Iterable, Union
from warnings import warn from warnings import warn
from ocrmypdf._logging import PageNumberFilter, TqdmConsole from ocrmypdf._logging import PageNumberFilter, TqdmConsole
@@ -217,7 +219,7 @@ def ocr( # pylint: disable=unused-argument
language: Iterable[str] = None, language: Iterable[str] = None,
image_dpi: int = None, image_dpi: int = None,
output_type=None, output_type=None,
sidecar: Optional[StrPath] = None, sidecar: StrPath | None = None,
jobs: int = None, jobs: int = None,
use_threads: bool = None, use_threads: bool = None,
title: str = None, title: str = None,
+2
View File
@@ -4,6 +4,8 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this # License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
# This file exists only mark builtin_plugins as a package. # This file exists only mark builtin_plugins as a package.
# The plugin manager will not load it, so anything defined here may not be # The plugin manager will not load it, so anything defined here may not be
# processed as a module. # processed as a module.
+13 -3
View File
@@ -4,13 +4,14 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this # License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
# © 2020 James R. Barlow: github.com/jbarlow83 # © 2020 James R. Barlow: github.com/jbarlow83
# #
# This Source Code Form is subject to the terms of the Mozilla Public # This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this # License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
"""OCRmyPDF's multiprocessing/multithreading abstraction layer.""" """OCRmyPDF's multiprocessing/multithreading abstraction layer."""
import logging import logging
@@ -104,6 +105,15 @@ def thread_init(q: Queue, user_init: UserInit, loglevel) -> None:
class StandardExecutor(Executor): class StandardExecutor(Executor):
"""Standard OCRmyPDF concurrent task executor.""" """Standard OCRmyPDF concurrent task executor."""
def _cancel_futures_kwargs(self):
"""Shim older Pythons that do not have Executor.shutdown(...cancel_futures=).
Remove this code when support for Python 3.8 is dropped.
"""
if sys.version_info[:2] < (3, 9):
return {}
return dict(cancel_futures=True)
def _execute( def _execute(
self, self,
*, *,
@@ -142,7 +152,7 @@ class StandardExecutor(Executor):
task_finished(result, pbar) task_finished(result, pbar)
except KeyboardInterrupt: except KeyboardInterrupt:
# Terminate pool so we exit instantly # Terminate pool so we exit instantly
executor.shutdown(wait=False, cancel_futures=True) executor.shutdown(wait=False, **self._cancel_futures_kwargs())
raise raise
except Exception: except Exception:
if not os.environ.get("PYTEST_CURRENT_TEST", ""): if not os.environ.get("PYTEST_CURRENT_TEST", ""):
@@ -151,7 +161,7 @@ class StandardExecutor(Executor):
# results will be discard. But if the condition above is True, # results will be discard. But if the condition above is True,
# then we are running in pytest, and we want everything to exit # then we are running in pytest, and we want everything to exit
# as cleanly as possible so that we get good error messages. # as cleanly as possible so that we get good error messages.
executor.shutdown(wait=False, cancel_futures=True) executor.shutdown(wait=False, **self._cancel_futures_kwargs())
raise raise
finally: finally:
# Terminate log listener # Terminate log listener
@@ -6,6 +6,8 @@
"""OCRmyPDF automatically installs these filters as plugins.""" """OCRmyPDF automatically installs these filters as plugins."""
from __future__ import annotations
from ocrmypdf import hookimpl from ocrmypdf import hookimpl
@@ -7,6 +7,8 @@
"""Built-in plugin to implement PDF page rasterization and PDF/A production.""" """Built-in plugin to implement PDF page rasterization and PDF/A production."""
from __future__ import annotations
import logging import logging
from ocrmypdf import hookimpl from ocrmypdf import hookimpl
+4 -2
View File
@@ -7,10 +7,12 @@
"""Built-in plugin to implement PDF page optimization.""" """Built-in plugin to implement PDF page optimization."""
from __future__ import annotations
import argparse import argparse
import logging import logging
from pathlib import Path from pathlib import Path
from typing import Sequence, Tuple from typing import Sequence
from ocrmypdf import Executor, PdfContext, hookimpl from ocrmypdf import Executor, PdfContext, hookimpl
from ocrmypdf._exec import jbig2enc, pngquant from ocrmypdf._exec import jbig2enc, pngquant
@@ -130,7 +132,7 @@ def optimize_pdf(
context: PdfContext, context: PdfContext,
executor: Executor, executor: Executor,
linearize: bool, linearize: bool,
) -> Tuple[Path, Sequence[str]]: ) -> tuple[Path, Sequence[str]]:
save_settings = dict( save_settings = dict(
linearize=linearize, linearize=linearize,
**get_pdf_save_settings(context.options.output_type), **get_pdf_save_settings(context.options.output_type),
@@ -7,6 +7,8 @@
"""Built-in plugin to implement OCR using Tesseract.""" """Built-in plugin to implement OCR using Tesseract."""
from __future__ import annotations
import logging import logging
import os import os
+4 -4
View File
@@ -6,8 +6,10 @@
"""Command line interface customization and validation.""" """Command line interface customization and validation."""
from __future__ import annotations
import argparse import argparse
from typing import Any, Callable, Mapping, Optional, TypeVar from typing import Any, Callable, Mapping, TypeVar
from ocrmypdf._version import PROGRAM_NAME as _PROGRAM_NAME from ocrmypdf._version import PROGRAM_NAME as _PROGRAM_NAME
from ocrmypdf._version import __version__ as _VERSION from ocrmypdf._version import __version__ as _VERSION
@@ -15,9 +17,7 @@ from ocrmypdf._version import __version__ as _VERSION
T = TypeVar('T', int, float) T = TypeVar('T', int, float)
def numeric( def numeric(basetype: Callable[[Any], T], min_: T | None = None, max_: T | None = None):
basetype: Callable[[Any], T], min_: Optional[T] = None, max_: Optional[T] = None
):
"""Validator for numeric params""" """Validator for numeric params"""
min_ = basetype(min_) if min_ is not None else None min_ = basetype(min_) if min_ is not None else None
max_ = basetype(max_) if max_ is not None else None max_ = basetype(max_) if max_ is not None else None
+2
View File
@@ -6,3 +6,5 @@
"""Data files used to generate certain PDFs.""" """Data files used to generate certain PDFs."""
from __future__ import annotations
+2
View File
@@ -6,6 +6,8 @@
"""OCRmyPDF's exceptions.""" """OCRmyPDF's exceptions."""
from __future__ import annotations
from enum import IntEnum from enum import IntEnum
from textwrap import dedent from textwrap import dedent
+5 -3
View File
@@ -20,6 +20,8 @@ be guaranteed, some workers may end up with too much work while others are idle.
It is less efficient than the standard implementation, so not th edefault. It is less efficient than the standard implementation, so not th edefault.
""" """
from __future__ import annotations
import logging import logging
import logging.handlers import logging.handlers
import signal import signal
@@ -28,7 +30,7 @@ from enum import Enum, auto
from itertools import islice, repeat, takewhile, zip_longest from itertools import islice, repeat, takewhile, zip_longest
from multiprocessing import Pipe, Process from multiprocessing import Pipe, Process
from multiprocessing.connection import Connection, wait from multiprocessing.connection import Connection, wait
from typing import Callable, Iterable, Iterator, List from typing import Callable, Iterable, Iterator
from ocrmypdf import Executor, hookimpl from ocrmypdf import Executor, hookimpl
from ocrmypdf._concurrent import NullProgressBar from ocrmypdf._concurrent import NullProgressBar
@@ -134,8 +136,8 @@ class LambdaExecutor(Executor):
if not grouped_args: if not grouped_args:
return return
processes: List[Process] = [] processes: list[Process] = []
connections: List[Connection] = [] connections: list[Connection] = []
for chunk in grouped_args: for chunk in grouped_args:
parent_conn, child_conn = Pipe() parent_conn, child_conn = Pipe()
+2
View File
@@ -6,6 +6,8 @@
"""Support functions.""" """Support functions."""
from __future__ import annotations
import logging import logging
import multiprocessing import multiprocessing
import os import os
+15 -8
View File
@@ -30,17 +30,24 @@
"""Transform .hocr and page image to text PDF.""" """Transform .hocr and page image to text PDF."""
from __future__ import annotations
import argparse import argparse
import os import os
import re import re
import warnings
from math import atan, cos, sin from math import atan, cos, sin
from pathlib import Path from pathlib import Path
from typing import Any, NamedTuple, Optional, Tuple, Union from typing import Any, NamedTuple, Optional, Tuple, Union
from xml.etree import ElementTree from xml.etree import ElementTree
from reportlab.lib.colors import black, cyan, magenta, red with warnings.catch_warnings():
from reportlab.lib.units import inch warnings.filterwarnings(
from reportlab.pdfgen.canvas import Canvas 'ignore', category=DeprecationWarning, message=r".*load_module.*"
)
from reportlab.lib.colors import black, cyan, magenta, red
from reportlab.lib.units import inch
from reportlab.pdfgen.canvas import Canvas
# According to Wikipedia these languages are supported in the ISO-8859-1 character # According to Wikipedia these languages are supported in the ISO-8859-1 character
# set, meaning reportlab can generate them and they are compatible with hocr, # set, meaning reportlab can generate them and they are compatible with hocr,
@@ -134,7 +141,7 @@ class HocrTransform:
{'': 'ff', '': 'ffi', '': 'ffl', '': 'fi', '': 'fl'} {'': 'ff', '': 'ffi', '': 'ffl', '': 'fi', '': 'fl'}
) )
def __init__(self, *, hocr_filename: Union[str, Path], dpi: float): def __init__(self, *, hocr_filename: str | Path, dpi: float):
self.dpi = dpi self.dpi = dpi
self.hocr = ElementTree.parse(os.fspath(hocr_filename)) self.hocr = ElementTree.parse(os.fspath(hocr_filename))
@@ -198,7 +205,7 @@ class HocrTransform:
return out return out
@classmethod @classmethod
def baseline(cls, element: Element) -> Tuple[float, float]: def baseline(cls, element: Element) -> tuple[float, float]:
""" """
Returns a tuple containing the baseline slope and intercept. Returns a tuple containing the baseline slope and intercept.
""" """
@@ -214,7 +221,7 @@ class HocrTransform:
""" """
return Rect._make((c / self.dpi * inch) for c in pxl) return Rect._make((c / self.dpi * inch) for c in pxl)
def _child_xpath(self, html_tag: str, html_class: Optional[str] = None) -> str: def _child_xpath(self, html_tag: str, html_class: str | None = None) -> str:
xpath = f".//{self.xmlns}{html_tag}" xpath = f".//{self.xmlns}{html_tag}"
if html_class: if html_class:
xpath += f"[@class='{html_class}']" xpath += f"[@class='{html_class}']"
@@ -241,7 +248,7 @@ class HocrTransform:
self, self,
*, *,
out_filename: Path, out_filename: Path,
image_filename: Optional[Path] = None, image_filename: Path | None = None,
show_bounding_boxes: bool = False, show_bounding_boxes: bool = False,
fontname: str = "Helvetica", fontname: str = "Helvetica",
invisible_text: bool = False, invisible_text: bool = False,
@@ -344,7 +351,7 @@ class HocrTransform:
def _do_line( def _do_line(
self, self,
pdf: Canvas, pdf: Canvas,
line: Optional[Element], line: Element | None,
elemclass: str, elemclass: str,
fontname: str, fontname: str,
invisible_text: bool, invisible_text: bool,
+20 -29
View File
@@ -7,6 +7,8 @@
"""Post-processing image optimization of OCR PDFs.""" """Post-processing image optimization of OCR PDFs."""
from __future__ import annotations
import logging import logging
import sys import sys
import tempfile import tempfile
@@ -14,18 +16,7 @@ import threading
from collections import defaultdict from collections import defaultdict
from os import fspath from os import fspath
from pathlib import Path from pathlib import Path
from typing import ( from typing import Callable, Iterator, MutableSet, NamedTuple, NewType, Sequence
Callable,
Dict,
Iterator,
List,
MutableSet,
NamedTuple,
NewType,
Optional,
Sequence,
Tuple,
)
from zlib import compress from zlib import compress
import img2pdf import img2pdf
@@ -78,7 +69,7 @@ def jpg_name(root: Path, xref: Xref) -> Path:
def extract_image_filter( def extract_image_filter(
pike: Pdf, root: Path, image: Stream, xref: Xref pike: Pdf, root: Path, image: Stream, xref: Xref
) -> Optional[Tuple[PdfImage, Tuple[Name, Object]]]: ) -> tuple[PdfImage, tuple[Name, Object]] | None:
del pike # unused args del pike # unused args
del root del root
@@ -135,7 +126,7 @@ def extract_image_filter(
def extract_image_jbig2( def extract_image_jbig2(
*, pike: Pdf, root: Path, image: Stream, xref: Xref, options *, pike: Pdf, root: Path, image: Stream, xref: Xref, options
) -> Optional[XrefExt]: ) -> XrefExt | None:
del options # unused arg del options # unused arg
result = extract_image_filter(pike, root, image, xref) result = extract_image_filter(pike, root, image, xref)
@@ -176,7 +167,7 @@ def extract_image_jbig2(
def extract_image_generic( def extract_image_generic(
*, pike: Pdf, root: Path, image: Stream, xref: Xref, options *, pike: Pdf, root: Path, image: Stream, xref: Xref, options
) -> Optional[XrefExt]: ) -> XrefExt | None:
result = extract_image_filter(pike, root, image, xref) result = extract_image_filter(pike, root, image, xref)
if result is None: if result is None:
return None return None
@@ -240,8 +231,8 @@ def extract_images(
pike: Pdf, pike: Pdf,
root: Path, root: Path,
options, options,
extract_fn: Callable[..., Optional[XrefExt]], extract_fn: Callable[..., XrefExt | None],
) -> Iterator[Tuple[int, XrefExt]]: ) -> Iterator[tuple[int, XrefExt]]:
"""Extract image using extract_fn """Extract image using extract_fn
Enumerate images on each page, lookup their xref/ID number in the PDF. Enumerate images on each page, lookup their xref/ID number in the PDF.
@@ -300,7 +291,7 @@ def extract_images(
def extract_images_generic( def extract_images_generic(
pike: Pdf, root: Path, options pike: Pdf, root: Path, options
) -> Tuple[List[Xref], List[Xref]]: ) -> tuple[list[Xref], list[Xref]]:
"""Extract any >=2bpp image we think we can improve""" """Extract any >=2bpp image we think we can improve"""
jpegs = [] jpegs = []
@@ -315,7 +306,7 @@ def extract_images_generic(
return jpegs, pngs return jpegs, pngs
def extract_images_jbig2(pike: Pdf, root: Path, options) -> Dict[int, List[XrefExt]]: def extract_images_jbig2(pike: Pdf, root: Path, options) -> dict[int, list[XrefExt]]:
"""Extract any bitonal image that we think we can improve as JBIG2""" """Extract any bitonal image that we think we can improve as JBIG2"""
jbig2_groups = defaultdict(list) jbig2_groups = defaultdict(list)
@@ -328,11 +319,11 @@ def extract_images_jbig2(pike: Pdf, root: Path, options) -> Dict[int, List[XrefE
def _produce_jbig2_images( def _produce_jbig2_images(
jbig2_groups: Dict[int, List[XrefExt]], root: Path, options, executor: Executor jbig2_groups: dict[int, list[XrefExt]], root: Path, options, executor: Executor
) -> None: ) -> None:
"""Produce JBIG2 images from their groups""" """Produce JBIG2 images from their groups"""
def jbig2_group_args(root: Path, groups: Dict[int, List[XrefExt]]): def jbig2_group_args(root: Path, groups: dict[int, list[XrefExt]]):
for group, xref_exts in groups.items(): for group, xref_exts in groups.items():
prefix = f'group{group:08d}' prefix = f'group{group:08d}'
yield ( yield (
@@ -341,7 +332,7 @@ def _produce_jbig2_images(
prefix, # =out_prefix prefix, # =out_prefix
) )
def jbig2_single_args(root, groups: Dict[int, List[XrefExt]]): def jbig2_single_args(root, groups: dict[int, list[XrefExt]]):
for group, xref_exts in groups.items(): for group, xref_exts in groups.items():
prefix = f'group{group:08d}' prefix = f'group{group:08d}'
# Second loop is to ensure multiple images per page are unpacked # Second loop is to ensure multiple images per page are unpacked
@@ -376,7 +367,7 @@ def _produce_jbig2_images(
def convert_to_jbig2( def convert_to_jbig2(
pike: Pdf, pike: Pdf,
jbig2_groups: Dict[int, List[XrefExt]], jbig2_groups: dict[int, list[XrefExt]],
root: Path, root: Path,
options, options,
executor: Executor, executor: Executor,
@@ -393,7 +384,7 @@ def convert_to_jbig2(
When the JBIG2 symbolic coder is not used, each JBIG2 stands on its own When the JBIG2 symbolic coder is not used, each JBIG2 stands on its own
and needs no dictionary. Currently this must be lossless JBIG2. and needs no dictionary. Currently this must be lossless JBIG2.
""" """
jbig2_globals_dict: Optional[Dictionary] jbig2_globals_dict: Dictionary | None
_produce_jbig2_images(jbig2_groups, root, options, executor) _produce_jbig2_images(jbig2_groups, root, options, executor)
@@ -419,7 +410,7 @@ def convert_to_jbig2(
) )
def _optimize_jpeg(args: Tuple[Xref, Path, Path, int]) -> Tuple[Xref, Optional[Path]]: def _optimize_jpeg(args: tuple[Xref, Path, Path, int]) -> tuple[Xref, Path | None]:
xref, in_jpg, opt_jpg, jpeg_quality = args xref, in_jpg, opt_jpg, jpeg_quality = args
with Image.open(in_jpg) as im: with Image.open(in_jpg) as im:
@@ -435,13 +426,13 @@ def _optimize_jpeg(args: Tuple[Xref, Path, Path, int]) -> Tuple[Xref, Optional[P
def transcode_jpegs( def transcode_jpegs(
pike: Pdf, jpegs: Sequence[Xref], root: Path, options, executor: Executor pike: Pdf, jpegs: Sequence[Xref], root: Path, options, executor: Executor
) -> None: ) -> None:
def jpeg_args() -> Iterator[Tuple[Xref, Path, Path, int]]: def jpeg_args() -> Iterator[tuple[Xref, Path, Path, int]]:
for xref in jpegs: for xref in jpegs:
in_jpg = jpg_name(root, xref) in_jpg = jpg_name(root, xref)
opt_jpg = in_jpg.with_suffix('.opt.jpg') opt_jpg = in_jpg.with_suffix('.opt.jpg')
yield xref, in_jpg, opt_jpg, options.jpeg_quality yield xref, in_jpg, opt_jpg, options.jpeg_quality
def finish_jpeg(result: Tuple[Xref, Optional[Path]], pbar): def finish_jpeg(result: tuple[Xref, Path | None], pbar):
xref, opt_jpg = result xref, opt_jpg = result
if opt_jpg: if opt_jpg:
compdata = opt_jpg.read_bytes() # JPEG can inserted into PDF as is compdata = opt_jpg.read_bytes() # JPEG can inserted into PDF as is
@@ -466,7 +457,7 @@ def transcode_jpegs(
def _find_deflatable_jpeg( def _find_deflatable_jpeg(
*, pike: Pdf, root: Path, image: Stream, xref: Xref, options *, pike: Pdf, root: Path, image: Stream, xref: Xref, options
) -> Optional[XrefExt]: ) -> XrefExt | None:
result = extract_image_filter(pike, root, image, xref) result = extract_image_filter(pike, root, image, xref)
if result is None: if result is None:
return None return None
@@ -478,7 +469,7 @@ def _find_deflatable_jpeg(
return None return None
def _deflate_jpeg(args: Tuple[Pdf, threading.Lock, Xref, int]) -> Tuple[Xref, bytes]: def _deflate_jpeg(args: tuple[Pdf, threading.Lock, Xref, int]) -> tuple[Xref, bytes]:
pike, lock, xref, complevel = args pike, lock, xref, complevel = args
with lock: with lock:
xobj = pike.get_object(xref, 0) xobj = pike.get_object(xref, 0)
+7 -5
View File
@@ -9,14 +9,16 @@
Utilities for PDF/A production and confirmation with Ghostspcript. Utilities for PDF/A production and confirmation with Ghostspcript.
""" """
from __future__ import annotations
import base64 import base64
from pathlib import Path from pathlib import Path
from typing import Dict, Iterator, Union from typing import Iterator
try: try:
from importlib_resources import files as package_files
except ImportError:
from importlib.resources import files as package_files from importlib.resources import files as package_files
except ImportError:
from importlib_resources import files as package_files # type: ignore
import pikepdf import pikepdf
@@ -25,7 +27,7 @@ SRGB_ICC_PROFILE_NAME = 'sRGB.icc'
def _postscript_objdef( def _postscript_objdef(
alias: str, alias: str,
dictionary: Dict[str, str], dictionary: dict[str, str],
*, *,
stream_name: str = None, stream_name: str = None,
stream_data: bytes = None, stream_data: bytes = None,
@@ -131,7 +133,7 @@ def file_claims_pdfa(filename: Path):
} }
valid_part_conforms = {'1A', '1B', '2A', '2B', '2U', '3A', '3B', '3U'} valid_part_conforms = {'1A', '1B', '2A', '2B', '2U', '3A', '3B', '3U'}
conformance = f'PDF/A-{pdfmeta.pdfa_status}' conformance = f'PDF/A-{pdfmeta.pdfa_status}'
pdfa_dict: Dict[str, Union[str, bool]] = {} pdfa_dict: dict[str, str | bool] = {}
if pdfmeta.pdfa_status in valid_part_conforms: if pdfmeta.pdfa_status in valid_part_conforms:
pdfa_dict['pass'] = True pdfa_dict['pass'] = True
pdfa_dict['output'] = 'pdfa' pdfa_dict['output'] = 'pdfa'
+2
View File
@@ -8,4 +8,6 @@
"""For extracting information about PDFs prior to OCR.""" """For extracting information about PDFs prior to OCR."""
from __future__ import annotations
from ocrmypdf.pdfinfo.info import Colorspace, Encoding, PdfInfo from ocrmypdf.pdfinfo.info import Colorspace, Encoding, PdfInfo
+28 -31
View File
@@ -8,6 +8,8 @@
"""Extract information about the content of a PDF.""" """Extract information about the content of a PDF."""
from __future__ import annotations
import atexit import atexit
import logging import logging
import re import re
@@ -21,16 +23,13 @@ from os import PathLike
from pathlib import Path from pathlib import Path
from typing import ( from typing import (
Container, Container,
Dict,
Iterable, Iterable,
Iterator, Iterator,
List,
Mapping, Mapping,
NamedTuple, NamedTuple,
Optional, Optional,
Sequence, Sequence,
Tuple, Tuple,
Union,
) )
from warnings import warn from warnings import warn
@@ -85,7 +84,7 @@ class Encoding(Enum):
FloatRect = Tuple[float, float, float, float] FloatRect = Tuple[float, float, float, float]
FRIENDLY_COLORSPACE: Dict[str, Colorspace] = { FRIENDLY_COLORSPACE: dict[str, Colorspace] = {
'/DeviceGray': Colorspace.gray, '/DeviceGray': Colorspace.gray,
'/CalGray': Colorspace.gray, '/CalGray': Colorspace.gray,
'/DeviceRGB': Colorspace.rgb, '/DeviceRGB': Colorspace.rgb,
@@ -103,7 +102,7 @@ FRIENDLY_COLORSPACE: Dict[str, Colorspace] = {
'/I': Colorspace.index, '/I': Colorspace.index,
} }
FRIENDLY_ENCODING: Dict[str, Encoding] = { FRIENDLY_ENCODING: dict[str, Encoding] = {
'/CCITTFaxDecode': Encoding.ccitt, '/CCITTFaxDecode': Encoding.ccitt,
'/DCTDecode': Encoding.jpeg, '/DCTDecode': Encoding.jpeg,
'/JPXDecode': Encoding.jpeg2000, '/JPXDecode': Encoding.jpeg2000,
@@ -117,7 +116,7 @@ FRIENDLY_ENCODING: Dict[str, Encoding] = {
'/RL': Encoding.runlength, '/RL': Encoding.runlength,
} }
FRIENDLY_COMP: Dict[Colorspace, int] = { FRIENDLY_COMP: dict[Colorspace, int] = {
Colorspace.gray: 1, Colorspace.gray: 1,
Colorspace.rgb: 3, Colorspace.rgb: 3,
Colorspace.cmyk: 4, Colorspace.cmyk: 4,
@@ -139,7 +138,7 @@ class XobjectSettings(NamedTuple):
"""Info about an XObject found in a PDF.""" """Info about an XObject found in a PDF."""
name: str name: str
shorthand: Tuple[float, float, float, float, float, float] shorthand: tuple[float, float, float, float, float, float]
stack_depth: int stack_depth: int
@@ -147,24 +146,24 @@ class InlineSettings(NamedTuple):
"""Info about an inline image found in a PDF.""" """Info about an inline image found in a PDF."""
iimage: PdfInlineImage iimage: PdfInlineImage
shorthand: Tuple[float, float, float, float, float, float] shorthand: tuple[float, float, float, float, float, float]
stack_depth: int stack_depth: int
class ContentsInfo(NamedTuple): class ContentsInfo(NamedTuple):
"""Info about various objects found in a PDF.""" """Info about various objects found in a PDF."""
xobject_settings: List[XobjectSettings] xobject_settings: list[XobjectSettings]
inline_images: List[InlineSettings] inline_images: list[InlineSettings]
found_vector: bool found_vector: bool
found_text: bool found_text: bool
name_index: Mapping[str, List[XobjectSettings]] name_index: Mapping[str, list[XobjectSettings]]
class TextboxInfo(NamedTuple): class TextboxInfo(NamedTuple):
"""Info about a text box found in a PDF.""" """Info about a text box found in a PDF."""
bbox: Tuple[float, float, float, float] bbox: tuple[float, float, float, float]
is_visible: bool is_visible: bool
is_corrupt: bool is_corrupt: bool
@@ -217,8 +216,8 @@ def _interpret_contents(contentstream: Object, initial_shorthand=UNIT_SQUARE):
stack = [] stack = []
ctm = PdfMatrix(initial_shorthand) ctm = PdfMatrix(initial_shorthand)
xobject_settings: List[XobjectSettings] = [] xobject_settings: list[XobjectSettings] = []
inline_images: List[InlineSettings] = [] inline_images: list[InlineSettings] = []
name_index = defaultdict(lambda: []) name_index = defaultdict(lambda: [])
found_vector = False found_vector = False
found_text = False found_text = False
@@ -342,21 +341,21 @@ class ImageInfo:
DPI_PREC = Decimal('1.000') DPI_PREC = Decimal('1.000')
_comp: Optional[int] _comp: int | None
_name: str _name: str
def __init__( def __init__(
self, self,
*, *,
name='', name='',
pdfimage: Optional[Object] = None, pdfimage: Object | None = None,
inline: Optional[PdfInlineImage] = None, inline: PdfInlineImage | None = None,
shorthand=None, shorthand=None,
): ):
self._name = str(name) self._name = str(name)
self._shorthand = shorthand self._shorthand = shorthand
pim: Union[PdfInlineImage, PdfImage] pim: PdfInlineImage | PdfImage
if inline is not None: if inline is not None:
self._origin = 'inline' self._origin = 'inline'
@@ -400,7 +399,7 @@ class ImageInfo:
self._comp = 4 self._comp = 4
else: else:
self._comp = 3 self._comp = 3
except UnsupportedImageTypeError as ex: except (AttributeError, UnsupportedImageTypeError) as ex:
self._comp = None self._comp = None
logger.warning( logger.warning(
f"An image with a corrupt or unreadable ICC profile was found. " f"An image with a corrupt or unreadable ICC profile was found. "
@@ -473,7 +472,7 @@ def _find_inline_images(contentsinfo: ContentsInfo) -> Iterator[ImageInfo]:
) )
def _image_xobjects(container) -> Iterator[Tuple[Object, str]]: def _image_xobjects(container) -> Iterator[tuple[Object, str]]:
"""Search for all XObject-based images in the container """Search for all XObject-based images in the container
Usually the container is a page, but it could also be a Form XObject Usually the container is a page, but it could also be a Form XObject
@@ -561,7 +560,7 @@ def _find_form_xobject_images(pdf: Pdf, container: Object, contentsinfo: Content
def _process_content_streams( def _process_content_streams(
*, pdf: Pdf, container: Object, shorthand=None *, pdf: Pdf, container: Object, shorthand=None
) -> Iterator[Union[VectorMarker, TextMarker, ImageInfo]]: ) -> Iterator[VectorMarker | TextMarker | ImageInfo]:
"""Find all individual instances of images drawn in the container """Find all individual instances of images drawn in the container
Usually the container is a page, but it may also be a Form XObject. Usually the container is a page, but it may also be a Form XObject.
@@ -691,8 +690,8 @@ def _pdf_pageinfo_concurrent(
max_workers, max_workers,
check_pages, check_pages,
detailed_analysis=False, detailed_analysis=False,
) -> Sequence[Optional['PageInfo']]: ) -> Sequence[PageInfo | None]:
pages: Sequence[Optional['PageInfo']] = [None] * len(pdf.pages) pages: Sequence[PageInfo | None] = [None] * len(pdf.pages)
def update_pageinfo(result, pbar): def update_pageinfo(result, pbar):
page = result page = result
@@ -744,9 +743,9 @@ def _pdf_pageinfo_concurrent(
class PageInfo: class PageInfo:
"""Information about type of contents on each page in a PDF.""" """Information about type of contents on each page in a PDF."""
_has_text: Optional[bool] _has_text: bool | None
_has_vector: Optional[bool] _has_vector: bool | None
_images: List[ImageInfo] _images: list[ImageInfo]
def __init__( def __init__(
self, self,
@@ -879,9 +878,7 @@ class PageInfo:
def images(self): def images(self):
return self._images return self._images
def get_textareas( def get_textareas(self, visible: bool | None = None, corrupt: bool | None = None):
self, visible: Optional[bool] = None, corrupt: Optional[bool] = None
):
def predicate(obj, want_visible, want_corrupt): def predicate(obj, want_visible, want_corrupt):
result = True result = True
if want_visible is not None: if want_visible is not None:
@@ -965,7 +962,7 @@ class PdfInfo:
self._has_acroform = True self._has_acroform = True
@property @property
def pages(self) -> Sequence[Optional[PageInfo]]: def pages(self) -> Sequence[PageInfo | None]:
return self._pages return self._pages
@property @property
@@ -982,7 +979,7 @@ class PdfInfo:
return self._has_acroform return self._has_acroform
@property @property
def filename(self) -> Union[str, Path]: def filename(self) -> str | Path:
if not isinstance(self._infile, (str, Path)): if not isinstance(self._infile, (str, Path)):
raise NotImplementedError("can't get filename from stream") raise NotImplementedError("can't get filename from stream")
return self._infile return self._infile
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import re import re
from math import copysign from math import copysign
from pathlib import Path from pathlib import Path
+11 -19
View File
@@ -6,19 +6,13 @@
"""OCRmyPDF pluggy plugin specification.""" """OCRmyPDF pluggy plugin specification."""
from __future__ import annotations
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from argparse import ArgumentParser, Namespace from argparse import ArgumentParser, Namespace
from logging import Handler from logging import Handler
from pathlib import Path from pathlib import Path
from typing import ( from typing import TYPE_CHECKING, AbstractSet, NamedTuple, Sequence
TYPE_CHECKING,
AbstractSet,
List,
NamedTuple,
Optional,
Sequence,
Tuple,
)
import pluggy import pluggy
@@ -177,7 +171,7 @@ def get_progressbar_class():
@hookspec @hookspec
def validate(pdfinfo: 'PdfInfo', options: Namespace) -> None: def validate(pdfinfo: PdfInfo, options: Namespace) -> None:
"""Called to give a plugin an opportunity to review *options* and *pdfinfo*. """Called to give a plugin an opportunity to review *options* and *pdfinfo*.
*options* contains the "work order" to process a particular file. *pdfinfo* *options* contains the "work order" to process a particular file. *pdfinfo*
@@ -203,8 +197,8 @@ def rasterize_pdf_page(
raster_device: str, raster_device: str,
raster_dpi: Resolution, raster_dpi: Resolution,
pageno: int, pageno: int,
page_dpi: Optional[Resolution], page_dpi: Resolution | None,
rotation: Optional[int], rotation: int | None,
filter_vector: bool, filter_vector: bool,
) -> Path: ) -> Path:
"""Rasterize one page of a PDF at resolution raster_dpi in canvas units. """Rasterize one page of a PDF at resolution raster_dpi in canvas units.
@@ -233,7 +227,7 @@ def rasterize_pdf_page(
@hookspec(firstresult=True) @hookspec(firstresult=True)
def filter_ocr_image(page: 'PageContext', image: 'Image.Image') -> 'Image.Image': def filter_ocr_image(page: PageContext, image: Image.Image) -> Image.Image:
"""Called to filter the image before it is sent to OCR. """Called to filter the image before it is sent to OCR.
This is the image that OCR sees, not what the user sees when they view the This is the image that OCR sees, not what the user sees when they view the
@@ -260,7 +254,7 @@ def filter_ocr_image(page: 'PageContext', image: 'Image.Image') -> 'Image.Image'
@hookspec(firstresult=True) @hookspec(firstresult=True)
def filter_page_image(page: 'PageContext', image_filename: Path) -> Path: def filter_page_image(page: PageContext, image_filename: Path) -> Path:
"""Called to filter the whole page before it is inserted into the PDF. """Called to filter the whole page before it is inserted into the PDF.
A whole page image is only produced when preprocessing command line arguments A whole page image is only produced when preprocessing command line arguments
@@ -300,9 +294,7 @@ def filter_page_image(page: 'PageContext', image_filename: Path) -> Path:
@hookspec(firstresult=True) @hookspec(firstresult=True)
def filter_pdf_page( def filter_pdf_page(page: PageContext, image_filename: Path, output_pdf: Path) -> Path:
page: 'PageContext', image_filename: Path, output_pdf: Path
) -> Path:
"""Called to convert a filtered whole page image into a PDF. """Called to convert a filtered whole page image into a PDF.
A whole page image is only produced when preprocessing command line arguments A whole page image is only produced when preprocessing command line arguments
@@ -445,7 +437,7 @@ def get_ocr_engine() -> OcrEngine:
@hookspec(firstresult=True) @hookspec(firstresult=True)
def generate_pdfa( def generate_pdfa(
pdf_pages: List[Path], pdf_pages: list[Path],
pdfmark: Path, pdfmark: Path,
output_file: Path, output_file: Path,
compression: str, compression: str,
@@ -501,7 +493,7 @@ def optimize_pdf(
context: PdfContext, context: PdfContext,
executor: Executor, executor: Executor,
linearize: bool, linearize: bool,
) -> Tuple[Path, Sequence[str]]: ) -> tuple[Path, Sequence[str]]:
"""Optimize a PDF after image, OCR and metadata processing. """Optimize a PDF after image, OCR and metadata processing.
If the input_pdf is a PDF/A, the plugin should modify input_pdf in a way If the input_pdf is a PDF/A, the plugin should modify input_pdf in a way
+2
View File
@@ -8,6 +8,8 @@
"""Utilities to measure OCR quality""" """Utilities to measure OCR quality"""
from __future__ import annotations
import re import re
from typing import Iterable from typing import Iterable
+43 -19
View File
@@ -7,16 +7,18 @@
"""Wrappers to manage subprocess calls""" """Wrappers to manage subprocess calls"""
from __future__ import annotations
import logging import logging
import os import os
import re import re
import sys import sys
from collections.abc import Mapping
from contextlib import suppress from contextlib import suppress
from functools import lru_cache from functools import lru_cache
from pathlib import Path
from subprocess import PIPE, STDOUT, CalledProcessError, CompletedProcess, Popen from subprocess import PIPE, STDOUT, CalledProcessError, CompletedProcess, Popen
from subprocess import run as subprocess_run from subprocess import run as subprocess_run
from typing import Callable, Optional, Type, Union from typing import Callable, Mapping, Sequence, Union
from packaging.version import Version from packaging.version import Version
@@ -26,11 +28,14 @@ from ocrmypdf.exceptions import MissingDependencyError
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
Args = Sequence[Union[Path, str]]
OsEnviron = os._Environ # pylint: disable=protected-access
def run( def run(
args, args: Args,
*, *,
env=None, env: OsEnviron | None = None,
logs_errors_to_stdout: bool = False, logs_errors_to_stdout: bool = False,
check: bool = False, check: bool = False,
**kwargs, **kwargs,
@@ -73,7 +78,12 @@ def run(
def run_polling_stderr( def run_polling_stderr(
args, *, callback: Callable[[str], None], check: bool = False, env=None, **kwargs args: Args,
*,
callback: Callable[[str], None],
check: bool = False,
env: OsEnviron | None = None,
**kwargs,
) -> CompletedProcess: ) -> CompletedProcess:
"""Run a process like ``ocrmypdf.subprocess.run``, and poll stderr. """Run a process like ``ocrmypdf.subprocess.run``, and poll stderr.
@@ -106,7 +116,9 @@ def run_polling_stderr(
return CompletedProcess(args, proc.returncode, None, stderr=stderr) return CompletedProcess(args, proc.returncode, None, stderr=stderr)
def _fix_process_args(args, env, kwargs): def _fix_process_args(
args: Args, env: OsEnviron | None, kwargs
) -> tuple[Args, OsEnviron, logging.Logger, bool]:
assert 'universal_newlines' not in kwargs, "Use text= instead of universal_newlines" assert 'universal_newlines' not in kwargs, "Use text= instead of universal_newlines"
if not env: if not env:
@@ -115,7 +127,7 @@ def _fix_process_args(args, env, kwargs):
# Search in spoof path if necessary # Search in spoof path if necessary
program = str(args[0]) program = str(args[0])
if os.name == 'nt': if sys.platform == 'win32':
# pylint: disable=import-outside-toplevel # pylint: disable=import-outside-toplevel
from ocrmypdf.subprocess._windows import fix_windows_args from ocrmypdf.subprocess._windows import fix_windows_args
@@ -123,14 +135,18 @@ def _fix_process_args(args, env, kwargs):
log.debug("Running: %s", args) log.debug("Running: %s", args)
process_log = log.getChild(os.path.basename(program)) process_log = log.getChild(os.path.basename(program))
text = kwargs.get('text', False) text = bool(kwargs.get('text', False))
return args, env, process_log, text return args, env, process_log, text
@lru_cache(maxsize=None) @lru_cache(maxsize=None)
def get_version( def get_version(
program: str, *, version_arg: str = '--version', regex=r'(\d+(\.\d+)*)', env=None program: str,
*,
version_arg: str = '--version',
regex=r'(\d+(\.\d+)*)',
env: OsEnviron | None = None,
) -> str: ) -> str:
"""Get the version of the specified program """Get the version of the specified program
@@ -229,7 +245,7 @@ a command prompt to install the missing package:
''' '''
def _get_platform(): def _get_platform() -> str:
if sys.platform.startswith('freebsd'): if sys.platform.startswith('freebsd'):
return 'freebsd' return 'freebsd'
elif sys.platform.startswith('linux'): elif sys.platform.startswith('linux'):
@@ -239,7 +255,7 @@ def _get_platform():
return sys.platform return sys.platform
def _error_trailer(program, package, **kwargs): def _error_trailer(program: str, package: str | Mapping[str, str], **kwargs) -> None:
del kwargs del kwargs
if isinstance(package, Mapping): if isinstance(package, Mapping):
package = package.get(_get_platform(), program) package = package.get(_get_platform(), program)
@@ -252,7 +268,9 @@ def _error_trailer(program, package, **kwargs):
log.info(WINDOWS_INSTALL_ADVICE.format(**locals())) log.info(WINDOWS_INSTALL_ADVICE.format(**locals()))
def _error_missing_program(program, package, required_for, recommended): def _error_missing_program(
program: str, package: str, required_for: str | None, recommended: bool
) -> None:
# pylint: disable=unused-argument # pylint: disable=unused-argument
if recommended: if recommended:
log.warning(MISSING_RECOMMEND_PROGRAM.format(**locals())) log.warning(MISSING_RECOMMEND_PROGRAM.format(**locals()))
@@ -263,7 +281,13 @@ def _error_missing_program(program, package, required_for, recommended):
_error_trailer(**locals()) _error_trailer(**locals())
def _error_old_version(program, package, need_version, found_version, required_for): def _error_old_version(
program: str,
package: str,
need_version: str,
found_version: str,
required_for: str | None,
) -> None:
# pylint: disable=unused-argument # pylint: disable=unused-argument
if required_for: if required_for:
log.error(OLD_VERSION_REQUIRED_FOR.format(**locals())) log.error(OLD_VERSION_REQUIRED_FOR.format(**locals()))
@@ -272,7 +296,7 @@ def _error_old_version(program, package, need_version, found_version, required_f
_error_trailer(**locals()) _error_trailer(**locals())
def _remove_leading_v(s): def _remove_leading_v(s: str) -> str:
if sys.version_info >= (3, 9): if sys.version_info >= (3, 9):
return s.removeprefix('v') return s.removeprefix('v')
@@ -285,12 +309,12 @@ def check_external_program(
*, *,
program: str, program: str,
package: str, package: str,
version_checker: Callable, version_checker: Callable[[], str],
need_version: str, need_version: str,
required_for: Optional[str] = None, required_for: str | None = None,
recommended=False, recommended: bool = False,
version_parser: Type[Version] = Version, version_parser: type[Version] = Version,
): ) -> None:
"""Check for required version of external program and raise exception if not. """Check for required version of external program and raise exception if not.
Args: Args:
+29 -18
View File
@@ -3,31 +3,44 @@
# This Source Code Form is subject to the terms of the Mozilla Public # This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this # License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
# type: ignore
# Non-Windows mypy now breaks when trying to typecheck winreg
"""Find Tesseract and Ghostscript binaries on Windows using the registry.""" """Find Tesseract and Ghostscript binaries on Windows using the registry."""
from __future__ import annotations
import logging import logging
import os import os
import shutil import shutil
import sys import sys
from itertools import chain from itertools import chain
from pathlib import Path from pathlib import Path
from typing import Any, Callable, Iterable, Iterator, Set, Tuple, TypeVar from typing import Any, Callable, Iterable, Iterator, TypeVar
if sys.version_info >= (3, 10):
from typing import TypeAlias
else:
from typing_extensions import TypeAlias # pragma: no cover
if sys.platform == 'win32':
# mypy understands 'if sys.platform' better than try/except ModuleNotFoundError
import winreg # pylint: disable=import-error
HKEYType: TypeAlias = winreg.HKEYType
else:
from unittest.mock import Mock
winreg = Mock(
spec=['HKEYType', 'EnumKey', 'EnumValue', 'HKEY_LOCAL_MACHINE', 'OpenKey']
)
# mypy does not understand winreg.HKeyType where winreg is a Mock (fair enough!)
HKEYType: TypeAlias = Any
try:
import winreg
except ModuleNotFoundError as _notfound_ex:
raise ModuleNotFoundError("This module is for Windows only") from _notfound_ex
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
T = TypeVar('T') T = TypeVar('T')
def ghostscript_version_key(s: str) -> Tuple[int, int, int]: def ghostscript_version_key(s: str) -> tuple[int, int, int]:
"""Compare Ghostscript version numbers.""" """Compare Ghostscript version numbers."""
try: try:
release = [int(elem) for elem in s.split('.', maxsplit=3)] release = [int(elem) for elem in s.split('.', maxsplit=3)]
@@ -38,9 +51,7 @@ def ghostscript_version_key(s: str) -> Tuple[int, int, int]:
return (0, 0, 0) return (0, 0, 0)
def registry_enum( def registry_enum(key: HKEYType, enum_fn: Callable[[HKEYType, int], T]) -> Iterator[T]:
key: winreg.HKEYType, enum_fn: Callable[[winreg.HKEYType, int], T]
) -> Iterator[T]:
limit = 999 limit = 999
n = 0 n = 0
while n < limit: while n < limit:
@@ -53,11 +64,11 @@ def registry_enum(
raise ValueError(f"Too many registry keys under {key}") raise ValueError(f"Too many registry keys under {key}")
def registry_subkeys(key: winreg.HKEYType) -> Iterator[str]: def registry_subkeys(key: HKEYType) -> Iterator[str]:
return registry_enum(key, winreg.EnumKey) return registry_enum(key, winreg.EnumKey)
def registry_values(key: winreg.HKEYType) -> Iterator[Tuple[str, Any, int]]: def registry_values(key: HKEYType) -> Iterator[tuple[str, Any, int]]:
return registry_enum(key, winreg.EnumValue) return registry_enum(key, winreg.EnumValue)
@@ -73,8 +84,8 @@ def registry_path_ghostscript(env=None) -> Iterator[Path]:
with winreg.OpenKey( with winreg.OpenKey(
winreg.HKEY_LOCAL_MACHINE, fr"SOFTWARE\Artifex\GPL Ghostscript\{latest_gs}" winreg.HKEY_LOCAL_MACHINE, fr"SOFTWARE\Artifex\GPL Ghostscript\{latest_gs}"
) as k: ) as k:
_, gs_path, _ = next(registry_values(k)) for _, gs_path, _ in registry_values(k):
yield Path(gs_path) / 'bin' yield Path(gs_path) / 'bin'
except OSError as e: except OSError as e:
log.warning(e) log.warning(e)
@@ -160,7 +171,7 @@ def unique_everseen(iterable: Iterable[T], key: Callable[[T], T]) -> Iterator[T]
"List unique elements, preserving order." "List unique elements, preserving order."
# unique_everseen('AAAABBBCCDAABBB') --> A B C D # unique_everseen('AAAABBBCCDAABBB') --> A B C D
# unique_everseen('ABBCcAD', str.lower) --> A B C D # unique_everseen('ABBCcAD', str.lower) --> A B C D
seen: Set[T] = set() seen: set[T] = set()
seen_add = seen.add seen_add = seen.add
for element in iterable: for element in iterable:
k = key(element) k = key(element)
+2
View File
@@ -4,4 +4,6 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this # License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
# Empty __init__.py file # Empty __init__.py file
+3 -1
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import os import os
import platform import platform
import sys import sys
@@ -52,7 +54,7 @@ def resources() -> Path:
@pytest.fixture @pytest.fixture
def ocrmypdf_exec() -> List[str]: def ocrmypdf_exec() -> list[str]:
return [sys.executable, '-m', 'ocrmypdf'] return [sys.executable, '-m', 'ocrmypdf']
+2
View File
@@ -19,6 +19,8 @@
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE # TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
from __future__ import annotations
from unittest.mock import patch from unittest.mock import patch
from ocrmypdf import hookimpl from ocrmypdf import hookimpl
+2
View File
@@ -19,6 +19,8 @@
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE # TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
from __future__ import annotations
from unittest.mock import patch from unittest.mock import patch
from ocrmypdf import hookimpl from ocrmypdf import hookimpl
+2
View File
@@ -19,6 +19,8 @@
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE # TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
from __future__ import annotations
from pathlib import Path from pathlib import Path
from subprocess import CalledProcessError from subprocess import CalledProcessError
from unittest.mock import patch from unittest.mock import patch
+2
View File
@@ -19,6 +19,8 @@
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE # TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
from __future__ import annotations
from subprocess import CalledProcessError from subprocess import CalledProcessError
from unittest.mock import patch from unittest.mock import patch
+2
View File
@@ -26,6 +26,8 @@ that is not UTF-8 compatible, so we are forced to check that we can convert it
and present it to the user. and present it to the user.
""" """
from __future__ import annotations
from contextlib import contextmanager from contextlib import contextmanager
from subprocess import CalledProcessError from subprocess import CalledProcessError
from unittest.mock import patch from unittest.mock import patch
@@ -19,6 +19,8 @@
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE # TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
from __future__ import annotations
from contextlib import contextmanager from contextlib import contextmanager
from subprocess import CalledProcessError from subprocess import CalledProcessError
from unittest.mock import patch from unittest.mock import patch
+2
View File
@@ -44,6 +44,8 @@ Assumes Tesseract 4.0.0-alpha or higher.
""" """
from __future__ import annotations
import argparse import argparse
import json import json
import logging import logging
+2
View File
@@ -19,6 +19,8 @@
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE # TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
from __future__ import annotations
import signal import signal
from contextlib import contextmanager from contextlib import contextmanager
from subprocess import CalledProcessError from subprocess import CalledProcessError
+2
View File
@@ -31,6 +31,8 @@ In 'pdf' mode, convert the image to PDF using another program.
In orientation check mode, report 0, 90, 180, 270... based on page number. In orientation check mode, report 0, 90, 180, 270... based on page number.
""" """
from __future__ import annotations
import pikepdf import pikepdf
from PIL import Image from PIL import Image
+2
View File
@@ -30,6 +30,8 @@ In 'pdf' mode, convert the image to PDF using another program.
In orientation check mode, report the orientation is upright. In orientation check mode, report the orientation is upright.
""" """
from __future__ import annotations
import pikepdf import pikepdf
from PIL import Image from PIL import Image
@@ -19,8 +19,6 @@
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE # TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
# type: ignore
"""Tesseract no-op plugin that simulates the OOM killer on page 4. """Tesseract no-op plugin that simulates the OOM killer on page 4.
OCRmyPDF can use a lot of memory, even that it might trigger the OCRmyPDF can use a lot of memory, even that it might trigger the
@@ -30,14 +28,18 @@ ensure we fail with an error rather than deadlock in such cases.
Page 4 was chosen because of this number's association with bad luck Page 4 was chosen because of this number's association with bad luck
in many East Asian cultures. in many East Asian cultures.
""" """
# type: ignore
from __future__ import annotations
import os import os
import signal import signal
import sys
from pathlib import Path from pathlib import Path
from ocrmypdf import hookimpl from ocrmypdf import hookimpl
# type: ignore
# Ugly hack that let us use the NoopOcrEngine without setting up packaging for our # Ugly hack that let us use the NoopOcrEngine without setting up packaging for our
# tests. # tests.
# This hack also requires us to set type: ignore # This hack also requires us to set type: ignore
@@ -47,7 +49,7 @@ exec(parent)
NoopOcrEngine = locals()['NoopOcrEngine'] NoopOcrEngine = locals()['NoopOcrEngine']
class Page4Engine(NoopOcrEngine): class Page4Engine(NoopOcrEngine): # type: ignore
def __str__(self): def __str__(self):
return f"NO-OP Page 4 {NoopOcrEngine.version()}" return f"NO-OP Page 4 {NoopOcrEngine.version()}"
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import logging import logging
import pytest import pytest
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import logging import logging
from io import BytesIO, StringIO from io import BytesIO, StringIO
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import pytest import pytest
from ocrmypdf.helpers import check_pdf from ocrmypdf.helpers import check_pdf
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import os import os
from subprocess import PIPE, run from subprocess import PIPE, run
+2
View File
@@ -4,6 +4,8 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this # License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import os import os
import pytest import pytest
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import logging import logging
import subprocess import subprocess
from decimal import Decimal from decimal import Decimal
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
from unittest.mock import patch from unittest.mock import patch
import pikepdf import pikepdf
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import logging import logging
import multiprocessing import multiprocessing
import os import os
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import re import re
from io import StringIO from io import StringIO
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
from unittest.mock import patch from unittest.mock import patch
import img2pdf import img2pdf
+2
View File
@@ -4,6 +4,8 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this # License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import logging import logging
import pytest import pytest
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import os import os
import shutil import shutil
from math import isclose from math import isclose
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import datetime import datetime
import warnings import warnings
from datetime import timezone from datetime import timezone
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
from os import fspath from os import fspath
from pathlib import Path from pathlib import Path
from unittest.mock import patch from unittest.mock import patch
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import pytest import pytest
import ocrmypdf import ocrmypdf
+2
View File
@@ -4,6 +4,8 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this # License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import os import os
import pikepdf import pikepdf
+2
View File
@@ -4,6 +4,8 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this # License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import pickle import pickle
from io import BytesIO from io import BytesIO
from math import isclose from math import isclose
+2
View File
@@ -4,6 +4,8 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this # License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
from unittest.mock import Mock from unittest.mock import Mock
import pytest import pytest
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
from math import isclose from math import isclose
import pytest import pytest
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import pytest import pytest
from ocrmypdf import quality as qual from ocrmypdf import quality as qual
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import operator import operator
from io import BytesIO from io import BytesIO
from math import cos, pi, sin from math import cos, pi, sin
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import os import os
import sys import sys
from pathlib import Path from pathlib import Path
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import logging import logging
import os import os
import subprocess import subprocess
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import logging import logging
from os import fspath from os import fspath
from unittest.mock import patch from unittest.mock import patch
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
from math import isclose from math import isclose
import pytest import pytest
+2
View File
@@ -5,6 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import annotations
import logging import logging
import os import os
from unittest.mock import patch from unittest.mock import patch