Replaces os.path/open()/os.stat()/os.chmod() calls with their Path
method equivalents across src, tests, misc, and bin, wrapping str
variables in Path(...) where they must stay str for other uses (e.g.
subprocess argv, CLI-arg formatting). helpers.safe_symlink() now
decodes StrOrBytesPath to a str Path via os.fsdecode() upfront, same
pattern already used elsewhere for the str|bytes union.
Pillow >=9.2 ships its own inline types (py.typed), which the separate
types-Pillow stub package shadows when both are installed. Since
types-Pillow lived in the `test` dependency group (not `dev`), a local
`uv sync --group dev --group test` env had it installed and mypy
silently preferred its stubs, while CI's lint job only syncs the
default `dev` group and used Pillow's own (correct) stubs - surfacing
5 real "Incompatible types in assignment" errors that never appeared
locally.
Fixed the underlying type errors in _pipeline.py and ghostscript.py:
`im` was inferred as ImageFile.ImageFile from `Image.open(...) as im`,
but later reassigned the result of `.resize()`/`.transpose()`, which
return the broader Image.Image - now declared explicitly as
`im: Image.Image` before the `with` block.
Fix the 13 errors that surfaced under mypy --check-untyped-defs so the
flag can be turned on permanently in pyproject.toml, and drop the
advisory exit-0 wrapper on the mypy pre-commit hook now that the tree is
clean.
- _plugin_manager: rename colliding loop vars (module/name were reused
with conflicting types) and guard spec/spec.loader from
spec_from_file_location; call __init__ via the class in __setstate__.
- __main__: pass Verbosity(options.verbose), not a bare int.
- subprocess/_check: widen package to str | Mapping[str, str] to match
_error_trailer's existing per-platform handling.
- optimize.main: annotate the standalone PdfContext(..., None, None) that
only ever reads context.options.
Reduces mypy errors in src/ocrmypdf and tests from 89 to 51.
- Replace the `deprecation` package with stdlib `warnings.deprecated`
(falling back to typing_extensions on <3.13); drop the dependency.
- Add pypdfium2/uharfbuzz/pi_heif to mypy's ignore_missing_imports
overrides (no upstream stubs); drop pluggy, which now ships py.typed.
- Add a tests.* mypy override so test functions aren't required to
annotate -> None.
Real bugs found and fixed along the way, not just annotations:
- OcrmypdfPluginManager had a `pluggy` property shadowing the `pluggy`
module import within its own class body, breaking every
`pluggy.PluginManager` annotation below it; renamed to
`pluggy_manager`.
- `_option_registry` was bolted onto OcrmypdfPluginManager from outside
and read via `getattr(..., None)` instead of being a declared
attribute; declared it properly.
- ValidationCoordinator.__init__ was typed to accept a raw
pluggy.PluginManager, but every caller passes the OcrmypdfPluginManager
wrapper.
- check_options_sidecar() did `options.output_file + '.txt'`, assuming
output_file is always a str; would raise a raw TypeError if ever hit
with a stream/bytes output. Added an explicit guard.
- is_file_writable() called Path(test_file), which raises TypeError on
a bytes path; fixed via os.fsdecode().
- copy_final() had a dead, unused `original_file` parameter; removed it.
- run_hocr_pipeline() constructed PdfContext with the raw, untriaged
input_file instead of the locally-copied origin_pdf, inconsistent
with the other two pipelines.
- _options.py had jbig2_threshold declared twice in the same model.
prek runs local hooks as plain execs against tools uv already provisions,
so ruff/mypy can never drift from the versions/config uv.lock pins
elsewhere and CI needs no separate hook-cache download.
- Add ruff and prek to the uv dev dependency group (ruff wasn't a
uv-managed dependency before; pre-commit silently vendored its own).
- Replace .pre-commit-config.yaml with prek.toml: keep the
pre-commit-hooks repo for generic file checks, convert ruff-format/
ruff-check to local `uv run ruff ...` hooks, and add a local mypy
hook that reports but never fails (87 pre-existing errors need a
separate cleanup before it can be made blocking).
- Add a `lint` job to CI that runs `prek run --all-files` and gate the
OS/Python test matrix on it so lint issues fail fast.
- Fix the ruff debt (format + lint) uncovered by actually running it,
since it was small and mechanical, so the new CI gate starts green.
Since v16.4.3, OCRmyPDF forced pdfminer's read buffer to 256 MiB to work
around a pdfminer bug that mishandled tokens split across the buffer
boundary (gh #1361). On Windows this caused a severe performance
regression (gh #1662): CPython's BufferedReader.read(n) eagerly allocates
an n-byte buffer on every read, so pdfminer's thousands of seek+read
cycles each paid a ~30 ms 256 MiB allocation (this allocation is lazy and
effectively free on Linux). For a typical PDF the "Scanning contents"
phase went from ~5s on Linux to ~60s on Windows.
The underlying pdfminer bug was fixed upstream in pdfminer.six 20250327
(pdfminer/pdfminer.six#1030), with a follow-up for tokens split across
streams in 20260107 (pdfminer/pdfminer.six#1158). Remove the monkeypatch
entirely and raise the minimum pdfminer.six to 20260107 so we rely on the
upstream fix instead.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Replace hatch-vcs dynamic versioning with static version in _version.py
and pyproject.toml. Split CI into build.yml (test + stage draft release
on main) and release.yml (publish from draft on tag push). Docker images
are built on main pushes and re-tagged with the release version on tag
push without rebuilding.
multiprocessing.Queue is a factory function, not a type, so it cannot
be used with the runtime | union operator. Move Queue, UserInit, and
WorkerInit type aliases into TYPE_CHECKING block to avoid evaluation
at runtime.
Python 3.11 is now the minimum supported version. This aligns with
the codebase's use of StrEnum (introduced in 3.11) and removes
compatibility shims that were only needed for older versions.
The pypdfium rasterizer was producing output images that differed by 1
pixel compared to Ghostscript due to floating-point precision issues in
dimension calculations.
Root cause:
- pypdfium used harmonic mean of x/y DPI to calculate a single scale
factor, losing the distinction between x and y DPI
- No DPI rounding like Ghostscript's 6-decimal precision
- Compound rounding errors when converting points to pixels
Solution:
1. Round DPI to 6 decimals to match Ghostscript's precision
2. Calculate expected output dimensions using separate x/y DPI values
3. Handle dimension swapping for 90°/270° rotations
4. Resize output image if off by 1-2 pixels (graceful correction)
This ensures pixel-perfect matching with Ghostscript while being
minimally invasive and only resizing when necessary.
Changes:
- Modified _render_page_to_bitmap() to calculate expected dimensions
- Modified _process_image_for_output() to correct small discrepancies
- Updated rasterize_pdf_page() to pass dimensions through pipeline
- Parametrized rotation tests to run with both rasterizers
All 45 rotation tests now pass with both pypdfium and ghostscript.
Fixes test_rotated_skew_timeout with pypdfium rasterizer.
Migrate watcher.py and pdf_text_diff.py from typer to cyclopts for
CLI argument parsing. Update pyproject.toml to reflect the dependency
change in the watcher optional feature.
Establish clear separation between user-facing optional dependencies
and developer-only dependency groups:
**Optional Dependencies (user features):**
- watcher: File watching service for batch processing
- webservice: Streamlit-based web UI
- Installable via: uv sync --extra <name> or pip install ocrmypdf[name]
**Dependency Groups (developer tools):**
- test: Testing infrastructure (merged from test + extended_test)
- docs: Documentation building tools
- streamlit-dev: Enhanced Streamlit development tools
- dev: General development tools (mypy, ipykernel)
- Installable via: uv sync --group <name> (uv only, NOT pip)
Breaking changes for developers:
- pip install -e .[test] no longer works → use uv sync --group test
- pip install -e .[docs] no longer works → use uv sync --group docs
- pip install -e .[extended_test] removed → merged into test group
No breaking changes for end users:
- pip install ocrmypdf[watcher] still works
- pip install ocrmypdf[webservice] still works
Updated:
- CI/CD workflows to use uv sync --group test
- Docker images to exclude test dependencies
- Documentation to recommend uv with pip as fallback
- pyproject.toml with clear comments explaining both systems
- Update pipeline to use fpdf2 renderer as default
- Remove legacy hocrtransform PDF renderer (_font.py, _hocr.py,
pdf_renderer.py)
- Update CLI and options for fpdf2 renderer
- Add fpdf2 dependency to pyproject.toml
- Update graft module for fpdf2 multi-page rendering
In 0.12.1, Typer was significantly reorganized.
- `typer-slim` is the library (for `import typer`)
- `typer-slim[standard]` adds optional dependencies (currently `rich`
and `shellingham`, basically equivalent to the old `typer[all]`)
- `typer-cli` is the `typer` command-line tool
- `typer` is now basically a metapackage that brings in *all of the
above*, and it no longer has an `all` extra
Pip will warn about this and proceed,
```
WARNING: typer 0.12.1 does not provide the extra 'all'
```
but there are other tools that will fail hard when asked to resolve a
(now) nonexistent extra.
Since this project doesn’t need the `typer` command-line tool, it looks
like changing the dependency to `typer-slim[standard]` is the best way
forward.
See https://typer.tiangolo.com/release-notes/#0121 and
tiangolo/typer#785 for further discussion
and details.
James R. Barlow
Chris MayoandGitHub
Ben BeasleyandGitHub