stage_release runs on every push to main and unconditionally deletes
and recreates the draft release for the version in _version.py. Since
_version.py isn't bumped until sometime after a release is tagged and
published by release.yml, every intervening push to main was clobbering
the just-published release back to draft status. Confirmed this hit
v17.3.0 through v17.8.0 (and briefly v17.8.1, manually fixed).
Skip the delete/recreate if a non-draft release already exists for
this tag.
A malformed PDF may store a non-dictionary object (an array, name, or
other type) at /Resources or /Resources /XObject. The pdfinfo image
scanner iterated these with .items()/.as_dict() and probed them with the
`in` operator, which raise TypeError/ValueError on non-dictionary pikepdf
objects and crashed PdfInfo on otherwise-processable files. OCRmyPDF's
domain is messy, machine-generated PDFs, so scanning must tolerate this.
Guard _image_xobjects and _find_form_xobject_images with
isinstance(x, Dictionary) before iterating, treating a non-dictionary
/Resources or /XObject as "no image XObjects". This is the same
robustness class as the pdfa.py find_nonembedded_cid_fonts fix, applied
to the pdfinfo image scanner.
Follow-on to the non-dictionary /Font and /XObject guard. Replace the
_dict_entries() helper with an isinstance(..., pikepdf.Dictionary) check
at each resource lookup, which pikepdf's metaclass supports directly and
which reads as exactly the invariant being enforced. Iterate via
as_dict().values() so the values are typed and mypy stays clean once the
Any from the untyped resources argument is narrowed away.
Also guard _cid_font_is_embedded against a non-dictionary /FontDescriptor:
`key in descriptor` raises ValueError on a non-dict, which the caller's
except (AttributeError, TypeError, KeyError) does not catch. Such a font
now counts as non-embedded and is reported, so PDF/A conversion is refused
rather than risking Ghostscript corrupting a pre-existing CID text layer.
Adds a regression test for the FontDescriptor case (issue #1713).
Replaces os.path/open()/os.stat()/os.chmod() calls with their Path
method equivalents across src, tests, misc, and bin, wrapping str
variables in Path(...) where they must stay str for other uses (e.g.
subprocess argv, CLI-arg formatting). helpers.safe_symlink() now
decodes StrOrBytesPath to a str Path via os.fsdecode() upfront, same
pattern already used elsewhere for the str|bytes union.
Pillow >=9.2 ships its own inline types (py.typed), which the separate
types-Pillow stub package shadows when both are installed. Since
types-Pillow lived in the `test` dependency group (not `dev`), a local
`uv sync --group dev --group test` env had it installed and mypy
silently preferred its stubs, while CI's lint job only syncs the
default `dev` group and used Pillow's own (correct) stubs - surfacing
5 real "Incompatible types in assignment" errors that never appeared
locally.
Fixed the underlying type errors in _pipeline.py and ghostscript.py:
`im` was inferred as ImageFile.ImageFile from `Image.open(...) as im`,
but later reassigned the result of `.resize()`/`.transpose()`, which
return the broader Image.Image - now declared explicitly as
`im: Image.Image` before the `with` block.
Fix the 13 errors that surfaced under mypy --check-untyped-defs so the
flag can be turned on permanently in pyproject.toml, and drop the
advisory exit-0 wrapper on the mypy pre-commit hook now that the tree is
clean.
- _plugin_manager: rename colliding loop vars (module/name were reused
with conflicting types) and guard spec/spec.loader from
spec_from_file_location; call __init__ via the class in __setstate__.
- __main__: pass Verbosity(options.verbose), not a bare int.
- subprocess/_check: widen package to str | Mapping[str, str] to match
_error_trailer's existing per-platform handling.
- optimize.main: annotate the standalone PdfContext(..., None, None) that
only ever reads context.options.
Resolves the last 16 mypy errors in the project (src/ocrmypdf and
tests are now fully clean).
fpdf_renderer/renderer.py (9 errors):
- add_page(format=...): fpdf2's own stub types this param as str, but
its docstring and get_page_format() helper confirm a (width, height)
tuple is accepted too - the stub annotation on add_page() itself is
the outlier. Used cast() to match the documented/actual behavior.
- pdf.current_font is typed CoreFont | TTFFont | None, but this
renderer only ever registers fonts via add_font() with a TTF file
(see _register_font/set_font call sites) - it never falls back to
fpdf2's built-in CoreFont. Added assertions (isinstance(font,
TTFFont) where shape_text()/escape_text() are needed, which
CoreFont lacks; plain not-None elsewhere) documenting that
invariant instead of narrowing defensively for a case that can't
happen here.
tests/test_pdf_renderer.py (7 errors): the ToUnicode/glyph-extraction
test helpers used `.get(key, {})` (a plain dict literal default) then
called `.values()`/`.items()` on the result. pikepdf.Object doesn't
declare `values()` in its stub (only `keys()`), so this silently
degraded to Object's catch-all `__getattr__` returning another Object,
which then failed as "not callable". Switched to `.get(key,
Dictionary()).as_dict()`, which returns pikepdf's properly-typed
_ObjectMapping helper.
Resolves all 11 remaining errors in _concurrent.py and
builtin_plugins/concurrency.py.
Root cause for 9 of the 11: setup_executor() was annotated to return
ocrmypdf's own Executor ABC as its second tuple element, but it
actually returns a concurrent.futures pool class (ThreadPoolExecutor
or ProcessPoolExecutor) - an already-existing FuturesExecutorClass
alias was defined for exactly this but never wired into the
signature. That wrong annotation made mypy check
`executor_class(initializer=..., initargs=...)` in _execute() against
Executor.__call__'s signature (which has entirely different
parameters and returns None), cascading into 8 further errors
(unexpected keyword args, "function does not return a value", "None
has no attribute __enter__/__exit__"). Fixing the one annotation
resolved all of them.
Also:
- Added a proper Queue[LogRecord | None] generic parameter (was bare
Queue, which needs an explicit type argument for mypy to infer
loq_queue's type across the use_threads branches).
- _concurrent.py: Executor.__call__'s task parameter defaults to
_task_noop (return type None) when the caller omits a task, but the
parameter is typed Callable[..., T] for an unbound per-call T. Used
cast() since task_finished's own no-op default already accepts Any,
so the mismatch is never actually exercised unsafely.
Resolves the remaining 22 pikepdf-related mypy errors in
pdfinfo/_image.py, pdfinfo/info.py, optimize.py, and _graft.py
(89 -> 27 errors remaining, all in the concurrency/fpdf_renderer
clusters).
Adds pikepdf_get_int()/pikepdf_get_bool() to helpers.py: safe
accessors for dict.get(key, default) results, whose static type is
the ambiguous `Object | int`/`Object | bool` and doesn't support
arithmetic/comparison against a plain int/bool.
Important correctness fix caught by the test suite: pikepdf only
returns pikepdf.Object wrappers under explicit_conversion() mode,
which this codebase never enables. By default (implicit mode), PDF
Integers/Booleans are already unboxed to native int/bool by the time
callers see them, so calling the `.as_int()`/`.as_bool()`/
`.as_decimal()` safe accessors unconditionally crashes with
AttributeError on the native-type case (test_oversized_page caught
this for UserUnit). Fixed by using int()/float() builtins, which work
polymorphically on both native numbers and pikepdf.Object (bool()
does not, so pikepdf_get_bool checks isinstance first).
Also:
- pdfinfo/info.py: pass page.obj (an Object) to
_process_content_streams() instead of page (a Page wrapper, not an
Object subtype).
- pdfinfo/_image.py: cast() around Matrix(array_object) - pikepdf's
stub omits the Object/Array constructor overload that the C++
implementation actually supports.
- optimize.py: cast() around Object.write()'s filter/decode_parms
args for the same reason.
- _graft.py: iterate parse_content_stream() via .operands/.operator
instead of tuple-unpacking, since ContentStreamInstruction supports
the legacy __getitem__-based iteration protocol but not __iter__,
which mypy doesn't statically recognize.
- pdfinfo/_worker.py: _pdf_pageinfo_concurrent's return type was
Sequence[PageInfo | None] but always returns a real list; narrowed
to match, fixing PdfInfo.pages' declared list[...] return type.
Reduces mypy errors in src/ocrmypdf and tests from 89 to 51.
- Replace the `deprecation` package with stdlib `warnings.deprecated`
(falling back to typing_extensions on <3.13); drop the dependency.
- Add pypdfium2/uharfbuzz/pi_heif to mypy's ignore_missing_imports
overrides (no upstream stubs); drop pluggy, which now ships py.typed.
- Add a tests.* mypy override so test functions aren't required to
annotate -> None.
Real bugs found and fixed along the way, not just annotations:
- OcrmypdfPluginManager had a `pluggy` property shadowing the `pluggy`
module import within its own class body, breaking every
`pluggy.PluginManager` annotation below it; renamed to
`pluggy_manager`.
- `_option_registry` was bolted onto OcrmypdfPluginManager from outside
and read via `getattr(..., None)` instead of being a declared
attribute; declared it properly.
- ValidationCoordinator.__init__ was typed to accept a raw
pluggy.PluginManager, but every caller passes the OcrmypdfPluginManager
wrapper.
- check_options_sidecar() did `options.output_file + '.txt'`, assuming
output_file is always a str; would raise a raw TypeError if ever hit
with a stream/bytes output. Added an explicit guard.
- is_file_writable() called Path(test_file), which raises TypeError on
a bytes path; fixed via os.fsdecode().
- copy_final() had a dead, unused `original_file` parameter; removed it.
- run_hocr_pipeline() constructed PdfContext with the raw, untriaged
input_file instead of the locally-copied origin_pdf, inconsistent
with the other two pipelines.
- _options.py had jbig2_threshold declared twice in the same model.
prek runs local hooks as plain execs against tools uv already provisions,
so ruff/mypy can never drift from the versions/config uv.lock pins
elsewhere and CI needs no separate hook-cache download.
- Add ruff and prek to the uv dev dependency group (ruff wasn't a
uv-managed dependency before; pre-commit silently vendored its own).
- Replace .pre-commit-config.yaml with prek.toml: keep the
pre-commit-hooks repo for generic file checks, convert ruff-format/
ruff-check to local `uv run ruff ...` hooks, and add a local mypy
hook that reports but never fails (87 pre-existing errors need a
separate cleanup before it can be made blocking).
- Add a `lint` job to CI that runs `prek run --all-files` and gate the
OS/Python test matrix on it so lint issues fail fast.
- Fix the ruff debt (format + lint) uncovered by actually running it,
since it was small and mechanical, so the new CI gate starts green.
The test asserted that --mode skip preserves the structure tree, but with
the default --output-type auto the output runs through Ghostscript PDF/A
conversion, which discards /StructTreeRoot on Ghostscript 10.x (9.x kept
it). This failed on macOS CI and locally while passing on the Ubuntu
runners' Ghostscript 9.55. Pin the test to --output-type pdf so it exercises
OCRmyPDF's own structure-tree handling without the version-dependent GS step,
and document the caveat in advanced.md.
Ghostscript's PDF/A conversion re-embeds non-embedded CID (CJK) fonts by
substituting a system font, which corrupts the character-to-Unicode
mapping and silently destroys an existing text layer -- commonly the OCR
layer Adobe Acrobat adds to scanned CJK documents.
Detect non-embedded CID/Type0 fonts before conversion: with
--output-type auto (the default) downgrade to a regular PDF and preserve
the text layer; with an explicit --output-type pdfa* stop with an error
rather than emit corrupted output. Simple non-embedded fonts (e.g. Latin)
are left alone -- Ghostscript substitutes them without corrupting the
text, and they are far too common to treat as conversion blockers.
Use --output-type pdf to keep the existing text layer, or --force-ocr to
rebuild it with embedded fonts.
Writing the output PDF to stdout (ocrmypdf in.pdf -) previously relied on
an honor system: no in-process code -- third-party libraries, plugins, or
stray print() calls -- was supposed to write to stdout, enforced only
indirectly. A single accidental write to fd 1 would silently corrupt the
output PDF.
Enforce this at the OS level. At CLI startup, before plugins load or any
worker process/thread starts, save the real stdout via os.dup() and point
fd 1 at stderr, so stray writes are diverted to stderr while only the final
"produce the PDF" step writes to the preserved descriptor. Exposed as the
opt-in public API function configure_stdout_protection(), mirroring
configure_logging(); it is not enabled inside ocr() so in-process library
users keep their own stdout.
Also fix check_requested_output_file() to test the preserved real stdout
for tty-ness, since after the redirect sys.stdout reports stderr's status.
Fold unreleased v17.7.2 notes into v17.8.0.
Some PDFs use a /Name dictionary key in /DocumentInfo whose bytes are not
valid UTF-8/PDFDocEncoding, e.g. a Latin-1 /Saks#e5r. Older pikepdf raised
UnicodeDecodeError while iterating such a block, crashing the pipeline
during PDF/A conversion. repair_docinfo_nuls is documented to log and
continue on a malformed DocumentInfo block, so catch UnicodeDecodeError
alongside TypeError.
Add a mock-based unit test that drives the decode-error branch (current
pikepdf surrogate-escapes instead of raising) and an end-to-end test over
the reporter's file, committed as docinfo_latin1_key.pdf.
ReadTheDocs stopped injecting the GitHub edit context when it migrated to
Addons, so the sphinx_rtd_theme "Edit on GitHub" breadcrumb link vanished,
leaving only the static "View page source" (_sources/*.txt) copy. Set the
html_context explicitly so each page links back to its source on GitHub.
When TESSDATA_PREFIX points at a hand-assembled tessdata folder lacking
the configs/ subdirectory (e.g. files pulled from tessdata_best),
Tesseract prints "read_params_file: Can't open hocr/txt" and produces no
output. The runtime now surfaces a clear error (v17.5.0); add the
matching documentation: a new errors.md entry and a note on the
TESSDATA_PREFIX docs.
A page whose MediaBox has a non-zero origin (e.g. from PDF Arranger crops)
was rendered blank by --force-ocr in v16.12.0, because fix_pagepdf_boxes
offset the CropBox with the wrong sign, pushing it entirely outside the
image-page MediaBox. The behavior was fixed in v17.0.0 but had no
end-to-end test: the existing box tests only assert that ocrmypdf runs,
which the blank-page bug passed (it exited 0 with a valid, empty PDF/A).
Add a test that renders the output and asserts the visible content
survives, parametrized over both the ghostscript and pypdfium rasterizers
since the bug reproduced regardless of rasterizer.
Harden the Docker images by dropping root privileges, and make the
bind-mount workflow less fiddly.
- Create a non-root `app` user (uid/gid 1000) in both images and add
`USER app` before the entrypoint, so ocrmypdf (and the
webservice/watcher) no longer run as root. This also fixes the
previously dangling `--chown=app:app`, which referenced a user that
was never created. The Ubuntu base ships a default `ubuntu`/1000 user,
so remove it first so `app` can take uid 1000 (parity with Alpine).
- Add `WORKDIR /data` (created and app-owned) so bind-mounted input and
output can be passed as relative paths without `--workdir`. The
webservice/watcher are now invoked by absolute path (`/app/*.py`)
since the working directory is no longer `/app`.
- Drop the redundant `ppa:alex-p/tesseract-ocr5` from the Ubuntu image:
Tesseract 5 ships in the Ubuntu archive as of 24.04, and the PPA had
no build for the 26.04 base, which broke the build outright.
- Rewrite docs/docker.md rootless-first: stdin/stdout piping as the
recommended permission-free path, then per-runtime volume guidance
(rootless Docker `--user 0:0`, Podman `--userns keep-id`, rootful
Docker as the special case). Update batch.md and the compose example
to match (absolute script paths, per-runtime `user:` guidance).
Adds a processing mode that removes the invisible (render mode 3) OCR text
layer in place. Unlike `--ocr-engine none --force-ocr`, it does not
rasterize the page, so images and visible content are preserved unchanged
and the output is smaller rather than larger. Options that require
rasterization or OCR (--deskew, --clean, --sidecar, etc.) are rejected.
Only invisible text is removed; text drawn as visible glyphs under an
opaque image (some OCR engines, and OCRmyPDF v2.2 and earlier) cannot be
removed this way, as documented.
Closes#1435.
When Tesseract reports a page with many diacritics, OCRmyPDF rewrites the
message to "lots of diacritics - possibly poor OCR", which hid the
original wording. Keep the interpreted hint but also emit Tesseract's raw
line at debug verbosity (-v 1) so users can see exactly what Tesseract
reported.
Closes#1566.
Ghostscript 10.x renders aliased glyphs that OCR frequently misreads as
extra word breaks or substituted characters. Enable text and graphics
anti-aliasing (-dTextAlphaBits=4 -dGraphicsAlphaBits=4) for the contone
raster devices, which empirically improves OCR accuracy on the
Ghostscript path, especially for small fonts at moderate DPI. The 1-bit
mono devices are excluded, since older Ghostscript rejects alpha bits on
them and pngmonod performs its own anti-aliased downscaling.
Also log which rasterizer rendered each page at debug verbosity and
clarify the --rasterizer help text, so quality reports are easier to
diagnose. The default rasterizer (auto) already prefers pypdfium2, which
anti-aliases; this primarily benefits --rasterizer ghostscript and
installs without pypdfium2.
Closes#1439.
The optimizer is best-effort: any image it cannot process is left
unchanged in the output, which remains valid. Previously, an extraction
failure (e.g. an exotic colorspace pikepdf cannot transcode) was logged
with log.exception, printing a full traceback at ERROR level that alarmed
users even though nothing was wrong with the output (issue #846).
Trap such failures with a concise warning that the image was left
unchanged and the output is still valid, and demote the traceback to
debug verbosity for diagnosis.
Ghostscript's `auto` image compression heuristic can transcode lossless
images to JPEG during PDF/A generation, which is surprising at
optimization levels that otherwise promise lossless-only operations
(issue #1124).
`--pdfa-image-compression=auto` (the default) now coerces to lossless at
-O0 so Ghostscript will not transcode lossless images to JPEG. -O1 and
above continue to defer to Ghostscript's heuristic; -O1 (the default
level) is kept as a historical exception because coercing it to lossless
substantially bloats output. Users wanting guaranteed lossless image
handling can pass --pdfa-image-compression=lossless or use -O0.
Also make `lossless` pass existing JPEGs through unchanged
(-dPassThroughJPEGImages=true) instead of re-encoding them with a
lossless codec, which only inflates already-lossy data.
Validate and repair the page-boundary boxes (MediaBox, CropBox, TrimBox,
ArtBox, BleedBox) of input PDFs in triage(), following PDF 2.0:
- Coerce coordinates written in invalid exponential notation, stored by
qpdf/pikepdf as strings (#1398).
- Normalize rectangles whose corners are given in reversed order, which
previously crashed with NegativeDimensionError (#1526).
- Clamp a crop/trim/art/bleed box that extends outside the MediaBox to
their intersection, or discard it when the intersection is empty, which
previously produced a zero-height effective page some viewers rejected
(#1400).
When a box is discarded, clamped, or reinterpreted, a warning recommends
visual inspection of the output. pdfinfo box reading shares the same
coercion helper so PdfInfo no longer crashes on malformed boxes.
Supersedes PR #1691. Thanks @ajdlinux.
Closes#1398, #1526, #1400
A tagged/structured PDF carries a logical structure tree
(/Root/StructTreeRoot, /MarkInfo) that maps marked page content to
semantic elements via MCIDs. When --force-ocr rasterizes pages or
--redo-ocr rewrites the text layer, those MCIDs are destroyed or
renumbered and the tree is left dangling. We cannot rebuild it to match
the new text, so discard it for force/redo modes, following the same
pattern as the thumbnail and search-index discards. --skip-text leaves
text pages untouched, so their structure is preserved.
Also broaden the default-mode "looks born-digital" stop signal to fire
on /StructTreeRoot, not just /MarkInfo/Marked, so structure-tree-only
PDFs are no longer silently OCR'd. The existing --tagged-pdf-mode ignore
escape hatch is unchanged.
A page may carry an optional /Thumb image XObject (ISO 32000-2 12.3.4),
a miniature rendering used only as a navigation aid. OCRmyPDF alters
page appearance (deskew, clean, rasterize, re-render) and plugins may
edit pages arbitrarily, so a retained thumbnail would be stale and no
longer match its page. Modern viewers generate thumbnails on demand, so
there is no loss of functionality.
OcrGrafter.finalize() now strips /Thumb from every page before saving,
alongside the existing search-index discard, covering both the OCR and
hOCR pipelines. Orphaned thumbnail XObjects are garbage-collected on save.
Adobe Acrobat can embed a proprietary full-text search index in the
document catalog at /Root/PieceInfo/SearchIndex. Only Acrobat reads it;
other viewers ignore it and search the text on the fly. Any change to a
PDF invalidates the index, so once OCRmyPDF rewrites the document the
retained index is stale and returns incorrect search results in Acrobat.
OcrGrafter.finalize() now discards it before saving (covering both the
OCR and hOCR pipelines), preserving any other PieceInfo owner data and
dropping an empty PieceInfo. Modern viewers rebuild a search index on
demand, so there is no loss of search capability.
test_gray_mask_ocrs_to_text loaded DejaVu, which only exists on the Linux
CI runners; the OSError fallback hardcoded a Linux-only path, so the test
failed with "cannot open resource" on macOS and Windows. Load the bundled
ocrmypdf.data/NotoSans-Regular.ttf via importlib.resources instead, which
is guaranteed present on every platform.
Promote rasterization device based on image-mask fill color so gray/colored
stencil text is not destroyed by 1-bit dithering before OCR; default the
1-bit Ghostscript device to pngmonod. Fixes#1688.
Address final review findings: a mask painted inside a Form XObject now
inherits the fill color in effect at the form's Do operator (previously it
reset to black, missing gray/color promotion one indirection deep). The cs
operator now resets the fill color to black per PDF spec, so a stale color
set before cs cannot leak to a subsequently drawn mask.
Track the current PDF fill color on the graphics stack alongside the CTM and
record an Ink classification (mono/gray/color) per image-draw event. Image
masks are painted with the current fill color, so this enables later device
promotion. Color operators are tolerant of malformed operands to preserve
robustness on untrusted input.
Closes#1615. The token 'end' (case-insensitive) is now accepted as an
alias for the document's last page, e.g. --pages 3-end. Resolution is
deferred until the page count is known from the input PDF.
PDFium does not support negative crop values to expand the render
area beyond the CropBox: such values only pad the output canvas with
white, leaving content outside the CropBox clipped. Set the in-memory
CropBox to the MediaBox before rendering instead. Reported in #1685.
Expose Ghostscript's -dJPEGQ and image downsampling switches as
advanced, plugin-scoped options for tuning PDF/A output, without
polluting the central OcrOptions registry. The optimizer's existing
--jpeg-quality remains the recommended JPEG quality control.
- GhostscriptOptions gains jpeg_quality and jpeg_maxdpi fields and CLI
args (advanced help text). jpeg_quality=0 is honored as Ghostscript's
maximum compression rather than being silently coerced to the default.
- _exec.ghostscript.generate_pdfa() forwards both values; when
jpeg_maxdpi is set, downsample threshold is pinned at 1.0.
- _get_plugin_options falls back to extra_attrs for namespaced fields
so plugins can own their options without registering them centrally.
- Documentation explains the rationale: Ghostscript is the legacy path
(pypdfium + verapdf is preferred in v17+), the optimizer is the
supported file-size lever, and lowering quality is almost always a
better trade than downsampling.
When Tesseract cannot find its 'hocr' or 'txt' config files in the
tessdata configs/ directory, it prints "read_params_file: Can't open"
warnings, exits 0, and produces no output. OCRmyPDF then crashed with a
confusing FileNotFoundError on the missing hOCR file (issue #1687).
Promote read_params_file warnings to TesseractConfigError with guidance
on the likely cause, and verify the expected output file exists after
Tesseract claims success as defense-in-depth for other silent-failure
modes.
The 2024 guard against runaway recursion in _find_image_xrefs_container
only deduplicated image xrefs, but Form XObject xrefs are never added to
include_xrefs/exclude_xrefs, so a self-referential or DAG-shaped Form
graph re-entered every branch until the depth limit fired -- producing
the reported flood of warnings (and minutes-long hangs) on PowerPoint
exports.
Thread a visited_forms set through the recursion so each Form XObject is
descended into at most once per document. With memoization in place the
depth limit is no longer a cycle defense, so demote its log to debug.
Add a regression test that synthesises a circular-Form PDF from the
existing formxobject.pdf fixture (no new binary fixture, no license
issues) and asserts zero "Recursion depth exceeded" warnings.
Split ocrmypdf.subprocess/__init__.py into three private submodules by
concern (_run, _version, _check) and reduce __init__ to re-exports.
Introduce ocrmypdf._exec._probe.ToolProbe to centralize the version()/
available() pattern each tool module was reimplementing, so the "is this
tool installed and suitable?" question is cleanly distinct from the
pure, picklable functions that do the work.
Also replace the ghostscript module-import log.addFilter() side effect
with an idempotent _ensure_log_filter_installed() called at the top of
each work function, so the DuplicateFilter is present in subprocess
workers without relying on import-time ordering.
Public API of ocrmypdf.subprocess is unchanged.
fpdf2 >= 2.8.7 emits a custom begincidchar Encoding CMap for CFF-based
CID fonts (e.g. NotoSansCJK). pdfminer.six returns <CMap: None> for such
CMaps, so text extraction yields empty output. Switch to pdftotext (poppler)
which handles the new encoding correctly.
The API previously clobbered PIL.Image.MAX_IMAGE_PIXELS unconditionally
on every call, so host applications (e.g. Paperless-NGX) that configured
the PIL limit before invoking ocrmypdf.ocr() saw their setting silently
overwritten with the 250 MP default. Make max_image_mpixels default to
None and only apply the override when the caller explicitly sets it.
The CLI default of 250 MP is unchanged.
Fixes#1665
The JPEG truncation bug (1-15 bytes) persists in Ghostscript 10.7.0.
Update the warning message to show the actual GS version instead of
hardcoding "10.6.x", and remove the stale date reference. Also make
the test warning filter match the new message format.
fpdf2's shape_text() produces RTL ligature glyphs (e.g. lam-alef) with
multi-character CMap entries whose character order gets reversed by the
bidi algorithm during text extraction, producing garbled output like
"سالح" instead of "سلاح".
For invisible text (the production OCR overlay path), bypass text shaping
and use encode_text() with pre-reversed strings. encode_text() maps
characters 1:1 in logical order, avoiding the ligature CMap issue. The
pre-reversal compensates for bidi reversal by text extractors. Since the
text is invisible (Tr=3), the lack of joining forms is harmless.
Add RTL text extraction tests that verify glyph stream order, ToUnicode
CMap 1:1 mappings, and correct logical order for Arabic (including
lam-alef ligature) and Hebrew scripts.
The inter-word Tz calculation stretched "word " to span from the current
word to the next, producing extreme horizontal scaling (300-500%) for
words far apart (e.g. in tables). Use per-word Tz instead — Td
positioning already handles inter-word gaps correctly.
Fixes#1635
Catch OSError (parent of both FileNotFoundError and
NotADirectoryError) in verapdf.available() so environments where
executing `verapdf` raises NotADirectoryError gracefully fall back
instead of crashing the pipeline. Fixes#1638.
Fixes#1642. Adds an early check in check_requested_output_file() that
raises OutputFileAccessError (exit code 5) if the destination file
already exists and --no-overwrite is set. The option is wired through
CLI, OcrOptions, and the Python API.
The jpg_quality and png_quality options default to None in the pydantic
model, but the fallback check only handled == 0. This caused a TypeError
when calling ocrmypdf.ocr() with optimize >= 2 without explicitly
setting quality values. Fixes#1641.
Replace hatch-vcs dynamic versioning with static version in _version.py
and pyproject.toml. Split CI into build.yml (test + stage draft release
on main) and release.yml (publish from draft on tag push). Docker images
are built on main pushes and re-tagged with the release version on tag
push without rebuilding.
James R. Barlow