Commit Graph
4247 Commits
Author SHA1 Message Date
James R. Barlow 9dcd882c83 Use uv to install docs with dependency groups 2026-01-31 00:05:27 -08:00
James R. Barlow 9d8aa5a0c3 v17.1.0 release notes v17.1.0 2026-01-30 16:15:50 -08:00
James R. Barlow e036a902ae Add --tagged-pdf-mode option to control Tagged PDF handling
Allow users to bypass the TaggedPDFError when processing Tagged PDFs
by setting --tagged-pdf-mode=ignore. This is useful when users know
they want to OCR a Tagged PDF despite the warning.

- 'default': Error if --mode is default, otherwise warn (current behavior)
- 'ignore': Always warn but continue processing (never error)
2026-01-30 16:15:43 -08:00
James R. Barlow 0a980fb11b Add Encoding.flate_jpeg to recognize deflated JPEG images
FlateDecode+DCTDecode compressed images are essentially deflated JPEGs,
typically created by OCRmyPDF's optimizer. This change ensures pdfinfo
correctly identifies them and should_visible_page_image_use_jpg treats
them as JPEG-origin images, allowing JPEG output when appropriate.
2026-01-30 12:53:59 -08:00
James R. Barlow 3abe8f71c7 v17.0.1 release notes v17.0.1 2026-01-30 00:15:13 -08:00
James R. Barlow 64f45b7fdb Fix pypdfium type checking 2026-01-30 00:14:02 -08:00
James R. Barlow 7e939ad44d Fix pypdfium rasterizer to respect raster_device colorspace
pypdfium was not converting images to the correct colorspace/mode based
on the raster_device parameter. For example, pngmono should produce a
1-bit image, but pypdfium was outputting full-color or grayscale images.

This caused images to be incorrectly promoted to PNG instead of being
preserved as CCITT/JBIG2 when using --force-ocr with pypdfium, because
the optimizer relies on the image being in the correct mode.

Changes:
- Render in grayscale for pngmono device (better input for 1-bit conversion)
- Add mode conversion to match Ghostscript's native device output:
  - pngmono: convert to mode '1' (1-bit)
  - pnggray/jpeggray: convert to mode 'L' (8-bit grayscale)
  - png256: convert to mode 'P' (8-bit indexed)
  - png16m/jpeg: convert to mode 'RGB'
2026-01-30 00:04:02 -08:00
James R. Barlow 297fb786a0 Update uv.lock (for protobuf) v17.0.0 2026-01-29 18:33:00 -08:00
James R. Barlow ad30dd94f7 Merge branch 'release/v17' 2026-01-29 18:31:11 -08:00
James R. Barlow e77f79ac6f Merge branch 'main' of github.com:ocrmypdf/OCRmyPDF 2026-01-29 18:30:54 -08:00
James R. Barlow c84fc56e45 Update CLI completions to match current options
Add new options: --mode, --ocr-engine, --rasterizer,
--continue-on-soft-render-error, --tesseract-non-ocr-timeout,
--tesseract-downsample-large-images, --tesseract-downsample-above,
--unpaper-args (fish), --plugin (fish).

Update --output-type to include 'auto' as default.
Update --pdf-renderer to include 'fpdf2' and mark hocr as deprecated.

Remove non-working options: --remove-background, --threshold.
2026-01-29 12:41:56 -08:00
James R. Barlow 0a0756b33e Tidy long lines and unnested with blocks 2026-01-27 15:28:27 -08:00
James R. Barlow c5d3ef4b17 Tighten ruff rules and modernize style 2026-01-27 14:04:52 -08:00
James R. Barlow 6b37583674 Refactor: move ocr_element to a better location 2026-01-27 14:01:30 -08:00
James R. Barlow de5f2b80f0 Further patching-out of fonts v17.0.0b1 2026-01-21 11:43:54 -08:00
James R. Barlow d951b4f0f7 Improve font fallback checking 2026-01-21 10:38:07 -08:00
James R. Barlow b386d39b3b tests: fix test_page_boxes when verapdf unavailable
The test expected MediaBox preservation for pdfa output, but this only
works when verapdf is available for speculative PDF/A conversion.
Without verapdf (Linux/Windows CI), Ghostscript normalizes the MediaBox.

Also convert pikepdf.Array to list in assertions for clearer error
messages, avoiding pytest repr issues with pikepdf objects.
2026-01-21 00:22:26 -08:00
James R. Barlow ec595a395b tests: little fixes 2026-01-20 23:23:43 -08:00
James R. Barlow bd29269c00 Various test fixes, mainly Windows issues 2026-01-20 22:28:06 -08:00
James R. Barlow 6fb7c5d95f Additional build fixes 2026-01-20 21:49:40 -08:00
James R. Barlow d57552c4f8 test: For Windows, ensure outputs are UTF-8 2026-01-20 21:37:13 -08:00
James R. Barlow f017c982cf watcher: use modern API 2026-01-20 21:25:12 -08:00
James R. Barlow 7ac51ac1a7 Fix type alias for Queue causing runtime TypeError
multiprocessing.Queue is a factory function, not a type, so it cannot
be used with the runtime | union operator. Move Queue, UserInit, and
WorkerInit type aliases into TYPE_CHECKING block to avoid evaluation
at runtime.
2026-01-20 20:38:33 -08:00
James R. Barlow db9f94de14 Ensure Noto font is installed where needed 2026-01-20 19:50:47 -08:00
James R. Barlow 37e7131a01 Drop support for Python 3.10, require Python 3.11+
Python 3.11 is now the minimum supported version. This aligns with
the codebase's use of StrEnum (introduced in 3.11) and removes
compatibility shims that were only needed for older versions.
2026-01-20 11:54:55 -08:00
James R. Barlow bc745d4d81 Replace magic Ghostscript raster device strings with StrEnum 2026-01-20 10:44:25 -08:00
James R. Barlow c818ad5e75 Drop deprecated NeverRaise exception 2026-01-20 10:43:21 -08:00
James R. Barlow 4b16228a4a docs: minor adjustments 2026-01-20 10:41:55 -08:00
James R. Barlow d40fca2590 Add verapdf to build for macOS 2026-01-20 10:41:43 -08:00
James R. Barlow 99f8106936 Update API documentation for OcrOptions-first calling convention
Document the new v17 API style where OcrOptions can be passed directly
to ocr(). Mark the positional argument style as legacy API for <v17
compatibility. Update examples to use modern syntax.
2026-01-20 10:30:33 -08:00
James R. Barlow ef88ba3f95 Add OcrOptions as first-class argument to ocr() function
Allow passing an OcrOptions object directly to ocr() as the first
positional argument, providing a cleaner API for programmatic use.
The old-style API with individual parameters remains fully supported.
2026-01-20 10:20:52 -08:00
James R. Barlow 2f4280b66c Comprrehensive documentation update in preparation for v17 2026-01-16 01:38:47 -08:00
James R. Barlow 6cf9d1c6ee Update release notes 2026-01-15 23:29:29 -08:00
James R. Barlow 6a7164a76c Update release notes with branch changes 2026-01-15 23:25:51 -08:00
James R. Barlow 3f328785f0 Fix pypdfium rasterizer to match Ghostscript dimensions
The pypdfium rasterizer was producing output images that differed by 1
pixel compared to Ghostscript due to floating-point precision issues in
dimension calculations.

Root cause:
- pypdfium used harmonic mean of x/y DPI to calculate a single scale
  factor, losing the distinction between x and y DPI
- No DPI rounding like Ghostscript's 6-decimal precision
- Compound rounding errors when converting points to pixels

Solution:
1. Round DPI to 6 decimals to match Ghostscript's precision
2. Calculate expected output dimensions using separate x/y DPI values
3. Handle dimension swapping for 90°/270° rotations
4. Resize output image if off by 1-2 pixels (graceful correction)

This ensures pixel-perfect matching with Ghostscript while being
minimally invasive and only resizing when necessary.

Changes:
- Modified _render_page_to_bitmap() to calculate expected dimensions
- Modified _process_image_for_output() to correct small discrepancies
- Updated rasterize_pdf_page() to pass dimensions through pipeline
- Parametrized rotation tests to run with both rasterizers

All 45 rotation tests now pass with both pypdfium and ghostscript.

Fixes test_rotated_skew_timeout with pypdfium rasterizer.
2026-01-14 14:37:24 -08:00
James R. Barlow 5acf21651f ruff lint and format 2026-01-13 01:50:57 -08:00
James R. Barlow 7bfe3ecd5b Fix double-compression of already-deflated JPEGs
Images with [FlateDecode, DCTDecode] filter chain were incorrectly
being marked for additional FlateDecode compression, resulting in
double-compressed data and invalid output PDFs.

Add _already_flate_encoded() helper to check if an image already has
FlateDecode in its filter chain, and skip such images in
_find_deflatable_jpeg().
2026-01-13 01:41:59 -08:00
James R. Barlow 5371cc5e39 Update test to match new error messag 2026-01-13 01:33:10 -08:00
James R. Barlow 4c7086c609 Replace typer with cyclopts CLI library in misc scripts
Migrate watcher.py and pdf_text_diff.py from typer to cyclopts for
CLI argument parsing. Update pyproject.toml to reflect the dependency
change in the watcher optional feature.
2026-01-13 00:43:14 -08:00
James R. Barlow bf76c8270c Rationalize optional dependencies vs dependency groups
Establish clear separation between user-facing optional dependencies
and developer-only dependency groups:

**Optional Dependencies (user features):**
- watcher: File watching service for batch processing
- webservice: Streamlit-based web UI
- Installable via: uv sync --extra <name> or pip install ocrmypdf[name]

**Dependency Groups (developer tools):**
- test: Testing infrastructure (merged from test + extended_test)
- docs: Documentation building tools
- streamlit-dev: Enhanced Streamlit development tools
- dev: General development tools (mypy, ipykernel)
- Installable via: uv sync --group <name> (uv only, NOT pip)

Breaking changes for developers:
- pip install -e .[test] no longer works → use uv sync --group test
- pip install -e .[docs] no longer works → use uv sync --group docs
- pip install -e .[extended_test] removed → merged into test group

No breaking changes for end users:
- pip install ocrmypdf[watcher] still works
- pip install ocrmypdf[webservice] still works

Updated:
- CI/CD workflows to use uv sync --group test
- Docker images to exclude test dependencies
- Documentation to recommend uv with pip as fallback
- pyproject.toml with clear comments explaining both systems
2026-01-13 00:34:55 -08:00
James R. Barlow 740f67091c Rename OCROptions to OcrOptions for consistency
Technically OCROptions is more Pythonic but we have several pre-existing classes named OcrWhatever. Go with the local flow.
2026-01-12 23:37:54 -08:00
James R. Barlow 36dea181e6 Update cookbook: Replace --tesseract-timeout 0 with --ocr-engine none
Update documentation examples to use the new --ocr-engine none option
instead of the deprecated --tesseract-timeout 0 idiom for disabling OCR.
2026-01-12 23:28:14 -08:00
James R. Barlow c69f293322 Add --mode/-m CLI argument with ProcessingMode enum
Introduce a new --mode (-m) argument that consolidates the three
mutually exclusive OCR processing options into a single enum:
- default: Error if text is found (standard behavior)
- force: Rasterize all content and run OCR (replaces --force-ocr)
- skip: Skip pages with existing text (replaces --skip-text)
- redo: Re-OCR pages, stripping old text layer (replaces --redo-ocr)

The legacy flags --force-ocr, --skip-text, and --redo-ocr remain as
silent aliases for backward compatibility. Both CLI and API usage
continue to work unchanged.
2026-01-12 15:23:08 -08:00
James R. Barlow e9fe061c30 Format fix 2026-01-12 10:25:24 -08:00
James R. Barlow c9ea07e954 Reduce chattiness of fonttools 2026-01-12 10:16:58 -08:00
James R. Barlow 0c3745a1a4 Add OCR engine selection framework and null OCR engine
Introduce --ocr-engine option to select between OCR engines:
- 'auto' (default): Uses Tesseract
- 'tesseract': Explicit Tesseract selection
- 'none': Skip OCR entirely (for PDF processing only)

Key changes:
- Extend OcrEngine ABC with generate_ocr() and supports_generate_ocr()
  for direct OcrElement tree output (bypasses hOCR)
- Add get_ocr_engine(options) hook parameter for engine selection
- Implement NullOcrEngine for --ocr-engine none
- Export OcrElement, OcrClass, BoundingBox from ocrmypdf package
- Add ocr_tree support to grafting pipeline

This prepares the foundation for pluggable OCR engines while maintaining
full backward compatibility with existing Tesseract-based workflows.
2026-01-12 10:11:14 -08:00
James R. Barlow 664c3e2a8e Update test cache for slow rotation tests 2026-01-10 16:30:25 -08:00
James R. Barlow 315d0df0e9 Fix incorrect rotation direction in pypdfium rasterizer
pypdfium2 expects clockwise rotation values, but OCRmyPDF tracks
rotation in counter-clockwise. Negate the rotation value to fix.

Also refactor nested try/finally blocks to use contextlib.closing()
for cleaner resource management.
2026-01-10 16:29:49 -08:00
James R. Barlow 3c94ada857 Fix tesseract_cache plugin to properly handle cache misses
- Check all required output files exist before declaring cache hit,
  not just stderr.bin
- Add 'hocr' to list of cached output file types
- Fix timeout=0.0 causing immediate timeout on cache miss by treating
  it as "no timeout"
2026-01-09 02:10:29 -08:00
James R. Barlow fcbdbac602 Update test_page_boxes MediaBox expectations for speculative PDF/A
When speculative PDF/A succeeds (verapdf available), Ghostscript is
bypassed and MediaBox is preserved rather than normalized to origin.
2026-01-09 01:25:31 -08:00