Commit Graph
228 Commits
Author SHA1 Message Date
James R. Barlow e036a902ae Add --tagged-pdf-mode option to control Tagged PDF handling
Allow users to bypass the TaggedPDFError when processing Tagged PDFs
by setting --tagged-pdf-mode=ignore. This is useful when users know
they want to OCR a Tagged PDF despite the warning.

- 'default': Error if --mode is default, otherwise warn (current behavior)
- 'ignore': Always warn but continue processing (never error)
2026-01-30 16:15:43 -08:00
James R. Barlow 0a980fb11b Add Encoding.flate_jpeg to recognize deflated JPEG images
FlateDecode+DCTDecode compressed images are essentially deflated JPEGs,
typically created by OCRmyPDF's optimizer. This change ensures pdfinfo
correctly identifies them and should_visible_page_image_use_jpg treats
them as JPEG-origin images, allowing JPEG output when appropriate.
2026-01-30 12:53:59 -08:00
James R. Barlow bc745d4d81 Replace magic Ghostscript raster device strings with StrEnum 2026-01-20 10:44:25 -08:00
James R. Barlow 5acf21651f ruff lint and format 2026-01-13 01:50:57 -08:00
James R. Barlow 740f67091c Rename OCROptions to OcrOptions for consistency
Technically OCROptions is more Pythonic but we have several pre-existing classes named OcrWhatever. Go with the local flow.
2026-01-12 23:37:54 -08:00
James R. Barlow c69f293322 Add --mode/-m CLI argument with ProcessingMode enum
Introduce a new --mode (-m) argument that consolidates the three
mutually exclusive OCR processing options into a single enum:
- default: Error if text is found (standard behavior)
- force: Rasterize all content and run OCR (replaces --force-ocr)
- skip: Skip pages with existing text (replaces --skip-text)
- redo: Re-OCR pages, stripping old text layer (replaces --redo-ocr)

The legacy flags --force-ocr, --skip-text, and --redo-ocr remain as
silent aliases for backward compatibility. Both CLI and API usage
continue to work unchanged.
2026-01-12 15:23:08 -08:00
James R. Barlow 0c3745a1a4 Add OCR engine selection framework and null OCR engine
Introduce --ocr-engine option to select between OCR engines:
- 'auto' (default): Uses Tesseract
- 'tesseract': Explicit Tesseract selection
- 'none': Skip OCR entirely (for PDF processing only)

Key changes:
- Extend OcrEngine ABC with generate_ocr() and supports_generate_ocr()
  for direct OcrElement tree output (bypasses hOCR)
- Add get_ocr_engine(options) hook parameter for engine selection
- Implement NullOcrEngine for --ocr-engine none
- Export OcrElement, OcrClass, BoundingBox from ocrmypdf package
- Add ocr_tree support to grafting pipeline

This prepares the foundation for pluggable OCR engines while maintaining
full backward compatibility with existing Tesseract-based workflows.
2026-01-12 10:11:14 -08:00
James R. Barlow 0c4ee5af4e Add 'auto' output type for best-effort PDF/A without Ghostscript
- Add new '--output-type auto' option (now the default) that produces
  best-effort PDF/A without requiring Ghostscript
- When verapdf is available, use speculative PDF/A conversion
- Without verapdf, pass through as PDF/A if safe (input claims PDF/A
  or --force-ocr was used), otherwise output as regular PDF
- Make Ghostscript check conditional - only required for pdfa* output types
- Update soft error tests to explicitly use --output-type pdfa since they
  exercise Ghostscript failure modes
- Fix Tesseract OSD error handling to check both stdout and stderr for
  known non-fatal messages like "Too few characters"
2026-01-09 00:56:00 -08:00
James R. Barlow 4cb488d0fc Skip speculative PDF/A when --pdfa-image-compression is set
When the user explicitly sets --pdfa-image-compression to something
other than 'auto', skip the speculative PDF/A conversion and use
Ghostscript instead. The speculative conversion (using pikepdf +
verapdf) doesn't apply image compression settings, so Ghostscript
is required to honor the user's compression preference.
2026-01-08 15:12:35 -08:00
James R. Barlow 900a60fd10 Add verapdf integration for speculative PDF/A conversion
Introduce a fast path for PDF/A conversion that uses pikepdf to add
PDF/A structures directly (sRGB ICC profile and XMP metadata), then
validates with verapdf. If validation passes, skip Ghostscript entirely.
If validation fails or verapdf is unavailable, fall back to the existing
Ghostscript conversion path.

New files:
- src/ocrmypdf/_exec/verapdf.py: CLI wrapper for verapdf validator
- tests/test_verapdf.py: Test suite for new functionality

Modified:
- pdfa.py: Add speculative_pdfa_conversion() and helpers
- _pipeline.py: Add try_speculative_pdfa() function
- _pipelines/_common.py: Integrate speculative path into postprocess()
2026-01-08 10:58:01 -08:00
James R. Barlow f5617ce44e Refactor OcrmypdfPluginManager to use composition over inheritance
Replace inheritance from pluggy.PluginManager with composition pattern,
providing a type-safe interface for all 16 hooks defined in pluginspec.py.
The underlying pluggy manager is now accessible via the .pluggy property
for advanced use cases like set_blocked().

This change enables IDE autocomplete and type checking for all hook calls
while maintaining full backward compatibility with the plugin system.
2026-01-07 17:23:13 -08:00
James R. Barlow 7a4b98974c Integrate fpdf2 renderer and remove legacy hOCR renderer
- Update pipeline to use fpdf2 renderer as default
- Remove legacy hocrtransform PDF renderer (_font.py, _hocr.py,
  pdf_renderer.py)
- Update CLI and options for fpdf2 renderer
- Add fpdf2 dependency to pyproject.toml
- Update graft module for fpdf2 multi-page rendering
2026-01-06 13:45:44 -08:00
James R. Barlow be425e7405 Refactor pdfinfo: split info.py into focused modules
Split the 1288-line info.py into smaller, single-responsibility modules:
- _types.py: Enums, type aliases, lookup dictionaries
- _contentstream.py: PDF content stream parsing, DPI calculation
- _image.py: ImageInfo class and image finding functions
- _worker.py: Concurrency/worker process handling
- info.py: PageInfo, PdfInfo classes (reduced to ~530 lines)

Public API unchanged - all existing imports continue to work.
2025-12-22 01:27:23 -08:00
James R. Barlow 57e2600566 Also process art and bleed boxes 2025-12-21 14:03:28 -08:00
James R. Barlow 41758766a1 Test and fix page box issues 2025-12-21 14:03:28 -08:00
James R. BarlowandClaude Opus 4.5 3e46b039ed feat: add use_cropbox parameter to align rasterizer APIs
Added use_cropbox parameter to rasterize_pdf_page hook to allow
choosing between MediaBox and CropBox rendering:

- Default is use_cropbox=False (MediaBox) for consistency with
  Ghostscript's existing behavior
- Ghostscript: passes -dUseCropBox when use_cropbox=True
- pypdfium: calculates crop values to expand from CropBox to MediaBox
  when use_cropbox=False

This aligns both rasterizers to produce the same output dimensions
by default, making the rasterizer choice transparent for page
geometry.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:29:17 -08:00
James R. BarlowandClaude Opus 4.5 ed813cec67 feat: add --rasterizer CLI option to select PDF rasterization backend
Add user control over which rasterizer is used for PDF page rendering:
- 'auto' (default): prefers pypdfium when available, falls back to Ghostscript
- 'pypdfium': force pypdfium2 (errors if not installed)
- 'ghostscript': force traditional Ghostscript rasterizer

Changes:
- Add rasterizer field with validation to OCROptions model
- Add --rasterizer CLI argument in the Advanced options group
- Update rasterize_pdf_page hookspec to pass options to plugins
- Update pypdfium plugin with check_options hook for availability check
- Update both plugins to respect the rasterizer option

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:29:17 -08:00
James R. Barlow 1f493ba789 refactor: post-AI code cleanup 2025-12-21 12:21:47 -08:00
James R. Barlowandaider afc85333ac feat: add OCROptions import and type hints to pipeline functions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 1ef9aaf659 fix: Correct PDF/A part extraction and handle hOCR API output file
This commit addresses two issues:
1. Properly extract the PDF/A part from output_type
2. Add a placeholder output_file for hOCR API tests when output_folder is used

The changes include:
- Modifying `from_namespace` to add a placeholder output_file
- Updating PDF/A part extraction logic to handle different output_type formats
- Ensuring correct PDF/A part is passed to Ghostscript

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:41:27 -08:00
James R. Barlow 206bab74bc Improve diagnostics for unidentified image 2025-01-01 17:13:57 -08:00
James R. Barlow ee5acbe94e Repair PDF before all processing
Some PDFs choke both pdfminer.six and Ghostscript but the issues can be fixed first.

Fixes #1403
2024-10-27 17:37:46 -07:00
James R. Barlow c283cf0a0d Fix incorrect return value 2024-10-27 12:32:40 -07:00
James R. Barlow 4fb1bb4de6 pipeline: fix typo in message 2024-06-30 00:08:31 -07:00
James R. Barlow d6342cb8c2 Add heif/heic input image support 2024-04-07 00:33:13 -07:00
James R. Barlow 065bddbc6c Reformat with ruff format 2024-04-07 00:25:32 -07:00
James R. Barlow 6a746a1cbb ruff linting/Python 3.10 cleanup 2024-02-14 12:41:51 -08:00
James R. Barlow 3b9d4b7f0a Attempt to deal with oddball mediaboxes 2024-02-11 15:34:54 -08:00
James R. Barlow 11f53fe9a9 First cut at propagating page boxes
This would fix the immediate issue, but does not address an offset mediabox.
2024-02-11 15:34:54 -08:00
James R. Barlow 7e608fd1df Use Courier as debug font 2024-01-06 14:24:25 -08:00
James R. Barlow a0b98abb94 Improve and externalize debug rendering settings 2023-12-24 00:10:09 -08:00
James R. Barlow 9adcad84da Overhaul debug rendering to support possible use of Helvetica for testing 2023-12-23 23:47:35 -08:00
James R. Barlow 5b2f2e6290 Merge branch 'feature/modernhocr' 2023-12-03 15:17:02 -08:00
James R. Barlow 2affa83efe Remove code that attempted to manage xattrs out of output file
Feature requested in issue #1179, but caused #1195. On further review,
there is no platform independent way to manage extended attributes
and it is not clear copying them through is necessarily the sensible
thing to do.

Closes #1179.
2023-11-29 23:25:51 -08:00
James R. Barlow d217856166 Make hocrdebug work, and try to handle CJK spacing better 2023-11-21 00:33:02 -08:00
James R. Barlow db2e5132e6 Remove some obsolete parameters 2023-11-20 00:10:55 -08:00
James R. Barlow 580252a1a0 Merge branch 'feature/gscan2pdf'
Reconcile release notes and copy_final() with new pipeline.
2023-10-30 00:01:28 -07:00
James R. Barlow ae123fd209 Try to retain/copy xattrs 2023-10-28 01:42:06 -07:00
James R. Barlow 56f9bc311d Improve verbosity of colorspace selection 2023-10-25 00:38:56 -07:00
James R. Barlow 7935914f55 Use empty .hocr file instead of dummy template for symmetry with sandwich 2023-10-24 00:54:31 -07:00
James R. Barlow 53c953a561 Fix use_threads logic for get_pdfinfo
Some debug code was level in place that forced pdfinfo to run with only
one worker when --use-threads was issued. That is how it ought to be,
since threaded pdfinfo workers just fight over the GIL and there is no
sense in parallelizing them.

Also, the user's --use-threads or --no-use-threads would be ignored in
the case of pdfinfo. By setting max_workers=1 we disabled worker processes.

This fixes how that decision is made (putting it in the relevant code, which
knows its constraints) and allows the user to influence the
thread/process decision again.
2023-10-24 00:54:30 -07:00
James R. Barlow 1a7738a925 Refactor -migrate metadata repair to new module 2023-10-24 00:54:30 -07:00
James R. Barlow cbb0868ae3 Add hocr to ocr pdf pipeline 2023-10-24 00:52:31 -07:00
James R. Barlow 8991d2cb33 Refactor main pipeline and start hocr pipeline 2023-10-24 00:52:31 -07:00
James R. Barlow 93fda0dd00 Detect and warn about Tagged PDFs 2023-10-12 01:03:09 -07:00
James R. Barlow 9346d1f970 Simplify deep nested with-block 2023-10-09 02:30:30 -07:00
James R. Barlow 3086cfc3d9 pipeline: documentation and tweak merge_sidecars 2023-10-04 00:05:19 -07:00
James R. Barlow 91a14660b3 Require Pillow >= 10.0.1 and drop shims for older versions 2023-10-04 00:04:28 -07:00
James R. Barlow 113a6b45bd ruff autofixes (mostly typing.* -> collections.abc.*) 2023-10-01 00:02:53 -07:00
James R. Barlow d5128c5cf5 Further improvements to image DPI calculation 2023-09-26 00:28:54 -07:00