Commit Graph
55 Commits
Author SHA1 Message Date
James R. Barlow 0d4c3bcdcf feat: add --mode strip to remove the OCR text layer without rasterizing
Adds a processing mode that removes the invisible (render mode 3) OCR text
layer in place. Unlike `--ocr-engine none --force-ocr`, it does not
rasterize the page, so images and visible content are preserved unchanged
and the output is smaller rather than larger. Options that require
rasterization or OCR (--deskew, --clean, --sidecar, etc.) are rejected.

Only invisible text is removed; text drawn as visible glyphs under an
opaque image (some OCR engines, and OCRmyPDF v2.2 and earlier) cannot be
removed this way, as documented.

Closes #1435.
2026-06-11 12:34:03 -07:00
James R. Barlow 9748208e68 Support 'end' alias for last page in --pages
Closes #1615. The token 'end' (case-insensitive) is now accepted as an
alias for the document's last page, e.g. --pages 3-end. Resolution is
deferred until the page count is known from the input PDF.
2026-05-26 12:18:09 -07:00
James R. Barlow efb83ad64f Add --ghostscript-jpeg-quality and --ghostscript-jpeg-maxdpi
Expose Ghostscript's -dJPEGQ and image downsampling switches as
advanced, plugin-scoped options for tuning PDF/A output, without
polluting the central OcrOptions registry. The optimizer's existing
--jpeg-quality remains the recommended JPEG quality control.

- GhostscriptOptions gains jpeg_quality and jpeg_maxdpi fields and CLI
  args (advanced help text). jpeg_quality=0 is honored as Ghostscript's
  maximum compression rather than being silently coerced to the default.
- _exec.ghostscript.generate_pdfa() forwards both values; when
  jpeg_maxdpi is set, downsample threshold is pinned at 1.0.
- _get_plugin_options falls back to extra_attrs for namespaced fields
  so plugins can own their options without registering them centrally.
- Documentation explains the rationale: Ghostscript is the legacy path
  (pypdfium + verapdf is preferred in v17+), the optimizer is the
  supported file-size lever, and lowering quality is almost always a
  better trade than downsampling.
2026-05-25 10:20:54 -07:00
James R. Barlow 1c89cacfef Respect host-set PIL.Image.MAX_IMAGE_PIXELS in Python API
The API previously clobbered PIL.Image.MAX_IMAGE_PIXELS unconditionally
on every call, so host applications (e.g. Paperless-NGX) that configured
the PIL limit before invoking ocrmypdf.ocr() saw their setting silently
overwritten with the 250 MP default. Make max_image_mpixels default to
None and only apply the override when the caller explicitly sets it.
The CLI default of 250 MP is unchanged.

Fixes #1665
2026-04-19 13:44:57 -07:00
James R. Barlow 5b9d6f979e Add --no-overwrite / -n option to prevent overwriting output files
Fixes #1642. Adds an early check in check_requested_output_file() that
raises OutputFileAccessError (exit code 5) if the destination file
already exists and --no-overwrite is set. The option is wired through
CLI, OcrOptions, and the Python API.
2026-03-10 01:58:57 -07:00
James R. Barlow 5890d1855e Fix Python API producing empty OCR due to tesseract_timeout defaulting to 0
OcrOptions.tesseract_timeout defaulted to 0.0, which caused
subprocess.run(timeout=0) to immediately raise TimeoutExpired before
Tesseract could produce any output. The CLI was unaffected because
argparse defaults --tesseract-timeout to 180. Change the OcrOptions
default to None so the plugin's own default (180s) is used.

Fixes #1636
2026-02-17 21:55:49 -08:00
James R. Barlow 92beb474a5 Normalize unpaper_args to list at construction time
Use a Pydantic field validator to convert string input to list[str]
during OcrOptions construction, simplifying the type from
`str | list[str] | None` to `list[str] | None`. Security validation
(path injection check) now happens at construction rather than in
check_options_preprocessing().
2026-01-31 12:05:37 -08:00
James R. Barlow e036a902ae Add --tagged-pdf-mode option to control Tagged PDF handling
Allow users to bypass the TaggedPDFError when processing Tagged PDFs
by setting --tagged-pdf-mode=ignore. This is useful when users know
they want to OCR a Tagged PDF despite the warning.

- 'default': Error if --mode is default, otherwise warn (current behavior)
- 'ignore': Always warn but continue processing (never error)
2026-01-30 16:15:43 -08:00
James R. Barlow 5acf21651f ruff lint and format 2026-01-13 01:50:57 -08:00
James R. Barlow 740f67091c Rename OCROptions to OcrOptions for consistency
Technically OCROptions is more Pythonic but we have several pre-existing classes named OcrWhatever. Go with the local flow.
2026-01-12 23:37:54 -08:00
James R. Barlow c69f293322 Add --mode/-m CLI argument with ProcessingMode enum
Introduce a new --mode (-m) argument that consolidates the three
mutually exclusive OCR processing options into a single enum:
- default: Error if text is found (standard behavior)
- force: Rasterize all content and run OCR (replaces --force-ocr)
- skip: Skip pages with existing text (replaces --skip-text)
- redo: Re-OCR pages, stripping old text layer (replaces --redo-ocr)

The legacy flags --force-ocr, --skip-text, and --redo-ocr remain as
silent aliases for backward compatibility. Both CLI and API usage
continue to work unchanged.
2026-01-12 15:23:08 -08:00
James R. Barlow 0c3745a1a4 Add OCR engine selection framework and null OCR engine
Introduce --ocr-engine option to select between OCR engines:
- 'auto' (default): Uses Tesseract
- 'tesseract': Explicit Tesseract selection
- 'none': Skip OCR entirely (for PDF processing only)

Key changes:
- Extend OcrEngine ABC with generate_ocr() and supports_generate_ocr()
  for direct OcrElement tree output (bypasses hOCR)
- Add get_ocr_engine(options) hook parameter for engine selection
- Implement NullOcrEngine for --ocr-engine none
- Export OcrElement, OcrClass, BoundingBox from ocrmypdf package
- Add ocr_tree support to grafting pipeline

This prepares the foundation for pluggable OCR engines while maintaining
full backward compatibility with existing Tesseract-based workflows.
2026-01-12 10:11:14 -08:00
James R. Barlow 0c4ee5af4e Add 'auto' output type for best-effort PDF/A without Ghostscript
- Add new '--output-type auto' option (now the default) that produces
  best-effort PDF/A without requiring Ghostscript
- When verapdf is available, use speculative PDF/A conversion
- Without verapdf, pass through as PDF/A if safe (input claims PDF/A
  or --force-ocr was used), otherwise output as regular PDF
- Make Ghostscript check conditional - only required for pdfa* output types
- Update soft error tests to explicitly use --output-type pdfa since they
  exercise Ghostscript failure modes
- Fix Tesseract OSD error handling to check both stdout and stderr for
  known non-fatal messages like "Too few characters"
2026-01-09 00:56:00 -08:00
James R. Barlow 7a4b98974c Integrate fpdf2 renderer and remove legacy hOCR renderer
- Update pipeline to use fpdf2 renderer as default
- Remove legacy hocrtransform PDF renderer (_font.py, _hocr.py,
  pdf_renderer.py)
- Update CLI and options for fpdf2 renderer
- Add fpdf2 dependency to pyproject.toml
- Update graft module for fpdf2 multi-page rendering
2026-01-06 13:45:44 -08:00
James R. Barlow e9bfce34f1 Fix ruff linting issues
- Use X | Y syntax in isinstance calls (UP038)
- Remove trailing whitespace from blank lines (W293)
2025-12-23 03:07:48 -08:00
James R. Barlow 16c2604a07 Remove lossy JBIG2 support, retain lossless JBIG2 only
Lossy JBIG2 has been removed due to well-documented risks of character
substitution errors (e.g., 6/8 confusion). The --jbig2-lossy and
--jbig2-page-group-size arguments are now deprecated and ignored with
a warning.

Changes:
- Remove jbig2_lossy and jbig2_page_group_size from OCROptions
- Simplify optimize.py to use single-image JBIG2 encoding only
  (no symbol dictionaries/JBIG2Globals)
- Remove convert_group() from jbig2enc.py
- Deprecate CLI args with warnings for backward compatibility
- Update documentation to explain lossless-only JBIG2
2025-12-23 02:45:07 -08:00
James R. Barlow 9ebba91466 Use plugin namespace access pattern throughout codebase
Migrate all code from flat accessor pattern (options.tesseract_timeout)
to the plugin namespace pattern (options.tesseract.timeout).

Key changes:
- Fix _get_plugin_options to raise AttributeError for unregistered
  namespaces instead of silently returning None
- Add _convert_value helper to convert PathLike to str for plugin
  model field compatibility
- Filter out _plugin_cache_* entries from JSON serialization to fix
  worker process serialization (test_simulate_oom_killer)
- Update tesseract_ocr.py, ghostscript.py, _validation_coordinator.py,
  and _pipelines/ocr.py to use options.tesseract.* and
  options.ghostscript.* accessors
- Update tests to use setup_plugin_infrastructure() for plugin
  model registration
2025-12-23 02:02:21 -08:00
James R. Barlow aec995aced Require plugin model registration for namespace access in OCROptions
- Update __getattr__ docstring to clarify that plugin models must be
  registered for namespace access (e.g., options.tesseract.timeout)
- Update test_json_serialization.py to properly register TesseractOptions
  before accessing plugin namespaces
- Worker processes now register plugin models for multiprocessing tests
- Exclude plugin cache keys from extra_attrs comparison in tests
2025-12-22 15:09:55 -08:00
James R. BarlowandClaude Opus 4.5 ed813cec67 feat: add --rasterizer CLI option to select PDF rasterization backend
Add user control over which rasterizer is used for PDF page rendering:
- 'auto' (default): prefers pypdfium when available, falls back to Ghostscript
- 'pypdfium': force pypdfium2 (errors if not installed)
- 'ghostscript': force traditional Ghostscript rasterizer

Changes:
- Add rasterizer field with validation to OCROptions model
- Add --rasterizer CLI argument in the Advanced options group
- Update rasterize_pdf_page hookspec to pass options to plugins
- Update pypdfium plugin with check_options hook for availability check
- Update both plugins to respect the rasterizer option

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:29:17 -08:00
James R. BarlowandClaude Opus 4.5 a4ee513cd4 refactor: clean up deprecated code and update plugin docs
- Remove outdated Phase comments from _options.py and cli.py
- Remove unused methods from PluginOptionRegistry:
  - get_extended_options_model() - replaced by __getattr__ in OCROptions
  - map_legacy_options() - unused
  - validate_plugin_options() - unused
- Update plugin documentation to document register_options hook
- Add documentation for nested plugin option access pattern

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:21:48 -08:00
James R. BarlowandClaude Opus 4.5 0ad7f5fc13 feat: add dynamic nested access to plugin options
Completes Phase 5 of the CLI refactoring plan by enabling nested
plugin option access (e.g., options.tesseract.timeout) alongside
the legacy flat access (options.tesseract_timeout).

Changes:
- Add module-level plugin option model registry in _options.py
- Add __getattr__ to OCROptions for dynamic namespace access
- Register plugin models in setup_plugin_infrastructure()
- Add test for nested plugin option access

Plugin option instances are lazily created from flat field values
and cached for subsequent access.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 95d9c3ed18 fix: add cross-cutting validation to OCROptions model
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider f91e41a209 fix: refine validation coordinator error handling
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 01ea6c2b8b feat: add ValidationCoordinator for cross-cutting validation
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider f02d733d31 feat: add legacy field mapping to plugin registry
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 28d6ea0f10 feat: Add CLI generation methods to plugin option models
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 40f01d85ae refactor: convert jbig2 properties to direct fields in OCROptions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider b7640bdb9c feat: implement plugin option models with backward compatibility
This commit introduces Pydantic models for plugin-specific options in OCRmyPDF, focusing on:
- Creating TesseractOptions, OptimizeOptions, and GhostscriptOptions
- Adding backward compatibility properties
- Maintaining existing CLI and API functionality
- Preparing for future plugin option registration system

The changes include:
- Added type-annotated option models with validation
- Updated OCROptions to include legacy fields
- Added backward compatibility properties for jbig2 options
- Prepared groundwork for dynamic plugin option handling

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlow b7737446e4 cli: push up imports 2025-12-21 12:21:48 -08:00
James R. Barlowandaider 4f9c4c3e52 refactor: reorganize CLI and options initialization
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlow 08ee5690bc Remove to_namespace and its user, since nothing triggers it 2025-12-21 12:21:48 -08:00
James R. Barlow 3f38ea4d80 Remove OCROptions getattr interface 2025-12-21 12:21:47 -08:00
James R. Barlowandaider f04b5504e8 test: fix hOCR pipeline output folder handling
The commit message captures the essence of the changes: we fixed how the output folder is handled in the hOCR pipeline by making it a proper field in OCROptions and updating the API functions accordingly.

Would you like me to generate a full commit message or is this sufficient?

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlow 69185e5819 refactor: remove custom attribute handling methods and add output_folder option 2025-12-21 12:21:47 -08:00
James R. Barlow e4f8ba8edc Fix computing of lossless_reconstruction and checking of redo_ocr conflicts 2025-12-21 12:21:47 -08:00
James R. Barlow 1f493ba789 refactor: post-AI code cleanup 2025-12-21 12:21:47 -08:00
James R. Barlowandaider e1d976168c feat: handle Pydantic serialization iterators in JSON serialization
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 60182ac8a8 fix: update JSON serialization tests to match default values
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlow 53db7b384b refactor: set default values for optional OCR configuration parameters 2025-12-21 12:21:47 -08:00
James R. Barlowandaider 9535b52d06 fix: revert default option values to preserve original behavior
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>

fix: set default optimize value to 1 to prevent NoneType comparison

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 21d69ffe87 fix: set default value for fast_web_view to prevent NoneType errors
This change ensures that `fast_web_view` always has a default value of 1.0, preventing the TypeError that was occurring in multiple test cases when trying to multiply `None` with an integer.

The modifications include:
1. Changing the type hint for `fast_web_view` from `float | None` to `float`
2. Setting a default value of 1.0
3. Adding a validation step in the `model_validator` to set the default value if not provided

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 3baeb83533 fix: Add compatibility for jpeg_quality and resolve strategy validation
This commit addresses several test failures by:
- Adding a compatibility property for `jpeg_quality`
- Changing default color conversion strategy from 'auto' to 'RGB'
- Ensuring `tesseract_config` is always a list
- Improving validation for color conversion and image compression strategies

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 134f4fcc28 fix: remove Union type hints and add default values in OCROptions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 3a0a7c546b feat: Implement JSON serialization for OCROptions with safe handling of Path and stream objects
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 7575dddefc fix: add lossless_reconstruction attribute to OCROptions for compatibility
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>

fix: resolve recursion error in lossless_reconstruction option

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>

fix: ensure lossless_reconstruction attribute is added to Namespace

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>

fix: ensure lossless_reconstruction attribute is correctly propagated to PageContext

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:42:49 -08:00
James R. Barlowandaider 66a3e8508e feat: add comprehensive validators to OCROptions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:42:23 -08:00
James R. Barlow 87478bc240 Fix options.jpg_quality issue 2025-12-13 11:42:23 -08:00
James R. Barlowandaider d18efcbbf1 fix: support flexible type inputs for pages and unpaper_args
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:41:27 -08:00
James R. Barlowandaider f9a4a2e240 fix: handle missing input_file in OCROptions namespace conversion
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:41:27 -08:00
James R. Barlowandaider 5f89100dc3 test: add lossless_reconstruction field to OCROptions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:41:27 -08:00