Commit Graph
40 Commits
Author SHA1 Message Date
James R. Barlow 16c2604a07 Remove lossy JBIG2 support, retain lossless JBIG2 only
Lossy JBIG2 has been removed due to well-documented risks of character
substitution errors (e.g., 6/8 confusion). The --jbig2-lossy and
--jbig2-page-group-size arguments are now deprecated and ignored with
a warning.

Changes:
- Remove jbig2_lossy and jbig2_page_group_size from OCROptions
- Simplify optimize.py to use single-image JBIG2 encoding only
  (no symbol dictionaries/JBIG2Globals)
- Remove convert_group() from jbig2enc.py
- Deprecate CLI args with warnings for backward compatibility
- Update documentation to explain lossless-only JBIG2
2025-12-23 02:45:07 -08:00
James R. Barlow 9ebba91466 Use plugin namespace access pattern throughout codebase
Migrate all code from flat accessor pattern (options.tesseract_timeout)
to the plugin namespace pattern (options.tesseract.timeout).

Key changes:
- Fix _get_plugin_options to raise AttributeError for unregistered
  namespaces instead of silently returning None
- Add _convert_value helper to convert PathLike to str for plugin
  model field compatibility
- Filter out _plugin_cache_* entries from JSON serialization to fix
  worker process serialization (test_simulate_oom_killer)
- Update tesseract_ocr.py, ghostscript.py, _validation_coordinator.py,
  and _pipelines/ocr.py to use options.tesseract.* and
  options.ghostscript.* accessors
- Update tests to use setup_plugin_infrastructure() for plugin
  model registration
2025-12-23 02:02:21 -08:00
James R. Barlow aec995aced Require plugin model registration for namespace access in OCROptions
- Update __getattr__ docstring to clarify that plugin models must be
  registered for namespace access (e.g., options.tesseract.timeout)
- Update test_json_serialization.py to properly register TesseractOptions
  before accessing plugin namespaces
- Worker processes now register plugin models for multiprocessing tests
- Exclude plugin cache keys from extra_attrs comparison in tests
2025-12-22 15:09:55 -08:00
James R. BarlowandClaude Opus 4.5 ed813cec67 feat: add --rasterizer CLI option to select PDF rasterization backend
Add user control over which rasterizer is used for PDF page rendering:
- 'auto' (default): prefers pypdfium when available, falls back to Ghostscript
- 'pypdfium': force pypdfium2 (errors if not installed)
- 'ghostscript': force traditional Ghostscript rasterizer

Changes:
- Add rasterizer field with validation to OCROptions model
- Add --rasterizer CLI argument in the Advanced options group
- Update rasterize_pdf_page hookspec to pass options to plugins
- Update pypdfium plugin with check_options hook for availability check
- Update both plugins to respect the rasterizer option

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:29:17 -08:00
James R. BarlowandClaude Opus 4.5 a4ee513cd4 refactor: clean up deprecated code and update plugin docs
- Remove outdated Phase comments from _options.py and cli.py
- Remove unused methods from PluginOptionRegistry:
  - get_extended_options_model() - replaced by __getattr__ in OCROptions
  - map_legacy_options() - unused
  - validate_plugin_options() - unused
- Update plugin documentation to document register_options hook
- Add documentation for nested plugin option access pattern

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:21:48 -08:00
James R. BarlowandClaude Opus 4.5 0ad7f5fc13 feat: add dynamic nested access to plugin options
Completes Phase 5 of the CLI refactoring plan by enabling nested
plugin option access (e.g., options.tesseract.timeout) alongside
the legacy flat access (options.tesseract_timeout).

Changes:
- Add module-level plugin option model registry in _options.py
- Add __getattr__ to OCROptions for dynamic namespace access
- Register plugin models in setup_plugin_infrastructure()
- Add test for nested plugin option access

Plugin option instances are lazily created from flat field values
and cached for subsequent access.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 95d9c3ed18 fix: add cross-cutting validation to OCROptions model
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider f91e41a209 fix: refine validation coordinator error handling
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 01ea6c2b8b feat: add ValidationCoordinator for cross-cutting validation
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider f02d733d31 feat: add legacy field mapping to plugin registry
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 28d6ea0f10 feat: Add CLI generation methods to plugin option models
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 40f01d85ae refactor: convert jbig2 properties to direct fields in OCROptions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider b7640bdb9c feat: implement plugin option models with backward compatibility
This commit introduces Pydantic models for plugin-specific options in OCRmyPDF, focusing on:
- Creating TesseractOptions, OptimizeOptions, and GhostscriptOptions
- Adding backward compatibility properties
- Maintaining existing CLI and API functionality
- Preparing for future plugin option registration system

The changes include:
- Added type-annotated option models with validation
- Updated OCROptions to include legacy fields
- Added backward compatibility properties for jbig2 options
- Prepared groundwork for dynamic plugin option handling

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlow b7737446e4 cli: push up imports 2025-12-21 12:21:48 -08:00
James R. Barlowandaider 4f9c4c3e52 refactor: reorganize CLI and options initialization
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlow 08ee5690bc Remove to_namespace and its user, since nothing triggers it 2025-12-21 12:21:48 -08:00
James R. Barlow 3f38ea4d80 Remove OCROptions getattr interface 2025-12-21 12:21:47 -08:00
James R. Barlowandaider f04b5504e8 test: fix hOCR pipeline output folder handling
The commit message captures the essence of the changes: we fixed how the output folder is handled in the hOCR pipeline by making it a proper field in OCROptions and updating the API functions accordingly.

Would you like me to generate a full commit message or is this sufficient?

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlow 69185e5819 refactor: remove custom attribute handling methods and add output_folder option 2025-12-21 12:21:47 -08:00
James R. Barlow e4f8ba8edc Fix computing of lossless_reconstruction and checking of redo_ocr conflicts 2025-12-21 12:21:47 -08:00
James R. Barlow 1f493ba789 refactor: post-AI code cleanup 2025-12-21 12:21:47 -08:00
James R. Barlowandaider e1d976168c feat: handle Pydantic serialization iterators in JSON serialization
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 60182ac8a8 fix: update JSON serialization tests to match default values
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlow 53db7b384b refactor: set default values for optional OCR configuration parameters 2025-12-21 12:21:47 -08:00
James R. Barlowandaider 9535b52d06 fix: revert default option values to preserve original behavior
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>

fix: set default optimize value to 1 to prevent NoneType comparison

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 21d69ffe87 fix: set default value for fast_web_view to prevent NoneType errors
This change ensures that `fast_web_view` always has a default value of 1.0, preventing the TypeError that was occurring in multiple test cases when trying to multiply `None` with an integer.

The modifications include:
1. Changing the type hint for `fast_web_view` from `float | None` to `float`
2. Setting a default value of 1.0
3. Adding a validation step in the `model_validator` to set the default value if not provided

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 3baeb83533 fix: Add compatibility for jpeg_quality and resolve strategy validation
This commit addresses several test failures by:
- Adding a compatibility property for `jpeg_quality`
- Changing default color conversion strategy from 'auto' to 'RGB'
- Ensuring `tesseract_config` is always a list
- Improving validation for color conversion and image compression strategies

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 134f4fcc28 fix: remove Union type hints and add default values in OCROptions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 3a0a7c546b feat: Implement JSON serialization for OCROptions with safe handling of Path and stream objects
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 7575dddefc fix: add lossless_reconstruction attribute to OCROptions for compatibility
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>

fix: resolve recursion error in lossless_reconstruction option

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>

fix: ensure lossless_reconstruction attribute is added to Namespace

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>

fix: ensure lossless_reconstruction attribute is correctly propagated to PageContext

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:42:49 -08:00
James R. Barlowandaider 66a3e8508e feat: add comprehensive validators to OCROptions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:42:23 -08:00
James R. Barlow 87478bc240 Fix options.jpg_quality issue 2025-12-13 11:42:23 -08:00
James R. Barlowandaider d18efcbbf1 fix: support flexible type inputs for pages and unpaper_args
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:41:27 -08:00
James R. Barlowandaider f9a4a2e240 fix: handle missing input_file in OCROptions namespace conversion
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:41:27 -08:00
James R. Barlowandaider 5f89100dc3 test: add lossless_reconstruction field to OCROptions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:41:27 -08:00
James R. Barlowandaider 1ef9aaf659 fix: Correct PDF/A part extraction and handle hOCR API output file
This commit addresses two issues:
1. Properly extract the PDF/A part from output_type
2. Add a placeholder output_file for hOCR API tests when output_folder is used

The changes include:
- Modifying `from_namespace` to add a placeholder output_file
- Updating PDF/A part extraction logic to handle different output_type formats
- Ensuring correct PDF/A part is passed to Ghostscript

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:41:27 -08:00
James R. Barlowandaider 4c4a1cfa17 fix: Resolve API test failures with pdf_renderer and output_file handling
This commit addresses several issues in the OCRmyPDF API:
- Fixed handling of 'auto' pdf_renderer by defaulting to 'hocr'
- Added placeholder for output_file when output_folder is present
- Updated model_fields access to use class method instead of instance attribute
- Improved error handling and default behavior in PDF rendering

Specifically:
- Modified `_options.py` to handle 'auto' pdf_renderer
- Updated attribute access to use class methods
- Added placeholder for output_file in special cases
- Updated `_pipelines/ocr.py` to handle 'auto' pdf_renderer

These changes resolve the test failures in `test_api.py` and improve the library's flexibility.

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:41:27 -08:00
James R. Barlowandaider 5251e21f7e refactor: migrate OCROptions validators to Pydantic V2
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:40:57 -08:00
James R. Barlowandaider 28eb923d9f feat: rename _extra_attrs to extra_attrs in OCROptions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:40:57 -08:00
James R. Barlowandaider 1579337ebe refactor: Create OCROptions model with Namespace compatibility
This commit introduces a new `OCROptions` class in `_options.py` that provides:
- Proper typing for OCRmyPDF options
- Pydantic validation
- Backward compatibility with `argparse.Namespace`
- Gradual migration support for the options system

Key changes:
- Added comprehensive option fields with type hints
- Implemented custom attribute access methods
- Created conversion methods between Namespace and OCROptions
- Updated type hints in multiple files to support both types
- Maintained existing validation logic

The new model allows for a step-by-step refactoring of the options handling throughout the project.

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:40:57 -08:00