Commit Graph
1502 Commits
Author SHA1 Message Date
James R. Barlow 41758766a1 Test and fix page box issues 2025-12-21 14:03:28 -08:00
James R. BarlowandClaude Opus 4.5 3e46b039ed feat: add use_cropbox parameter to align rasterizer APIs
Added use_cropbox parameter to rasterize_pdf_page hook to allow
choosing between MediaBox and CropBox rendering:

- Default is use_cropbox=False (MediaBox) for consistency with
  Ghostscript's existing behavior
- Ghostscript: passes -dUseCropBox when use_cropbox=True
- pypdfium: calculates crop values to expand from CropBox to MediaBox
  when use_cropbox=False

This aligns both rasterizers to produce the same output dimensions
by default, making the rasterizer choice transparent for page
geometry.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:29:17 -08:00
James R. BarlowandClaude Opus 4.5 ae783b4ae6 fix: add thread safety lock to pypdfium plugin
pypdfium2/PDFium is not thread-safe - concurrent calls from different
threads can crash or corrupt the process. Added a module-level lock to
serialize all pdfium operations.

PIL image processing and file I/O are done outside the lock since they
are thread-safe, minimizing lock contention.

For maximum parallelism, users can use process-based parallelism
(use_threads=False) where each process has its own pdfium instance.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:29:17 -08:00
James R. BarlowandClaude Opus 4.5 b9f488d65c test: add comprehensive tests for --rasterizer option
Add test_rasterizer.py with tests covering:
- Basic rasterizer option validation ('auto', 'ghostscript', 'pypdfium')
- Rasterizer + --rotate-pages interaction
- PDFs with nonstandard MediaBox/TrimBox/CropBox
- Direct hook tests verifying plugins respect the option

Also fix pluggy parameter passing: make 'options' a required parameter
(no default) in the hookspec so pluggy forwards it to implementations.
Update test plugins and test_rotation.py to pass the new parameter.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:29:17 -08:00
James R. BarlowandClaude Opus 4.5 ed813cec67 feat: add --rasterizer CLI option to select PDF rasterization backend
Add user control over which rasterizer is used for PDF page rendering:
- 'auto' (default): prefers pypdfium when available, falls back to Ghostscript
- 'pypdfium': force pypdfium2 (errors if not installed)
- 'ghostscript': force traditional Ghostscript rasterizer

Changes:
- Add rasterizer field with validation to OCROptions model
- Add --rasterizer CLI argument in the Advanced options group
- Update rasterize_pdf_page hookspec to pass options to plugins
- Update pypdfium plugin with check_options hook for availability check
- Update both plugins to respect the rasterizer option

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:29:17 -08:00
James R. BarlowandClaude Opus 4.5 938ce8e285 fix: make pypdfium plugin optional with Ghostscript fallback
- Remove check_options hook from pypdfium that raised error when
  pypdfium2 wasn't installed
- Return None from pypdfium's rasterize_pdf_page when pypdfium2 is
  unavailable, allowing the hook to fall through
- Restore Ghostscript's rasterize_pdf_page hook as fallback

This allows OCRmyPDF to work without pypdfium2 installed, using
Ghostscript for rasterization as before.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:29:17 -08:00
James R. Barlow cf3fb6e89b Fix raster_device settings for pypdfium rasterizer 2025-12-21 12:29:17 -08:00
James R. Barlowandaider 3482ea5fe5 refactor: Modularize rasterize_pdf_page into separate PDF, page, and image processing functions
Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat>
2025-12-21 12:29:17 -08:00
James R. Barlow e85c5bbb4d refactor: Simplify error message and code formatting in pypdfium plugin 2025-12-21 12:29:17 -08:00
James R. Barlowandaider 740b0bddc6 feat: add pypdfium2 rasterization plugin for OCRmyPDF
Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat>
2025-12-21 12:29:17 -08:00
James R. BarlowandClaude Opus 4.5 a4ee513cd4 refactor: clean up deprecated code and update plugin docs
- Remove outdated Phase comments from _options.py and cli.py
- Remove unused methods from PluginOptionRegistry:
  - get_extended_options_model() - replaced by __getattr__ in OCROptions
  - map_legacy_options() - unused
  - validate_plugin_options() - unused
- Update plugin documentation to document register_options hook
- Add documentation for nested plugin option access pattern

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:21:48 -08:00
James R. BarlowandClaude Opus 4.5 0ad7f5fc13 feat: add dynamic nested access to plugin options
Completes Phase 5 of the CLI refactoring plan by enabling nested
plugin option access (e.g., options.tesseract.timeout) alongside
the legacy flat access (options.tesseract_timeout).

Changes:
- Add module-level plugin option model registry in _options.py
- Add __getattr__ to OCROptions for dynamic namespace access
- Register plugin models in setup_plugin_infrastructure()
- Add test for nested plugin option access

Plugin option instances are lazily created from flat field values
and cached for subsequent access.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider b89bb3b524 fix: add blocked language validation for osd and equ
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 95d9c3ed18 fix: add cross-cutting validation to OCROptions model
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider f91e41a209 fix: refine validation coordinator error handling
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider f6fcdfa618 fix: allow 0 as valid value for jbig2_page_group_size
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 01ea6c2b8b feat: add ValidationCoordinator for cross-cutting validation
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider f02d733d31 feat: add legacy field mapping to plugin registry
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 28d6ea0f10 feat: Add CLI generation methods to plugin option models
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 6913ec7cb8 feat: add PluginOptionRegistry for dynamic option models
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 40f01d85ae refactor: convert jbig2 properties to direct fields in OCROptions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider b7640bdb9c feat: implement plugin option models with backward compatibility
This commit introduces Pydantic models for plugin-specific options in OCRmyPDF, focusing on:
- Creating TesseractOptions, OptimizeOptions, and GhostscriptOptions
- Adding backward compatibility properties
- Maintaining existing CLI and API functionality
- Preparing for future plugin option registration system

The changes include:
- Added type-annotated option models with validation
- Updated OCROptions to include legacy fields
- Added backward compatibility properties for jbig2 options
- Prepared groundwork for dynamic plugin option handling

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 62ad37b276 refactor: centralize plugin manager setup
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlow e4fa9dbc8f Drop obsolete subclass of ArgumentParser 2025-12-21 12:21:48 -08:00
James R. Barlow b7737446e4 cli: push up imports 2025-12-21 12:21:48 -08:00
James R. Barlowandaider 4ed0e4510c fix: add type checking imports for OCROptions and pluggy
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlowandaider 4f9c4c3e52 refactor: reorganize CLI and options initialization
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:48 -08:00
James R. Barlow 08ee5690bc Remove to_namespace and its user, since nothing triggers it 2025-12-21 12:21:48 -08:00
James R. Barlow 3f38ea4d80 Remove OCROptions getattr interface 2025-12-21 12:21:47 -08:00
James R. Barlowandaider f04b5504e8 test: fix hOCR pipeline output folder handling
The commit message captures the essence of the changes: we fixed how the output folder is handled in the hOCR pipeline by making it a proper field in OCROptions and updating the API functions accordingly.

Would you like me to generate a full commit message or is this sufficient?

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlow 69185e5819 refactor: remove custom attribute handling methods and add output_folder option 2025-12-21 12:21:47 -08:00
James R. Barlow ade3ecd5a1 fix: add error handling in hOCR pipeline 2025-12-21 12:21:47 -08:00
James R. Barlow e4f8ba8edc Fix computing of lossless_reconstruction and checking of redo_ocr conflicts 2025-12-21 12:21:47 -08:00
James R. Barlow 1225c0a45e fix: correct typing issues in concurrent and optimize modules 2025-12-21 12:21:47 -08:00
James R. Barlowandaider f0c292f4e1 refactor: Remove CLI-parser dependencies in experimental API functions
This commit updates `_pdf_to_hocr` and `_hocr_to_ocr_pdf` to use direct OCROptions construction, eliminating the last vestiges of CLI-parser dependency in the experimental APIs.

Key changes:
- Removed `parser = get_parser()` calls
- Added plugin validation similar to main `ocr()` function
- Simplified plugin manager hook calls
- Added None value filtering to use OCROptions defaults
- Maintained error handling and extra_attrs logic

The refactoring makes these experimental APIs truly API-first and simplifies the code by removing unnecessary CLI-related complexity.

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlow 1f493ba789 refactor: post-AI code cleanup 2025-12-21 12:21:47 -08:00
James R. Barlowandaider e1d976168c feat: handle Pydantic serialization iterators in JSON serialization
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 60182ac8a8 fix: update JSON serialization tests to match default values
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlow 53db7b384b refactor: set default values for optional OCR configuration parameters 2025-12-21 12:21:47 -08:00
James R. Barlowandaider ff250afa51 fix: add helper function to calculate effective JBIG2 page group size
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider cdb976db41 fix: handle None or zero jbig2_page_group_size in extract_images_jbig2
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 9535b52d06 fix: revert default option values to preserve original behavior
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>

fix: set default optimize value to 1 to prevent NoneType comparison

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 21d69ffe87 fix: set default value for fast_web_view to prevent NoneType errors
This change ensures that `fast_web_view` always has a default value of 1.0, preventing the TypeError that was occurring in multiple test cases when trying to multiply `None` with an integer.

The modifications include:
1. Changing the type hint for `fast_web_view` from `float | None` to `float`
2. Setting a default value of 1.0
3. Adding a validation step in the `model_validator` to set the default value if not provided

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 3baeb83533 fix: Add compatibility for jpeg_quality and resolve strategy validation
This commit addresses several test failures by:
- Adding a compatibility property for `jpeg_quality`
- Changing default color conversion strategy from 'auto' to 'RGB'
- Ensuring `tesseract_config` is always a list
- Improving validation for color conversion and image compression strategies

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 134f4fcc28 fix: remove Union type hints and add default values in OCROptions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider a869a4ac42 fix: filter out None values from OCROptions kwargs
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 48a2fdb0f2 refactor: replace _kwargs_to_cmdline with direct OCROptions construction in experimental API functions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 91a2d39845 refactor: replace command line synthesis with direct OCROptions construction
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 3a0a7c546b feat: Implement JSON serialization for OCROptions with safe handling of Path and stream objects
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider d0a46a0359 feat: convert CLI Namespace to OCROptions in main entry point
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00