Commit Graph
25 Commits
Author SHA1 Message Date
James R. Barlow dfdb32995e Fix mypy errors: drop deprecation dep, fix PathOrIO union bugs
Reduces mypy errors in src/ocrmypdf and tests from 89 to 51.

- Replace the `deprecation` package with stdlib `warnings.deprecated`
  (falling back to typing_extensions on <3.13); drop the dependency.
- Add pypdfium2/uharfbuzz/pi_heif to mypy's ignore_missing_imports
  overrides (no upstream stubs); drop pluggy, which now ships py.typed.
- Add a tests.* mypy override so test functions aren't required to
  annotate -> None.

Real bugs found and fixed along the way, not just annotations:
- OcrmypdfPluginManager had a `pluggy` property shadowing the `pluggy`
  module import within its own class body, breaking every
  `pluggy.PluginManager` annotation below it; renamed to
  `pluggy_manager`.
- `_option_registry` was bolted onto OcrmypdfPluginManager from outside
  and read via `getattr(..., None)` instead of being a declared
  attribute; declared it properly.
- ValidationCoordinator.__init__ was typed to accept a raw
  pluggy.PluginManager, but every caller passes the OcrmypdfPluginManager
  wrapper.
- check_options_sidecar() did `options.output_file + '.txt'`, assuming
  output_file is always a str; would raise a raw TypeError if ever hit
  with a stream/bytes output. Added an explicit guard.
- is_file_writable() called Path(test_file), which raises TypeError on
  a bytes path; fixed via os.fsdecode().
- copy_final() had a dead, unused `original_file` parameter; removed it.
- run_hocr_pipeline() constructed PdfContext with the raw, untriaged
  input_file instead of the locally-copied origin_pdf, inconsistent
  with the other two pipelines.
- _options.py had jbig2_threshold declared twice in the same model.
2026-07-07 00:25:33 -07:00
James R. Barlow c5d3ef4b17 Tighten ruff rules and modernize style 2026-01-27 14:04:52 -08:00
James R. Barlow 740f67091c Rename OCROptions to OcrOptions for consistency
Technically OCROptions is more Pythonic but we have several pre-existing classes named OcrWhatever. Go with the local flow.
2026-01-12 23:37:54 -08:00
James R. Barlow 0c3745a1a4 Add OCR engine selection framework and null OCR engine
Introduce --ocr-engine option to select between OCR engines:
- 'auto' (default): Uses Tesseract
- 'tesseract': Explicit Tesseract selection
- 'none': Skip OCR entirely (for PDF processing only)

Key changes:
- Extend OcrEngine ABC with generate_ocr() and supports_generate_ocr()
  for direct OcrElement tree output (bypasses hOCR)
- Add get_ocr_engine(options) hook parameter for engine selection
- Implement NullOcrEngine for --ocr-engine none
- Export OcrElement, OcrClass, BoundingBox from ocrmypdf package
- Add ocr_tree support to grafting pipeline

This prepares the foundation for pluggable OCR engines while maintaining
full backward compatibility with existing Tesseract-based workflows.
2026-01-12 10:11:14 -08:00
James R. Barlow 0e946a7498 Clarify messageabout number of workers 2026-01-07 16:41:18 -08:00
James R. Barlow 7a4b98974c Integrate fpdf2 renderer and remove legacy hOCR renderer
- Update pipeline to use fpdf2 renderer as default
- Remove legacy hocrtransform PDF renderer (_font.py, _hocr.py,
  pdf_renderer.py)
- Update CLI and options for fpdf2 renderer
- Add fpdf2 dependency to pyproject.toml
- Update graft module for fpdf2 multi-page rendering
2026-01-06 13:45:44 -08:00
James R. Barlow 9ebba91466 Use plugin namespace access pattern throughout codebase
Migrate all code from flat accessor pattern (options.tesseract_timeout)
to the plugin namespace pattern (options.tesseract.timeout).

Key changes:
- Fix _get_plugin_options to raise AttributeError for unregistered
  namespaces instead of silently returning None
- Add _convert_value helper to convert PathLike to str for plugin
  model field compatibility
- Filter out _plugin_cache_* entries from JSON serialization to fix
  worker process serialization (test_simulate_oom_killer)
- Update tesseract_ocr.py, ghostscript.py, _validation_coordinator.py,
  and _pipelines/ocr.py to use options.tesseract.* and
  options.ghostscript.* accessors
- Update tests to use setup_plugin_infrastructure() for plugin
  model registration
2025-12-23 02:02:21 -08:00
James R. Barlow 1f493ba789 refactor: post-AI code cleanup 2025-12-21 12:21:47 -08:00
James R. Barlowandaider 5f685aef6e fix: handle None jobs in concurrent processing
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:47 -08:00
James R. Barlowandaider 1d74c2831f refactor: update pipeline entry points to use OCROptions directly
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:46 -08:00
James R. Barlowandaider f5bfd2fd3e Add compat function for pages_from_ranges
fix: handle Pydantic validation errors with correct exit code

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-21 12:21:46 -08:00
James R. Barlowandaider 4c4a1cfa17 fix: Resolve API test failures with pdf_renderer and output_file handling
This commit addresses several issues in the OCRmyPDF API:
- Fixed handling of 'auto' pdf_renderer by defaulting to 'hocr'
- Added placeholder for output_file when output_folder is present
- Updated model_fields access to use class method instead of instance attribute
- Improved error handling and default behavior in PDF rendering

Specifically:
- Modified `_options.py` to handle 'auto' pdf_renderer
- Updated attribute access to use class methods
- Added placeholder for output_file in special cases
- Updated `_pipelines/ocr.py` to handle 'auto' pdf_renderer

These changes resolve the test failures in `test_api.py` and improve the library's flexibility.

Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-12-13 11:41:27 -08:00
James R. Barlow 9a6150ae53 Refactor get_pdfinfo common code 2024-10-27 12:09:33 -07:00
James R. Barlow f3d6387bca Fix "OCR" progress bar not matching actual progress 2024-05-21 01:35:14 -07:00
James R. Barlow 065bddbc6c Reformat with ruff format 2024-04-07 00:25:32 -07:00
James R. Barlow 3b9d4b7f0a Attempt to deal with oddball mediaboxes 2024-02-11 15:34:54 -08:00
James R. Barlow 11f53fe9a9 First cut at propagating page boxes
This would fix the immediate issue, but does not address an offset mediabox.
2024-02-11 15:34:54 -08:00
James R. Barlow 580252a1a0 Merge branch 'feature/gscan2pdf'
Reconcile release notes and copy_final() with new pipeline.
2023-10-30 00:01:28 -07:00
James R. Barlow a31f17bb9d Update comments and make worker functions private 2023-10-24 00:56:00 -07:00
James R. Barlow d2dbea6cf8 Reorganize progress bars so they can be typed properly 2023-10-24 00:54:31 -07:00
James R. Barlow 46a279a49a Improve passing of arguments to workers
The executor system was built around passing only a single
argument to workers, which was
always PageContext. For other tasks, all actual arguments were packed in
a tuple, which meant we needed intermediate functions to unpack the
tuple.

The situation is now rationlized and resembles how Python handles
argument passing to familiar multiprocessing tools.
2023-10-24 00:54:31 -07:00
James R. Barlow 6127f7abd6 tqdm_kwargs to progress_kwargs 2023-10-24 00:54:31 -07:00
James R. Barlow 16eb5627a7 Fix unused imports and other trivia 2023-10-24 00:54:31 -07:00
James R. Barlow 62c4f65fc3 Remove duplicate thread local storage of page numbers 2023-10-24 00:54:31 -07:00
James R. Barlow e400112f32 Improve ._pipelines naming 2023-10-24 00:54:31 -07:00