Comprrehensive documentation update in preparation for v17
This commit is contained in:
+176
-23
@@ -58,6 +58,38 @@ disk space.
|
||||
OCRmyPDF provides many features to control the behavior of the OCR
|
||||
engine, Tesseract.
|
||||
|
||||
### OCR processing mode
|
||||
|
||||
:::{versionadded} 17.0.0
|
||||
The `--mode` (`-m`) argument consolidates OCR processing options.
|
||||
:::
|
||||
|
||||
OCRmyPDF provides a unified `--mode` argument to control how pages with
|
||||
existing text are handled:
|
||||
|
||||
| Mode | Behavior | Legacy equivalent |
|
||||
|------|----------|-------------------|
|
||||
| `default` | Error if text is found | (no flag) |
|
||||
| `force` | Rasterize all content and run OCR | `--force-ocr` |
|
||||
| `skip` | Skip pages with existing text | `--skip-text` |
|
||||
| `redo` | Re-OCR pages, stripping old OCR layer | `--redo-ocr` |
|
||||
|
||||
```bash
|
||||
# Skip pages that already have text
|
||||
ocrmypdf --mode skip input.pdf output.pdf
|
||||
# or equivalently:
|
||||
ocrmypdf -m skip input.pdf output.pdf
|
||||
|
||||
# Force OCR on all pages (rasterizes everything)
|
||||
ocrmypdf --mode force input.pdf output.pdf
|
||||
|
||||
# Re-do OCR, replacing old invisible text
|
||||
ocrmypdf --mode redo input.pdf output.pdf
|
||||
```
|
||||
|
||||
The legacy flags (`--force-ocr`, `--skip-text`, `--redo-ocr`) remain as
|
||||
silent aliases for backward compatibility.
|
||||
|
||||
### When OCR is skipped
|
||||
|
||||
If a page in a PDF seems to have text, by default OCRmyPDF will exit
|
||||
@@ -65,13 +97,13 @@ without modifying the PDF. This is to ensure that PDFs that were
|
||||
previously OCRed or were "born digital" rather than scanned are not
|
||||
processed.
|
||||
|
||||
If `--skip-text` is issued, then no image processing or OCR will be
|
||||
If `--mode skip` (or `--skip-text`) is issued, then no image processing or OCR will be
|
||||
performed on pages that already have text. The page will be copied to
|
||||
the output. This may be useful for documents that contain both "born
|
||||
digital" and scanned content, or to use OCRmyPDF to normalize and
|
||||
convert to PDF/A regardless of their contents.
|
||||
|
||||
If `--redo-ocr` is issued, then a detailed text analysis is performed.
|
||||
If `--mode redo` (or `--redo-ocr`) is issued, then a detailed text analysis is performed.
|
||||
Text is categorized as either visible or invisible. Invisible text (OCR)
|
||||
is stripped out. Then an image of each page is created with visible text
|
||||
masked out. The page image is sent for OCR, and any additional text is
|
||||
@@ -82,7 +114,7 @@ technically printable or visible in some way, perhaps by drawing it and
|
||||
then painting over it. OCRmyPDF cannot distinguish this type of OCR
|
||||
text from real text, so it will not be "redone".
|
||||
|
||||
If `--force-ocr` is issued, then all pages will be rasterized to
|
||||
If `--mode force` (or `--force-ocr`) is issued, then all pages will be rasterized to
|
||||
images, discarding any hidden OCR text, rasterizing any printable
|
||||
text, and flattening form fields or interactive objects into their visual
|
||||
representation. This is useful for redoing OCR, for fixing OCR text
|
||||
@@ -257,44 +289,85 @@ Their use may interfere with `--rotate-pages` and other features.
|
||||
It is currently not possible to use advanced Tesseract OCR features, such as creating
|
||||
OCR information, when using Tesseract through OCRmyPDF.
|
||||
|
||||
## Changing the PDF renderer
|
||||
## Choosing a PDF rasterizer
|
||||
|
||||
:::{versionadded} 17.0.0
|
||||
:::
|
||||
|
||||
rasterizing
|
||||
|
||||
: Converting a PDF to an image for display.
|
||||
: Converting a PDF page to an image for OCR processing.
|
||||
|
||||
OCRmyPDF supports two PDF rasterizers:
|
||||
|
||||
| Rasterizer | Package | Advantages | Disadvantages |
|
||||
|------------|---------|------------|---------------|
|
||||
| pypdfium2 | Python package | Faster, fewer version issues | Requires pypdfium2 package |
|
||||
| Ghostscript | System binary | More widely packaged | Version consistency issues, restrictive AGPLv3 |
|
||||
|
||||
The `--rasterizer` argument controls which rasterizer is used:
|
||||
|
||||
```bash
|
||||
# Automatic selection (default) - prefers pypdfium when available
|
||||
ocrmypdf --rasterizer auto input.pdf output.pdf
|
||||
|
||||
# Force pypdfium2
|
||||
ocrmypdf --rasterizer pypdfium input.pdf output.pdf
|
||||
|
||||
# Force Ghostscript
|
||||
ocrmypdf --rasterizer ghostscript input.pdf output.pdf
|
||||
```
|
||||
|
||||
pypdfium2 is a Python binding for pdfium, the PDF rendering library used
|
||||
by Google Chrome and Chromium. It generally produces output identical to
|
||||
Ghostscript but with better performance.
|
||||
|
||||
:::{note}
|
||||
If pypdfium2 is not installed and `--rasterizer pypdfium` is requested,
|
||||
OCRmyPDF will exit with an error. Install it with: `pip install pypdfium2`
|
||||
:::
|
||||
|
||||
## Changing the PDF renderer
|
||||
|
||||
rendering
|
||||
|
||||
: Creating a new PDF from other data (such as an existing PDF).
|
||||
|
||||
OCRmyPDF has these PDF renderers: `sandwich` and `hocr`. The
|
||||
:::{versionchanged} 17.0.0
|
||||
The fpdf2 renderer is now the default, replacing the legacy hOCR renderer.
|
||||
:::
|
||||
|
||||
OCRmyPDF uses PDF renderers to create the invisible text layer. The
|
||||
renderer may be selected using `--pdf-renderer`. The default is
|
||||
`auto` which lets OCRmyPDF select the renderer to use. Currently,
|
||||
`auto` always selects `hocr`.
|
||||
`auto` which selects `fpdf2`.
|
||||
|
||||
### The `hocr` renderer
|
||||
### The `fpdf2` renderer (default)
|
||||
|
||||
:::{versionchanged} 16.0.0
|
||||
:::{versionadded} 17.0.0
|
||||
:::
|
||||
|
||||
The fpdf2 renderer creates text layers using the fpdf2 library. It provides:
|
||||
|
||||
- Full multilingual support including RTL languages (Arabic, Hebrew, Persian)
|
||||
- Accurate text positioning aligned with OCR bounding boxes
|
||||
- Improved "Occulta" glyphless font handling:
|
||||
- Zero-width markers are properly handled
|
||||
- Double-width CJK characters are properly sized
|
||||
- Direct OcrElement tree input (no hOCR intermediate format required)
|
||||
|
||||
The fpdf2 renderer is the recommended choice for all installations.
|
||||
|
||||
:::{note}
|
||||
The fpdf2 renderer may be slightly slower than the legacy hocrtransform
|
||||
renderer for some workloads. This is an area of ongoing optimization.
|
||||
:::
|
||||
|
||||
In both renderers, a text-only layer is rendered and sandwiched (overlaid)
|
||||
on to either the original PDF page, or newly rasterized version of the
|
||||
original PDF page (when `--force-ocr` is used). In this way, loss
|
||||
original PDF page (when `--mode force` is used). In this way, loss
|
||||
of PDF information is generally avoided. (You may need to disable PDF/A
|
||||
conversion and optimization to eliminate all lossy transformations.)
|
||||
|
||||
The current approach used by the new hOCR renderer is a re-implementation
|
||||
of Tesseract's PDF renderer, using the same Glyphless font and general
|
||||
ideas, but fixing many technical issues that impeded it. The new hocr
|
||||
provides better text placement accuracy, avoids issues with word
|
||||
segmentation, and provides better positioning of skewed text.
|
||||
|
||||
Using the experimental API, it is also possible to edit the OCR output
|
||||
from Tesseract, using any tool that is capable of editing hOCR files.
|
||||
|
||||
Older versions of this renderer did not support non-Latin languages, but
|
||||
it is now universal.
|
||||
|
||||
### The `sandwich` renderer
|
||||
|
||||
The `sandwich` renderer uses Tesseract's text-only PDF feature,
|
||||
@@ -310,6 +383,11 @@ When image preprocessing features like `--deskew` are used, the
|
||||
original PDF will be rendered as a full page and the OCR layer will be
|
||||
placed on top.
|
||||
|
||||
### Legacy renderer options
|
||||
|
||||
The `hocr` and `hocrdebug` renderer options are deprecated and
|
||||
automatically redirect to `fpdf2`. They will be removed in a future version.
|
||||
|
||||
## Rendering and rasterizing options
|
||||
|
||||
:::{versionadded} 14.3.0
|
||||
@@ -341,6 +419,81 @@ curves. In this case, you may want to use a different color conversion
|
||||
strategy. The `--color-conversion-strategy` option allows you to select a
|
||||
different strategy, such as `RGB`.
|
||||
|
||||
## PDF/A output modes
|
||||
|
||||
:::{versionchanged} 17.0.0
|
||||
The default `--output-type` is now `auto` instead of `pdfa`.
|
||||
:::
|
||||
|
||||
OCRmyPDF can produce PDF/A compliant output for long-term archival. The
|
||||
`--output-type` argument controls PDF/A conversion:
|
||||
|
||||
| Output type | Behavior |
|
||||
|-------------|----------|
|
||||
| `auto` | Best-effort PDF/A without requiring Ghostscript (default) |
|
||||
| `pdfa` | PDF/A-2b via Ghostscript |
|
||||
| `pdfa-1` | PDF/A-1b via Ghostscript |
|
||||
| `pdfa-2` | PDF/A-2b via Ghostscript (same as `pdfa`) |
|
||||
| `pdfa-3` | PDF/A-3b via Ghostscript |
|
||||
| `pdf` | Standard PDF, no PDF/A conversion |
|
||||
| `none` | No output file (useful with `--sidecar`) |
|
||||
|
||||
### Speculative PDF/A conversion
|
||||
|
||||
:::{versionadded} 17.0.0
|
||||
:::
|
||||
|
||||
When `--output-type auto` is used (the default), OCRmyPDF attempts a
|
||||
fast "speculative" PDF/A conversion that avoids Ghostscript when possible:
|
||||
|
||||
1. OCRmyPDF adds an sRGB ICC profile and PDF/A XMP metadata using pikepdf
|
||||
2. If verapdf is available, it validates the result
|
||||
3. If validation passes, Ghostscript is skipped entirely
|
||||
4. If validation fails or verapdf is unavailable, falls back to Ghostscript
|
||||
|
||||
This approach is faster and avoids some Ghostscript limitations (such as
|
||||
image transcoding), but only works for PDFs that are already "mostly"
|
||||
PDF/A compliant.
|
||||
|
||||
### PDF/A conversion flow
|
||||
|
||||
The following diagram illustrates the PDF/A conversion decision tree:
|
||||
|
||||
```{mermaid}
|
||||
flowchart TD
|
||||
A[Start] --> B{--output-type?}
|
||||
B -->|pdf| C[Output standard PDF]
|
||||
B -->|pdfa/pdfa-N| D[Use Ghostscript]
|
||||
B -->|auto| E[Attempt speculative conversion]
|
||||
|
||||
E --> F["Add sRGB ICC + XMP metadata (pikepdf)"]
|
||||
F --> G{verapdf available?}
|
||||
|
||||
G -->|No| H{Ghostscript available?}
|
||||
G -->|Yes| I[Validate with verapdf]
|
||||
|
||||
I --> J{Validation passed?}
|
||||
J -->|Yes| K[Output PDF/A - Ghostscript skipped]
|
||||
J -->|No| H
|
||||
|
||||
H -->|Yes| D
|
||||
H -->|No| L[Output standard PDF + WARNING]
|
||||
|
||||
D --> M[Ghostscript PDF/A conversion]
|
||||
M --> N[Output PDF/A]
|
||||
|
||||
style K fill:#90EE90
|
||||
style N fill:#90EE90
|
||||
style L fill:#FFB6C1
|
||||
```
|
||||
|
||||
:::{warning}
|
||||
**Breaking change:** If neither Ghostscript nor verapdf is installed,
|
||||
`--output-type auto` will produce a standard PDF instead of PDF/A.
|
||||
This is a change from previous versions where Ghostscript was required
|
||||
and PDF/A was always produced.
|
||||
:::
|
||||
|
||||
## Return code policy
|
||||
|
||||
OCRmyPDF writes all messages to `stderr`. `stdout` is reserved for
|
||||
|
||||
@@ -41,6 +41,7 @@ extensions = [
|
||||
'sphinx.ext.napoleon',
|
||||
'sphinx.ext.imgconverter', # PDF docs needs this for SVG to PNG conversion
|
||||
'sphinx_issues',
|
||||
'sphinxcontrib.mermaid',
|
||||
]
|
||||
|
||||
myst_enable_extensions = ['colon_fence', 'attrs_block', 'attrs_inline', 'substitution']
|
||||
|
||||
@@ -260,6 +260,66 @@ You can also optimize all images without performing any OCR:
|
||||
ocrmypdf --ocr-engine none --optimize 3 --skip-text input.pdf output.pdf
|
||||
```
|
||||
|
||||
## Using v17 features
|
||||
|
||||
### Select a rasterizer
|
||||
|
||||
:::{versionadded} 17.0.0
|
||||
:::
|
||||
|
||||
OCRmyPDF can use pypdfium2 or Ghostscript to rasterize PDF pages. pypdfium2
|
||||
is generally faster and is preferred when available.
|
||||
|
||||
```bash
|
||||
# Automatic selection (default) - prefers pypdfium when available
|
||||
ocrmypdf --rasterizer auto input.pdf output.pdf
|
||||
|
||||
# Explicitly use pypdfium2 (requires pip install pypdfium2)
|
||||
ocrmypdf --rasterizer pypdfium input.pdf output.pdf
|
||||
|
||||
# Explicitly use Ghostscript
|
||||
ocrmypdf --rasterizer ghostscript input.pdf output.pdf
|
||||
```
|
||||
|
||||
### PDF/A without Ghostscript
|
||||
|
||||
:::{versionadded} 17.0.0
|
||||
:::
|
||||
|
||||
With verapdf installed, OCRmyPDF can produce PDF/A without using Ghostscript
|
||||
for conversion. This is faster and avoids some Ghostscript limitations.
|
||||
|
||||
```bash
|
||||
# Uses speculative conversion with verapdf validation (default)
|
||||
ocrmypdf --output-type auto input.pdf output.pdf
|
||||
|
||||
# Explicitly request Ghostscript-based PDF/A conversion
|
||||
ocrmypdf --output-type pdfa input.pdf output.pdf
|
||||
```
|
||||
|
||||
### Using --mode instead of legacy flags
|
||||
|
||||
:::{versionadded} 17.0.0
|
||||
:::
|
||||
|
||||
The `--mode` (`-m`) flag consolidates OCR behavior options:
|
||||
|
||||
```bash
|
||||
# Instead of --skip-text
|
||||
ocrmypdf --mode skip input.pdf output.pdf
|
||||
|
||||
# Instead of --force-ocr
|
||||
ocrmypdf --mode force input.pdf output.pdf
|
||||
|
||||
# Instead of --redo-ocr
|
||||
ocrmypdf --mode redo input.pdf output.pdf
|
||||
|
||||
# Short form
|
||||
ocrmypdf -m skip input.pdf output.pdf
|
||||
```
|
||||
|
||||
The legacy flags continue to work as aliases.
|
||||
|
||||
### Process only certain pages
|
||||
|
||||
You can ask OCRmyPDF to only apply [image processing](#image-processing)
|
||||
|
||||
+37
-7
@@ -599,21 +599,45 @@ OCRmyPDF currently requires these external programs and libraries to be
|
||||
installed, and must be satisfied using the operating system package
|
||||
manager. `pip` cannot provide them.
|
||||
|
||||
:::{versionchanged} 17.0.0
|
||||
Ghostscript is now optional. pypdfium2 can be used for PDF rasterization,
|
||||
and verapdf can validate speculative PDF/A conversion.
|
||||
:::
|
||||
|
||||
The following versions are required:
|
||||
|
||||
- Python 3.10 or newer
|
||||
- Ghostscript 9.54 or newer
|
||||
- Tesseract 4.1.1 or newer
|
||||
- jbig2enc 0.29 or newer
|
||||
- pngquant 2.5 or newer
|
||||
- unpaper 6.1
|
||||
- One of: Ghostscript 9.54+ **or** pypdfium2 (Python package)
|
||||
- One of: Ghostscript 9.54+ **or** verapdf (for PDF/A output)
|
||||
- fpdf2 2.8 or newer (Python package)
|
||||
- jbig2enc 0.29 or newer (optional)
|
||||
- pngquant 2.5 or newer (optional)
|
||||
- unpaper 6.1 (optional)
|
||||
|
||||
:::{note}
|
||||
For the best user experience, install both Ghostscript and pypdfium2.
|
||||
pypdfium2 is faster for rasterization, while Ghostscript provides
|
||||
broader compatibility and is required for certain PDF/A conversions.
|
||||
:::
|
||||
|
||||
We recommend 64-bit versions of all software. (32-bit versions are not
|
||||
supported, although on Linux, they may still work.)
|
||||
|
||||
jbig2enc, pngquant, and unpaper are optional. If missing certain
|
||||
features are disabled. OCRmyPDF will discover them as soon as they are
|
||||
available.
|
||||
**fpdf2** is a required dependency that provides the text layer
|
||||
rendering engine. It replaces the legacy hOCR-based renderer with improved
|
||||
multilingual support. Install with: `pip install fpdf2`
|
||||
|
||||
**pypdfium2**, if present, provides fast PDF page rasterization using
|
||||
the pdfium library (the same library used by Google Chrome). It is
|
||||
preferred over Ghostscript when available due to better performance.
|
||||
Install with: `pip install pypdfium2`
|
||||
|
||||
**verapdf**, if present, enables fast speculative PDF/A conversion.
|
||||
OCRmyPDF attempts to create PDF/A by adding metadata and ICC profiles
|
||||
using pikepdf, then validates with verapdf. If validation passes,
|
||||
Ghostscript is skipped entirely. See your distribution's package manager
|
||||
or visit [verapdf.org](https://verapdf.org/).
|
||||
|
||||
**jbig2enc**, if present, will be used to optimize the encoding of
|
||||
monochrome images. This can significantly reduce the file size of the
|
||||
@@ -623,6 +647,12 @@ available for Ubuntu or Debian due to lingering concerns about patent
|
||||
issues, but can easily be built from source. To add JBIG2 encoding, see
|
||||
{ref}`jbig2`.
|
||||
|
||||
:::{warning}
|
||||
Lossy JBIG2 encoding (`--jbig2-lossy`) has been removed in v17.0.0 due to
|
||||
well-documented risks of character substitution errors. Only lossless
|
||||
JBIG2 compression is now supported.
|
||||
:::
|
||||
|
||||
**pngquant**, if present, is optionally used to optimize the encoding of
|
||||
PNG-style images in PDFs (actually, any that are that losslessly
|
||||
encoded) by lossily quantizing to a smaller color palette. It is only
|
||||
|
||||
+25
-5
@@ -77,10 +77,17 @@ straightforward, and any PDF viewer can handle PDF/A files.
|
||||
|
||||
OCRmyPDF analyzes each page of a PDF to determine the required colorspace
|
||||
and resolution (DPI) for capturing all the information on that page without
|
||||
losing content. It uses
|
||||
[Ghostscript](http://ghostscript.com/) to rasterize each page and subsequently
|
||||
performs OCR on the rasterized image to generate an OCR "layer." This layer
|
||||
is then integrated back into the original PDF.
|
||||
losing content. It uses a PDF rasterizer (pypdfium2 or
|
||||
[Ghostscript](http://ghostscript.com/)) to convert each page to an image and
|
||||
subsequently performs OCR on the rasterized image to generate an OCR "layer."
|
||||
This layer is then integrated back into the original PDF.
|
||||
|
||||
:::{versionchanged} 17.0.0
|
||||
OCRmyPDF now supports pypdfium2 as an alternative rasterizer to Ghostscript.
|
||||
pypdfium2 is a Python binding for pdfium, the PDF rendering library used by
|
||||
Google Chrome. The `--rasterizer auto` setting (default) prefers pypdfium2
|
||||
when available.
|
||||
:::
|
||||
|
||||
While it is possible to use a program like Ghostscript or ImageMagick to
|
||||
obtain an image and then run that image through Tesseract OCR, this process
|
||||
@@ -156,7 +163,16 @@ These limitations are inherent to any software relying on Tesseract:
|
||||
the text and its bounding box. As such, the generated PDF does not
|
||||
contain any information about the document's structure.
|
||||
|
||||
Ghostscript also imposes some limitations:
|
||||
### Ghostscript considerations
|
||||
|
||||
:::{versionchanged} 17.0.0
|
||||
Ghostscript is no longer strictly required. OCRmyPDF can use pypdfium2
|
||||
for rasterization and verapdf for PDF/A validation.
|
||||
:::
|
||||
|
||||
While Ghostscript remains a capable and feature-rich tool with a long history,
|
||||
recent releases have introduced some compatibility challenges that OCRmyPDF
|
||||
v17 addresses through alternative codepaths. When Ghostscript is used:
|
||||
|
||||
- PDFs containing JPEG 2000-encoded content may be converted to JPEG
|
||||
encoding, which may introduce compression artifacts, if Ghostscript
|
||||
@@ -173,6 +189,10 @@ Ghostscript also imposes some limitations:
|
||||
- Ghostscript's PDF/A conversion may remove or deactivate
|
||||
hyperlinks and other active content.
|
||||
|
||||
When pypdfium2 and verapdf are available, many of these limitations can be
|
||||
avoided by using the speculative PDF/A conversion path (enabled by default
|
||||
with `--output-type auto`).
|
||||
|
||||
You can use `--output-type pdf` to disable PDF/A conversion and produce
|
||||
a standard, non-archival PDF.
|
||||
|
||||
|
||||
+114
-6
@@ -20,12 +20,48 @@ with much stiffer build requirements. If you want to use OCRmyPDF on
|
||||
some novel platform or distribution, first make sure you can package
|
||||
pikepdf.
|
||||
|
||||
### Non-Python dependencies
|
||||
### Core dependencies
|
||||
|
||||
Note that we have non-Python dependencies. In particular, OCRmyPDF
|
||||
requires Ghostscript and Tesseract OCR to be installed and needs to be
|
||||
able to locate their binaries on the system PATH. On Windows, OCRmyPDF
|
||||
will also check the registry for their locations.
|
||||
:::{versionchanged} 17.0.0
|
||||
Ghostscript is no longer strictly required. OCRmyPDF now supports alternative
|
||||
codepaths for both PDF rasterization and PDF/A conversion.
|
||||
:::
|
||||
|
||||
OCRmyPDF has the following runtime dependencies:
|
||||
|
||||
**For PDF rasterization** (converting PDF pages to images for OCR):
|
||||
|
||||
- `pypdfium2` (Python package) - OR -
|
||||
- `ghostscript` (system binary)
|
||||
- Recommendation: Install both for best compatibility
|
||||
|
||||
**For PDF/A conversion**:
|
||||
|
||||
- `verapdf` (system binary) with pikepdf's speculative conversion - OR -
|
||||
- `ghostscript` (system binary)
|
||||
- Recommendation: Install both for best compatibility
|
||||
|
||||
**For OCR**:
|
||||
- `tesseract-ocr` (system binary) - Required for MVP
|
||||
|
||||
**For text rendering** (expressing OCR results in PDF):
|
||||
- `fpdf2` (Python package) - Required for text layer rendering
|
||||
- `uharfbuzz` (Python package) - Required for text layer rendering
|
||||
- `font-noto` (system package) - Recommended for text layer rendering
|
||||
|
||||
**Other dependencies**:
|
||||
- `unpaper` (system binary) - Optional, enables `--clean` and `--clean-final`
|
||||
- `pngquant` (system binary) - Optional, enables `--optimize 2` and `--optimize 3`
|
||||
- `jbig2enc` (system binary) - Optional, improves compression of monochrome images
|
||||
|
||||
While Ghostscript remains a capable and feature-rich tool with a long history,
|
||||
recent releases have introduced some compatibility challenges that OCRmyPDF v17
|
||||
addresses through alternative codepaths. For the best user experience, packagers
|
||||
should install both Ghostscript and the alternative tools (pypdfium2, verapdf)
|
||||
when available.
|
||||
|
||||
On Windows, OCRmyPDF will also check the registry for Tesseract and Ghostscript
|
||||
locations.
|
||||
|
||||
Tesseract OCR relies on SIMD for performance and only has proper support
|
||||
for this on ARM and x86\_64. Performance may be poor on other processor
|
||||
@@ -48,7 +84,79 @@ override versioning for some reason.
|
||||
OCRmyPDF will use jbig2enc, a JBIG2 encoder, if one can be found. Some
|
||||
distributions have shied away from packaging JBIG2 because it contains
|
||||
patented algorithms, but all patents have expired since 2017. If
|
||||
possible, consider packaging it too to improve OCRmyPDF\'s compression.
|
||||
possible, consider packaging it too to improve OCRmyPDF's compression.
|
||||
|
||||
:::{note}
|
||||
Lossy JBIG2 encoding has been removed in v17.0.0 due to well-documented
|
||||
risks of character substitution errors. Previously we provided this feature
|
||||
on a "caveat emptor" basis but in the interest of focusing and eliminating
|
||||
risks, we decided to remove this option. Now, only lossless JBIG2 compression
|
||||
is supported.
|
||||
:::
|
||||
|
||||
### Dependency matrix for packagers
|
||||
|
||||
:::{versionadded} 17.0.0
|
||||
:::
|
||||
|
||||
The following table summarizes the dependency options introduced in v17.0.0:
|
||||
|
||||
| Feature | Option 1 | Option 2 | Notes |
|
||||
|---------|----------|----------|-------|
|
||||
| PDF rasterization | pypdfium2 (Python) | ghostscript (binary) | pypdfium2 preferred when available |
|
||||
| PDF/A conversion | verapdf + pikepdf | ghostscript | verapdf validates speculative conversion |
|
||||
| Text rendering | fpdf2 (Python) | - | Required, replaces legacy hOCR renderer |
|
||||
| OCR | tesseract-ocr | `--ocr-engine none` | Can be skipped entirely |
|
||||
|
||||
**Minimum viable installation:**
|
||||
|
||||
- tesseract-ocr + (pypdfium2 OR ghostscript) + fpdf2
|
||||
|
||||
**Recommended installation:**
|
||||
|
||||
- tesseract-ocr + pypdfium2 + ghostscript + verapdf + fpdf2 + unpaper + pngquant + jbig2enc
|
||||
|
||||
:::{warning}
|
||||
If Ghostscript is not installed and verapdf is not available, PDF/A output
|
||||
cannot be produced. The output will be a standard PDF instead. This is a
|
||||
breaking change for rare configurations that previously relied on PDF/A
|
||||
output without Ghostscript alternatives.
|
||||
:::
|
||||
|
||||
**Sample debian/control dependency specification**
|
||||
|
||||
```
|
||||
Depends:
|
||||
fonts-noto,
|
||||
fpdf2 (>= 2.8),
|
||||
ghostscript (>= 9.55), # Not strictly required, but best user experience
|
||||
icc-profiles-free,
|
||||
img2pdf,
|
||||
python3-coloredlogs,
|
||||
python3-deprecation,
|
||||
python3-pdfminer (>= 20181108+dfsg-3),
|
||||
python3-pikepdf (>= 8.14.0),
|
||||
python3-pil,
|
||||
python3-pluggy,
|
||||
python3-reportlab,
|
||||
python3-rich,
|
||||
python3-uharfbuzz, # Not currently in Debian
|
||||
tesseract-ocr (>= 5.0.0),
|
||||
zlib1g,
|
||||
${misc:Depends},
|
||||
${python3:Depends},
|
||||
Recommends:
|
||||
cyclopts, # Not currently in Debian
|
||||
jbig2
|
||||
paddleocr, # Not currently in Debian
|
||||
pngquant,
|
||||
pypdfium2, # Not currently in Debian
|
||||
unpaper,
|
||||
verapdf, # Not currently in Debian
|
||||
Suggests:
|
||||
ocrmypdf-doc,
|
||||
python-watchdog,
|
||||
```
|
||||
|
||||
### Command line completions
|
||||
|
||||
|
||||
+141
-1
@@ -184,12 +184,57 @@ Plugin options can be accessed in two ways:
|
||||
Both access patterns are equivalent and return the same values.
|
||||
|
||||
:::{note}
|
||||
**Plugin Interface Change**: Starting in OCRmyPDF v16.13.0, plugin hooks receive
|
||||
**Plugin Interface Change**: Starting in OCRmyPDF v17.0.0, plugin hooks receive
|
||||
`OcrOptions` objects instead of `argparse.Namespace` objects. Most plugins will
|
||||
continue working due to duck-typing compatibility, but plugin developers should
|
||||
update their type hints accordingly.
|
||||
:::
|
||||
|
||||
### Migration guide for plugin developers
|
||||
|
||||
:::{versionadded} 17.0.0
|
||||
:::
|
||||
|
||||
**Update imports:**
|
||||
|
||||
```python
|
||||
from ocrmypdf._options import OcrOptions
|
||||
```
|
||||
|
||||
**Update type hints:**
|
||||
|
||||
```python
|
||||
# Before (v16 and earlier)
|
||||
def check_options(options: argparse.Namespace) -> None:
|
||||
...
|
||||
|
||||
# After (v17+)
|
||||
def check_options(options: OcrOptions) -> None:
|
||||
...
|
||||
```
|
||||
|
||||
**Attribute access unchanged:**
|
||||
|
||||
```python
|
||||
# These work exactly as before
|
||||
options.languages
|
||||
options.output_type
|
||||
options.tesseract_timeout
|
||||
```
|
||||
|
||||
**Remove in-place modifications:**
|
||||
|
||||
```python
|
||||
# Before (v16 pattern - no longer recommended)
|
||||
def check_options(options):
|
||||
options.some_computed_value = compute_value(options)
|
||||
|
||||
# After (v17 pattern - compute at point of use)
|
||||
def some_function(options):
|
||||
computed = compute_value(options)
|
||||
use_computed(computed)
|
||||
```
|
||||
|
||||
### Execution and progress reporting
|
||||
|
||||
```{eval-rst}
|
||||
@@ -274,3 +319,98 @@ update their type hints accordingly.
|
||||
```{eval-rst}
|
||||
.. autofunction:: ocrmypdf.pluginspec.is_optimization_enabled
|
||||
```
|
||||
|
||||
### Working with OcrElement trees
|
||||
|
||||
:::{versionadded} 17.0.0
|
||||
:::
|
||||
|
||||
OCRmyPDF v17 introduces the `OcrElement` dataclass for representing OCR
|
||||
output in an engine-agnostic format. This enables plugins to work with
|
||||
OCR results without parsing hOCR XML.
|
||||
|
||||
**Key classes:**
|
||||
|
||||
```python
|
||||
from ocrmypdf import OcrElement, OcrClass, BoundingBox
|
||||
|
||||
# OcrElement - represents any OCR structural unit
|
||||
page = OcrElement(
|
||||
ocr_class=OcrClass.PAGE,
|
||||
bbox=BoundingBox(0, 0, 612, 792),
|
||||
children=[...]
|
||||
)
|
||||
|
||||
# BoundingBox - axis-aligned bounding box (left, top, right, bottom)
|
||||
bbox = BoundingBox(left=100, top=50, right=300, bottom=80)
|
||||
|
||||
# OcrClass - constants for element types
|
||||
OcrClass.PAGE # "ocr_page"
|
||||
OcrClass.LINE # "ocr_line"
|
||||
OcrClass.WORD # "ocrx_word"
|
||||
OcrClass.PARAGRAPH # "ocr_par"
|
||||
```
|
||||
|
||||
**Navigating the tree:**
|
||||
|
||||
```python
|
||||
# Get all words in a page
|
||||
words = page.words # Returns list[OcrElement]
|
||||
|
||||
# Get all lines
|
||||
lines = page.lines
|
||||
|
||||
# Get combined text
|
||||
text = page.get_text_recursive()
|
||||
|
||||
# Iterate by class
|
||||
for para in page.paragraphs:
|
||||
print(para.get_text_recursive())
|
||||
```
|
||||
|
||||
**OCR engine plugins:**
|
||||
|
||||
Plugins implementing custom OCR engines can now output `OcrElement` trees
|
||||
directly via the `generate_ocr()` method, bypassing hOCR entirely:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from ocrmypdf.pluginspec import OcrEngine
|
||||
from ocrmypdf import OcrElement, OcrClass, BoundingBox
|
||||
|
||||
class MyOcrEngine(OcrEngine):
|
||||
def generate_ocr(
|
||||
self,
|
||||
input_file: Path,
|
||||
options,
|
||||
context,
|
||||
) -> OcrElement:
|
||||
# Perform OCR and return OcrElement tree directly
|
||||
# No need to generate hOCR XML
|
||||
return OcrElement(
|
||||
ocr_class=OcrClass.PAGE,
|
||||
bbox=BoundingBox(0, 0, width, height),
|
||||
dpi=300,
|
||||
children=[
|
||||
OcrElement(
|
||||
ocr_class=OcrClass.LINE,
|
||||
bbox=BoundingBox(100, 50, 500, 80),
|
||||
children=[
|
||||
OcrElement(
|
||||
ocr_class=OcrClass.WORD,
|
||||
bbox=BoundingBox(100, 50, 200, 80),
|
||||
text="Hello",
|
||||
),
|
||||
# ... more words
|
||||
]
|
||||
),
|
||||
# ... more lines
|
||||
]
|
||||
)
|
||||
|
||||
def supports_generate_ocr(self) -> bool:
|
||||
return True # Indicate this engine uses generate_ocr()
|
||||
```
|
||||
|
||||
This approach is simpler than generating hOCR and allows modern OCR
|
||||
engines to integrate more naturally with OCRmyPDF.
|
||||
|
||||
+11
-37
@@ -38,6 +38,10 @@ official when it's tagged and posted to PyPI.
|
||||
- **Lossy JBIG2 removed**: The `--jbig2-lossy` and `--jbig2-page-group-size` options have been
|
||||
removed due to well-documented risks of character substitution errors. These options are now
|
||||
deprecated and will emit warnings if used. Only lossless JBIG2 compression is supported.
|
||||
- **PDF/A output behavior change**: If neither Ghostscript nor verapdf is installed,
|
||||
`--output-type auto` (the new default) will produce a standard PDF instead of PDF/A. This is
|
||||
a change from previous versions where Ghostscript was required and PDF/A was always produced.
|
||||
This configuration is rare but users should be aware of the change.
|
||||
|
||||
**New features**
|
||||
|
||||
@@ -57,9 +61,14 @@ official when it's tagged and posted to PyPI.
|
||||
- **verapdf integration**: Added optional verapdf validation for fast PDF/A conversion. When
|
||||
available, OCRmyPDF attempts speculative PDF/A conversion using pikepdf, validates with verapdf,
|
||||
and skips Ghostscript if validation passes.
|
||||
- **Optional Ghostscript**: As a consequence of the changes above, Ghostscript is no longer a require dependency. It is optional.
|
||||
- **Optional Ghostscript**: As a consequence of the changes above, Ghostscript is no longer a required dependency. It is optional.
|
||||
- **fpdf2 text renderer**: Replaced legacy hOCR text renderer with new fpdf2-based implementation,
|
||||
providing better multilingual support and more accurate text positioning.
|
||||
- **Improved Occulta glyphless font**: The new Occulta font provides better handling of
|
||||
zero-width markers and double-width CJK characters for accurate text layer positioning.
|
||||
- **Expanded multilingual font support**: Added FontProvider infrastructure with language-aware
|
||||
font selection for Devanagari (Hindi, Sanskrit, Marathi, Nepali), CJK (Chinese, Japanese,
|
||||
Korean), Arabic script, and many other scripts. System font discovery reduces package size.
|
||||
- **Simplified mode selection**: New `--mode` (`-m`) argument consolidates processing options:
|
||||
- `default`: Error if text is found (standard behavior)
|
||||
- `force`: Rasterize all content and run OCR (replaces `--force-ocr`)
|
||||
@@ -101,42 +110,7 @@ official when it's tagged and posted to PyPI.
|
||||
- Optional: `verapdf` for fast PDF/A validation (new dependency)
|
||||
- Requires: `fpdf2` for text layer rendering (new dependency)
|
||||
- Recommended: `typer` with `cyclopts` in misc scripts (new dependency)
|
||||
|
||||
Summarizing, in Debian "control" style, our runtime dependency spec would look like:
|
||||
|
||||
```
|
||||
Depends:
|
||||
fonts-noto,
|
||||
fpdf2 (>= 2.8),
|
||||
ghostscript (>= 9.18~dfsg~), # Not strictly required, but best user experience
|
||||
icc-profiles-free,
|
||||
img2pdf,
|
||||
python3-coloredlogs,
|
||||
python3-deprecation,
|
||||
python3-hypothesis,
|
||||
python3-pdfminer (>= 20181108+dfsg-3),
|
||||
python3-pikepdf (>= 8.14.0),
|
||||
python3-pil,
|
||||
python3-pluggy,
|
||||
python3-reportlab,
|
||||
python3-rich,
|
||||
python3-uharfbuzz, # Not currently in Debian
|
||||
tesseract-ocr (>= 4.0.0),
|
||||
zlib1g,
|
||||
${misc:Depends},
|
||||
${python3:Depends},
|
||||
Recommends:
|
||||
cyclopts, # Not currently in Debian
|
||||
jbig2
|
||||
paddleocr, # Not currently in Debian
|
||||
pngquant,
|
||||
pypdfium2, # Not currently in Debian
|
||||
unpaper,
|
||||
verapdf, # Not currently in Debian
|
||||
Suggests:
|
||||
ocrmypdf-doc,
|
||||
python-watchdog,
|
||||
```
|
||||
- See docs/maintainers.md for details.
|
||||
|
||||
**Migration guide for plugin developers**
|
||||
|
||||
|
||||
+1
-1
@@ -171,5 +171,5 @@ test = [
|
||||
# Extended test capabilities (merged from extended_test)
|
||||
"pymupdf>=1.24.14",
|
||||
]
|
||||
docs = ["myst-parser>=4.0.1", "sphinx", "sphinx-issues", "sphinx-rtd-theme"]
|
||||
docs = ["myst-parser>=4.0.1", "sphinx", "sphinx-issues", "sphinx-rtd-theme", "sphinxcontrib-mermaid"]
|
||||
streamlit-dev = ["streamlit>=1.40.2", "streamlit-pdf-viewer>=0.0.19"]
|
||||
|
||||
@@ -1460,6 +1460,7 @@ docs = [
|
||||
{ name = "sphinx", version = "8.2.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
|
||||
{ name = "sphinx-issues" },
|
||||
{ name = "sphinx-rtd-theme" },
|
||||
{ name = "sphinxcontrib-mermaid" },
|
||||
]
|
||||
streamlit-dev = [
|
||||
{ name = "streamlit" },
|
||||
@@ -1510,6 +1511,7 @@ docs = [
|
||||
{ name = "sphinx" },
|
||||
{ name = "sphinx-issues" },
|
||||
{ name = "sphinx-rtd-theme" },
|
||||
{ name = "sphinxcontrib-mermaid" },
|
||||
]
|
||||
streamlit-dev = [
|
||||
{ name = "streamlit", specifier = ">=1.40.2" },
|
||||
@@ -2775,6 +2777,21 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c2/42/4c8646762ee83602e3fb3fbe774c2fac12f317deb0b5dbeeedd2d3ba4b77/sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl", hash = "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178", size = 5071, upload-time = "2019-01-21T16:10:14.333Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sphinxcontrib-mermaid"
|
||||
version = "2.0.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "jinja2" },
|
||||
{ name = "pyyaml" },
|
||||
{ name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
|
||||
{ name = "sphinx", version = "8.2.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/96/a5/65a5c439cc14ba80483b9891e9350f11efb80cd3bdccb222f0c738068c78/sphinxcontrib_mermaid-2.0.0.tar.gz", hash = "sha256:cf4f7d453d001132eaba5d1fdf53d42049f02e913213cf8337427483bfca26f4", size = 18194, upload-time = "2026-01-13T17:13:42.563Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/f9/de/bd96c69b62e967bffd02c6d89dfca9471b04e761c466725fc39746abf41d/sphinxcontrib_mermaid-2.0.0-py3-none-any.whl", hash = "sha256:59a73249bbee2c74b1a4db036f8e8899ade65982bdda6712cf22b4f4e9874bb5", size = 14055, upload-time = "2026-01-13T17:13:41.481Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sphinxcontrib-qthelp"
|
||||
version = "2.0.0"
|
||||
|
||||
Reference in New Issue
Block a user