rst -> md migration in progress
This commit is contained in:
James R. Barlow
+248
@@ -0,0 +1,248 @@
|
||||
% SPDX-FileCopyrightText: 2022 James R. Barlow
|
||||
% SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
|
||||
Batch processing
|
||||
================
|
||||
|
||||
This article provides information about running OCRmyPDF on multiple
|
||||
files or configuring it as a service triggered by file system events.
|
||||
|
||||
Batch jobs
|
||||
----------
|
||||
|
||||
Consider using the excellent [GNU
|
||||
Parallel](https://www.gnu.org/software/parallel/) to apply OCRmyPDF to
|
||||
multiple files at once.
|
||||
|
||||
Both `parallel` and `ocrmypdf` will try to use all available processors.
|
||||
To maximize parallelism without overloading your system with processes,
|
||||
consider using `parallel -j 2` to limit parallel to running two jobs at
|
||||
once.
|
||||
|
||||
This command will run `ocrmypdf` on all files named `*.pdf` in the
|
||||
current directory and write them to the previously created `output/`
|
||||
folder. It will not search subdirectories.
|
||||
|
||||
The `--tag` argument tells parallel to print the filename as a prefix
|
||||
whenever a message is printed, so that one can trace any errors to the
|
||||
file that produced them.
|
||||
|
||||
:::{code} bash
|
||||
parallel --tag -j 2 ocrmypdf '{}' 'output/{}' ::: *.pdf
|
||||
:::
|
||||
|
||||
OCRmyPDF automatically repairs PDFs before parsing and gathering
|
||||
information from them.
|
||||
|
||||
Directory trees
|
||||
---------------
|
||||
|
||||
This will walk through a directory tree and run OCR on all files in
|
||||
place, and printing each filename in between runs:
|
||||
|
||||
:::{code} bash
|
||||
find . -name '*.pdf' -printf '%p\n' -exec ocrmypdf '{}' '{}' \;
|
||||
:::
|
||||
|
||||
This only runs one `ocrmypdf` process at a time. This variation uses
|
||||
`find` to create a directory list and `parallel` to parallelize runs of
|
||||
`ocrmypdf`, again updating files in place.
|
||||
|
||||
:::{code} bash
|
||||
find . -name '*.pdf' | parallel --tag -j 2 ocrmypdf '{}' '{}'
|
||||
:::
|
||||
|
||||
In a Windows batch file, use
|
||||
|
||||
:::{code} bat
|
||||
for /r %%f in (*.pdf) do ocrmypdf %%f %%f
|
||||
:::
|
||||
|
||||
With a Docker container, you will need to stream through standard input
|
||||
and output:
|
||||
|
||||
:::{code} bash
|
||||
find . -name '*.pdf' -print0 | xargs -0 | while read pdf; do
|
||||
pdfout=$(mktemp)
|
||||
docker run --rm -i jbarlow83/ocrmypdf - - <$pdf >$pdfout && cp $pdfout $pdf
|
||||
done
|
||||
:::
|
||||
|
||||
### Sample script
|
||||
|
||||
This user contributed script also provides an example of batch
|
||||
processing.
|
||||
|
||||
:::{literalinclude} ../misc/batch.py
|
||||
---
|
||||
caption: misc/batch.py
|
||||
---
|
||||
:::
|
||||
|
||||
### Synology DiskStations
|
||||
|
||||
Synology DiskStations (Network Attached Storage devices) can run the
|
||||
Docker image of OCRmyPDF if the Synology [Docker
|
||||
package](https://www.synology.com/en-global/dsm/packages/Docker) is
|
||||
installed. Attached is a script to address particular quirks of using
|
||||
OCRmyPDF on one of these devices.
|
||||
|
||||
At the time this script was written, it only worked for x86-based
|
||||
Synology products. It is not known if it will work on ARM-based Synology
|
||||
products. Further adjustments might be needed to deal with the
|
||||
Synology\'s relatively limited CPU and RAM.
|
||||
|
||||
:::{literalinclude} ../misc/synology.py
|
||||
---
|
||||
caption: misc/synology.py - Sample script for Synology DiskStations
|
||||
---
|
||||
:::
|
||||
|
||||
### Huge batch jobs
|
||||
|
||||
If you have thousands of files to work with, contact the author.
|
||||
Consulting work related to OCRmyPDF helps fund this open source project
|
||||
and all inquiries are appreciated.
|
||||
|
||||
Hot (watched) folders
|
||||
---------------------
|
||||
|
||||
### Watched folders with watcher.py
|
||||
|
||||
OCRmyPDF has a folder watcher called watcher.py, which is currently
|
||||
included in source distributions but not part of the main program. It
|
||||
may be used natively or may run in a Docker container. Native instances
|
||||
tend to give better performance. watcher.py works on all platforms.
|
||||
|
||||
Users may need to customize the script to meet their requirements.
|
||||
|
||||
:::{code} bash
|
||||
pip3 install ocrmypdf[watcher]
|
||||
|
||||
env OCR_INPUT_DIRECTORY=/mnt/input-pdfs \
|
||||
OCR_OUTPUT_DIRECTORY=/mnt/output-pdfs \
|
||||
OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1 \
|
||||
python3 watcher.py
|
||||
:::
|
||||
|
||||
:::{list-table} watcher.py environment variables
|
||||
---
|
||||
header-rows: 1
|
||||
---
|
||||
|
||||
* - Environment variable
|
||||
- Description
|
||||
* - OCR\_INPUT\_DIRECTORY
|
||||
- Set input directory to monitor (recursive)
|
||||
* - OCR\_OUTPUT\_DIRECTORY
|
||||
- Set output directory (should not be under input)
|
||||
* - OCR\_ARCHIVE\_DIRECTORY
|
||||
- Set archive directory for processed originals (should not be under input, requires `OCR_ON_SUCCESS_ARCHIVE` to be set)
|
||||
* - OCR\_ON\_SUCCESS\_DELETE
|
||||
- This will move the processed original file to `OCR_ARCHIVE_DIRECTORY` if the exit code is 0 (OK). Note that `OCR_ON_SUCCESS_DELETE` takes precedence over this option, i.e. if both options are set, the input file will be deleted.
|
||||
* - OCR\_OUTPUT\_DIRECTORY\_YEAR\_MONTH
|
||||
- This will place files in the output in `{output}/{year}/{month}/{filename}`
|
||||
* - OCR\_DESKEW
|
||||
- Apply deskew to crooked input PDFs
|
||||
* - OCR\_JSON\_SETTINGS
|
||||
- A JSON string specifying any other arguments for `ocrmypdf.ocr`, e.g. `'OCR_JSON_SETTINGS={"rotate_pages": true, "optimize": "3"}'`.
|
||||
* - OCR\_POLL\_NEW\_FILE\_SECONDS
|
||||
- Polling interval
|
||||
* - OCR\_LOGLEVEL
|
||||
- Level of log messages t
|
||||
:::
|
||||
|
||||
One could configure a networked scanner or scanning computer to drop
|
||||
files in the watched folder.
|
||||
|
||||
### Watched folders with Docker
|
||||
|
||||
The watcher service is included in the OCRmyPDF Docker image. To run it:
|
||||
|
||||
:::{code} bash
|
||||
docker run \
|
||||
--volume <path to files to convert>:/input \
|
||||
--volume <path to store results>:/output \
|
||||
--volume <path to store processed originals>:/processed \
|
||||
--env OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1 \
|
||||
--env OCR_ON_SUCCESS_ARCHIVE=1 \
|
||||
--env OCR_DESKEW=1 \
|
||||
--env PYTHONUNBUFFERED=1 \
|
||||
--interactive --tty --entrypoint python3 \
|
||||
jbarlow83/ocrmypdf \
|
||||
watcher.py
|
||||
:::
|
||||
|
||||
This service will watch for a file that matches `/input/\*.pdf`, convert
|
||||
it to a OCRed PDF in `/output/`, and move the processed original to
|
||||
`/processed`. The parameters to this image are:
|
||||
|
||||
:::{list-table} Watcher Docker Parameters
|
||||
:header-rows: 1
|
||||
|
||||
* - Parameter
|
||||
- Description
|
||||
* - `--volume <path to files to convert>:/input`
|
||||
- Files placed in this location will be OCRed
|
||||
* - `--volume <path to store results>:/output`
|
||||
- This is where OCRed files will be stored
|
||||
* - `--volume <path to store processed originals>:/processed`
|
||||
- Archive processed originals here
|
||||
* - `--env OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1`
|
||||
- Define environment variable `OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1` to place files in the output in `{output}/{year}/{month}/{filename}`
|
||||
* - `--env OCR_ON_SUCCESS_ARCHIVE=1`
|
||||
- Define environment variable `OCR_ON_SUCCESS_ARCHIVE` to move processed originals
|
||||
* - `--env OCR_DESKEW=1`
|
||||
- Define environment variable `OCR_DESKEW` to apply deskew to crooked input PDFs
|
||||
* - `--env PYTHONBUFFERED=1`
|
||||
- This will force `STDOUT` to be unbuffered and allow you to see messages in docker logs
|
||||
:::
|
||||
|
||||
This service relies on polling to check for changes to the filesystem.
|
||||
It may not be suitable for some environments, such as filesystems shared
|
||||
on a slow network.
|
||||
|
||||
A configuration manager such as Docker Compose could be used to ensure
|
||||
that the service is always available.
|
||||
|
||||
:::{literalinclude} ../misc/docker-compose.example.yml
|
||||
---
|
||||
caption: misc/docker-compose.example.yml
|
||||
---
|
||||
:::
|
||||
|
||||
### Caveats
|
||||
|
||||
- `watchmedo` may not work properly on a networked file system,
|
||||
depending on the capabilities of the file system client and server.
|
||||
- This simple recipe does not filter for the type of file system
|
||||
event, so file copies, deletes and moves, and directory operations,
|
||||
will all be sent to ocrmypdf, producing errors in several cases.
|
||||
Disable your watched folder if you are doing anything other than
|
||||
copying files to it.
|
||||
- If the source and destination directory are the same, watchmedo may
|
||||
create an infinite loop.
|
||||
- On BSD, FreeBSD and older versions of macOS, you may need to
|
||||
increase the number of file descriptors to monitor more files, using
|
||||
`ulimit -n 1024` to watch a folder of up to 1024 files.
|
||||
|
||||
### Alternatives
|
||||
|
||||
- On Linux, [systemd user
|
||||
services](https://wiki.archlinux.org/index.php/Systemd/User) can be
|
||||
configured to automatically perform OCR on a collection of files.
|
||||
- [Watchman](https://facebook.github.io/watchman/) is a more powerful
|
||||
alternative to `watchmedo`.
|
||||
|
||||
macOS Automator
|
||||
---------------
|
||||
|
||||
You can use the Automator app with macOS, to create a Workflow or Quick
|
||||
Action. Use a *Run Shell Script* action in your workflow. In the context
|
||||
of Automator, the `PATH` may be set differently your Terminal\'s `PATH`;
|
||||
you may need to explicitly set the PATH to include `ocrmypdf`. The
|
||||
following example may serve as a starting point:
|
||||
|
||||
|
||||
|
||||
You may customize the command sent to ocrmypdf.
|
||||
-228
@@ -1,228 +0,0 @@
|
||||
.. SPDX-FileCopyrightText: 2022 James R. Barlow
|
||||
..
|
||||
.. SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
|
||||
================
|
||||
Batch processing
|
||||
================
|
||||
|
||||
This article provides information about running OCRmyPDF on multiple
|
||||
files or configuring it as a service triggered by file system events.
|
||||
|
||||
Batch jobs
|
||||
==========
|
||||
|
||||
Consider using the excellent `GNU
|
||||
Parallel <https://www.gnu.org/software/parallel/>`__ to apply OCRmyPDF
|
||||
to multiple files at once.
|
||||
|
||||
Both ``parallel`` and ``ocrmypdf`` will try to use all available
|
||||
processors. To maximize parallelism without overloading your system with
|
||||
processes, consider using ``parallel -j 2`` to limit parallel to running
|
||||
two jobs at once.
|
||||
|
||||
This command will run ``ocrmypdf`` on all files named ``*.pdf`` in the
|
||||
current directory and write them to the previously created ``output/``
|
||||
folder. It will not search subdirectories.
|
||||
|
||||
The ``--tag`` argument tells parallel to print the filename as a prefix
|
||||
whenever a message is printed, so that one can trace any errors to the
|
||||
file that produced them.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
parallel --tag -j 2 ocrmypdf '{}' 'output/{}' ::: *.pdf
|
||||
|
||||
OCRmyPDF automatically repairs PDFs before parsing and gathering
|
||||
information from them.
|
||||
|
||||
Directory trees
|
||||
===============
|
||||
|
||||
This will walk through a directory tree and run OCR on all files in
|
||||
place, and printing each filename in between runs:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
find . -name '*.pdf' -printf '%p\n' -exec ocrmypdf '{}' '{}' \;
|
||||
|
||||
This only runs one ``ocrmypdf`` process at a time. This variation uses
|
||||
``find`` to create a directory list and ``parallel`` to parallelize runs
|
||||
of ``ocrmypdf``, again updating files in place.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
find . -name '*.pdf' | parallel --tag -j 2 ocrmypdf '{}' '{}'
|
||||
|
||||
In a Windows batch file, use
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
for /r %%f in (*.pdf) do ocrmypdf %%f %%f
|
||||
|
||||
With a Docker container, you will need to stream through standard input and output:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
find . -name '*.pdf' -print0 | xargs -0 | while read pdf; do
|
||||
pdfout=$(mktemp)
|
||||
docker run --rm -i jbarlow83/ocrmypdf - - <$pdf >$pdfout && cp $pdfout $pdf
|
||||
done
|
||||
|
||||
Sample script
|
||||
-------------
|
||||
|
||||
This user contributed script also provides an example of batch
|
||||
processing.
|
||||
|
||||
.. literalinclude:: ../misc/batch.py
|
||||
:caption: misc/batch.py
|
||||
|
||||
Synology DiskStations
|
||||
---------------------
|
||||
|
||||
Synology DiskStations (Network Attached Storage devices) can run the
|
||||
Docker image of OCRmyPDF if the Synology `Docker
|
||||
package <https://www.synology.com/en-global/dsm/packages/Docker>`__ is
|
||||
installed. Attached is a script to address particular quirks of using
|
||||
OCRmyPDF on one of these devices.
|
||||
|
||||
At the time this script was written, it only worked for x86-based Synology
|
||||
products. It is not known if it will work on ARM-based Synology products.
|
||||
Further adjustments might be needed to deal with the Synology's relatively
|
||||
limited CPU and RAM.
|
||||
|
||||
.. literalinclude:: ../misc/synology.py
|
||||
:caption: misc/synology.py - Sample script for Synology DiskStations
|
||||
|
||||
Huge batch jobs
|
||||
---------------
|
||||
|
||||
If you have thousands of files to work with, contact the author.
|
||||
Consulting work related to OCRmyPDF helps fund this open source project
|
||||
and all inquiries are appreciated.
|
||||
|
||||
Hot (watched) folders
|
||||
=====================
|
||||
|
||||
Watched folders with watcher.py
|
||||
-------------------------------
|
||||
|
||||
OCRmyPDF has a folder watcher called watcher.py, which is currently included in source
|
||||
distributions but not part of the main program. It may be used natively or may run
|
||||
in a Docker container. Native instances tend to give better performance. watcher.py
|
||||
works on all platforms.
|
||||
|
||||
Users may need to customize the script to meet their requirements.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip3 install ocrmypdf[watcher]
|
||||
|
||||
env OCR_INPUT_DIRECTORY=/mnt/input-pdfs \
|
||||
OCR_OUTPUT_DIRECTORY=/mnt/output-pdfs \
|
||||
OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1 \
|
||||
python3 watcher.py
|
||||
|
||||
.. csv-table:: watcher.py environment variables
|
||||
:header: "Environment variable", "Description"
|
||||
:widths: 50, 50
|
||||
|
||||
"OCR_INPUT_DIRECTORY", "Set input directory to monitor (recursive)"
|
||||
"OCR_OUTPUT_DIRECTORY", "Set output directory (should not be under input)"
|
||||
"OCR_ARCHIVE_DIRECTORY", "Set archive directory for processed originals (should not be under input, requires ``OCR_ON_SUCCESS_ARCHIVE`` to be set)"
|
||||
"OCR_ON_SUCCESS_DELETE", "This will delete the input file if the exit code is 0 (OK)"
|
||||
"OCR_ON_SUCCESS_ARCHIVE", "This will move the processed original file to ``OCR_ARCHIVE_DIRECTORY`` if the exit code is 0 (OK). Note that ``OCR_ON_SUCCESS_DELETE`` takes precedence over this option, i.e. if both options are set, the input file will be deleted."
|
||||
"OCR_OUTPUT_DIRECTORY_YEAR_MONTH", "This will place files in the output in ``{output}/{year}/{month}/{filename}``"
|
||||
"OCR_DESKEW", "Apply deskew to crooked input PDFs"
|
||||
"OCR_JSON_SETTINGS", "A JSON string specifying any other arguments for ``ocrmypdf.ocr``, e.g. ``'OCR_JSON_SETTINGS={""rotate_pages"": true, ""optimize"": ""3""}'``."
|
||||
"OCR_POLL_NEW_FILE_SECONDS", "Polling interval"
|
||||
"OCR_LOGLEVEL", "Level of log messages to report"
|
||||
|
||||
One could configure a networked scanner or scanning computer to drop files in the
|
||||
watched folder.
|
||||
|
||||
Watched folders with Docker
|
||||
---------------------------
|
||||
|
||||
The watcher service is included in the OCRmyPDF Docker image. To run it:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run \
|
||||
--volume <path to files to convert>:/input \
|
||||
--volume <path to store results>:/output \
|
||||
--volume <path to store processed originals>:/processed \
|
||||
--env OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1 \
|
||||
--env OCR_ON_SUCCESS_ARCHIVE=1 \
|
||||
--env OCR_DESKEW=1 \
|
||||
--env PYTHONUNBUFFERED=1 \
|
||||
--interactive --tty --entrypoint python3 \
|
||||
jbarlow83/ocrmypdf \
|
||||
watcher.py
|
||||
|
||||
This service will watch for a file that matches ``/input/\*.pdf``,
|
||||
convert it to a OCRed PDF in ``/output/``, and move the processed
|
||||
original to ``/processed``. The parameters to this image are:
|
||||
|
||||
.. csv-table:: watcher.py parameters for Docker
|
||||
:header: "Parameter", "Description"
|
||||
:widths: 50, 50
|
||||
|
||||
"``--volume <path to files to convert>:/input``", "Files placed in this location will be OCRed"
|
||||
"``--volume <path to store results>:/output``", "This is where OCRed files will be stored"
|
||||
"``--volume <path to store processed originals>:/processed``", "Archive processed originals here"
|
||||
"``--env OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1``", "Define environment variable ``OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1`` to place files in the output in ``{output}/{year}/{month}/{filename}``"
|
||||
"``--env OCR_ON_SUCCESS_ARCHIVE=1``", "Define environment variable ``OCR_ON_SUCCESS_ARCHIVE`` to move processed originals"
|
||||
"``--env OCR_DESKEW=1``", "Define environment variable ``OCR_DESKEW`` to apply deskew to crooked input PDFs"
|
||||
"``--env PYTHONBUFFERED=1``", "This will force ``STDOUT`` to be unbuffered and allow you to see messages in docker logs"
|
||||
|
||||
This service relies on polling to check for changes to the filesystem. It
|
||||
may not be suitable for some environments, such as filesystems shared on a
|
||||
slow network.
|
||||
|
||||
A configuration manager such as Docker Compose could be used to ensure that the
|
||||
service is always available.
|
||||
|
||||
.. literalinclude:: ../misc/docker-compose.example.yml
|
||||
:language: yaml
|
||||
:caption: misc/docker-compose.example.yml
|
||||
|
||||
Caveats
|
||||
-------
|
||||
|
||||
- ``watchmedo`` may not work properly on a networked file system,
|
||||
depending on the capabilities of the file system client and server.
|
||||
- This simple recipe does not filter for the type of file system event,
|
||||
so file copies, deletes and moves, and directory operations, will all
|
||||
be sent to ocrmypdf, producing errors in several cases. Disable your
|
||||
watched folder if you are doing anything other than copying files to
|
||||
it.
|
||||
- If the source and destination directory are the same, watchmedo may
|
||||
create an infinite loop.
|
||||
- On BSD, FreeBSD and older versions of macOS, you may need to increase
|
||||
the number of file descriptors to monitor more files, using
|
||||
``ulimit -n 1024`` to watch a folder of up to 1024 files.
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
- On Linux, `systemd user services <https://wiki.archlinux.org/index.php/Systemd/User>`__
|
||||
can be configured to automatically perform OCR on a collection of files.
|
||||
|
||||
- `Watchman <https://facebook.github.io/watchman/>`__ is a more
|
||||
powerful alternative to ``watchmedo``.
|
||||
|
||||
macOS Automator
|
||||
===============
|
||||
|
||||
You can use the Automator app with macOS, to create a Workflow or Quick
|
||||
Action. Use a *Run Shell Script* action in your workflow. In the context
|
||||
of Automator, the ``PATH`` may be set differently your Terminal's
|
||||
``PATH``; you may need to explicitly set the PATH to include
|
||||
``ocrmypdf``. The following example may serve as a starting point:
|
||||
|
||||
.. figure:: images/macos-workflow.png
|
||||
:alt: Example macOS Automator workflow
|
||||
|
||||
You may customize the command sent to ocrmypdf.
|
||||
+4
-1
@@ -36,6 +36,7 @@ import datetime
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = [
|
||||
'myst_parser',
|
||||
'sphinx.ext.autodoc',
|
||||
'sphinx.ext.intersphinx',
|
||||
'sphinx.ext.autosummary',
|
||||
@@ -44,6 +45,8 @@ extensions = [
|
||||
'sphinx_issues',
|
||||
]
|
||||
|
||||
myst_enable_extensions = ['colon_fence', 'attrs_block', 'attrs_inline']
|
||||
|
||||
# Extension settings
|
||||
intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
|
||||
napoleon_use_rtype = False
|
||||
@@ -53,7 +56,7 @@ issues_github_path = "ocrmypdf/OCRmyPDF"
|
||||
templates_path = ['_templates']
|
||||
|
||||
# The suffix(es) of source filenames.
|
||||
source_suffix = {'.rst': 'restructuredtext', '.md': 'markdown'}
|
||||
source_suffix = {'.rst': 'restructuredtext', '.md': 'markdown', '.txt': 'markdown'}
|
||||
|
||||
# The encoding of source files.
|
||||
#
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
% SPDX-FileCopyrightText: 2025 James R. Barlow
|
||||
% SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
|
||||
# Contributing guidelines
|
||||
|
||||
Contributions are welcome!
|
||||
|
||||
## Big changes
|
||||
|
||||
Please open a new issue to discuss or propose a major change. Not only
|
||||
is it fun to discuss big ideas, but we might save each other\'s time
|
||||
too. Perhaps some of the work you\'re contemplating is already half-done
|
||||
in a development branch.
|
||||
|
||||
## Code style
|
||||
|
||||
We use `ruff` for code formatting.
|
||||
The settings for these programs are in `pyproject.toml`. Pull requests
|
||||
should follow the style guide. One difference we use from \"black\"
|
||||
style is that strings shown to the user are always in double quotes
|
||||
(`"`) and strings for internal uses are in single quotes (`'`).
|
||||
|
||||
## Tests
|
||||
|
||||
New features should come with tests that confirm their correctness.
|
||||
|
||||
## New dependencies
|
||||
|
||||
If you are proposing a change that will require a new dependency, we
|
||||
prefer dependencies that are already packaged by Debian or Red Hat. This
|
||||
makes life much easier for our downstream package maintainers. A package
|
||||
that is only available on PyPI or GitHub, and not more widely packaged,
|
||||
may not be accepted.
|
||||
|
||||
We are unlikely to accept a dependency on CUDA or other GPU-based
|
||||
libraries, because these are still difficult to package and install on
|
||||
many systems. We recommend implementing these changes as plugins.
|
||||
|
||||
Python dependencies must also be license-compatible. GPLv3 or AGPLv3 are
|
||||
likely incompatible with the project\'s license, but LGPLv3 is
|
||||
compatible.
|
||||
|
||||
## New non-Python dependencies
|
||||
|
||||
OCRmyPDF uses several external programs (Tesseract, Ghostscript and
|
||||
others) for its functionality. In general we prefer to avoid adding new
|
||||
external programs, and if we are to add external programs, we prefer
|
||||
those that are already packaged by Debian or Red Hat.
|
||||
|
||||
## Plugins
|
||||
|
||||
Some new features may be a good fit for a plugin. Plugins are a way to
|
||||
add features to OCRmyPDF without adding them to the core program.
|
||||
Plugins are installed separately from OCRmyPDF. They are written in
|
||||
Python and can be installed from PyPI. See the [plugin
|
||||
documentation](https://ocrmypdf.readthedocs.io/en/latest/plugins.html).
|
||||
|
||||
We are happy to link users to your plugin from the documentation.
|
||||
|
||||
## Style guide: Is it OCRmyPDF or ocrmypdf?
|
||||
|
||||
The program/project is OCRmyPDF and the name of the executable or
|
||||
library is ocrmypdf.
|
||||
|
||||
## Copyright and license
|
||||
|
||||
For contributions over 10 lines of code, please add your name to list of
|
||||
copyright holders for that file. The core program is licensed under
|
||||
MPL-2.0, test files and documentation under CC-BY-SA 4.0, and
|
||||
miscellaneous files under MIT, with a few minor exceptions. Please
|
||||
contribute only content that you own or have the right to contribute
|
||||
under these licenses.
|
||||
@@ -1,77 +0,0 @@
|
||||
.. SPDX-FileCopyrightText: 2022 James R. Barlow
|
||||
..
|
||||
.. SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
|
||||
=======================
|
||||
Contributing guidelines
|
||||
=======================
|
||||
|
||||
Contributions are welcome!
|
||||
|
||||
Big changes
|
||||
===========
|
||||
|
||||
Please open a new issue to discuss or propose a major change. Not only is it fun
|
||||
to discuss big ideas, but we might save each other's time too. Perhaps some of the
|
||||
work you're contemplating is already half-done in a development branch.
|
||||
|
||||
Code style
|
||||
==========
|
||||
|
||||
We use PEP8, ``black`` for code formatting and ``ruff`` for everything else. The
|
||||
settings for these programs are in ``pyproject.toml``. Pull
|
||||
requests should follow the style guide. One difference we use from "black" style
|
||||
is that strings shown to the user are always in double quotes (``"``) and strings
|
||||
for internal uses are in single quotes (``'``).
|
||||
|
||||
Tests
|
||||
=====
|
||||
|
||||
New features should come with tests that confirm their correctness.
|
||||
|
||||
New dependencies
|
||||
================
|
||||
|
||||
If you are proposing a change that will require a new dependency, we
|
||||
prefer dependencies that are already packaged by Debian or Red Hat. This makes
|
||||
life much easier for our downstream package maintainers. A package that is only
|
||||
available on PyPI or GitHub, and not more widely packaged, may not be accepted.
|
||||
|
||||
We are unlikely to accept a dependency on CUDA or other GPU-based libraries,
|
||||
because these are still difficult to package and install on many systems.
|
||||
We recommend implementing these changes as plugins.
|
||||
|
||||
Python dependencies must also be license-compatible. GPLv3 or AGPLv3 are likely
|
||||
incompatible with the project's license, but LGPLv3 is compatible.
|
||||
|
||||
New non-Python dependencies
|
||||
===========================
|
||||
|
||||
OCRmyPDF uses several external programs (Tesseract, Ghostscript and others) for
|
||||
its functionality. In general we prefer to avoid adding new external programs,
|
||||
and if we are to add external programs, we prefer those that are already
|
||||
packaged by Debian or Red Hat.
|
||||
|
||||
Plugins
|
||||
=======
|
||||
|
||||
Some new features may be a good fit for a plugin. Plugins are a way to add
|
||||
features to OCRmyPDF without adding them to the core program. Plugins are
|
||||
installed separately from OCRmyPDF. They are written in Python and can be
|
||||
installed from PyPI. See the `plugin documentation <https://ocrmypdf.readthedocs.io/en/latest/plugins.html>`_.
|
||||
|
||||
We are happy to link users to your plugin from the documentation.
|
||||
|
||||
Style guide: Is it OCRmyPDF or ocrmypdf?
|
||||
========================================
|
||||
|
||||
The program/project is OCRmyPDF and the name of the executable or library is ocrmypdf.
|
||||
|
||||
Copyright and license
|
||||
=====================
|
||||
|
||||
For contributions over 10 lines of code, please add your name to list of
|
||||
copyright holders for that file. The core program is licensed under MPL-2.0,
|
||||
test files and documentation under CC-BY-SA 4.0, and miscellaneous files under
|
||||
MIT, with a few minor exceptions. Please contribute only content that you own
|
||||
or have the right to contribute under these licenses.
|
||||
@@ -0,0 +1,404 @@
|
||||
% SPDX-FileCopyrightText: 2025 James R. Barlow
|
||||
% SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
|
||||
Cookbook
|
||||
========
|
||||
|
||||
Basic examples
|
||||
--------------
|
||||
|
||||
### Help!
|
||||
|
||||
ocrmypdf has built-in help.
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --help
|
||||
:::
|
||||
|
||||
### Add an OCR layer and convert to PDF/A
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf input.pdf output.pdf
|
||||
:::
|
||||
|
||||
### Add an OCR layer and output a standard PDF
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --output-type pdf input.pdf output.pdf
|
||||
:::
|
||||
|
||||
### Create a PDF/A with all color and grayscale images converted to JPEG
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --output-type pdfa --pdfa-image-compression jpeg input.pdf output.pdf
|
||||
:::
|
||||
|
||||
### Modify a file in place
|
||||
|
||||
The file will only be overwritten if OCRmyPDF is successful.
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf myfile.pdf myfile.pdf
|
||||
:::
|
||||
|
||||
### Correct page rotation
|
||||
|
||||
OCR will attempt to automatic correct the rotation of each page. This
|
||||
can help fix a scanning job that contains a mix of landscape and
|
||||
portrait pages.
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --rotate-pages myfile.pdf myfile.pdf
|
||||
:::
|
||||
|
||||
You can increase (decrease) the parameter `--rotate-pages-threshold` to
|
||||
make page rotation more (less) aggressive. The threshold number is the
|
||||
ratio of how confidence the OCR engine is that the document image should
|
||||
be changed, compared to kept the same. The default value is quite
|
||||
conservative; on some files it may not attempt rotations at all unless
|
||||
it is very confident that the current rotation is wrong. A lower value
|
||||
of `2.0` will produce more rotations, and more false positives. Run with
|
||||
`-v1` to see the confidence level for each page to see if there may be a
|
||||
better value for your files.
|
||||
|
||||
If the page is \"just a little off horizontal\", like a crooked picture,
|
||||
then you want `--deskew`. `--rotate-pages` is for when the cardinal
|
||||
angle is wrong.
|
||||
|
||||
### OCR languages other than English
|
||||
|
||||
OCRmyPDF assumes the document is in English unless told otherwise. OCR
|
||||
quality may be poor if the wrong language is used.
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf -l fra LeParisien.pdf LeParisien.pdf
|
||||
ocrmypdf -l eng+fra Bilingual-English-French.pdf Bilingual-English-French.pdf
|
||||
:::
|
||||
|
||||
Language packs must be installed for all languages specified. See
|
||||
`Installing additional language packs <lang-packs>`{.interpreted-text
|
||||
role="ref"}.
|
||||
|
||||
Unfortunately, the Tesseract OCR engine has no ability to detect the
|
||||
language when it is unknown.
|
||||
|
||||
### Produce PDF and text file containing OCR text
|
||||
|
||||
This produces a file named \"output.pdf\" and a companion text file
|
||||
named \"output.txt\".
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --sidecar output.txt input.pdf output.pdf
|
||||
:::
|
||||
|
||||
:::{note}
|
||||
The sidecar file contains the **OCR text** found by OCRmyPDF. If the
|
||||
document contains pages that already have text, that text will not
|
||||
appear in the sidecar. If the option `--pages` is used, only those pages
|
||||
on which OCR was performed will be included in the sidecar. If certain
|
||||
pages were skipped because of options like `--skip-big` or
|
||||
`--tesseract-timeout`, those pages will not be in the sidecar.
|
||||
|
||||
If you don\'t want to generate the output PDF, use `--output-type=none`
|
||||
to avoid generating one. Set the output filename to `-` (i.e. redirect
|
||||
to stdout).
|
||||
|
||||
To extract all text from a PDF, whether generated from OCR or otherwise,
|
||||
use a program like Poppler\'s `pdftotext` or `pdfgrep`.
|
||||
:::
|
||||
|
||||
### OCR images, not PDFs
|
||||
|
||||
#### Option: use Tesseract
|
||||
|
||||
If you are starting with images, you can just use Tesseract directly to
|
||||
convert images to PDFs:
|
||||
|
||||
:::{code} bash
|
||||
tesseract my-image.jpg output-prefix pdf
|
||||
:::
|
||||
|
||||
:::{code} bash
|
||||
# When there are multiple images
|
||||
tesseract text-file-containing-list-of-image-filenames.txt output-prefix pdf
|
||||
:::
|
||||
|
||||
Tesseract\'s PDF output is quite good -- OCRmyPDF uses it internally, in
|
||||
some cases. However, OCRmyPDF has many features not available in
|
||||
Tesseract like image processing, metadata control, and PDF/A generation.
|
||||
|
||||
#### Option: use img2pdf
|
||||
|
||||
You can also use a program like
|
||||
[img2pdf](https://gitlab.mister-muffin.de/josch/img2pdf) to convert your
|
||||
images to PDFs, and then pipe the results to run ocrmypdf. The `-` tells
|
||||
ocrmypdf to read standard input.
|
||||
|
||||
:::{code} bash
|
||||
img2pdf my-images*.jpg | ocrmypdf - myfile.pdf
|
||||
:::
|
||||
|
||||
`img2pdf` is recommended because it does an excellent job at generating
|
||||
PDFs without transcoding images.
|
||||
|
||||
#### Option: use OCRmyPDF (single images only)
|
||||
|
||||
For convenience, OCRmyPDF can also convert single images to PDFs on its
|
||||
own. If the resolution (dots per inch, DPI) of an image is not set or is
|
||||
incorrect, it can be overridden with `--image-dpi`. (As 1 inch is 2.54
|
||||
cm, 1 dpi = 0.39 dpcm).
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --image-dpi 300 image.png myfile.pdf
|
||||
:::
|
||||
|
||||
If you have multiple images, you must use `img2pdf` to convert the
|
||||
images to PDF.
|
||||
|
||||
#### Not recommended
|
||||
|
||||
We caution against using ImageMagick or Ghostscript to convert images to
|
||||
PDF, since they may transcode images or produce downsampled images,
|
||||
sometimes without warning.
|
||||
|
||||
Image processing
|
||||
----------------
|
||||
|
||||
OCRmyPDF perform some image processing on each page of a PDF, if
|
||||
desired. The same processing is applied to each page. It is suggested
|
||||
that the user review files after image processing as these commands
|
||||
might remove desirable content, especially from poor quality scans.
|
||||
|
||||
- `--rotate-pages` attempts to determine the correct orientation for
|
||||
each page and rotates the page if necessary.
|
||||
- `--remove-background` attempts to detect and remove a noisy
|
||||
background from grayscale or color images. Monochrome images are
|
||||
ignored. This should not be used on documents that contain color
|
||||
photos as it may remove them.
|
||||
- `--deskew` will correct pages that were scanned at a skewed angle by
|
||||
rotating them back into place.
|
||||
- `--clean` uses [unpaper](https://www.flameeyes.eu/projects/unpaper)
|
||||
to clean up pages before OCR, but does not alter the final output.
|
||||
This makes it less likely that OCR will try to find text in
|
||||
background noise.
|
||||
- `--clean-final` uses unpaper to clean up pages before OCR and
|
||||
inserts the page into the final output. You will want to review each
|
||||
page to ensure that unpaper did not remove something important.
|
||||
|
||||
:::{note}
|
||||
In many cases image processing will rasterize PDF pages as images,
|
||||
potentially losing quality.
|
||||
:::
|
||||
|
||||
:::{warning}
|
||||
`--clean-final` and `--remove-background` may leave undesirable visual
|
||||
artifacts in some images where their algorithms have shortcomings. Files
|
||||
should be visually reviewed after using these options.
|
||||
:::
|
||||
|
||||
### Example: OCR and correct document skew (crooked scan)
|
||||
|
||||
Deskew:
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --deskew input.pdf output.pdf
|
||||
:::
|
||||
|
||||
Image processing commands can be combined. The order in which options
|
||||
are given does not matter. OCRmyPDF always applies the steps of the
|
||||
image processing pipeline in the same order (rotate, remove background,
|
||||
deskew, clean).
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --deskew --clean --rotate-pages input.pdf output.pdf
|
||||
:::
|
||||
|
||||
Don\'t actually OCR my PDF
|
||||
--------------------------
|
||||
|
||||
If you set `--tesseract-timeout 0` OCRmyPDF will apply its image
|
||||
processing without performing OCR (by causing OCR to time out). This
|
||||
works if all you want to is to apply image processing or PDF/A
|
||||
conversion.
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --tesseract-timeout=0 --remove-background input.pdf output.pdf
|
||||
:::
|
||||
|
||||
::: {.versionchanged}
|
||||
v14.1.0
|
||||
|
||||
Prior to this version, `--tesseract-timeout 0` would prevent other uses
|
||||
of Tesseract, such as deskewing, from working. This is no longer the
|
||||
case. Use `--tesseract-non-ocr-timeout` to control the timeout for
|
||||
non-OCR operations, if needed.
|
||||
:::
|
||||
|
||||
### Remove all text or OCR from my PDF
|
||||
|
||||
This is getting ridiculous, but OCRmyPDF can complete strip all textual
|
||||
information from a PDF and reconstruct it as a \"bag of images\" PDF.
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --tesseract-timeout 0 --force-ocr input.pdf output.pdf
|
||||
:::
|
||||
|
||||
Why would you want to do this? Perhaps you have a PDF where OCR fails to
|
||||
produce useful results, and just want to get rid of all OCR information.
|
||||
This command also removes OCR generated by third party tools.
|
||||
|
||||
### Optimize images without performing OCR
|
||||
|
||||
You can also optimize all images without performing any OCR:
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --tesseract-timeout=0 --optimize 3 --skip-text input.pdf output.pdf
|
||||
:::
|
||||
|
||||
### Process only certain pages
|
||||
|
||||
You can ask OCRmyPDF to only apply [image processing](#image-processing)
|
||||
and OCR to certain pages.
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --pages 2,3,13-17 input.pdf output.pdf
|
||||
:::
|
||||
|
||||
Hyphens denote a range of pages and commas separate page numbers. If you
|
||||
prefer to use spaces, quote all of the page numbers:
|
||||
`--pages '2, 3, 5, 7'`.
|
||||
|
||||
OCRmyPDF will warn if your list of page numbers contains duplicates or
|
||||
overlapping pages. OCRmyPDF does not currently account for document page
|
||||
numbers, such as an introduction section of a book that uses Roman
|
||||
numerals. It simply counts the number of virtual pieces of paper since
|
||||
the start. If your list of pages is out of numerical order, OCRmyPDF
|
||||
will sort it for you.
|
||||
|
||||
Regardless of the argument to `--pages`, OCRmyPDF will optimize all
|
||||
pages/images in the file and convert it to PDF/A, unless you disable
|
||||
those options. Both of these steps are \"whole file\" operations. In
|
||||
this example, we want to OCR only the title and otherwise change the PDF
|
||||
as little as possible:
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --pages 1 --output-type pdf --optimize 0 input.pdf output.pdf
|
||||
:::
|
||||
|
||||
Redo existing OCR
|
||||
-----------------
|
||||
|
||||
To redo OCR on a file OCRed with other OCR software or a previous
|
||||
version of OCRmyPDF and/or Tesseract, you may use the `--redo-ocr`
|
||||
argument. (Normally, OCRmyPDF will exit with an error if asked to modify
|
||||
a file with OCR.)
|
||||
|
||||
This may be helpful for users who want to take advantage of accuracy
|
||||
improvements in Tesseract for files they previously OCRed with an
|
||||
earlier version of Tesseract and OCRmyPDF.
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --redo-ocr input.pdf output.pdf
|
||||
:::
|
||||
|
||||
This method will replace OCR without rasterizing, reducing quality or
|
||||
removing vector content. If a file contains a mix of pure digital text
|
||||
and OCR, digital text will be ignored and OCR will be replaced. As such
|
||||
this mode is incompatible with image processing options, since they
|
||||
alter the appearance of the file.
|
||||
|
||||
In some cases, existing OCR cannot be detected or replaced. Files
|
||||
produced by OCRmyPDF v2.2 or earlier, for example, are internally
|
||||
represented as having visible text with an opaque image drawn on top.
|
||||
This situation cannot be detected.
|
||||
|
||||
If `--redo-ocr` does not work, you can use `--force-ocr`, which will
|
||||
force rasterization of all pages, potentially reducing quality or losing
|
||||
vector content.
|
||||
|
||||
Improving OCR quality
|
||||
---------------------
|
||||
|
||||
The [Image processing](#image-processing) features can improve OCR
|
||||
quality.
|
||||
|
||||
Rotating pages and deskewing helps to ensure that the page orientation
|
||||
is correct before OCR begins. Removing the background and/or cleaning
|
||||
the page can also improve results. The `--oversample DPI` argument can
|
||||
be specified to resample images to higher resolution before attempting
|
||||
OCR; this can improve results as well.
|
||||
|
||||
OCR quality will suffer if the resolution of input images is not correct
|
||||
(since the range of pixel sizes that will be checked for possible fonts
|
||||
will also be incorrect).
|
||||
|
||||
PDF optimization
|
||||
----------------
|
||||
|
||||
By default OCRmyPDF will attempt to perform lossless optimizations on
|
||||
the images inside PDFs after OCR is complete. Optimization is performed
|
||||
even if no OCR text is found.
|
||||
|
||||
The `--optimize N` (short form `-O`) argument controls optimization,
|
||||
where `N` ranges from 0 to 3 inclusive, analogous to the optimization
|
||||
levels in the GCC compiler.
|
||||
|
||||
:::{list-table}
|
||||
---
|
||||
widths: auto
|
||||
header-rows: 1
|
||||
---
|
||||
|
||||
* - Level
|
||||
- Comments
|
||||
* - ``--optimize=0``
|
||||
- Disables optimization.
|
||||
* - ``--optimize 1``
|
||||
- Enables lossless optimizations, such as transcoding images to more
|
||||
efficient formats. Also compress other uncompressed objects in the
|
||||
PDF and enables the more efficient "object streams" within the PDF.
|
||||
(If ``--jbig2-lossy`` is issued, then lossy JBIG2 optimization is used.
|
||||
The decision to use lossy JBIG2 is separate from standard optimization
|
||||
settings.)
|
||||
* - ``--optimize 2``
|
||||
- All of the above, and enables lossy optimizations and color quantization.
|
||||
* - ``--optimize 3``
|
||||
- All of the above, and enables more aggressive optimizations and targets lower image quality.
|
||||
:::
|
||||
|
||||
Optimization is improved when a JBIG2 encoder is available and when
|
||||
`pngquant` is installed. If either of these components are missing, then
|
||||
some types of images cannot be optimized.
|
||||
|
||||
The types of optimization available may expand over time. By default,
|
||||
OCRmyPDF compresses data streams inside PDFs, and will change
|
||||
inefficient compression modes to more modern versions. A program like
|
||||
`qpdf` can be used to change encodings, e.g. to inspect the internals
|
||||
for a PDF.
|
||||
|
||||
:::{code} bash
|
||||
ocrmypdf --optimize 3 in.pdf out.pdf # Make it small
|
||||
:::
|
||||
|
||||
Some users may consider enabling lossy JBIG2. See:
|
||||
`jbig2-lossy`{.interpreted-text role="ref"}.
|
||||
|
||||
:::{note}
|
||||
Image processing and PDF/A conversion can also introduce lossy
|
||||
transformations to your PDF images, even when `--optimize 1` is in use.
|
||||
:::
|
||||
|
||||
Digitally signed PDFs
|
||||
---------------------
|
||||
|
||||
OCRmyPDF cannot preserve digital signatures in PDFs and also add OCR to
|
||||
them. By default, it will refuse to modify a signed PDF regardless of
|
||||
other settings. You can override this behavior with
|
||||
`--invalidate-digital-signatures`; as the name suggests, any digital
|
||||
signatures will be invalidated.
|
||||
|
||||
OCRmyPDF cannot open documents that are encrypted with a digital
|
||||
certificate.
|
||||
|
||||
Versions of OCRmyPDF prior to 14.4.0 would invalidate existing digital
|
||||
signatures without warning.
|
||||
@@ -1,410 +0,0 @@
|
||||
.. SPDX-FileCopyrightText: 2022 James R. Barlow
|
||||
..
|
||||
.. SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
|
||||
========
|
||||
Cookbook
|
||||
========
|
||||
|
||||
Basic examples
|
||||
==============
|
||||
|
||||
Help!
|
||||
-----
|
||||
|
||||
ocrmypdf has built-in help.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --help
|
||||
|
||||
Add an OCR layer and convert to PDF/A
|
||||
-------------------------------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf input.pdf output.pdf
|
||||
|
||||
Add an OCR layer and output a standard PDF
|
||||
------------------------------------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --output-type pdf input.pdf output.pdf
|
||||
|
||||
Create a PDF/A with all color and grayscale images converted to JPEG
|
||||
--------------------------------------------------------------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --output-type pdfa --pdfa-image-compression jpeg input.pdf output.pdf
|
||||
|
||||
Modify a file in place
|
||||
----------------------
|
||||
|
||||
The file will only be overwritten if OCRmyPDF is successful.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf myfile.pdf myfile.pdf
|
||||
|
||||
Correct page rotation
|
||||
---------------------
|
||||
|
||||
OCR will attempt to automatic correct the rotation of each page. This
|
||||
can help fix a scanning job that contains a mix of landscape and
|
||||
portrait pages.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --rotate-pages myfile.pdf myfile.pdf
|
||||
|
||||
You can increase (decrease) the parameter ``--rotate-pages-threshold``
|
||||
to make page rotation more (less) aggressive. The threshold number is the ratio
|
||||
of how confidence the OCR engine is that the document image should be changed,
|
||||
compared to kept the same. The default value is quite conservative; on some files
|
||||
it may not attempt rotations at all unless it is very confident that the current
|
||||
rotation is wrong. A lower value of ``2.0`` will produce more rotations, and
|
||||
more false positives. Run with ``-v1`` to see the confidence level for each
|
||||
page to see if there may be a better value for your files.
|
||||
|
||||
If the page is "just a little off horizontal", like a crooked picture,
|
||||
then you want ``--deskew``. ``--rotate-pages`` is for when the cardinal
|
||||
angle is wrong.
|
||||
|
||||
OCR languages other than English
|
||||
--------------------------------
|
||||
|
||||
OCRmyPDF assumes the document is in English unless told otherwise. OCR
|
||||
quality may be poor if the wrong language is used.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf -l fra LeParisien.pdf LeParisien.pdf
|
||||
ocrmypdf -l eng+fra Bilingual-English-French.pdf Bilingual-English-French.pdf
|
||||
|
||||
Language packs must be installed for all languages specified. See
|
||||
:ref:`Installing additional language packs <lang-packs>`.
|
||||
|
||||
Unfortunately, the Tesseract OCR engine has no ability to detect the
|
||||
language when it is unknown.
|
||||
|
||||
Produce PDF and text file containing OCR text
|
||||
---------------------------------------------
|
||||
|
||||
This produces a file named "output.pdf" and a companion text file named
|
||||
"output.txt".
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --sidecar output.txt input.pdf output.pdf
|
||||
|
||||
.. note::
|
||||
|
||||
The sidecar file contains the **OCR text** found by OCRmyPDF. If the document
|
||||
contains pages that already have text, that text will not appear in the
|
||||
sidecar. If the option ``--pages`` is used, only those pages on which OCR
|
||||
was performed will be included in the sidecar. If certain pages were skipped
|
||||
because of options like ``--skip-big`` or ``--tesseract-timeout``, those pages
|
||||
will not be in the sidecar.
|
||||
|
||||
If you don't want to generate the output PDF, use ``--output-type=none`` to
|
||||
avoid generating one. Set the output filename to ``-`` (i.e. redirect to stdout).
|
||||
|
||||
To extract all text from a PDF, whether generated from OCR or otherwise,
|
||||
use a program like Poppler's ``pdftotext`` or ``pdfgrep``.
|
||||
|
||||
OCR images, not PDFs
|
||||
--------------------
|
||||
|
||||
Option: use Tesseract
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
If you are starting with images, you can just use Tesseract directly to
|
||||
convert images to PDFs:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
tesseract my-image.jpg output-prefix pdf
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# When there are multiple images
|
||||
tesseract text-file-containing-list-of-image-filenames.txt output-prefix pdf
|
||||
|
||||
Tesseract's PDF output is quite good – OCRmyPDF uses it internally, in
|
||||
some cases. However, OCRmyPDF has many features not available in
|
||||
Tesseract like image processing, metadata control, and PDF/A generation.
|
||||
|
||||
Option: use img2pdf
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
You can also use a program like
|
||||
`img2pdf <https://gitlab.mister-muffin.de/josch/img2pdf>`__ to convert
|
||||
your images to PDFs, and then pipe the results to run ocrmypdf. The
|
||||
``-`` tells ocrmypdf to read standard input.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
img2pdf my-images*.jpg | ocrmypdf - myfile.pdf
|
||||
|
||||
``img2pdf`` is recommended because it does an excellent job at
|
||||
generating PDFs without transcoding images.
|
||||
|
||||
Option: use OCRmyPDF (single images only)
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
For convenience, OCRmyPDF can also convert single images to PDFs on its
|
||||
own. If the resolution (dots per inch, DPI) of an image is not set or is
|
||||
incorrect, it can be overridden with ``--image-dpi``. (As 1 inch is 2.54
|
||||
cm, 1 dpi = 0.39 dpcm).
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --image-dpi 300 image.png myfile.pdf
|
||||
|
||||
If you have multiple images, you must use ``img2pdf`` to convert the
|
||||
images to PDF.
|
||||
|
||||
Not recommended
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
We caution against using ImageMagick or Ghostscript to convert images to
|
||||
PDF, since they may transcode images or produce downsampled images,
|
||||
sometimes without warning.
|
||||
|
||||
Image processing
|
||||
================
|
||||
|
||||
OCRmyPDF perform some image processing on each page of a PDF, if
|
||||
desired. The same processing is applied to each page. It is suggested
|
||||
that the user review files after image processing as these commands
|
||||
might remove desirable content, especially from poor quality scans.
|
||||
|
||||
- ``--rotate-pages`` attempts to determine the correct orientation for
|
||||
each page and rotates the page if necessary.
|
||||
- ``--remove-background`` attempts to detect and remove a noisy
|
||||
background from grayscale or color images. Monochrome images are
|
||||
ignored. This should not be used on documents that contain color
|
||||
photos as it may remove them.
|
||||
- ``--deskew`` will correct pages that were scanned at a skewed angle by
|
||||
rotating them back into place.
|
||||
- ``--clean`` uses
|
||||
`unpaper <https://www.flameeyes.eu/projects/unpaper>`__ to clean up
|
||||
pages before OCR, but does not alter the final output. This makes it
|
||||
less likely that OCR will try to find text in background noise.
|
||||
- ``--clean-final`` uses unpaper to clean up pages before OCR and
|
||||
inserts the page into the final output. You will want to review each
|
||||
page to ensure that unpaper did not remove something important.
|
||||
|
||||
.. note::
|
||||
|
||||
In many cases image processing will rasterize PDF pages as images,
|
||||
potentially losing quality.
|
||||
|
||||
.. warning::
|
||||
|
||||
``--clean-final`` and ``--remove-background`` may leave undesirable
|
||||
visual artifacts in some images where their algorithms have
|
||||
shortcomings. Files should be visually reviewed after using these
|
||||
options.
|
||||
|
||||
Example: OCR and correct document skew (crooked scan)
|
||||
-----------------------------------------------------
|
||||
|
||||
Deskew:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --deskew input.pdf output.pdf
|
||||
|
||||
Image processing commands can be combined. The order in which options
|
||||
are given does not matter. OCRmyPDF always applies the steps of the
|
||||
image processing pipeline in the same order (rotate, remove background,
|
||||
deskew, clean).
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --deskew --clean --rotate-pages input.pdf output.pdf
|
||||
|
||||
Don't actually OCR my PDF
|
||||
=========================
|
||||
|
||||
If you set ``--tesseract-timeout 0`` OCRmyPDF will apply its image
|
||||
processing without performing OCR (by causing OCR to time out). This works
|
||||
if all you want to is to apply image processing or PDF/A conversion.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --tesseract-timeout=0 --remove-background input.pdf output.pdf
|
||||
|
||||
.. versionchanged:: v14.1.0
|
||||
|
||||
Prior to this version, ``--tesseract-timeout 0`` would prevent other
|
||||
uses of Tesseract, such as deskewing, from working. This is no longer
|
||||
the case. Use ``--tesseract-non-ocr-timeout`` to control the timeout
|
||||
for non-OCR operations, if needed.
|
||||
|
||||
Remove all text or OCR from my PDF
|
||||
----------------------------------
|
||||
|
||||
This is getting ridiculous, but OCRmyPDF can complete strip all textual
|
||||
information from a PDF and reconstruct it as a "bag of images" PDF.
|
||||
|
||||
.. code-block::
|
||||
|
||||
ocrmypdf --tesseract-timeout 0 --force-ocr input.pdf output.pdf
|
||||
|
||||
Why would you want to do this? Perhaps you have a PDF where OCR
|
||||
fails to produce useful results, and just want to get rid of all OCR information.
|
||||
This command also removes OCR generated by third party tools.
|
||||
|
||||
Optimize images without performing OCR
|
||||
--------------------------------------
|
||||
|
||||
You can also optimize all images without performing any OCR:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --tesseract-timeout=0 --optimize 3 --skip-text input.pdf output.pdf
|
||||
|
||||
Process only certain pages
|
||||
--------------------------
|
||||
|
||||
You can ask OCRmyPDF to only apply `image processing <#image-processing>`__
|
||||
and OCR to certain pages.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --pages 2,3,13-17 input.pdf output.pdf
|
||||
|
||||
Hyphens denote a range of pages and commas separate page numbers. If you prefer
|
||||
to use spaces, quote all of the page numbers: ``--pages '2, 3, 5, 7'``.
|
||||
|
||||
OCRmyPDF will warn if your list of page numbers contains duplicates or
|
||||
overlapping pages. OCRmyPDF does not currently account for document page numbers,
|
||||
such as an introduction section of a book that uses Roman numerals. It simply
|
||||
counts the number of virtual pieces of paper since the start. If your list of
|
||||
pages is out of numerical order, OCRmyPDF will sort it for you.
|
||||
|
||||
Regardless of the argument to ``--pages``, OCRmyPDF will optimize all pages/images
|
||||
in the file and convert it to PDF/A, unless you disable those options. Both of these
|
||||
steps are "whole file" operations. In this example, we want to OCR only the title
|
||||
and otherwise change the PDF as little as possible:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --pages 1 --output-type pdf --optimize 0 input.pdf output.pdf
|
||||
|
||||
Redo existing OCR
|
||||
=================
|
||||
|
||||
To redo OCR on a file OCRed with other OCR software or a previous
|
||||
version of OCRmyPDF and/or Tesseract, you may use the ``--redo-ocr``
|
||||
argument. (Normally, OCRmyPDF will exit with an error if asked to modify
|
||||
a file with OCR.)
|
||||
|
||||
This may be helpful for users who want to take advantage of accuracy
|
||||
improvements in Tesseract for files they previously OCRed with an
|
||||
earlier version of Tesseract and OCRmyPDF.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --redo-ocr input.pdf output.pdf
|
||||
|
||||
This method will replace OCR without rasterizing, reducing quality or
|
||||
removing vector content. If a file contains a mix of pure digital text
|
||||
and OCR, digital text will be ignored and OCR will be replaced. As such
|
||||
this mode is incompatible with image processing options, since they
|
||||
alter the appearance of the file.
|
||||
|
||||
In some cases, existing OCR cannot be detected or replaced. Files
|
||||
produced by OCRmyPDF v2.2 or earlier, for example, are internally
|
||||
represented as having visible text with an opaque image drawn on top.
|
||||
This situation cannot be detected.
|
||||
|
||||
If ``--redo-ocr`` does not work, you can use ``--force-ocr``, which will
|
||||
force rasterization of all pages, potentially reducing quality or losing
|
||||
vector content.
|
||||
|
||||
Improving OCR quality
|
||||
=====================
|
||||
|
||||
The `Image processing <#image-processing>`__ features can improve OCR
|
||||
quality.
|
||||
|
||||
Rotating pages and deskewing helps to ensure that the page orientation
|
||||
is correct before OCR begins. Removing the background and/or cleaning
|
||||
the page can also improve results. The ``--oversample DPI`` argument can
|
||||
be specified to resample images to higher resolution before attempting
|
||||
OCR; this can improve results as well.
|
||||
|
||||
OCR quality will suffer if the resolution of input images is not correct
|
||||
(since the range of pixel sizes that will be checked for possible fonts
|
||||
will also be incorrect).
|
||||
|
||||
PDF optimization
|
||||
================
|
||||
|
||||
By default OCRmyPDF will attempt to perform lossless optimizations on
|
||||
the images inside PDFs after OCR is complete. Optimization is performed
|
||||
even if no OCR text is found.
|
||||
|
||||
The ``--optimize N`` (short form ``-O``) argument controls optimization,
|
||||
where ``N`` ranges from 0 to 3 inclusive, analogous to the optimization
|
||||
levels in the GCC compiler.
|
||||
|
||||
.. list-table::
|
||||
:widths: auto
|
||||
:header-rows: 1
|
||||
|
||||
* - Level
|
||||
- Comments
|
||||
* - ``--optimize 0``
|
||||
- Disables optimization.
|
||||
* - ``--optimize 1``
|
||||
- Enables lossless optimizations, such as transcoding images to more
|
||||
efficient formats. Also compress other uncompressed objects in the
|
||||
PDF and enables the more efficient "object streams" within the PDF.
|
||||
(If ``--jbig2-lossy`` is issued, then lossy JBIG2 optimization is used.
|
||||
The decision to use lossy JBIG2 is separate from standard optimization
|
||||
settings.)
|
||||
* - ``--optimize 2``
|
||||
- All of the above, and enables lossy optimizations and color quantization.
|
||||
* - ``--optimize 3``
|
||||
- All of the above, and enables more aggressive optimizations and targets lower image quality.
|
||||
|
||||
Optimization is improved when a JBIG2 encoder is available and when
|
||||
``pngquant`` is installed. If either of these components are missing,
|
||||
then some types of images cannot be optimized.
|
||||
|
||||
The types of optimization available may expand over time. By default,
|
||||
OCRmyPDF compresses data streams inside PDFs, and will change
|
||||
inefficient compression modes to more modern versions. A program like
|
||||
``qpdf`` can be used to change encodings, e.g. to inspect the internals
|
||||
for a PDF.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --optimize 3 in.pdf out.pdf # Make it small
|
||||
|
||||
Some users may consider enabling lossy JBIG2. See: :ref:`jbig2-lossy`.
|
||||
|
||||
.. note::
|
||||
|
||||
Image processing and PDF/A conversion can also introduce lossy transformations
|
||||
to your PDF images, even when ``--optimize 1`` is in use.
|
||||
|
||||
|
||||
Digitally signed PDFs
|
||||
=====================
|
||||
|
||||
OCRmyPDF cannot preserve digital signatures in PDFs and also add OCR to them.
|
||||
By default, it will refuse to modify a signed PDF regardless of other settings. You can
|
||||
override this behavior with ``--invalidate-digital-signatures``; as the name suggests,
|
||||
any digital signatures will be invalidated.
|
||||
|
||||
OCRmyPDF cannot open documents that are encrypted with a digital certificate.
|
||||
|
||||
Versions of OCRmyPDF prior to 14.4.0 would invalidate existing digital signatures
|
||||
without warning.
|
||||
+245
@@ -0,0 +1,245 @@
|
||||
# OCRmyPDF Docker image {#docker}
|
||||
|
||||
OCRmyPDF is also available in Docker images that packages recent
|
||||
versions of all dependencies.
|
||||
|
||||
For users who already have Docker installed this may be an easy and
|
||||
convenient option.
|
||||
|
||||
On platforms other than Linux, Docker runs in a virtual machine, and so
|
||||
may be less performant. You may also want to adjust the Docker virtual
|
||||
machine\'s memory and CPU allocation. On Linux, the Docker image runs
|
||||
natively and performance is comparable to a system installation.
|
||||
|
||||
{#docker-install}
|
||||
## Installing the Docker image
|
||||
|
||||
If you have [Docker](https://docs.docker.com/) installed on your system,
|
||||
you can install a Docker image of the latest release.
|
||||
|
||||
If you can run this command successfully, your system is ready to
|
||||
download and execute the image:
|
||||
|
||||
:::{code} bash
|
||||
docker run hello-world
|
||||
:::
|
||||
|
||||
:::{list-table} Docker Images
|
||||
:header-rows: 1
|
||||
|
||||
* - Image
|
||||
- Architecture
|
||||
- Description
|
||||
* - `jbarlow83/ocrmypdf-alpine`
|
||||
- x86_64 and arm64
|
||||
- Recommended image, based on Alpine Linux.
|
||||
* - `jbarlow83/ocrmypdf-ubuntu`
|
||||
- x86_64 and arm64
|
||||
- Alternate image, based on Ubuntu. When the Alpine image is considered stable and available for arm64, this image will be deprecated.
|
||||
* - `jbarlow83/ocrmypdf`
|
||||
- x86_64 and arm64
|
||||
- Currently an alias for ocrmypdf-ubuntu. When the Alpine image is considered stable and available for arm64, this name will point to the Alpine image. If you don\'t know about the difference between Alpine and Ubuntu, use this image.
|
||||
:::
|
||||
|
||||
To install:
|
||||
|
||||
:::{code} bash
|
||||
docker pull jbarlow83/ocrmypdf-alpine
|
||||
:::
|
||||
|
||||
The `ocrmypdf` image is also available, but is deprecated and will be
|
||||
removed in the future.
|
||||
|
||||
OCRmyPDF will use all available CPU cores. See the Docker documentation
|
||||
for [adjusting memory and CPU on other
|
||||
platforms](https://docs.docker.com/config/containers/resource_constraints/)
|
||||
if you are using Docker on macOS or Windows, where you may need to
|
||||
manually assign more resources. On Linux, all resources will be
|
||||
available automatically.
|
||||
|
||||
The underlying operating system and other details in Docker images are
|
||||
considered implementation details and **subject to change at minor
|
||||
releases**. If you are modifying the image, you should pin the version
|
||||
you intend to use.
|
||||
|
||||
## Using the Docker image on the command line
|
||||
|
||||
**Unlike typical Docker containers**, in this section the OCRmyPDF
|
||||
Docker container is ephemeral -- it runs for one OCR job and terminates,
|
||||
just like a command line program. We are using Docker to deliver an
|
||||
application (as opposed to the more conventional case, where a Docker
|
||||
container runs as a server). For that reason we usually use the `--rm`
|
||||
argument to delete the container when it exits.
|
||||
|
||||
To start a Docker container (instance of the image):
|
||||
|
||||
:::{code} bash
|
||||
docker run --rm -i jbarlow83/ocrmypdf-alpine (... all other arguments here...) - -
|
||||
:::
|
||||
|
||||
For convenience, create a shell alias to hide the Docker command. It is
|
||||
easier to send the input file as stdin and read the output from stdout
|
||||
-- **this avoids the messy permission issues with Docker entirely**.
|
||||
|
||||
:::{code} bash
|
||||
alias docker_ocrmypdf='docker run --rm -i jbarlow83/ocrmypdf-alpine'
|
||||
docker_ocrmypdf --version # runs docker version
|
||||
docker_ocrmypdf - - <input.pdf >output.pdf
|
||||
:::
|
||||
|
||||
Or in the wonderful [fish shell](https://fishshell.com/):
|
||||
|
||||
:::{code} fish
|
||||
alias docker_ocrmypdf 'docker run --rm jbarlow83/ocrmypdf-alpine'
|
||||
funcsave docker_ocrmypdf
|
||||
:::
|
||||
|
||||
Alternately, you could mount the local current working directory as a
|
||||
Docker volume:
|
||||
|
||||
:::{code} bash
|
||||
alias docker_ocrmypdf='docker run --rm -i --user "$(id -u):$(id -g)" --workdir /data -v "$PWD:/data" jbarlow83/ocrmypdf-alpine'
|
||||
docker_ocrmypdf /data/input.pdf /data/output.pdf
|
||||
:::
|
||||
|
||||
## Podman
|
||||
|
||||
Especially if you use [Podman](https://podman.io/) (or have SELinux
|
||||
enabled on your system), you may need to add `--userns keep-id` there,
|
||||
otherwise you may get access errors, because the user is otherwise not
|
||||
mapped to the same UID as on the host:
|
||||
|
||||
:::{code} bash
|
||||
alias podman_ocrmypdf='podman run --rm -i --user "$(id -u):$(id -g)" --userns keep-id --workdir /data -v "$PWD:/data" ocrmypdf'
|
||||
podman_ocrmypdf /data/input.pdf /data/output.pdf
|
||||
:::
|
||||
|
||||
If you use SELinux you may additionally need to add the `:Z` [suffix to
|
||||
the
|
||||
volume](https://docs.podman.io/en/stable/markdown/podman-run.1.html#volume-v-source-volume-host-dir-container-dir-options)
|
||||
or disable SELinux for the container using
|
||||
`--security-opt label=disable`, which is suggested for system files as
|
||||
they should not be re-labelled. Please refer to the „Note" section at
|
||||
the end of the linked podman documentation for details.
|
||||
|
||||
{#docker-lang-packs}
|
||||
## Adding languages to the Docker image
|
||||
|
||||
By default the Docker image includes English, German, Simplified
|
||||
Chinese, French, Portuguese and Spanish, the most popular languages for
|
||||
OCRmyPDF users based on feedback. You may add other languages by
|
||||
creating a new Dockerfile based on the public one.
|
||||
|
||||
:::{code} dockerfile
|
||||
FROM jbarlow83/ocrmypdf
|
||||
|
||||
# Example: add Italian
|
||||
RUN apt install tesseract-ocr-ita
|
||||
:::
|
||||
|
||||
To install language packs (training data) such as the
|
||||
[tessdata\_best](https://github.com/tesseract-ocr/tessdata_best) suite
|
||||
or custom data, you first need to determine the version of Tesseract
|
||||
data files, which may differ from the Tesseract program version. Use
|
||||
this command to determine the data file version:
|
||||
|
||||
:::{code} bash
|
||||
docker run -i --rm --entrypoint /bin/ls jbarlow83/ocrmypdf /usr/share/tesseract-ocr
|
||||
:::
|
||||
|
||||
As of 2021, the data file version is probably `4.00`.
|
||||
|
||||
You can then add new data with either a Dockerfile:
|
||||
|
||||
:::{code} dockerfile
|
||||
FROM jbarlow83/ocrmypdf:{TAG}
|
||||
|
||||
# Example: add a tessdata_best file
|
||||
COPY chi_tra_vert.traineddata /usr/share/tesseract-ocr/<data version>/tessdata/
|
||||
:::
|
||||
|
||||
When creating your own image, you should always pin a specific version
|
||||
of the OCRmyPDF Docker image. This ensures that your image will not
|
||||
break when a new version of OCRmyPDF is released.
|
||||
|
||||
Alternately, you can copy training data into a Docker container as
|
||||
follows:
|
||||
|
||||
:::{code} bash
|
||||
docker cp mycustomtraining.traineddata name_of_container:/usr/share/tesseract-ocr/<tesseract version>/tessdata/
|
||||
:::
|
||||
|
||||
Extending the Docker image
|
||||
--------------------------
|
||||
|
||||
You can extend the Docker image with your own customizations, similar to
|
||||
the way it is extended to add language packs.
|
||||
|
||||
Note that the Docker image is subject to change at any time. For
|
||||
example, the base image may be updated to a newer version of Ubuntu or
|
||||
Debian. Such changes will be noted in the release notes but might occur
|
||||
at minor versions releases, unless the way a \"casual\" user of the
|
||||
Docker image is affected.
|
||||
|
||||
If you extend the Docker image, you should pin a specific version of the
|
||||
OCRmyPDF Docker image.
|
||||
|
||||
Executing the test suite
|
||||
------------------------
|
||||
|
||||
The OCRmyPDF test suite is installed with image. To run it:
|
||||
|
||||
:::{code} bash
|
||||
docker run --rm --entrypoint python jbarlow83/ocrmypdf -m pytest
|
||||
:::
|
||||
|
||||
Accessing the shell
|
||||
-------------------
|
||||
|
||||
To use the shell in the Docker image:
|
||||
|
||||
:::{code} bash
|
||||
docker run -it --entrypoint sh jbarlow83/ocrmypdf
|
||||
:::
|
||||
|
||||
Using the OCRmyPDF web service wrapper
|
||||
--------------------------------------
|
||||
|
||||
The OCRmyPDF Docker image includes an example, barebones HTTP web
|
||||
service. The webservice may be launched as follows:
|
||||
|
||||
:::{code} bash
|
||||
docker run --entrypoint python -p 5000:5000 jbarlow83/ocrmypdf webservice.py
|
||||
:::
|
||||
|
||||
We omit the `--rm` parameter so that the container will not be
|
||||
automatically deleted when it exits.
|
||||
|
||||
This will configure the machine to listen on port 5000. On Linux
|
||||
machines this is port 5000 of localhost. On macOS or Windows machines
|
||||
running Docker, this is port 5000 of the virtual machine that runs your
|
||||
Docker images. You can find its IP address using the command
|
||||
`docker-machine ip`.
|
||||
|
||||
Unlike command line usage this program will open a socket and wait for
|
||||
connections.
|
||||
|
||||
:::{warning}
|
||||
The OCRmyPDF web service wrapper is intended for demonstration or
|
||||
development. It provides no security, no authentication, no protection
|
||||
against denial of service attacks, and no load balancing. The default
|
||||
Flask WSGI server is used, which is intended for development only. The
|
||||
server is single-threaded and so can respond to only one client at a
|
||||
time. While running OCR, it cannot respond to any other clients.
|
||||
:::
|
||||
|
||||
Clients must keep their open connection while waiting for OCR to
|
||||
complete. This may entail setting a long timeout; this interface is more
|
||||
useful for internal HTTP API calls.
|
||||
|
||||
Unlike the rest of OCRmyPDF, this web service is licensed under the
|
||||
Affero GPLv3 (AGPLv3) since Ghostscript is also licensed in this way.
|
||||
|
||||
In addition to the above, please read our
|
||||
`general remarks on using OCRmyPDF as a service <ocr-service>`{.interpreted-text
|
||||
role="ref"}.
|
||||
-254
@@ -1,254 +0,0 @@
|
||||
.. SPDX-FileCopyrightText: 2022 James R. Barlow
|
||||
..
|
||||
.. SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
|
||||
.. _docker:
|
||||
|
||||
=====================
|
||||
OCRmyPDF Docker image
|
||||
=====================
|
||||
|
||||
OCRmyPDF is also available in Docker images that packages recent
|
||||
versions of all dependencies.
|
||||
|
||||
For users who already have Docker installed this may be an easy and
|
||||
convenient option.
|
||||
|
||||
On platforms other than Linux, Docker runs in a virtual machine, and so may
|
||||
be less performant. You may also want to adjust the Docker virtual machine's
|
||||
memory and CPU allocation. On Linux, the Docker image runs natively and
|
||||
performance is comparable to a system installation.
|
||||
|
||||
.. _docker-install:
|
||||
|
||||
Installing the Docker image
|
||||
===========================
|
||||
|
||||
If you have `Docker <https://docs.docker.com/>`__ installed on your
|
||||
system, you can install a Docker image of the latest release.
|
||||
|
||||
If you can run this command successfully, your system is ready to download and
|
||||
execute the image:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run hello-world
|
||||
|
||||
.. list-table:: Docker images
|
||||
:widths: 30 20 50
|
||||
:header-rows: 1
|
||||
|
||||
* - Image
|
||||
- Architecture
|
||||
- Description
|
||||
* - ``jbarlow83/ocrmypdf-alpine``
|
||||
- x86_64 and arm64
|
||||
- Recommended image, based on Alpine Linux.
|
||||
* - ``jbarlow83/ocrmypdf-ubuntu``
|
||||
- x86_64 and arm64
|
||||
- Alternate image, based on Ubuntu. When the Alpine image is considered
|
||||
stable and available for arm64, this image will be deprecated.
|
||||
* - ``jbarlow83/ocrmypdf``
|
||||
- x86_64 and arm64
|
||||
- Currently an alias for ocrmypdf-ubuntu. When the Alpine image is
|
||||
considered stable and available for arm64, this name point to the
|
||||
Alpine image. If you don't about the difference between Alpine and
|
||||
Ubuntu, use this image.
|
||||
|
||||
To install:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker pull jbarlow83/ocrmypdf-alpine
|
||||
|
||||
The ``ocrmypdf`` image is also available, but is deprecated and will be removed
|
||||
in the future.
|
||||
|
||||
OCRmyPDF will use all available CPU cores. See the Docker documentation for
|
||||
`adjusting memory and CPU on other platforms <https://docs.docker.com/config/containers/resource_constraints/>`__
|
||||
if you are using Docker on macOS or Windows, where you may need to manually assign
|
||||
more resources. On Linux, all resources will be available automatically.
|
||||
|
||||
The underlying operating system and other details in Docker images are considered
|
||||
implementation details and **subject to change at minor releases**. If you are
|
||||
modifying the image, you should pin the version you intend to use.
|
||||
|
||||
Using the Docker image on the command line
|
||||
==========================================
|
||||
|
||||
**Unlike typical Docker containers**, in this section the OCRmyPDF Docker
|
||||
container is ephemeral – it runs for one OCR job and terminates, just like a
|
||||
command line program. We are using Docker to deliver an application (as opposed
|
||||
to the more conventional case, where a Docker container runs as a server).
|
||||
For that reason we usually use the ``--rm`` argument to delete the container
|
||||
when it exits.
|
||||
|
||||
To start a Docker container (instance of the image):
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run --rm -i jbarlow83/ocrmypdf-alpine (... all other arguments here...) - -
|
||||
|
||||
For convenience, create a shell alias to hide the Docker command. It is
|
||||
easier to send the input file as stdin and read the output from
|
||||
stdout – **this avoids the messy permission issues with Docker entirely**.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
alias docker_ocrmypdf='docker run --rm -i jbarlow83/ocrmypdf-alpine'
|
||||
docker_ocrmypdf --version # runs docker version
|
||||
docker_ocrmypdf - - <input.pdf >output.pdf
|
||||
|
||||
Or in the wonderful `fish shell <https://fishshell.com/>`__:
|
||||
|
||||
.. code-block:: fish
|
||||
|
||||
alias docker_ocrmypdf 'docker run --rm jbarlow83/ocrmypdf-alpine'
|
||||
funcsave docker_ocrmypdf
|
||||
|
||||
Alternately, you could mount the local current working directory as a
|
||||
Docker volume:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
alias docker_ocrmypdf='docker run --rm -i --user "$(id -u):$(id -g)" --workdir /data -v "$PWD:/data" jbarlow83/ocrmypdf-alpine'
|
||||
docker_ocrmypdf /data/input.pdf /data/output.pdf
|
||||
|
||||
Podman
|
||||
======
|
||||
|
||||
Especially if you use `Podman <https://podman.io/>`__ (or have SELinux enabled on your
|
||||
system), you may need to add ``--userns keep-id`` there, otherwise you may get access
|
||||
errors, because the user is otherwise not mapped to the same UID as on the host:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
alias podman_ocrmypdf='podman run --rm -i --user "$(id -u):$(id -g)" --userns keep-id --workdir /data -v "$PWD:/data" ocrmypdf'
|
||||
podman_ocrmypdf /data/input.pdf /data/output.pdf
|
||||
|
||||
If you use SELinux you may additionally need to add the ``:Z`` `suffix to the volume
|
||||
<https://docs.podman.io/en/stable/markdown/podman-run.1.html#volume-v-source-volume-host-dir-container-dir-options>`__
|
||||
or disable SELinux for the container using ``--security-opt label=disable``, which is
|
||||
suggested for system files as they should not be re-labelled. Please refer to the „Note”
|
||||
section at the end of the linked podman documentation for details.
|
||||
|
||||
.. _docker-lang-packs:
|
||||
|
||||
Adding languages to the Docker image
|
||||
====================================
|
||||
|
||||
By default the Docker image includes English, German, Simplified Chinese,
|
||||
French, Portuguese and Spanish, the most popular languages for OCRmyPDF
|
||||
users based on feedback. You may add other languages by creating a new
|
||||
Dockerfile based on the public one.
|
||||
|
||||
.. code-block:: dockerfile
|
||||
|
||||
FROM jbarlow83/ocrmypdf
|
||||
|
||||
# Example: add Italian
|
||||
RUN apt install tesseract-ocr-ita
|
||||
|
||||
To install language packs (training data) such as the
|
||||
`tessdata_best <https://github.com/tesseract-ocr/tessdata_best>`_ suite or
|
||||
custom data, you first need to determine the version of Tesseract data files, which
|
||||
may differ from the Tesseract program version. Use this command to determine the data
|
||||
file version:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run -i --rm --entrypoint /bin/ls jbarlow83/ocrmypdf /usr/share/tesseract-ocr
|
||||
|
||||
As of 2021, the data file version is probably ``4.00``.
|
||||
|
||||
You can then add new data with either a Dockerfile:
|
||||
|
||||
.. code-block:: dockerfile
|
||||
|
||||
FROM jbarlow83/ocrmypdf:{TAG}
|
||||
|
||||
# Example: add a tessdata_best file
|
||||
COPY chi_tra_vert.traineddata /usr/share/tesseract-ocr/<data version>/tessdata/
|
||||
|
||||
When creating your own image, you should always pin a specific version of the
|
||||
OCRmyPDF Docker image. This ensures that your image will not break when a new
|
||||
version of OCRmyPDF is released.
|
||||
|
||||
Alternately, you can copy training data into a Docker container as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker cp mycustomtraining.traineddata name_of_container:/usr/share/tesseract-ocr/<tesseract version>/tessdata/
|
||||
|
||||
Extending the Docker image
|
||||
==========================
|
||||
|
||||
You can extend the Docker image with your own customizations, similar to the way
|
||||
it is extended to add language packs.
|
||||
|
||||
Note that the Docker image is subject to change at any time. For example, the base
|
||||
image may be updated to a newer version of Ubuntu or Debian. Such changes will be
|
||||
noted in the release notes but might occur at minor versions releases, unless the
|
||||
way a "casual" user of the Docker image is affected.
|
||||
|
||||
If you extend the Docker image, you should pin a specific version of the OCRmyPDF
|
||||
Docker image.
|
||||
|
||||
Executing the test suite
|
||||
========================
|
||||
|
||||
The OCRmyPDF test suite is installed with image. To run it:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run --rm --entrypoint python jbarlow83/ocrmypdf -m pytest
|
||||
|
||||
Accessing the shell
|
||||
===================
|
||||
|
||||
To use the shell in the Docker image:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run -it --entrypoint sh jbarlow83/ocrmypdf
|
||||
|
||||
Using the OCRmyPDF web service wrapper
|
||||
======================================
|
||||
|
||||
The OCRmyPDF Docker image includes an example, barebones HTTP web
|
||||
service. The webservice may be launched as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run --entrypoint python -p 5000:5000 jbarlow83/ocrmypdf webservice.py
|
||||
|
||||
We omit the ``--rm`` parameter so that the container will not be
|
||||
automatically deleted when it exits.
|
||||
|
||||
This will configure the machine to listen on port 5000. On Linux machines
|
||||
this is port 5000 of localhost. On macOS or Windows machines running
|
||||
Docker, this is port 5000 of the virtual machine that runs your Docker
|
||||
images. You can find its IP address using the command ``docker-machine ip``.
|
||||
|
||||
Unlike command line usage this program will open a socket and wait for
|
||||
connections.
|
||||
|
||||
.. warning::
|
||||
|
||||
The OCRmyPDF web service wrapper is intended for demonstration or
|
||||
development. It provides no security, no authentication, no
|
||||
protection against denial of service attacks, and no load balancing.
|
||||
The default Flask WSGI server is used, which is intended for
|
||||
development only. The server is single-threaded and so can respond to
|
||||
only one client at a time. While running OCR, it cannot respond to
|
||||
any other clients.
|
||||
|
||||
Clients must keep their open connection while waiting for OCR to
|
||||
complete. This may entail setting a long timeout; this interface is more
|
||||
useful for internal HTTP API calls.
|
||||
|
||||
Unlike the rest of OCRmyPDF, this web service is licensed under the
|
||||
Affero GPLv3 (AGPLv3) since Ghostscript is also licensed in this way.
|
||||
|
||||
In addition to the above, please read our
|
||||
:ref:`general remarks on using OCRmyPDF as a service <ocr-service>`.
|
||||
Reference in New Issue
Block a user