Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a371655052 | ||
|
|
a6ce35b13a | ||
|
|
45added738 | ||
|
|
6e20439c91 | ||
|
|
72e056436c | ||
|
|
e02ba19097 | ||
|
|
d3b858f994 | ||
|
|
19045c4f21 | ||
|
|
f4d89fe6cc | ||
|
|
ab85c0f5a9 | ||
|
|
32693b683d | ||
|
|
b5dc276ba1 | ||
|
|
7c38c71794 | ||
|
|
a80e7a127b | ||
|
|
cf3309555f | ||
|
|
f80dd0d86a | ||
|
|
1ba2bce486 | ||
|
|
050dd1f5a8 | ||
|
|
e44a57aec0 | ||
|
|
d94d2671c3 | ||
|
|
5124daa79f | ||
|
|
7293847da7 | ||
|
|
59fd0ac587 | ||
|
|
90619b308c | ||
|
|
d0d49ce989 | ||
|
|
bf0224faa4 | ||
|
|
ae2f8ed8f1 | ||
|
|
14ac9b0560 | ||
|
|
dbe6148d41 | ||
|
|
36d4c2dbbc | ||
|
|
adbffb7bd9 | ||
|
|
05ecb6ca46 | ||
|
|
0a7b60cda5 | ||
|
|
5f211ecf6f | ||
|
|
417ee067a2 | ||
|
|
c4649dabef | ||
|
|
6eadd65dfb | ||
|
|
e8ed510543 | ||
|
|
22d35c199d | ||
|
|
5a82ad63c9 | ||
|
|
4769a6c50b | ||
|
|
9004009adc | ||
|
|
11221f9912 |
@@ -59,6 +59,7 @@ RUN add-apt-repository -y ppa:alex-p/tesseract-ocr-devel
|
|||||||
|
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
ghostscript \
|
ghostscript \
|
||||||
|
fonts-droid-fallback \
|
||||||
jbig2dec \
|
jbig2dec \
|
||||||
img2pdf \
|
img2pdf \
|
||||||
libsm6 libxext6 libxrender-dev \
|
libsm6 libxext6 libxrender-dev \
|
||||||
|
|||||||
@@ -0,0 +1,51 @@
|
|||||||
|
name: General issues
|
||||||
|
description: Installation, packages, dependencies, "nothing works", test suite failures...
|
||||||
|
title: "[Bug]: "
|
||||||
|
labels: ["bug", "triage"]
|
||||||
|
assignees:
|
||||||
|
- jbarlow83
|
||||||
|
body:
|
||||||
|
- type: markdown
|
||||||
|
attributes:
|
||||||
|
value: |
|
||||||
|
Thanks for taking the time to fill out this bug report!
|
||||||
|
- type: textarea
|
||||||
|
id: what-happened
|
||||||
|
attributes:
|
||||||
|
label: What were you trying to do?
|
||||||
|
description: Also tell us, what did you expect to happen?
|
||||||
|
placeholder: Tell us what you see!
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
|
- type: dropdown
|
||||||
|
id: packaging-system
|
||||||
|
attributes:
|
||||||
|
label: Where are you installing from?
|
||||||
|
multiple: true
|
||||||
|
options:
|
||||||
|
- PyPI (pip, poetry, pipx, etc.)
|
||||||
|
- Linux package manager (apt, dnf, etc.)
|
||||||
|
- Wndows package manager (chocolatey, etc.)
|
||||||
|
- Homebrew
|
||||||
|
- Docker container
|
||||||
|
- Ubuntu snap
|
||||||
|
- Conda
|
||||||
|
- source build
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
|
- type: dropdown
|
||||||
|
id: operating-system
|
||||||
|
attributes:
|
||||||
|
label: What operating system are you working on?
|
||||||
|
multiple: true
|
||||||
|
options:
|
||||||
|
- Linux
|
||||||
|
- Windows
|
||||||
|
- macOS
|
||||||
|
- BSD
|
||||||
|
- type: textarea
|
||||||
|
id: logs
|
||||||
|
attributes:
|
||||||
|
label: Relevant log output
|
||||||
|
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
|
||||||
|
render: plain text
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
name: Problem with specific file
|
||||||
|
description: Something went wrong while trying to OCR a specific file
|
||||||
|
title: "[Bug]: "
|
||||||
|
labels: ["bug", "triage"]
|
||||||
|
assignees:
|
||||||
|
- jbarlow83
|
||||||
|
body:
|
||||||
|
- type: markdown
|
||||||
|
attributes:
|
||||||
|
value: |
|
||||||
|
Thanks for taking the time to describe this issue with a particular file.
|
||||||
|
- type: textarea
|
||||||
|
id: what-happened
|
||||||
|
attributes:
|
||||||
|
label: Describe the bug
|
||||||
|
description: A clear and concise description of what the bug is.
|
||||||
|
placeholder: Tell us what you see!
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
|
- type: textarea
|
||||||
|
id: reproduce
|
||||||
|
attributes:
|
||||||
|
label: Steps to reproduce
|
||||||
|
description: Please include steps to reproduce
|
||||||
|
value: |
|
||||||
|
1. Run ocrmypdf -v1 ...arguments... input.pdf output.pdf
|
||||||
|
2. Open output.pdf
|
||||||
|
3. ...
|
||||||
|
render: plain text
|
||||||
|
- type: textarea
|
||||||
|
id: files
|
||||||
|
attributes:
|
||||||
|
label: Files
|
||||||
|
description: Please attach the input and output files, or any screenshots that may be helpful.
|
||||||
|
placeholder: Drag and drop files here
|
||||||
|
- type: dropdown
|
||||||
|
id: packaging-system
|
||||||
|
attributes:
|
||||||
|
label: How did you download and install the software?
|
||||||
|
multiple: true
|
||||||
|
options:
|
||||||
|
- PyPI (pip, poetry, pipx, etc.)
|
||||||
|
- Linux package manager (apt, dnf, etc.)
|
||||||
|
- Windows package manager (chocolatey, etc.)
|
||||||
|
- Homebrew
|
||||||
|
- Docker container
|
||||||
|
- Ubuntu snap
|
||||||
|
- Conda
|
||||||
|
- source build
|
||||||
|
- type: input
|
||||||
|
id: version
|
||||||
|
attributes:
|
||||||
|
label: OCRmyPDF version
|
||||||
|
description: Paste "ocrmypdf --version" here
|
||||||
|
placeholder: ocrmypdf --version
|
||||||
|
- type: textarea
|
||||||
|
id: logs
|
||||||
|
attributes:
|
||||||
|
label: Relevant log output
|
||||||
|
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
|
||||||
|
placeholder: Run OCRmyPDF with verbosity `-v1` to get more detailed logging output.
|
||||||
|
render: plain text
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
name: Feature request
|
||||||
|
description: Suggest an idea for this project
|
||||||
|
title: "[Feature]: "
|
||||||
|
labels: ["enhancement", "triage"]
|
||||||
|
assignees:
|
||||||
|
- jbarlow83
|
||||||
|
body:
|
||||||
|
- type: textarea
|
||||||
|
id: feature
|
||||||
|
attributes:
|
||||||
|
label: Describe the proposed feature
|
||||||
|
description: A clear and concise description of what the desired is.
|
||||||
@@ -1,23 +0,0 @@
|
|||||||
---
|
|
||||||
name: Feature request
|
|
||||||
about: Suggest an idea for this project
|
|
||||||
title: ''
|
|
||||||
labels: ''
|
|
||||||
assignees: ''
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Is your feature request related to a problem? Please describe.**
|
|
||||||
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
|
|
||||||
|
|
||||||
**Describe the solution you'd like**
|
|
||||||
A clear and concise description of what you want to happen.
|
|
||||||
|
|
||||||
**Describe alternatives you've considered**
|
|
||||||
A clear and concise description of any alternative solutions or features you've considered. Please include the versions of OCRmyPDF and other supporting programs (Tesseract OCR, Ghostscript) - maybe an alternative already exists in a newer version.
|
|
||||||
|
|
||||||
**Example file**
|
|
||||||
If your issue concerns how OCRmyPDF processes certain files, and please provide an example file that helps illustrate how OCRmyPDF's output could be improve. You could also look in ``tests/resources`` and see if any of those files demonstrates your issue.
|
|
||||||
|
|
||||||
**Additional context**
|
|
||||||
Add any other context or screenshots about the feature request here.
|
|
||||||
@@ -1,33 +0,0 @@
|
|||||||
---
|
|
||||||
name: General issues
|
|
||||||
about: Installation, packages, dependencies, "nothing works", test suite failures...
|
|
||||||
title: "[BUG]"
|
|
||||||
labels: ''
|
|
||||||
assignees: ''
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Describe the bug**
|
|
||||||
What's the problem?
|
|
||||||
|
|
||||||
**To Reproduce**
|
|
||||||
Steps to reproduce the behavior.
|
|
||||||
|
|
||||||
**Expected behavior**
|
|
||||||
What did you expected to happen?
|
|
||||||
|
|
||||||
**Screenshots**
|
|
||||||
If applicable, add screenshots to help explain your problem.
|
|
||||||
|
|
||||||
**System (please complete the following information):**
|
|
||||||
- OS:
|
|
||||||
- Python version:
|
|
||||||
- OCRmyPDF version:
|
|
||||||
- Platform: x64 or ARM
|
|
||||||
|
|
||||||
**Installation**
|
|
||||||
How did you install OCRmyPDF? Did you install it from your operating system's
|
|
||||||
package manager, or using pip?
|
|
||||||
|
|
||||||
**Additional context**
|
|
||||||
Add any other context about the problem here.
|
|
||||||
@@ -1,40 +0,0 @@
|
|||||||
---
|
|
||||||
name: Problem with specific file
|
|
||||||
about: Something went wrong while trying to OCR a specific file
|
|
||||||
title: "[BUG]"
|
|
||||||
labels: ''
|
|
||||||
assignees: ''
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Describe the bug**
|
|
||||||
A clear and concise description of what the bug is.
|
|
||||||
|
|
||||||
**To Reproduce**
|
|
||||||
What command line or API call were you trying to run?
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ocrmypdf ...arguments... input.pdf output.pdf
|
|
||||||
```
|
|
||||||
|
|
||||||
Run with verbosity or higher `-v1` to see more detailed logging. This information may be helpful.
|
|
||||||
|
|
||||||
**Example file**
|
|
||||||
If your issue is a problem that affects only certain files, and we will require an input file (PDF or image) that demonstrates your issue.
|
|
||||||
|
|
||||||
Please provide an input file with no personal or confidential information. At your option you may [GPG-encrypt the file](https://github.com/ocrmypdf/OCRmyPDF/wiki) for OCRmyPDF's author only.
|
|
||||||
|
|
||||||
Links to files hosted elsewhere are perfectly acceptable. You could also look in ``tests/resources`` and see if any of those files reproduce your issue.
|
|
||||||
|
|
||||||
*(Issues without example files usually cannot be resolved. It's like reporting an issue against a web browser without providing a URL.)*
|
|
||||||
|
|
||||||
**Expected behavior**
|
|
||||||
A clear and concise description of what you expected to happen.
|
|
||||||
|
|
||||||
**Screenshots**
|
|
||||||
If applicable, add screenshots to help explain your problem.
|
|
||||||
|
|
||||||
**System**
|
|
||||||
- OS: [e.g. Linux, Windows, macOS]
|
|
||||||
- OCRmyPDF Version: ``ocrmypdf --version``
|
|
||||||
- How did you install ocrmypdf? Did you use a system package manager, `pip`, or a Docker image?
|
|
||||||
@@ -5,7 +5,7 @@ name: Test and deploy
|
|||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
branches:
|
branches:
|
||||||
- master
|
- main
|
||||||
- ci
|
- ci
|
||||||
- release/*
|
- release/*
|
||||||
- feature/*
|
- feature/*
|
||||||
@@ -249,12 +249,38 @@ jobs:
|
|||||||
name: artifact
|
name: artifact
|
||||||
path: dist
|
path: dist
|
||||||
|
|
||||||
- uses: pypa/gh-action-pypi-publish@master
|
- uses: pypa/gh-action-pypi-publish@release/v1
|
||||||
with:
|
with:
|
||||||
user: __token__
|
user: __token__
|
||||||
password: ${{ secrets.TOKEN_PYPI }}
|
password: ${{ secrets.TOKEN_PYPI }}
|
||||||
# repository_url: https://test.pypi.org/legacy/
|
# repository_url: https://test.pypi.org/legacy/
|
||||||
|
|
||||||
|
create_release:
|
||||||
|
name: Create GitHub release
|
||||||
|
needs: [wheel_sdist_linux, test_linux, test_macos, test_windows]
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/v')
|
||||||
|
permissions:
|
||||||
|
# Required to create a release
|
||||||
|
contents: write
|
||||||
|
steps:
|
||||||
|
- uses: actions/download-artifact@v3
|
||||||
|
with:
|
||||||
|
name: artifact
|
||||||
|
path: dist
|
||||||
|
|
||||||
|
- name: Create Release
|
||||||
|
id: create-release
|
||||||
|
uses: shogo82148/actions-create-release@v1
|
||||||
|
|
||||||
|
- name: Upload Assets
|
||||||
|
uses: shogo82148/actions-upload-release-asset@v1
|
||||||
|
with:
|
||||||
|
upload_url: ${{ steps.create-release.outputs.upload_url }}
|
||||||
|
asset_path: |
|
||||||
|
./dist/*.whl
|
||||||
|
./dist/*.tar.gz
|
||||||
|
|
||||||
docker:
|
docker:
|
||||||
name: Build Docker images
|
name: Build Docker images
|
||||||
needs: [wheel_sdist_linux, test_linux, test_macos, test_windows]
|
needs: [wheel_sdist_linux, test_linux, test_macos, test_windows]
|
||||||
@@ -264,9 +290,9 @@ jobs:
|
|||||||
- name: Set image tag to release or branch
|
- name: Set image tag to release or branch
|
||||||
run: echo "DOCKER_IMAGE_TAG=${GITHUB_REF##*/}" >> $GITHUB_ENV
|
run: echo "DOCKER_IMAGE_TAG=${GITHUB_REF##*/}" >> $GITHUB_ENV
|
||||||
|
|
||||||
- name: If master, set to latest
|
- name: If main, set to latest
|
||||||
run: echo 'DOCKER_IMAGE_TAG=latest' >> $GITHUB_ENV
|
run: echo 'DOCKER_IMAGE_TAG=latest' >> $GITHUB_ENV
|
||||||
if: env.DOCKER_IMAGE_TAG == 'master'
|
if: env.DOCKER_IMAGE_TAG == 'main'
|
||||||
|
|
||||||
- name: Set Docker Hub repository to username
|
- name: Set Docker Hub repository to username
|
||||||
run: echo "DOCKER_REPOSITORY=jbarlow83" >> $GITHUB_ENV
|
run: echo "DOCKER_REPOSITORY=jbarlow83" >> $GITHUB_ENV
|
||||||
|
|||||||
@@ -41,6 +41,7 @@ Files:
|
|||||||
tests/resources/trivial.pdf
|
tests/resources/trivial.pdf
|
||||||
tests/resources/truetype_font_nomapping.pdf
|
tests/resources/truetype_font_nomapping.pdf
|
||||||
tests/resources/type3_font_nomapping.pdf
|
tests/resources/type3_font_nomapping.pdf
|
||||||
|
misc/screencast/*
|
||||||
Copyright: (C) 2022 James R. Barlow
|
Copyright: (C) 2022 James R. Barlow
|
||||||
License: CC-BY-SA-4.0
|
License: CC-BY-SA-4.0
|
||||||
|
|
||||||
|
|||||||
@@ -42,6 +42,8 @@ ocrmypdf # it's a scriptable command line program
|
|||||||
- Scales properly to handle files with thousands of pages
|
- Scales properly to handle files with thousands of pages
|
||||||
- Battle-tested on millions of PDFs
|
- Battle-tested on millions of PDFs
|
||||||
|
|
||||||
|
<img src="misc/screencast/demo.svg" alt="Demo of OCRmyPDF in a terminal session">
|
||||||
|
|
||||||
For details: please consult the [documentation](https://ocrmypdf.readthedocs.io/en/latest/).
|
For details: please consult the [documentation](https://ocrmypdf.readthedocs.io/en/latest/).
|
||||||
|
|
||||||
## Motivation
|
## Motivation
|
||||||
|
|||||||
+32
-6
@@ -10,7 +10,7 @@ Control of unpaper
|
|||||||
|
|
||||||
OCRmyPDF uses ``unpaper`` to provide the implementation of the
|
OCRmyPDF uses ``unpaper`` to provide the implementation of the
|
||||||
``--clean`` and ``--clean-final`` arguments.
|
``--clean`` and ``--clean-final`` arguments.
|
||||||
`unpaper <https://github.com/Flameeyes/unpaper/blob/master/doc/basic-concepts.md>`__
|
`unpaper <https://github.com/Flameeyes/unpaper/blob/main/doc/basic-concepts.md>`__
|
||||||
provides a variety of image processing filters to improve images.
|
provides a variety of image processing filters to improve images.
|
||||||
|
|
||||||
By default, OCRmyPDF uses only ``unpaper`` arguments that were found to
|
By default, OCRmyPDF uses only ``unpaper`` arguments that were found to
|
||||||
@@ -104,7 +104,7 @@ was requested, the preprocessed image layer will be inserted.
|
|||||||
If you want to adjust the amount of time spent on OCR, change
|
If you want to adjust the amount of time spent on OCR, change
|
||||||
``--tesseract-timeout``. You can also automatically skip images that
|
``--tesseract-timeout``. You can also automatically skip images that
|
||||||
exceed a certain number of megapixels with ``--skip-big``. (A 300 DPI,
|
exceed a certain number of megapixels with ``--skip-big``. (A 300 DPI,
|
||||||
8.5×11" page is 8.4 megapixels.)
|
8.5×11" page image is 8.4 megapixels.)
|
||||||
|
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
@@ -241,6 +241,17 @@ PDF.js viewer.
|
|||||||
|
|
||||||
This works in all versions of Tesseract.
|
This works in all versions of Tesseract.
|
||||||
|
|
||||||
|
Rendering and rasterizing options
|
||||||
|
=================================
|
||||||
|
|
||||||
|
.. versionadded:: 14.3.0
|
||||||
|
|
||||||
|
The ``--continue-on-soft-render-error`` option allows OCRmyPDF to
|
||||||
|
proceed if a page cannot be rasterized rendered. This is useful if you are
|
||||||
|
trying to get the best possible OCR from a PDF that is not well-formed,
|
||||||
|
and you are willing to accept some pages that may not visually match the
|
||||||
|
input, and that may not OCR well.
|
||||||
|
|
||||||
Return code policy
|
Return code policy
|
||||||
==================
|
==================
|
||||||
|
|
||||||
@@ -299,16 +310,31 @@ stable user interface. They may be imported from
|
|||||||
- The program was interrupted by pressing Ctrl+C.
|
- The program was interrupted by pressing Ctrl+C.
|
||||||
|
|
||||||
|
|
||||||
|
.. _tmpdir:
|
||||||
|
|
||||||
|
Changing temporary storage location
|
||||||
|
===================================
|
||||||
|
|
||||||
|
OCRmyPDF generates many temporary files during processing.
|
||||||
|
|
||||||
|
To change where temporary files are stored, change the ``TMPDIR``
|
||||||
|
environment variable for ocrmypdf's environment. (Python's
|
||||||
|
``tempfile.gettempdir()`` returns the root directory in which temporary
|
||||||
|
files will be stored.) For example, one could redirect ``TMPDIR`` to a
|
||||||
|
large RAM disk to avoid wear on HDD/SSD and potentially improve
|
||||||
|
performance.
|
||||||
|
|
||||||
|
On Windows, the ``TEMP`` environment variable is used instead.
|
||||||
|
|
||||||
Debugging the intermediate files
|
Debugging the intermediate files
|
||||||
================================
|
================================
|
||||||
|
|
||||||
OCRmyPDF normally saves its intermediate results to a temporary folder
|
OCRmyPDF normally saves its intermediate results to a temporary folder
|
||||||
and deletes this folder when it exits, whether it succeeded or failed.
|
and deletes this folder when it exits, whether it succeeded or failed.
|
||||||
|
|
||||||
If the ``-k`` argument is issued on the command line, OCRmyPDF will keep
|
If the ``--keep-temporary-files`` (``-k```) argument is issued on the
|
||||||
the temporary folder and print the location, whether it succeeded or
|
command line, OCRmyPDF will keep the temporary folder and print the location,
|
||||||
failed (provided the Python interpreter did not crash). An example
|
whether it succeeded or failed. An example message is:
|
||||||
message is:
|
|
||||||
|
|
||||||
.. code-block:: none
|
.. code-block:: none
|
||||||
|
|
||||||
|
|||||||
+15
-15
@@ -151,14 +151,14 @@ The watcher service is included in the OCRmyPDF Docker image. To run it:
|
|||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
docker run \
|
docker run \
|
||||||
-v <path to files to convert>:/input \
|
--volume <path to files to convert>:/input \
|
||||||
-v <path to store results>:/output \
|
--volume <path to store results>:/output \
|
||||||
-v <path to store processed originals>:/archive \
|
--volume <path to store processed originals>:/archive \
|
||||||
-e OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1 \
|
--env OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1 \
|
||||||
-e OCR_ON_SUCCESS_ARCHIVE=1 \
|
--env OCR_ON_SUCCESS_ARCHIVE=1 \
|
||||||
-e OCR_DESKEW=1 \
|
--env OCR_DESKEW=1 \
|
||||||
-e PYTHONUNBUFFERED=1 \
|
--env PYTHONUNBUFFERED=1 \
|
||||||
-it --entrypoint python3 \
|
--interactive --tty --entrypoint python3 \
|
||||||
jbarlow83/ocrmypdf \
|
jbarlow83/ocrmypdf \
|
||||||
watcher.py
|
watcher.py
|
||||||
|
|
||||||
@@ -170,13 +170,13 @@ original to ``/archive``. The parameters to this image are:
|
|||||||
:header: "Parameter", "Description"
|
:header: "Parameter", "Description"
|
||||||
:widths: 50, 50
|
:widths: 50, 50
|
||||||
|
|
||||||
"``-v <path to files to convert>:/input``", "Files placed in this location will be OCRed"
|
"``--volume <path to files to convert>:/input``", "Files placed in this location will be OCRed"
|
||||||
"``-v <path to store results>:/output``", "This is where OCRed files will be stored"
|
"``--volume <path to store results>:/output``", "This is where OCRed files will be stored"
|
||||||
"``-v <path to store processed originals>:/archive``", "Archive processed originals here"
|
"``--volume <path to store processed originals>:/archive``", "Archive processed originals here"
|
||||||
"``-e OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1``", "Define environment variable ``OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1`` to place files in the output in ``{output}/{year}/{month}/{filename}``"
|
"``--env OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1``", "Define environment variable ``OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1`` to place files in the output in ``{output}/{year}/{month}/{filename}``"
|
||||||
"``-e OCR_ON_SUCCESS_ARCHIVE=1``", "Define environment variable ``OCR_ON_SUCCESS_ARCHIVE`` to move processed originals"
|
"``--env OCR_ON_SUCCESS_ARCHIVE=1``", "Define environment variable ``OCR_ON_SUCCESS_ARCHIVE`` to move processed originals"
|
||||||
"``-e OCR_DESKEW=1``", "Define environment variable ``OCR_DESKEW`` to apply deskew to crooked input PDFs"
|
"``--env OCR_DESKEW=1``", "Define environment variable ``OCR_DESKEW`` to apply deskew to crooked input PDFs"
|
||||||
"``-e PYTHONBUFFERED=1``", "This will force ``STDOUT`` to be unbuffered and allow you to see messages in docker logs"
|
"``--env PYTHONBUFFERED=1``", "This will force ``STDOUT`` to be unbuffered and allow you to see messages in docker logs"
|
||||||
|
|
||||||
This service relies on polling to check for changes to the filesystem. It
|
This service relies on polling to check for changes to the filesystem. It
|
||||||
may not be suitable for some environments, such as filesystems shared on a
|
may not be suitable for some environments, such as filesystems shared on a
|
||||||
|
|||||||
@@ -0,0 +1,87 @@
|
|||||||
|
.. _ocr-service:
|
||||||
|
|
||||||
|
==================
|
||||||
|
Online deployments
|
||||||
|
==================
|
||||||
|
|
||||||
|
OCRmyPDF is designed to be used as a command line tool, but it can be
|
||||||
|
used in a web service. This document describes some considerations for
|
||||||
|
doing so.
|
||||||
|
|
||||||
|
A basic web service implementation is provided in the source code
|
||||||
|
repository, as ``misc/webservice.py``. It is only demonstration quality
|
||||||
|
and is not intended for production use.
|
||||||
|
|
||||||
|
OCRmyPDF is not designed for use as a public web service where a
|
||||||
|
malicious user could upload a chosen PDF. In particular, it is not
|
||||||
|
necessarily secure against PDF malware or PDFs that cause denial of
|
||||||
|
service. For further discussino of security, see :ref:`security`.
|
||||||
|
|
||||||
|
OCRmyPDF relies on Ghostscript, and therefore, if deployed
|
||||||
|
online one should be prepared to comply with Ghostscript's Affero GPL
|
||||||
|
license, and any other licenses.
|
||||||
|
|
||||||
|
Setting aside these concerns, a side effect of OCRmyPDF is that it may
|
||||||
|
incidentally sanitize PDFs containing certain types of malware. It
|
||||||
|
repairs the PDF with pikepdf/libqpdf, which could correct malformed PDF
|
||||||
|
structures that are part of an attack. When PDF/A output is selected
|
||||||
|
(the default), the input PDF is partially reconstructed by Ghostscript.
|
||||||
|
When ``--force-ocr`` is used, all pages are rasterized and reconverted
|
||||||
|
to PDF, which could remove malware in embedded images.
|
||||||
|
|
||||||
|
Limiting CPU usage
|
||||||
|
------------------
|
||||||
|
|
||||||
|
OCRmyPDF will attempt to use all available CPUs and storage, so
|
||||||
|
executing ``nice ocrmypdf`` or limiting the number of jobs with the
|
||||||
|
``--jobs`` argument may ensure the server remains responsive. Another option
|
||||||
|
would be to run OCRmyPDF jobs inside a Docker container, a virtual machine,
|
||||||
|
or a cloud instance, which can impose its own limits on CPU usage and be
|
||||||
|
terminated "from orbit" if it fails to complete.
|
||||||
|
|
||||||
|
Temporary storage requirements
|
||||||
|
------------------------------
|
||||||
|
|
||||||
|
OCRmyPDF will use a large amount of temporary storage for its work,
|
||||||
|
proportional to the total number of pixels needed to rasterize the PDF.
|
||||||
|
The raster image of a 8.5×11" color page at 300 DPI takes 25 MB
|
||||||
|
uncompressed; OCRmyPDF saves its intermediates as PNG, but that still
|
||||||
|
means it requires about 9 MB per intermediate based on average
|
||||||
|
compression ratios. Multiple intermediates per page are also required,
|
||||||
|
depending on the command line given. A rule of thumb would be to allow
|
||||||
|
100 MB of temporary storage per page in a file – meaning that a small
|
||||||
|
cloud servers or small VM partitions should be provisioned with plenty
|
||||||
|
of extra space, if say, a 500 page file might be sent.
|
||||||
|
|
||||||
|
To change the temporary directory, see :ref:`tmpdir`.
|
||||||
|
|
||||||
|
On Amazon Web Services or other cloud vendors, consider setting your
|
||||||
|
temporary directory to `empheral
|
||||||
|
storage <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html>`__.
|
||||||
|
|
||||||
|
Timeouts
|
||||||
|
--------
|
||||||
|
|
||||||
|
To prevent excessively long OCR jobs consider setting
|
||||||
|
``--tesseract-timeout`` and/or ``--skip-big`` arguments. ``--skip-big``
|
||||||
|
is particularly helpful if your PDFs include documents such as reports
|
||||||
|
on standard page sizes with large images attached - often large images
|
||||||
|
are not worth OCR'ing anyway.
|
||||||
|
|
||||||
|
Document management systems
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
If you are looking for a full document management system, consider
|
||||||
|
`paperless-ngx <https://github.com/paperless-ngx/paperless-ngx>`__,
|
||||||
|
which is a web application that uses OCRmyPDF to automatically OCR and
|
||||||
|
archive documents.
|
||||||
|
|
||||||
|
Commercial OCR alternatives
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
The author also provides professional services that include OCR and
|
||||||
|
building databases around PDFs, and is happy to provide consultation.
|
||||||
|
|
||||||
|
Abbyy Cloud OCR is viable commercial alternative with a web services
|
||||||
|
API. Amazon Textract, Google Cloud Vision, and Microsoft Azure
|
||||||
|
Computer Vision provide advanced OCR but have less PDF rendering capability.
|
||||||
@@ -18,7 +18,7 @@ work you're contemplating is already half-done in a development branch.
|
|||||||
Code style
|
Code style
|
||||||
==========
|
==========
|
||||||
|
|
||||||
We use PEP8, ``black`` for code formatting and ``isort`` for import sorting. The
|
We use PEP8, ``black`` for code formatting and ``ruff`` for everything else. The
|
||||||
settings for these programs are in ``pyproject.toml`` and ``setup.cfg``. Pull
|
settings for these programs are in ``pyproject.toml`` and ``setup.cfg``. Pull
|
||||||
requests should follow the style guide. One difference we use from "black" style
|
requests should follow the style guide. One difference we use from "black" style
|
||||||
is that strings shown to the user are always in double quotes (``"``) and strings
|
is that strings shown to the user are always in double quotes (``"``) and strings
|
||||||
|
|||||||
@@ -372,3 +372,17 @@ Some users may consider enabling lossy JBIG2. See: :ref:`jbig2-lossy`.
|
|||||||
|
|
||||||
Image processing and PDF/A conversion can also introduce lossy transformations
|
Image processing and PDF/A conversion can also introduce lossy transformations
|
||||||
to your PDF images, even when ``--optimize 1`` is in use.
|
to your PDF images, even when ``--optimize 1`` is in use.
|
||||||
|
|
||||||
|
|
||||||
|
Digitally signed PDFs
|
||||||
|
=====================
|
||||||
|
|
||||||
|
OCRmyPDF cannot preserve digital signatures in PDFs and also add to OCR to them.
|
||||||
|
By default, it will refuse to modify a signed PDF regardless of other settings. You can
|
||||||
|
override this behavior with ``--invalidate-digital-signatures``; as the name suggests,
|
||||||
|
any digital signatures will be invalidated.
|
||||||
|
|
||||||
|
OCRmyPDF cannot open documents that are encrypted with a digital certificate.
|
||||||
|
|
||||||
|
Versions of OCRmyPDF prior to 14.4.0 would invalidate existing digital signatures
|
||||||
|
without warning.
|
||||||
+2
-1
@@ -20,7 +20,6 @@ image processing and OCR to existing PDFs.
|
|||||||
introduction
|
introduction
|
||||||
release_notes
|
release_notes
|
||||||
installation
|
installation
|
||||||
optimizer
|
|
||||||
languages
|
languages
|
||||||
jbig2
|
jbig2
|
||||||
|
|
||||||
@@ -29,9 +28,11 @@ image processing and OCR to existing PDFs.
|
|||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
|
|
||||||
cookbook
|
cookbook
|
||||||
|
optimizer
|
||||||
docker
|
docker
|
||||||
advanced
|
advanced
|
||||||
batch
|
batch
|
||||||
|
cloud
|
||||||
performance
|
performance
|
||||||
pdfsecurity
|
pdfsecurity
|
||||||
errors
|
errors
|
||||||
|
|||||||
@@ -637,7 +637,7 @@ system-wide:
|
|||||||
|
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
git clone -b master https://github.com/ocrmypdf/OCRmyPDF.git
|
git clone -b main https://github.com/ocrmypdf/OCRmyPDF.git
|
||||||
python3 -m venv venv
|
python3 -m venv venv
|
||||||
source venv/bin/activate
|
source venv/bin/activate
|
||||||
cd OCRmyPDF
|
cd OCRmyPDF
|
||||||
@@ -664,7 +664,7 @@ To install all of the development and test requirements:
|
|||||||
|
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
git clone -b master https://github.com/ocrmypdf/OCRmyPDF.git
|
git clone -b main https://github.com/ocrmypdf/OCRmyPDF.git
|
||||||
python -m venv
|
python -m venv
|
||||||
source venv/bin/activate
|
source venv/bin/activate
|
||||||
cd OCRmyPDF
|
cd OCRmyPDF
|
||||||
|
|||||||
+7
-7
@@ -12,7 +12,7 @@ OCRmyPDF uses Tesseract for OCR, and relies on its language packs for all langua
|
|||||||
On most platforms, English is installed with Tesseract by default, but not always.
|
On most platforms, English is installed with Tesseract by default, but not always.
|
||||||
|
|
||||||
Tesseract supports `most
|
Tesseract supports `most
|
||||||
languages <https://github.com/tesseract-ocr/tesseract/blob/master/doc/tesseract.1.asc#languages>`__.
|
languages <https://github.com/tesseract-ocr/tesseract/blob/main/doc/tesseract.1.asc#languages>`__.
|
||||||
Languages are identified by standardized three-letter codes (called ISO 639-2 Alpha-3).
|
Languages are identified by standardized three-letter codes (called ISO 639-2 Alpha-3).
|
||||||
Tesseract's documentation also lists the three-letter code for your language.
|
Tesseract's documentation also lists the three-letter code for your language.
|
||||||
Some are anglicized, e.g. Spanish is ``spa`` rather than ``esp``, while others
|
Some are anglicized, e.g. Spanish is ``spa`` rather than ``esp``, while others
|
||||||
@@ -70,13 +70,13 @@ This enables these languages for all packages (e.g. including aspell).
|
|||||||
|
|
||||||
# Display a list of all Tesseract language packs
|
# Display a list of all Tesseract language packs
|
||||||
equery uses app-text/tessdata_fast
|
equery uses app-text/tessdata_fast
|
||||||
|
|
||||||
# Add English and German language support for Tesseract only
|
# Add English and German language support for Tesseract only
|
||||||
echo 'app-text/tessdata_fast l10n_de l10n_en' >> /etc/portage/package.use
|
echo 'app-text/tessdata_fast l10n_de l10n_en' >> /etc/portage/package.use
|
||||||
|
|
||||||
# Add global English and German language support (the `l10n_` from equery has to be omitted)
|
# Add global English and German language support (the `l10n_` from equery has to be omitted)
|
||||||
echo L10N="de en" >> /etc/portage/make.conf
|
echo L10N="de en" >> /etc/portage/make.conf
|
||||||
|
|
||||||
# update system to reflect changed USE flags
|
# update system to reflect changed USE flags
|
||||||
emerge --update --deep --newuse @world
|
emerge --update --deep --newuse @world
|
||||||
|
|
||||||
@@ -101,7 +101,7 @@ derived Docker image as
|
|||||||
Windows users
|
Windows users
|
||||||
=============
|
=============
|
||||||
|
|
||||||
The Tesseract installer provided by Chocolatey currently includes only English language.
|
The Tesseract installer provided by Chocolatey currently includes only English language.
|
||||||
To install other languages, download the respective language pack (``.traineddata`` file)
|
To install other languages, download the respective language pack (``.traineddata`` file)
|
||||||
from https://github.com/tesseract-ocr/tessdata/ and place it in
|
from https://github.com/tesseract-ocr/tessdata/ and place it in
|
||||||
``C:\\Program Files\\Tesseract-OCR\\tessdata`` (or wherever Tesseract OCR is installed).
|
``C:\\Program Files\\Tesseract-OCR\\tessdata`` (or wherever Tesseract OCR is installed).
|
||||||
|
|||||||
+30
-9
@@ -13,14 +13,33 @@ tuned. Optimization occurs after OCR, and only if OCR succeeded. It does not
|
|||||||
perform other possible optimizations such as deduplicating resources,
|
perform other possible optimizations such as deduplicating resources,
|
||||||
consolidating fonts, simplifying vector drawings, or anything of that nature.
|
consolidating fonts, simplifying vector drawings, or anything of that nature.
|
||||||
|
|
||||||
Optimization ranges from ``-O0`` through ``-O3``, where ``0`` disables
|
.. list-table:: Title
|
||||||
optimization and ``3`` implements all options. ``1``, the default, performs only
|
:widths: 33 6 60
|
||||||
safe and lossless optimizations. (This is similar to GCC's optimization
|
:header-rows: 1
|
||||||
parameter.) The exact type of optimizations performed will vary over time.
|
|
||||||
|
|
||||||
PDF optimization requires third-party, optional tools for certain optimizations.
|
* - Optimization level
|
||||||
If these are not installed or cannot be found by OCRmyPDF, optimization will not
|
- Shorthand
|
||||||
be as good.
|
- Description
|
||||||
|
* - ``--optimize 0``
|
||||||
|
- ``-O0``
|
||||||
|
- Disable most optimizations.
|
||||||
|
* - ``--optimize 1`` (default)
|
||||||
|
- ``-O1``
|
||||||
|
- Safe and lossless optimizations.
|
||||||
|
* - ``--optimize 2``
|
||||||
|
- ``-O2``
|
||||||
|
- Safe and lossy optimizations.
|
||||||
|
* - ``--optimize 3``
|
||||||
|
- ``-O3``
|
||||||
|
- Aggressive lossy optimizations.
|
||||||
|
|
||||||
|
The exact type of optimizations performed will vary over time, and depend on
|
||||||
|
the availability of third-party tools.
|
||||||
|
|
||||||
|
Despite optimizations, OCRmyPDF might still increase the overall file size,
|
||||||
|
since it must embed information about the recognized text, and depending on the
|
||||||
|
settings chosen, may not be able to represent the output file as compactly as
|
||||||
|
the input file.
|
||||||
|
|
||||||
Optimizations that always occurs
|
Optimizations that always occurs
|
||||||
================================
|
================================
|
||||||
@@ -37,12 +56,14 @@ Fast web view
|
|||||||
OCRmyPDF automatically optimizes PDFs for "fast web view" in Adobe Acrobat's
|
OCRmyPDF automatically optimizes PDFs for "fast web view" in Adobe Acrobat's
|
||||||
parlance, or equivalently, linearizes PDFs so that the resources they reference
|
parlance, or equivalently, linearizes PDFs so that the resources they reference
|
||||||
are presented in the order a viewer needs them for sequential display. This
|
are presented in the order a viewer needs them for sequential display. This
|
||||||
reduces the latency of viewing a PDF both online and from local storage. This
|
reduces the latency of viewing a PDF both online and from local storage, in
|
||||||
actually slightly increases the file size.
|
exchange for a slight increase in file size.
|
||||||
|
|
||||||
To disable this optimization and all others, use ``ocrmypdf --optimize 0 ...``
|
To disable this optimization and all others, use ``ocrmypdf --optimize 0 ...``
|
||||||
or the shorthand ``-O0``.
|
or the shorthand ``-O0``.
|
||||||
|
|
||||||
|
Adobe Acrobat might not report the file as being "fast web view".
|
||||||
|
|
||||||
Lossless optimizations
|
Lossless optimizations
|
||||||
======================
|
======================
|
||||||
|
|
||||||
|
|||||||
+6
-88
@@ -29,13 +29,9 @@ attack vectors.
|
|||||||
In short, PDFs `may contain
|
In short, PDFs `may contain
|
||||||
viruses <https://security.stackexchange.com/questions/64052/can-a-pdf-file-contain-a-virus>`__.
|
viruses <https://security.stackexchange.com/questions/64052/can-a-pdf-file-contain-a-virus>`__.
|
||||||
|
|
||||||
This
|
If you do not trust a PDF or its source, do not open it or use OCRmyPDF
|
||||||
`article <https://theinvisiblethings.blogspot.ca/2013/02/converting-untrusted-pdfs-into-trusted.html>`__
|
on it. Consider using a Docker container or virtual machine to isolate
|
||||||
describes a high-paranoia method which allows potentially hostile PDFs
|
an untrusted PDF from your system.
|
||||||
to be viewed and rasterized safely in a disposable virtual machine. A
|
|
||||||
trusted PDF created in this manner is converted to images and loses all
|
|
||||||
information making it searchable and losing all compression. OCRmyPDF
|
|
||||||
could be used to restore searchability.
|
|
||||||
|
|
||||||
How OCRmyPDF processes PDFs
|
How OCRmyPDF processes PDFs
|
||||||
===========================
|
===========================
|
||||||
@@ -43,11 +39,11 @@ How OCRmyPDF processes PDFs
|
|||||||
OCRmyPDF must open and interpret your PDF in order to insert an OCR
|
OCRmyPDF must open and interpret your PDF in order to insert an OCR
|
||||||
layer. First, it runs all PDFs through
|
layer. First, it runs all PDFs through
|
||||||
`pikepdf <https://github.com/pikepdf/pikepdf>`__, a library based on
|
`pikepdf <https://github.com/pikepdf/pikepdf>`__, a library based on
|
||||||
`qpdf <https://github.com/qpdf/qpdf>`__, a program that repairs PDFs
|
`QPDF <https://github.com/qpdf/qpdf>`__, a program that repairs PDFs
|
||||||
with syntax errors. This is done because, in the author's experience, a
|
with syntax errors. This is done because, in the author's experience, a
|
||||||
significant number of PDFs in the wild, especially those created by
|
significant number of PDFs in the wild, especially those created by
|
||||||
scanners, are not well-formed files. qpdf makes it more likely that
|
scanners, are not well-formed files. QPDF makes it more likely that
|
||||||
OCRmyPDF will succeed, but offers no security guarantees. qpdf is also
|
OCRmyPDF will succeed, but offers no security guarantees. QPDF is also
|
||||||
used to split the PDF into single page PDFs.
|
used to split the PDF into single page PDFs.
|
||||||
|
|
||||||
Finally, OCRmyPDF rasterizes each page of the PDF using
|
Finally, OCRmyPDF rasterizes each page of the PDF using
|
||||||
@@ -58,84 +54,6 @@ into the existing PDF or it may essentially reconstruct ("re-fry") a
|
|||||||
visually identical PDF that may be quite different at the binary level.
|
visually identical PDF that may be quite different at the binary level.
|
||||||
That said, OCRmyPDF is not a tool designed for sanitizing PDFs.
|
That said, OCRmyPDF is not a tool designed for sanitizing PDFs.
|
||||||
|
|
||||||
.. _ocr-service:
|
|
||||||
|
|
||||||
Using OCRmyPDF online or as a service
|
|
||||||
=====================================
|
|
||||||
|
|
||||||
OCRmyPDF is not designed for use as a public web service where a
|
|
||||||
malicious user could upload a chosen PDF. In particular, it is not
|
|
||||||
necessarily secure against PDF malware or PDFs that cause denial of
|
|
||||||
service. OCRmyPDF relies on Ghostscript, and therefore, if deployed
|
|
||||||
online one should be prepared to comply with Ghostscript's Affero GPL
|
|
||||||
license, and any other licenses.
|
|
||||||
|
|
||||||
Setting aside these concerns, a side effect of OCRmyPDF is that it may
|
|
||||||
incidentally sanitize PDFs containing certain types of malware. It
|
|
||||||
repairs the PDF with pikepdf/libqpdf, which could correct malformed PDF
|
|
||||||
structures that are part of an attack. When PDF/A output is selected
|
|
||||||
(the default), the input PDF is partially reconstructed by Ghostscript.
|
|
||||||
When ``--force-ocr`` is used, all pages are rasterized and reconverted
|
|
||||||
to PDF, which could remove malware in embedded images.
|
|
||||||
|
|
||||||
OCRmyPDF should be relatively safe to use in a trusted intranet, with
|
|
||||||
some considerations:
|
|
||||||
|
|
||||||
Limiting CPU usage
|
|
||||||
------------------
|
|
||||||
|
|
||||||
OCRmyPDF will attempt to use all available CPUs and storage, so
|
|
||||||
executing ``nice ocrmypdf`` or limiting the number of jobs with the
|
|
||||||
``-j`` argument may ensure the server remains available. Another option
|
|
||||||
would be to run OCRmyPDF jobs inside a Docker container, a virtual machine,
|
|
||||||
or a cloud instance, which can impose its own limits on CPU usage and be
|
|
||||||
terminated "from orbit" if it fails to complete.
|
|
||||||
|
|
||||||
Temporary storage requirements
|
|
||||||
------------------------------
|
|
||||||
|
|
||||||
OCRmyPDF will use a large amount of temporary storage for its work,
|
|
||||||
proportional to the total number of pixels needed to rasterize the PDF.
|
|
||||||
The raster image of a 8.5×11" color page at 300 DPI takes 25 MB
|
|
||||||
uncompressed; OCRmyPDF saves its intermediates as PNG, but that still
|
|
||||||
means it requires about 9 MB per intermediate based on average
|
|
||||||
compression ratios. Multiple intermediates per page are also required,
|
|
||||||
depending on the command line given. A rule of thumb would be to allow
|
|
||||||
100 MB of temporary storage per page in a file – meaning that a small
|
|
||||||
cloud servers or small VM partitions should be provisioned with plenty
|
|
||||||
of extra space, if say, a 500 page file might be sent.
|
|
||||||
|
|
||||||
To check temporary storage usage on actual files, run
|
|
||||||
``ocrmypdf -k ...`` which will preserve and print the path to temporary
|
|
||||||
storage when the job is done.
|
|
||||||
|
|
||||||
To change where temporary files are stored, change the ``TMPDIR``
|
|
||||||
environment variable for ocrmypdf's environment. (Python's
|
|
||||||
``tempfile.gettempdir()`` returns the root directory in which temporary
|
|
||||||
files will be stored.) For example, one could redirect ``TMPDIR`` to a
|
|
||||||
large RAM disk to avoid wear on HDD/SSD and potentially improve
|
|
||||||
performance. On Amazon Web Services, ``TMPDIR`` can be set to `empheral
|
|
||||||
storage <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html>`__.
|
|
||||||
|
|
||||||
Timeouts
|
|
||||||
--------
|
|
||||||
|
|
||||||
To prevent excessively long OCR jobs consider setting
|
|
||||||
``--tesseract-timeout`` and/or ``--skip-big`` arguments. ``--skip-big``
|
|
||||||
is particularly helpful if your PDFs include documents such as reports
|
|
||||||
on standard page sizes with large images attached - often large images
|
|
||||||
are not worth OCR'ing anyway.
|
|
||||||
|
|
||||||
Commercial alternatives
|
|
||||||
-----------------------
|
|
||||||
|
|
||||||
The author also provides professional services that include OCR and
|
|
||||||
building databases around PDFs, and is happy to provide consultation.
|
|
||||||
|
|
||||||
Abbyy Cloud OCR is viable commercial alternative with a web services
|
|
||||||
API. Amazon Textract, Google Cloud Vision, and Microsoft Azure
|
|
||||||
Computer Vision provide advanced OCR but have less PDF rendering capability.
|
|
||||||
|
|
||||||
Password protection, digital signatures and certification
|
Password protection, digital signatures and certification
|
||||||
=========================================================
|
=========================================================
|
||||||
|
|
||||||
|
|||||||
@@ -28,6 +28,36 @@ tagged yet.
|
|||||||
|
|
||||||
.. |OCRmyPDF PyPI| image:: https://img.shields.io/pypi/v/ocrmypdf.svg
|
.. |OCRmyPDF PyPI| image:: https://img.shields.io/pypi/v/ocrmypdf.svg
|
||||||
|
|
||||||
|
v14.4.0
|
||||||
|
=======
|
||||||
|
|
||||||
|
- Digitally signed PDFs are now detected. If the PDF is signed, OCRmyPDF will
|
||||||
|
refuse to modify it. Previously, only encrypted PDFs were detected, not
|
||||||
|
those that were signed but not encrypted. :issue:`1040`
|
||||||
|
- In addition, `--invalidate-digital-signatures` can be used to override the
|
||||||
|
above behavior and modify the PDF anyway. :issue:`1040`
|
||||||
|
- tqdm progress bars replaced with "rich" progress bars. The rich library is
|
||||||
|
a new dependency. Certain APIs that used tqdm are now deprecated and will
|
||||||
|
be removed in the next major release.
|
||||||
|
- Improved integration with GitHub Releases. Thanks to @stumpylog.
|
||||||
|
|
||||||
|
v14.3.0
|
||||||
|
=======
|
||||||
|
|
||||||
|
- Renamed master branch to main.
|
||||||
|
- Improve PDF rasterization accuracy by using the ``-dPDFSTOPONERROR`` option
|
||||||
|
to Ghostscript. Use ``--continue-on-soft-render-error`` if you want to render
|
||||||
|
the PDF anyway. The plugin specification was adjusted to support this feature;
|
||||||
|
plugin authors may want to adapt PDF rasterizing and rendering
|
||||||
|
plugins. :issue:`1083`
|
||||||
|
- The calculated deskew angle is now recorded in the logged output. :issue:`1101`
|
||||||
|
- Metadata can now be unset by setting a metadata type such as ``--title`` to an
|
||||||
|
empty string. :issue:`1117,1059`
|
||||||
|
- Fixed random order of languages due to use of a set. This may have caused output
|
||||||
|
to vary when multiple languages were set for OCR. :issue:`1113`
|
||||||
|
- Clarified the optimization ratio reported in the log output.
|
||||||
|
- Documentation improvements.
|
||||||
|
|
||||||
v14.2.1
|
v14.2.1
|
||||||
=======
|
=======
|
||||||
|
|
||||||
|
|||||||
@@ -31,6 +31,7 @@ __ocrmypdf_arguments()
|
|||||||
--force-ocr (OCR documents that already have printable text)
|
--force-ocr (OCR documents that already have printable text)
|
||||||
--skip-text (skip OCR on any pages that already contain text)
|
--skip-text (skip OCR on any pages that already contain text)
|
||||||
--redo-ocr (redo OCR on any pages that seem to have OCR already)
|
--redo-ocr (redo OCR on any pages that seem to have OCR already)
|
||||||
|
--invalidate-digital-signatures (remove digital signatures from PDF)
|
||||||
--skip-big (skip OCR on pages larger than this many MPixels)
|
--skip-big (skip OCR on pages larger than this many MPixels)
|
||||||
--optimize (select optimization level)
|
--optimize (select optimization level)
|
||||||
--jpeg-quality (JPEG quality [0..100])
|
--jpeg-quality (JPEG quality [0..100])
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ complete -c ocrmypdf -l remove-vectors -d "don't send vector objects to OCR"
|
|||||||
complete -c ocrmypdf -s f -l force-ocr -d "OCR documents that already have printable text"
|
complete -c ocrmypdf -s f -l force-ocr -d "OCR documents that already have printable text"
|
||||||
complete -c ocrmypdf -s s -l skip-ocr -d "skip OCR on pages that text, otherwise try OCR"
|
complete -c ocrmypdf -s s -l skip-ocr -d "skip OCR on pages that text, otherwise try OCR"
|
||||||
complete -c ocrmypdf -l redo-ocr -d "redo OCR on any pages that seem to have OCR already"
|
complete -c ocrmypdf -l redo-ocr -d "redo OCR on any pages that seem to have OCR already"
|
||||||
|
complete -c ocrmypdf -l invalidate-digital-signatures -d "invalidate digital signatures and allow OCR to proceed"
|
||||||
|
|
||||||
complete -c ocrmypdf -s k -l keep-temporary-files -d "keep temporary files (debug)"
|
complete -c ocrmypdf -s k -l keep-temporary-files -d "keep temporary files (debug)"
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,31 @@
|
|||||||
|
<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->
|
||||||
|
|
||||||
|
To regenerate
|
||||||
|
=============
|
||||||
|
|
||||||
|
Using asciinema and svg-term (`npm install -g svg-term-cli`).
|
||||||
|
|
||||||
|
Create `~/.config/asciinema/config` to disable prompt.
|
||||||
|
|
||||||
|
```
|
||||||
|
[record]
|
||||||
|
|
||||||
|
command = fish --init-command 'alias fish_prompt="echo \>\ "'
|
||||||
|
```
|
||||||
|
|
||||||
|
Run asciinema
|
||||||
|
|
||||||
|
```
|
||||||
|
asciinema rec new_input.cast
|
||||||
|
```
|
||||||
|
|
||||||
|
Re-record faster version with fewer pauses
|
||||||
|
|
||||||
|
```
|
||||||
|
asciinema rec demo.cast -c "asciinema play new_input.cast --speed 2 --idle-time-limit 0.5"
|
||||||
|
```
|
||||||
|
|
||||||
|
Convert to SVG
|
||||||
|
```
|
||||||
|
svg-term --in=misc/screencast/demo.cast --out=misc/screencast/demo.svg --window
|
||||||
|
```
|
||||||
@@ -0,0 +1,65 @@
|
|||||||
|
{"version": 2, "width": 131, "height": 24, "timestamp": 1687247006, "env": {"SHELL": "/usr/bin/fish", "TERM": "xterm-256color"}}
|
||||||
|
[0.103649, "o", "\u001b[?2004h\u001b]7; \u0007"]
|
||||||
|
[0.104223, "o", "\u001b]0;fish \u0007\u001b[30m\u001b(B\u001b[m\r> \u001b[K\r\u001b[C\u001b[C"]
|
||||||
|
[0.604542, "o", "o\r\u001b[3C\b\u001b[38;2;255;0;0mo\r\u001b[3C\u001b[30m\u001b(B\u001b[m\u001b[38;2;85;85;85mcrmypdf multipage.pdf multipage_with_ocr.pdf\r\u001b[3C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[0.679571, "o", "\u001b[38;2;255;0;0mc\u001b[38;2;85;85;85mrmypdf multipage.pdf multipage_with_ocr.pdf\r\u001b[4C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[0.767271, "o", "\u001b[38;2;255;0;0mr\u001b[38;2;85;85;85mmypdf multipage.pdf multipage_with_ocr.pdf\r\u001b[5C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[0.814505, "o", "\u001b[38;2;255;0;0mm\u001b[38;2;85;85;85mypdf multipage.pdf multipage_with_ocr.pdf\r\u001b[6C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[0.938919, "o", "\u001b[38;2;255;0;0my\u001b[38;2;85;85;85mpdf multipage.pdf multipage_with_ocr.pdf\r\u001b[7C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[0.967347, "o", "\u001b[38;2;255;0;0mp\u001b[38;2;85;85;85mdf multipage.pdf multipage_with_ocr.pdf\r\u001b[8C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[1.009954, "o", "\u001b[38;2;255;0;0md\u001b[38;2;85;85;85mf multipage.pdf multipage_with_ocr.pdf\r\u001b[9C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[1.034488, "o", "\u001b[38;2;255;0;0mf\u001b[38;2;85;85;85m multipage.pdf multipage_with_ocr.pdf\r\u001b[10C\u001b[30m\u001b(B\u001b[m\b\b\b\b\b\b\b\b\u001b[38;2;0;95;215mocrmypdf\u001b[38;2;85;85;85m multipage.pdf multipage_with_ocr.pdf\r\u001b[10C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[1.069226, "o", "\u001b[38;2;0;95;215m \u001b[38;2;85;85;85mmultipage.pdf multipage_with_ocr.pdf\r\u001b[11C\u001b[30m\u001b(B\u001b[m\b \u001b[38;2;85;85;85mmultipage.pdf multipage_with_ocr.pdf\r\u001b[11C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[1.569682, "o", "-\u001b[K\r\u001b[12C\u001b[38;2;85;85;85m-version\r\u001b[12C\u001b[30m\u001b(B\u001b[m\b\u001b[38;2;0;175;255m-\u001b[38;2;85;85;85m-version\r\u001b[12C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[1.642096, "o", "\u001b[38;2;0;175;255m-\u001b[38;2;85;85;85mversion\r\u001b[13C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[1.71793, "o", "\u001b[38;2;0;175;255ms\u001b[30m\u001b(B\u001b[m\u001b[K\r\u001b[14C"]
|
||||||
|
[1.771483, "o", "\u001b[38;2;0;175;255mk\r\u001b[15C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[1.864664, "o", "\u001b[38;2;0;175;255mi\r\u001b[16C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[1.876085, "o", "\u001b[38;2;0;175;255mp\r\u001b[17C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[2.092979, "o", "\u001b[38;2;0;175;255m-\r\u001b[18C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[2.138821, "o", "\u001b[38;2;0;175;255mt\r\u001b[19C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[2.18017, "o", "\u001b[38;2;0;175;255me\r\u001b[20C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[2.268222, "o", "\u001b[38;2;0;175;255mx\r\u001b[21C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[2.277031, "o", "\u001b[38;2;0;175;255mt\r\u001b[22C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[2.322469, "o", "\u001b[38;2;0;175;255m \r\u001b[23C\u001b[30m\u001b(B\u001b[m\b \r\u001b[23C"]
|
||||||
|
[2.824696, "o", "m\r\u001b[24C\b\u001b[38;2;0;175;255m\u001b[4mm\r\u001b[24C\u001b[30m\u001b(B\u001b[m\u001b[38;2;85;85;85masks.pdf \r\u001b[24C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[2.923234, "o", "\u001b[38;2;0;175;255m\u001b[4mu\u001b[30m\u001b(B\u001b[m\u001b[K\r\u001b[25C\u001b[38;2;85;85;85mltipage.pdf \r\u001b[25C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[2.960685, "o", "\u001b[38;2;0;175;255m\u001b[4ml\u001b[38;2;85;85;85m\u001b[24mtipage.pdf \r\u001b[26C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[3.03365, "o", "\u001b[38;2;0;175;255m\u001b[4mt\u001b[38;2;85;85;85m\u001b[24mipage.pdf \r\u001b[27C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[3.479338, "o", "\u001b[38;2;0;175;255m\u001b[4mipage.pdf \r\u001b[37C\u001b[30m\u001b(B\u001b[m\b \r\u001b[37C"]
|
||||||
|
[3.754818, "o", "m\r\u001b[38C\b\u001b[38;2;0;175;255m\u001b[4mm\r\u001b[38C\u001b[30m\u001b(B\u001b[m\u001b[38;2;85;85;85masks.pdf \r\u001b[38C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[3.873318, "o", "\u001b[38;2;0;175;255m\u001b[4mu\u001b[30m\u001b(B\u001b[m\u001b[K\r\u001b[39C\u001b[38;2;85;85;85mltipage.pdf \r\u001b[39C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[3.926829, "o", "\u001b[38;2;0;175;255m\u001b[4ml\u001b[38;2;85;85;85m\u001b[24mtipage.pdf \r\u001b[40C\u001b[30m\u001b(B\u001b[m"]
|
||||||
|
[4.272251, "o", "\u001b[38;2;0;175;255m\u001b[4mtipage.pdf \r\u001b[51C\u001b[30m\u001b(B\u001b[m\b \r\u001b[51C"]
|
||||||
|
[4.343464, "o", "\r\u001b[50C"]
|
||||||
|
[4.416286, "o", "\r\u001b[49C"]
|
||||||
|
[4.490574, "o", "\r\u001b[48C"]
|
||||||
|
[4.564115, "o", "\r\u001b[47C"]
|
||||||
|
[4.630398, "o", "\r\u001b[46C"]
|
||||||
|
[4.76825, "o", "\u001b[38;2;0;175;255m\u001b[4m_.pd\u001b[30m\u001b(B\u001b[mf \r\u001b[47C\u001b[10D\u001b[38;2;0;175;255mmultipage_.pdf\u001b[30m\u001b(B\u001b[m \r\u001b[47C"]
|
||||||
|
[5.012506, "o", "\u001b[38;2;0;175;255mo.pd\u001b[30m\u001b(B\u001b[mf \r\u001b[48C\u001b[3C\u001b[38;2;0;175;255mf\u001b[30m\u001b(B\u001b[m \r\u001b[48C"]
|
||||||
|
[5.053615, "o", "\u001b[38;2;0;175;255mc.pd\u001b[30m\u001b(B\u001b[mf \r\u001b[49C\u001b[3C\u001b[38;2;0;175;255mf\u001b[30m\u001b(B\u001b[m \r\u001b[49C"]
|
||||||
|
[5.103957, "o", "\u001b[38;2;0;175;255mr.pd\u001b[30m\u001b(B\u001b[mf \r\u001b[50C\u001b[3C\u001b[38;2;0;175;255mf\u001b[30m\u001b(B\u001b[m \r\u001b[50C"]
|
||||||
|
[5.226183, "o", "\r\u001b[55C"]
|
||||||
|
[5.728321, "o", "\r\n\u001b[30m\u001b(B\u001b[m\u001b[?2004l\u001b]0;ocrmypdf --skip-text multipage.pdf multipage_ocr.pdf /home/jb/src/ocrmypdf/tests/resources\u0007\u001b[30m\u001b(B\u001b[m\r"]
|
||||||
|
[5.801032, "o", "\rScanning contents: 0%| | 0/6 [00:00<?, ?page/s]"]
|
||||||
|
[5.802664, "o", "\rScanning contents: 100%|█████████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 1270.68page/s]\r\n"]
|
||||||
|
[5.802747, "o", "Start processing 6 pages concurrently\r\n"]
|
||||||
|
[5.803488, "o", "\rOCR: 0%| | 0.0/6.0 [00:00<?, ?page/s]"]
|
||||||
|
[5.804896, "o", "\r \r 4 skipping all processing on this page\r\n\rOCR: 0%| | 0.0/6.0 [00:00<?, ?page/s]"]
|
||||||
|
[5.896969, "o", "\rOCR: 25%|█████████████████████▎ | 1.5/6.0 [00:00<00:00, 8.12page/s]"]
|
||||||
|
[6.170021, "o", "\rOCR: 42%|███████████████████████████████████▍ | 2.5/6.0 [00:00<00:01, 3.05page/s]"]
|
||||||
|
[6.292338, "o", "\rOCR: 58%|█████████████████████████████████████████████████▌ | 3.5/6.0 [00:00<00:00, 3.39page/s]"]
|
||||||
|
[6.586017, "o", "\rOCR: 75%|███████████████████████████████████████████████████████████████▊ | 4.5/6.0 [00:01<00:00, 2.49page/s]"]
|
||||||
|
[7.087058, "o", "\rOCR: 92%|█████████████████████████████████████████████████████████████████████████████▉ | 5.5/6.0 [00:06<00:00, 1.98s/page]\rOCR: 100%|█████████████████████████████████████████████████████████████████████████████████████| 6.0/6.0 [00:06<00:00, 1.09s/page]\r\nPostprocessing...\r\n"]
|
||||||
|
[7.104927, "o", "\rPDF/A conversion: 0%| | 0/6 [00:00<?, ?page/s]"]
|
||||||
|
[7.607392, "o", "\rPDF/A conversion: 50%|██████████████████████████████████████ | 3/6 [00:01<00:01, 1.61page/s]"]
|
||||||
|
[7.653781, "o", "\rPDF/A conversion: 83%|███████████████████████████████████████████████████████████████▎ | 5/6 [00:01<00:00, 2.90page/s]"]
|
||||||
|
[7.774532, "o", "\rPDF/A conversion: 100%|████████████████████████████████████████████████████████████████████████████| 6/6 [00:02<00:00, 2.71page/s]\r\n"]
|
||||||
|
[7.778252, "o", "\u001b[33mSome input metadata could not be copied because it is not permitted in PDF/A. You may wish to examine the output PDF's XMP metadata.\u001b[0m\r\n"]
|
||||||
|
[8.280789, "o", "\rRecompressing JPEGs: 0image [00:00, ?image/s]\rRecompressing JPEGs: 0image [00:00, ?image/s]\r\n\rDeflating JPEGs: 0%| | 0/4 [00:00<?, ?image/s]\rDeflating JPEGs: 100%|███████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 238.28image/s]\r\n"]
|
||||||
|
[8.28149, "o", "\rJBIG2: 0item [00:00, ?item/s]\rJBIG2: 0item [00:00, ?item/s]\r\n"]
|
||||||
|
[8.289998, "o", "Image optimization ratio: 1.01 savings: 1.3%\r\nTotal file size ratio: 1.02 savings: 1.6%\r\n"]
|
||||||
|
[8.291209, "o", "Output file is a PDF/A-2B (as expected)\r\n"]
|
||||||
|
[8.361316, "o", "\u001b[2m⏎\u001b(B\u001b[m \r⏎ \r\u001b[K\u001b[?2004h\u001b]0;fish /home/jb/src/ocrmypdf/tests/resources\u0007\u001b[30m\u001b(B\u001b[m> \u001b[K\r\u001b[C\u001b[C"]
|
||||||
|
[8.862206, "o", "\r\n\u001b[30m\u001b(B\u001b[m\u001b[30m\u001b(B\u001b[m\u001b[?2004l"]
|
||||||
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 29 KiB |
+1
-22
@@ -17,7 +17,6 @@ license = {text = "MPL-2.0"}
|
|||||||
requires-python = ">=3.8"
|
requires-python = ">=3.8"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"Pillow>=8.2.0",
|
"Pillow>=8.2.0",
|
||||||
"coloredlogs>=14.0",
|
|
||||||
"deprecation>=2.1.0",
|
"deprecation>=2.1.0",
|
||||||
"img2pdf>=0.3.0", # pure Python
|
"img2pdf>=0.3.0", # pure Python
|
||||||
"packaging>=20",
|
"packaging>=20",
|
||||||
@@ -25,6 +24,7 @@ dependencies = [
|
|||||||
"pikepdf>=5.0.1",
|
"pikepdf>=5.0.1",
|
||||||
"pluggy>=0.13.0",
|
"pluggy>=0.13.0",
|
||||||
"reportlab>=3.5.66",
|
"reportlab>=3.5.66",
|
||||||
|
"rich>=13",
|
||||||
"tqdm>=4",
|
"tqdm>=4",
|
||||||
"importlib-resources>=5;python_version<'3.9'", # until Python 3.9
|
"importlib-resources>=5;python_version<'3.9'", # until Python 3.9
|
||||||
"typing-extensions>=4;python_version<'3.10'",
|
"typing-extensions>=4;python_version<'3.10'",
|
||||||
@@ -137,27 +137,6 @@ exclude_lines = [
|
|||||||
"if TYPE_CHECKING:"
|
"if TYPE_CHECKING:"
|
||||||
]
|
]
|
||||||
|
|
||||||
[tool.isort]
|
|
||||||
profile = "black"
|
|
||||||
known_first_party = "ocrmypdf"
|
|
||||||
known_third_party = [
|
|
||||||
"PIL",
|
|
||||||
"flask",
|
|
||||||
"img2pdf",
|
|
||||||
"ocrmypdf",
|
|
||||||
"pdfminer",
|
|
||||||
"pikepdf",
|
|
||||||
"pkg_resources",
|
|
||||||
"pluggy",
|
|
||||||
"pytest",
|
|
||||||
"reportlab",
|
|
||||||
"setuptools",
|
|
||||||
"sphinx_rtd_theme",
|
|
||||||
"tqdm",
|
|
||||||
"watchdog",
|
|
||||||
"werkzeug"
|
|
||||||
]
|
|
||||||
|
|
||||||
[tool.pytest.ini_options]
|
[tool.pytest.ini_options]
|
||||||
minversion = "6.0"
|
minversion = "6.0"
|
||||||
norecursedirs = ["lib", ".pc", ".git", "venv", "output", "cache", "resources"]
|
norecursedirs = ["lib", ".pc", ".git", "venv", "output", "cache", "resources"]
|
||||||
|
|||||||
@@ -30,7 +30,7 @@ except AttributeError:
|
|||||||
|
|
||||||
log = logging.getLogger(__name__)
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
# Most reliable what to get the bitness of Python interpreter, according to Python docs
|
# Most reliable way to get the bitness of Python interpreter, according to Python docs
|
||||||
_IS_64BIT = sys.maxsize > 2**32
|
_IS_64BIT = sys.maxsize > 2**32
|
||||||
|
|
||||||
_GSWIN = None
|
_GSWIN = None
|
||||||
@@ -63,6 +63,7 @@ def rasterize_pdf(
|
|||||||
page_dpi: Resolution | None = None,
|
page_dpi: Resolution | None = None,
|
||||||
rotation: int | None = None,
|
rotation: int | None = None,
|
||||||
filter_vector: bool = False,
|
filter_vector: bool = False,
|
||||||
|
stop_on_error: bool = False,
|
||||||
):
|
):
|
||||||
"""Rasterize one page of a PDF at resolution raster_dpi in canvas units."""
|
"""Rasterize one page of a PDF at resolution raster_dpi in canvas units."""
|
||||||
raster_dpi = raster_dpi.round(6)
|
raster_dpi = raster_dpi.round(6)
|
||||||
@@ -83,6 +84,7 @@ def rasterize_pdf(
|
|||||||
f'-r{raster_dpi.x:f}x{raster_dpi.y:f}',
|
f'-r{raster_dpi.x:f}x{raster_dpi.y:f}',
|
||||||
]
|
]
|
||||||
+ (['-dFILTERVECTOR'] if filter_vector else [])
|
+ (['-dFILTERVECTOR'] if filter_vector else [])
|
||||||
|
+ (['-dPDFSTOPONERROR'] if stop_on_error else [])
|
||||||
+ [
|
+ [
|
||||||
'-o',
|
'-o',
|
||||||
'-',
|
'-',
|
||||||
@@ -161,6 +163,7 @@ def generate_pdfa(
|
|||||||
pdf_version: str = '1.5',
|
pdf_version: str = '1.5',
|
||||||
pdfa_part: str = '2',
|
pdfa_part: str = '2',
|
||||||
progressbar_class=None,
|
progressbar_class=None,
|
||||||
|
stop_on_error: bool = False,
|
||||||
):
|
):
|
||||||
# Ghostscript's compression is all or nothing. We can either force all images
|
# Ghostscript's compression is all or nothing. We can either force all images
|
||||||
# to JPEG, force all to Flate/PNG, or let it decide how to encode the images.
|
# to JPEG, force all to Flate/PNG, or let it decide how to encode the images.
|
||||||
@@ -193,6 +196,10 @@ def generate_pdfa(
|
|||||||
# https://bugs.ghostscript.com/show_bug.cgi?id=705187
|
# https://bugs.ghostscript.com/show_bug.cgi?id=705187
|
||||||
compression_args.append('-dNEWPDF=false')
|
compression_args.append('-dNEWPDF=false')
|
||||||
|
|
||||||
|
if os.name == 'nt':
|
||||||
|
# Windows has lots of fatal "permission denied" errors
|
||||||
|
stop_on_error = False
|
||||||
|
|
||||||
# nb no need to specify ProcessColorModel when ColorConversionStrategy
|
# nb no need to specify ProcessColorModel when ColorConversionStrategy
|
||||||
# is set; see:
|
# is set; see:
|
||||||
# https://bugs.ghostscript.com/show_bug.cgi?id=699392
|
# https://bugs.ghostscript.com/show_bug.cgi?id=699392
|
||||||
@@ -207,6 +214,7 @@ def generate_pdfa(
|
|||||||
"-dAutoRotatePages=/None",
|
"-dAutoRotatePages=/None",
|
||||||
"-sColorConversionStrategy=" + strategy,
|
"-sColorConversionStrategy=" + strategy,
|
||||||
]
|
]
|
||||||
|
+ (['-dPDFSTOPONERROR'] if stop_on_error else [])
|
||||||
+ compression_args
|
+ compression_args
|
||||||
+ [
|
+ [
|
||||||
"-dJPEGQ=95",
|
"-dJPEGQ=95",
|
||||||
|
|||||||
@@ -233,6 +233,7 @@ def get_deskew(
|
|||||||
parsed = _parse_tesseract_output(p.stdout)
|
parsed = _parse_tesseract_output(p.stdout)
|
||||||
deskew_radians = float(parsed.get('Deskew angle', 0))
|
deskew_radians = float(parsed.get('Deskew angle', 0))
|
||||||
deskew_degrees = 180 / pi * deskew_radians
|
deskew_degrees = 180 / pi * deskew_radians
|
||||||
|
log.debug(f"Deskew angle: {deskew_degrees:.3f}")
|
||||||
return deskew_degrees
|
return deskew_degrees
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ from ocrmypdf.exceptions import MissingDependencyError, SubprocessOutputError
|
|||||||
from ocrmypdf.subprocess import get_version, run
|
from ocrmypdf.subprocess import get_version, run
|
||||||
|
|
||||||
# unpaper documentation:
|
# unpaper documentation:
|
||||||
# https://github.com/Flameeyes/unpaper/blob/master/doc/basic-concepts.md
|
# https://github.com/Flameeyes/unpaper/blob/main/doc/basic-concepts.md
|
||||||
|
|
||||||
|
|
||||||
if sys.version_info >= (3, 10):
|
if sys.version_info >= (3, 10):
|
||||||
|
|||||||
@@ -8,6 +8,17 @@ from __future__ import annotations
|
|||||||
import logging
|
import logging
|
||||||
from contextlib import suppress
|
from contextlib import suppress
|
||||||
|
|
||||||
|
from rich.console import Console
|
||||||
|
from rich.logging import RichHandler
|
||||||
|
from rich.progress import (
|
||||||
|
BarColumn,
|
||||||
|
MofNCompleteColumn,
|
||||||
|
Progress,
|
||||||
|
TaskProgressColumn,
|
||||||
|
TextColumn,
|
||||||
|
TimeRemainingColumn,
|
||||||
|
)
|
||||||
|
from rich.table import Column
|
||||||
from tqdm import tqdm
|
from tqdm import tqdm
|
||||||
|
|
||||||
|
|
||||||
@@ -41,3 +52,59 @@ class TqdmConsole:
|
|||||||
def flush(self):
|
def flush(self):
|
||||||
with suppress(AttributeError):
|
with suppress(AttributeError):
|
||||||
self.file.flush()
|
self.file.flush()
|
||||||
|
|
||||||
|
|
||||||
|
class RichLoggingHandler(RichHandler):
|
||||||
|
def __init__(self, console: Console, **kwargs):
|
||||||
|
super().__init__(
|
||||||
|
console=console, show_level=False, show_time=False, markup=True, **kwargs
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class RichTqdmProgressAdapter:
|
||||||
|
"""Adapt tqdm API to rich progress bar."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
console: Console,
|
||||||
|
desc: str,
|
||||||
|
total: float | None = None,
|
||||||
|
unit: str | None = None,
|
||||||
|
unit_scale: float | None = 1.0,
|
||||||
|
disable: bool = False,
|
||||||
|
**kwargs,
|
||||||
|
):
|
||||||
|
self.progress = Progress(
|
||||||
|
TextColumn(
|
||||||
|
"[progress.description]{task.description}",
|
||||||
|
table_column=Column(min_width=20),
|
||||||
|
),
|
||||||
|
BarColumn(),
|
||||||
|
TaskProgressColumn(),
|
||||||
|
MofNCompleteColumn(),
|
||||||
|
TimeRemainingColumn(),
|
||||||
|
console=console,
|
||||||
|
auto_refresh=True,
|
||||||
|
redirect_stderr=True,
|
||||||
|
redirect_stdout=False,
|
||||||
|
disable=disable,
|
||||||
|
**kwargs,
|
||||||
|
)
|
||||||
|
self.unit_scale = unit_scale
|
||||||
|
self.progress_bar = self.progress.add_task(
|
||||||
|
desc, total=total * self.unit_scale, unit=unit
|
||||||
|
)
|
||||||
|
|
||||||
|
def __enter__(self):
|
||||||
|
self.progress.start()
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, exc_type, exc_value, traceback):
|
||||||
|
self.progress.refresh()
|
||||||
|
self.progress.stop()
|
||||||
|
return False
|
||||||
|
|
||||||
|
def update(self, value=None):
|
||||||
|
advance = self.unit_scale if value is None else value
|
||||||
|
self.progress.update(self.progress_bar, advance=advance)
|
||||||
|
|||||||
+133
-37
@@ -27,6 +27,7 @@ from ocrmypdf._jobcontext import PageContext, PdfContext
|
|||||||
from ocrmypdf._version import PROGRAM_NAME
|
from ocrmypdf._version import PROGRAM_NAME
|
||||||
from ocrmypdf._version import __version__ as VERSION
|
from ocrmypdf._version import __version__ as VERSION
|
||||||
from ocrmypdf.exceptions import (
|
from ocrmypdf.exceptions import (
|
||||||
|
DigitalSignatureError,
|
||||||
DpiError,
|
DpiError,
|
||||||
EncryptedPdfError,
|
EncryptedPdfError,
|
||||||
InputFileError,
|
InputFileError,
|
||||||
@@ -37,6 +38,7 @@ from ocrmypdf.helpers import IMG2PDF_KWARGS, Resolution, safe_symlink
|
|||||||
from ocrmypdf.hocrtransform import HocrTransform
|
from ocrmypdf.hocrtransform import HocrTransform
|
||||||
from ocrmypdf.pdfa import generate_pdfa_ps
|
from ocrmypdf.pdfa import generate_pdfa_ps
|
||||||
from ocrmypdf.pdfinfo import Colorspace, Encoding, PageInfo, PdfInfo
|
from ocrmypdf.pdfinfo import Colorspace, Encoding, PageInfo, PdfInfo
|
||||||
|
from ocrmypdf.pluginspec import OrientationConfidence
|
||||||
|
|
||||||
# Remove this workaround when we require Pillow >= 10
|
# Remove this workaround when we require Pillow >= 10
|
||||||
try:
|
try:
|
||||||
@@ -51,6 +53,20 @@ VECTOR_PAGE_DPI = 400
|
|||||||
|
|
||||||
|
|
||||||
def triage_image_file(input_file: Path, output_file: Path, options) -> None:
|
def triage_image_file(input_file: Path, output_file: Path, options) -> None:
|
||||||
|
"""Triage the input image file.
|
||||||
|
|
||||||
|
If the input file is an image, check its resolution and convert it to PDF.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
input_file: The path to the input file.
|
||||||
|
output_file: The path to the output file.
|
||||||
|
options: An object containing the options passed to the OCRmyPDF command.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
UnsupportedImageFormatError: If the input file is not a supported image format.
|
||||||
|
DpiError: If the input image has no resolution (DPI) in its metadata or if the
|
||||||
|
resolution is not credible.
|
||||||
|
"""
|
||||||
log.info("Input file is not a PDF, checking if it is an image...")
|
log.info("Input file is not a PDF, checking if it is an image...")
|
||||||
try:
|
try:
|
||||||
im = Image.open(input_file)
|
im = Image.open(input_file)
|
||||||
@@ -65,34 +81,32 @@ def triage_image_file(input_file: Path, output_file: Path, options) -> None:
|
|||||||
if im.info['dpi'] <= (96, 96) and not options.image_dpi:
|
if im.info['dpi'] <= (96, 96) and not options.image_dpi:
|
||||||
log.info("Image size: (%d, %d)", *im.size)
|
log.info("Image size: (%d, %d)", *im.size)
|
||||||
log.info("Image resolution: (%d, %d)", *im.info['dpi'])
|
log.info("Image resolution: (%d, %d)", *im.info['dpi'])
|
||||||
log.error(
|
raise DpiError(
|
||||||
"Input file is an image, but the resolution (DPI) is "
|
"Input file is an image, but the resolution (DPI) is "
|
||||||
"not credible. Estimate the resolution at which the "
|
"not credible. Estimate the resolution at which the "
|
||||||
"image was scanned and specify it using --image-dpi."
|
"image was scanned and specify it using --image-dpi."
|
||||||
)
|
)
|
||||||
raise DpiError()
|
|
||||||
elif not options.image_dpi:
|
elif not options.image_dpi:
|
||||||
log.info("Image size: (%d, %d)", *im.size)
|
log.info("Image size: (%d, %d)", *im.size)
|
||||||
log.error(
|
raise DpiError(
|
||||||
"Input file is an image, but has no resolution (DPI) "
|
"Input file is an image, but has no resolution (DPI) "
|
||||||
"in its metadata. Estimate the resolution at which "
|
"in its metadata. Estimate the resolution at which "
|
||||||
"image was scanned and specify it using --image-dpi."
|
"image was scanned and specify it using --image-dpi."
|
||||||
)
|
)
|
||||||
raise DpiError()
|
|
||||||
|
|
||||||
if im.mode in ('RGBA', 'LA'):
|
if im.mode in ('RGBA', 'LA'):
|
||||||
log.error(
|
raise UnsupportedImageFormatError(
|
||||||
"The input image has an alpha channel. Remove the alpha "
|
"The input image has an alpha channel. Remove the alpha "
|
||||||
"channel first."
|
"channel first."
|
||||||
)
|
)
|
||||||
raise UnsupportedImageFormatError()
|
|
||||||
|
|
||||||
if 'iccprofile' not in im.info:
|
if 'iccprofile' not in im.info:
|
||||||
if im.mode == 'RGB':
|
if im.mode == 'RGB':
|
||||||
log.info("Input image has no ICC profile, assuming sRGB")
|
log.info("Input image has no ICC profile, assuming sRGB")
|
||||||
elif im.mode == 'CMYK':
|
elif im.mode == 'CMYK':
|
||||||
log.error("Input CMYK image has no ICC profile, not usable")
|
raise UnsupportedImageFormatError(
|
||||||
raise UnsupportedImageFormatError()
|
"Input CMYK image has no ICC profile, not usable"
|
||||||
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
log.info("Image seems valid. Try converting to PDF...")
|
log.info("Image seems valid. Try converting to PDF...")
|
||||||
@@ -110,7 +124,6 @@ def triage_image_file(input_file: Path, output_file: Path, options) -> None:
|
|||||||
)
|
)
|
||||||
log.info("Successfully converted to PDF, processing...")
|
log.info("Successfully converted to PDF, processing...")
|
||||||
except img2pdf.ImageOpenError as e:
|
except img2pdf.ImageOpenError as e:
|
||||||
log.error(e)
|
|
||||||
raise UnsupportedImageFormatError() from e
|
raise UnsupportedImageFormatError() from e
|
||||||
|
|
||||||
|
|
||||||
@@ -180,18 +193,21 @@ def validate_pdfinfo_options(context: PdfContext) -> None:
|
|||||||
options = context.options
|
options = context.options
|
||||||
|
|
||||||
if pdfinfo.needs_rendering:
|
if pdfinfo.needs_rendering:
|
||||||
log.error(
|
raise InputFileError(
|
||||||
"This PDF contains dynamic XFA forms created by Adobe LiveCycle "
|
"This PDF contains dynamic XFA forms created by Adobe LiveCycle "
|
||||||
"Designer and can only be read by Adobe Acrobat or Adobe Reader."
|
"Designer and can only be read by Adobe Acrobat or Adobe Reader."
|
||||||
)
|
)
|
||||||
raise InputFileError()
|
if pdfinfo.has_signature:
|
||||||
|
if options.invalidate_digital_signatures:
|
||||||
|
log.warning("All digital signatures will be invalidated")
|
||||||
|
else:
|
||||||
|
raise DigitalSignatureError()
|
||||||
if pdfinfo.has_acroform:
|
if pdfinfo.has_acroform:
|
||||||
if options.redo_ocr:
|
if options.redo_ocr:
|
||||||
log.error(
|
raise InputFileError(
|
||||||
"This PDF has a user fillable form. --redo-ocr is not "
|
"This PDF has a user fillable form. --redo-ocr is not "
|
||||||
"currently possible on such files."
|
"currently possible on such files."
|
||||||
)
|
)
|
||||||
raise InputFileError()
|
|
||||||
else:
|
else:
|
||||||
log.warning(
|
log.warning(
|
||||||
"This PDF has a fillable form. "
|
"This PDF has a fillable form. "
|
||||||
@@ -208,6 +224,7 @@ def validate_pdfinfo_options(context: PdfContext) -> None:
|
|||||||
|
|
||||||
|
|
||||||
def _vector_page_dpi(pageinfo: PageInfo) -> int:
|
def _vector_page_dpi(pageinfo: PageInfo) -> int:
|
||||||
|
"""Get a DPI to use for vector pages, if the page has vector content."""
|
||||||
return VECTOR_PAGE_DPI if pageinfo.has_vector or pageinfo.has_text else 0
|
return VECTOR_PAGE_DPI if pageinfo.has_vector or pageinfo.has_text else 0
|
||||||
|
|
||||||
|
|
||||||
@@ -227,7 +244,10 @@ def get_page_dpi(pageinfo: PageInfo, options) -> Resolution:
|
|||||||
|
|
||||||
|
|
||||||
def get_page_square_dpi(pageinfo: PageInfo, options) -> Resolution:
|
def get_page_square_dpi(pageinfo: PageInfo, options) -> Resolution:
|
||||||
"""Get the DPI when we require xres == yres, scaled to physical units."""
|
"""Get the DPI when we require xres == yres, scaled to physical units.
|
||||||
|
|
||||||
|
Page DPI includes UserUnit scaling.
|
||||||
|
"""
|
||||||
xres = pageinfo.dpi.x or 0.0
|
xres = pageinfo.dpi.x or 0.0
|
||||||
yres = pageinfo.dpi.y or 0.0
|
yres = pageinfo.dpi.y or 0.0
|
||||||
userunit = float(pageinfo.userunit) or 1.0
|
userunit = float(pageinfo.userunit) or 1.0
|
||||||
@@ -243,7 +263,12 @@ def get_page_square_dpi(pageinfo: PageInfo, options) -> Resolution:
|
|||||||
|
|
||||||
|
|
||||||
def get_canvas_square_dpi(pageinfo: PageInfo, options) -> Resolution:
|
def get_canvas_square_dpi(pageinfo: PageInfo, options) -> Resolution:
|
||||||
"""Get the DPI when we require xres == yres, in Postscript units."""
|
"""Get the DPI when we require xres == yres, in Postscript units.
|
||||||
|
|
||||||
|
Canvas DPI is independent of PDF UserUnit scaling, which is
|
||||||
|
used to describe situations where the PDF user space is not 1:1 with
|
||||||
|
the physical units of the page.
|
||||||
|
"""
|
||||||
units = float(
|
units = float(
|
||||||
max(
|
max(
|
||||||
(pageinfo.dpi.x) or VECTOR_PAGE_DPI,
|
(pageinfo.dpi.x) or VECTOR_PAGE_DPI,
|
||||||
@@ -256,6 +281,7 @@ def get_canvas_square_dpi(pageinfo: PageInfo, options) -> Resolution:
|
|||||||
|
|
||||||
|
|
||||||
def is_ocr_required(page_context: PageContext) -> bool:
|
def is_ocr_required(page_context: PageContext) -> bool:
|
||||||
|
"""Check if the page needs to be OCR'd."""
|
||||||
pageinfo = page_context.pageinfo
|
pageinfo = page_context.pageinfo
|
||||||
options = page_context.options
|
options = page_context.options
|
||||||
|
|
||||||
@@ -331,6 +357,7 @@ def is_ocr_required(page_context: PageContext) -> bool:
|
|||||||
|
|
||||||
|
|
||||||
def rasterize_preview(input_file: Path, page_context: PageContext) -> Path:
|
def rasterize_preview(input_file: Path, page_context: PageContext) -> Path:
|
||||||
|
"""Generate a lower quality preview image."""
|
||||||
output_file = page_context.get_path('rasterize_preview.jpg')
|
output_file = page_context.get_path('rasterize_preview.jpg')
|
||||||
canvas_dpi = get_canvas_square_dpi(page_context.pageinfo, page_context.options)
|
canvas_dpi = get_canvas_square_dpi(page_context.pageinfo, page_context.options)
|
||||||
page_dpi = get_page_square_dpi(page_context.pageinfo, page_context.options)
|
page_dpi = get_page_square_dpi(page_context.pageinfo, page_context.options)
|
||||||
@@ -343,12 +370,15 @@ def rasterize_preview(input_file: Path, page_context: PageContext) -> Path:
|
|||||||
page_dpi=page_dpi,
|
page_dpi=page_dpi,
|
||||||
rotation=0,
|
rotation=0,
|
||||||
filter_vector=False,
|
filter_vector=False,
|
||||||
|
stop_on_soft_error=not page_context.options.continue_on_soft_render_error,
|
||||||
)
|
)
|
||||||
return output_file
|
return output_file
|
||||||
|
|
||||||
|
|
||||||
def describe_rotation(page_context: PageContext, orient_conf, correction: int) -> str:
|
def describe_rotation(
|
||||||
"""Describe the page rotation we are going to perform."""
|
page_context: PageContext, orient_conf: OrientationConfidence, correction: int
|
||||||
|
) -> str:
|
||||||
|
"""Describe the page rotation we are going to perform (or not perform)."""
|
||||||
direction = {0: '⇧', 90: '⇨', 180: '⇩', 270: '⇦'}
|
direction = {0: '⇧', 90: '⇨', 180: '⇩', 270: '⇦'}
|
||||||
turns = {0: ' ', 90: '⬏', 180: '↻', 270: '⬑'}
|
turns = {0: ' ', 90: '⬏', 180: '↻', 270: '⬑'}
|
||||||
|
|
||||||
@@ -375,7 +405,7 @@ def describe_rotation(page_context: PageContext, orient_conf, correction: int) -
|
|||||||
|
|
||||||
|
|
||||||
def get_orientation_correction(preview: Path, page_context: PageContext) -> int:
|
def get_orientation_correction(preview: Path, page_context: PageContext) -> int:
|
||||||
"""Work out orientation correct for each page.
|
"""Work out orientation correction for each page.
|
||||||
|
|
||||||
We ask Ghostscript to draw a preview page, which will rasterize with the
|
We ask Ghostscript to draw a preview page, which will rasterize with the
|
||||||
current /Rotate applied, and then ask OCR which way the page is
|
current /Rotate applied, and then ask OCR which way the page is
|
||||||
@@ -409,8 +439,22 @@ def rasterize(
|
|||||||
page_context: PageContext,
|
page_context: PageContext,
|
||||||
correction: int = 0,
|
correction: int = 0,
|
||||||
output_tag: str = '',
|
output_tag: str = '',
|
||||||
remove_vectors=None,
|
remove_vectors: bool | None = None,
|
||||||
):
|
) -> Path:
|
||||||
|
"""Rasterize a PDF page to a PNG image.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
input_file: The input PDF file path.
|
||||||
|
page_context: The page context object.
|
||||||
|
correction: The orientation correction angle. Defaults to 0.
|
||||||
|
output_tag: The output tag. Defaults to ''.
|
||||||
|
remove_vectors: Whether to remove vectors. Defaults to None, which means
|
||||||
|
the value from the page context options will be used. If the value
|
||||||
|
is True or False, it will override the page context options.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Path: The output PNG file path.
|
||||||
|
"""
|
||||||
colorspaces = ['pngmono', 'pnggray', 'png256', 'png16m']
|
colorspaces = ['pngmono', 'pnggray', 'png256', 'png16m']
|
||||||
device_idx = 0
|
device_idx = 0
|
||||||
|
|
||||||
@@ -455,6 +499,7 @@ def rasterize(
|
|||||||
pageno=pageinfo.pageno + 1,
|
pageno=pageinfo.pageno + 1,
|
||||||
rotation=correction,
|
rotation=correction,
|
||||||
filter_vector=remove_vectors,
|
filter_vector=remove_vectors,
|
||||||
|
stop_on_soft_error=not page_context.options.continue_on_soft_render_error,
|
||||||
)
|
)
|
||||||
return output_file
|
return output_file
|
||||||
|
|
||||||
@@ -470,6 +515,15 @@ def preprocess_remove_background(input_file: Path, page_context: PageContext) ->
|
|||||||
|
|
||||||
|
|
||||||
def preprocess_deskew(input_file: Path, page_context: PageContext) -> Path:
|
def preprocess_deskew(input_file: Path, page_context: PageContext) -> Path:
|
||||||
|
"""Deskews the input image using the OCR engine and saves the output to a file.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
input_file: The input image file to deskew.
|
||||||
|
page_context: The context of the page being processed.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Path: The path to the deskewed image file.
|
||||||
|
"""
|
||||||
output_file = page_context.get_path('pp_deskew.png')
|
output_file = page_context.get_path('pp_deskew.png')
|
||||||
dpi = get_page_square_dpi(page_context.pageinfo, page_context.options)
|
dpi = get_page_square_dpi(page_context.pageinfo, page_context.options)
|
||||||
|
|
||||||
@@ -565,7 +619,16 @@ def ocr_engine_hocr(input_file: Path, page_context: PageContext) -> tuple[Path,
|
|||||||
|
|
||||||
|
|
||||||
def should_visible_page_image_use_jpg(pageinfo: PageInfo) -> bool:
|
def should_visible_page_image_use_jpg(pageinfo: PageInfo) -> bool:
|
||||||
# If all images were JPEGs originally, produce a JPEG as output
|
"""Determines whether the visible page image should be saved as a JPEG.
|
||||||
|
|
||||||
|
If all images were JPEGs originally, permit a JPEG as output.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
pageinfo: The PageInfo object containing information about the page.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
A boolean indicating whether the visible page image should be saved as a JPEG.
|
||||||
|
"""
|
||||||
return bool(pageinfo.images) and all(
|
return bool(pageinfo.images) and all(
|
||||||
im.enc == Encoding.jpeg for im in pageinfo.images
|
im.enc == Encoding.jpeg for im in pageinfo.images
|
||||||
)
|
)
|
||||||
@@ -736,6 +799,7 @@ def convert_to_pdfa(input_pdf: Path, input_ps_stub: Path, context: PdfContext) -
|
|||||||
if options.progress_bar
|
if options.progress_bar
|
||||||
else None
|
else None
|
||||||
),
|
),
|
||||||
|
stop_on_soft_error=not options.continue_on_soft_render_error,
|
||||||
)
|
)
|
||||||
|
|
||||||
return output_file
|
return output_file
|
||||||
@@ -789,24 +853,45 @@ def metadata_fixup(working_file: Path, context: PdfContext) -> Path:
|
|||||||
|
|
||||||
with pikepdf.open(context.origin) as original, pikepdf.open(working_file) as pdf:
|
with pikepdf.open(context.origin) as original, pikepdf.open(working_file) as pdf:
|
||||||
docinfo = get_docinfo(original, context)
|
docinfo = get_docinfo(original, context)
|
||||||
with pdf.open_metadata() as meta:
|
with pdf.open_metadata() as meta_pdf:
|
||||||
meta.load_from_docinfo(docinfo, delete_missing=False, raise_failure=False)
|
meta_pdf.load_from_docinfo(
|
||||||
|
docinfo, delete_missing=False, raise_failure=False
|
||||||
|
)
|
||||||
# If xmp:CreateDate is missing, set it to the modify date to
|
# If xmp:CreateDate is missing, set it to the modify date to
|
||||||
# match Ghostscript, for consistency
|
# ensure consistency with Ghostscript.
|
||||||
if 'xmp:CreateDate' not in meta:
|
if 'xmp:CreateDate' not in meta_pdf:
|
||||||
meta['xmp:CreateDate'] = meta.get('xmp:ModifyDate', '')
|
meta_pdf['xmp:CreateDate'] = meta_pdf.get('xmp:ModifyDate', '')
|
||||||
|
|
||||||
with original.open_metadata(
|
with original.open_metadata(
|
||||||
set_pikepdf_as_editor=False, update_docinfo=False, strict=False
|
set_pikepdf_as_editor=False, update_docinfo=False, strict=False
|
||||||
) as meta_original:
|
) as meta_original:
|
||||||
if meta.get('dc:title') == 'Untitled':
|
if meta_pdf.get('dc:title') == 'Untitled':
|
||||||
# Ghostscript likes to set title to Untitled if omitted from input.
|
# Ghostscript likes to set title to Untitled if omitted from input.
|
||||||
# Reverse this, because PDF/A TechNote 0003:Metadata in PDF/A-1
|
# Reverse this, because PDF/A TechNote 0003:Metadata in PDF/A-1
|
||||||
# and the XMP Spec do not make this recommendation.
|
# and the XMP Spec do not make this recommendation.
|
||||||
if 'dc:title' not in meta_original:
|
if 'dc:title' not in meta_original:
|
||||||
del meta['dc:title']
|
del meta_pdf['dc:title']
|
||||||
missing = set(meta_original.keys()) - set(meta.keys())
|
# If the user explicitly specified an empty string for any of the
|
||||||
report_on_metadata(missing)
|
# following, they should be unset and not reported as missing in
|
||||||
|
# the output pdf. Note that some metadata fields use differing names
|
||||||
|
# between PDF-A and PDF.
|
||||||
|
for meta in [meta_pdf, meta_original]:
|
||||||
|
if options.title == '' and 'dc:title' in meta:
|
||||||
|
del meta['dc:title'] # PDF-A and PDF
|
||||||
|
if options.author == '':
|
||||||
|
if 'dc:creator' in meta:
|
||||||
|
del meta['dc:creator'] # PDF-A (Not xmp:CreatorTool)
|
||||||
|
if 'pdf:Author' in meta:
|
||||||
|
del meta['pdf:Author'] # PDF
|
||||||
|
if options.subject == '':
|
||||||
|
if 'dc:description' in meta:
|
||||||
|
del meta['dc:description'] # PDF-A
|
||||||
|
if 'dc:subject' in meta:
|
||||||
|
del meta['dc:subject'] # PDF
|
||||||
|
if options.keywords == '' and 'pdf:Keywords' in meta:
|
||||||
|
del meta['pdf:Keywords'] # PDF-A and PDF
|
||||||
|
meta_missing = set(meta_original.keys()) - set(meta_pdf.keys())
|
||||||
|
report_on_metadata(meta_missing)
|
||||||
|
|
||||||
optimizing = context.plugin_manager.hook.is_optimization_enabled(
|
optimizing = context.plugin_manager.hook.is_optimization_enabled(
|
||||||
context=context
|
context=context
|
||||||
@@ -822,6 +907,18 @@ def metadata_fixup(working_file: Path, context: PdfContext) -> Path:
|
|||||||
return output_file
|
return output_file
|
||||||
|
|
||||||
|
|
||||||
|
def _file_size_ratio(
|
||||||
|
input_file: Path, output_file: Path
|
||||||
|
) -> tuple[float | None, float | None]:
|
||||||
|
input_size = input_file.stat().st_size
|
||||||
|
output_size = output_file.stat().st_size
|
||||||
|
if output_size == 0:
|
||||||
|
return None, None
|
||||||
|
ratio = input_size / output_size
|
||||||
|
savings = 1 - output_size / input_size
|
||||||
|
return ratio, savings
|
||||||
|
|
||||||
|
|
||||||
def optimize_pdf(
|
def optimize_pdf(
|
||||||
input_file: Path, context: PdfContext, executor: Executor
|
input_file: Path, context: PdfContext, executor: Executor
|
||||||
) -> tuple[Path, Sequence[str]]:
|
) -> tuple[Path, Sequence[str]]:
|
||||||
@@ -834,13 +931,12 @@ def optimize_pdf(
|
|||||||
linearize=should_linearize(input_file, context),
|
linearize=should_linearize(input_file, context),
|
||||||
)
|
)
|
||||||
|
|
||||||
input_size = input_file.stat().st_size
|
ratio, savings = _file_size_ratio(input_file, output_file)
|
||||||
output_size = output_file.stat().st_size
|
if ratio:
|
||||||
if output_size > 0:
|
log.info(f"Image optimization ratio: {ratio:.2f} savings: {(savings):.1%}")
|
||||||
ratio = input_size / output_size
|
ratio, savings = _file_size_ratio(context.origin, output_file)
|
||||||
savings = 1 - output_size / input_size
|
if ratio:
|
||||||
log.info(f"Optimize ratio: {ratio:.2f} savings: {(savings):.1%}")
|
log.info(f"Total file size ratio: {ratio:.2f} savings: {(savings):.1%}")
|
||||||
|
|
||||||
return output_pdf, messages
|
return output_pdf, messages
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -51,15 +51,17 @@ def check_platform() -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def check_options_languages(options: Namespace, ocr_engine_languages: set[str]) -> None:
|
def check_options_languages(
|
||||||
|
options: Namespace, ocr_engine_languages: list[str]
|
||||||
|
) -> None:
|
||||||
if not options.languages:
|
if not options.languages:
|
||||||
options.languages = {DEFAULT_LANGUAGE}
|
options.languages = [DEFAULT_LANGUAGE]
|
||||||
system_lang = locale.getlocale()[0]
|
system_lang = locale.getlocale()[0]
|
||||||
if system_lang and not system_lang.startswith('en'):
|
if system_lang and not system_lang.startswith('en'):
|
||||||
log.debug("No language specified; assuming --language %s", DEFAULT_LANGUAGE)
|
log.debug("No language specified; assuming --language %s", DEFAULT_LANGUAGE)
|
||||||
if not ocr_engine_languages:
|
if not ocr_engine_languages:
|
||||||
return
|
return
|
||||||
missing_languages = options.languages - ocr_engine_languages
|
missing_languages = set(options.languages) - set(ocr_engine_languages)
|
||||||
if missing_languages:
|
if missing_languages:
|
||||||
lang_text = '\n'.join(lang for lang in missing_languages)
|
lang_text = '\n'.join(lang for lang in missing_languages)
|
||||||
msg = (
|
msg = (
|
||||||
@@ -71,15 +73,16 @@ def check_options_languages(options: Namespace, ocr_engine_languages: set[str])
|
|||||||
"See the online documentation for instructions:\n"
|
"See the online documentation for instructions:\n"
|
||||||
" https://ocrmypdf.readthedocs.io/en/latest/languages.html\n"
|
" https://ocrmypdf.readthedocs.io/en/latest/languages.html\n"
|
||||||
"\n"
|
"\n"
|
||||||
"Note: most languages are identified by a 3-digit ISO 639-2 Code.\n"
|
"Note: most languages are identified by a 3-letter ISO 639-2 Code.\n"
|
||||||
"For example, English is 'eng', German is 'deu', and Spanish is 'spa'."
|
"For example, English is 'eng', German is 'deu', and Spanish is 'spa'.\n"
|
||||||
|
"Simplified Chinese is 'chi_sim' and Traditional Chinese is 'chi_tra'."
|
||||||
"\n"
|
"\n"
|
||||||
)
|
)
|
||||||
raise MissingDependencyError(msg)
|
raise MissingDependencyError(msg)
|
||||||
|
|
||||||
|
|
||||||
def check_options_output(options: Namespace) -> None:
|
def check_options_output(options: Namespace) -> None:
|
||||||
is_latin = options.languages.issubset(HOCR_OK_LANGS)
|
is_latin = set(options.languages).issubset(HOCR_OK_LANGS)
|
||||||
|
|
||||||
if options.pdf_renderer.startswith('hocr') and not is_latin:
|
if options.pdf_renderer.startswith('hocr') and not is_latin:
|
||||||
log.warning(
|
log.warning(
|
||||||
|
|||||||
+1
-10
@@ -15,9 +15,6 @@ from pathlib import Path
|
|||||||
from typing import AnyStr, BinaryIO, Iterable, Union
|
from typing import AnyStr, BinaryIO, Iterable, Union
|
||||||
from warnings import warn
|
from warnings import warn
|
||||||
|
|
||||||
import coloredlogs
|
|
||||||
from humanfriendly.terminal import enable_ansi_support
|
|
||||||
|
|
||||||
from ocrmypdf._logging import PageNumberFilter, TqdmConsole
|
from ocrmypdf._logging import PageNumberFilter, TqdmConsole
|
||||||
from ocrmypdf._plugin_manager import get_plugin_manager
|
from ocrmypdf._plugin_manager import get_plugin_manager
|
||||||
from ocrmypdf._sync import run_pipeline
|
from ocrmypdf._sync import run_pipeline
|
||||||
@@ -112,14 +109,7 @@ def configure_logging(
|
|||||||
else:
|
else:
|
||||||
fmt = '%(pageno)s%(message)s'
|
fmt = '%(pageno)s%(message)s'
|
||||||
|
|
||||||
use_colors = progress_bar_friendly
|
|
||||||
formatter = None
|
formatter = None
|
||||||
if use_colors:
|
|
||||||
use_colors = enable_ansi_support()
|
|
||||||
if use_colors:
|
|
||||||
use_colors = coloredlogs.terminal_supports_colors()
|
|
||||||
if use_colors:
|
|
||||||
formatter = coloredlogs.ColoredFormatter(fmt=fmt)
|
|
||||||
|
|
||||||
if not formatter:
|
if not formatter:
|
||||||
formatter = logging.Formatter(fmt=fmt)
|
formatter = logging.Formatter(fmt=fmt)
|
||||||
@@ -248,6 +238,7 @@ def ocr( # noqa: ruff: disable=D417
|
|||||||
user_words: os.PathLike | None = None,
|
user_words: os.PathLike | None = None,
|
||||||
user_patterns: os.PathLike | None = None,
|
user_patterns: os.PathLike | None = None,
|
||||||
fast_web_view: float | None = None,
|
fast_web_view: float | None = None,
|
||||||
|
continue_on_soft_render_error: bool | None = None,
|
||||||
plugins: Iterable[StrPath] | None = None,
|
plugins: Iterable[StrPath] | None = None,
|
||||||
plugin_manager=None,
|
plugin_manager=None,
|
||||||
keep_temporary_files: bool | None = None,
|
keep_temporary_files: bool | None = None,
|
||||||
|
|||||||
@@ -16,10 +16,10 @@ from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor, as_compl
|
|||||||
from contextlib import suppress
|
from contextlib import suppress
|
||||||
from typing import Callable, Iterable, Type, Union
|
from typing import Callable, Iterable, Type, Union
|
||||||
|
|
||||||
from tqdm import tqdm
|
from rich.console import Console as RichConsole
|
||||||
|
|
||||||
from ocrmypdf import Executor, hookimpl
|
from ocrmypdf import Executor, hookimpl
|
||||||
from ocrmypdf._logging import TqdmConsole
|
from ocrmypdf._logging import RichLoggingHandler, RichTqdmProgressAdapter
|
||||||
from ocrmypdf.exceptions import InputFileError
|
from ocrmypdf.exceptions import InputFileError
|
||||||
from ocrmypdf.helpers import remove_all_log_handlers
|
from ocrmypdf.helpers import remove_all_log_handlers
|
||||||
|
|
||||||
@@ -168,13 +168,20 @@ def get_executor(progressbar_class):
|
|||||||
return StandardExecutor(pbar_class=progressbar_class)
|
return StandardExecutor(pbar_class=progressbar_class)
|
||||||
|
|
||||||
|
|
||||||
|
RICH_CONSOLE = RichConsole(stderr=True)
|
||||||
|
|
||||||
|
|
||||||
@hookimpl
|
@hookimpl
|
||||||
def get_progressbar_class():
|
def get_progressbar_class():
|
||||||
"""Return the default progress bar class."""
|
"""Return the default progress bar class."""
|
||||||
return tqdm
|
|
||||||
|
def partial_RichTqdmProgressAdapter(*args, **kwargs):
|
||||||
|
return RichTqdmProgressAdapter(*args, **kwargs, console=RICH_CONSOLE)
|
||||||
|
|
||||||
|
return partial_RichTqdmProgressAdapter
|
||||||
|
|
||||||
|
|
||||||
@hookimpl
|
@hookimpl
|
||||||
def get_logging_console():
|
def get_logging_console():
|
||||||
"""Return the default logging console handler."""
|
"""Return the default logging console handler."""
|
||||||
return logging.StreamHandler(stream=TqdmConsole(sys.stderr))
|
return RichLoggingHandler(console=RICH_CONSOLE)
|
||||||
|
|||||||
@@ -45,6 +45,7 @@ def rasterize_pdf_page(
|
|||||||
page_dpi,
|
page_dpi,
|
||||||
rotation,
|
rotation,
|
||||||
filter_vector,
|
filter_vector,
|
||||||
|
stop_on_soft_error,
|
||||||
):
|
):
|
||||||
"""Rasterize a single page of a PDF file using Ghostscript."""
|
"""Rasterize a single page of a PDF file using Ghostscript."""
|
||||||
ghostscript.rasterize_pdf(
|
ghostscript.rasterize_pdf(
|
||||||
@@ -56,6 +57,7 @@ def rasterize_pdf_page(
|
|||||||
page_dpi=page_dpi,
|
page_dpi=page_dpi,
|
||||||
rotation=rotation,
|
rotation=rotation,
|
||||||
filter_vector=filter_vector,
|
filter_vector=filter_vector,
|
||||||
|
stop_on_error=stop_on_soft_error,
|
||||||
)
|
)
|
||||||
return output_file
|
return output_file
|
||||||
|
|
||||||
@@ -69,6 +71,7 @@ def generate_pdfa(
|
|||||||
pdf_version,
|
pdf_version,
|
||||||
pdfa_part,
|
pdfa_part,
|
||||||
progressbar_class,
|
progressbar_class,
|
||||||
|
stop_on_soft_error,
|
||||||
):
|
):
|
||||||
"""Generate a PDF/A from the list of PDF pages and PDF/A metadata."""
|
"""Generate a PDF/A from the list of PDF pages and PDF/A metadata."""
|
||||||
ghostscript.generate_pdfa(
|
ghostscript.generate_pdfa(
|
||||||
@@ -78,5 +81,6 @@ def generate_pdfa(
|
|||||||
pdf_version=pdf_version,
|
pdf_version=pdf_version,
|
||||||
pdfa_part=pdfa_part,
|
pdfa_part=pdfa_part,
|
||||||
progressbar_class=progressbar_class,
|
progressbar_class=progressbar_class,
|
||||||
|
stop_on_error=stop_on_soft_error,
|
||||||
)
|
)
|
||||||
return output_file
|
return output_file
|
||||||
|
|||||||
+18
-3
@@ -84,16 +84,16 @@ class LanguageSetAction(argparse.Action):
|
|||||||
def __init__(self, option_strings, dest, default=None, **kwargs):
|
def __init__(self, option_strings, dest, default=None, **kwargs):
|
||||||
"""Initialize the action."""
|
"""Initialize the action."""
|
||||||
if default is None:
|
if default is None:
|
||||||
default = set()
|
default = list()
|
||||||
super().__init__(option_strings, dest, default=default, **kwargs)
|
super().__init__(option_strings, dest, default=default, **kwargs)
|
||||||
|
|
||||||
def __call__(self, parser, namespace, values, option_string=None):
|
def __call__(self, parser, namespace, values, option_string=None):
|
||||||
"""Add a language to the set."""
|
"""Add a language to the set."""
|
||||||
dest = getattr(namespace, self.dest)
|
dest = getattr(namespace, self.dest)
|
||||||
if '+' in values:
|
if '+' in values:
|
||||||
dest.update(lang for lang in values.split('+'))
|
[dest.append(lang) for lang in values.split('+')]
|
||||||
else:
|
else:
|
||||||
dest.add(values)
|
dest.append(values)
|
||||||
|
|
||||||
|
|
||||||
def get_parser():
|
def get_parser():
|
||||||
@@ -359,6 +359,13 @@ Online documentation is located at:
|
|||||||
help="Skip OCR on pages larger than the specified amount of megapixels, "
|
help="Skip OCR on pages larger than the specified amount of megapixels, "
|
||||||
"but include skipped pages in final output",
|
"but include skipped pages in final output",
|
||||||
)
|
)
|
||||||
|
ocrsettings.add_argument(
|
||||||
|
'--invalidate-digital-signatures',
|
||||||
|
action='store_true',
|
||||||
|
help="Normally, OCRmyPDF will refuse to OCR a PDF that has a digital "
|
||||||
|
"signature. This option allows OCR to proceed, but the digital signature "
|
||||||
|
"will be invalidated.",
|
||||||
|
)
|
||||||
|
|
||||||
advanced = parser.add_argument_group(
|
advanced = parser.add_argument_group(
|
||||||
"Advanced", "Advanced options to control OCRmyPDF"
|
"Advanced", "Advanced options to control OCRmyPDF"
|
||||||
@@ -420,6 +427,14 @@ Online documentation is located at:
|
|||||||
"which do not benefit. If the threshold is 0 it will be apply to all files. "
|
"which do not benefit. If the threshold is 0 it will be apply to all files. "
|
||||||
"Set the threshold very high to disable.",
|
"Set the threshold very high to disable.",
|
||||||
)
|
)
|
||||||
|
advanced.add_argument(
|
||||||
|
'--continue-on-soft-render-error',
|
||||||
|
action='store_true',
|
||||||
|
help="Continue processing pages after a recoverable PDF rendering error. "
|
||||||
|
"A recoverable error is one that does not prevent the page from being "
|
||||||
|
"rendered, but may result in visual differences compared to the input "
|
||||||
|
"file. Missing fonts are a typical source of these errors.",
|
||||||
|
)
|
||||||
advanced.add_argument(
|
advanced.add_argument(
|
||||||
'--plugin',
|
'--plugin',
|
||||||
dest='plugins',
|
dest='plugins',
|
||||||
|
|||||||
@@ -108,6 +108,18 @@ class EncryptedPdfError(ExitCodeException):
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class DigitalSignatureError(ExitCodeException):
|
||||||
|
"""PDF has a digital signature."""
|
||||||
|
|
||||||
|
exit_code = ExitCode.input_file
|
||||||
|
message = dedent(
|
||||||
|
"""\
|
||||||
|
Input PDF has a digital signature. OCR would alter the document,
|
||||||
|
invalidating the signature.
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class TesseractConfigError(ExitCodeException):
|
class TesseractConfigError(ExitCodeException):
|
||||||
"""Tesseract config can't be parsed."""
|
"""Tesseract config can't be parsed."""
|
||||||
|
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ worker communicates only with the main process.
|
|||||||
|
|
||||||
This is not without drawbacks. If the tasks are not "even" in size, which cannot
|
This is not without drawbacks. If the tasks are not "even" in size, which cannot
|
||||||
be guaranteed, some workers may end up with too much work while others are idle.
|
be guaranteed, some workers may end up with too much work while others are idle.
|
||||||
It is less efficient than the standard implementation, so not th edefault.
|
It is less efficient than the standard implementation, so not the default.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|||||||
@@ -21,7 +21,9 @@ from typing import Container, Iterable, Iterator, Mapping, NamedTuple, Sequence,
|
|||||||
from warnings import warn
|
from warnings import warn
|
||||||
|
|
||||||
from pikepdf import (
|
from pikepdf import (
|
||||||
|
Name,
|
||||||
Object,
|
Object,
|
||||||
|
Page,
|
||||||
Pdf,
|
Pdf,
|
||||||
PdfImage,
|
PdfImage,
|
||||||
PdfInlineImage,
|
PdfInlineImage,
|
||||||
@@ -491,15 +493,15 @@ def _image_xobjects(container) -> Iterator[tuple[Object, str]]:
|
|||||||
since the object does not know its own name.
|
since the object does not know its own name.
|
||||||
|
|
||||||
"""
|
"""
|
||||||
if '/Resources' not in container:
|
if Name.Resources not in container:
|
||||||
return
|
return
|
||||||
resources = container['/Resources']
|
resources = container[Name.Resources]
|
||||||
if '/XObject' not in resources:
|
if Name.XObject not in resources:
|
||||||
return
|
return
|
||||||
for key, candidate in resources['/XObject'].items():
|
for key, candidate in resources[Name.XObject].items():
|
||||||
if candidate is None or '/Subtype' not in candidate:
|
if candidate is None or Name.Subtype not in candidate:
|
||||||
continue
|
continue
|
||||||
if candidate['/Subtype'] == '/Image':
|
if candidate[Name.Subtype] == Name.Image:
|
||||||
pdfimage = candidate
|
pdfimage = candidate
|
||||||
yield (pdfimage, key)
|
yield (pdfimage, key)
|
||||||
|
|
||||||
@@ -535,15 +537,15 @@ def _find_form_xobject_images(pdf: Pdf, container: Object, contentsinfo: Content
|
|||||||
The container may be a page, or a parent Form XObject.
|
The container may be a page, or a parent Form XObject.
|
||||||
|
|
||||||
"""
|
"""
|
||||||
if '/Resources' not in container:
|
if Name.Resources not in container:
|
||||||
return
|
return
|
||||||
resources = container['/Resources']
|
resources = container[Name.Resources]
|
||||||
if '/XObject' not in resources:
|
if Name.XObject not in resources:
|
||||||
return
|
return
|
||||||
xobjs = resources['/XObject'].as_dict()
|
xobjs = resources[Name.XObject].as_dict()
|
||||||
for xobj in xobjs:
|
for xobj in xobjs:
|
||||||
candidate = xobjs[xobj]
|
candidate = xobjs[xobj]
|
||||||
if candidate is None or candidate['/Subtype'] != '/Form':
|
if candidate is None or candidate[Name.Subtype] != Name.Form:
|
||||||
continue
|
continue
|
||||||
|
|
||||||
form_xobject = candidate
|
form_xobject = candidate
|
||||||
@@ -581,16 +583,19 @@ def _process_content_streams(
|
|||||||
downsampling.
|
downsampling.
|
||||||
|
|
||||||
"""
|
"""
|
||||||
if container.get('/Type') == '/Page' and '/Contents' in container:
|
if container.get(Name.Type) == Name.Page and Name.Contents in container:
|
||||||
initial_shorthand = shorthand or UNIT_SQUARE
|
initial_shorthand = shorthand or UNIT_SQUARE
|
||||||
elif container.get('/Type') == '/XObject' and container['/Subtype'] == '/Form':
|
elif (
|
||||||
|
container.get(Name.Type) == Name.XObject
|
||||||
|
and container[Name.Subtype] == Name.Form
|
||||||
|
):
|
||||||
# Set the CTM to the state it was when the "Do" operator was
|
# Set the CTM to the state it was when the "Do" operator was
|
||||||
# encountered that is drawing this instance of the Form XObject
|
# encountered that is drawing this instance of the Form XObject
|
||||||
ctm = PdfMatrix(shorthand) if shorthand else PdfMatrix.identity()
|
ctm = PdfMatrix(shorthand) if shorthand else PdfMatrix.identity()
|
||||||
|
|
||||||
# A Form XObject may provide its own matrix to map form space into
|
# A Form XObject may provide its own matrix to map form space into
|
||||||
# user space. Get this if one exists
|
# user space. Get this if one exists
|
||||||
form_shorthand = container.get('/Matrix', PdfMatrix.identity())
|
form_shorthand = container.get(Name.Matrix, PdfMatrix.identity())
|
||||||
form_matrix = PdfMatrix(form_shorthand)
|
form_matrix = PdfMatrix(form_shorthand)
|
||||||
|
|
||||||
# Concatenate form matrix with CTM to ensure CTM is correct for
|
# Concatenate form matrix with CTM to ensure CTM is correct for
|
||||||
@@ -771,7 +776,7 @@ class PageInfo:
|
|||||||
check_pages: Container[int],
|
check_pages: Container[int],
|
||||||
detailed_analysis: bool,
|
detailed_analysis: bool,
|
||||||
):
|
):
|
||||||
page = pdf.pages[pageno]
|
page: Page = pdf.pages[pageno]
|
||||||
mediabox = [Decimal(d) for d in page.MediaBox.as_list()]
|
mediabox = [Decimal(d) for d in page.MediaBox.as_list()]
|
||||||
width_pt = mediabox[2] - mediabox[0]
|
width_pt = mediabox[2] - mediabox[0]
|
||||||
height_pt = mediabox[3] - mediabox[1]
|
height_pt = mediabox[3] - mediabox[1]
|
||||||
@@ -779,7 +784,7 @@ class PageInfo:
|
|||||||
check_this_page = pageno in check_pages
|
check_this_page = pageno in check_pages
|
||||||
|
|
||||||
if check_this_page and detailed_analysis:
|
if check_this_page and detailed_analysis:
|
||||||
pscript5_mode = str(pdf.docinfo.get('/Creator')).startswith('PScript5')
|
pscript5_mode = str(pdf.docinfo.get(Name.Creator)).startswith('PScript5')
|
||||||
miner = get_page_analysis(infile, pageno, pscript5_mode)
|
miner = get_page_analysis(infile, pageno, pscript5_mode)
|
||||||
self._textboxes = list(simplify_textboxes(miner, get_text_boxes))
|
self._textboxes = list(simplify_textboxes(miner, get_text_boxes))
|
||||||
bboxes = (box.bbox for box in self._textboxes)
|
bboxes = (box.bbox for box in self._textboxes)
|
||||||
@@ -789,17 +794,13 @@ class PageInfo:
|
|||||||
self._textboxes = []
|
self._textboxes = []
|
||||||
self._has_text = None # i.e. "no information"
|
self._has_text = None # i.e. "no information"
|
||||||
|
|
||||||
userunit = page.get('/UserUnit', Decimal(1.0))
|
userunit = page.get(Name.UserUnit, Decimal(1.0))
|
||||||
if not isinstance(userunit, Decimal):
|
if not isinstance(userunit, Decimal):
|
||||||
userunit = Decimal(userunit)
|
userunit = Decimal(userunit)
|
||||||
self._userunit = userunit
|
self._userunit = userunit
|
||||||
self._width_inches = width_pt * userunit / Decimal(72.0)
|
self._width_inches = width_pt * userunit / Decimal(72.0)
|
||||||
self._height_inches = height_pt * userunit / Decimal(72.0)
|
self._height_inches = height_pt * userunit / Decimal(72.0)
|
||||||
|
self._rotate = int(getattr(page.obj, 'Rotate', 0))
|
||||||
try:
|
|
||||||
self._rotate = int(page['/Rotate'])
|
|
||||||
except KeyError:
|
|
||||||
self._rotate = 0
|
|
||||||
|
|
||||||
userunit_shorthand = (userunit, 0, 0, userunit, 0, 0)
|
userunit_shorthand = (userunit, 0, 0, userunit, 0, 0)
|
||||||
|
|
||||||
@@ -953,6 +954,10 @@ DEFAULT_EXECUTOR = SerialExecutor()
|
|||||||
class PdfInfo:
|
class PdfInfo:
|
||||||
"""Get summary information about a PDF."""
|
"""Get summary information about a PDF."""
|
||||||
|
|
||||||
|
_has_acroform: bool = False
|
||||||
|
_has_signature: bool = False
|
||||||
|
_needs_rendering: bool = False
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
infile,
|
infile,
|
||||||
@@ -980,13 +985,13 @@ class PdfInfo:
|
|||||||
check_pages=check_pages,
|
check_pages=check_pages,
|
||||||
detailed_analysis=detailed_analysis,
|
detailed_analysis=detailed_analysis,
|
||||||
)
|
)
|
||||||
self._needs_rendering = pdf.Root.get('/NeedsRendering', False)
|
self._needs_rendering = pdf.Root.get(Name.NeedsRendering, False)
|
||||||
self._has_acroform = False
|
if Name.AcroForm in pdf.Root:
|
||||||
if '/AcroForm' in pdf.Root:
|
if len(pdf.Root.AcroForm.get(Name.Fields, [])) > 0:
|
||||||
if len(pdf.Root.AcroForm.get('/Fields', [])) > 0:
|
|
||||||
self._has_acroform = True
|
self._has_acroform = True
|
||||||
elif '/XFA' in pdf.Root.AcroForm:
|
elif Name.XFA in pdf.Root.AcroForm:
|
||||||
self._has_acroform = True
|
self._has_acroform = True
|
||||||
|
self._has_signature = bool(pdf.Root.AcroForm.get(Name.SigFlags, 0) & 1)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def pages(self) -> Sequence[PageInfo | None]:
|
def pages(self) -> Sequence[PageInfo | None]:
|
||||||
@@ -1006,9 +1011,14 @@ class PdfInfo:
|
|||||||
|
|
||||||
@property
|
@property
|
||||||
def has_acroform(self) -> bool:
|
def has_acroform(self) -> bool:
|
||||||
"""Return True if any page has an AcroForm."""
|
"""Return True if the document catalog has an AcroForm."""
|
||||||
return self._has_acroform
|
return self._has_acroform
|
||||||
|
|
||||||
|
@property
|
||||||
|
def has_signature(self) -> bool:
|
||||||
|
"""Return True if the document annotations has a digital signature."""
|
||||||
|
return self._has_signature
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def filename(self) -> str | Path:
|
def filename(self) -> str | Path:
|
||||||
"""Return filename of PDF."""
|
"""Return filename of PDF."""
|
||||||
|
|||||||
@@ -203,6 +203,7 @@ def rasterize_pdf_page(
|
|||||||
page_dpi: Resolution | None,
|
page_dpi: Resolution | None,
|
||||||
rotation: int | None,
|
rotation: int | None,
|
||||||
filter_vector: bool,
|
filter_vector: bool,
|
||||||
|
stop_on_soft_error: bool,
|
||||||
) -> Path:
|
) -> Path:
|
||||||
"""Rasterize one page of a PDF at resolution raster_dpi in canvas units.
|
"""Rasterize one page of a PDF at resolution raster_dpi in canvas units.
|
||||||
|
|
||||||
@@ -213,19 +214,26 @@ def rasterize_pdf_page(
|
|||||||
Args:
|
Args:
|
||||||
input_file: The PDF to rasterize.
|
input_file: The PDF to rasterize.
|
||||||
output_file: The desired name of the rasterized image.
|
output_file: The desired name of the rasterized image.
|
||||||
raster_device: Type of image to produce at output_file
|
raster_device: Type of image to produce at output_file.
|
||||||
raster_dpi: Resolution at which to rasterize page
|
raster_dpi: Resolution in dots per inch at which to rasterize page.
|
||||||
pageno: Page number to rasterize (beginning at page 1)
|
pageno: Page number to rasterize (beginning at page 1).
|
||||||
page_dpi: Resolution, overriding output image DPI
|
page_dpi: Resolution, overriding output image DPI.
|
||||||
rotation: Cardinal angle, clockwise, to rotate page
|
rotation: Cardinal angle, clockwise, to rotate page.
|
||||||
filter_vector: If True, remove vector graphics objects
|
filter_vector: If True, remove vector graphics objects.
|
||||||
|
stop_on_soft_error: If there is an "soft error" such that PDF page image
|
||||||
|
generation can proceed, but may visually differ from the original,
|
||||||
|
the implementer of this hook should raise a detailed exception. If
|
||||||
|
``False``, continue processing and report by logging it. If the hook
|
||||||
|
cannot proceed, it should always raise an exception, regardless of
|
||||||
|
this setting. One "soft error" would be a missing font that is
|
||||||
|
required to properly rasterize the PDF.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Path: output_file if successful
|
Path: output_file if successful
|
||||||
Note:
|
Note:
|
||||||
This hook will be called from child processes. Modifying global state
|
This hook will be called from child processes. Modifying global state
|
||||||
will not affect the main process or other child processes.
|
will not affect the main process or other child processes.
|
||||||
|
|
||||||
|
|
||||||
Note:
|
Note:
|
||||||
This is a :ref:`firstresult hook<firstresult>`.
|
This is a :ref:`firstresult hook<firstresult>`.
|
||||||
"""
|
"""
|
||||||
@@ -462,6 +470,7 @@ def generate_pdfa(
|
|||||||
pdf_version: str,
|
pdf_version: str,
|
||||||
pdfa_part: str,
|
pdfa_part: str,
|
||||||
progressbar_class,
|
progressbar_class,
|
||||||
|
stop_on_soft_error: bool,
|
||||||
) -> Path:
|
) -> Path:
|
||||||
"""Generate a PDF/A.
|
"""Generate a PDF/A.
|
||||||
|
|
||||||
@@ -492,6 +501,12 @@ def generate_pdfa(
|
|||||||
and the name of the work units ("page"). Then ``instance.update()``
|
and the name of the work units ("page"). Then ``instance.update()``
|
||||||
will be called when a work unit is completed. If ``None``, no
|
will be called when a work unit is completed. If ``None``, no
|
||||||
progress information is reported.
|
progress information is reported.
|
||||||
|
stop_on_soft_error: If there is an "soft error" such that PDF/A generation
|
||||||
|
can proceed and produce a valid PDF/A, but output may be invalid or
|
||||||
|
may not visually resemble the original, the implementer of this hook
|
||||||
|
should raise a detailed exception. If ``False``, continue processing
|
||||||
|
and report by logging it. If the hook cannot proceed, it should always
|
||||||
|
raise an exception, regardless of this setting.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Path: If successful, the hook should return ``output_file``.
|
Path: If successful, the hook should return ``output_file``.
|
||||||
|
|||||||
@@ -118,8 +118,6 @@ def run_polling_stderr(
|
|||||||
def _fix_process_args(
|
def _fix_process_args(
|
||||||
args: Args, env: OsEnviron | None, kwargs
|
args: Args, env: OsEnviron | None, kwargs
|
||||||
) -> tuple[Args, OsEnviron, logging.Logger, bool]:
|
) -> tuple[Args, OsEnviron, logging.Logger, bool]:
|
||||||
assert 'universal_newlines' not in kwargs, "Use text= instead of universal_newlines"
|
|
||||||
|
|
||||||
if not env:
|
if not env:
|
||||||
env = os.environ
|
env = os.environ
|
||||||
|
|
||||||
|
|||||||
@@ -31,6 +31,7 @@ def generate_pdfa(pdf_pages, pdfmark, output_file, compression, pdf_version, pdf
|
|||||||
pdf_version=pdf_version,
|
pdf_version=pdf_version,
|
||||||
pdfa_part=pdfa_part,
|
pdfa_part=pdfa_part,
|
||||||
progressbar_class=None,
|
progressbar_class=None,
|
||||||
|
stop_on_soft_error=True,
|
||||||
)
|
)
|
||||||
mock.assert_called_once()
|
mock.assert_called_once()
|
||||||
return output_file
|
return output_file
|
||||||
|
|||||||
@@ -33,6 +33,7 @@ def generate_pdfa(pdf_pages, pdfmark, output_file, compression, pdf_version, pdf
|
|||||||
pdf_version=pdf_version,
|
pdf_version=pdf_version,
|
||||||
pdfa_part=pdfa_part,
|
pdfa_part=pdfa_part,
|
||||||
progressbar_class=None,
|
progressbar_class=None,
|
||||||
|
stop_on_soft_error=True,
|
||||||
)
|
)
|
||||||
mock.assert_called()
|
mock.assert_called()
|
||||||
return output_file
|
return output_file
|
||||||
|
|||||||
@@ -39,6 +39,7 @@ def rasterize_pdf_page(
|
|||||||
page_dpi=page_dpi,
|
page_dpi=page_dpi,
|
||||||
rotation=rotation,
|
rotation=rotation,
|
||||||
filter_vector=filter_vector,
|
filter_vector=filter_vector,
|
||||||
|
stop_on_soft_error=True,
|
||||||
)
|
)
|
||||||
mock.assert_called()
|
mock.assert_called()
|
||||||
return output_file
|
return output_file
|
||||||
|
|||||||
@@ -0,0 +1,47 @@
|
|||||||
|
# SPDX-FileCopyrightText: 2022 James R. Barlow
|
||||||
|
# SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
from subprocess import CalledProcessError
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from ocrmypdf import hookimpl
|
||||||
|
from ocrmypdf.builtin_plugins import ghostscript
|
||||||
|
from ocrmypdf.subprocess import run
|
||||||
|
|
||||||
|
|
||||||
|
def fail_if_stoponerror(args, **kwargs):
|
||||||
|
if '-dPDFSTOPONERROR' in args:
|
||||||
|
raise CalledProcessError(1, 'gs', output=b"", stderr=b"PDF STOP ON ERROR")
|
||||||
|
return run(args, **kwargs)
|
||||||
|
|
||||||
|
|
||||||
|
@hookimpl
|
||||||
|
def rasterize_pdf_page(
|
||||||
|
input_file,
|
||||||
|
output_file,
|
||||||
|
raster_device,
|
||||||
|
raster_dpi,
|
||||||
|
pageno,
|
||||||
|
page_dpi,
|
||||||
|
rotation,
|
||||||
|
filter_vector,
|
||||||
|
stop_on_soft_error,
|
||||||
|
) -> Path:
|
||||||
|
with patch('ocrmypdf._exec.ghostscript.run') as mock:
|
||||||
|
mock.side_effect = fail_if_stoponerror
|
||||||
|
ghostscript.rasterize_pdf_page(
|
||||||
|
input_file=input_file,
|
||||||
|
output_file=output_file,
|
||||||
|
raster_device=raster_device,
|
||||||
|
raster_dpi=raster_dpi,
|
||||||
|
pageno=pageno,
|
||||||
|
page_dpi=page_dpi,
|
||||||
|
rotation=rotation,
|
||||||
|
filter_vector=filter_vector,
|
||||||
|
stop_on_soft_error=stop_on_soft_error,
|
||||||
|
)
|
||||||
|
mock.assert_called()
|
||||||
|
return output_file
|
||||||
@@ -28,6 +28,7 @@ def generate_pdfa(pdf_pages, pdfmark, output_file, compression, pdf_version, pdf
|
|||||||
pdf_version=pdf_version,
|
pdf_version=pdf_version,
|
||||||
pdfa_part=pdfa_part,
|
pdfa_part=pdfa_part,
|
||||||
progressbar_class=None,
|
progressbar_class=None,
|
||||||
|
stop_on_soft_error=True,
|
||||||
)
|
)
|
||||||
mock.assert_called()
|
mock.assert_called()
|
||||||
return output_file
|
return output_file
|
||||||
|
|||||||
@@ -0,0 +1,44 @@
|
|||||||
|
# SPDX-FileCopyrightText: 2022 James R. Barlow
|
||||||
|
# SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
from subprocess import CalledProcessError
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from ocrmypdf import hookimpl
|
||||||
|
from ocrmypdf.builtin_plugins import ghostscript
|
||||||
|
from ocrmypdf.subprocess import run_polling_stderr
|
||||||
|
|
||||||
|
|
||||||
|
def fail_if_stoponerror(args, **kwargs):
|
||||||
|
if '-dPDFSTOPONERROR' in args:
|
||||||
|
raise CalledProcessError(1, 'gs', output=b"", stderr=b"PDF STOP ON ERROR")
|
||||||
|
return run_polling_stderr(args, **kwargs)
|
||||||
|
|
||||||
|
|
||||||
|
@hookimpl
|
||||||
|
def generate_pdfa(
|
||||||
|
pdf_pages,
|
||||||
|
pdfmark,
|
||||||
|
output_file,
|
||||||
|
compression,
|
||||||
|
pdf_version,
|
||||||
|
pdfa_part,
|
||||||
|
stop_on_soft_error,
|
||||||
|
):
|
||||||
|
with patch('ocrmypdf._exec.ghostscript.run_polling_stderr') as mock:
|
||||||
|
mock.side_effect = fail_if_stoponerror
|
||||||
|
ghostscript.generate_pdfa(
|
||||||
|
pdf_pages=pdf_pages,
|
||||||
|
pdfmark=pdfmark,
|
||||||
|
output_file=output_file,
|
||||||
|
compression=compression,
|
||||||
|
pdf_version=pdf_version,
|
||||||
|
pdfa_part=pdfa_part,
|
||||||
|
progressbar_class=None,
|
||||||
|
stop_on_soft_error=stop_on_soft_error,
|
||||||
|
)
|
||||||
|
mock.assert_called()
|
||||||
|
return output_file
|
||||||
@@ -0,0 +1,115 @@
|
|||||||
|
%PDF-1.3
|
||||||
|
%¿÷¢þ
|
||||||
|
1 0 obj
|
||||||
|
<< /Pages 2 0 R /Type /Catalog >>
|
||||||
|
endobj
|
||||||
|
2 0 obj
|
||||||
|
<< /Count 1 /Kids [ 3 0 R ] /Type /Pages >>
|
||||||
|
endobj
|
||||||
|
3 0 obj
|
||||||
|
<< /MediaBox [ 0 0 720 720 ] /Parent 2 0 R /Resources << >> /Type /Page >>
|
||||||
|
endobj
|
||||||
|
xref
|
||||||
|
0 4
|
||||||
|
0000000000 65535 f
|
||||||
|
0000000015 00000 n
|
||||||
|
0000000064 00000 n
|
||||||
|
0000000123 00000 n
|
||||||
|
trailer << /Root 1 0 R /Size 4 /ID [<52bba3c78160d0c6e851b59110e5d076><52bba3c78160d0c6e851b59110e5d076>] >>
|
||||||
|
startxref
|
||||||
|
213
|
||||||
|
%%EOF
|
||||||
|
%BeginExifToolUpdate
|
||||||
|
1 0 obj
|
||||||
|
<<
|
||||||
|
/Pages 2 0 R
|
||||||
|
/Type /Catalog
|
||||||
|
/Metadata 5 0 R
|
||||||
|
>>
|
||||||
|
endobj
|
||||||
|
4 0 obj
|
||||||
|
<<
|
||||||
|
/Author (yXaryipxyRk9dVjWjSSaVaNCKeLRgEVzPRMp)
|
||||||
|
/Keywords (s9EeALwUg7urA7fnnhm5EtUyC54sW2WPUzqh)
|
||||||
|
/Subject (t49vimctvnuH7ZeAjAkv52ACvWFjcnm5MPJr)
|
||||||
|
/Title (NFY5f7Ft2DWMkxLhXwxvFf7eWR2KeK3vEDcd)
|
||||||
|
>>
|
||||||
|
endobj
|
||||||
|
5 0 obj
|
||||||
|
<<
|
||||||
|
/Type /Metadata
|
||||||
|
/Subtype /XML
|
||||||
|
/Length 3232
|
||||||
|
>>
|
||||||
|
stream
|
||||||
|
<?xpacket begin='' id='W5M0MpCehiHzreSzNTczkc9d'?>
|
||||||
|
<x:xmpmeta xmlns:x='adobe:ns:meta/' x:xmptk='Image::ExifTool 12.60'>
|
||||||
|
<rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'>
|
||||||
|
|
||||||
|
<rdf:Description rdf:about=''
|
||||||
|
xmlns:dc='http://purl.org/dc/elements/1.1/'>
|
||||||
|
<dc:subject>
|
||||||
|
<rdf:Bag>
|
||||||
|
<rdf:li>t49vimctvnuH7ZeAjAkv52ACvWFjcnm5MPJr</rdf:li>
|
||||||
|
</rdf:Bag>
|
||||||
|
</dc:subject>
|
||||||
|
<dc:title>
|
||||||
|
<rdf:Alt>
|
||||||
|
<rdf:li xml:lang='x-default'>NFY5f7Ft2DWMkxLhXwxvFf7eWR2KeK3vEDcd</rdf:li>
|
||||||
|
</rdf:Alt>
|
||||||
|
</dc:title>
|
||||||
|
</rdf:Description>
|
||||||
|
|
||||||
|
<rdf:Description rdf:about=''
|
||||||
|
xmlns:pdf='http://ns.adobe.com/pdf/1.3/'>
|
||||||
|
<pdf:Author>yXaryipxyRk9dVjWjSSaVaNCKeLRgEVzPRMp</pdf:Author>
|
||||||
|
<pdf:Keywords>s9EeALwUg7urA7fnnhm5EtUyC54sW2WPUzqh</pdf:Keywords>
|
||||||
|
</rdf:Description>
|
||||||
|
</rdf:RDF>
|
||||||
|
</x:xmpmeta>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<?xpacket end='w'?>
|
||||||
|
endstream
|
||||||
|
endobj
|
||||||
|
xref
|
||||||
|
0 2
|
||||||
|
0000000000 65535 f
|
||||||
|
0000000452 00000 n
|
||||||
|
4 2
|
||||||
|
0000000517 00000 n
|
||||||
|
0000000728 00000 n
|
||||||
|
trailer
|
||||||
|
<<
|
||||||
|
/Root 1 0 R
|
||||||
|
/Size 6
|
||||||
|
/ID [ <52bba3c78160d0c6e851b59110e5d076> <54bba3c78160d0c6e851b59110e5d076> ]
|
||||||
|
/Info 4 0 R
|
||||||
|
/Prev 213
|
||||||
|
>>
|
||||||
|
%EndExifToolUpdate 431
|
||||||
|
startxref
|
||||||
|
4043
|
||||||
|
%%EOF
|
||||||
+25
-3
@@ -5,6 +5,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
|
import pikepdf
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
import ocrmypdf
|
import ocrmypdf
|
||||||
@@ -19,10 +20,11 @@ def acroform(resources):
|
|||||||
return resources / 'acroform.pdf'
|
return resources / 'acroform.pdf'
|
||||||
|
|
||||||
|
|
||||||
def test_acroform_and_redo(acroform, caplog, no_outpdf):
|
def test_acroform_and_redo(acroform, no_outpdf):
|
||||||
with pytest.raises(ocrmypdf.exceptions.InputFileError):
|
with pytest.raises(
|
||||||
|
ocrmypdf.exceptions.InputFileError, match='--redo-ocr is not currently possible'
|
||||||
|
):
|
||||||
check_ocrmypdf(acroform, no_outpdf, '--redo-ocr')
|
check_ocrmypdf(acroform, no_outpdf, '--redo-ocr')
|
||||||
assert '--redo-ocr is not currently possible' in caplog.text
|
|
||||||
|
|
||||||
|
|
||||||
def test_acroform_message(acroform, caplog, outpdf):
|
def test_acroform_message(acroform, caplog, outpdf):
|
||||||
@@ -30,3 +32,23 @@ def test_acroform_message(acroform, caplog, outpdf):
|
|||||||
check_ocrmypdf(acroform, outpdf, '--plugin', 'tests/plugins/tesseract_noop.py')
|
check_ocrmypdf(acroform, outpdf, '--plugin', 'tests/plugins/tesseract_noop.py')
|
||||||
assert 'fillable form' in caplog.text
|
assert 'fillable form' in caplog.text
|
||||||
assert '--force-ocr' in caplog.text
|
assert '--force-ocr' in caplog.text
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def digitally_signed(acroform, outdir):
|
||||||
|
out = outdir / 'acroform_signed.pdf'
|
||||||
|
with pikepdf.open(acroform) as pdf:
|
||||||
|
pdf.Root.AcroForm.SigFlags = 3
|
||||||
|
pdf.save(out)
|
||||||
|
yield out
|
||||||
|
|
||||||
|
|
||||||
|
def test_digital_signature(digitally_signed, no_outpdf):
|
||||||
|
with pytest.raises(ocrmypdf.exceptions.DigitalSignatureError):
|
||||||
|
check_ocrmypdf(digitally_signed, no_outpdf)
|
||||||
|
|
||||||
|
|
||||||
|
def test_digital_signature_invalidate(digitally_signed, no_outpdf):
|
||||||
|
check_ocrmypdf(
|
||||||
|
digitally_signed, no_outpdf, '--force-ocr', '--invalidate-digital-signatures'
|
||||||
|
)
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ from ocrmypdf import helpers
|
|||||||
from .conftest import running_in_docker
|
from .conftest import running_in_docker
|
||||||
|
|
||||||
needs_symlink = pytest.mark.skipif(os.name == 'nt', reason='needs posix symlink')
|
needs_symlink = pytest.mark.skipif(os.name == 'nt', reason='needs posix symlink')
|
||||||
|
windows_only = pytest.mark.skipif(os.name != 'nt', reason="Windows test")
|
||||||
|
|
||||||
|
|
||||||
class TestSafeSymlink:
|
class TestSafeSymlink:
|
||||||
@@ -93,7 +94,7 @@ class TestFileIsWritable:
|
|||||||
assert not helpers.is_file_writable(pathmock)
|
assert not helpers.is_file_writable(pathmock)
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.skipif(os.name != 'nt', reason="Windows test")
|
@windows_only
|
||||||
def test_gs_install_locations():
|
def test_gs_install_locations():
|
||||||
# pylint: disable=import-outside-toplevel
|
# pylint: disable=import-outside-toplevel
|
||||||
from ocrmypdf.subprocess._windows import _gs_version_in_path_key
|
from ocrmypdf.subprocess._windows import _gs_version_in_path_key
|
||||||
@@ -104,7 +105,7 @@ def test_gs_install_locations():
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.skipif(os.name != 'nt', reason="Windows test")
|
@windows_only
|
||||||
def test_shim_paths(tmp_path):
|
def test_shim_paths(tmp_path):
|
||||||
# pylint: disable=import-outside-toplevel
|
# pylint: disable=import-outside-toplevel
|
||||||
from ocrmypdf.subprocess._windows import shim_env_path
|
from ocrmypdf.subprocess._windows import shim_env_path
|
||||||
|
|||||||
+53
-1
@@ -76,6 +76,50 @@ def test_override_metadata(output_type, resources, outpdf):
|
|||||||
assert pdfa_info['output'] == output_type
|
assert pdfa_info['output'] == output_type
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize('output_type', ['pdfa', 'pdf', 'pdfa-1', 'pdfa-2', 'pdfa-3'])
|
||||||
|
@pytest.mark.parametrize('field', ['title', 'author', 'subject', 'keywords'])
|
||||||
|
def test_unset_metadata(output_type, field, resources, outpdf):
|
||||||
|
input_file = resources / 'meta.pdf'
|
||||||
|
|
||||||
|
# magic strings contained in the input pdf metadata
|
||||||
|
meta = {
|
||||||
|
'title': b'NFY5f7Ft2DWMkxLhXwxvFf7eWR2KeK3vEDcd',
|
||||||
|
'author': b'yXaryipxyRk9dVjWjSSaVaNCKeLRgEVzPRMp',
|
||||||
|
'subject': b't49vimctvnuH7ZeAjAkv52ACvWFjcnm5MPJr',
|
||||||
|
'keywords': b's9EeALwUg7urA7fnnhm5EtUyC54sW2WPUzqh'}
|
||||||
|
|
||||||
|
p = run_ocrmypdf(
|
||||||
|
input_file,
|
||||||
|
outpdf,
|
||||||
|
f'--{field}',
|
||||||
|
'',
|
||||||
|
'--output-type',
|
||||||
|
output_type,
|
||||||
|
'--plugin',
|
||||||
|
'tests/plugins/tesseract_noop.py',
|
||||||
|
)
|
||||||
|
|
||||||
|
assert p.returncode == ExitCode.ok, p.stderr
|
||||||
|
|
||||||
|
# We mainly want to ensure that when '' is passed, the corresponding
|
||||||
|
# metadata is unset in the output pdf. Since metedata is not compressed,
|
||||||
|
# the best way to gaurentee the metadata of interest didn't carry
|
||||||
|
# forward is to just check to ensure the corresponding magic string
|
||||||
|
# isn't contained anywhere in the output pdf. We'll also check to ensure
|
||||||
|
# it's in the input pdf and that any values not unset are still in the
|
||||||
|
# output pdf.
|
||||||
|
with open(input_file, 'rb') as before, open(outpdf, 'rb') as after:
|
||||||
|
before_data = before.read()
|
||||||
|
after_data = after.read()
|
||||||
|
|
||||||
|
for k, v in meta.items():
|
||||||
|
assert v in before_data
|
||||||
|
if k == field:
|
||||||
|
assert v not in after_data
|
||||||
|
else:
|
||||||
|
assert v in after_data
|
||||||
|
|
||||||
|
|
||||||
def test_high_unicode(resources, no_outpdf):
|
def test_high_unicode(resources, no_outpdf):
|
||||||
# Ghostscript doesn't support high Unicode, so neither do we, to be
|
# Ghostscript doesn't support high Unicode, so neither do we, to be
|
||||||
# safe
|
# safe
|
||||||
@@ -350,7 +394,15 @@ def test_malformed_docinfo(caplog, resources, outdir):
|
|||||||
pike.save(outdir / 'layers.rendered.pdf', fix_metadata_version=False)
|
pike.save(outdir / 'layers.rendered.pdf', fix_metadata_version=False)
|
||||||
|
|
||||||
options = get_parser().parse_args(
|
options = get_parser().parse_args(
|
||||||
args=['-j', '1', '--output-type', 'pdfa-2', 'a.pdf', 'b.pdf']
|
args=[
|
||||||
|
'-j',
|
||||||
|
'1',
|
||||||
|
'--continue-on-soft-render-error',
|
||||||
|
'--output-type',
|
||||||
|
'pdfa-2',
|
||||||
|
'a.pdf',
|
||||||
|
'b.pdf',
|
||||||
|
]
|
||||||
)
|
)
|
||||||
pdfinfo = PdfInfo(outdir / 'layers.rendered.pdf')
|
pdfinfo = PdfInfo(outdir / 'layers.rendered.pdf')
|
||||||
context = PdfContext(
|
context = PdfContext(
|
||||||
|
|||||||
@@ -275,6 +275,7 @@ def test_rasterize_rotates(resources, tmp_path):
|
|||||||
pageno=1,
|
pageno=1,
|
||||||
rotation=90,
|
rotation=90,
|
||||||
filter_vector=False,
|
filter_vector=False,
|
||||||
|
stop_on_soft_error=True,
|
||||||
)
|
)
|
||||||
with Image.open(img) as im:
|
with Image.open(img) as im:
|
||||||
assert im.size == (83, 200), "Image not rotated"
|
assert im.size == (83, 200), "Image not rotated"
|
||||||
@@ -289,6 +290,7 @@ def test_rasterize_rotates(resources, tmp_path):
|
|||||||
pageno=1,
|
pageno=1,
|
||||||
rotation=180,
|
rotation=180,
|
||||||
filter_vector=False,
|
filter_vector=False,
|
||||||
|
stop_on_soft_error=True,
|
||||||
)
|
)
|
||||||
assert Image.open(img).size == (200, 83), "Image not rotated"
|
assert Image.open(img).size == (200, 83), "Image not rotated"
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,63 @@
|
|||||||
|
# SPDX-FileCopyrightText: 2023 James R. Barlow
|
||||||
|
# SPDX-License-Identifier: MPL-2.0
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from ocrmypdf.exceptions import ExitCode
|
||||||
|
|
||||||
|
from .conftest import run_ocrmypdf
|
||||||
|
|
||||||
|
|
||||||
|
def test_raster_continue_on_soft_error(resources, outpdf):
|
||||||
|
p = run_ocrmypdf(
|
||||||
|
resources / 'francais.pdf',
|
||||||
|
outpdf,
|
||||||
|
'--continue-on-soft-render-error',
|
||||||
|
'--plugin',
|
||||||
|
'tests/plugins/tesseract_noop.py',
|
||||||
|
'--plugin',
|
||||||
|
'tests/plugins/gs_raster_soft_error.py',
|
||||||
|
)
|
||||||
|
assert p.returncode == ExitCode.ok
|
||||||
|
|
||||||
|
|
||||||
|
def test_raster_stop_on_soft_error(resources, outpdf):
|
||||||
|
p = run_ocrmypdf(
|
||||||
|
resources / 'francais.pdf',
|
||||||
|
outpdf,
|
||||||
|
'--plugin',
|
||||||
|
'tests/plugins/tesseract_noop.py',
|
||||||
|
'--plugin',
|
||||||
|
'tests/plugins/gs_raster_soft_error.py',
|
||||||
|
)
|
||||||
|
assert p.returncode == ExitCode.child_process_error
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_continue_on_soft_error(resources, outpdf):
|
||||||
|
p = run_ocrmypdf(
|
||||||
|
resources / 'francais.pdf',
|
||||||
|
outpdf,
|
||||||
|
'--continue-on-soft-render-error',
|
||||||
|
'--plugin',
|
||||||
|
'tests/plugins/tesseract_noop.py',
|
||||||
|
'--plugin',
|
||||||
|
'tests/plugins/gs_render_soft_error.py',
|
||||||
|
)
|
||||||
|
assert p.returncode == ExitCode.ok
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.skipif(os.name == 'nt', reason='Ghostscript on Windows errors out')
|
||||||
|
def test_render_stop_on_soft_error(resources, outpdf):
|
||||||
|
p = run_ocrmypdf(
|
||||||
|
resources / 'francais.pdf',
|
||||||
|
outpdf,
|
||||||
|
'--plugin',
|
||||||
|
'tests/plugins/tesseract_noop.py',
|
||||||
|
'--plugin',
|
||||||
|
'tests/plugins/gs_render_soft_error.py',
|
||||||
|
)
|
||||||
|
assert p.returncode == ExitCode.child_process_error
|
||||||
@@ -166,8 +166,8 @@ def test_language_warning(caplog):
|
|||||||
with patch(
|
with patch(
|
||||||
'ocrmypdf._validation.locale.getlocale', return_value=('en_US', 'UTF-8')
|
'ocrmypdf._validation.locale.getlocale', return_value=('en_US', 'UTF-8')
|
||||||
) as mock:
|
) as mock:
|
||||||
vd.check_options_languages(opts, {'eng'})
|
vd.check_options_languages(opts, ['eng'])
|
||||||
assert opts.languages == {'eng'}
|
assert opts.languages == ['eng']
|
||||||
assert '' in caplog.text
|
assert '' in caplog.text
|
||||||
mock.assert_called_once()
|
mock.assert_called_once()
|
||||||
|
|
||||||
@@ -175,8 +175,8 @@ def test_language_warning(caplog):
|
|||||||
with patch(
|
with patch(
|
||||||
'ocrmypdf._validation.locale.getlocale', return_value=('fr_FR', 'UTF-8')
|
'ocrmypdf._validation.locale.getlocale', return_value=('fr_FR', 'UTF-8')
|
||||||
) as mock:
|
) as mock:
|
||||||
vd.check_options_languages(opts, {'eng'})
|
vd.check_options_languages(opts, ['eng'])
|
||||||
assert opts.languages == {'eng'}
|
assert opts.languages == ['eng']
|
||||||
assert 'assuming --language' in caplog.text
|
assert 'assuming --language' in caplog.text
|
||||||
mock.assert_called_once()
|
mock.assert_called_once()
|
||||||
|
|
||||||
@@ -282,7 +282,7 @@ def test_two_languages():
|
|||||||
parser=get_parser(),
|
parser=get_parser(),
|
||||||
language='fakelang1+fakelang2',
|
language='fakelang1+fakelang2',
|
||||||
),
|
),
|
||||||
{'fakelang1', 'fakelang2'},
|
['fakelang1', 'fakelang2'],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user