Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e748fdcf6f | ||
|
|
8c17c9918e | ||
|
|
ea0dd99d0b | ||
|
|
e0cc67afae | ||
|
|
04f9cbe364 | ||
|
|
99afebd033 | ||
|
|
a6feacc810 | ||
|
|
65e4b1672f | ||
|
|
46cc0dd190 | ||
|
|
678b9fb603 | ||
|
|
49ab0c1f0b | ||
|
|
ab490a7736 | ||
|
|
e4ce1dae35 | ||
|
|
179b812acb | ||
|
|
7f170517ec |
+10
-9
@@ -2,11 +2,9 @@ sudo: required
|
||||
dist: trusty
|
||||
language: python
|
||||
|
||||
cache: pip
|
||||
|
||||
cache:
|
||||
pip: true
|
||||
directories:
|
||||
- $HOME/.ccache
|
||||
- packages
|
||||
- tests/cache
|
||||
|
||||
@@ -28,19 +26,22 @@ before_install:
|
||||
# pip
|
||||
- pip install --upgrade pip
|
||||
|
||||
# Download, make and install unpaper (using ccache)
|
||||
# install unpaper from a previously built .deb since ubuntu's version is too old
|
||||
- mkdir -p packages
|
||||
- "[ -f packages/unpaper_6.1-1.deb ] || wget -q https://dl.dropboxusercontent.com/u/28971240/unpaper_6.1-1.deb -O packages/unpaper_6.1-1.deb"
|
||||
- sudo dpkg -i packages/unpaper_6.1-1.deb
|
||||
|
||||
install:
|
||||
- pip install -r requirements.txt
|
||||
- pip install -r test_requirements.txt
|
||||
- pip install .
|
||||
- pip install -r requirements.txt -r test_requirements.txt
|
||||
|
||||
script:
|
||||
- python setup.py clean
|
||||
- python setup.py install
|
||||
- python setup.py test
|
||||
# "pip install" installs the ocrmypdf into the system environment.
|
||||
# The working copy of ocrmypdf is not an installed version and its cffi
|
||||
# module has not been compiled. As such, move the local copy out of the
|
||||
# way and ensure that the system copy gets tested.
|
||||
- mv ocrmypdf dont_import_this_ocrmypdf
|
||||
- pytest
|
||||
|
||||
os:
|
||||
- linux
|
||||
|
||||
+12
-3
@@ -3,19 +3,28 @@ RELEASE NOTES
|
||||
|
||||
OCRmyPDF uses `semantic versioning <http://semver.org/>`_.
|
||||
|
||||
v4.4.1:
|
||||
=======
|
||||
|
||||
- To prevent a `TIFF output error <https://github.com/python-pillow/Pillow/issues/2206>`_ caused by img2pdf >= 0.2.1 and Pillow <= 3.4.2, dependencies have been tightened
|
||||
- The Tesseract 4.00 simultaenous process limit was increased from 1 to 2, since it was observed that 1 lowers performance
|
||||
- Documentation improvements to describe the ``--tesseract-config`` feature
|
||||
- Added test cases and fixed error handling for ``--tesseract-config``
|
||||
- Tweaks to setup.py to deal with issues in the v4.4 release
|
||||
|
||||
v4.4:
|
||||
=====
|
||||
|
||||
- Tesseract 4.00 is now supported on an experimental basis.
|
||||
|
||||
+ A new rendering option ``--pdf-renderer tess4`` exploits Tesseract 4's new text-only output PDF mode. See the documentation on PDF Renderers for details.
|
||||
+ The ``--tesseract-oem`` argument allows control over the Tesseract 4 OCR
|
||||
engine mode (tesseract's ``--oem``). Use ``--tesseract-oem 2`` to enforce the new LSTM mode.
|
||||
+ The ``--tesseract-oem`` argument allows control over the Tesseract 4 OCR engine mode (tesseract's ``--oem``). Use ``--tesseract-oem 2`` to enforce the new LSTM mode.
|
||||
+ Fixed poor performance with Tesseract 4.00 on Linux
|
||||
|
||||
- Fixed an issue that caused corruption of output to stdout in some cases
|
||||
- Removed test for Pillow JPEG and PNG support, as the minimum supported version of Pillow now enforces this
|
||||
- OCRmyPDF now tests that the intended destination file is writable before proceeding
|
||||
- The test suite now requires ``pytest-helpers-namespace`` to run (but not install)
|
||||
- Significant code reorganization to make OCRmyPDF re-entrant and improve performance. All changes should be backward compatible for the v4.x series.
|
||||
|
||||
+ However, OCRmyPDF's dependency "ruffus" is not re-entrant, so no Python API is available. Scripts should continue to use the command line interface.
|
||||
@@ -485,4 +494,4 @@ v2.2-stable (2014-09-29):
|
||||
|
||||
OCRmyPDF versions 1 and 2 were implemented as shell scripts. OCRmyPDF 3.0+ is a fork that gradually replaced all shell scripts with Python while maintaining the existing command line arguments. No one is maintaining old versions.
|
||||
|
||||
For details on older versions, see the `final version of its release notes <https://github.com/fritz-hh/OCRmyPDF/blob/7fd3dbdf42ca53a619412ce8add7532c5e81a9d1/RELEASE_NOTES.md>`_.
|
||||
For details on older versions, see the `final version of its release notes <https://github.com/fritz-hh/OCRmyPDF/blob/7fd3dbdf42ca53a619412ce8add7532c5e81a9d1/RELEASE_NOTES.md>`_.
|
||||
|
||||
@@ -0,0 +1,127 @@
|
||||
Advanced features
|
||||
=================
|
||||
|
||||
Control of OCR options
|
||||
----------------------
|
||||
|
||||
OCRmyPDF provides many features to control the behavior of the OCR engine, Tesseract.
|
||||
|
||||
When OCR is skipped
|
||||
"""""""""""""""""""
|
||||
|
||||
If a page in a PDF seems to have text, by default OCRmyPDF will exit without modifying the PDF. This is to ensure that PDFs that were previously OCRed or were "born digital" rather than scanned are not processed.
|
||||
|
||||
If ``--skip-text`` is issued, then no OCR will be performed on pages that already have text. The page will be copied to the output. This may be useful for documents that contain both "born digital" and scanned content, or to use OCRmyPDF to normalize and convert to PDF/A regardless of their contents.
|
||||
|
||||
If ``--force-ocr`` is issued, then all pages will be rasterized to images, discarding any hidden OCR text, and rasterizing any printable text. This is useful for redoing OCR, for fixing OCR text with a damaged character map (text is selectable but not searchable), and destroying redacted information.
|
||||
|
||||
|
||||
Time and image size limits
|
||||
""""""""""""""""""""""""""
|
||||
|
||||
By default, OCRmyPDF permits tesseract to run for only three minutes (180 seconds) per page. This is usually more than enough time to find all text on a reasonably sized page with modern hardware.
|
||||
|
||||
If a page is skipped, it will be inserted without OCR. If preprocessing was requested, the preprocessed image layer will be inserted.
|
||||
|
||||
If you want to adjust the amount of time spent on OCR, change ``--tesseract-timeout``. You can also automatically skip images that exceed a certain number of megapixels with ``--skip-big``. (A 300 DPI, 8.5×11" page is 8.4 megapixels.)
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# Allow 300 seconds for OCR; skip any page larger than 50 megapixels
|
||||
ocrmypdf --tesseract-timeout 300 --skip-big 50 bigfile.pdf output.pdf
|
||||
|
||||
Overriding default tesseract
|
||||
""""""""""""""""""""""""""""
|
||||
|
||||
OCRmyPDF checks the environment variable ``OCRMYPDF_TESSERACT`` for the full path *to the tesseract executable* first.
|
||||
|
||||
For example, if you are testing tesseract 4.00 and don't wish to disturb your tesseract 3.04 installation, you can launch OCRmyPDF as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
env \
|
||||
OCRMYPDF_TESSERACT=/home/user/src/tesseract4/api/tesseract \
|
||||
TESSDATA_PREFIX=/home/user/src/tesseract4 \
|
||||
ocrmypdf --pdf-renderer tess4 --tesseract-oem 2 input.pdf output.pdf
|
||||
|
||||
* ``TESSDATA_PREFIX`` directs tesseract 4.0 to use LSTM training data. This is a tesseract environment variable.
|
||||
* ``--pdf-renderer tess4`` takes advantage of new tesseract 4.0 PDF renderer in OCRmyPDF. (Tesseract 4.0 only.)
|
||||
* ``--tesseract-oem 1`` requests tesseract 4.0's new LSTM engine. (Tesseract 4.0 only.)
|
||||
|
||||
Overriding other support programs
|
||||
"""""""""""""""""""""""""""""""""
|
||||
|
||||
In addition to tesseract, OCRmyPDF uses the following external binaries:
|
||||
|
||||
* ``gs`` (Ghostscript)
|
||||
* ``unpaper``
|
||||
* ``qpdf``
|
||||
|
||||
In each case OCRmyPDF will check the environment variable ``OCRMYPDF_{program}`` before asking the system to find ``{program}`` on the PATH. For example, you could redirect OCRmyPDF to ``OCRMYPDF_GS`` to override Ghostscript.
|
||||
|
||||
Changing tesseract configuration variables
|
||||
""""""""""""""""""""""""""""""""""""""""""
|
||||
|
||||
You can override tesseract's default `control parameters <https://github.com/tesseract-ocr/tesseract/wiki/ControlParams>`_ with a configuration file.
|
||||
|
||||
As an example, this configuration will disable Tesseract's dictionary for current language. Normally the dictionary is helpful for interpolating words that are unclear, but it may interfere with OCR if the document does not contain many words (for example, a list of part numbers).
|
||||
|
||||
Create a file named "no-dict.cfg" with these contents:
|
||||
|
||||
::
|
||||
|
||||
load_system_dawg 0
|
||||
language_model_penalty_non_dict_word 0
|
||||
language_model_penalty_non_freq_dict_word 0
|
||||
|
||||
then run ocrmypdf as follows (along with any other desired arguments):
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf --tesseract-config no-dict.cfg input.pdf output.pdf
|
||||
|
||||
.. warning::
|
||||
|
||||
Some combinations of control parameters will break Tesseract or break assumptions that OCRmyPDF makes about Tesseract's output.
|
||||
|
||||
|
||||
Changing the PDF renderer
|
||||
-------------------------
|
||||
|
||||
rasterizing
|
||||
Converting a PDF to an image for display.
|
||||
|
||||
rendering
|
||||
Creating a new PDF from other data (such as an existing PDF).
|
||||
|
||||
|
||||
OCRmyPDF has three PDF renderers: ``hocr``, ``tesseract`` and ``tess4``. The renderer may be selected using ``--pdf-renderer``. The default is ``auto`` which lets OCRmyPDF select the renderer to use. Currently, ``auto`` always selects ``hocr``.
|
||||
|
||||
The ``hocr`` renderer
|
||||
"""""""""""""""""""""
|
||||
|
||||
The ``hocr`` renderer is the default because it works in most cases. In this mode the whole PDF is rasterized, the raster image is run through OCR to generate a .hocr file, which is an HTML-like file that specifies the location of all identified words.
|
||||
|
||||
The .hocr file is then rendered as a PDF and merged with the image layer.
|
||||
|
||||
The image layer is copied from the original PDF page if possible, avoiding potentially lossy transcoding or loss of other PDF information. If preprocessing is specified, then the image layer is a new PDF.
|
||||
|
||||
This is the only option for tesseract 3.02 and older.
|
||||
|
||||
|
||||
The ``tesseract`` renderer
|
||||
""""""""""""""""""""""""""
|
||||
|
||||
The tesseract renderer uses tesseract's capability to produce a PDF directly. In version 3, tesseract automatically combined the image layer and text, meaning that this mode *always* transcodes and loses potentially loses image quality and other PDF information.
|
||||
|
||||
It does a much better job on non-Latin text.
|
||||
|
||||
In a future release this will become the "tess3" renderer and ultimately will be dropped.
|
||||
|
||||
|
||||
The ``tess4`` renderer
|
||||
""""""""""""""""""""""
|
||||
|
||||
The tess4 renderer uses tesseract 4.00 alpha's text-only PDF feature added in January 2017. This combines the advantages of the tesseract and hocr renderers, transcoding the image layer only if required by preprocessing options.
|
||||
|
||||
Ghostscript PDF/A still sometimes inserts spaces between words when the tess4 renderer is used, affecting search quality. ``--output-pdf pdf`` may be used to avoid this issue.
|
||||
+15
-13
@@ -59,7 +59,7 @@ By default OCRmyPDF assumes the document is English.
|
||||
ocrmypdf -l fre LeParisien.pdf LeParisien.pdf
|
||||
ocrmypdf -l eng+fre Bilingual-English-French.pdf Bilingual-English-French.pdf
|
||||
|
||||
Language packs must be installed for all languages specified. See :ref:`Installing additional language packs <languages>`.
|
||||
Language packs must be installed for all languages specified. See :ref:`Installing additional language packs <lang-packs>`.
|
||||
|
||||
|
||||
Produce PDF and text file containing OCR text
|
||||
@@ -68,10 +68,16 @@ Produce PDF and text file containing OCR text
|
||||
This produces a file named "output.pdf" and a companion text file named "output.txt". The ``pdftotext`` program from
|
||||
`Poppler <https://poppler.freedesktop.org/>`_ is used to extract text from the finished PDF.
|
||||
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ocrmypdf input.pdf - | tee output.pdf | pdftotext - output.txt
|
||||
|
||||
.. note::
|
||||
|
||||
To get pdftotext, Debian/Ubuntu users may ``apt-get install poppler-utils``
|
||||
and macOS users may ``brew install poppler`` respectively.
|
||||
|
||||
|
||||
OCR images, not PDFs
|
||||
--------------------
|
||||
@@ -111,6 +117,14 @@ OCRmyPDF perform some image processing on each page of a PDF, if desired. The s
|
||||
|
||||
* ``--clean-final`` uses unpaper to clean up pages before OCR and inserts the page into the final output. You will want to review each page to ensure that unpaper did not remove something important.
|
||||
|
||||
.. note::
|
||||
|
||||
In many cases image processing will rasterize PDF pages as images, potentially losing quality.
|
||||
|
||||
.. warning::
|
||||
|
||||
``--clean-final`` and ``-remove-background`` may leave undesirable visual artifacts in some images where their algorithms have shortcomings. Files should be visually reviewed after using these options.
|
||||
|
||||
|
||||
OCR and correct document skew (crooked scan)
|
||||
""""""""""""""""""""""""""""""""""""""""""""
|
||||
@@ -127,16 +141,4 @@ Image processing commands can be combined. The order in which options are given
|
||||
|
||||
ocrmypdf --deskew --clean --rotate-pages input.pdf output.pdf
|
||||
|
||||
Control of OCR options
|
||||
----------------------
|
||||
|
||||
By default, OCRmyPDF permits tesseract to run for only three minutes (180 seconds) per page. This is usually more than enough time to find all text on a reasonably sized page with modern hardware. A skipped page will be inserted into the output without any OCR text.
|
||||
|
||||
If you want to adjust the amount of time spent on OCR, change ``--tesseract-timeout``. You can also automatically skip images that exceed a certain number of megapixels.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# Allow 300 seconds for OCR; skip any page larger than 50 megapixels
|
||||
ocrmypdf --tesseract-timeout 300 --skip-big 50 bigfile.pdf output.pdf
|
||||
|
||||
|
||||
|
||||
+1
-1
@@ -20,8 +20,8 @@ Contents:
|
||||
installation
|
||||
languages
|
||||
cookbook
|
||||
advanced
|
||||
batch
|
||||
renderers
|
||||
security
|
||||
errors
|
||||
|
||||
|
||||
@@ -1,40 +0,0 @@
|
||||
PDF Renderers
|
||||
=============
|
||||
|
||||
rasterizing
|
||||
Converting a PDF to an image for display.
|
||||
|
||||
rendering
|
||||
Creating a new PDF from other data (such as an existing PDF).
|
||||
|
||||
|
||||
OCRmyPDF has three PDF renderers: ``hocr``, ``tesseract`` and ``tess4``. The renderer may be selected using ``--pdf-renderer``. The default is ``auto`` which lets OCRmyPDF select the renderer to use. Currently it always uses ``hocr``.
|
||||
|
||||
The hocr renderer
|
||||
-----------------
|
||||
|
||||
The ``hocr`` renderer is the default because it works in most cases. In this mode the whole PDF is rasterized, the raster image is run through OCR to generate a .hocr file, which is an HTML-like file that specifies the location of all identified words.
|
||||
|
||||
The .hocr file is then rendered as a PDF and merged with the image layer.
|
||||
|
||||
The image layer is copied from the original PDF page if possible, avoiding potentially lossy transcoding or loss of other PDF information. If preprocessing is specified, then the image layer is a new PDF.
|
||||
|
||||
This is the only option for tesseract 3.02 and older.
|
||||
|
||||
|
||||
The tesseract renderer
|
||||
----------------------
|
||||
|
||||
The tesseract renderer uses tesseract's capability to produce a PDF directly. In version 3, tesseract automatically combined the image layer and text, meaning that this mode always transcodes and loses potentially loses quality and other PDF information.
|
||||
|
||||
It does a much better job on non-Latin text.
|
||||
|
||||
In a future release this will become the "tess3" renderer and ultimately will be dropped.
|
||||
|
||||
|
||||
The tess4 renderer
|
||||
------------------
|
||||
|
||||
The tess4 renderer uses tesseract 4.00 alpha's text-only PDF feature added in January 2017. This combines the advantages of the tesseract and hocr renderers, transcoding the image layer only if required by preprocessing options.
|
||||
|
||||
Ghostscript PDF/A still sometimes inserts spaces between words when the tess4 renderer is used, affecting search quality. ``--output-pdf pdf`` may be used to avoid this issue.
|
||||
@@ -207,7 +207,7 @@ advanced = parser.add_argument_group(
|
||||
"Advanced options for power users")
|
||||
advanced.add_argument(
|
||||
'--tesseract-config', action='append', metavar='CFG', default=[],
|
||||
help="additional Tesseract configuration files")
|
||||
help="additional Tesseract configuration files -- see documentation")
|
||||
advanced.add_argument(
|
||||
'--tesseract-pagesegmode', action='store', type=int, metavar='PSM',
|
||||
choices=range(0, 14),
|
||||
|
||||
@@ -14,6 +14,7 @@ class ExitCode(IntEnum):
|
||||
already_done_ocr = 6
|
||||
child_process_error = 7
|
||||
encrypted_pdf = 8
|
||||
invalid_config = 9
|
||||
other_error = 15
|
||||
ctrl_c = 130
|
||||
|
||||
@@ -52,3 +53,7 @@ class SubprocessOutputError(ExitCodeException):
|
||||
|
||||
class EncryptedPdfError(ExitCodeException):
|
||||
exit_code = ExitCode.encrypted_pdf
|
||||
|
||||
|
||||
class TesseractConfigError(ExitCodeException):
|
||||
exit_code = ExitCode.invalid_config
|
||||
|
||||
@@ -6,7 +6,7 @@ import os
|
||||
import re
|
||||
import shutil
|
||||
from functools import lru_cache
|
||||
from ..exceptions import MissingDependencyError
|
||||
from ..exceptions import MissingDependencyError, TesseractConfigError
|
||||
from ..helpers import page_number
|
||||
from . import get_program
|
||||
from collections import namedtuple
|
||||
@@ -186,6 +186,8 @@ def tesseract_log_output(log, stdout, input_file):
|
||||
log.warning(prefix + "unsure about page orientation")
|
||||
elif 'error' in line.lower() or 'exception' in line.lower():
|
||||
log.error(prefix + line.strip())
|
||||
elif 'read_params_file' in line.lower():
|
||||
log.error(prefix + line.strip())
|
||||
else:
|
||||
log.info(prefix + line.strip())
|
||||
|
||||
@@ -236,6 +238,8 @@ def generate_hocr(input_file, output_hocr, language: list, engine_mode,
|
||||
_generate_null_hocr(output_hocr, input_file)
|
||||
except CalledProcessError as e:
|
||||
tesseract_log_output(log, e.output, input_file)
|
||||
if 'read_params_file: parameter not found' in e.output:
|
||||
raise TesseractConfigError() from e
|
||||
if 'Image too large' in e.output:
|
||||
_generate_null_hocr(output_hocr, input_file)
|
||||
return
|
||||
@@ -243,6 +247,7 @@ def generate_hocr(input_file, output_hocr, language: list, engine_mode,
|
||||
raise e from e
|
||||
else:
|
||||
tesseract_log_output(log, stdout, input_file)
|
||||
|
||||
if os.path.exists(badxml + '.html'):
|
||||
# Tesseract 3.02 appends suffix ".html" on its own (.badxml.html)
|
||||
shutil.move(badxml + '.html', badxml)
|
||||
@@ -305,6 +310,9 @@ def generate_pdf(input_image, skip_pdf, output_pdf, language: list,
|
||||
shutil.copy(skip_pdf, output_pdf)
|
||||
except CalledProcessError as e:
|
||||
tesseract_log_output(log, e.output, input_image)
|
||||
if 'read_params_file: parameter not found' in e.output:
|
||||
raise TesseractConfigError() from e
|
||||
|
||||
if 'Image too large' in e.output:
|
||||
shutil.copy(skip_pdf, output_pdf)
|
||||
return
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
from cffi import FFI
|
||||
|
||||
ffi = FFI()
|
||||
ffi.set_source("ocrmypdf.lib._leptonica", None)
|
||||
ffi.cdef("""
|
||||
ffibuilder = FFI()
|
||||
ffibuilder.set_source("ocrmypdf.lib._leptonica", None)
|
||||
ffibuilder.cdef("""
|
||||
typedef signed char l_int8;
|
||||
typedef unsigned char l_uint8;
|
||||
typedef short l_int16;
|
||||
@@ -56,7 +56,7 @@ typedef struct Box BOX;
|
||||
|
||||
""")
|
||||
|
||||
ffi.cdef("""
|
||||
ffibuilder.cdef("""
|
||||
PIX * pixRead ( const char *filename );
|
||||
PIX * pixScale ( PIX *pixs, l_float32 scalex, l_float32 scaley );
|
||||
l_int32 pixFindSkew ( PIX *pixs, l_float32 *pangle, l_float32 *pconf );
|
||||
@@ -189,4 +189,4 @@ void lept_free(void *ptr);
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
ffi.compile()
|
||||
ffibuilder.compile(verbose=True)
|
||||
|
||||
@@ -948,7 +948,7 @@ def build_pipeline(options, work_folder, log, context):
|
||||
task_ocr_tesseract_hocr.graphviz(fillcolor='"#00cc66"')
|
||||
task_ocr_tesseract_hocr.active_if(options.pdf_renderer == 'hocr')
|
||||
if tesseract.v4():
|
||||
task_ocr_tesseract_hocr.jobs_limit(1) # Uses multi-core on its own
|
||||
task_ocr_tesseract_hocr.jobs_limit(2) # Uses multi-core on its own
|
||||
|
||||
task_select_visible_page_image = main_pipeline.collate(
|
||||
task_func=select_visible_page_image,
|
||||
@@ -1001,7 +1001,7 @@ def build_pipeline(options, work_folder, log, context):
|
||||
task_ocr_tesseract_textonly_pdf.graphviz(fillcolor='"#ff69b4"')
|
||||
task_ocr_tesseract_textonly_pdf.active_if(options.pdf_renderer == 'tess4')
|
||||
if tesseract.v4():
|
||||
task_ocr_tesseract_textonly_pdf.jobs_limit(1)
|
||||
task_ocr_tesseract_textonly_pdf.jobs_limit(2)
|
||||
|
||||
task_combine_layers = main_pipeline.collate(
|
||||
task_func=combine_layers,
|
||||
@@ -1024,7 +1024,7 @@ def build_pipeline(options, work_folder, log, context):
|
||||
task_ocr_tesseract_and_render_pdf.graphviz(fillcolor='"#66ccff"')
|
||||
task_ocr_tesseract_and_render_pdf.active_if(options.pdf_renderer == 'tesseract')
|
||||
if tesseract.v4():
|
||||
task_ocr_tesseract_and_render_pdf.jobs_limit(1) # Uses multi-core
|
||||
task_ocr_tesseract_and_render_pdf.jobs_limit(2) # Uses multi-core
|
||||
|
||||
# PDF/A
|
||||
task_generate_postscript_stub = main_pipeline.transform(
|
||||
|
||||
+2
-2
@@ -2,8 +2,8 @@
|
||||
# setup.py lists a separate set of requirements that are looser to simplify
|
||||
# installation
|
||||
ruffus == 2.6.3
|
||||
Pillow == 3.4.2
|
||||
Pillow == 4.0.0
|
||||
reportlab == 3.3.0
|
||||
PyPDF2 == 1.26
|
||||
img2pdf == 0.2.1
|
||||
img2pdf == 0.2.3
|
||||
cffi == 1.9.1
|
||||
|
||||
@@ -9,7 +9,7 @@ if sys.version_info < (3, 4):
|
||||
print("Python 3.4 or newer is required", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
from setuptools import setup # nopep8
|
||||
from setuptools import setup, find_packages # nopep8
|
||||
from subprocess import STDOUT, check_output, CalledProcessError # nopep8
|
||||
from collections.abc import Mapping # nopep8
|
||||
import re # nopep8
|
||||
@@ -192,7 +192,7 @@ setup(
|
||||
author='James R. Barlow',
|
||||
author_email='jim@purplerock.ca',
|
||||
license='MIT',
|
||||
packages=['ocrmypdf'],
|
||||
packages=find_packages(exclude=["tests", "tests.*"]),
|
||||
keywords=['PDF', 'OCR', 'optical character recognition', 'PDF/A', 'scanning'],
|
||||
classifiers=[
|
||||
"Programming Language :: Python :: 3",
|
||||
@@ -214,22 +214,20 @@ setup(
|
||||
"Topic :: Text Processing :: Linguistic",
|
||||
],
|
||||
setup_requires=[
|
||||
'setuptools_scm',
|
||||
'cffi>=1.5.0',
|
||||
'pytest-runner',
|
||||
'pytest-helpers-namespace'
|
||||
'setuptools_scm', # so that version will work
|
||||
'cffi>=1.9.1' # to build the leptonica module
|
||||
],
|
||||
use_scm_version={'version_scheme': 'post-release'},
|
||||
cffi_modules=[
|
||||
'ocrmypdf/lib/compile_leptonica.py:ffi'
|
||||
'ocrmypdf/lib/compile_leptonica.py:ffibuilder'
|
||||
],
|
||||
install_requires=[
|
||||
'ruffus==2.6.3', # pinned - ocrmypdf implements a 2.6.3 workaround
|
||||
'Pillow>=3.1.1', # Pillow is pretty stable
|
||||
'Pillow>=4.0.0', # Pillow < 4 has BytesIO/TIFF bug w/img2pdf 0.2.3
|
||||
'reportlab>=3.2.0', # oldest released version with sane image handling
|
||||
'PyPDF2>=1.26', # pure Python, so track HEAD closely
|
||||
'img2pdf>=0.2.1', # pure Python, so track HEAD closely
|
||||
'cffi>=1.5.0' # oldest version ever tested
|
||||
'img2pdf>=0.2.3', # pure Python, so track HEAD closely
|
||||
'cffi>=1.9.1' # must be a setup and install requirement
|
||||
],
|
||||
tests_require=tests_require,
|
||||
entry_points={
|
||||
|
||||
@@ -1,2 +1,2 @@
|
||||
pytest >= 2.8
|
||||
pytest >= 3.0
|
||||
pytest-helpers-namespace
|
||||
@@ -37,6 +37,7 @@ def real_tesseract():
|
||||
def main():
|
||||
operation = sys.argv[-1]
|
||||
# For anything unexpected operation, defer to real tesseract binary
|
||||
# Currently this includes all use of "--tesseract-config"
|
||||
if operation != 'hocr' and operation != 'pdf' and operation != 'stdout':
|
||||
real_tesseract()
|
||||
return # Not reachable
|
||||
@@ -52,6 +53,11 @@ def main():
|
||||
['tesseract', '--version'],
|
||||
stderr=subprocess.STDOUT)
|
||||
|
||||
if b'4.00.00alpha' in tess_version:
|
||||
# Tesseract 4.x alpha is a moving target, don't cache it
|
||||
real_tesseract()
|
||||
return
|
||||
|
||||
m.update(tess_version)
|
||||
|
||||
# Insert this source file into the hash function, to ensure that any
|
||||
@@ -65,12 +71,21 @@ def main():
|
||||
lang = sys.argv[sys.argv.index('-l') + 1]
|
||||
m.update(lang.encode())
|
||||
except ValueError:
|
||||
pass
|
||||
try:
|
||||
psm = sys.argv[sys.argv.index('-psm') + 1]
|
||||
m.update(psm.encode())
|
||||
except ValueError:
|
||||
pass
|
||||
m.update(b'default-lang')
|
||||
|
||||
psm_arg = ''
|
||||
if '--psm' in sys.argv:
|
||||
psm_arg = '--psm'
|
||||
elif '-psm' in sys.argv:
|
||||
psm_arg = '-psm'
|
||||
if psm_arg:
|
||||
try:
|
||||
psm = sys.argv[sys.argv.index(psm_arg) + 1]
|
||||
m.update(psm.encode())
|
||||
except ValueError:
|
||||
m.update(b'default-psm')
|
||||
else:
|
||||
m.update(b'default-psm')
|
||||
|
||||
if operation == 'stdout' and psm != '0':
|
||||
real_tesseract()
|
||||
|
||||
@@ -690,3 +690,51 @@ def test_destination_not_writable(spoof_tesseract_noop, resources, outdir):
|
||||
resources / 'jbig2.pdf', protected_file,
|
||||
env=spoof_tesseract_noop)
|
||||
assert p.returncode == ExitCode.file_access_error, "Expected error"
|
||||
|
||||
|
||||
def test_tesseract_config_valid(resources, outdir):
|
||||
cfg_file = outdir / 'test.cfg'
|
||||
with cfg_file.open('w') as f:
|
||||
f.write('''\
|
||||
load_system_dawg 0
|
||||
language_model_penalty_non_dict_word 0
|
||||
language_model_penalty_non_freq_dict_word 0
|
||||
''')
|
||||
|
||||
check_ocrmypdf(
|
||||
resources / 'ccitt.pdf', outdir / 'out.pdf',
|
||||
'--tesseract-config', str(cfg_file))
|
||||
|
||||
|
||||
@pytest.mark.parametrize('renderer', [
|
||||
'hocr',
|
||||
'tesseract',
|
||||
])
|
||||
def test_tesseract_config_notfound(renderer, resources, outdir):
|
||||
cfg_file = outdir / 'nofile.cfg'
|
||||
|
||||
p, out, err = run_ocrmypdf(
|
||||
resources / 'ccitt.pdf', outdir / 'out.pdf',
|
||||
'--pdf-renderer', renderer,
|
||||
'--tesseract-config', str(cfg_file))
|
||||
assert "Can't open" in err, "No error message about missing config file"
|
||||
assert p.returncode == ExitCode.ok
|
||||
|
||||
|
||||
@pytest.mark.parametrize('renderer', [
|
||||
'hocr',
|
||||
'tesseract',
|
||||
])
|
||||
def test_tesseract_config_invalid(renderer, resources, outdir):
|
||||
cfg_file = outdir / 'test.cfg'
|
||||
with cfg_file.open('w') as f:
|
||||
f.write('''\
|
||||
THIS FILE IS INVALID
|
||||
''')
|
||||
|
||||
p, out, err = run_ocrmypdf(
|
||||
resources / 'ccitt.pdf', outdir / 'out.pdf',
|
||||
'--pdf-renderer', renderer,
|
||||
'--tesseract-config', str(cfg_file))
|
||||
assert "parameter not found" in err, "No error message"
|
||||
assert p.returncode == ExitCode.invalid_config
|
||||
|
||||
Reference in New Issue
Block a user