Compare commits

...
8 Commits
Author SHA1 Message Date
James R. Barlow 3a7c3417bb Don’t check tags and branch at the same time as Travis doesn’t get this
Travis is weird
2017-06-13 13:14:34 -07:00
James R. Barlow d792ef7222 Give the ‘auto’ renderer setting more test covfefe 2017-06-13 13:13:58 -07:00
James R. Barlow 2c24f67deb Rename “tess4” renderer to “sandwich” and make it default in Tess 3.05.01
Tesseract 3.05.01 backported the textonly_pdf=1 which allows the use
of this superior PDF renderer prior to 4.00 alpha. This means that
the tess4 name is no longer accurate, so call it a sandwich because of
its merge-preserve characteristic. Preserve the tess4 name. Fix the
documentation and tests to reflect this.

Make it the default, because it’s better. It does not have the issues
the “tesseract” renderer does prior to Tess 3.05.00 with rendering
PDFs that Ghostscript corrupts, and it produces better output without
re-rastering.

Deprecate some old stuff to avoid the test suite growing obscenely
large.
2017-06-13 13:09:12 -07:00
James R. Barlow 9e75e28d0c Homebrew needs x11 to compile Pillow 2017-06-13 11:03:26 -07:00
James R. Barlow 3232643809 Support “textonly PDF” renderer in Tesseract 3.05.01 2017-06-13 10:18:08 -07:00
James R. Barlow f7ee9e90ce Document what is meant by the ocrmypdf “API” 2017-06-13 10:15:11 -07:00
James R. Barlow 47298be132 Remove Python <3.5 test 2017-06-13 10:14:28 -07:00
James R. Barlow a88fa83515 Travis: fix deploy conditions for homebrew autobrew 2017-05-31 02:29:32 -07:00
14 changed files with 179 additions and 123 deletions
+1 -2
View File
@@ -84,11 +84,10 @@ deploy:
script: /usr/bin/true script: /usr/bin/true
on: on:
branch: master branch: master
tags: false
condition: $TRAVIS_OS_NAME == "osx" condition: $TRAVIS_OS_NAME == "osx"
after_deploy: | after_deploy: |
if [[ "$TRAVIS_OS_NAME" == "osx" && "$TRAVIS_BRANCH" == "master" ]]; then if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
bash .travis/osx_brew.sh bash .travis/osx_brew.sh
elif [[ "$TRAVIS_PYTHON_VERSION" == "3.6" && "$TRAVIS_OS_NAME" == "linux" ]]; then elif [[ "$TRAVIS_PYTHON_VERSION" == "3.6" && "$TRAVIS_OS_NAME" == "linux" ]]; then
curl -H "Content-Type: application/json" --data '{"build": true}' -X POST https://registry.hub.docker.com/u/jbarlow83/ocrmypdf/trigger/$DOCKERHUB_OCRMYPDF_TOKEN/ curl -H "Content-Type: application/json" --data '{"build": true}' -X POST https://registry.hub.docker.com/u/jbarlow83/ocrmypdf/trigger/$DOCKERHUB_OCRMYPDF_TOKEN/
+1
View File
@@ -15,6 +15,7 @@ class Ocrmypdf < Formula
${ocrmypdf_sha256} ${ocrmypdf_sha256}
depends_on :python3 depends_on :python3
depends_on :x11 # Pillow needs XQuartz
depends_on "pkg-config" => :build depends_on "pkg-config" => :build
depends_on "libffi" depends_on "libffi"
depends_on "tesseract" depends_on "tesseract"
+16 -21
View File
@@ -42,10 +42,9 @@ For example, if you are testing tesseract 4.00 and don't wish to disturb your te
env \ env \
OCRMYPDF_TESSERACT=/home/user/src/tesseract4/api/tesseract \ OCRMYPDF_TESSERACT=/home/user/src/tesseract4/api/tesseract \
TESSDATA_PREFIX=/home/user/src/tesseract4 \ TESSDATA_PREFIX=/home/user/src/tesseract4 \
ocrmypdf --pdf-renderer tess4 --tesseract-oem 2 input.pdf output.pdf ocrmypdf --tesseract-oem 2 input.pdf output.pdf
* ``TESSDATA_PREFIX`` directs tesseract 4.0 to use LSTM training data. This is a tesseract environment variable. * ``TESSDATA_PREFIX`` directs tesseract 4.0 to use LSTM training data. This is a tesseract environment variable.
* ``--pdf-renderer tess4`` takes advantage of new tesseract 4.0 PDF renderer in OCRmyPDF. (Tesseract 4.0 only.)
* ``--tesseract-oem 1`` requests tesseract 4.0's new LSTM engine. (Tesseract 4.0 only.) * ``--tesseract-oem 1`` requests tesseract 4.0's new LSTM engine. (Tesseract 4.0 only.)
Overriding other support programs Overriding other support programs
@@ -95,33 +94,29 @@ rendering
Creating a new PDF from other data (such as an existing PDF). Creating a new PDF from other data (such as an existing PDF).
OCRmyPDF has three PDF renderers: ``hocr``, ``tesseract`` and ``tess4``. The renderer may be selected using ``--pdf-renderer``. The default is ``auto`` which lets OCRmyPDF select the renderer to use. Currently, ``auto`` always selects ``hocr``. OCRmyPDF has three PDF renderers: ``sandwich``, ``hocr``, ``tesseract``. The renderer may be selected using ``--pdf-renderer``. The default is ``auto`` which lets OCRmyPDF select the renderer to use. Currently, ``auto`` selects ``sandwich`` for Tesseract 3.05.01, and newer, ``hocr`` for older versions of Tesseract.
The ``sandwich`` renderer
"""""""""""""""""""""""""
The ``sandwich`` renderer uses Tesseract's new text-only PDF feature, which produces a PDF page that lays out the OCR in invisible text. This page is then "sandwiched" onto the original PDF page, allowing lossless application of OCR even to PDF pages that contain other vector objects.
When image preprocessing features like ``--deskew`` are used, the original PDF will be rendered as a full page and the OCR layer will be placed on top.
This renderer requires Tesseract 3.05.01 or newer.
The ``hocr`` renderer The ``hocr`` renderer
""""""""""""""""""""" """""""""""""""""""""
The ``hocr`` renderer is the default because it works in most cases. In this mode the whole PDF is rasterized, the raster image is run through OCR to generate a .hocr file, which is an HTML-like file that specifies the location of all identified words. The ``hocr`` renderer works with older versions of Tesseract. The image layer is copied from the original PDF page if possible, avoiding potentially lossy transcoding or loss of other PDF information. If preprocessing is specified, then the image layer is a new PDF.
The .hocr file is then rendered as a PDF and merged with the image layer.
The image layer is copied from the original PDF page if possible, avoiding potentially lossy transcoding or loss of other PDF information. If preprocessing is specified, then the image layer is a new PDF.
This is the only option for tesseract 3.02 and older.
This works in all versions of Tesseract.
The ``tesseract`` renderer The ``tesseract`` renderer
"""""""""""""""""""""""""" """"""""""""""""""""""""""
The tesseract renderer uses tesseract's capability to produce a PDF directly. In version 3, tesseract automatically combined the image layer and text, meaning that this mode *always* transcodes and loses potentially loses image quality and other PDF information. The ``tesseract`` renderer creates a PDF with the image and text layers precomposed, meaning that it always transcodes, loses image quality and rasterizes and vector objects. It does a better job on non-Latin text and document structure than ``hocr``.
It does a much better job on non-Latin text. If a PDF created with this renderer using Tesseract versions older than 3.05.00 is then passed through Ghostscript's pdfwrite feature, the OCR text *may* be corrupted. The ``--output-type=pdfa`` argument will produce a warning in this situation.
In a future release this will become the "tess3" renderer and ultimately will be dropped. *This renderer is deprecated and will be removed whenever support for older versions of Tesseract is dropped.*
The ``tess4`` renderer
""""""""""""""""""""""
The tess4 renderer uses tesseract 4.00 alpha's text-only PDF feature added in January 2017. This combines the advantages of the tesseract and hocr renderers, transcoding the image layer only if required by preprocessing options.
Ghostscript PDF/A still sometimes inserts spaces between words when the tess4 renderer is used, affecting search quality. ``--output-pdf pdf`` may be used to avoid this issue.
+13 -2
View File
@@ -1,7 +1,18 @@
Release notes Release notes
============= =============
OCRmyPDF uses `semantic versioning <http://semver.org/>`_. OCRmyPDF uses `semantic versioning <http://semver.org/>`_ for its command line interface.
The OCRmyPDF package itself does not contain a public API, although it is fairly stable and breaking changes are usually timed with a major release. A future release will clearly define the stable public API.
v5.2
----
- When using Tesseract 3.05.01 or newer, OCRmyPDF will select the "sandwich" PDF renderer by default, unless another PDF renderer is specified with the ``--pdf-renderer`` argument. The previous behavior was to select ``--pdf-renderer=hocr``.
- The "tesseract" PDF renderer is now deprecated, since it can cause problems with Ghostscript on Tesseract 3.05.00
- The "tess4" PDF renderer has been renamed to "sandwich". "tess4" is now a deprecated alias for "sandwich".
v5.1 v5.1
---- ----
@@ -12,7 +23,7 @@ v5.1
v5.0.1 v5.0.1
------ ------
- Fixed issue #169, exception due to failure to create sidecar text files on some versions of Tesseract 3.04, including the jbarlow83/ocrmypdf image - Fixed issue #169, exception due to failure to create sidecar text files on some versions of Tesseract 3.04, including the jbarlow83/ocrmypdf Docker image
v5.0 v5.0
+31 -25
View File
@@ -250,15 +250,18 @@ advanced.add_argument(
"3 - default.") "3 - default.")
) )
advanced.add_argument( advanced.add_argument(
'--pdf-renderer', choices=['auto', 'tesseract', 'hocr', 'tess4'], default='auto', '--pdf-renderer',
choices=['auto', 'tesseract', 'hocr', 'tess4', 'sandwich'], default='auto',
help="Choose OCR PDF renderer - the default option is to let OCRmyPDF " help="Choose OCR PDF renderer - the default option is to let OCRmyPDF "
"choose. The 'tesseract' PDF renderer is more accurate and does a " "choose."
"better job and document structure such as recognizing columns. It " "auto - let OCRmyPDF choose; "
"also does a better job on non-Latin languages. However, it does " "sandwich - default renderer for Tesseract 3.05.01 and newer; "
"not work as well when older versions of Tesseract or Ghostscript " "hocr - default renderer for older versions of Tesseract; "
"are installed, and some combinations of arguments to do not work " "tesseract - gives better results for non-Latin languages and "
"with --pdf-renderer tesseract. The 'tess4' PDF renderer is similar " "Tesseract older than 3.05.01 but has problems with some versions "
"to 'tesseract', requires tesseract 4, and gives superior results.") " of Ghostscript; deprecated"
"tess4 - deprecated alias for 'sandwich'"
)
advanced.add_argument( advanced.add_argument(
'--tesseract-timeout', default=180.0, type=float, metavar='SECONDS', '--tesseract-timeout', default=180.0, type=float, metavar='SECONDS',
help='Give up on OCR after the timeout, but copy the preprocessed page ' help='Give up on OCR after the timeout, but copy the preprocessed page '
@@ -312,28 +315,31 @@ def check_options_languages(options, _log):
def check_options_output(options, log): def check_options_output(options, log):
if options.pdf_renderer == 'auto': if options.pdf_renderer == 'auto':
options.pdf_renderer = 'hocr' if tesseract.has_textonly_pdf():
options.pdf_renderer = 'sandwich'
else:
options.pdf_renderer = 'hocr'
if options.pdf_renderer in ('tesseract', 'tess4'): if options.pdf_renderer == 'tess4':
if tesseract.version() < '3.05': log.warning("The 'tess4' PDF renderer has been renamed to 'sandwich'. "
log.warning( "Please use --pdf-renderer=sandwich.")
"The setting --pdf-renderer=tesseract is not recommend for " options.pdf_renderer = 'sandwich'
" use with tesseract versions less than 3.05, because it "
" produces OCR text that is incompatible with Ghostscript and "
" some other software.")
elif tesseract.version() == '4.00.00alpha':
log.warning(
"The setting --pdf-renderer={tesseract,tess4} is not"
" recommended for builds of tesseract 4.00.00alpha older than"
" February 2017. Make sure you are using a recent build.")
if options.debug_rendering and options.pdf_renderer == 'tesseract': if options.pdf_renderer == 'tesseract':
log.warning("The 'tesseract' PDF renderer is deprecated.")
if tesseract.version() < '3.05' and options.output_type == 'pdfa':
log.warning(
"For best results use --pdf-renderer=tesseract "
"--output-type=pdf to disable PDF/A generation via "
"Ghostscript, which is known to corrupt the OCR text of "
"some PDFs produced your version of Tesseract.")
if options.debug_rendering and options.pdf_renderer != 'hocr':
log.info( log.info(
"Ignoring --debug-rendering because it is not supported with" "Ignoring --debug-rendering because it requires --pdf-renderer=hocr")
"--pdf-renderer=tesseract.")
lossless_reconstruction = False lossless_reconstruction = False
if options.pdf_renderer in ('hocr', 'tess4'): if options.pdf_renderer in ('hocr', 'sandwich'):
if not any((options.deskew, options.clean_final, options.force_ocr, if not any((options.deskew, options.clean_final, options.force_ocr,
options.remove_background)): options.remove_background)):
lossless_reconstruction = True lossless_reconstruction = True
+21 -19
View File
@@ -63,26 +63,28 @@ def v4():
return (version() >= '4') return (version() >= '4')
@lru_cache(maxsize=1)
def has_textonly_pdf(): def has_textonly_pdf():
if version() == '4.00.00alpha': """Does Tesseract have textonly_pdf capability?
# textonly_pdf added during the 4.00.00alpha cycle, so we must test
# more carefully to see if it is present Available in 3.05.01, and v4.00.00alpha since January 2017. Best to
args_tess = [ parse the parameter list
get_program('tesseract'), """
'--print-parameters' args_tess = [
] get_program('tesseract'),
try: '--print-parameters'
params = check_output( ]
args_tess, close_fds=True, universal_newlines=True, try:
stderr=STDOUT) params = check_output(
except CalledProcessError as e: args_tess, close_fds=True, universal_newlines=True,
print("Could not --print-parameters from tesseract", stderr=STDOUT)
file=sys.stderr) except CalledProcessError as e:
raise MissingDependencyError from e print("Could not --print-parameters from tesseract",
if 'textonly_pdf' in params: file=sys.stderr)
return True raise MissingDependencyError from e
else: if 'textonly_pdf' in params:
return v4() return True
return False
def psm(): def psm():
+13 -6
View File
@@ -595,7 +595,7 @@ def select_image_layer(
# as it accurately describes the image. It would be possible to # as it accurately describes the image. It would be possible to
# resample the image at this stage back to non-square DPI to more # resample the image at this stage back to non-square DPI to more
# closely resemble the input, except that the hocr renderer does not # closely resemble the input, except that the hocr renderer does not
# understand non-square DPI. The tess4 renderer would be fine. # understand non-square DPI. The sandwich renderer would be fine.
dpi = get_page_square_dpi(pageinfo, options) dpi = get_page_square_dpi(pageinfo, options)
layout_fun = img2pdf.get_fixed_dpi_layout_fun((dpi, dpi)) layout_fun = img2pdf.get_fixed_dpi_layout_fun((dpi, dpi))
@@ -812,9 +812,16 @@ def get_pdfmark(base_pdf, options):
if options.subject: if options.subject:
pdfmark['/Subject'] = options.subject pdfmark['/Subject'] = options.subject
pdfmark['/Creator'] = '{0} {1} / Tesseract OCR{2} {3}'.format( if options.pdf_renderer == 'tesseract':
renderer_tag = 'OCR+PDF'
elif options.pdf_renderer == 'sandwich':
renderer_tag = 'OCR-PDF'
else:
renderer_tag = 'OCR'
pdfmark['/Creator'] = '{0} {1} / Tesseract {2} {3}'.format(
PROGRAM_NAME, VERSION, PROGRAM_NAME, VERSION,
'+PDF' if options.pdf_renderer == 'tesseract' else '', renderer_tag,
tesseract.version()) tesseract.version())
return pdfmark return pdfmark
@@ -1083,7 +1090,7 @@ def build_pipeline(options, work_folder, log, context):
task_select_image_layer.graphviz( task_select_image_layer.graphviz(
fillcolor='"#00cc66"', shape='diamond') fillcolor='"#00cc66"', shape='diamond')
task_select_image_layer.active_if( task_select_image_layer.active_if(
options.pdf_renderer == 'hocr' or options.pdf_renderer == 'tess4') options.pdf_renderer == 'hocr' or options.pdf_renderer == 'sandwich')
task_render_hocr_page = main_pipeline.transform( task_render_hocr_page = main_pipeline.transform(
task_func=render_hocr_page, task_func=render_hocr_page,
@@ -1113,7 +1120,7 @@ def build_pipeline(options, work_folder, log, context):
os.path.join(work_folder, r'\1.text.txt')], os.path.join(work_folder, r'\1.text.txt')],
extras=[log, context]) extras=[log, context])
task_ocr_tesseract_textonly_pdf.graphviz(fillcolor='"#ff69b4"') task_ocr_tesseract_textonly_pdf.graphviz(fillcolor='"#ff69b4"')
task_ocr_tesseract_textonly_pdf.active_if(options.pdf_renderer == 'tess4') task_ocr_tesseract_textonly_pdf.active_if(options.pdf_renderer == 'sandwich')
if tesseract.v4(): if tesseract.v4():
task_ocr_tesseract_textonly_pdf.jobs_limit(2) task_ocr_tesseract_textonly_pdf.jobs_limit(2)
@@ -1126,7 +1133,7 @@ def build_pipeline(options, work_folder, log, context):
output=os.path.join(work_folder, r'\1.rendered.pdf'), output=os.path.join(work_folder, r'\1.rendered.pdf'),
extras=[log, context]) extras=[log, context])
task_combine_layers.graphviz(fillcolor='"#00cc66"') task_combine_layers.graphviz(fillcolor='"#00cc66"')
task_combine_layers.active_if(options.pdf_renderer == 'hocr' or options.pdf_renderer == 'tess4') task_combine_layers.active_if(options.pdf_renderer == 'hocr' or options.pdf_renderer == 'sandwich')
# Tesseract OCR+PDF # Tesseract OCR+PDF
task_ocr_tesseract_and_render_pdf = main_pipeline.collate( task_ocr_tesseract_and_render_pdf = main_pipeline.collate(
+3
View File
@@ -22,6 +22,9 @@ def main():
elif sys.argv[1] == '--list-langs': elif sys.argv[1] == '--list-langs':
print('List of available languages (1):\neng', file=sys.stderr) print('List of available languages (1):\neng', file=sys.stderr)
sys.exit(0) sys.exit(0)
elif sys.argv[1] == '--print-parameters':
print('A parameter list would go here', file=sys.stderr)
sys.exit(0)
elif sys.argv[-2] == 'hocr': elif sys.argv[-2] == 'hocr':
print("Image too large: (33830, 14959)\n" print("Image too large: (33830, 14959)\n"
"Error during processing.", file=sys.stderr) "Error during processing.", file=sys.stderr)
+6
View File
@@ -85,6 +85,12 @@ def main():
except ValueError: except ValueError:
m.update(b'default-lang') m.update(b'default-lang')
try:
textonly = sys.argv[sys.argv.index('-c') + 1]
m.update(textonly.encode())
except ValueError:
m.update(b'textonly_pdf=0')
psm_arg = '' psm_arg = ''
if '--psm' in sys.argv: if '--psm' in sys.argv:
psm_arg = '--psm' psm_arg = '--psm'
+3
View File
@@ -29,6 +29,9 @@ def main():
elif sys.argv[1] == '--list-langs': elif sys.argv[1] == '--list-langs':
print('List of available languages (1):\neng', file=sys.stderr) print('List of available languages (1):\neng', file=sys.stderr)
sys.exit(0) sys.exit(0)
elif sys.argv[1] == '--print-parameters':
print('List of parameters would go here', file=sys.stderr)
sys.exit(0)
elif sys.argv[-2] == 'hocr': elif sys.argv[-2] == 'hocr':
print("KABOOM! Tesseract failed for some reason", file=sys.stderr) print("KABOOM! Tesseract failed for some reason", file=sys.stderr)
sys.exit(128 + signal.SIGSEGV) sys.exit(128 + signal.SIGSEGV)
+27 -8
View File
@@ -2,6 +2,7 @@
# © 2016 James R. Barlow: github.com/jbarlow83 # © 2016 James R. Barlow: github.com/jbarlow83
import sys import sys
import img2pdf import img2pdf
import PyPDF2 as pypdf
from PIL import Image from PIL import Image
@@ -17,7 +18,7 @@ In orientation check mode, report the orientation is upright.
""" """
VERSION_STRING = '''tesseract 3.04.00 VERSION_STRING = '''tesseract 3.05.01
leptonica-1.72 leptonica-1.72
libjpeg 8d : libpng 1.6.19 : libtiff 4.0.6 : zlib 1.2.5 libjpeg 8d : libpng 1.6.19 : libtiff 4.0.6 : zlib 1.2.5
SPOOFED SPOOFED
@@ -53,6 +54,10 @@ def main():
elif sys.argv[1] == '--list-langs': elif sys.argv[1] == '--list-langs':
print('List of available languages (1):\neng', file=sys.stderr) print('List of available languages (1):\neng', file=sys.stderr)
sys.exit(0) sys.exit(0)
elif sys.argv[1] == '--print-parameters':
print("Some parameters", file=sys.stderr)
print("textonly_pdf\t1\tSome help text")
sys.exit(0)
elif sys.argv[-2] == 'hocr': elif sys.argv[-2] == 'hocr':
inputf = sys.argv[-4] inputf = sys.argv[-4]
output = sys.argv[-3] output = sys.argv[-3]
@@ -63,13 +68,27 @@ def main():
with open(output + '.txt', 'w') as f: with open(output + '.txt', 'w') as f:
f.write('') f.write('')
elif sys.argv[-2] == 'pdf': elif sys.argv[-2] == 'pdf':
inputf = sys.argv[-4] if 'textonly_pdf=1' in sys.argv:
output = sys.argv[-3] inputf = sys.argv[-4]
pdf_bytes = img2pdf.convert([inputf], dpi=300) output = sys.argv[-3]
with open(output + '.pdf', 'wb') as f: with Image.open(inputf) as im:
f.write(pdf_bytes) dpi = im.info['dpi']
with open(output + '.txt', 'w') as f: imsize = im.size[0] * dpi[0] / 72, im.size[1] * dpi[1] / 72
f.write('')
pdf_out = pypdf.PdfFileWriter()
pdf_out.addBlankPage(imsize[0], imsize[1])
with open(output + '.pdf', 'wb') as f:
pdf_out.write(f)
with open(output + '.txt', 'w') as f:
f.write('')
else:
inputf = sys.argv[-4]
output = sys.argv[-3]
pdf_bytes = img2pdf.convert([inputf], dpi=300)
with open(output + '.pdf', 'wb') as f:
f.write(pdf_bytes)
with open(output + '.txt', 'w') as f:
f.write('')
elif sys.argv[-1] == 'stdout': elif sys.argv[-1] == 'stdout':
inputf = sys.argv[-2] inputf = sys.argv[-2]
print("""Orientation: 0 print("""Orientation: 0
+16 -35
View File
@@ -10,7 +10,7 @@ import PyPDF2 as pypdf
from ocrmypdf.exceptions import ExitCode from ocrmypdf.exceptions import ExitCode
from ocrmypdf import leptonica from ocrmypdf import leptonica
from ocrmypdf.pdfa import file_claims_pdfa from ocrmypdf.pdfa import file_claims_pdfa
from ocrmypdf.exec import ghostscript from ocrmypdf.exec import ghostscript, tesseract
import logging import logging
from math import isclose from math import isclose
@@ -20,6 +20,11 @@ run_ocrmypdf = pytest.helpers.run_ocrmypdf
spoof = pytest.helpers.spoof spoof = pytest.helpers.spoof
RENDERERS = ['hocr', 'tesseract']
if tesseract.has_textonly_pdf():
RENDERERS.append('sandwich')
@pytest.fixture @pytest.fixture
def spoof_tesseract_crash(): def spoof_tesseract_crash():
return spoof(tesseract='tesseract_crash.py') return spoof(tesseract='tesseract_crash.py')
@@ -128,7 +133,7 @@ def test_remove_background(spoof_tesseract_noop, resources, outdir):
@pytest.mark.parametrize( @pytest.mark.parametrize(
"pdf", "pdf",
['palette.pdf', 'cmyk.pdf', 'ccitt.pdf', 'jbig2.pdf', 'lichtenstein.pdf']) ['palette.pdf', 'cmyk.pdf', 'ccitt.pdf', 'jbig2.pdf', 'lichtenstein.pdf'])
@pytest.mark.parametrize("renderer", ['hocr', 'tesseract']) @pytest.mark.parametrize("renderer", ['auto', 'tesseract'])
@pytest.mark.parametrize("output_type", ['pdf', 'pdfa']) @pytest.mark.parametrize("output_type", ['pdf', 'pdfa'])
def test_exotic_image(spoof_tesseract_cache, pdf, renderer, output_type, def test_exotic_image(spoof_tesseract_cache, pdf, renderer, output_type,
resources, outdir): resources, outdir):
@@ -210,10 +215,7 @@ def test_high_unicode(spoof_tesseract_noop, resources, no_outpdf):
assert p.returncode == ExitCode.bad_args, err assert p.returncode == ExitCode.bad_args, err
@pytest.mark.parametrize('renderer', [ @pytest.mark.parametrize('renderer', RENDERERS)
'hocr',
'tesseract',
])
def test_oversample(spoof_tesseract_cache, renderer, resources, outpdf): def test_oversample(spoof_tesseract_cache, renderer, resources, outpdf):
oversampled_pdf = check_ocrmypdf( oversampled_pdf = check_ocrmypdf(
resources / 'skew.pdf', outpdf, '--oversample', '350', resources / 'skew.pdf', outpdf, '--oversample', '350',
@@ -303,10 +305,7 @@ def test_monochrome_correlation(resources, outdir):
assert corr > 0.90 assert corr > 0.90
@pytest.mark.parametrize('renderer', [ @pytest.mark.parametrize('renderer', RENDERERS)
'hocr',
'tesseract',
])
def test_autorotate(spoof_tesseract_cache, renderer, resources, outdir): def test_autorotate(spoof_tesseract_cache, renderer, resources, outdir):
# cardinal.pdf contains four copies of an image rotated in each cardinal # cardinal.pdf contains four copies of an image rotated in each cardinal
# direction - these ones are "burned in" not tagged with /Rotate # direction - these ones are "burned in" not tagged with /Rotate
@@ -341,10 +340,7 @@ def test_autorotate_threshold(
assert eval(correlation_test) assert eval(correlation_test)
@pytest.mark.parametrize('renderer', [ @pytest.mark.parametrize('renderer',RENDERERS)
'hocr',
'tesseract',
])
def test_ocr_timeout(renderer, resources, outpdf): def test_ocr_timeout(renderer, resources, outpdf):
out = check_ocrmypdf(resources / 'skew.pdf', outpdf, out = check_ocrmypdf(resources / 'skew.pdf', outpdf,
'--tesseract-timeout', '1.0') '--tesseract-timeout', '1.0')
@@ -359,7 +355,7 @@ def test_skip_big(spoof_tesseract_cache, resources, outpdf):
assert not pdfinfo[0].has_text assert not pdfinfo[0].has_text
@pytest.mark.parametrize('renderer', ['hocr', 'tesseract']) @pytest.mark.parametrize('renderer', RENDERERS)
@pytest.mark.parametrize('output_type', ['pdf', 'pdfa']) @pytest.mark.parametrize('output_type', ['pdf', 'pdfa'])
def test_maximum_options(spoof_tesseract_cache, renderer, output_type, def test_maximum_options(spoof_tesseract_cache, renderer, output_type,
resources, outpdf): resources, outpdf):
@@ -515,10 +511,7 @@ def test_tesseract_crash_autorotate(spoof_tesseract_crash,
print(err) print(err)
@pytest.mark.parametrize('renderer', [ @pytest.mark.parametrize('renderer', RENDERERS)
'hocr',
'tesseract',
])
def test_tesseract_image_too_big(renderer, spoof_tesseract_big_image_error, def test_tesseract_image_too_big(renderer, spoof_tesseract_big_image_error,
resources, outpdf): resources, outpdf):
check_ocrmypdf( check_ocrmypdf(
@@ -566,10 +559,7 @@ def test_non_square_resolution(renderer, spoof_tesseract_cache,
assert in_pageinfo[0].yres == out_pageinfo[0].yres assert in_pageinfo[0].yres == out_pageinfo[0].yres
@pytest.mark.parametrize('renderer', [ @pytest.mark.parametrize('renderer', RENDERERS)
'hocr',
'tesseract'
])
def test_convert_to_square_resolution(renderer, spoof_tesseract_cache, def test_convert_to_square_resolution(renderer, spoof_tesseract_cache,
resources, outpdf): resources, outpdf):
from math import isclose from math import isclose
@@ -773,10 +763,7 @@ language_model_penalty_non_freq_dict_word 0
'--tesseract-config', str(cfg_file)) '--tesseract-config', str(cfg_file))
@pytest.mark.parametrize('renderer', [ @pytest.mark.parametrize('renderer', RENDERERS)
'hocr',
'tesseract',
])
def test_tesseract_config_notfound(renderer, resources, outdir): def test_tesseract_config_notfound(renderer, resources, outdir):
cfg_file = outdir / 'nofile.cfg' cfg_file = outdir / 'nofile.cfg'
@@ -788,10 +775,7 @@ def test_tesseract_config_notfound(renderer, resources, outdir):
assert p.returncode == ExitCode.ok assert p.returncode == ExitCode.ok
@pytest.mark.parametrize('renderer', [ @pytest.mark.parametrize('renderer', RENDERERS)
'hocr',
'tesseract',
])
def test_tesseract_config_invalid(renderer, resources, outdir): def test_tesseract_config_invalid(renderer, resources, outdir):
cfg_file = outdir / 'test.cfg' cfg_file = outdir / 'test.cfg'
with cfg_file.open('w') as f: with cfg_file.open('w') as f:
@@ -813,10 +797,7 @@ def test_form_xobject(spoof_tesseract_noop, resources, outpdf):
env=spoof_tesseract_noop) env=spoof_tesseract_noop)
@pytest.mark.parametrize('renderer', [ @pytest.mark.parametrize('renderer', RENDERERS)
'hocr',
'tesseract',
])
def test_pagesize_consistency(renderer, resources, outpdf): def test_pagesize_consistency(renderer, resources, outpdf):
from math import isclose from math import isclose
+14 -2
View File
@@ -11,14 +11,26 @@ pytestmark = pytest.mark.skipif(tesseract.v4(),
reason="tesseract 3.x required") reason="tesseract 3.x required")
def test_textonly_pdf_on_tess3(resources, no_outpdf): @pytest.mark.skipif(tesseract.has_textonly_pdf(),
reason="check that missing dep is reported on old tess3")
def test_textonly_pdf_on_older_tess3(resources, no_outpdf):
p, _, _ = pytest.helpers.run_ocrmypdf( p, _, _ = pytest.helpers.run_ocrmypdf(
resources / 'linn.pdf', resources / 'linn.pdf',
no_outpdf, '--pdf-renderer', 'tess4') no_outpdf, '--pdf-renderer', 'sandwich')
assert p.returncode == ExitCode.missing_dependency assert p.returncode == ExitCode.missing_dependency
@pytest.mark.skipif(not tesseract.has_textonly_pdf(),
reason="check that feature is exercised on new test3")
def test_textonly_pdf_on_newer_tess3(resources, no_outpdf):
p, _, _ = pytest.helpers.run_ocrmypdf(
resources / 'linn.pdf',
no_outpdf, '--pdf-renderer', 'sandwich')
assert p.returncode == ExitCode.ok
def test_oem_on_tess3(resources, no_outpdf): def test_oem_on_tess3(resources, no_outpdf):
p, _, err = pytest.helpers.run_ocrmypdf( p, _, err = pytest.helpers.run_ocrmypdf(
resources / 'aspect.pdf', resources / 'aspect.pdf',
+14 -3
View File
@@ -8,6 +8,7 @@ from ocrmypdf import pdfinfo
import sys import sys
import os import os
import PyPDF2 as pypdf import PyPDF2 as pypdf
from contextlib import contextmanager
spoof = pytest.helpers.spoof spoof = pytest.helpers.spoof
@@ -32,14 +33,25 @@ def ensure_tess4():
raise EnvironmentError("Can't find Tesseract 4") raise EnvironmentError("Can't find Tesseract 4")
@contextmanager
def modified_os_environ(env):
old_env = os.environ.copy()
os.environ = env
yield
os.environ = old_env
def tess4_available(): def tess4_available():
"""Check if a tesseract 4 binary is available, even if it's not the """Check if a tesseract 4 binary is available, even if it's not the
official "tesseract" on PATH official "tesseract" on PATH
""" """
try: try:
ensure_tess4() # ensure_tess4 locates the tess4 binary we are going to check
return True env = ensure_tess4()
with modified_os_environ(env):
# Now jump into this environment and make sure it really is Tess4
return tesseract.v4() and tesseract.has_textonly_pdf()
except EnvironmentError: except EnvironmentError:
pass pass
@@ -63,7 +75,6 @@ def test_textonly_pdf(ensure_tess4, resources, outdir):
env=ensure_tess4) env=ensure_tess4)
@pytest.mark.skipif(sys.version_info < (3, 5), reason="needs math.isclose")
def test_pagesize_consistency_tess4(ensure_tess4, resources, outpdf): def test_pagesize_consistency_tess4(ensure_tess4, resources, outpdf):
from math import isclose from math import isclose