Add vector-only PDF test case

Never updated the checking logic to deal with a pure vector file with no text that needs an OCR layer. This is doable, so allow it.

.travis/autobrew.py

@@ -14,11 +14,11 @@ class Ocrmypdf < Formula
   ${ocrmypdf_url}
   ${ocrmypdf_sha256}
 
+  depends_on "pkg-config" => :build
   depends_on "freetype"
   depends_on "ghostscript"
   depends_on "jpeg"
   depends_on "libpng"
-  depends_on "pkg-config" => :build
   depends_on "python3"
   depends_on "qpdf"
   depends_on "tesseract"
@@ -55,28 +55,8 @@ ${resources}
     # Since we use Python 3, we require a UTF-8 locale
     ENV["LC_ALL"] = "en_US.UTF-8"
 
-    # One page Postscript with the wording "Testing" on the page
-    # This is more compact than including a test PDF
-    (testpath/"test.ps").write(
-      <<~EOS
-        %!PS
-        /Times-Roman findfont
-        20 scalefont
-        setfont
-        gsave
-        newpath
-        200 400 moveto
-        (Testing) show
-        closepath
-        stroke
-        showpage
-      EOS
-    )
-
-    system "#{Formula["ghostscript"].opt_bin}/ps2pdf", testpath/"test.ps", testpath/"test.pdf"
-
     # Use ocrmypdf -f to rasterize the PDF to image before doing OCR
-    system "#{bin}/ocrmypdf", "-f", "-q", "--deskew", testpath/"test.pdf", testpath/"ocr.pdf"
+    system "#{bin}/ocrmypdf", "-f", "-q", "--deskew", test_fixtures("test.pdf"), testpath/"ocr.pdf"
   end
 end
 """)

Dockerfile

@@ -1,6 +1,6 @@
 # OCRmyPDF
 #
-FROM      ubuntu:17.04
+FROM      ubuntu:17.10
 MAINTAINER James R. Barlow <jim@purplerock.ca>
 
 RUN apt-get update && apt-get install -y --no-install-recommends \

Dockerfile.tess4

@@ -1,6 +1,6 @@
 # OCRmyPDF
 #
-FROM 	ubuntu:17.04
+FROM 	ubuntu:17.10
 MAINTAINER James R. Barlow <jim@purplerock.ca>
 
 RUN apt-get update && apt-get install -y --no-install-recommends \

docs/installation.rst

@@ -285,6 +285,8 @@ Now we need to install ``pip`` and let it install ocrmypdf:
     wget -O - -o /dev/null https://bootstrap.pypa.io/get-pip.py | python3.6
     pip3.6 install ocrmypdf
 
+The ``wget`` command will download a program and run it.
+
 These installation instructions omit the optional dependency ``unpaper``, which is only available at version 0.4.2 in Ubuntu 14.04. The author could not find a backport of ``unpaper``, and created a .deb package to do the job of installing unpaper 6.1 (for x86 64-bit only):
 
 .. code-block:: bash

docs/introduction.rst

@@ -16,7 +16,7 @@ OCRmyPDF uses `Tesseract <https://github.com/tesseract-ocr/tesseract>`_, the bes
 About PDFs
 ----------
 
-PDFs are page description files that attempts to preserve a layout exactly. They can contain `vector graphic files <http://vector-conversions.com/vectorizing/raster_vs_vector.html>`_ that can contain raster objects such as scanned images. Because PDFs can contain multiple pages (unlike many image formats) and can contain fonts and text, it is a good formats for exchanging scanned documents.
+PDFs are page description files that attempts to preserve a layout exactly. They  contain `vector graphics <http://vector-conversions.com/vectorizing/raster_vs_vector.html>`_ that can contain raster objects such as scanned images. Because PDFs can contain multiple pages (unlike many image formats) and can contain fonts and text, it is a good formats for exchanging scanned documents.
 
 .. image:: bitmap_vs_svg.svg
 
@@ -42,15 +42,13 @@ PDF/A has a few drawbacks.  Some PDF viewers include an alert that the file is a
 What OCRmyPDF does
 ------------------
 
-OCRmyPDF analyzes each page of a PDF to determine the colorspace and resolution (DPI) needed to capture all of the information on that page without losing content.  It uses `Ghostscript <http://ghostscript.com/>`_ to rasterize the page, and then performs on OCR on the rasterized image.  It is not enough to simply extract the images from each page and run OCR on them individually.  Of course one could use Ghostscript or another PDF rasterizer and then pass the image to Tesseract.  OCRmyPDF automates this process and produces a minimally changed output file that contains the same information, colorspace and resolution.
+OCRmyPDF analyzes each page of a PDF to determine the colorspace and resolution (DPI) needed to capture all of the information on that page without losing content.  It uses `Ghostscript <http://ghostscript.com/>`_ to rasterize the page, and then performs on OCR on the rasterized image to create an OCR "layer". The layer is then grafted back onto the original PDF.
 
-The Tesseract OCR engine can output 'hOCR' files, which are XML files that contain a description of the text it found on the page.  OCRmyPDF will render a new PDF that contains only the hidden text layer, and merge this with the original page.
+While one can use a program like Ghostscript or ImageMagick to get an image and put the image through Tesseract, that actually creates a new PDF and many details may be lost. OCRmyPDF can produce a minimally changed PDF as output.
 
-Alternately, OCRmyPDF can use the Tesseract OCR engine to directly output PDFs for each page, then merge them.
+OCRmyPDF also some image processing options like deskew which improve the appearance of files and quality of OCR. When these are used, the OCR layer is grafted onto the processed image instead.
 
-By default, OCRmyPDF will convert the file to a PDF/A.  This behavior can be disabled with the ``--output-type pdf`` argument.
-
-Depending on the settings selected, OCRmyPDF may "graft" the OCR layer into the existing PDF, or reconstruct a visually equivalent new PDF.
+By default, OCRmyPDF produces archival PDFs – PDF/A, which are a stricter subset of PDF features designed for long term archives. If regular PDFs are desired, this can be disabled with ``--output-type pdf``.
 
 
 Why you shouldn't do this manually

docs/release_notes.rst

@@ -5,6 +5,14 @@ OCRmyPDF uses `semantic versioning <http://semver.org/>`_ for its command line i
 
 The OCRmyPDF package itself does not contain a public API, although it is fairly stable and breaking changes are usually timed with a major release. A future release will clearly define the stable public API.
 
+v5.6.0
+------
+
+-   Fix issue #216: preserve "text as curves" PDFs without rasterizing file
+-   Related to the above, messages about rasterizing are more consistent
+-   For consistency versions minor releases will now get the trailing .0 they always should have had.
+
+
 v5.5
 ----
 

ocrmypdf/pipeline.py

@@ -244,7 +244,30 @@ def get_canvas_square_dpi(pageinfo, options):
 def is_ocr_required(pageinfo, log, options):
     page = pageinfo.pageno + 1
     ocr_required = True
-    if not pageinfo.images:
+
+    if pageinfo.has_text:
+        msg = "{0:4d}: page already has text! – {1}"
+
+        if not options.force_ocr and not options.skip_text:
+            log.error(msg.format(page,
+                                 "aborting (use --force-ocr to force OCR)"))
+            raise PriorOcrFoundError()
+        elif options.force_ocr:
+            log.info(msg.format(page,
+                                "rasterizing text and running OCR anyway"))
+            ocr_required = True
+        elif options.skip_text:
+            log.info(msg.format(page,
+                                "skipping all processing on this page"))
+            ocr_required = False
+    elif not pageinfo.images and not options.lossless_reconstruction:
+        # We found a page with no images and no text. That means it may
+        # have vector art that the user wants to OCR. If we determined
+        # lossless reconstruction is not possible then we have to rasterize
+        # the image. So if OCR is being forced, take that to mean YES, go 
+        # ahead and rasterize. If not forced, then pretend there's no text
+        # on the page at all so we don't lose anything.
+        # This could be made smarter by explicitly searching for vector art.
         if options.force_ocr and options.oversample:
             # The user really wants to reprocess this file
             log.info(
@@ -263,23 +286,9 @@ def is_ocr_required(pageinfo, log, options):
         else:
             log.info(
                 "{0:4d}: page has no images - "
-                "skipping all processing on this page".format(page))
-            ocr_required = False
-
-    elif pageinfo.has_text:
-        msg = "{0:4d}: page already has text! – {1}"
-
-        if not options.force_ocr and not options.skip_text:
-            log.error(msg.format(page,
-                                 "aborting (use --force-ocr to force OCR)"))
-            raise PriorOcrFoundError()
-        elif options.force_ocr:
-            log.info(msg.format(page,
-                                "rasterizing text and running OCR anyway"))
-            ocr_required = True
-        elif options.skip_text:
-            log.info(msg.format(page,
-                                "skipping all processing on this page"))
+                "skipping all processing on this page to avoid losing detail. "
+                "Use --force-ocr if you wish to perform OCR on pages that "
+                "have vector content.".format(page))
             ocr_required = False
 
     if ocr_required and options.skip_big and pageinfo.images:

tests/resources/README.rst

@@ -100,6 +100,9 @@ under the terms of the license in LICENSE.rst.
     *   - trivial.pdf
         - @jbarlow83
         - smallest possible valid PDF-1.3 with all required fields
+    *   - vector.pdf
+        - @Catscratch
+        - a PDF with vector art and text rendered as curves with no fonts
 
 
 Assemblies

tests/test_main.py

@@ -433,7 +433,7 @@ def test_klingon(resources, outpdf):
 
 def test_missing_docinfo(spoof_tesseract_noop, resources, outpdf):
     p, out, err = run_ocrmypdf(
-        resources / 'missing_docinfo.pdf', outpdf, '-l', 'eng', '-c',
+        resources / 'missing_docinfo.pdf', outpdf, '-l', 'eng', '--skip-text',
         env=spoof_tesseract_noop)
     assert p.returncode == ExitCode.ok, err
 
@@ -1081,4 +1081,20 @@ def test_decompression_bomb(resources, outpdf):
         outpdf,
         '--max-image-mpixels', '2000'
     )
-    assert p.returncode == 0
+    assert p.returncode == 0
+
+
+def test_text_curves(spoof_tesseract_noop, resources, outpdf):
+    check_ocrmypdf(
+        resources / 'vector.pdf', outpdf, env=spoof_tesseract_noop)
+
+    info = PdfInfo(outpdf)
+    assert len(info.pages[0].images) == 0, "added images to the vector PDF"
+
+    check_ocrmypdf(
+        resources / 'vector.pdf', outpdf, '--force-ocr',
+        env=spoof_tesseract_noop)
+    
+    info = PdfInfo(outpdf)
+    assert len(info.pages[0].images) != 0, "force did not rasterize"
+