v6.1.3 notes

Closes #247

.docker/Dockerfile

@@ -28,7 +28,7 @@ RUN python3 -m venv --system-site-packages /appenv
 # (i.e. "pip install ."). It's unclear to me why this is the case.
 RUN . /appenv/bin/activate; \
   pip install --upgrade pip \
-  && pip install ocrmypdf
+  && pip install ocrmypdf[fitz]
 
 # Now copy the application in, mainly to get the test suite.
 # Do this now to make the best use of Docker cache.

.docker/Dockerfile.tess4

@@ -38,7 +38,7 @@ RUN python3 -m venv --system-site-packages /appenv
 # (i.e. "pip install ."). It's unclear to me why this is the case.
 RUN . /appenv/bin/activate; \
   pip install --upgrade pip \
-  && pip install ocrmypdf
+  && pip install ocrmypdf[fitz]
 
 # Now copy the application in, mainly to get the test suite.
 # Do this now to make the best use of Docker cache.

.travis.yml

@@ -55,7 +55,7 @@ install:
 script:
 - tesseract --version
 - qpdf --version
-- pytest
+- pytest -n auto
 
 deploy:
   # release for main pypi

docs/batch.rst

@@ -18,7 +18,7 @@ The ``--tag`` argument tells parallel to print the filename as a prefix whenever
 
 	parallel --tag -j 2 ocrmypdf '{}' 'output/{}' ::: *.pdf
 
-OCRmyPDF automaticaly repairs PDFs before parsing and gathering information from them.  If you are already repairing PDFs with ``qpdf`` prior to attempting OCR, or you can use ``--skip-repair`` to skip this step.  It may improve performance for large files, since repairing PDFs is single-threaded.
+OCRmyPDF automatically repairs PDFs before parsing and gathering information from them.  If you are already repairing PDFs with ``qpdf`` prior to attempting OCR, or you can use ``--skip-repair`` to skip this step.  It may improve performance for large files, since repairing PDFs is single-threaded.
 
 Directory trees
 ---------------

docs/installation.rst

@@ -3,6 +3,15 @@ Installation
 
 OCRmyPDF requires Python 3.5 (or newer) and Tesseract 3.04 (or newer). 
 
+Python 3.6 and Tesseract 4.x are recommended for best OCR results and best performance.
+
+OCRmyPDF 6.x adds a dependency on PyMuPDF ("fitz"). This library is not widely available in platform distributions, and it improves OCRmyPDF in certain conditions. Consider installing OCRmyPDF from the Python binary wheels, which include a precompiled version of this library.
+
+
+.. contents:: Platform-specific steps
+    :depth: 1
+    :local:
+
 Installing on Debian and Ubuntu 16.10 or newer
 ----------------------------------------------
 
@@ -12,8 +21,14 @@ Users of Debian 9 ("stretch") or later or Ubuntu 16.10 or later may simply
 
     apt-get install ocrmypdf
 
-Installing on macOS
+To see what versions are available, check the `Debian Package Tracker <https://tracker.debian.org/pkg/ocrmypdf>`_ or `Ubuntu launchpad.net <https://launchpad.net/ocrmypdf>`_.
--------------------
+
+Installing on macOS with Homebrew
+---------------------------------
+
+.. image:: https://img.shields.io/homebrew/v/ocrmypdf.svg   
+    :alt: homebrew
+    :target: http://brewformulas.org/Ocrmypdf
 
 OCRmyPDF is now a standard `Homebrew <https://brew.sh>`_ formula. To install on macOS:
 
@@ -36,11 +51,9 @@ Installing the Docker image
 
 For many users, installing the Docker image will be easier than installing all of OCRmyPDF's dependencies. For Windows, it is the only option.
 
-If you have `Docker <https://docs.docker.com/>`_ installed on your system, you can install
+If you have `Docker <https://docs.docker.com/>`_ installed on your system, you can install a Docker image of the latest release.
-a Docker image of the latest release.
 
-Follow the Docker installation instructions for your platform.  If you can run this command
+Follow the Docker installation instructions for your platform.  If you can run this command successfully, your system is ready to download and execute the image:
-successfully, your system is ready to download and execute the image:
 
 .. code-block:: bash
 
@@ -58,16 +71,22 @@ OCRmyPDF will use all available CPU cores.  By default, the VirtualBox machine i
 
 Assuming you have a Docker engine running, you can download one of the three available images:
 
-+-----------------------------+---------------------------------------------+---------------------------------------------------------------------------------+
+.. list-table::
-| Image name                  | Download command                            | Notes                                                                           |
+    :widths: auto
-+-----------------------------+---------------------------------------------+---------------------------------------------------------------------------------+
+    :header-rows: 1
-| ocrmypdf                    | ``docker pull jbarlow83/ocrmypdf``          | Latest ocrmypdf with Tesseract 3.04. Includes English, French, German, Spanish. |
+
-+-----------------------------+---------------------------------------------+---------------------------------------------------------------------------------+
+    *   - Image name
-| ocrmypdf-polyglot           | ``docker pull jbarlow83/ocrmypdf-polyglot`` | As above, with all available language packs.                                    |
+        - Download command
-+-----------------------------+---------------------------------------------+---------------------------------------------------------------------------------+
+        - Notes
-| ocrmypdf-tess4              | ``docker pull jbarlow83/ocrmypdf-tess4``    | Latest ocrmypdf with Tesseract 4.00.00alpha and English, French, German,        |
+    *   - ocrmypdf
-|                             |                                             | Spanish, Portuguese, Chinese Simplified, Arabic and Russian (the top 8).        |
+        - ``docker pull jbarlow83/ocrmypdf``
-+-----------------------------+---------------------------------------------+---------------------------------------------------------------------------------+
+        - Latest ocrmypdf with Tesseract 3.x. Includes English, French, German, Spanish.
+    *   - ocrmypdf-polyglot
+        - ``docker pull jbarlow83/ocrmypdf-polyglot``
+        - As above, with all available language packs.
+    *   - ocrmypdf-tess4
+        - ``docker pull jbarlow83/ocrmypdf-tess4``
+        - Latest ocrmypdf with Tesseract 4.x and English, French, German, Spanish, Portuguese, Chinese Simplified, Arabic and Russian (the top 8).
 
 For example:
 
@@ -169,24 +188,25 @@ Install the required Tesseract OCR engine with the language packs you plan to us
 
     brew install tesseract --with-all-languages  # Option 2: for all language packs
 
-Update the homebrew pip and install Pillow:
+Update the homebrew pip:
 
 .. code-block:: bash
 
     pip3 install --upgrade pip
-    pip3 install --upgrade pillow
 
 You can then install OCRmyPDF from PyPI, for the current user:
 
 .. code-block:: bash
 
-    pip3 install --user ocrmypdf
+    pip3 install --user ocrmypdf[fitz]
 
 or system-wide:
 
 .. code-block:: bash
 
-    pip3 install ocrmypdf
+    pip3 install ocrmypdf[fitz]
+
+``[fitz]`` includes the optional dependency on PyMuPDF, which improves OCRmyPDF's output in many cases.
 
 The command line program should now be available:
 
@@ -215,29 +235,29 @@ If you wish install OCRmyPDF for the current user:
 
 .. code-block:: bash
 
-    pip3 install --user ocrmypdf
+    pip3 install --user ocrmypdf[fitz]
 
 Alternately, system-wide. Note that this may modify the system Python environment:
 
 .. code-block:: bash
 
-    sudo pip3 install ocrmypdf
+    sudo pip3 install ocrmypdf[fitz]
 
-If you wish to install OCRmyPDF to a virtual environment to isolate the system Python, you can
+If you wish to install OCRmyPDF to a virtual environment to isolate the system Python, you can follow these steps.
-follow these steps.
 
 .. code-block:: bash
 
     python3 -m venv venv-ocrmypdf
     source venv-ocrmypdf/bin/activate
-    pip3 install ocrmypdf
+    pip3 install ocrmypdf[fitz]
+
+``[fitz]`` includes the optional dependency on PyMuPDF, which improves OCRmyPDF's output in many cases.
 
 
 Installing on Ubuntu 14.04 LTS
 ------------------------------
 
-Installing on Ubuntu 14.04 LTS (trusty) is more difficult than some other options,
+Installing on Ubuntu 14.04 LTS (trusty) is more difficult than some other options, because it is older and does not provide ``pip``.
-because it is older and does not provide ``pip``.
 
 Update apt-get:
 
@@ -256,10 +276,7 @@ Install system dependencies:
         libffi-dev \
         qpdf
 
-We will need backports of Ghostscript 9.16, libav-11 (for unpaper 6.1),
+We will need backports of Ghostscript 9.16, libav-11 (for unpaper 6.1), Tesseract 4.00 (alpha), and Python 3.6. This will replace Ghostscript and Tesseract 3.x on your system. Python 3.6 will be installed alongside the system Python 3.
-Tesseract 4.00 (alpha), and Python 3.6. This will replace Ghostscript and
-Tesseract 3.x on your system. Python 3.6 will be installed alongside the system
-Python 3.
 
 If you prefer to not modify your system in this matter, consider using a Docker container.
 
@@ -285,7 +302,7 @@ Now we need to install ``pip`` and let it install ocrmypdf:
 .. code-block:: bash
 
     wget -O - -o /dev/null https://bootstrap.pypa.io/get-pip.py | python3.6
-    pip3.6 install ocrmypdf
+    pip3.6 install ocrmypdf[fitz]
 
 The ``wget`` command will download a program and run it.
 
@@ -312,6 +329,9 @@ Installing on Windows
 
 Direct installation on Windows is not possible.  Install the _`Docker` container as described above.  Ensure that your command prompt can run the docker "hello world" container.
 
+It would probably not be too difficult to run on Windows.  The main reason this has been avoided is the difficulty of packaging and installing the various non-Python dependencies: Tesseract, QPDF, Ghostscript, Leptonica.  Pull requests to add or improve Windows support would be quite welcome.
+
+
 Running on Windows
 ~~~~~~~~~~~~~~~~~~
 
@@ -343,15 +363,7 @@ Or, to install in `development mode <https://pythonhosted.org/setuptools/setupto
 
     pip3 install -e git+https://github.com/jbarlow83/OCRmyPDF.git
 
-On certain Linux distributions such as Ubuntu, you may need to use 
+You may find it easiest to install in a virtual environment, rather than system-wide:
-run the install command as superuser:
-
-.. code-block:: bash
-
-    sudo pip3 install [-e] git+https://github.com/jbarlow83/OCRmyPDF.git
-
-Note that this will alter your system's Python distribution. If you prefer 
-to not install as superuser, you can install the package in a Python virtual environment:
 
 .. code-block:: bash
 

docs/release_notes.rst

@@ -5,6 +5,12 @@ OCRmyPDF uses `semantic versioning <http://semver.org/>`_ for its command line i
 
 The OCRmyPDF package itself does not contain a public API, although it is fairly stable and breaking changes are usually timed with a major release. A future release will clearly define the stable public API.
 
+v6.1.3
+------
+
+-   Fix issue #247, ``/CreationDate`` metadata not copied from input to output.
+
+
 v6.1.2
 ------
 

setup.cfg

@@ -11,7 +11,6 @@ ignore =
 [tool:pytest]
 norecursedirs = lib .pc .git output cache resources
 testpaths = tests
-addopts = -n auto
 
 [metadata]
 license_file = LICENSE

setup.py

@@ -190,9 +190,6 @@ if not forced and command.startswith('install') or \
         package='qpdf',
         version_check_args=['--version']
     )
-else:
-    print('Skipping external program tests because of --force')
-
 
 if 'upload' in sys.argv[1:]:
     print('Use twine to upload the package - setup.py upload is insecure')

src/ocrmypdf/pdfa.py

@@ -19,6 +19,8 @@
 
 from string import Template
 from binascii import hexlify
+from datetime import datetime
+from xml.parsers.expat import ExpatError
 import pkg_resources
 import PyPDF2 as pypdf
 
@@ -98,7 +100,79 @@ def encode_text_string(s: str) -> str:
     return ascii_hex_str
 
 
+def encode_pdf_date(d: datetime) -> str:
+    """Encode Python datetime object as PDF date string
+
+    From Adobe pdfmark manual:    
+    (D:YYYYMMDDHHmmSSOHH'mm')
+    D: is an optional prefix. YYYY is the year. All fields after the year are
+    optional. MM is the month (01-12), DD is the day (01-31), HH is the
+    hour (00-23), mm are the minutes (00-59), and SS are the seconds
+    (00-59). The remainder of the string defines the relation of local
+    time to GMT. O is either + for a positive difference (local time is
+    later than GMT) or - (minus) for a negative difference. HH' is the
+    absolute value of the offset from GMT in hours, and mm' is the
+    absolute value of the offset in minutes. If no GMT information is
+    specified, the relation between the specified time and GMT is
+    considered unknown. Regardless of whether or not GMT
+    information is specified, the remainder of the string should specify
+    the local time.
+    """
+
+    pdfmark_date_fmt = r'%Y%m%d%H%M%S'
+    s = d.strftime(pdfmark_date_fmt)
+
+    tz = d.strftime('%z')
+    if tz == 'Z':
+        s += "+00'00'"
+    elif tz != '':
+        sign, tz_hours, tz_mins = tz[0], tz[1:3], tz[3:5]
+        s += "{}{}'{tz}'".format(sign, tz_hours, tz_mins)
+    return s
+
+
+def decode_pdf_date(s: str) -> datetime:
+    pdfmark_date_fmts = (
+        r'%Y%m%d%H%M%S%z',  # +0430 etc
+        r'%Y%m%d%H%M%S',    # no time zone
+        r'%Y%m%d%H%M%SZ')   # trailing Z
+
+    if s.startswith('D:'):
+        s = s[2:]
+    for fmt in pdfmark_date_fmts:
+        try:
+            return datetime.strptime(s, fmt)
+        except ValueError:
+            continue
+    return None
+
+
+def _get_pdfmark_dates(pdfmark):
+    """Encode dates for pdfmark Postscript.  The best way to deal with a
+    missing date entry is set it to null, because if the key is omitted 
+    Ghostscript will set it to now - we do not want to erase the fact that
+    the value was unknown.  Setting to an empty string breaks Ghostscript
+    9.22 as reported here:
+    https://bugs.ghostscript.com/show_bug.cgi?id=699182
+    """
+
+    for key in ('/CreationDate', '/ModDate'):
+        if key not in pdfmark:
+            continue
+        if pdfmark[key].strip() == '':
+            yield '  {} null'.format(key)
+            continue
+        date_str = pdfmark[key]
+        if date_str.startswith('D:'):
+            date_str = date_str[2:]
+        yield '  {} (D:{})'.format(key, date_str)
+
+
 def _get_pdfa_def(icc_profile, icc_identifier, pdfmark):
+    """Create a Postscript file for Ghostscript.  pdfmark contains the various
+    objects as strings; these must be encoded in ASCII, and dates have a 
+    special format."""
+
     # Ghostscript <= 9.21 has a bug where null entries in DOCINFO might produce
     # ERROR: VMerror (-25) on closing pdfwrite device.
     # https://bugs.ghostscript.com/show_bug.cgi?id=697684
@@ -107,12 +181,12 @@ def _get_pdfa_def(icc_profile, icc_identifier, pdfmark):
     docinfo_line_template = '  {key} <{value}>'
 
     def docinfo_gen():
+        yield from _get_pdfmark_dates(pdfmark)
         for key in docinfo_keys:
             if key in pdfmark and pdfmark[key].strip() != '':
                 line = docinfo_line_template.format(
                     key=key, value=encode_text_string(pdfmark[key]))
                 yield line
-
     docinfo = '\n'.join(docinfo_gen())
 
     t = Template(pdfa_def_template)
@@ -145,9 +219,12 @@ def file_claims_pdfa(filename):
 
     This checks if the XMP metadata contains a PDF/A marker.
     """
-
     pdf = pypdf.PdfFileReader(filename)
-    xmp = pdf.getXmpMetadata()
+    try:
+        xmp = pdf.getXmpMetadata()
+    except ExpatError:
+        return {'pass': False, 'output': 'pdf',
+                'conformance': 'Invalid XML metadata'}
 
     try:
         pdfa_nodes = xmp.getNodesInNamespace(

src/ocrmypdf/pipeline.py

@@ -18,6 +18,7 @@
 from contextlib import suppress
 from shutil import copyfileobj
 from pathlib import Path
+from datetime import datetime
 import sys
 import os
 import shutil
@@ -31,7 +32,7 @@ from ruffus import formatter, regex, Pipeline, suffix
 
 from .hocrtransform import HocrTransform
 from .pdfinfo import PdfInfo, Encoding, Colorspace
-from .pdfa import generate_pdfa_ps
+from .pdfa import generate_pdfa_ps, encode_pdf_date
 from .helpers import re_symlink, is_iterable_notstr, page_number
 from .exec import ghostscript, tesseract, qpdf
 from .lib import fitz
@@ -231,6 +232,12 @@ def repair_and_parse_pdf(
         )
         raise InputFileError()
     
+    if len(pdfinfo.pages) > 2000 and sys.version_info[0:2] <= (3, 5):
+        log.warning(
+            "Performance regressions are known occur with Python 3.5 for "
+            "high page count files.  Python 3.6 or newer is recommended."
+        )
+
     context.set_pdfinfo(pdfinfo)
     log.debug(pdfinfo)
 
@@ -624,19 +631,18 @@ def select_visible_page_image(
         log.debug('{:4d}: JPEG input -> JPEG output'.format(
             page_number(image)))
         # If all images were JPEGs originally, produce a JPEG as output
-        im = Image.open(image)
+        with Image.open(image) as im:
+            # At this point the image should be a .png, but deskew, unpaper
+            # might have removed the DPI information. In this case, fall back to
+            # square DPI used to rasterize. When the preview image was
+            # rasterized, it was also converted to square resolution, which is
+            # what we want to give tesseract, so keep it square.
+            fallback_dpi = get_page_square_dpi(pageinfo, options)
+            dpi = im.info.get('dpi', (fallback_dpi, fallback_dpi))
 
-        # At this point the image should be a .png, but deskew, unpaper might
+            # Pillow requires integer DPI
-        # have removed the DPI information. In this case, fall back to square
+            dpi = round(dpi[0]), round(dpi[1])
-        # DPI used to rasterize. When the preview image was rasterized, it
+            im.save(output_file, format='JPEG', dpi=dpi)
-        # was also converted to square resolution, which is what we want to
-        # give tesseract, so keep it square.
-        fallback_dpi = get_page_square_dpi(pageinfo, options)
-        dpi = im.info.get('dpi', (fallback_dpi, fallback_dpi))
-
-        # Pillow requires integer DPI
-        dpi = round(dpi[0]), round(dpi[1])
-        im.save(output_file, format='JPEG', dpi=dpi)
     else:
         re_symlink(image, output_file, log)
 
@@ -658,25 +664,25 @@ def select_image_layer(
         log.debug("{:4d}: page eligible for lossless reconstruction".format(
             page_number(page_pdf)))
         re_symlink(page_pdf, output_file, log)
-    else:
+        return
-        pageinfo = get_pageinfo(image, context)
 
-        # We rasterize a square DPI version of each page because most image
+    pageinfo = get_pageinfo(image, context)
-        # processing tools don't support rectangular DPI. Use the square DPI
-        # as it accurately describes the image. It would be possible to
-        # resample the image at this stage back to non-square DPI to more
-        # closely resemble the input, except that the hocr renderer does not
-        # understand non-square DPI. The sandwich renderer would be fine.
-        dpi = get_page_square_dpi(pageinfo, options)
-        layout_fun = img2pdf.get_fixed_dpi_layout_fun((dpi, dpi))
 
-        with open(image, 'rb') as imfile, \
+    # We rasterize a square DPI version of each page because most image
-                open(output_file, 'wb') as pdf:
+    # processing tools don't support rectangular DPI. Use the square DPI as it
-            log.debug('{:4d}: convert'.format(page_number(page_pdf)))
+    # accurately describes the image. It would be possible to resample the image
-            img2pdf.convert(
+    # at this stage back to non-square DPI to more closely resemble the input,
-                imfile, with_pdfrw=False,
+    # except that the hocr renderer does not understand non-square DPI. The
-                layout_fun=layout_fun, outputstream=pdf)
+    # sandwich renderer would be fine.
-            log.debug('{:4d}: convert done'.format(page_number(page_pdf)))
+    dpi = get_page_square_dpi(pageinfo, options)
+    layout_fun = img2pdf.get_fixed_dpi_layout_fun((dpi, dpi))
+
+    with open(image, 'rb') as imfile, open(output_file, 'wb') as pdf:
+        log.debug('{:4d}: convert'.format(page_number(page_pdf)))
+        img2pdf.convert(
+            imfile, with_pdfrw=False,
+            layout_fun=layout_fun, outputstream=pdf)
+        log.debug('{:4d}: convert done'.format(page_number(page_pdf)))
 
 
 def render_hocr_page(
@@ -872,12 +878,8 @@ def get_pdfmark(base_pdf, options):
         except (KeyError, TypeError):
             return ''
 
-    pdfmark = {
+    pdfmark = {k: from_document_info(k) for k in 
-        '/Title': from_document_info('/Title'),
+        ('/Title', '/Author', '/Keywords', '/Subject', '/CreationDate')}
-        '/Author': from_document_info('/Author'),
-        '/Keywords': from_document_info('/Keywords'),
-        '/Subject': from_document_info('/Subject'),
-    }
     if options.title:
         pdfmark['/Title'] = options.title
     if options.author:
@@ -898,6 +900,7 @@ def get_pdfmark(base_pdf, options):
         PROGRAM_NAME, VERSION,
         renderer_tag,
         tesseract.version())
+    pdfmark['/ModDate'] = encode_pdf_date(datetime.utcnow())
     return pdfmark
 
 
@@ -1031,7 +1034,7 @@ def merge_pages_mupdf(
     reader_metadata = pypdf.PdfFileReader(metadata_file)
     pdfmark = get_pdfmark(reader_metadata, options)
     pdfmark['/Producer'] = 'PyMuPDF ' + fitz.version[0]
-    pymupdf_metadata = {k[1:].lower() : v for k, v in pdfmark.items()}
+    pymupdf_metadata = {(k[1].lower() + k[2:]) : v for k, v in pdfmark.items()}
 
     for pdf_page in pdf_pages:
         page = fitz.open(pdf_page)

tests/resources/README.rst

@@ -46,80 +46,97 @@ The following test resources were crafted specifically for this project, and are
 licensed under the specified license.
 
 .. list-table:: 
-    :widths: 20 20 60
+    :widths: 20 40 15 15 10
     :header-rows: 1
 
     *   - File
-        - Contributor
         - Purpose
+        - Contributor
+        - Copyright Holder
         - License
     *   - aspect.pdf
-        - @jbarlow83
         - test image with 200 x 100 DPI resolution
-        - CC BY-SA 4.0
+        - @jbarlow83
+        - @jbarlow83
+        - CC-BY-SA 4.0
     *   - blank.pdf
-        - @jbarlow83
         - blank PDF generated by Adobe Illustrator CC 17, containing a lot of application-specific metadata/bloat
-        - CC BY-SA 4.0
+        - @jbarlow83
+        - @jbarlow83
+        - CC-BY-SA 4.0
     *   - cmyk.pdf
-        - @jbarlow83
         - a CMYK image created in Photoshop
-        - CC BY-SA 4.0        
+        - @jbarlow83
+        - @jbarlow83
+        - CC-BY-SA 4.0        
     *   - crom.png
-        - @jbarlow83
         - test for non-dictionary words
-        - CC BY-SA 4.0        
+        - @jbarlow83
+        - @jbarlow83
+        - CC-BY-SA 4.0        
     *   - enormous.pdf
-        - @jbarlow83
         - very large PDF page
-        - CC BY-SA 4.0
+        - @jbarlow83
+        - @jbarlow83
+        - CC-BY-SA 4.0
     *   - epson.pdf
+        - a linearized PDF containing some unusual indirect objects, created by an Epson printer; printout of a Wikipedia article (CC-BY-SA)
         - @lowesjam
-        - a linearized PDF containing some unusual indirect objects, created by an Epson printer; printout of a Wikipedia article (CC BY-SA)
+        - Wikipedia authors
-        - MIT
+        - CC-BY-SA 3.0
     *   - formxobject.pdf
-        - @jbarlow83
         - hand-crafted PDF containing an image inside a Form XObject
-        - CC BY-SA 4.0
+        - @jbarlow83
+        - @jbarlow83
+        - CC-BY-SA 4.0
     *   - francais.pdf
-        - @jbarlow83
         - a page containing French accents (diacritics)  
-        - CC BY-SA 4.0
+        - @jbarlow83
+        - @jbarlow83
+        - CC-BY-SA 4.0
     *   - hugemono.pdf
-        - @jbarlow83
         - large monochrome 35000x35000 image in JBIG2 encoding 
-        - CC BY-SA 4.0
-    *   - invalid.pdf
         - @jbarlow83
+        - @jbarlow83
+        - CC-BY-SA 4.0
+    *   - invalid.pdf
         - a PDF file header followed by EOF marker
-        - CC BY-SA 4.0
+        - @jbarlow83
+        - @jbarlow83
+        - CC-BY-SA 4.0
     *   - masks.pdf
+        - file containing explicit masks and a stencil mask drawn without a proper transformation matrix; printout of a German Wikipedia article (CC-BY-SA)
         - @supergrobi
-        - file containing explicit masks and a stencil mask drawn without a proper transformation matrix; printout of a German Wikipedia article (CC BY-SA)
+        - Wikipedia authors
-        - MIT
+        - CC-BY-SA 3.0
     *   - missing_docinfo.pdf
         - @jbarlow83
         - PDF file with no /DocumentInfo section 
-        - CC BY-SA 4.0
+        - CC-BY-SA 4.0
     *   - overlay.pdf
-        - @maxandersen
         - PDF file generated by PDFPen pro that triggered content stream parse errors
+        - @maxandersen
+        - @maxandersen
         - MIT
     *   - negzero.pdf
-        - @jbarlow83
         - copy of formxobject.pdf with token that qpdf doesn't like
-        - CC BY-SA 4.0
+        - @jbarlow83
+        - @jbarlow83
+        - CC-BY-SA 4.0
     *   - no_contents.pdf
-        - @jbarlow83
         - synthetic PDF with a blank page that has no /Contents entry
-        - CC BY-SA 4.0
-    *   - trivial.pdf
         - @jbarlow83
+        - @jbarlow83
+        - CC-BY-SA 4.0
+    *   - trivial.pdf
         - smallest possible valid PDF-1.3 with all required fields
-        - CC BY-SA 4.0
+        - @jbarlow83
+        - @jbarlow83
+        - CC-BY-SA 4.0
     *   - vector.pdf
-        - @Catscratch
         - a PDF with vector art and text rendered as curves with no fonts
+        - @Catscratch
+        - @Catscratch
         - MIT
 
 

tests/test_metadata.py

@@ -18,8 +18,9 @@
 
 import pytest
 import PyPDF2 as pypdf
+import datetime
 
-from ocrmypdf.pdfa import file_claims_pdfa
+from ocrmypdf.pdfa import file_claims_pdfa, encode_pdf_date, decode_pdf_date
 from ocrmypdf.exceptions import ExitCode
 from ocrmypdf.lib import fitz
 
@@ -115,3 +116,40 @@ def test_bookmarks_preserved(spoof_tesseract_noop, output_type, ocr_option,
     print(before_toc)
     print(after_toc)
     assert before_toc == after_toc
+
+
+def seconds_between_dates(date1, date2):
+    return (date2 - date1).total_seconds()
+
+
+@pytest.mark.parametrize('infile', ['trivial.pdf', 'jbig2.pdf'])
+@pytest.mark.parametrize('output_type', ['pdf', 'pdfa'])
+def test_creation_date_preserved(spoof_tesseract_noop, output_type, resources,
+                                 infile, outpdf):
+    input_file = resources / infile
+
+    before = pypdf.PdfFileReader(str(input_file)).getDocumentInfo()
+    check_ocrmypdf(
+        input_file, outpdf, '--output-type', output_type, 
+        env=spoof_tesseract_noop)
+    after = pypdf.PdfFileReader(str(outpdf)).getDocumentInfo()
+
+    if not before:
+        # If there was input creation date, none should be output
+        # because of Ghostscript quirks we set it to null
+        # This test would be better if we had a test file with /DocumentInfo but
+        # no /CreationDate, which we don't
+        assert not after['/CreationDate'] or \
+                isinstance(after['/CreationDate'], pypdf.generic.NullObject)
+    else:
+        # We expect that the creation date stayed the same
+        date_before = decode_pdf_date(before['/CreationDate'])
+        date_after = decode_pdf_date(after['/CreationDate'])
+        assert seconds_between_dates(date_before, date_after) < 1000
+
+    # We expect that the modified date is quite recent
+    date_after = decode_pdf_date(after['/ModDate'])
+    assert seconds_between_dates(
+        date_after, datetime.datetime.utcnow()) < 1000
+
+