From 2ac8e8a0cc1ebd313b43699beda4225726d919ed Mon Sep 17 00:00:00 2001 From: "James R. Barlow" Date: Thu, 8 Sep 2016 11:29:53 -0700 Subject: [PATCH] More doc tweaks, mainly introduction --- docs/introduction.rst | 15 +++++++++------ ocrmypdf/pdfa.py | 11 ++++++----- 2 files changed, 15 insertions(+), 11 deletions(-) diff --git a/docs/introduction.rst b/docs/introduction.rst index f4815d68..32511c78 100644 --- a/docs/introduction.rst +++ b/docs/introduction.rst @@ -21,7 +21,7 @@ PDFs are `vector graphic files `_ to rasterize the page, and then performs on OCR on the rasterized image. It is not enough to simply extract the images from each page and run OCR on them individually. Of course one could use Ghostscript or another PDF rasterizer directly; the advantage of OCRmyPDF its ability to automate the process and produce a minimally changed output file. +OCRmyPDF analyzes each page of a PDF to determine the colorspace and resolution (DPI) needed to capture all of the information on that page without losing content. It uses `Ghostscript `_ to rasterize the page, and then performs on OCR on the rasterized image. It is not enough to simply extract the images from each page and run OCR on them individually. Of course one could use Ghostscript or another PDF rasterizer and then pass the image to Tesseract. OCRmyPDF automates this process and produces a minimally changed output file that contains the same information, colorspace and resolution. The Tesseract OCR engine can output 'hOCR' files, which are XML files that contain a description of the text it found on the page. OCRmyPDF will render a new PDF that contains only the hidden text layer, and merge this with the original page. @@ -51,16 +51,19 @@ Limitations OCRmyPDF is limited by the Tesseract OCR engine. As such it experiences these limitations, as do any other programs that rely on Tesseract: -* The OCR is not as accurate quality as commercial solutions such as Abbyy. +* The OCR is not as accurate as commercial solutions such as Abbyy. * It is not capable of recognizing handwriting. -* It may report OCR text as gibberish. +* It may find gibberish and report this as OCR output. +* If a document contains languages outside of those given in the ``-l LANG`` arguments, results may be poor. * It is not always good at analyzing the natural reading order of documents. For example, it may fail to recognize that a document contains two columns and join text across the columns. * Poor quality scans may produce poor quality OCR. Garbage in, garbage out. OCRmyPDF is also limited by the PDF specification: -* PDF encodes the position of text glyphs but does not encode document structure. There is no markup that divides a document in sections, paragraphs, sentences, or even words. As such all elements of document structure including the spaces between words must be derived heuristically. Some PDF viewers do a better job of this than others. +* PDF encodes the position of text glyphs but does not encode document structure. There is no markup that divides a document in sections, paragraphs, sentences, or even words (since blank spaces are not represented). As such all elements of document structure including the spaces between words must be derived heuristically. Some PDF viewers do a better job of this than others. Ghostscript also imposes some limitations: -* PDFs containing JBIG2-encoded content will be converted to CCITT Group4 encoding, which has lower compression ratios, if Ghostscript PDF/A is enabled. \ No newline at end of file +* PDFs containing JBIG2-encoded content will be converted to CCITT Group4 encoding, which has lower compression ratios, if Ghostscript PDF/A is enabled. + +OCRmyPDF is currently not designed to be used as a Python API; it is designed to be run as a command line tool. ``import ocrmypf`` currently attempts to process the command line on ``sys.argv`` at import time so it has side effects that will interfere with its use as a package. The API it presents should not be considered stable. \ No newline at end of file diff --git a/ocrmypdf/pdfa.py b/ocrmypdf/pdfa.py index 056add5c..115de315 100644 --- a/ocrmypdf/pdfa.py +++ b/ocrmypdf/pdfa.py @@ -70,13 +70,14 @@ def encode_text_string(s: str) -> str: From PDF 32000-1:2008 a string object may be included in hexademical form if it is enclosed in angle brackets. For general Unicode the string should - be UTF-16 (big endian) with byte order marks. A non-hexademical - representation is doable but this is preferable since it allows the output - Postscript file to be completely ASCII and no escaping of Postscript - characters is necessary. + be UTF-16 (big endian) with byte order marks. Many strings including all + ASCII strings fall could be encoded as PdfDocEncoding literals provided + that certain Postscript sequences are escaped. But it's far simpler to + encode everything as UTF-16. ''' - # Sometimes lazy C programmer leave their NULs at the end of strings + # Sometimes lazy C programmers leave their NULs at the end of strings they + # insert into PDFs # tests/resources/aspect.pdf is one example (created by ImageMagick) s = s.replace('\x00', '')