Commit Graph
471 Commits
Author SHA1 Message Date
James R. Barlow 6756016572 Add license notice to all files
Source files to GPL3

Exceptions:
-tests/spoof/* to MIT
-hocrtransform.py
-_unicodefun.py

Test resources to CC BY-SA 4.0 except when otherwise noted.

Add GPL license.
2018-03-24 02:33:24 -07:00
James R. Barlow f42123afc3 pipeline: make removal of merge_qpdf more explicit 2018-03-24 02:30:05 -07:00
James R. Barlow 1425ffd274 pipeline: Merge branch 'feature/mumerge' into test
Replaces qpdf page merging
2018-03-24 02:26:01 -07:00
James R. Barlow efecf42566 Add PyMuPDF and use to detect text on pages 2018-03-24 02:16:53 -07:00
James R. Barlow 74bdfc07fb mumerge: fix regressions 2018-03-24 01:18:22 -07:00
James R. Barlow 376dfdba1c Fix text/image files not closed in combine_layers 2018-03-23 13:48:37 -07:00
James R. Barlow 3795d6720f Try out pymupdf merging
With garbage collection it reduces waste on the worst case file.
That's nice. 1 MB -> 105 MB -> 1.5 MB.

Indicates really problem is using PyPDF2 to watermark.

Currently hacked into --output-type pdf.
2018-03-23 13:39:16 -07:00
James R. Barlow 537aaf56d7 Remove duplication between page merge functions 2018-03-23 13:38:53 -07:00
James R. Barlow dea8fcfb5b Optimize page splitting by multiprocessing
Previously page splitting occurred in a single process because it was
not believed to affect performance much. It turned out to be an expensive
operation.

It now scales better with large page sizes although this has a negative
effect on small files.

Overall time changes as follows:

7 page file, 9.02s -> 9.56s
731 page file, 213s -> 97s

WITH --tesseract-timeout 0 --output-type pdf --skip-text

i.e. you don't get a 2.2x speed gain when OCR is available.

Squashed a commit to ix test suite failure on --rotate-pages
Squashed a commit to remove debug code
2018-03-23 13:07:51 -07:00
James R. Barlow 4f1f3b9b51 Move available_cpu_count to helpers 2018-03-23 13:07:51 -07:00
James R. Barlow dfeb8812ad Document some instances of 0 vs 1-based page numbering, import cleanup 2018-03-23 13:07:31 -07:00
James R. Barlow e5f27b7a12 Solve text detection issue with PyMuPDF 2018-03-15 22:29:56 -07:00
James R. Barlow 319aff6d09 Merge better-hocr 2018-03-15 16:59:59 -07:00
James R. Barlow a614fa3400 hocr: simplify some math expressions and add comments 2018-03-14 17:05:40 -07:00
James R. Barlow 0089a84c94 hocr: Make interword spaces default and non-optional for hocr
Update documentation to match.
2018-03-13 14:51:47 -07:00
James R. Barlow 90676e1c6a hocr: Remove baseline dashes 2018-03-13 14:45:31 -07:00
James R. Barlow 062901be43 Some cleanup and variable renaming 2018-03-13 14:29:22 -07:00
James R. Barlow b195d79b50 Refactoring 2018-03-13 11:04:34 -07:00
James R. Barlow 6d7ee98721 Force Tesseract 4 to be single threaded
Gives better performance (throughput basis) than the existing solution
and scales better on powerful boxes.
2018-03-13 08:54:52 -07:00
James R. Barlow d631c80024 Suppress debug message when merging large files 2018-03-09 11:10:45 -08:00
James R. Barlow f1f0033875 Suppress spurious debug message in --output-type pdf 2018-03-09 11:07:28 -08:00
James R. Barlow 995f8c106b hocr: account for baseline offset to position text more accurately 2018-03-09 07:45:41 -08:00
Jim Barlow 7cc104b138 hocr: account for skewed baseline 2018-03-05 11:16:40 -05:00
James R. Barlow b3a7299a62 hocr: refactor/improve PEP8 a bit 2018-03-05 10:47:36 -05:00
James R. Barlow 0e7a4deaec hocr: add baseline function, hocr doc link 2018-03-05 10:47:36 -05:00
James R. Barlow b4d66650bd hocr: adjust text cursor with relative moves 2018-03-05 10:47:36 -05:00
James R. Barlow 4986afca28 hocr: Refactor use of text object
We don't need to declare the font on each word.

No improvement for removing trailing space or adding \n
2018-03-05 10:47:36 -05:00
James R. Barlow 2b6004a82b hocr: Make words on line use the line height
Seems to improve the behavior and appearance
of selected text a fair bit.
2018-03-05 10:47:36 -05:00
James R. Barlow 04c54a7c31 Suppress spurious debug message in --output-type pdf 2018-03-05 10:47:36 -05:00
James R. Barlow 7ae6c5ae87 Trial merge interword-spaces 2018-03-02 23:47:06 -08:00
James R. Barlow 4046766ca5 Fix Python 3.5 test suite failure on symlinks
Did not account for API difference in pathlib
2018-03-02 16:57:46 -08:00
James R. Barlow 810390df0b pipeline: Refactor duplicate with clause 2018-03-02 16:57:15 -08:00
James R. Barlow de749bc7ae Fix regression - output to stdout broken 2018-03-02 15:35:51 -08:00
Tucker Barbour 9fd9c7a51f Scale BoundingBox and Text elements to account for additional space.
Here we are manually scaling the pt width used for the BoundingBox and
the Text element when manually adding whitespace to account for
limitations of the PDF.js viewer. This fixes an initial regression
noticed when selecting text elements in Chrome and PDFium. The width
of the Text element and BoundBox had not been adjusted for the
additional whitespace so the highlighting was offset slightly.
2018-03-02 11:18:47 +00:00
Tucker BarbourandCharles Forcey e6e34251c6 Add option to explicitly add interword spaces to HOCR pdf-renderer
This commit includes an optional work around for limitations of the
PDF.js viewer described in
https://github.com/jbarlow83/OCRmyPDF/issues/133. Here is explicitly
add an addition space to text elements before drawing them on the PDF
canvas when using the HOCR renderer. This option does not apply to
other pdf renderers in OCRmyPDF and is turned off by default.
2018-03-01 13:15:03 -05:00
James R. Barlow 74ca736333 Issue #223: improve text of encrypted PDF error message 2018-02-27 15:08:22 -08:00
James R. Barlow 5e4fd8b0b9 compile_leptonica should have shebang 2018-02-24 12:49:36 -08:00
James R. Barlow 8ab8132411 lint: unused variables, wildcard imports 2018-02-24 12:48:52 -08:00
James R. Barlow 6899dd46e4 lint: remove extraneous backslash 2018-02-24 12:42:37 -08:00
James R. Barlow 8ad0697a20 lint: remove duplicate property definition 2018-02-24 12:42:03 -08:00
James R. Barlow b47e5672e6 Remove old test case that no longer works 2018-02-24 12:40:14 -08:00
James R. Barlow 45c7bd9a60 lint: Remove shebangs from non-executable files 2018-02-24 12:38:58 -08:00
James R. Barlow e7bcb95635 Fix pylint errors 2018-02-24 11:59:01 -08:00
James R. Barlow 3de83627a9 Handle output to /dev/null or directory (#219)
Previously we threw an exception if the output name was a directory (only after doing OCR) and would trigger a PermissionError on trying to flip permission bits of /dev/null due to shutil.copyfile implementation. Instead of copying file use shutil.copyfileobj which should also respect umask etc.
2018-02-19 22:15:07 -08:00
James R. Barlow 1dfc32d7e6 Preserve "text as curves" vector content
Never updated the checking logic to deal with a pure vector file with no text that needs an OCR layer. This is doable, so allow it.
2018-02-07 16:05:48 -08:00
James R. Barlow f5e07c9427 Fix Ghostscript parameter order 2018-01-10 16:33:26 -08:00
James R. Barlow 75dcb90621 Niceties: when environment variable overload is used clarify we're not checking the PATH 2018-01-10 16:33:03 -08:00
James R. Barlow 882fc2257c Add --max-image-mpixels argument to support Pillow 5.0 2018-01-10 15:43:59 -08:00
James R. Barlow 6bf1f970a0 Fix some parameter validation for --output-type pdfa-1 and pdfa-2 2018-01-10 11:50:08 -08:00
James R. Barlow 7d451f101f Detect old versions of Ghostscript and warn about them (#208) 2018-01-10 11:47:39 -08:00