Commit Graph
122 Commits
Author SHA1 Message Date
James R. Barlow 4356983707 Suppress overly long stack traces on traverse_ruffus_exception 2016-07-31 02:06:44 -07:00
James R. Barlow 2414b79ee6 More cleanup of exception related errors 2016-07-31 01:48:13 -07:00
James R. Barlow 968e1546f0 Refactor image file triage 2016-07-31 01:47:57 -07:00
James R. Barlow f385772d21 Refactor "is this an iterable that's not a string?" test 2016-07-29 15:25:02 -07:00
James R. Barlow d257c83520 Most tests were failing at split_pages()
It seems that ruffus sometimes decides to send a ['inputfile.pdf']
instead of a bare string.
2016-07-29 14:59:17 -07:00
James R. Barlow 7b72ffec4f ocrmyimage: better handling of missing/invalid DPI 2016-07-29 14:38:07 -07:00
James R. Barlow 757f6826dc ocrmyimage - Attempt conversion to PDF if input file is not a PDF
First cut.

May have broken ruffus errors again too.
2016-07-29 14:03:19 -07:00
James R. Barlow d70e3d3753 ruffus exceptions: for clarity only, don't iterate strings
It's a good habit to ensure any iterator test is explicit about
allowing or disallowing strings.
2016-07-29 13:31:24 -07:00
James R. Barlow fef35e4eb2 Fix handling of DPI for rare case of JPEG recompression after deskew/clean
This test is exercised by page 4 of multipage.pdf. If all images are
JPEGs, and one of deskew/clean removes DPI information, make sure that
we can get the right information back and that the DPI stays square.
2016-07-29 01:34:52 -07:00
James R. Barlow 8f77576dc4 Fix non-square image resolution for "hocr" case; use img2pdf 0.2.1
Tesseract renderer not immediately fixable.
2016-07-28 16:43:51 -07:00
James R. Barlow b3fcf24a26 Refactor DPI: fix regressions in test suite
Some called functions are particular about the data format of DPI and
don't like to deal with the Decimal() returned by PyPDF2. Convert to
float and int where needed.
2016-07-28 00:19:32 -07:00
James R. Barlow 16e4d342d2 Bug fix: --force-ocr should still run on pages with no images
Useful for people who want to reprocess text.

This also requires --oversample because DPI is undefined. To be fixed
in next commit.
2016-07-27 15:06:49 -07:00
James R. Barlow bbd02926e1 Add helpful error message for PDFs that use algorithm 4 2016-06-23 13:13:17 -07:00
James R. Barlow 349ec5c81f Provide more helpful error message if pypdf can't merge pages 2016-04-28 14:02:12 -07:00
James R. Barlow fe14cb57c0 Fix ruffus exception output
I found this issue in ruffus 2.6.3
https://github.com/bunbun/ruffus/issues/65
also discussed here
https://github.com/bunbun/ruffus/pull/67

ruffus 2.6.3 RethrownJobError don't follow the normal conventions and
so its exception causes problems when they cross process boundaries.
This change carefully examines the various forms of ruffus exception
objects that can appear in 2.6.3 and parses them more carefully. It
also removes any direct posting of the exception to the logger because
this triggers another serializing of the exception object, mutating it
further.
2016-04-28 00:38:50 -07:00
James R. Barlow e877d37ac8 --rotate-pages: Only apply rotation if we're reasonable confident
Take the threshold from tesseract's default value for -psm 1.
2016-04-14 13:49:44 -07:00
James R. Barlow 322085933b unpaper: fix check for missing and old versions, add test case 2016-03-10 15:37:09 -08:00
James R. Barlow 6a380ee99c Fix temporary file placed in wrong folder 2016-02-27 00:51:47 -08:00
James R. Barlow dad2198394 Log information about detected page orientations in a summary line 2016-02-26 01:07:59 -08:00
James R. Barlow e40fdc502d Always dump stack trace for unexpected errors 2016-02-26 01:06:59 -08:00
James R. Barlow 71fbda8bf6 Adjust page orientation parsing to deal with change in Tess 3.04.01 2016-02-20 01:32:56 -08:00
James R. Barlow f3b0434a87 Improve ability to capture error messages from tesseract on a crash 2016-02-19 03:48:49 -08:00
James R. Barlow d4ef3411e0 Suppress --pdf-renderer tesseract warning in Docker image
Since the corrected font is provided in the Docker image, there's no
reason to show the warning.
2016-02-17 01:03:20 -08:00
James R. Barlow 60b2eb1455 Fix JPEG DPI: Pillow expects dpi=(x,y) 2016-02-16 07:29:20 -08:00
James R. Barlow 71e493a810 Fix case of JPEG missing DPI field 2016-02-16 05:29:32 -08:00
James R. Barlow c50e3f1329 Complain about older tesseracts that don't have sharp2.ttf installed 2016-02-15 16:43:41 -08:00
James R. Barlow 7c691c21ab Fix image layer rotation for pages with nonzero crop boxes 2016-02-10 17:48:33 -08:00
James R. Barlow 4ec51729d8 Partial fix for images not anchored to (0, 0) 2016-02-10 17:14:48 -08:00
James R. Barlow 6510bcad19 DPI information not transferred automatically from PNG to JPEG 2016-02-09 02:18:54 -08:00
James R. Barlow 1928a64cae Better logging output for autorotation 2016-02-08 23:42:25 -08:00
James R. Barlow 1ba8b1aa4b unpaper is lousy at deskewing, so let leptonica do it 2016-02-08 15:26:33 -08:00
James R. Barlow 2752bda80b Merge branch 'feature/leptdeskew' into feature/logging
Need leptonica for testing now, I think
# Conflicts:
#	ocrmypdf/tesseract.py
#	requirements.txt
#	setup.py
2016-02-08 12:34:48 -08:00
James R. Barlow d30a879e2d Fix test suite by running select_image_for_pdf unconditionally
The purpose of this change that caused the problem was a minor
optimization for the tesseract renderer path that had it pull an image
from select_image_for_pdf so that it could use a JPEG instead of PNG,
instead of taking it from preprocess_clean where it would only get a PNG
and make large files.
2016-02-08 02:33:03 -08:00
James R. Barlow b0114c9174 More logging improvements 2016-02-08 01:31:15 -08:00
James R. Barlow d2ba8c501f Restore invisibletext for normal output 2016-02-08 01:14:39 -08:00
James R. Barlow 6a7ed7d359 Make logging output a lot more useful 2016-02-08 00:58:14 -08:00
James R. Barlow 6289afa1a6 Better: custom logging factory to avoid whatever ruffus is doing 2016-02-08 00:18:52 -08:00
James R. Barlow 9bb6fa04cb Return logging to a semblance of normalcy 2016-02-08 00:09:31 -08:00
James R. Barlow afb6f6f5c9 Render preview as .jpg instead of .png
Smaller file size of JPEG seems to help performance, although the
difference is only about 1%.
2016-02-07 15:49:10 -08:00
James R. Barlow 178aee4687 Make rotation optional (for now it's off, possibly should be on) 2016-02-07 15:43:45 -08:00
James R. Barlow 8484caddfb Tweak pipeline, allowing --pdf-renderer to use JPEGs instead of PNGs 2016-02-07 15:36:51 -08:00
James R. Barlow 08313316de Cleanup auto-rotation 2016-02-07 15:06:54 -08:00
James R. Barlow 1d0eca5c63 All four rotation directions working 2016-02-07 06:09:01 -08:00
James R. Barlow fe89232a30 Fix autorotate for some lossless cases 2016-02-07 05:59:46 -08:00
James R. Barlow 4b51b521e2 Implement autorotate (provided lossless reconstruction is disabled)
Works for a single page file, probably

Although arguably rotation is not quite lossless, and the two could be
mutually exclusive anyway, so maybe this is it. Did not check in some
debugging changes (lossless=False, text debugging=True)

PyPDF seems to get merging wrong when one of the pages is rotated.
2016-02-07 03:27:33 -08:00
James R. Barlow daa3916430 Fix img2pdf 0.2 usage
All tests pass when forced to rely on img2pdf, so seems okay
2016-02-05 15:13:26 -08:00
James R. Barlow e9b87cefcc Try img2pdf 0.2 2016-02-05 14:38:37 -08:00
James R. Barlow b49f5a7d77 Support optionally using leptonica to deskew
unpaper doesn't seem to be good at deskewing. It fails on test case
with a lot of italics. I think it also struggles on pages with a lot
of whitespace. Leptonica continues to shine here.

However, this is only a first crack at Leptonica. The leptonica module
should be redone to use cffi (more extensible).

Also considering the possibility of making all Lept calls in a forked
process to insulate the calling process from C code crashes and the
messy redirect of stdout/stderr to read Leptonica's errors.

I don't think the redirect is a huge problem as long as multiprocesses
rather than multithreads are used. The ruffus child process that is
handling a page is single threaded and will not be affected by the
redirection. It just feels dirty. The main reason to consider a child
process is crash isolation.
2016-01-19 17:43:40 -08:00
James R. Barlow 37c508f3f8 Better versioning: no silly version files, but wrong ver in development
Small price to pay.
2016-01-19 16:07:52 -08:00
James R. Barlow 26e36422cc More fiddling with version 2016-01-19 15:07:21 -08:00