I found this issue in ruffus 2.6.3
https://github.com/bunbun/ruffus/issues/65
also discussed here
https://github.com/bunbun/ruffus/pull/67
ruffus 2.6.3 RethrownJobError don't follow the normal conventions and
so its exception causes problems when they cross process boundaries.
This change carefully examines the various forms of ruffus exception
objects that can appear in 2.6.3 and parses them more carefully. It
also removes any direct posting of the exception to the logger because
this triggers another serializing of the exception object, mutating it
further.
The purpose of this change that caused the problem was a minor
optimization for the tesseract renderer path that had it pull an image
from select_image_for_pdf so that it could use a JPEG instead of PNG,
instead of taking it from preprocess_clean where it would only get a PNG
and make large files.
Works for a single page file, probably
Although arguably rotation is not quite lossless, and the two could be
mutually exclusive anyway, so maybe this is it. Did not check in some
debugging changes (lossless=False, text debugging=True)
PyPDF seems to get merging wrong when one of the pages is rotated.
unpaper doesn't seem to be good at deskewing. It fails on test case
with a lot of italics. I think it also struggles on pages with a lot
of whitespace. Leptonica continues to shine here.
However, this is only a first crack at Leptonica. The leptonica module
should be redone to use cffi (more extensible).
Also considering the possibility of making all Lept calls in a forked
process to insulate the calling process from C code crashes and the
messy redirect of stdout/stderr to read Leptonica's errors.
I don't think the redirect is a huge problem as long as multiprocesses
rather than multithreads are used. The ruffus child process that is
handling a page is single threaded and will not be affected by the
redirection. It just feels dirty. The main reason to consider a child
process is crash isolation.
Ruffus treats omitted parameter as -j1. For our purposes it makes more
sense for omitting the parameter to mean "use all CPUs". As such we
must be able to distinguish -j1 from the parameter -j being omitted.
Telling ruffus to ignore the argument actually just makes it not auto
generate the argument. We can add an argument back with the same name.
James R. Barlow