menekis/OCRmyPDF
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,476 Commits 5 Branches 303 Tags
e02f6c1e97c4353834f7c982ec2d79c15b60aef7
Commit Graph
59 Commits
Author SHA1 Message Date
James R. Barlow e02f6c1e97 Support plugin invocation with API 2020-05-02 03:34:31 -07:00
James R. Barlow be107b4fed Set up filter_ocr_image hook 2020-05-01 02:56:41 -07:00
James R. Barlow 8d2535e327 Get pluggy to work with forking workers 2020-05-01 02:39:50 -07:00
James R. Barlow 5eb4fe0052 Refactor plugin setup to get_plugin_manager 2020-05-01 02:18:31 -07:00
James R. Barlow d8ff4485f8 Move samefile to helpers 2020-05-01 02:18:11 -07:00
James R. Barlow 82bce463ae Start pluggy-based plugin system 2020-05-01 02:15:23 -07:00
James R. Barlow 8f5c95f0f4 Remove last vestiges of command line usage of qpdf - change to check_pdf 2020-04-26 05:33:26 -07:00
James R. Barlow 18c4aa10bf Adjust number of workers for concurrent page scanning 2020-04-26 04:21:15 -07:00
James R. Barlow 991db17fde Remove Ghostscript-based text extraction 
While faster than Python based methods, we've outgrown the limited
amount of information Ghostscript provides with this feature, and it
repeats an analysis we have to do anyway to learn what images are
present.
 2020-04-26 04:02:07 -07:00
James R. Barlow 8c381a0227 Replace task_initargs with use of partial() 2020-04-26 03:49:20 -07:00
James R. Barlow af3c3c6466 Further refactoring of concurrency concerns 2020-04-26 03:49:20 -07:00
James R. Barlow db3e75e33e Refactor multiprocessing pool 2020-04-26 03:49:13 -07:00
James R. Barlow c2919f2e1c Reinstate logging of page numbers 2020-04-15 00:05:23 -07:00
James R. Barlow d146d2b65c The Great Logging Refactor 
Remove all instances of logger object being passed as parameters.
This was a holdover from ruffus, and complicated a lot of simple things.
 2020-04-14 23:59:33 -07:00
James R. Barlow 2490be8490 Fix debug.log not being deleted on Windows (probably) 
Fixes #515
 2020-03-29 21:53:56 -07:00
James R. Barlow e4cc9fcba7 Wrong number of threads to use shown when OMP_THREAD_LIMIT is defined 2020-03-23 01:06:55 -07:00
James R. Barlow bcf77375c0 Fix grammar in output message 2020-01-28 07:33:28 -08:00
James R. Barlow ce97af5a79 Add OCR quality measurement API 2020-01-17 03:10:27 -08:00
James R. Barlow 123fde174d Don't use debug.log in pytest 
pytest does not reset the state of logging if we install a file handler,
which will cause FileNotFoundError after the temporary folder is removed.

Semi-related:
https://github.com/pytest-dev/pytest/issues/5502
 2020-01-06 01:46:19 -08:00
James R. Barlow 6f5d77d930 Also generate log file in temp folder on verbose mode 2020-01-05 21:33:32 -08:00
James R. Barlow e2a563cc76 logging: create a debug log when -k parameter is issued 2020-01-01 16:47:15 -08:00
James R. Barlow 2f1c743227 Rewrite main pool loop 
pytest-cov documentation recommends using explicit
management of multiprocessing.Pool rather than the context manager.
This is supposed to work better for collecting coverage data, particularly
on Windows.
 2019-12-31 16:23:41 -08:00
James R. Barlow 63de7e1677 Improve error message for unreadable input files 2019-12-30 16:14:52 -08:00
James R. Barlow c5edff2c2f Sort imports 2019-12-19 15:31:18 -08:00
James R. Barlow b8b7ecfe7f Fix DecompressionBomb related errors due to Windows process differences 2019-12-04 21:10:27 -08:00
James R. Barlow b7f63bc93d Make devnull check compatible with Windows 2019-12-04 17:04:08 -08:00
James R. Barlow 4d26867dee Delinting 2019-09-20 17:17:11 -07:00
James R. Barlow 78e8bf9cbf Use at most 3 Tesseract threads 
Based on a user suggestion and
tesseract-ocr/tesseract#2611, I reviewed thread limits and found that
thread limit of 3 is still beneficial, but not 4.

> time env OMP_THREAD_LIMIT=2 tesseract omp4.png stdout >/dev/null
Warning: Invalid resolution 0 dpi. Using 70 instead.
Estimating resolution as 143
116.67user 1.67system 1:26.26elapsed 137%CPU (0avgtext+0avgdata 356752maxresident)k
2213inputs+0outputs (18major+131059minor)pagefaults 0swaps
> time env OMP_THREAD_LIMIT=3 tesseract omp4.png stdout >/dev/null
Warning: Invalid resolution 0 dpi. Using 70 instead.
Estimating resolution as 143
136.89user 1.63system 1:19.56elapsed 174%CPU (0avgtext+0avgdata 356784maxresident)k
821inputs+0outputs (0major+131080minor)pagefaults 0swaps
> time env OMP_THREAD_LIMIT=4 tesseract omp4.png stdout >/dev/null
Warning: Invalid resolution 0 dpi. Using 70 instead.
Estimating resolution as 143
161.31user 1.51system 1:18.80elapsed 206%CPU (0avgtext+0avgdata 356632maxresident)k
8477inputs+0outputs (12major+131074minor)pagefaults 0swaps
> time env OMP_THREAD_LIMIT=8 tesseract omp4.png stdout >/dev/null
Warning: Invalid resolution 0 dpi. Using 70 instead.
Estimating resolution as 143
160.30user 1.62system 1:18.01elapsed 207%CPU (0avgtext+0avgdata 356640maxresident)k
821inputs+0outputs (0major+131078minor)pagefaults 0swaps
 2019-09-20 17:12:36 -07:00
James R. Barlow eb104b405d Avoid circular imports for __version__ 2019-07-27 05:02:19 -07:00
James R. Barlow 5f00e4f9d8 Sort imports 2019-07-27 04:51:52 -07:00
James R. Barlow 187283192b Fix reporting output file size skipped 
Due to change to using finally for clean up
 2019-06-30 15:08:10 -07:00
James R. Barlow 3331a686fa Fix tess_threads clamped to 1 2019-06-22 00:59:33 -07:00
James R. Barlow 51ed381bfc Rename weave -> graft 2019-06-13 01:16:56 -07:00
James R. Barlow 5ee45411c9 Decide on OMP_THREAD_LIMIT more intelligently 2019-06-13 01:02:07 -07:00
James R. Barlow 066a293462 If verbose, print stacktrace on KeyboardInterrupt 2019-06-12 13:55:43 -07:00
James R. Barlow 81fc95556c Add progress bar for PdfInfo step 2019-06-05 03:08:04 -07:00
James R. Barlow 93f1b73579 Fix --remove-vectors which was broken in API migration 
It got dropped during the change. This feature has also been altered so that
the final visual appearance of the file is not affected, only the OCR image.
 2019-06-05 02:04:45 -07:00
James R. Barlow df9e286e9c Make bypassed exception clearer 2019-06-01 01:35:15 -07:00
James R. Barlow 45a361d112 Add option to use threads instead of processes 
Mainly since they are more convenient for debugging
 2019-05-31 01:56:16 -07:00
James R. Barlow d0efdf643c Cleanup working files when done with a particular file, rather than end of process 2019-05-23 01:25:08 -07:00
James R. Barlow 22298b31be Fix distinction between clean and clean_final lost in API refactor 2019-05-23 01:19:58 -07:00
James R. Barlow a139e64c67 api: short-circuit exception handler, as caller should provide their own 2019-05-22 18:30:30 -07:00
James R. Barlow 09ca1bee97 Add progress bar to optimize and add option to disable it 2019-05-22 15:31:48 -07:00
James R. Barlow 32a076c039 Refactor validation and exceptions 
CLI now tracks check_options exceptions. API now works more like
an API, without an exception handler,
because the caller should provide one.
 2019-05-20 18:01:17 -07:00
James R. Barlow 7ee0c52a57 Refactor cli into basic high level api 2019-05-19 22:34:45 -07:00
James R. Barlow cfd67ab6aa Fixing threading._RLock exception on Python 3.6 
Issue was the usual business: objects that cross process boundaries need
to be picklable and Python 3.6 is more strict about this. The logger
object in particular interfered, so now we suppress it and rebuild it
in process.
 2019-05-17 14:24:47 -07:00
James R. Barlow e528adc603 pylint removal 2019-05-17 01:09:06 -07:00
James R. Barlow 0cb4e854e5 Replace ProcessPoolExecutor with multiprocessing.Pool 
There seems to be no reasonable way to handle Ctrl-C with a
ProcessPoolExecutor. Or at least you have to press it several times to
actually kill. Pool does the job.
 2019-05-17 00:48:06 -07:00
James R. Barlow 13ab23ba54 Refactor weave_layers, introduce progress bar 
Fixes a bug in this branch where --sidecar would fail by trying to iterator
the executor futures twice.
 2019-05-16 14:57:31 -07:00
James R. Barlow 50bd129d7a logging: don't pass log object to validation 2019-05-16 01:58:48 -07:00