James R. Barlow
32a076c039
Refactor validation and exceptions
...
CLI now tracks check_options exceptions. API now works more like
an API, without an exception handler,
because the caller should provide one.
2019-05-20 18:01:17 -07:00
James R. Barlow
7ee0c52a57
Refactor cli into basic high level api
2019-05-19 22:34:45 -07:00
James R. Barlow
cfd67ab6aa
Fixing threading._RLock exception on Python 3.6
...
Issue was the usual business: objects that cross process boundaries need
to be picklable and Python 3.6 is more strict about this. The logger
object in particular interfered, so now we suppress it and rebuild it
in process.
2019-05-17 14:24:47 -07:00
James R. Barlow
e528adc603
pylint removal
2019-05-17 01:09:06 -07:00
James R. Barlow
0cb4e854e5
Replace ProcessPoolExecutor with multiprocessing.Pool
...
There seems to be no reasonable way to handle Ctrl-C with a
ProcessPoolExecutor. Or at least you have to press it several times to
actually kill. Pool does the job.
2019-05-17 00:48:06 -07:00
James R. Barlow
13ab23ba54
Refactor weave_layers, introduce progress bar
...
Fixes a bug in this branch where --sidecar would fail by trying to iterator
the executor futures twice.
2019-05-16 14:57:31 -07:00
James R. Barlow
50bd129d7a
logging: don't pass log object to validation
2019-05-16 01:58:48 -07:00
James R. Barlow
471cdea232
Move app specific settings a library may not want to __main__
2019-05-16 01:29:26 -07:00
James R. Barlow
f4b87915df
Fix --redo-ocr
2019-04-15 13:11:26 -07:00
mawi
c92ccc6134
fix: tests
2019-04-08 14:57:42 +02:00
mawi
6590875756
feat: add triage step
...
remove tqdm demo
2019-04-08 10:26:56 +02:00
mawi
01bbf064e0
feat: add tqdm progress bar
...
This is just a POC. Will be removed.
2019-04-05 19:52:38 +02:00
mawi
fc1c4f12f5
feat: add concurrent.futures pipeline
2019-04-05 18:48:34 +02:00
mawi
783a128bd1
feat: move to sync (none ETL) implementation - remove ruffus
2019-04-04 21:02:38 +02:00
Martin Wind
b214aa5b38
feat: move to sync (none ETL) implementation
2019-04-03 19:59:43 +02:00
Martin Wind
aa512b6181
feat: move to sync (none ETL) implementation (WIP)
2019-04-02 20:03:09 +02:00
James R. Barlow
mawi