menekis/OCRmyPDF
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Remove most Python 3.6 special casing

Browse Source
This commit is contained in:
James R. Barlow
2021-11-13 00:27:48 -08:00
parent 5abfb14c2a
commit 380b981763
13 changed files with 26 additions and 134 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/build.yml
+1 -6
View File
@@ -20,8 +20,6 @@ jobs:
strategy:
matrix:
include:
- os: ubuntu-18.04
python: 3.6
- os: ubuntu-18.04
python: 3.7
- os: ubuntu-20.04
@@ -30,8 +28,6 @@ jobs:
python: 3.9
- os: ubuntu-latest
python: 3.9
- os: ubuntu-20.04
python: "pypy-3.6"
- os: ubuntu-latest
python: "pypy-3.7"
- os: ubuntu-latest
@@ -146,7 +142,6 @@ jobs:
exempi \
ghostscript \
jbig2enc \
leptonica \
openjpeg \
pngquant \
tesseract
@@ -227,7 +222,7 @@ jobs:
- uses: actions/setup-python@v2
name: Install Python
with:
python-version: "3.6"
python-version: "3.7"
- name: Make wheels and sdist
run: |
.pre-commit-config.yaml
+1 -1
View File
@@ -25,7 +25,7 @@ repos:
rev: v2.29.0
hooks:
- id: pyupgrade
args: ["--py36-plus"]
args: ["--py37-plus"]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.910-1
hooks:
README.md
+1 -1
View File
@@ -110,7 +110,7 @@ Please report issues on our [GitHub issues](https://github.com/jbarlow83/OCRmyPD
## Requirements
In addition to the required Python version (3.6+), OCRmyPDF requires external program installations of Ghostscript and Tesseract OCR. OCRmyPDF is pure Python, and runs on pretty much everything: Linux, macOS, Windows and FreeBSD.
In addition to the required Python version (3.7+), OCRmyPDF requires external program installations of Ghostscript and Tesseract OCR. OCRmyPDF is pure Python, and runs on pretty much everything: Linux, macOS, Windows and FreeBSD.
## Press & Media
docs/installation.rst
+12 -14
View File
@@ -217,8 +217,8 @@ Ubuntu 16.04 LTS
----------------
No package is available for Ubuntu 16.04. OCRmyPDF 8.0 and newer require
Python 3.6. Ubuntu 16.04 ships Python 3.5, but you can install Python
3.6 on it. Or, you can skip Python 3.6 and install OCRmyPDF 7.x or older
Python 3.7. Ubuntu 16.04 ships Python 3.5, but you can install Python
3.7 on it. Or, you can skip Python 3.7 and install OCRmyPDF 7.x or older
- for that procedure, please see the installation documentation for the
version of OCRmyPDF you plan to use.
@@ -229,7 +229,7 @@ version of OCRmyPDF you plan to use.
sudo apt-get update
sudo apt-get install -y software-properties-common python-software-properties
sudo add-apt-repository -y \
ppa:jonathonf/python-3.6 \
ppa:jonathonf/python-3.7 \
ppa:alex-p/tesseract-ocr
sudo apt-get update
sudo apt-get install -y \
@@ -237,22 +237,22 @@ version of OCRmyPDF you plan to use.
libexempi3 \
libffi6 \
pngquant \
python3.6 \
python3.7 \
qpdf \
tesseract-ocr \
unpaper
This will install a Python 3.6 binary at ``/usr/bin/python3.6``
This will install a Python 3.7 binary at ``/usr/bin/python3.7``
alongside the system's Python 3.5. Do not remove the system Python. This
will also install Tesseract 4.0 from a PPA, since the version available
in Ubuntu 16.04 is too old for OCRmyPDF.
Now install pip for Python 3.6. This will install the Python 3.6 version
Now install pip for Python 3.7. This will install the Python 3.7 version
of ``pip`` at ``/usr/local/bin/pip``.
.. code-block:: bash
curl https://bootstrap.pypa.io/get-pip.py | sudo python3.6
curl https://bootstrap.pypa.io/get-pip.py | sudo python3.7
**Install OCRmyPDF**
@@ -272,7 +272,7 @@ environment variable contains ``$HOME/.local/bin``.
.. code-block:: bash
export PATH=$HOME/.local/bin:$PATH
pip3.6 install --user ocrmypdf
pip3.7 install --user ocrmypdf
To add JBIG2 encoding, see :ref:`jbig2`.
@@ -572,7 +572,7 @@ Cygwin64
First install the the following prerequisite Cygwin packages using ``setup-x86_64.exe``::
python36 (or later)
python37 (or later)
python3?-devel
python3?-pip
python3?-lxml
@@ -708,16 +708,14 @@ OCRmyPDF currently requires these external programs and libraries to be
installed, and must be satisfied using the operating system package
manager. ``pip`` cannot provide them.
- Python 3.6 or newer
- Python 3.7 or newer
- Ghostscript 9.15 or newer
- qpdf 8.1.0 or newer
- Tesseract 4.0.0-beta or newer
As of ocrmypdf 7.2.1, the following versions are recommended:
- Python 3.7 or 3.8
- Python 3.9 or newer
- Ghostscript 9.23 or newer
- qpdf 8.2.1
- Tesseract 4.0.0 or newer
- jbig2enc 0.29 or newer
- pngquant 2.5 or newer
@@ -749,7 +747,7 @@ unfortunately, the ``pip install`` command cannot satisfy all of them.
Installing HEAD revision from sources
=====================================
If you have ``git`` and Python 3.6 or newer installed, you can install
If you have ``git`` and Python 3.7 or newer installed, you can install
from source. When the ``pip`` installer runs, it will alert you if
dependencies are missing.
docs/release_notes.rst
+2 -3
View File
@@ -14,9 +14,8 @@ wish to use some of its features for working with PDFs.
.. note::
Python 3.6 reaches end of life on December 23, 2021. We will end support
for Python 3.6 around that time. The change will be marked with a major
release.
Python 3.6 reaches end of life on December 23, 2021. We will continue to support
OCRmyPDF v12.x if major issues are identified until that date.
v12.7.2
=======
setup.cfg
+3 -5
View File
@@ -24,7 +24,6 @@ classifiers =
Operating System :: POSIX :: Linux
Programming Language :: Python :: 3
Programming Language :: Python :: 3 :: Only
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
@@ -50,14 +49,13 @@ install_requires =
coloredlogs>=14.0 # strictly optional
img2pdf>=0.3.0,<0.5 # pure Python
pdfminer.six!=20200720,>=20191110,<=20211012
pikepdf>=2.10.0
pikepdf>=4.0.0
pluggy>=0.13.0,<2
reportlab>=3.5.66
tqdm>=4
importlib-metadata>=4;python_version<'3.8' # until Python 3.8
importlib-resources>=5;python_version<'3.9' # until Python 3.9
pikepdf<3;implementation_name=="pypy" and python_version=='3.6'
python_requires = >=3.6
python_requires = >=3.7
include_package_data = True
package_dir =
=src
@@ -98,7 +96,7 @@ ocrmypdf =
py.typed
[bdist_wheel]
python-tag = py36
python-tag = py37
[aliases]
test = pytest
src/ocrmypdf/__main__.py
+1 -4
View File
@@ -15,7 +15,7 @@ from multiprocessing import set_start_method
from ocrmypdf import __version__
from ocrmypdf._plugin_manager import get_parser_options_plugins
from ocrmypdf._sync import run_pipeline
from ocrmypdf._validation import check_closed_streams, check_options
from ocrmypdf._validation import check_options
from ocrmypdf.api import Verbosity, configure_logging
from ocrmypdf.exceptions import (
BadArgsError,
@@ -34,9 +34,6 @@ def sigbus(*args):
def run(args=None):
_parser, options, plugin_manager = get_parser_options_plugins(args=args)
if not check_closed_streams(options):
return ExitCode.bad_args
if hasattr(os, 'nice'):
os.nice(5)
src/ocrmypdf/_logging.py
+1 -9
View File
@@ -28,22 +28,14 @@ class TqdmConsole:
This routes log messages through tqdm so that it can print them above the
progress bar, and then refresh the progress bar, rather than overwriting
it which looks messy.
For some reason Python 3.6 prints extra empty messages from time to time,
so we suppress those.
"""
def __init__(self, file):
self.file = file
self.py36 = sys.version_info[0:2] == (3, 6)
def write(self, msg):
# When no progress bar is active, tqdm.write() routes to print()
if self.py36:
if msg.strip() != '':
tqdm.write(msg.rstrip(), end='\n', file=self.file)
else:
tqdm.write(msg.rstrip(), end='\n', file=self.file)
tqdm.write(msg.rstrip(), end='\n', file=self.file)
def flush(self):
with suppress(AttributeError):
src/ocrmypdf/_validation.py
-48
View File
@@ -275,54 +275,6 @@ def check_options(options, plugin_manager):
_check_options(options, plugin_manager, ocr_engine_languages)
def check_closed_streams(options): # pragma: no cover
"""Work around Python issue with multiprocessing forking on closed streams
https://bugs.python.org/issue28326
Attempting to a fork/exec a new Python process when any of std{in,out,err}
are closed or not flushable for some reason may raise an exception.
Fix this by opening devnull if the handle seems to be closed. Do this
globally to avoid tracking all places that fork.
Seems to be specific to multiprocessing.Process not all Python process
forkers.
The error actually occurs when the stream object is not flushable,
but replacing an open stream object that is not flushable with
/dev/null is a bad idea since it will create a silent failure. Replacing
a closed handle with /dev/null seems safe.
"""
if sys.version_info[0:3] >= (3, 6, 4):
return True # Issued fixed in Python 3.6.4+
if sys.stderr is None:
sys.stderr = open(os.devnull, 'w')
if sys.stdin is None:
if options.input_file == '-':
log.error("Trying to read from stdin but stdin seems closed")
return False
sys.stdin = open(os.devnull)
if sys.stdout is None:
if options.output_file == '-':
# Can't replace stdout if the user is piping
# If this case can even happen, it must be some kind of weird
# stream.
log.error(
"Output was set to stdout '-' but the stream attached to "
"stdout does not support the flush() system call. This "
"will fail."
)
return False
sys.stdout = open(os.devnull, 'w')
return True
def create_input_file(options, work_folder: Path) -> Tuple[Path, str]:
if options.input_file == '-':
# stdin
src/ocrmypdf/subprocess/__init__.py
+1 -9
View File
@@ -117,15 +117,7 @@ def _fix_process_args(args, env, kwargs):
log.debug("Running: %s", args)
process_log = log.getChild(os.path.basename(program))
text = kwargs.get('text', False)
if sys.version_info < (3, 7):
if os.name == 'nt':
# Can't use close_fds=True on Windows with Python 3.6 or older
# https://bugs.python.org/issue19575, etc.
kwargs['close_fds'] = False
if 'text' in kwargs:
# Convert run(...text=) to run(...universal_newlines=) for Python 3.6
kwargs['universal_newlines'] = kwargs['text']
del kwargs['text']
return args, env, process_log, text
tests/conftest.py
+1 -1
View File
@@ -129,7 +129,7 @@ def run_ocrmypdf(input_file, output_file, *args, text=True):
p_args,
stdout=PIPE,
stderr=PIPE,
universal_newlines=text, # When dropping support for Python 3.6 change to text=
text=text,
env=env,
check=False,
)
tests/test_main.py
+2 -2
View File
@@ -598,7 +598,7 @@ def test_compression_preserved(ocrmypdf_exec, resources, image, outpdf):
stdout=PIPE,
stderr=PIPE,
stdin=input_stream,
universal_newlines=True, # When dropping support for Python 3.6 change to text=
text=True,
check=False,
)
@@ -659,7 +659,7 @@ def test_compression_changed(ocrmypdf_exec, resources, image, compression, outpd
stdout=PIPE,
stderr=PIPE,
stdin=input_stream,
universal_newlines=True, # When dropping support for Python 3.6 change to text=
text=True,
check=False,
)
assert p.returncode == ExitCode.ok, p.stderr
tests/test_stdio.py
-31
View File
@@ -53,37 +53,6 @@ def test_stdout(ocrmypdf_exec, resources, outpdf):
assert check_pdf(output_file)
@pytest.mark.skipif(
sys.version_info[0:3] >= (3, 6, 4), reason="issue fixed in Python 3.6.4"
)
@pytest.mark.skipif(os.name == 'nt', reason="POSIX problem")
def test_closed_streams(ocrmypdf_exec, resources, outpdf):
input_file = str(resources / 'francais.pdf')
output_file = str(outpdf)
def evil_closer():
os.close(0)
os.close(1)
p_args = ocrmypdf_exec + [
input_file,
output_file,
'--plugin',
'tests/plugins/tesseract_noop.py',
]
p = Popen( # pylint: disable=subprocess-popen-preexec-fn
p_args,
close_fds=True,
stdout=None,
stderr=PIPE,
stdin=None,
preexec_fn=evil_closer,
)
_out, err = p.communicate()
print(err.decode())
assert p.returncode == ExitCode.ok
@pytest.mark.skipif(sys.version_info >= (3, 7, 0), reason='better utf-8')
@pytest.mark.skipif(
Path('/etc/alpine-release').exists(), reason="invalid test on alpine"