Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
204c9d6ae1 | ||
|
|
6eb393590b | ||
|
|
07c6654057 | ||
|
|
4e15eb8d14 | ||
|
|
8b01ab8ad2 | ||
|
|
e0a522ad50 | ||
|
|
a1a8788c5a | ||
|
|
cccdc178c3 | ||
|
|
4eacb3454f | ||
|
|
82b8b41e80 | ||
|
|
581c5020ab | ||
|
|
3ef8872a1e | ||
|
|
28eec73eed | ||
|
|
bfe4a5b329 | ||
|
|
29097837d6 | ||
|
|
a40361db3c | ||
|
|
8b29e3cbab | ||
|
|
b170be120b | ||
|
|
9a6cd95e5f | ||
|
|
d464d3122e | ||
|
|
1327ab37d4 | ||
|
|
67553fc5c6 | ||
|
|
306a903854 | ||
|
|
b93cf51c0f | ||
|
|
6b994221c6 | ||
|
|
8b5b02e0d8 | ||
|
|
624df9bb23 |
+3
-13
@@ -149,12 +149,6 @@ stages:
|
||||
- job: "macOS_Mojave"
|
||||
pool:
|
||||
vmImage: "macos-10.14"
|
||||
strategy:
|
||||
matrix:
|
||||
Python37:
|
||||
python.version: ""
|
||||
Python38:
|
||||
python.version: "python@3.8"
|
||||
steps:
|
||||
# https://github.com/actions/virtual-environments/issues/664
|
||||
# - task: UsePythonVersion@0
|
||||
@@ -163,11 +157,8 @@ stages:
|
||||
- bash: |
|
||||
brew update
|
||||
brew unlink python@2
|
||||
if [ "$(python.version)" != "" ]; then
|
||||
brew upgrade $(python.version)
|
||||
else
|
||||
echo "Using Python `python3 --version`"
|
||||
fi
|
||||
brew upgrade python
|
||||
echo "Using Python `python3 --version`"
|
||||
displayName: "Update brew and Python"
|
||||
- bash: |
|
||||
brew install \
|
||||
@@ -177,8 +168,7 @@ stages:
|
||||
leptonica \
|
||||
openjpeg \
|
||||
pngquant \
|
||||
tesseract \
|
||||
unpaper
|
||||
tesseract
|
||||
displayName: "Install system packages"
|
||||
- bash: |
|
||||
pip3 install --upgrade pip
|
||||
|
||||
@@ -190,7 +190,8 @@ of ocrmypdf, and install the following dependencies:
|
||||
python3-reportlab \
|
||||
qpdf \
|
||||
tesseract-ocr \
|
||||
zlib1g
|
||||
zlib1g \
|
||||
unpaper
|
||||
|
||||
We will need a newer version of ``pip`` then was available for Ubuntu 18.04:
|
||||
|
||||
|
||||
@@ -12,6 +12,51 @@ may be unreliable. Use the API to depend on precise behavior.
|
||||
The public API may be useful in scripts that launch OCRmyPDF processes or that
|
||||
wish to use some of its features for working with PDFs.
|
||||
|
||||
v11.2.1
|
||||
=======
|
||||
|
||||
- Fixed an issue where optimization of a 1-bit image with a color palette or
|
||||
associated ICC that was optimized to JBIG2 could have its colors inverted.
|
||||
|
||||
v11.2.0
|
||||
=======
|
||||
|
||||
- Fixed an issue with optimizing PNG-type images that had soft masks or image masks.
|
||||
This is a regression introduced in (or about) v11.1.0.
|
||||
- Improved type checking of the ``plugins`` parameter for the ``ocrmypdf.ocr``
|
||||
API call.
|
||||
|
||||
v11.1.2
|
||||
=======
|
||||
|
||||
- Fixed hOCR renderer writing the text in roughly reverse order. This should not
|
||||
affect reasonably smart PDF readers that properly locate the position of all
|
||||
text, but may confuse those that rely on the order of objects in the content
|
||||
stream. (#642)
|
||||
|
||||
v11.1.1
|
||||
=======
|
||||
|
||||
- We now avoid using named temporary files when using pngquant allowing containerized
|
||||
pngquant installs to be used.
|
||||
- Clarified an error message.
|
||||
- Highest number of 1's in a release ever!
|
||||
|
||||
v11.1.0
|
||||
=======
|
||||
|
||||
- Fixed page rotation issues: #634, #589.
|
||||
- Fixed some cases where optimization created an invalid image such as a
|
||||
1-bit "RGB" image: #629, #620.
|
||||
- Page numbers are now displayed in debug logs when pages are being grafted.
|
||||
- ocrmypdf.optimize.rewrite_png and ocrmypdf.optimize.rewrite_png_as_g4 were
|
||||
marked deprecated. Strictly speaking these should have been internal APIs,
|
||||
but they were never hidden.
|
||||
- As a precaution, pikepdf mmap-based file access has been disabled due to a
|
||||
rare race condition that causes a crash when certain objects are deallocated.
|
||||
The problem is likely in pikepdf's dependency pybind11.
|
||||
- Extended the example plugin to demonstrate conversion to mono.
|
||||
|
||||
v11.0.2
|
||||
=======
|
||||
|
||||
|
||||
+30
-4
@@ -18,6 +18,25 @@
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
|
||||
"""
|
||||
An example of an OCRmyPDF plugin.
|
||||
|
||||
This plugin adds two new command line arguments
|
||||
--grayscale-ocr: converts the image to grayscale before performing OCR on it
|
||||
(This is occasionally useful for images whose color confounds OCR. It only
|
||||
affects the image shown to OCR. The image is not saved.)
|
||||
--mono-page: converts pages all pages in the output file to black and white
|
||||
|
||||
To use this from the command line:
|
||||
ocrmypdf --plugin path/to/example_plugin.py --mono-page input.pdf output.pdf
|
||||
|
||||
To use this as an API:
|
||||
import ocrmypdf
|
||||
ocrmypdf.ocr('input.pdf', 'output.pdf',
|
||||
plugins=['path/to/example_plugin.py'], mono_page=True
|
||||
)
|
||||
"""
|
||||
|
||||
import logging
|
||||
|
||||
from PIL import Image
|
||||
@@ -30,6 +49,7 @@ log = logging.getLogger(__name__)
|
||||
@hookimpl
|
||||
def add_options(parser):
|
||||
parser.add_argument('--grayscale-ocr', action='store_true')
|
||||
parser.add_argument('--mono-page', action='store_true')
|
||||
|
||||
|
||||
@hookimpl
|
||||
@@ -52,7 +72,13 @@ def filter_ocr_image(page, image):
|
||||
|
||||
@hookimpl
|
||||
def filter_page_image(page, image_filename):
|
||||
output = image_filename.with_suffix('.jpg')
|
||||
with Image.open(image_filename) as im:
|
||||
im.save(output)
|
||||
return output
|
||||
if page.options.mono_page:
|
||||
with Image.open(image_filename) as im:
|
||||
im = im.convert('1')
|
||||
im.save(image_filename)
|
||||
return image_filename
|
||||
else:
|
||||
output = image_filename.with_suffix('.jpg')
|
||||
with Image.open(image_filename) as im:
|
||||
im.save(output)
|
||||
return output
|
||||
|
||||
@@ -7,7 +7,11 @@
|
||||
|
||||
"""Interface to pngquant executable"""
|
||||
|
||||
from contextlib import contextmanager
|
||||
from io import BytesIO
|
||||
from os import fspath
|
||||
from pathlib import Path
|
||||
from subprocess import PIPE
|
||||
from tempfile import NamedTemporaryFile
|
||||
|
||||
from PIL import Image
|
||||
@@ -28,34 +32,32 @@ def available():
|
||||
return True
|
||||
|
||||
|
||||
def quantize(input_file, output_file, quality_min, quality_max):
|
||||
input_file = fspath(input_file)
|
||||
output_file = fspath(output_file)
|
||||
if input_file.endswith('.jpg'):
|
||||
with Image.open(input_file) as im, NamedTemporaryFile(suffix='.png') as tmp:
|
||||
im.save(tmp)
|
||||
args = [
|
||||
'pngquant',
|
||||
'--force',
|
||||
'--skip-if-larger',
|
||||
'--output',
|
||||
output_file,
|
||||
'--quality',
|
||||
f'{quality_min}-{quality_max}',
|
||||
'--',
|
||||
tmp.name,
|
||||
]
|
||||
run(args)
|
||||
@contextmanager
|
||||
def input_as_png(input_file: Path):
|
||||
if not input_file.name.endswith('.png'):
|
||||
with Image.open(input_file) as im:
|
||||
bio = BytesIO()
|
||||
im.save(bio, format='png')
|
||||
bio.seek(0)
|
||||
yield bio
|
||||
else:
|
||||
with open(input_file, 'rb') as f:
|
||||
yield f
|
||||
|
||||
|
||||
def quantize(input_file: Path, output_file: Path, quality_min: int, quality_max: int):
|
||||
with input_as_png(input_file) as input_stream:
|
||||
args = [
|
||||
'pngquant',
|
||||
'--force',
|
||||
'--skip-if-larger',
|
||||
'--output',
|
||||
output_file,
|
||||
'--quality',
|
||||
f'{quality_min}-{quality_max}',
|
||||
'--',
|
||||
input_file,
|
||||
'--', # pngquant: stop processing arguments
|
||||
'-', # pngquant: stream input and output
|
||||
]
|
||||
run(args)
|
||||
result = run(args, stdin=input_stream, stdout=PIPE, stderr=PIPE, check=False)
|
||||
|
||||
if result.returncode == 0:
|
||||
# input_file could be the same as output_file, so we defer the write
|
||||
output_file.write_bytes(result.stdout)
|
||||
|
||||
@@ -107,7 +107,7 @@ def run(input_file, output_file, dpi, mode_args):
|
||||
|
||||
def validate_custom_args(args: str):
|
||||
unpaper_args = shlex.split(args)
|
||||
if any('/' in arg for arg in unpaper_args):
|
||||
if any(('/' in arg or arg == '.' or arg == '..') for arg in unpaper_args):
|
||||
raise ValueError('No filenames allowed in --unpaper-args')
|
||||
return unpaper_args
|
||||
|
||||
|
||||
+26
-20
@@ -109,7 +109,6 @@ class OcrGrafter:
|
||||
if textpdf and not self.font:
|
||||
self.font, self.font_key = self._find_font(textpdf)
|
||||
|
||||
emplaced_page = False
|
||||
content_rotation = self.pdfinfo[pageno].rotation
|
||||
path_image = Path(image).resolve() if image else None
|
||||
if path_image is not None and path_image != self.path_base:
|
||||
@@ -123,17 +122,21 @@ class OcrGrafter:
|
||||
local_image_page = self.pdf_base.pages[-1]
|
||||
self.pdf_base.pages[pageno].emplace(local_image_page)
|
||||
del self.pdf_base.pages[-1]
|
||||
emplaced_page = True
|
||||
# The pdf_image_page will always be created with any /Rotate applied
|
||||
# applied already
|
||||
content_rotation = 0
|
||||
|
||||
if emplaced_page:
|
||||
content_rotation = autorotate_correction
|
||||
text_rotation = autorotate_correction
|
||||
text_misaligned = (text_rotation - content_rotation) % 360
|
||||
log.debug(
|
||||
f"Rotations for page {pageno}: [text, auto, misalign, content] = "
|
||||
f"{text_rotation}, {autorotate_correction}, "
|
||||
f"{text_misaligned}, {content_rotation}"
|
||||
)
|
||||
if content_rotation != 0:
|
||||
# Text can be misaligned on a /Rotate'd page.
|
||||
# That is because we rasterize pages with /Rotate applied,
|
||||
# so that the OCR image text is upright and comes back upright.
|
||||
text_misaligned = (autorotate_correction - content_rotation) % 360
|
||||
log.debug(
|
||||
f"Text rotation: (autorotate, content) -> text misalignment = "
|
||||
f"({autorotate_correction}, {content_rotation}) -> {text_misaligned}"
|
||||
)
|
||||
else:
|
||||
text_misaligned = 0
|
||||
|
||||
if textpdf and self.font:
|
||||
# Graft the text layer onto this page, whether new or old
|
||||
@@ -143,15 +146,18 @@ class OcrGrafter:
|
||||
textpdf=textpdf,
|
||||
font=self.font,
|
||||
font_key=self.font_key,
|
||||
rotation=text_misaligned,
|
||||
text_rotation=text_misaligned,
|
||||
procset=self.procset,
|
||||
strip_old_text=strip_old,
|
||||
)
|
||||
|
||||
# Correct the rotation if applicable
|
||||
self.pdf_base.pages[pageno].Rotate = (
|
||||
content_rotation - autorotate_correction
|
||||
) % 360
|
||||
# Correct the page rotation
|
||||
page_rotation = (content_rotation - autorotate_correction) % 360
|
||||
self.pdf_base.pages[pageno].Rotate = page_rotation
|
||||
log.debug(
|
||||
f"Page rotation: (content, auto) -> page = "
|
||||
f"({content_rotation}, {autorotate_correction}) -> {page_rotation}"
|
||||
)
|
||||
|
||||
if self.emplacements % MAX_REPLACE_PAGES == 0:
|
||||
self.save_and_reload()
|
||||
@@ -226,7 +232,7 @@ class OcrGrafter:
|
||||
font: pikepdf.Object,
|
||||
font_key: pikepdf.Object,
|
||||
procset: pikepdf.Object,
|
||||
rotation: int,
|
||||
text_rotation: int,
|
||||
strip_old_text: bool,
|
||||
):
|
||||
"""Insert the text layer from text page 0 on to pdf_base at page_num"""
|
||||
@@ -256,13 +262,13 @@ class OcrGrafter:
|
||||
corner = pikepdf.PdfMatrix().translated(mediabox[0], mediabox[1])
|
||||
# -rotation because the input is a clockwise angle and this formula
|
||||
# uses CCW
|
||||
rotation = -rotation % 360
|
||||
rotate = pikepdf.PdfMatrix().rotated(rotation)
|
||||
text_rotation = -text_rotation % 360
|
||||
rotate = pikepdf.PdfMatrix().rotated(text_rotation)
|
||||
|
||||
# Because of rounding of DPI, we might get a text layer that is not
|
||||
# identically sized to the target page. Scale to adjust. Normally this
|
||||
# is within 0.998.
|
||||
if rotation in (90, 270):
|
||||
if text_rotation in (90, 270):
|
||||
wt, ht = ht, wt
|
||||
scale_x = wp / wt
|
||||
scale_y = hp / ht
|
||||
|
||||
+13
-9
@@ -252,15 +252,19 @@ def exec_concurrent(context: PdfContext):
|
||||
ocrgraft = OcrGrafter(context)
|
||||
|
||||
def update_page(result: PageResult, pbar):
|
||||
sidecars[result.pageno] = result.text
|
||||
pbar.update()
|
||||
ocrgraft.graft_page(
|
||||
pageno=result.pageno,
|
||||
image=result.pdf_page_from_image,
|
||||
textpdf=result.ocr,
|
||||
autorotate_correction=result.orientation_correction,
|
||||
)
|
||||
pbar.update()
|
||||
try:
|
||||
tls.pageno = result.pageno + 1
|
||||
sidecars[result.pageno] = result.text
|
||||
pbar.update()
|
||||
ocrgraft.graft_page(
|
||||
pageno=result.pageno,
|
||||
image=result.pdf_page_from_image,
|
||||
textpdf=result.ocr,
|
||||
autorotate_correction=result.orientation_correction,
|
||||
)
|
||||
pbar.update()
|
||||
finally:
|
||||
tls.pageno = None
|
||||
|
||||
exec_progress_pool(
|
||||
use_threads=context.options.use_threads,
|
||||
|
||||
@@ -308,17 +308,6 @@ def check_closed_streams(options): # pragma: no cover
|
||||
return True
|
||||
|
||||
|
||||
def log_page_orientations(pdfinfo):
|
||||
direction = {0: 'n', 90: 'e', 180: 's', 270: 'w'}
|
||||
orientations = []
|
||||
for n, page in enumerate(pdfinfo):
|
||||
angle = page.rotation or 0
|
||||
if angle != 0:
|
||||
orientations.append('{0}{1}'.format(n + 1, direction.get(angle, '')))
|
||||
if orientations:
|
||||
log.info('Page orientations detected: %s', ' '.join(orientations))
|
||||
|
||||
|
||||
def create_input_file(options, work_folder: Path) -> Tuple[Path, str]:
|
||||
if options.input_file == '-':
|
||||
# stdin
|
||||
|
||||
+3
-1
@@ -226,7 +226,7 @@ def ocr( # pylint: disable=unused-argument
|
||||
user_words: os.PathLike = None,
|
||||
user_patterns: os.PathLike = None,
|
||||
fast_web_view: float = None,
|
||||
plugins: Iterable[str] = None,
|
||||
plugins: Iterable[Union[str, Path]] = None,
|
||||
keep_temporary_files: bool = None,
|
||||
progress_bar: bool = None,
|
||||
**kwargs,
|
||||
@@ -280,6 +280,8 @@ def ocr( # pylint: disable=unused-argument
|
||||
"""
|
||||
if not plugins:
|
||||
plugins = []
|
||||
elif isinstance(plugins, (str, Path)):
|
||||
plugins = [plugins]
|
||||
else:
|
||||
plugins = list(plugins)
|
||||
|
||||
|
||||
@@ -39,13 +39,11 @@ def check_options(options):
|
||||
if gs_version < '9.20' and options.output_type != 'pdf' and not is_latin:
|
||||
# https://bugs.ghostscript.com/show_bug.cgi?id=696874
|
||||
# Ghostscript < 9.20 fails to encode multibyte characters properly
|
||||
msg = (
|
||||
"The installed version of Ghostscript does not work correctly "
|
||||
"with the OCR languages you specified. Use --output-type pdf or "
|
||||
log.warning(
|
||||
f"The installed version of Ghostscript ({gs_version}) does not work "
|
||||
"correctly with the OCR languages you specified. Use --output-type pdf or "
|
||||
"upgrade to Ghostscript 9.20 or later to avoid this issue."
|
||||
)
|
||||
msg += f"Found Ghostscript {gs_version}"
|
||||
log.warning(msg)
|
||||
|
||||
if options.output_type == 'pdfa':
|
||||
options.output_type = 'pdfa-2'
|
||||
|
||||
@@ -213,11 +213,15 @@ def clamp(n: T, smallest: T, largest: T) -> T:
|
||||
|
||||
|
||||
def pikepdf_enable_mmap():
|
||||
try:
|
||||
if pikepdf._qpdf.set_access_default_mmap(True):
|
||||
log.debug("pikepdf mmap enabled")
|
||||
except AttributeError:
|
||||
log.debug("pikepdf mmap not available")
|
||||
# try:
|
||||
# if pikepdf._qpdf.set_access_default_mmap(True):
|
||||
# log.debug("pikepdf mmap enabled")
|
||||
# except AttributeError:
|
||||
# log.debug("pikepdf mmap not available")
|
||||
# We found a race condition probably related to pybind issue #2252 that can
|
||||
# cause a crash. For now, disable pikepdf mmap to be on the safe side.
|
||||
log.debug("pikepdf mmap disabled")
|
||||
return
|
||||
|
||||
|
||||
def deprecated(func):
|
||||
|
||||
@@ -167,7 +167,10 @@ class HocrTransform:
|
||||
def topdown_position(self, element):
|
||||
pxl_line_coords = self.element_coordinates(element)
|
||||
line_box = self.pt_from_pixel(pxl_line_coords)
|
||||
return -line_box.y2
|
||||
# Coordinates here are still in the hocr coordinate system, so 0 on the y axis
|
||||
# is the top of the page and increasing values of y will move towards the
|
||||
# bottom of the page.
|
||||
return line_box.y2
|
||||
|
||||
def to_pdf(
|
||||
self,
|
||||
|
||||
@@ -58,6 +58,24 @@ if not _libpath:
|
||||
---------------------------------------------------------------------
|
||||
"""
|
||||
)
|
||||
if os.name == 'nt':
|
||||
# On Windows, recent versions of libpng require zlib. We have to make sure
|
||||
# the zlib version being loaded is the same one that libpng was built with.
|
||||
# This tries to import zlib from Tesseract's installation folder, falling back
|
||||
# to find_library() if liblept is being loaded from somewhere else.
|
||||
# Loading zlib from other places could cause a version mismatch
|
||||
_zlib_path = os.path.join(os.path.dirname(_libpath), 'zlib1.dll')
|
||||
if not os.path.exists(_zlib_path):
|
||||
_zlib_path = find_library('zlib')
|
||||
try:
|
||||
zlib = ffi.dlopen(_zlib_path)
|
||||
except ffi.error as e:
|
||||
raise MissingDependencyError(
|
||||
"""
|
||||
Could not load the zlib library. It could be that Tesseract is not installed properly,
|
||||
we can't find the installation on your system PATH environment variable.
|
||||
"""
|
||||
) from e
|
||||
try:
|
||||
lept = ffi.dlopen(_libpath)
|
||||
lept.setMsgSeverity(lept.L_SEVERITY_WARNING)
|
||||
|
||||
+66
-29
@@ -10,6 +10,7 @@ import sys
|
||||
import tempfile
|
||||
from collections import defaultdict
|
||||
from functools import partial
|
||||
from io import BytesIO
|
||||
from os import fspath
|
||||
from pathlib import Path
|
||||
from typing import (
|
||||
@@ -27,6 +28,7 @@ from typing import (
|
||||
Union,
|
||||
)
|
||||
|
||||
import img2pdf
|
||||
import pikepdf
|
||||
from pikepdf import Dictionary, Name, Object, Pdf, PdfImage
|
||||
from PIL import Image
|
||||
@@ -37,7 +39,7 @@ from ocrmypdf._concurrent import exec_progress_pool
|
||||
from ocrmypdf._exec import jbig2enc, pngquant
|
||||
from ocrmypdf._jobcontext import PdfContext
|
||||
from ocrmypdf.exceptions import OutputFileAccessError
|
||||
from ocrmypdf.helpers import safe_symlink
|
||||
from ocrmypdf.helpers import deprecated, safe_symlink
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -75,23 +77,26 @@ def extract_image_filter(
|
||||
if image.Subtype != Name.Image:
|
||||
return None
|
||||
if image.Length < 100:
|
||||
log.debug("Skipping small image, xref %s", xref)
|
||||
log.debug(f"Skipping small image, xref {xref}")
|
||||
return None
|
||||
|
||||
pim = PdfImage(image)
|
||||
|
||||
if len(pim.filter_decodeparms) > 1:
|
||||
log.debug("Skipping multiply filtered, xref %s", xref)
|
||||
log.debug(f"Skipping multiply filtered image, xref {xref}")
|
||||
return None
|
||||
filtdp = pim.filter_decodeparms[0]
|
||||
|
||||
if pim.bits_per_component > 8:
|
||||
log.debug(f"Skipping wide gamut image, xref {xref}")
|
||||
return None # Don't mess with wide gamut images
|
||||
|
||||
if filtdp[0] == Name.JPXDecode:
|
||||
log.debug(f"Skipping JPEG2000 iamge, xref {xref}")
|
||||
return None # Don't do JPEG2000
|
||||
|
||||
if Name.Decode in image:
|
||||
log.debug(f"Skipping image with Decode table, xref {xref}")
|
||||
return None # Don't mess with custom Decode tables
|
||||
|
||||
return pim, filtdp
|
||||
@@ -111,12 +116,23 @@ def extract_image_jbig2(
|
||||
and jbig2enc.available()
|
||||
):
|
||||
try:
|
||||
# Save any colorspace associated with the image, so that we
|
||||
# will export a pure 1-bit PNG with no palette or ICC profile.
|
||||
# Showing the palette or ICC to jbig2enc will cause it to perform
|
||||
# colorspace transform to 1bpp, which will conflict the palette or
|
||||
# ICC if it exists.
|
||||
colorspace = pim.obj.ColorSpace
|
||||
# Set to DeviceGray temporarily; we already in 1 bpc.
|
||||
pim.obj.ColorSpace = pikepdf.Name.DeviceGray
|
||||
imgname = root / f'{xref:08d}'
|
||||
with imgname.open('wb') as f:
|
||||
ext = pim.extract_to(stream=f)
|
||||
imgname.rename(imgname.with_suffix(ext))
|
||||
except pikepdf.UnsupportedImageTypeError:
|
||||
return None
|
||||
finally:
|
||||
# Restore image colorspace after temporarily setting it to DeviceGray
|
||||
pim.obj.ColorSpace = colorspace
|
||||
return XrefExt(xref, ext)
|
||||
return None
|
||||
|
||||
@@ -227,7 +243,9 @@ def extract_images(
|
||||
# Ignore soft masks
|
||||
smask_xref = Xref(image.SMask.objgen[0])
|
||||
exclude_xrefs.add(smask_xref)
|
||||
log.debug(f"Skipping image {smask_xref} because it is an SMask")
|
||||
include_xrefs.add(xref)
|
||||
log.debug(f"Treating {xref} as an optimization candidate")
|
||||
if xref not in pageno_for_xref:
|
||||
pageno_for_xref[xref] = pageno
|
||||
|
||||
@@ -393,6 +411,47 @@ def transcode_jpegs(pike: Pdf, jpegs: Sequence[Xref], root: Path, options) -> No
|
||||
im_obj.write(compdata.read(), filter=Name.DCTDecode)
|
||||
|
||||
|
||||
def _transcode_png(pike: Pdf, filename: Path, xref: Xref) -> bool:
|
||||
output = filename.with_suffix('.png.pdf')
|
||||
with output.open('wb') as f:
|
||||
img2pdf.convert(fspath(filename), outputstream=f)
|
||||
|
||||
with pikepdf.open(output) as pdf_image:
|
||||
foreign_image = next(pdf_image.pages[0].images.values())
|
||||
local_image = pike.copy_foreign(foreign_image)
|
||||
|
||||
im_obj = pike.get_object(xref, 0)
|
||||
im_obj.write(
|
||||
local_image.read_raw_bytes(),
|
||||
filter=local_image.Filter,
|
||||
decode_parms=local_image.DecodeParms,
|
||||
)
|
||||
|
||||
# Don't copy keys from the new image...
|
||||
del_keys = set(im_obj.keys()) - set(local_image.keys())
|
||||
# ...except for the keep_fields, which are essential to displaying
|
||||
# the image correctly and preserving its metadata. (/Decode arrays
|
||||
# and /SMaskInData are implicitly discarded prior to this point.)
|
||||
keep_fields = {
|
||||
'/ID',
|
||||
'/Intent',
|
||||
'/Interpolate',
|
||||
'/Mask',
|
||||
'/Metadata',
|
||||
'/OC',
|
||||
'/OPI',
|
||||
'/SMask',
|
||||
'/StructParent',
|
||||
}
|
||||
del_keys -= keep_fields
|
||||
for key in local_image.keys():
|
||||
if key != Name.Length and str(key) not in keep_fields:
|
||||
im_obj[key] = local_image[key]
|
||||
for key in del_keys:
|
||||
del im_obj[key]
|
||||
return True
|
||||
|
||||
|
||||
def transcode_pngs(
|
||||
pike: Pdf,
|
||||
images: Sequence[Xref],
|
||||
@@ -435,34 +494,11 @@ def transcode_pngs(
|
||||
)
|
||||
|
||||
for xref in modified:
|
||||
im_obj = pike.get_object(xref, 0)
|
||||
try:
|
||||
pix = leptonica.Pix.open(png_name(root, xref))
|
||||
if pix.mode == '1':
|
||||
compdata = pix.generate_pdf_ci_data(leptonica.lept.L_G4_ENCODE, 0)
|
||||
else:
|
||||
compdata = leptonica.CompressedData.open(png_name(root, xref))
|
||||
except leptonica.LeptonicaError as e:
|
||||
# Most likely this means file not found, i.e. quantize did not
|
||||
# produce an improved version
|
||||
log.error(e)
|
||||
continue
|
||||
|
||||
# If re-coded image is larger don't use it - we test here because
|
||||
# pngquant knows the size of the temporary output file but not the actual
|
||||
# object in the PDF
|
||||
if len(compdata) > int(im_obj.stream_dict.Length):
|
||||
log.debug(
|
||||
f"pngquant: pngquant did not improve over original image "
|
||||
f"{len(compdata)} > {int(im_obj.stream_dict.Length)}"
|
||||
)
|
||||
continue
|
||||
if compdata.type == leptonica.lept.L_FLATE_ENCODE:
|
||||
rewrite_png(pike, im_obj, compdata)
|
||||
elif compdata.type == leptonica.lept.L_G4_ENCODE:
|
||||
rewrite_png_as_g4(pike, im_obj, compdata)
|
||||
filename = png_name(root, xref)
|
||||
_transcode_png(pike, filename, xref)
|
||||
|
||||
|
||||
@deprecated
|
||||
def rewrite_png_as_g4(pike: Pdf, im_obj: Object, compdata) -> None:
|
||||
im_obj.BitsPerComponent = 1
|
||||
im_obj.Width = compdata.w
|
||||
@@ -483,6 +519,7 @@ def rewrite_png_as_g4(pike: Pdf, im_obj: Object, compdata) -> None:
|
||||
return
|
||||
|
||||
|
||||
@deprecated
|
||||
def rewrite_png(pike: Pdf, im_obj: Object, compdata) -> None:
|
||||
# When a PNG is inserted into a PDF, we more or less copy the IDAT section from
|
||||
# the PDF and transfer the rest of the PNG headers to PDF image metadata.
|
||||
|
||||
@@ -154,11 +154,29 @@ def filter_page_image(page: 'PageContext', image_filename: Path) -> Path:
|
||||
produced for a given page, this function will not be called. This is not
|
||||
the image that will be shown to OCR.
|
||||
|
||||
ocrmypdf will create the PDF page based on the image format used. If you
|
||||
If the function does not want to modify the image, it should return
|
||||
``image_filename``. The hook may overwrite ``image_filename`` with a new file.
|
||||
|
||||
The output image should preserve the same physical unit dimensions, that is
|
||||
(width * dpi_x, height * dpi_y). That is, if the image is resized, the DPI
|
||||
must be adjusted by the reciprocal. If this is not preserved, the PDF page
|
||||
will be resized and the OCR layer misaligned. OCRmyPDF does not nothing
|
||||
to enforce these constraints; it is up to the plugin to do sensible things.
|
||||
|
||||
OCRmyPDF will create the PDF page based on the image format used. If you
|
||||
convert the image to a JPEG, the output page will be created as a JPEG, etc.
|
||||
Note that the ocrmypdf image optimization stage may ultimately chose a
|
||||
If you change the colorspace, that change will be kept. Note that the
|
||||
OCRmyPDF image optimization stage, if enabled, may ultimately chose a
|
||||
different format.
|
||||
|
||||
If the return value is a file that does not exist, ``FileNotFoundError``
|
||||
will occur. The return value should be a path to a file in the same folder
|
||||
as ``image_filename``.
|
||||
|
||||
Implementation detail: If the value returned is falsy, OCRmyPDF will ignore
|
||||
the return value and assume the input file was unmodified. This is deprecated.
|
||||
To leave the image unmodified, ``image_filename`` should be returned.
|
||||
|
||||
Note:
|
||||
This hook will be called from child processes. Modifying global state
|
||||
will not affect the main process or other child processes.
|
||||
|
||||
@@ -55,7 +55,7 @@ def test_old_ghostscript(caplog):
|
||||
vd._check_options(
|
||||
*make_opts_pm(language='chi_sim', output_type='pdfa'), {'chi_sim'}
|
||||
)
|
||||
assert 'Ghostscript does not work correctly' in caplog.text
|
||||
assert 'does not work correctly' in caplog.text
|
||||
|
||||
with patch('ocrmypdf._exec.ghostscript.version', return_value='9.18'), patch(
|
||||
'ocrmypdf._exec.tesseract.has_textonly_pdf', return_value=True
|
||||
|
||||
Reference in New Issue
Block a user