Compare commits

...
23 Commits
Author SHA1 Message Date
James R. Barlow 5629e960b9 v12.5.0 release notes update 2021-09-15 00:27:32 -07:00
James R. Barlow 79fd8d01a5 info: fix incorrect handling of inline images and other typing fixes 2021-09-15 00:26:15 -07:00
James R. Barlow 79fe7a0a85 graft: remove separate implementation of unparse_content_stream 2021-09-15 00:25:06 -07:00
James R. Barlow b4b32a35b5 optimize: fix typing consistency 2021-09-15 00:09:08 -07:00
James R. Barlow 4634b3db55 v12.5.0 release notes 2021-09-14 17:26:07 -07:00
James R. Barlow f5053158d4 hocrtransform: fix regression causing hocr text to be not rendered
Fixes #828
2021-09-14 17:23:09 -07:00
James R. Barlow dfa4ce1612 pyproject: configure mypy 2021-09-14 17:22:41 -07:00
James R. Barlow 585595a98e pre-commit: autoupdate 2021-09-14 17:22:29 -07:00
James R. Barlow f6396fbaac pre-commit: drop isort mirror 2021-09-14 17:20:43 -07:00
James R. Barlow 3859bae85e Merge branch 'feature/drop-pkg-resources' 2021-09-14 00:31:36 -07:00
James R. Barlow ee1a7baae7 requirements: drop setuptools
With importlib_* we no longer setuptools's pkg_resources.
2021-09-14 00:28:42 -07:00
James R. Barlow a4da05b66b docs: various fixes
As suggested by @Chealer

Closes #829, #830, #831, #832
2021-09-14 00:24:18 -07:00
James R. Barlow 4d67812d51 importlib helpers don't provide importlib.thing, but importlib_thing
Fix everywhere.
2021-09-14 00:15:07 -07:00
James R. Barlow 3534742ef9 ci: Enable build of feature branches 2021-09-13 01:35:14 -07:00
James R. Barlow 8bfd46c80d docs: update copyright 2021-09-13 01:10:49 -07:00
James R. Barlow cc6e9cecc0 Replace pkg_resources version lookup with importlib.metadata 2021-09-13 01:10:49 -07:00
James R. Barlow 208657f840 pdfa: replace pkg_resources with importlib.resources 2021-09-13 01:10:49 -07:00
James R. Barlow f3de980447 Introduce importlib-resources,metadata backports for Python < 3.9 2021-09-13 01:10:43 -07:00
James R. Barlow eb8992e58b Update release notes 2021-09-09 15:53:08 -07:00
James R. Barlow 72ad618ae6 Merge branch 'master' of github.com:jbarlow83/OCRmyPDF 2021-09-09 15:25:54 -07:00
James R. Barlow f07d0c39bb Require pikepdf<3 on PyPy 3.6
Because cibuildwheel does not build wheels for PyPy 3.6 anymore,
so pikepdf does not offer one.
2021-09-09 15:25:38 -07:00
James R. Barlow 9c5c7d9be0 release notes: mention Py3.6 EOL 2021-09-08 00:04:21 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
0b19b084e2 build(deps): bump pillow from 8.2.0 to 8.3.2 in /requirements/main.txt (#825)
Bumps [pillow](https://github.com/python-pillow/Pillow) from 8.2.0 to 8.3.2.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/master/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/8.2.0...8.3.2)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-08 00:03:35 -07:00
15 changed files with 155 additions and 110 deletions
+1
View File
@@ -6,6 +6,7 @@ on:
- master - master
- ci - ci
- release/* - release/*
- feature/*
tags: tags:
- v* - v*
paths-ignore: paths-ignore:
+5 -8
View File
@@ -7,16 +7,13 @@ repos:
- id: check-toml - id: check-toml
- id: check-yaml - id: check-yaml
- id: debug-statements - id: debug-statements
- repo: https://github.com/asottile/seed-isort-config - repo: https://github.com/pycqa/isort
rev: v2.2.0 rev: 5.9.3
hooks:
- id: seed-isort-config
- repo: https://github.com/pre-commit/mirrors-isort
rev: v5.9.3 # pick the isort version you'd like to use from https://github.com/pre-commit/mirrors-isort/releases
hooks: hooks:
- id: isort - id: isort
args: ["--profile", "black"]
- repo: https://github.com/psf/black - repo: https://github.com/psf/black
rev: 21.7b0 rev: 21.9b0
hooks: hooks:
- id: black - id: black
language_version: python language_version: python
@@ -26,7 +23,7 @@ repos:
hooks: hooks:
- id: setup-cfg-fmt - id: setup-cfg-fmt
- repo: https://github.com/asottile/pyupgrade - repo: https://github.com/asottile/pyupgrade
rev: v2.24.0 rev: v2.26.0
hooks: hooks:
- id: pyupgrade - id: pyupgrade
args: ["--py36-plus"] args: ["--py36-plus"]
+3 -4
View File
@@ -56,7 +56,7 @@ master_doc = 'index'
# General information about the project. # General information about the project.
project = 'ocrmypdf' project = 'ocrmypdf'
copyright = ( copyright = (
'2020, James R. Barlow. Licensed under Creative Commons Attribution-ShareAlike 4.0.' '2021, James R. Barlow. Licensed under Creative Commons Attribution-ShareAlike 4.0.'
) )
author = 'James R. Barlow' author = 'James R. Barlow'
@@ -88,11 +88,10 @@ if on_rtd:
] ]
sys.modules.update((mod_name, Mock()) for mod_name in MOCK_MODULES) sys.modules.update((mod_name, Mock()) for mod_name in MOCK_MODULES)
from importlib_metadata import version as package_version
from pkg_resources import get_distribution, DistributionNotFound
# The full version, including alpha/beta/rc tags. # The full version, including alpha/beta/rc tags.
release = get_distribution('ocrmypdf').version release = package_version('ocrmypdf').version
version = '.'.join(release.split('.')[:2]) version = '.'.join(release.split('.')[:2])
+14 -9
View File
@@ -2,7 +2,12 @@
Introduction Introduction
============ ============
OCRmyPDF is a Python 3 application and library that adds OCR layers to PDFs. OCRmyPDF is an application and library that adds text "layers" to images
in PDFs, making scanned image PDFs searchable. It uses OCR to guess what text
is contained in images. It is written in Python. OCRmyPDF supports plugins
that allow customization of its processing steps, and is very tolerant of
PDFs that contain scanned images and "born digital" content that needs no
text recognition.
About OCR About OCR
========= =========
@@ -26,7 +31,7 @@ exactly. They contain `vector
graphics <http://vector-conversions.com/vectorizing/raster_vs_vector.html>`__ graphics <http://vector-conversions.com/vectorizing/raster_vs_vector.html>`__
that can contain raster objects such as scanned images. Because PDFs can that can contain raster objects such as scanned images. Because PDFs can
contain multiple pages (unlike many image formats) and can contain fonts contain multiple pages (unlike many image formats) and can contain fonts
and text, it is a good formats for exchanging scanned documents. and text, it is a good format for exchanging scanned documents.
|image| |image|
@@ -35,9 +40,9 @@ have one image. Some scanners or scanning software will segment pages
into monochromatic text and color regions for example, to improve the into monochromatic text and color regions for example, to improve the
compression ratio and appearance of the page. compression ratio and appearance of the page.
Rasterizing a PDF is the process of generating an image suitable for Rasterizing a PDF is the process of generating corresponding raster images.
display or analyzing with an OCR engine. OCR engines like Tesseract work OCR engines like Tesseract work with images, not scalable vector graphics
with images, not vector objects. or mixed raster-vector-text graphics such as PDF.
About PDF/A About PDF/A
=========== ===========
@@ -76,7 +81,7 @@ OCRmyPDF analyzes each page of a PDF to determine the colorspace and
resolution (DPI) needed to capture all of the information on that page resolution (DPI) needed to capture all of the information on that page
without losing content. It uses without losing content. It uses
`Ghostscript <http://ghostscript.com/>`__ to rasterize the page, and `Ghostscript <http://ghostscript.com/>`__ to rasterize the page, and
then performs on OCR on the rasterized image to create an OCR "layer". then performs on OCR the rasterized image to create an OCR "layer".
The layer is then grafted back onto the original PDF. The layer is then grafted back onto the original PDF.
While one can use a program like Ghostscript or ImageMagick to get an While one can use a program like Ghostscript or ImageMagick to get an
@@ -84,9 +89,9 @@ image and put the image through Tesseract, that actually creates a new
PDF and many details may be lost. OCRmyPDF can produce a minimally PDF and many details may be lost. OCRmyPDF can produce a minimally
changed PDF as output. changed PDF as output.
OCRmyPDF also some image processing options like deskew which improve OCRmyPDF also provides some image processing options, like deskew, which
the appearance of files and quality of OCR. When these are used, the OCR improves the appearance of files and quality of OCR. When these are used,
layer is grafted onto the processed image instead. the OCR layer is grafted onto the processed image instead.
By default, OCRmyPDF produces archival PDFs PDF/A, which are a By default, OCRmyPDF produces archival PDFs PDF/A, which are a
stricter subset of PDF features designed for long term archives. If stricter subset of PDF features designed for long term archives. If
+19
View File
@@ -12,6 +12,25 @@ may be unreliable. Use the API to depend on precise behavior.
The public API may be useful in scripts that launch OCRmyPDF processes or that The public API may be useful in scripts that launch OCRmyPDF processes or that
wish to use some of its features for working with PDFs. wish to use some of its features for working with PDFs.
.. note::
Python 3.6 reaches end of life on December 23, 2021. We will end support
for Python 3.6 around that time. The change will be marked with a major
release.
v12.5.0
=======
- Fixed build failure for the combination of PyPy 3.6 and pikepdf 3.0. This
combination can work in a source build but does not work with wheels.
- Accepted bot that wanted to upgrade our deprecated requirements.txt.
- Documentation updates.
- Replace pkg_resources and install dependency on setuptools with
importlib-metadata and importlib-resources.
- Fixed regression in hocrtransform causing text to be omitted when this
renderer was used.
- Fixed some typing errors.
v12.4.0 v12.4.0
======= =======
+9 -1
View File
@@ -71,4 +71,12 @@ norecursedirs = ["lib", ".pc", ".git", "venv", "output", "cache", "resources"]
testpaths = ["tests"] testpaths = ["tests"]
addopts = "-n auto" addopts = "-n auto"
markers = ["slow"] markers = ["slow"]
filterwarnings = ["ignore:.*XMLParser.*:DeprecationWarning"] filterwarnings = ["ignore:.*XMLParser.*:DeprecationWarning"]
[tool.mypy]
[[tool.mypy.overrides]]
module = [
'pluggy', 'tqdm', 'coloredlogs', 'img2pdf', 'cffi', '_cffi_backend', 'pdfminer.*', 'reportlab.*'
]
ignore_missing_imports = true
+1 -1
View File
@@ -5,6 +5,6 @@ img2pdf == 0.4.0
pdfminer.six == 20201018 pdfminer.six == 20201018
pikepdf == 2.10.0 pikepdf == 2.10.0
pluggy == 0.13.1 pluggy == 0.13.1
Pillow == 8.2.0 Pillow == 8.3.2
reportlab == 3.5.66 reportlab == 3.5.66
tqdm == 4.59.0 tqdm == 4.59.0
+3 -1
View File
@@ -49,11 +49,13 @@ install_requires =
cffi>=1.9.1 # must be a setup and install requirement cffi>=1.9.1 # must be a setup and install requirement
coloredlogs>=14.0 # strictly optional coloredlogs>=14.0 # strictly optional
img2pdf>=0.3.0,<0.5 # pure Python img2pdf>=0.3.0,<0.5 # pure Python
importlib-metadata>=4 # until Python 3.8
importlib-resources>=5 # until Python 3.9
pdfminer.six!=20200720,>=20191110,<=20201018 pdfminer.six!=20200720,>=20191110,<=20201018
pikepdf>=2.10.0 pikepdf>=2.10.0
pikepdf<3;implementation_name=="pypy" and python_version=='3.6'
pluggy>=0.13.0,<2 pluggy>=0.13.0,<2
reportlab>=3.5.66 reportlab>=3.5.66
setuptools
tqdm>=4 tqdm>=4
python_requires = >=3.6 python_requires = >=3.6
include_package_data = True include_package_data = True
+38 -50
View File
@@ -11,8 +11,19 @@ from contextlib import suppress
from pathlib import Path from pathlib import Path
from typing import Optional from typing import Optional
import pikepdf from pikepdf import (
from pikepdf.objects import Dictionary, Name Dictionary,
Name,
Object,
Operator,
Page,
Pdf,
PdfError,
PdfMatrix,
Stream,
parse_content_stream,
unparse_content_stream,
)
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
MAX_REPLACE_PAGES = 100 MAX_REPLACE_PAGES = 100
@@ -47,51 +58,28 @@ def strip_invisible_text(pdf, page):
render_mode = 0 render_mode = 0
text_objects = [] text_objects = []
rich_page = pikepdf.Page(page) rich_page = Page(page)
rich_page.contents_coalesce() rich_page.contents_coalesce()
for operands, operator in pikepdf.parse_content_stream(page, ''): for operands, operator in parse_content_stream(page, ''):
if not in_text_obj: if not in_text_obj:
if operator == pikepdf.Operator('BT'): if operator == Operator('BT'):
in_text_obj = True in_text_obj = True
render_mode = 0 render_mode = 0
text_objects.append((operands, operator)) text_objects.append((operands, operator))
else: else:
stream.append((operands, operator)) stream.append((operands, operator))
else: else:
if operator == pikepdf.Operator('Tr'): if operator == Operator('Tr'):
render_mode = operands[0] render_mode = operands[0]
text_objects.append((operands, operator)) text_objects.append((operands, operator))
if operator == pikepdf.Operator('ET'): if operator == Operator('ET'):
in_text_obj = False in_text_obj = False
if render_mode != 3: if render_mode != 3:
stream.extend(text_objects) stream.extend(text_objects)
text_objects.clear() text_objects.clear()
def convert(op): content_stream = unparse_content_stream(stream)
try: page.Contents = Stream(pdf, content_stream)
return op.unparse()
except AttributeError:
return str(op).encode('ascii')
if hasattr(pikepdf, 'unparse_content_stream'):
content_stream = pikepdf.unparse_content_stream(stream)
else:
lines = []
for operands, operator in stream:
if operator == pikepdf.Operator('INLINE IMAGE'):
iim = operands[0]
line = iim.unparse()
else:
line = (
b' '.join(convert(op) for op in operands)
+ b' '
+ operator.unparse()
)
lines.append(line)
content_stream = b'\n'.join(lines)
page.Contents = pikepdf.Stream(pdf, content_stream)
class OcrGrafter: class OcrGrafter:
@@ -99,14 +87,14 @@ class OcrGrafter:
self.context = context self.context = context
self.path_base = context.origin self.path_base = context.origin
self.pdf_base = pikepdf.open(self.path_base) self.pdf_base = Pdf.open(self.path_base)
self.font, self.font_key = None, None self.font, self.font_key = None, None
self.pdfinfo = context.pdfinfo self.pdfinfo = context.pdfinfo
self.output_file = context.get_path('graft_layers.pdf') self.output_file = context.get_path('graft_layers.pdf')
self.procset = self.pdf_base.make_indirect( self.procset = self.pdf_base.make_indirect(
pikepdf.Object.parse(b'[ /PDF /Text /ImageB /ImageC /ImageI ]') Object.parse(b'[ /PDF /Text /ImageB /ImageC /ImageI ]')
) )
self.emplacements = 1 self.emplacements = 1
@@ -130,7 +118,7 @@ class OcrGrafter:
# We are updating the old page with a rasterized PDF of the new # We are updating the old page with a rasterized PDF of the new
# page (without changing objgen, to preserve references) # page (without changing objgen, to preserve references)
log.debug("Emplacement update") log.debug("Emplacement update")
with pikepdf.open(image) as pdf_image: with Pdf.open(path_image) as pdf_image:
self.emplacements += 1 self.emplacements += 1
foreign_image_page = pdf_image.pages[0] foreign_image_page = pdf_image.pages[0]
self.pdf_base.pages.append(foreign_image_page) self.pdf_base.pages.append(foreign_image_page)
@@ -203,7 +191,7 @@ class OcrGrafter:
self.pdf_base.save(next_file) self.pdf_base.save(next_file)
self.pdf_base.close() self.pdf_base.close()
self.pdf_base = pikepdf.open(next_file) self.pdf_base = Pdf.open(next_file)
self.procset = self.pdf_base.pages[0].Resources.ProcSet self.procset = self.pdf_base.pages[0].Resources.ProcSet
self.font, self.font_key = None, None # Ensure we reacquire this information self.font, self.font_key = None, None # Ensure we reacquire this information
self.interim_count += 1 self.interim_count += 1
@@ -219,7 +207,7 @@ class OcrGrafter:
font, font_key = None, None font, font_key = None, None
possible_font_names = ('/f-0-0', '/F1') possible_font_names = ('/f-0-0', '/F1')
try: try:
with pikepdf.open(text) as pdf_text: with Pdf.open(text) as pdf_text:
try: try:
pdf_text_fonts = pdf_text.pages[0].Resources.get('/Font', {}) pdf_text_fonts = pdf_text.pages[0].Resources.get('/Font', {})
except (AttributeError, IndexError, KeyError): except (AttributeError, IndexError, KeyError):
@@ -233,7 +221,7 @@ class OcrGrafter:
if pdf_text_font: if pdf_text_font:
font = self.pdf_base.copy_foreign(pdf_text_font) font = self.pdf_base.copy_foreign(pdf_text_font)
return font, font_key return font, font_key
except (FileNotFoundError, pikepdf.PdfError): except (FileNotFoundError, PdfError):
# PdfError occurs if a 0-length file is written e.g. due to OCR timeout # PdfError occurs if a 0-length file is written e.g. due to OCR timeout
return None, None return None, None
@@ -242,9 +230,9 @@ class OcrGrafter:
*, *,
page_num: int, page_num: int,
textpdf: Path, textpdf: Path,
font: pikepdf.Object, font: Object,
font_key: pikepdf.Object, font_key: Object,
procset: pikepdf.Object, procset: Object,
text_rotation: int, text_rotation: int,
strip_old_text: bool, strip_old_text: bool,
): ):
@@ -255,7 +243,7 @@ class OcrGrafter:
return return
# This is a pointer indicating a specific page in the base file # This is a pointer indicating a specific page in the base file
with pikepdf.open(textpdf) as pdf_text: with Pdf.open(textpdf) as pdf_text:
pdf_text_contents = pdf_text.pages[0].Contents.read_bytes() pdf_text_contents = pdf_text.pages[0].Contents.read_bytes()
base_page = self.pdf_base.pages.p(page_num) base_page = self.pdf_base.pages.p(page_num)
@@ -270,13 +258,13 @@ class OcrGrafter:
mediabox = [float(base_page.MediaBox[v]) for v in range(4)] mediabox = [float(base_page.MediaBox[v]) for v in range(4)]
wp, hp = mediabox[2] - mediabox[0], mediabox[3] - mediabox[1] wp, hp = mediabox[2] - mediabox[0], mediabox[3] - mediabox[1]
translate = pikepdf.PdfMatrix().translated(-wt / 2, -ht / 2) translate = PdfMatrix().translated(-wt / 2, -ht / 2)
untranslate = pikepdf.PdfMatrix().translated(wp / 2, hp / 2) untranslate = PdfMatrix().translated(wp / 2, hp / 2)
corner = pikepdf.PdfMatrix().translated(mediabox[0], mediabox[1]) corner = PdfMatrix().translated(mediabox[0], mediabox[1])
# -rotation because the input is a clockwise angle and this formula # -rotation because the input is a clockwise angle and this formula
# uses CCW # uses CCW
text_rotation = -text_rotation % 360 text_rotation = -text_rotation % 360
rotate = pikepdf.PdfMatrix().rotated(text_rotation) rotate = PdfMatrix().rotated(text_rotation)
# Because of rounding of DPI, we might get a text layer that is not # Because of rounding of DPI, we might get a text layer that is not
# identically sized to the target page. Scale to adjust. Normally this # identically sized to the target page. Scale to adjust. Normally this
@@ -287,7 +275,7 @@ class OcrGrafter:
scale_y = hp / ht scale_y = hp / ht
# log.debug('%r', scale_x, scale_y) # log.debug('%r', scale_x, scale_y)
scale = pikepdf.PdfMatrix().scaled(scale_x, scale_y) scale = PdfMatrix().scaled(scale_x, scale_y)
# Translate the text so it is centered at (0, 0), rotate it there, adjust # Translate the text so it is centered at (0, 0), rotate it there, adjust
# for a size different between initial and text PDF, then untranslate, and # for a size different between initial and text PDF, then untranslate, and
@@ -310,14 +298,14 @@ class OcrGrafter:
pdf_draw_xobj = ( pdf_draw_xobj = (
(b'q %s cm\n' % ctm.encode()) + (b'%s Do\n' % text_xobj_name) + b'\nQ\n' (b'q %s cm\n' % ctm.encode()) + (b'%s Do\n' % text_xobj_name) + b'\nQ\n'
) )
new_text_layer = pikepdf.Stream(self.pdf_base, pdf_draw_xobj) new_text_layer = Stream(self.pdf_base, pdf_draw_xobj)
if strip_old_text: if strip_old_text:
strip_invisible_text(self.pdf_base, base_page) strip_invisible_text(self.pdf_base, base_page)
if hasattr(pikepdf.Page, 'contents_add'): if hasattr(Page, 'contents_add'):
# pikepdf >= 2.14 adds this method and deprecates the one below # pikepdf >= 2.14 adds this method and deprecates the one below
pikepdf.Page(base_page).contents_add(new_text_layer, prepend=True) Page(base_page).contents_add(new_text_layer, prepend=True)
else: else:
# pikepdf < 2.14 # pikepdf < 2.14
base_page.page_contents_add( base_page.page_contents_add(
+2 -2
View File
@@ -5,9 +5,9 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/. # file, You can obtain one at http://mozilla.org/MPL/2.0/.
import pkg_resources from importlib_metadata import version as _package_version
PROGRAM_NAME = 'ocrmypdf' PROGRAM_NAME = 'ocrmypdf'
# Official PEP 396 # Official PEP 396
__version__ = pkg_resources.get_distribution('ocrmypdf').version __version__ = _package_version('ocrmypdf')
+8
View File
@@ -0,0 +1,8 @@
# © 2021 James R. Barlow: github.com/jbarlow83
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
"""Data files used to generate certain PDFs."""
+1 -1
View File
@@ -349,7 +349,7 @@ class HocrTransform:
interword_spaces: bool, interword_spaces: bool,
show_bounding_boxes: bool, show_bounding_boxes: bool,
): ):
if line is not None: if line is None:
return return
pxl_line_coords = self.element_coordinates(line) pxl_line_coords = self.element_coordinates(line)
line_box = self.pt_from_pixel(pxl_line_coords) line_box = self.pt_from_pixel(pxl_line_coords)
+22 -14
View File
@@ -25,8 +25,16 @@ from typing import (
) )
import img2pdf import img2pdf
import pikepdf from pikepdf import (
from pikepdf import Dictionary, Name, Object, Pdf, PdfImage Dictionary,
Name,
Object,
ObjectStreamMode,
Pdf,
PdfImage,
Stream,
UnsupportedImageTypeError,
)
from PIL import Image from PIL import Image
from ocrmypdf import leptonica from ocrmypdf import leptonica
@@ -63,7 +71,7 @@ def jpg_name(root: Path, xref: Xref) -> Path:
def extract_image_filter( def extract_image_filter(
pike: Pdf, root: Path, image: Object, xref: Xref pike: Pdf, root: Path, image: Stream, xref: Xref
) -> Optional[Tuple[PdfImage, Tuple[Name, Object]]]: ) -> Optional[Tuple[PdfImage, Tuple[Name, Object]]]:
del pike # unused args del pike # unused args
del root del root
@@ -104,7 +112,7 @@ def extract_image_filter(
def extract_image_jbig2( def extract_image_jbig2(
*, pike: pikepdf.Pdf, root: Path, image: Object, xref: Xref, options *, pike: Pdf, root: Path, image: Stream, xref: Xref, options
) -> Optional[XrefExt]: ) -> Optional[XrefExt]:
del options # unused arg del options # unused arg
@@ -123,16 +131,16 @@ def extract_image_jbig2(
# Showing the palette or ICC to jbig2enc will cause it to perform # Showing the palette or ICC to jbig2enc will cause it to perform
# colorspace transform to 1bpp, which will conflict the palette or # colorspace transform to 1bpp, which will conflict the palette or
# ICC if it exists. # ICC if it exists.
colorspace = pim.obj.get(pikepdf.Name.ColorSpace, None) colorspace = pim.obj.get(Name.ColorSpace, None)
if colorspace is not None or pim.image_mask: if colorspace is not None or pim.image_mask:
try: try:
# Set to DeviceGray temporarily; we already in 1 bpc. # Set to DeviceGray temporarily; we already in 1 bpc.
pim.obj.ColorSpace = pikepdf.Name.DeviceGray pim.obj.ColorSpace = Name.DeviceGray
imgname = root / f'{xref:08d}' imgname = root / f'{xref:08d}'
with imgname.open('wb') as f: with imgname.open('wb') as f:
ext = pim.extract_to(stream=f) ext = pim.extract_to(stream=f)
imgname.rename(imgname.with_suffix(ext)) imgname.rename(imgname.with_suffix(ext))
except pikepdf.UnsupportedImageTypeError: except UnsupportedImageTypeError:
return None return None
finally: finally:
# Restore image colorspace after temporarily setting it to DeviceGray # Restore image colorspace after temporarily setting it to DeviceGray
@@ -145,7 +153,7 @@ def extract_image_jbig2(
def extract_image_generic( def extract_image_generic(
*, pike: Pdf, root: Path, image: PdfImage, xref: Xref, options *, pike: Pdf, root: Path, image: Stream, xref: Xref, options
) -> Optional[XrefExt]: ) -> Optional[XrefExt]:
result = extract_image_filter(pike, root, image, xref) result = extract_image_filter(pike, root, image, xref)
if result is None: if result is None:
@@ -178,7 +186,7 @@ def extract_image_generic(
with imgname.open('wb') as f: with imgname.open('wb') as f:
ext = pim.extract_to(stream=f) ext = pim.extract_to(stream=f)
imgname.rename(imgname.with_suffix(ext)) imgname.rename(imgname.with_suffix(ext))
except pikepdf.UnsupportedImageTypeError: except UnsupportedImageTypeError:
return None return None
return XrefExt(xref, ext) return XrefExt(xref, ext)
elif ( elif (
@@ -374,7 +382,7 @@ def convert_to_jbig2(
jbig2_symfile = root / (prefix + '.sym') jbig2_symfile = root / (prefix + '.sym')
if jbig2_symfile.exists(): if jbig2_symfile.exists():
jbig2_globals_data = jbig2_symfile.read_bytes() jbig2_globals_data = jbig2_symfile.read_bytes()
jbig2_globals = pikepdf.Stream(pike, jbig2_globals_data) jbig2_globals = Stream(pike, jbig2_globals_data)
jbig2_globals_dict = Dictionary(JBIG2Globals=jbig2_globals) jbig2_globals_dict = Dictionary(JBIG2Globals=jbig2_globals)
elif options.jbig2_page_group_size == 1: elif options.jbig2_page_group_size == 1:
jbig2_globals_dict = None jbig2_globals_dict = None
@@ -445,7 +453,7 @@ def _transcode_png(pike: Pdf, filename: Path, xref: Xref) -> bool:
with output.open('wb') as f: with output.open('wb') as f:
img2pdf.convert(fspath(filename), outputstream=f) img2pdf.convert(fspath(filename), outputstream=f)
with pikepdf.open(output) as pdf_image: with Pdf.open(output) as pdf_image:
foreign_image = next(pdf_image.pages[0].images.values()) foreign_image = next(pdf_image.pages[0].images.values())
local_image = pike.copy_foreign(foreign_image) local_image = pike.copy_foreign(foreign_image)
@@ -544,7 +552,7 @@ def optimize(
if options.jbig2_page_group_size == 0: if options.jbig2_page_group_size == 0:
options.jbig2_page_group_size = 10 if options.jbig2_lossy else 1 options.jbig2_page_group_size = 10 if options.jbig2_lossy else 1
with pikepdf.Pdf.open(input_file) as pike: with Pdf.open(input_file) as pike:
root = output_file.parent / 'images' root = output_file.parent / 'images'
root.mkdir(exist_ok=True) root.mkdir(exist_ok=True)
@@ -576,7 +584,7 @@ def optimize(
if savings < 0: if savings < 0:
log.info("Image optimization did not improve the file - discarded") log.info("Image optimization did not improve the file - discarded")
# We still need to save the file # We still need to save the file
with pikepdf.open(input_file) as pike: with Pdf.open(input_file) as pike:
pike.remove_unreferenced_resources() pike.remove_unreferenced_resources()
pike.save(output_file, **save_settings) pike.save(output_file, **save_settings)
else: else:
@@ -623,7 +631,7 @@ def main(infile, outfile, level, jobs=1):
dict( dict(
compress_streams=True, compress_streams=True,
preserve_pdfa=True, preserve_pdfa=True,
object_stream_mode=pikepdf.ObjectStreamMode.generate, object_stream_mode=ObjectStreamMode.generate,
), ),
) )
copy(fspath(tmpout), fspath(outfile)) copy(fspath(tmpout), fspath(outfile))
+10 -6
View File
@@ -13,13 +13,17 @@ import base64
from pathlib import Path from pathlib import Path
from typing import Dict, Iterator, Union from typing import Dict, Iterator, Union
import importlib_resources
import pikepdf import pikepdf
import pkg_resources import pkg_resources # deprecated
# Deprecated
ICC_PROFILE_RELPATH = 'data/sRGB.icc' ICC_PROFILE_RELPATH = 'data/sRGB.icc'
# Deprecated
SRGB_ICC_PROFILE = pkg_resources.resource_filename('ocrmypdf', ICC_PROFILE_RELPATH) SRGB_ICC_PROFILE = pkg_resources.resource_filename('ocrmypdf', ICC_PROFILE_RELPATH)
SRGB_ICC_PROFILE_NAME = 'sRGB.icc'
def _postscript_objdef( def _postscript_objdef(
alias: str, alias: str,
@@ -97,12 +101,12 @@ def generate_pdfa_ps(target_filename: Path, icc: str = 'sRGB'):
References: References:
Adobe PDFMARK Reference: https://www.adobe.com/content/dam/acom/en/devnet/acrobat/pdfs/pdfmark_reference.pdf Adobe PDFMARK Reference: https://www.adobe.com/content/dam/acom/en/devnet/acrobat/pdfs/pdfmark_reference.pdf
""" """
if icc == 'sRGB': if icc != 'sRGB':
icc_profile = SRGB_ICC_PROFILE
else:
raise NotImplementedError("Only supporting sRGB") raise NotImplementedError("Only supporting sRGB")
bytes_icc_profile = Path(icc_profile).read_bytes() bytes_icc_profile = importlib_resources.read_binary(
'ocrmypdf.data', SRGB_ICC_PROFILE_NAME
)
ps = '\n'.join(_make_postscript(icc, bytes_icc_profile, 3)) ps = '\n'.join(_make_postscript(icc, bytes_icc_profile, 3))
# We should have encoded everything to pure ASCII by this point, and # We should have encoded everything to pure ASCII by this point, and
+19 -13
View File
@@ -30,8 +30,14 @@ from typing import (
) )
from warnings import warn from warnings import warn
import pikepdf from pikepdf import (
from pikepdf import Name, Object, Pdf, PdfInlineImage, PdfMatrix Object,
Pdf,
PdfImage,
PdfInlineImage,
PdfMatrix,
parse_content_stream,
)
from ocrmypdf._concurrent import Executor, SerialExecutor from ocrmypdf._concurrent import Executor, SerialExecutor
from ocrmypdf.exceptions import EncryptedPdfError, InputFileError from ocrmypdf.exceptions import EncryptedPdfError, InputFileError
@@ -181,9 +187,7 @@ def _interpret_contents(contentstream: Object, initial_shorthand=UNIT_SQUARE):
operator_whitelist = ' '.join(vector_ops | text_showing_ops | image_ops) operator_whitelist = ' '.join(vector_ops | text_showing_ops | image_ops)
for n, graphobj in enumerate( for n, graphobj in enumerate(
_normalize_stack( _normalize_stack(parse_content_stream(contentstream, operator_whitelist))
pikepdf.parse_content_stream(contentstream, operator_whitelist)
)
): ):
operands, operator = graphobj operands, operator = graphobj
if operator == 'q': if operator == 'q':
@@ -303,18 +307,20 @@ class ImageInfo:
*, *,
name='', name='',
pdfimage: Optional[Object] = None, pdfimage: Optional[Object] = None,
inline: Optional[Object] = None, inline: Optional[PdfInlineImage] = None,
shorthand=None, shorthand=None,
): ):
self._name = str(name) self._name = str(name)
self._shorthand = shorthand self._shorthand = shorthand
pim: Union[PdfInlineImage, PdfImage]
if inline is not None: if inline is not None:
self._origin = 'inline' self._origin = 'inline'
pim = inline.iimage pim = inline
elif pdfimage is not None: elif pdfimage is not None:
self._origin = 'xobject' self._origin = 'xobject'
pim = pikepdf.PdfImage(pdfimage) pim = PdfImage(pdfimage)
else: else:
raise ValueError("Either pdfimage or inline must be set") raise ValueError("Either pdfimage or inline must be set")
self._width = pim.width self._width = pim.width
@@ -335,7 +341,7 @@ class ImageInfo:
self._enc = None self._enc = None
try: try:
self._color = FRIENDLY_COLORSPACE.get(pim.colorspace) self._color = FRIENDLY_COLORSPACE.get(pim.colorspace or '')
except NotImplementedError: except NotImplementedError:
self._color = None self._color = None
if self._enc == Encoding.jpeg2000: if self._enc == Encoding.jpeg2000:
@@ -418,7 +424,7 @@ def _find_inline_images(contentsinfo: ContentsInfo) -> Iterator[ImageInfo]:
for n, inline in enumerate(contentsinfo.inline_images): for n, inline in enumerate(contentsinfo.inline_images):
yield ImageInfo( yield ImageInfo(
name='inline-%02d' % n, shorthand=inline.shorthand, inline=inline name='inline-%02d' % n, shorthand=inline.shorthand, inline=inline.iimage
) )
@@ -613,7 +619,7 @@ def _pdf_pageinfo_sync_init(pdf: Pdf, infile: Path, pdfminer_loglevel):
# If the pdf is not opened, open a copy for our worker process to use # If the pdf is not opened, open a copy for our worker process to use
if pdf is None: if pdf is None:
worker_pdf = pikepdf.open(infile) worker_pdf = Pdf.open(infile)
def on_process_close(): def on_process_close():
worker_pdf.close() worker_pdf.close()
@@ -627,7 +633,7 @@ def _pdf_pageinfo_sync(args):
pdf = thread_pdf if thread_pdf is not None else worker_pdf pdf = thread_pdf if thread_pdf is not None else worker_pdf
with ExitStack() as stack: with ExitStack() as stack:
if not pdf: # When called with SerialExecutor if not pdf: # When called with SerialExecutor
pdf = stack.enter_context(pikepdf.open(infile)) pdf = stack.enter_context(Pdf.open(infile))
page = PageInfo(pdf, pageno, infile, check_pages, detailed_analysis) page = PageInfo(pdf, pageno, infile, check_pages, detailed_analysis)
return page return page
@@ -888,7 +894,7 @@ class PdfInfo:
if check_pages is None: if check_pages is None:
check_pages = range(0, 1_000_000_000) check_pages = range(0, 1_000_000_000)
with pikepdf.open(infile) as pdf: with Pdf.open(infile) as pdf:
if pdf.is_encrypted: if pdf.is_encrypted:
raise EncryptedPdfError() # Triggered by encryption with empty passwd raise EncryptedPdfError() # Triggered by encryption with empty passwd
self._pages = _pdf_pageinfo_concurrent( self._pages = _pdf_pageinfo_concurrent(