Fix Windows redo-ocr performance regression; drop pdfminer BUFSIZ workaround (#1706)

Since v16.4.3, OCRmyPDF forced pdfminer's read buffer to 256 MiB to work
around a pdfminer bug that mishandled tokens split across the buffer
boundary (gh #1361). On Windows this caused a severe performance
regression (gh #1662): CPython's BufferedReader.read(n) eagerly allocates
an n-byte buffer on every read, so pdfminer's thousands of seek+read
cycles each paid a ~30 ms 256 MiB allocation (this allocation is lazy and
effectively free on Linux). For a typical PDF the "Scanning contents"
phase went from ~5s on Linux to ~60s on Windows.

The underlying pdfminer bug was fixed upstream in pdfminer.six 20250327
(pdfminer/pdfminer.six#1030), with a follow-up for tokens split across
streams in 20260107 (pdfminer/pdfminer.six#1158). Remove the monkeypatch
entirely and raise the minimum pdfminer.six to 20260107 so we rely on the
upstream fix instead.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

docs/releasenotes/version17.md

@@ -3,6 +3,23 @@
 
 # v17
 
+## v17.7.1
+
+- Fixed a severe, Windows-specific performance regression in the "Scanning
+  contents" phase, most visible with `--redo-ocr` ({issue}`1662`). Since
+  v16.4.3, OCRmyPDF forced pdfminer's read buffer to 256 MiB to work around a
+  pdfminer bug that mishandled tokens split across the buffer boundary
+  ({issue}`1361`). On Windows, CPython's `BufferedReader.read()` eagerly
+  allocates a buffer of the requested size on every read, so the oversized
+  buffer made each of pdfminer's thousands of reads cost tens of milliseconds
+  (this allocation is lazy, and effectively free, on Linux). The underlying
+  pdfminer bug was fixed upstream in pdfminer.six 20250327
+  ([#1030](https://github.com/pdfminer/pdfminer.six/pull/1030)), with a
+  follow-up for tokens split across streams in 20260107
+  ([#1158](https://github.com/pdfminer/pdfminer.six/pull/1158)), so the
+  workaround has been removed and the minimum pdfminer.six version raised to
+  20260107.
+
 ## v17.7.0
 
 - The Docker images now run as a non-root user (`app`, uid/gid 1000) by default

pyproject.toml

@@ -16,7 +16,7 @@ dependencies = [
   "fpdf2>=2.8.0",
   "img2pdf>=0.5",
   "packaging>=20",
-  "pdfminer.six>=20220319",
+  "pdfminer.six>=20260107",  # fixes parsing of tokens split across the read buffer/streams (gh #1361)
   "pi-heif",                # Heif image format - maintainers: if this is removed, it will NOT break
   "pikepdf>=10",
   "Pillow>=10.0.1",

src/ocrmypdf/pdfinfo/layout.py

@@ -17,7 +17,6 @@ import pdfminer
 import pdfminer.encodingdb
 import pdfminer.pdfdevice
 import pdfminer.pdfinterp
-import pdfminer.psparser
 from deprecation import deprecated
 from pdfminer.converter import PDFLayoutAnalyzer
 from pdfminer.layout import LAParams, LTChar, LTPage, LTTextBox
@@ -60,12 +59,6 @@ def pdfsimplefont__init__(
 
 PDFSimpleFont.__init__ = pdfsimplefont__init__
 
-# Patch pdfminer.six buffer size
-# The parser doesn't properly handle keyword tokens are split across the end of the
-# buffer, so increase the buffer size something far larger than will ever be seen.
-pdfminer.psparser.PSBaseParser.BUFSIZ = 256 * 1024 * 1024
-
-
 def pdftype3font__pscript5_get_height(self):
     """Monkeypatch for PScript5.dll PDFs.