Fix mypy errors in concurrency abstractions

Resolves all 11 remaining errors in _concurrent.py and
builtin_plugins/concurrency.py.

Root cause for 9 of the 11: setup_executor() was annotated to return
ocrmypdf's own Executor ABC as its second tuple element, but it
actually returns a concurrent.futures pool class (ThreadPoolExecutor
or ProcessPoolExecutor) - an already-existing FuturesExecutorClass
alias was defined for exactly this but never wired into the
signature. That wrong annotation made mypy check
`executor_class(initializer=..., initargs=...)` in _execute() against
Executor.__call__'s signature (which has entirely different
parameters and returns None), cascading into 8 further errors
(unexpected keyword args, "function does not return a value", "None
has no attribute __enter__/__exit__"). Fixing the one annotation
resolved all of them.

Also:
- Added a proper Queue[LogRecord | None] generic parameter (was bare
  Queue, which needs an explicit type argument for mypy to infer
  loq_queue's type across the use_threads branches).
- _concurrent.py: Executor.__call__'s task parameter defaults to
  _task_noop (return type None) when the caller omits a task, but the
  parameter is typed Callable[..., T] for an unbound per-call T. Used
  cast() since task_finished's own no-op default already accepts Any,
  so the mismatch is never actually exercised unsafely.
This commit is contained in:
James R. Barlow
2026-07-07 00:49:07 -07:00
parent 212b28e602
commit 3f1aceade2
2 changed files with 14 additions and 4 deletions
+5 -2
View File
@@ -8,7 +8,7 @@ from __future__ import annotations
import threading
from abc import ABC, abstractmethod
from collections.abc import Callable, Iterable
from typing import Any, TypeVar
from typing import Any, TypeVar, cast
from ocrmypdf._progressbar import NullProgressBar, ProgressBar
@@ -72,7 +72,10 @@ class Executor(ABC):
if not task_finished:
task_finished = _task_finished_noop
if not task:
task = _task_noop
# _task_noop always returns None, but T is unbound here (it's
# only meaningful when a real task is supplied); task_finished's
# own no-op default accepts Any, so this is safe.
task = cast('Callable[..., T]', _task_noop)
with self.pool_lock:
self._execute(
+9 -2
View File
@@ -27,9 +27,12 @@ from ocrmypdf.exceptions import InputFileError
from ocrmypdf.helpers import remove_all_log_handlers
if TYPE_CHECKING:
from logging import LogRecord
from typing import TypeAlias
Queue: TypeAlias = multiprocessing.queues.Queue | queue.Queue
Queue: TypeAlias = (
multiprocessing.queues.Queue[LogRecord | None] | queue.Queue[LogRecord | None]
)
UserInit: TypeAlias = Callable[[], None]
WorkerInit: TypeAlias = Callable[[Queue, UserInit, int], None]
@@ -99,7 +102,9 @@ def thread_init(q: Queue, user_init: UserInit, loglevel) -> None:
return
def setup_executor(use_threads: bool) -> tuple[Queue, Executor, WorkerInit]:
def setup_executor(
use_threads: bool,
) -> tuple[Queue, FuturesExecutorClass, WorkerInit]:
if not use_threads:
# Some execution environments like AWS Lambda and Termux do not support
# semaphores. Check if semaphore support is available, and if not, fall back
@@ -112,6 +117,8 @@ def setup_executor(use_threads: bool) -> tuple[Queue, Executor, WorkerInit]:
except ImportError:
use_threads = True
loq_queue: Queue
executor_class: FuturesExecutorClass
if use_threads:
loq_queue = queue.Queue(-1)
executor_class = ThreadPoolExecutor