pynenc.runner.thread_runner

Module Contents

Classes

ThreadInfo
ThreadRunner	ThreadRunner is a concrete implementation of BaseRunner that executes tasks in separate threads.

API

class pynenc.runner.thread_runner.ThreadInfo

Bases: typing.NamedTuple

thread: threading.Thread

None

invocation: pynenc.invocation.dist_invocation.DistributedInvocation

None

class pynenc.runner.thread_runner.ThreadRunner(app: pynenc.app.Pynenc, runner_cache: Optional[dict] = None, extra_id: Optional[str] = None)

Bases: pynenc.runner.base_runner.BaseRunner

ThreadRunner is a concrete implementation of BaseRunner that executes tasks in separate threads.

It manages task invocations, handling their execution and lifecycle within individual threads. This runner is suitable for I/O-bound tasks and scenarios where shared memory between tasks is required.

Initialization

wait_invocation: set[pynenc.invocation.dist_invocation.DistributedInvocation]

None

threads: dict[str, pynenc.runner.thread_runner.ThreadInfo]

None

max_threads: int

None

waiting_threads: int

None

final_invocations: OrderedDict[DistributedInvocation, None]

None

conf() → pynenc.conf.config_runner.ConfigThreadRunner

property cache: dict

The cache for the ThreadRunner instance.

Returns:

A dictionary representing the cache for the ThreadRunner.

static mem_compatible() → bool

Indicates if the runner is compatible with in-memory components.

Returns:

True, as each task is executed in a separate thread with shared memory.

property max_parallel_slots: int

The maximum number of parallel tasks that the runner can handle.

Returns:

An integer representing the maximum number of parallel tasks.

_on_start() → None

Internal method called when the ThreadRunner starts. Initializes the data infrastructures for managing invocations and threads.

_on_stop() → None

Internal method called when the ThreadRunner stops. Joins all running threads and updates their invocation statuses.

_on_stop_runner_loop() → None

Internal method called after receiving a signal to stop the runner loop.

property available_threads: int

Returns the number of available thread slots for new invocations. Joins finished threads so they no longer count against the limit.

Returns:

An integer representing available thread slots.

runner_loop_iteration() → None

Executes one iteration of the ThreadRunner loop. Handles the execution and monitoring of task invocations in separate threads.

_waiting_for_results(running_invocation: pynenc.invocation.dist_invocation.DistributedInvocation, result_invocations: list[pynenc.invocation.dist_invocation.DistributedInvocation], runner_args: Optional[dict[str, Any]] = None) → None

Handles invocations waiting for results by polling a local final cache instead of pausing threads.

Parameters:

• running_invocation – The invocation that is waiting for results.

• result_invocations – A list of invocations whose results are being awaited.

• runner_args – Additional arguments required for the ThreadRunner.