pynenc.runner.thread_runner#

Module Contents#

Classes#

ThreadInfo

ThreadRunner

ThreadRunner is a concrete implementation of BaseRunner that executes tasks in separate threads.

API#

class pynenc.runner.thread_runner.ThreadInfo[source]#

Bases: typing.NamedTuple

thread: threading.Thread#

None

invocation: pynenc.invocation.dist_invocation.DistributedInvocation#

None

class pynenc.runner.thread_runner.ThreadRunner(app: pynenc.app.Pynenc)[source]#

Bases: pynenc.runner.base_runner.BaseRunner

ThreadRunner is a concrete implementation of BaseRunner that executes tasks in separate threads.

It manages task invocations, handling their execution and lifecycle within individual threads. This runner is suitable for I/O-bound tasks and scenarios where shared memory between tasks is required.

Initialization

wait_conditions: dict[pynenc.invocation.dist_invocation.DistributedInvocation, threading.Condition]#

None

wait_invocation: dict[pynenc.invocation.dist_invocation.DistributedInvocation, set[pynenc.invocation.dist_invocation.DistributedInvocation]]#

None

threads: dict[str, pynenc.runner.thread_runner.ThreadInfo]#

None

max_threads: int#

None

waiting_threads: int#

None

conf() → pynenc.conf.config_runner.ConfigThreadRunner#
static mem_compatible() → bool[source]#

Indicates if the runner is compatible with in-memory components.

Returns:

True, as each task is executed in a separate thread with shared memory.

property max_parallel_slots: int#

The maximum number of parallel tasks that the runner can handle.

Returns:

An integer representing the maximum number of parallel tasks.

_on_start() → None[source]#

Internal method called when the ThreadRunner starts. Initializes the data structures for managing invocations and threads.

_on_stop() → None[source]#

Internal method called when the ThreadRunner stops. Joins all running threads and updates their invocation statuses.

_on_stop_runner_loop() → None[source]#

Internal method called after receiving a signal to stop the runner loop.

property available_threads: int#

Returns the number of available thread slots for new invocations.

Returns:

An integer representing available thread slots.

runner_loop_iteration() → None[source]#

Executes one iteration of the ThreadRunner loop. Handles the execution and monitoring of task invocations in separate threads.

waiting_for_results(running_invocation: Optional[pynenc.invocation.dist_invocation.DistributedInvocation], result_invocations: list[pynenc.invocation.dist_invocation.DistributedInvocation], runner_args: Optional[dict[str, Any]] = None) → None[source]#

Handles invocations that are waiting for results from other invocations. Pauses the current thread and registers it to wait for the results of specified invocations.

Parameters:

  • running_invocation – The invocation that is waiting for results.

  • result_invocations – A list of invocations whose results are being awaited.

  • runner_args – Additional arguments required for the ThreadRunner.