pynenc.runner.base_runner

Module Contents

Classes

BaseRunner	The BaseRunner class defines the interface for a runner that executes task invocations.
DummyRunner	This runner is a placeholder for the Pynenc app. It will be used when the app is defined in any other Python environment than a Pynenc runner.

API

class pynenc.runner.base_runner.BaseRunner(app: pynenc.app.Pynenc)

Bases: abc.ABC

The BaseRunner class defines the interface for a runner that executes task invocations.

It interacts with various components of the Pynenc system, like the broker and orchestrator, and is responsible for handling the execution and life cycle of task invocations.

• The runner’s behavior can vary depending on the execution environment (e.g., subprocess, async, cloud function, multiprocessing).

• It is designed to be subclassed for specific execution environments.

Initialization

runner_id() → str

Unique identifier for the runner instance.

Returns:

A string representing the unique identifier of the runner.

conf() → pynenc.conf.config_runner.ConfigRunner

abstract static mem_compatible() → bool

Indicates if the runner is compatible with in-memory components.

Important

In memory components can only be used for testing purposes in shared memory space.

Returns:

True if compatible, False otherwise.

abstract property max_parallel_slots: int

The maximum number of parallel tasks that the runner can handle.

Returns:

An integer representing the maximum number of parallel tasks.

abstract _on_start() → None

This method is called when the runner starts

on_start() → None

This method is called when the runner starts

abstract _on_stop() → None

This method is called when the runner stops

on_stop() → None

This method is called when the runner stops

abstract runner_loop_iteration() → None

One iteration of the runner loop. Subclasses should implement this method to process invocations.

abstract _on_stop_runner_loop() → None

This method is called after the runner loop signal is received

stop_runner_loop(signum: Optional[int] = None, frame: Optional[types.FrameType] = None) → None

Stops the runner loop, typically in response to a signal.

Parameters:

• signum – Signal number.

• frame – Frame object at the time the signal was received.

abstract waiting_for_results(running_invocation: Optional[pynenc.invocation.dist_invocation.DistributedInvocation], result_invocation: list[pynenc.invocation.dist_invocation.DistributedInvocation], runner_args: Optional[dict[str, Any]] = None) → None

Method called when an invocation is waiting for results from other invocations.

Note

This method is called from the result method of an invocation

The runner has the oportunity to define the waiting behaviour of the running invocation in this method Otherwise the running invocation will infinetely loop until the result invocation is ready

Note

The running invocation may be None, when the result was called from outside a runner (e.g. user environment)
In that case will be handle by the DummyRunner (default in the pynenc app to handle this cases)

Subclasses can define the waiting behavior of the running invocation in this method.

Parameters:

• running_invocation – The invocation that is waiting for results.

• result_invocation – A list of invocations whose results are being awaited.

• runner_args – Additional arguments passed to the runner, specific to the runner’s implementation.

run() → None

Starts the runner, initiating its main loop.

class pynenc.runner.base_runner.DummyRunner(app: pynenc.app.Pynenc)

Bases: pynenc.runner.base_runner.BaseRunner

This runner is a placeholder for the Pynenc app. It will be used when the app is defined in any other Python environment than a Pynenc runner.

Examples include:

• A script that defines the app, decorates some tasks, routes them, and then finishes. Such a script does not plan to run anything itself but triggers tasks that will later run in actual runners.

Initialization

static mem_compatible() → bool

_on_start() → None

_on_stop() → None

_on_stop_runner_loop() → None

runner_loop_iteration() → None

property max_parallel_slots: int

waiting_for_results(running_invocation: Optional[pynenc.invocation.dist_invocation.DistributedInvocation], result_invocation: list[pynenc.invocation.dist_invocation.DistributedInvocation], runner_args: Optional[dict[str, Any]] = None) → None