pynenc.runner.persistent_process_runner

Module Contents

Classes

PersistentProcessRunner

PersistentProcessRunner maintains a fixed number of processes that continuously run tasks.

Functions

persistent_process_main

Main function for persistent process that executes invocations sequentially.

API

pynenc.runner.persistent_process_runner.persistent_process_main(app: pynenc.app.Pynenc, *, runner_cache: dict, stop_event: multiprocessing.synchronize.Event, parent_runner_ctx_json: str, child_runner_id: str) → None

Main function for persistent process that executes invocations sequentially.

The child_runner_id is pre-generated by the parent process before spawning, allowing the parent to track which runner_ids are still alive via OS-level process checks. The parent reports heartbeats for alive children in its main loop.

Parameters:

• app (Pynenc) – The Pynenc application instance.

• runner_cache (dict) – Shared cache dictionary across all processes.

• stop_event (Event) – Multiprocessing event to signal graceful shutdown.

• parent_runner_ctx_json (str) – JSON serialized parent runner context.

• child_runner_id (str) – Pre-generated runner_id for this child worker.

class pynenc.runner.persistent_process_runner.PersistentProcessRunner(app: pynenc.app.Pynenc, runner_cache: dict | None = None, runner_context: pynenc.runner.runner_context.RunnerContext | None = None)

Bases: pynenc.runner.base_runner.BaseRunner

PersistentProcessRunner maintains a fixed number of processes that continuously run tasks.

This runner spawns child worker processes (PPRWorker) that execute invocations. The parent pre-generates runner_ids for children before spawning, enabling parent-based health reporting via OS-level process checks.

Initialization

conf() → pynenc.conf.config_runner.ConfigPersistentProcessRunner

static mem_compatible() → bool

Indicates if the runner supports in-memory components.

property max_parallel_slots: int

Returns the maximum number of concurrent processes.

get_active_child_runner_ids() → list[str]

Returns runner_ids of alive child workers for parent-based health reporting.

Returns:

List of runner_ids for child workers with alive processes.

_log_shutdown(signum: int | None) → None

static _ensure_spawn_start_method() → None

Set multiprocessing start method to ‘spawn’ if not already set (needed on macOS).

_on_start() → None

Initializes the runner and spawns initial processes.

_spawn_persistent_process() → str | None

Spawns a new persistent process and returns its runner_id.

Pre-generates the child runner_id before spawning so the parent can track which runner_ids are still alive via OS-level process checks.

Returns:

The runner_id of the spawned child, or None if spawn failed.

_terminate_all_processes() → None

Terminates all running processes with graceful shutdown attempt.

_on_stop() → None

Cleans up all resources when runner stops.

_on_stop_runner_loop() → None

Handles immediate stop from signal.

runner_loop_iteration() → None

Maintains the configured number of running processes.

_waiting_for_results(running_invocation_id: pynenc.identifiers.invocation_id.InvocationId, result_invocation_ids: list[pynenc.identifiers.invocation_id.InvocationId], runner_args: dict[str, Any] | None = None) → None

In this simplified version, we don’t pause/resume processes. The invocation will just be marked as paused and the process will continue with other invocations.