pynenc.conf.config_orchestrator

Module Contents

Classes

ConfigOrchestrator	Main config of the orchestrator components.
ConfigOrchestratorRedis	Specific Configuration for the Redis Orchestrator

API

class pynenc.conf.config_orchestrator.ConfigOrchestrator(config_values: Optional[dict[str, Any]] = None, config_filepath: Optional[str] = None)

Bases: pynenc.conf.config_base.ConfigPynencBase

Main config of the orchestrator components.

Variables:

• cycle_control (ConfigField[bool]) – This boolean flag enables the orchestrator to detect cycles of calls to subtasks. For example, if task1 calls task2 and task2 calls back to task1, this can create an endless loop. The cycle control functionality is enabled by default to prevent such scenarios. Users can choose to disable it if needed.

• blocking_control (ConfigField[bool]) – This boolean flag activates control over tasks that are blocking on other tasks. If a task invocation is waiting on another invocation, it notifies the runner, which temporarily removes it from the processing queue and uses the slot for another task invocation. Once the required invocation finishes, the dependent invocation is placed back into the run queue. This feature also prioritizes invocations that have many dependencies over new ones, ensuring efficient task management.

• auto_final_invocation_purge_hours (ConfigField[float]) – This float value, defaulting to 24.0 hours, sets the duration after which the orchestrator purges all invocations older than the specified time. This purge mechanism helps keep the orchestrator lightweight and fast, as it should ideally operate with minimal latency. Detailed information about the invocations is stored in the result backend, which can handle more data and afford to be slower.

Initialization

cycle_control

‘ConfigField(…)’

blocking_control

‘ConfigField(…)’

auto_final_invocation_purge_hours

‘ConfigField(…)’

class pynenc.conf.config_orchestrator.ConfigOrchestratorRedis(config_values: Optional[dict[str, Any]] = None, config_filepath: Optional[str] = None)

Bases: pynenc.conf.config_orchestrator.ConfigOrchestrator, pynenc.conf.config_redis.ConfigRedis

Specific Configuration for the Redis Orchestrator

Variables:

• max_pending_resolution_threads (ConfigField[int]) – This integer value controls the maximum number of worker threads used by the ThreadPoolExecutor for resolving PENDING status invocations. When an invocation enters PENDING state, a background thread is used to check if it needs to be resolved (when the pending timeout expires). This limits the number of concurrent Redis connections used for this process, helping to prevent connection pool exhaustion while still allowing enough parallelism to handle many pending invocations. Default is 50 threads.

• redis_retry_max_attempts (ConfigField[int]) – Maximum number of retry attempts for Redis operations that encounter specific exceptions such as StatusNotFound. After this many failed attempts, the operation will raise the exception. Default is 3 attempts.

• redis_retry_base_delay_sec (ConfigField[float]) – Base delay in seconds between retry attempts. The retry system uses exponential backoff, where each retry waits progressively longer (base_delay * 2^attempt). Default is 0.1 seconds.

• redis_retry_max_delay_sec (ConfigField[float]) – Maximum delay between retry attempts in seconds, regardless of the exponential backoff calculation. This ensures that even after many retries, the delay won’t exceed a reasonable upper bound. Default is 1.0 second.

Initialization

max_pending_resolution_threads

‘ConfigField(…)’

redis_retry_max_attempts

‘ConfigField(…)’

redis_retry_base_delay_sec

‘ConfigField(…)’

redis_retry_max_delay_sec

‘ConfigField(…)’