pynenc.broker.base_broker

Module Contents

Classes

BaseBroker

An Abstract Base Class for Message Brokers in Pynenc**

API

class pynenc.broker.base_broker.BaseBroker(app: pynenc.app.Pynenc)

Bases: abc.ABC

An Abstract Base Class for Message Brokers in Pynenc**

This class serves as the foundational structure for implementing various message brokers. It currently supports a simple FIFO queue and is extendable for more complex functionalities like priority queues and integration with different databases and message queues.

Parameters:

app (Pynenc) – A reference to the Pynenc application.

Note

The BaseBroker is currently implemented with an in-memory queue (MemBroker) for testing and demonstration, and a Redis-backed queue (RedisBroker) for production use.

Attention

The implementation is currently limited to a FIFO queue. Future enhancements will include support for priority queues and compatibility with other databases and message queues like RabbitMQ.

Hint

The class is designed to be flexible and expandable, allowing for easy integration of additional features and message brokers in the future.

See also

For more advanced or production-ready features, refer to the specific implementations like RedisBroker.

The route_call method creates a new invocation and routes it, demonstrating a basic usage of the broker.

Examples

    # Assuming `app` is an instance of Pynenc and `call` is a valid Call object
    broker = BaseBroker(app)
    invocation = broker.route_call(call)

Initialization

conf() → pynenc.conf.config_broker.ConfigBroker

abstract route_invocation(invocation: pynenc.invocation.dist_invocation.DistributedInvocation) → None

Abstract method for routing a given invocation.

This method should define the process of handling and dispatching a given DistributedInvocation within the broker system. Implementations might involve sending the invocation to a queue or handling it internally.

Parameters:

invocation (DistributedInvocation) – The invocation to be routed.

abstract route_invocations(invocations: list[pynenc.invocation.dist_invocation.DistributedInvocation]) → None

Routes multiple invocations at once.

This method is used for batch processing of invocations to improve performance when parallelizing large numbers of tasks.

Default implementation sequentially routes each invocation. Subclasses can override this with more efficient batch processing implementations.

Parameters:

invocations (list[DistributedInvocation]) – The invocations to be routed.

abstract retrieve_invocation() → Optional[pynenc.invocation.dist_invocation.DistributedInvocation]

Method to retrieve a distributed invocation.

Implementations of this method should detail how to retrieve the next available invocation from the broker’s queue or storage system. It is expected to return a DistributedInvocation if one is available, or None if the queue is empty.

Returns:

The next invocation to be processed, or None.

abstract count_invocations() → int

Method to count the number of invocations in the queue.

This method should return the current count of pending invocations in the broker’s queue. It’s useful for monitoring and managing the queue’s state.

Returns:

The number of invocations in the queue.

abstract purge() → None

Method to purge the message queue.

This method is intended to clear or reset the state of the broker’s queue, removing all pending invocations. It’s crucial for error handling and managing the queue in specific situations like maintenance or reset.

route_call(call: Call[Params, Result]) → pynenc.invocation.dist_invocation.DistributedInvocation[pynenc.types.Params, pynenc.types.Result]

Creates and routes a new DistributedInvocation based on the given call.

This method instantiates a DistributedInvocation with the provided call and the current invocation context. It then routes this invocation using the route_invocation method. This demonstrates the basic use of the broker’s functionality.

Parameters:

call (Call[Params, Result]) – The call object to be transformed into an invocation.

Returns:

The routed invocation.

Note

The method also logs the routing process for debugging purposes.