Skip to content

InferenceBatcher ​

python
InferenceBatcher(action_selector: ActionSelector, action_masker: ActionMasker, inference_batch_size: int) -> None

Buffers pending observations and runs batched action selection.

Initialize the inference batcher.

Parameters:

  • action_selector (ActionSelector) – Selector used to compute actions for a batch of states.
  • action_masker (ActionMasker) – Action masker applied during action selection.
  • inference_batch_size (int) – Number of queued items required before automatic flush.

Source

Properties ​

action_selector ​

python
action_selector: ActionSelector

Get the action selector used for action selection.

Returns:

  • ActionSelector – The ActionSelector instance.

Source

pending_count ​

python
pending_count: int

Get the number of queued inference items.

Returns:

  • int – The number of pending observations.

Source

Methods ​

flush ​

python
flush(on_inference_complete: Callable[[int, int], None]) -> None

Run action selection for all queued observations.

Parameters:

  • on_inference_complete (Callable[[int, int], None]) – Callback invoked for each (env_id, action) pair.

Source

queue ​

python
queue(env_id: int, state: np.ndarray) -> bool

Queue an observation for the next batched inference call.

Parameters:

  • env_id (int) – The environment ID.
  • state (ndarray) – The state observation to batch.

Returns:

  • bool – True if the batch has reached the configured flush size.

Source