OakEmu-RL v0.1.0

Appearance

PPOActorStepBuffers ​

python
PPOActorStepBuffers(observations: np.ndarray, actions: np.ndarray, log_probs: np.ndarray, values: np.ndarray, rewards: np.ndarray, dones: np.ndarray, trainable: np.ndarray) -> None

Mutable per-step PPO actor arrays used for action selection and rollout writes.

Source

Properties ​

stores_rollout_step ​

python
stores_rollout_step: bool

Return whether any environment in this step contributes learner data.

Returns:

  • bool – True when PPO will consume at least one environment step.

Source

Methods ​

add_to_rollout_buffer ​

python
add_to_rollout_buffer(rollout_buffer: RolloutBuffer) -> None

Append the current step buffers to the rollout buffer.

Parameters:

  • rollout_buffer (RolloutBuffer) – The rollout buffer being filled by the actor.

Source

create ​

python
create(*, num_envs: int, observation_shape: tuple[int, ...], observation_dtype: np.dtype[Any]) -> PPOActorStepBuffers

Create the preallocated step buffers for one PPO actor.

Parameters:

  • num_envs (int) – Number of environments stepped in lockstep.
  • observation_shape (tuple[int, ...]) – Observation shape stored in the rollout buffer.
  • observation_dtype (dtype[Any]) – Observation dtype stored in the rollout buffer.

Returns:

  • PPOActorStepBuffers – The allocated step buffers.

Source

record_action_bundle_for_env_ids ​

python
record_action_bundle_for_env_ids(action_bundle: PPOActionBundle, *, env_ids: list[int], trainable: list[bool]) -> None

Copy one PPO action bundle into the step buffers for a subset of environments.

Parameters:

  • action_bundle (PPOActionBundle) – The action bundle produced by the actor model.
  • env_ids (list[int]) – Environment IDs aligned with the batched action bundle.
  • trainable (list[bool]) – Whether each environment should contribute PPO training data.

Source

record_step_result ​

python
record_step_result(env_id: int, reward: float, *, done: bool) -> None

Copy one environment result into the step buffers.

Parameters:

  • env_id (int) – The environment ID.
  • reward (float) – The reward for the environment step.
  • done (bool) – Whether the environment episode ended on this step.

Source

reset_action_selection ​

python
reset_action_selection() -> None

Reset per-step policy-routing flags before selecting actions.

Source

reset_step_results ​

python
reset_step_results() -> None

Clear per-step rewards and done flags.

Source

Values ​

actions ​

python
actions: np.ndarray

Source

dones ​

python
dones: np.ndarray

Source

log_probs ​

python
log_probs: np.ndarray

Source

observations ​

python
observations: np.ndarray

Source

rewards ​

python
rewards: np.ndarray

Source

trainable ​

python
trainable: np.ndarray

Source

values ​

python
values: np.ndarray

Source