OakEmu-RL v0.1.0

Appearance

UniformReplayBuffer ​

Experience replay buffer for storing and sampling transitions. Uses pre-allocated numpy arrays for efficiency.

Source

Methods ​

add ​

python
add(state: list[np.ndarray], action: list[int], reward: list[float], next_state: list[np.ndarray], done: list[bool], n_steps: list[int], reward_shaping_metadata: Mapping[str, np.ndarray | int | float | list[int | float]] | None = None) -> None

Add a batch of transitions to the buffer.

Parameters:

  • state (list[ndarray]) – List of states.
  • action (list[int]) – List of actions.
  • reward (list[float]) – List of rewards.
  • next_state (list[ndarray]) – List of next states.
  • done (list[bool]) – List of done flags.
  • n_steps (list[int]) – List of step counts.
  • reward_shaping_metadata (Mapping[str, ndarray | int | float | list[int | float]] | None) – Optional compact reward-shaping metadata columns.

Source

load ​

python
load(path: Path) -> None

Load the buffer state from a file using numpy's compressed format.

Parameters:

  • path (Path) – Path to the save file.

Source

load_from_path ​

python
load_from_path(path: Path) -> UniformReplayBuffer

Create a new UniformReplayBuffer from a saved file using numpy's compressed format.

Parameters:

  • path (Path) – Path to the save file.

Returns:

  • UniformReplayBuffer – A new UniformReplayBuffer instance.

Source

sample_into ​

python
sample_into(batch_size: int, *, arrays_out: HostBatchArrays, reward_shaping_metadata_out: dict[str, np.ndarray] | None = None, alpha: float = 0.6, beta: float = 0.4) -> tuple[None, None]

Sample a random batch of transitions directly into caller-provided output arrays.

Parameters:

  • batch_size (int) – Number of transitions to sample.
  • arrays_out (HostBatchArrays) – Output arrays for the sampled batch; arrays_out.weights is unused.
  • reward_shaping_metadata_out (dict[str, ndarray] | None) – Optional output arrays for sampled reward-shaping metadata.
  • alpha (float) – Unused for uniform replay.
  • beta (float) – Unused for uniform replay.

Returns:

  • tuple[None, None] – (None, None) because uniform replay does not track sampled indices/versions.

Source

save ​

python
save(path: Path) -> None

Save the buffer state to a file using numpy's compressed format.

Parameters:

  • path (Path) – Path to the save file.

Source