OakEmu-RL v0.1.0

Appearance

PPOStateAdapter ​

python
PPOStateAdapter(config: PPOExperimentConfiguration, learner_model: torch.nn.Module, learner_model_container: ThreadSafeModel, actor_model: torch.nn.Module, optimizer: torch.optim.Optimizer, trainer: ActorRuntime, learner: LearnerRuntime, tester: TesterRuntime | None, tester_model: torch.nn.Module | None) -> None

Captures and restores PPO-specific runtime state.

Initialize the PPO state adapter.

Parameters:

  • config (PPOExperimentConfiguration) – PPO experiment configuration.
  • learner_model (Module) – Learner model module.
  • learner_model_container (ThreadSafeModel) – Thread-safe learner model wrapper.
  • actor_model (Module) – Actor model module.
  • optimizer (Optimizer) – Optimizer used by the learner.
  • trainer (ActorRuntime) – Trainer runtime object.
  • learner (LearnerRuntime) – Learner runtime object.
  • tester (TesterRuntime | None) – Optional tester runtime object.
  • tester_model (Module | None) – Optional tester model module.

Source

Methods ​

create_checkpoint_files ​

python
create_checkpoint_files(*, elapsed_time: float, total_rollbacks: int, run_name: str) -> list[CheckpointFile]

Build the PPO-specific checkpoint files for the current runtime state.

Parameters:

  • elapsed_time (float) – Total elapsed experiment time.
  • total_rollbacks (int) – Total number of performed rollbacks.
  • run_name (str) – The run name to persist.

Returns:

  • list[CheckpointFile] – The list of checkpoint files to save.

Source

restore_for_rollback ​

python
restore_for_rollback(checkpoint: Checkpoint) -> int

Restore only the learner/optimizer state required for automatic rollback.

Parameters:

  • checkpoint (Checkpoint) – The checkpoint selected for rollback.

Returns:

  • int – The actor step recorded inside the restored checkpoint.

Source

restore_full ​

python
restore_full(checkpoint: Checkpoint) -> PPORestoreMetadata

Restore full experiment runtime state from a checkpoint.

Parameters:

  • checkpoint (Checkpoint) – The checkpoint to restore from.

Returns:

  • PPORestoreMetadata – Metadata needed by the experiment after restore.

Source