OakEmu-RL v0.1.0

Appearance

DQNStateAdapter ​

python
DQNStateAdapter(config: DQNExperimentConfiguration, learner_model: SupportsNoiseReset, learner_model_container: ThreadSafeModel, target_model: DQNModel, actor_model: DQNModel, optimizer: torch.optim.Optimizer, trainer: TrainerRuntime, learner: LearnerRuntime, tester: TesterRuntime | None, tester_model: DQNModel | None) -> None

Captures and restores DQN-specific runtime state.

Initialize the DQN state adapter.

Parameters:

  • config (DQNExperimentConfiguration) – DQN experiment configuration.
  • learner_model (SupportsNoiseReset) – Learner model module.
  • learner_model_container (ThreadSafeModel) – Thread-safe learner model wrapper.
  • target_model (DQNModel) – Target model module.
  • actor_model (DQNModel) – Actor model module.
  • optimizer (Optimizer) – Optimizer used by the learner.
  • trainer (TrainerRuntime) – Trainer runtime object.
  • learner (LearnerRuntime) – Learner runtime object.
  • tester (TesterRuntime | None) – Optional tester runtime object.
  • tester_model (DQNModel | None) – Optional tester model module.

Source

Methods ​

create_checkpoint_files ​

python
create_checkpoint_files(*, elapsed_time: float, total_rollbacks: int, replay_buffer: ReplayBuffer) -> list[CheckpointFile]

Build the DQN-specific checkpoint files for the current runtime state.

Parameters:

  • elapsed_time (float) – Total elapsed experiment time.
  • total_rollbacks (int) – Total number of performed rollbacks.
  • replay_buffer (ReplayBuffer) – Replay buffer to optionally store alongside DQN state.

Returns:

  • list[CheckpointFile] – The list of checkpoint files to save.

Source

restore_for_rollback ​

python
restore_for_rollback(checkpoint: Checkpoint) -> int

Restore only the learner/target/optimizer state required for automatic rollback.

Parameters:

  • checkpoint (Checkpoint) – The checkpoint selected for rollback.

Returns:

  • int – The actor step recorded inside the restored checkpoint.

Source

restore_full ​

python
restore_full(checkpoint: Checkpoint) -> DQNRestoreMetadata

Restore full experiment runtime state from a checkpoint.

Parameters:

  • checkpoint (Checkpoint) – The checkpoint to restore from.

Returns:

  • DQNRestoreMetadata – Metadata needed by the experiment after restore.

Source