OakEmu-RL v0.1.0

Appearance

Trainer ​

python
Trainer(context: RuntimeContext[DQNExperimentConfiguration], actor_model: DQNModel, learner_model: ThreadSafeModel, replay_buffer: ReplayBuffer, replay_writer: ReplayWriter, reward_shaping: RewardShapingPipeline | None = None, env_factory: Callable[[], OakEmuMultiEnv] | None = None) -> None

Trainer component responsible for interacting with the environment and collecting transitions.

Initialize the Trainer.

Parameters:

  • context (RuntimeContext[DQNExperimentConfiguration]) – The shared runtime context.
  • actor_model (DQNModel) – The local DQN model for action selection.
  • learner_model (ThreadSafeModel) – The learner model container (for weight sync).
  • replay_buffer (ReplayBuffer) – The experience replay buffer.
  • replay_writer (ReplayWriter) – Dedicated replay-writer runtime for replay inserts.
  • reward_shaping (RewardShapingPipeline | None) – Optional actor-owned reward-shaping pipeline.
  • env_factory (Callable[[], OakEmuMultiEnv] | None) – Optional multi-environment factory; see Agent.

Source

Properties ​

replay_buffer ​

python
replay_buffer: ReplayBuffer

Get the replay buffer.

Returns:

  • ReplayBuffer – The ReplayBuffer instance.

Source

replay_queue_depth ​

python
replay_queue_depth: int

Get the number of replay batches waiting to be inserted.

Returns:

  • int – The replay queue depth.

Source

staged_transition_count ​

python
staged_transition_count: int

Get the number of transitions currently staged on the actor.

Returns:

  • int – The number of locally buffered transitions.

Source

steps ​

python
steps: int

Get the total number of environment steps.

Returns:

  • int – The total step count.

Source

trajectory_runtime ​

python
trajectory_runtime: TrajectoryEpisodeCoordinator

Get the shared trajectory runtime coordinator.

Returns:

  • TrajectoryEpisodeCoordinator – The trainer's trajectory runtime coordinator.

Source

Methods ​

freeze_replay_pipeline ​

python
freeze_replay_pipeline() -> AbstractContextManager[None]

Freeze replay submission, flush actor-side staging, and drain the replay writer.

Returns:

  • AbstractContextManager[None] – A context manager that keeps replay submission frozen for the duration of the block.

Source

replace_replay_buffer ​

python
replace_replay_buffer(replay_buffer: ReplayBuffer) -> None

Replace the replay buffer and reconnect the replay writer.

Parameters:

  • replay_buffer (ReplayBuffer) – The new replay buffer.

Source

restore_runtime_state ​

python
restore_runtime_state(state: ActorRuntimeState) -> None

Restore checkpointed trainer counters and trajectory runtime state.

Not named restore_state (unlike Agent.restore_state): the trainer restores a strictly wider payload than the base episode-totals state, so this is a distinct method rather than an incompatibly-typed override.

Parameters:

  • state (ActorRuntimeState) – The trainer runtime state to restore.

Source

run ​

python
run() -> None

Main loop for the Trainer, interacting with the environment and adding data to the replay buffer.

Source