OakEmu-RL v0.1.0

Appearance

Learner ​

python
Learner(context: RuntimeContext[DQNExperimentConfiguration], learner_model: ThreadSafeModel, target_model: DQNModel, replay_buffer: ReplayBuffer, optimizer: torch.optim.Optimizer, lr_schedule: Schedule, action_masker: ActionMasker | None = None, reward_shaping: RewardShapingPipeline | None = None) -> None

Learner component responsible for updating the DQN model using transitions from the replay buffer.

Initialize the Learner.

Parameters:

  • context (RuntimeContext[DQNExperimentConfiguration]) – The shared runtime context.
  • learner_model (ThreadSafeModel) – The learner model container (with lock).
  • target_model (DQNModel) – The target DQN model.
  • replay_buffer (ReplayBuffer) – The experience replay buffer.
  • optimizer (Optimizer) – The PyTorch optimizer.
  • lr_schedule (Schedule) – Schedule for the learning rate.
  • action_masker (ActionMasker | None) – Optional ActionMasker to restrict next-action selection in target Q-value computation to only actions the actor can take.
  • reward_shaping (RewardShapingPipeline | None) – Optional learner-owned reward-shaping pipeline.

Source

Properties ​

action_masker ​

python
action_masker: ActionMasker | None

Get the action masker.

Source

config ​

python
config: DQNExperimentConfiguration

Get the experiment configuration.

Source

iterations ​

python
iterations: int

Get the number of learner iterations performed.

Source

replay_buffer ​

python
replay_buffer: ReplayBuffer

Get the replay buffer.

Source

target_updates ​

python
target_updates: int

Get the number of target model updates performed.

Source

total_skipped_updates ​

python
total_skipped_updates: int

Get the total number of skipped PER priority updates.

Source

Methods ​

replace_replay_buffer ​

python
replace_replay_buffer(replay_buffer: ReplayBuffer) -> None

Replace the replay buffer used for sampling.

Parameters:

  • replay_buffer (ReplayBuffer) – The new replay buffer.

Source

restore_state ​

python
restore_state(state: DQNLearnerRuntimeState) -> None

Restore checkpointed learner counters.

Parameters:

  • state (DQNLearnerRuntimeState) – The learner runtime state to restore.

Source

run ​

python
run(stop_event: Event, current_step: Callable[[], int], policy: CheckpointPolicy) -> None

Main learner loop: repeatedly perform learning steps, notify the tester on target updates, feed automatic rollback monitoring, and manage periodic/final checkpointing.

Parameters:

  • stop_event (Event) – Signals when training should stop.
  • current_step (Callable[[], int]) – Returns the current global environment step count.
  • policy (CheckpointPolicy) – The checkpoint/rollback seam this loop drives.

Source

step ​

python
step(env_steps: int) -> bool

Perform a single learning step.

Parameters:

  • env_steps (int) – Total number of environment steps performed across all actors.

Returns:

  • bool – True if a learning step was performed, False if throttled or not enough data.

Source