OakEmu-RL v0.1.0

Appearance

ActionPolicy ​

python
ActionPolicy(action_masker: ActionMasker, exploration_strategy: ExplorationStrategy, actor_model: DQNModel, learner_model_container: ThreadSafeModel, num_steps_sampled_before_learning_starts: int, *, queue_for_inference: Callable[[int, np.ndarray], None], record_random_step: Callable[[], None]) -> None

Selects actions for each actor environment and enqueues them for execution.

During warmup, random-controlled trajectory replay, or epsilon-greedy exploration, an action is sampled and enqueued immediately; otherwise the environment is queued for batched greedy inference, with the resulting action applied once the batch completes.

Initialize the action policy.

Parameters:

  • action_masker (ActionMasker) – Restricts sampled/greedy actions to the game's training action set.
  • exploration_strategy (ExplorationStrategy) – Decides when to explore and the current exploration value.
  • actor_model (DQNModel) – The local model, for exploration-driven noise resets.
  • learner_model_container (ThreadSafeModel) – The learner model container, for weight sync after inference.
  • num_steps_sampled_before_learning_starts (int) – Steps before learning starts, during which actions are always sampled randomly regardless of the exploration strategy.
  • queue_for_inference (Callable[[int, ndarray], None]) – Queues an environment/observation for batched greedy inference.
  • record_random_step (Callable[[], None]) – Records a random-controlled trajectory-replay step.

Source

Methods ​

get_exploration_value ​

python
get_exploration_value(step: int) -> float | None

Get the current exploration value (epsilon), for metrics logging.

Parameters:

  • step (int) – The current environment step.

Returns:

  • float | None – The exploration value, or None for exploration strategies without one (for example noisy nets).

Source

on_inference_complete ​

python
on_inference_complete(env_state: EnvironmentState, action: int) -> None

Record the greedy action chosen by batched inference.

Parameters:

  • env_state (EnvironmentState) – The state for the environment that completed inference.
  • action (int) – The selected greedy action.

Source

reset_noise_before_inference ​

python
reset_noise_before_inference(step: int, num_environments: int) -> None

Reset the actor model's noise before a batched inference flush, if the exploration strategy calls for it.

Parameters:

  • step (int) – The current environment step.
  • num_environments (int) – The number of actor environments.

Source

select_and_enqueue_action ​

python
select_and_enqueue_action(env: OakEmuMultiEnv, env_id: int, env_state: EnvironmentState, *, step: int) -> None

Select an action and either enqueue it immediately or queue for batched inference.

During warmup or exploration, a random action is selected and enqueued immediately. Otherwise the environment is queued for batched greedy inference.

Parameters:

  • env (OakEmuMultiEnv) – The multi-environment instance.
  • env_id (int) – The environment ID.
  • env_state (EnvironmentState) – The state for this environment.
  • step (int) – The current environment step.

Source

sync_weights_after_inference ​

python
sync_weights_after_inference() -> None

Sync weights from the learner model if an update is available, resetting noise afterward if the exploration strategy calls for it.

Source