OakEmu-RL v0.1.0

Appearance

EpisodeScheduler ​

python
EpisodeScheduler(metrics: MetricsTracker, env_states: list[TEnvState], trajectory_runtime: TrajectoryEpisodeCoordinator, reset_parameter_curriculum: ResetParameterCurriculum | None = None, replay_observation_cache: ReplayObservationCache | None = None, max_materialisations_in_flight: int = 0, restart_pool: RestartPool | None = None) -> None

Schedules the next episode source for each actor environment.

On each environment's turn to start a new episode, first offers the slot to a queued BC materialisation replay (a learner-side data fetch, not training experience); otherwise chooses and enqueues the next real episode start (midpoint/random/normal) via the trajectory runtime. Also completes materialisation replays when their results arrive.

BC materialisation, the reset-parameter curriculum, and the materialisation-limit are all optional: an actor that doesn't use any of them gets a scheduler that reduces to a single call into the trajectory runtime per episode start.

Initialize the scheduler.

Parameters:

  • metrics (MetricsTracker) – Metrics tracker, for replay-cache metrics.
  • env_states (list[[EpisodeScheduler[TEnvState]](#mrkwatkins.oakemu_rl.framework.EpisodeScheduler.EpisodeScheduler[TEnvState])]) – The actor's per-environment state list (shared by reference).
  • trajectory_runtime (TrajectoryEpisodeCoordinator) – The trajectory coordinator to source episode starts from.
  • reset_parameter_curriculum (ResetParameterCurriculum | None) – Optional curriculum choosing reset parameters per episode.
  • replay_observation_cache (ReplayObservationCache | None) – Optional shared replay observation cache.
  • max_materialisations_in_flight (int) – Maximum number of concurrent BC materialisation replays.
  • restart_pool (RestartPool | None) – Optional pool of restart points harvested from played episodes.

Source

Methods ​

enqueue_episode_start ​

python
enqueue_episode_start(env: OakEmuMultiEnv, env_id: int, *, step: int) -> None

Choose and enqueue the next episode source for an environment.

Parameters:

  • env (OakEmuMultiEnv) – The multi-environment to enqueue on.
  • env_id (int) – The environment ID.
  • step (int) – The step to use for trajectory-source scheduling.

Source

handle_replay_result ​

python
handle_replay_result(env: OakEmuMultiEnv, result: MultiEnvResult, *, step: int) -> None

Complete one BC materialisation replay: cache its observations and resume normal episodes.

Materialisation replays are data fetches for the learner-side BC sampler, not training experience: they contribute no steps, no episode, and no action counts.

Parameters:

  • env (OakEmuMultiEnv) – The multi-environment the replay ran on.
  • result (MultiEnvResult) – The completed materialisation replay result.
  • step (int) – The step to use for replay-cache metrics and the resumed episode start.

Source

set_bc_materialisation_source ​

python
set_bc_materialisation_source(source: BCMaterialisationSource | None) -> None

Attach the learner-side BC sampler's materialisation queue.

Parameters:

  • source (BCMaterialisationSource | None) – The request source, or None to disable materialisation.

Source