OakEmu-RL v0.1.0

Appearance

BackwardCurriculumConfiguration ​

A strict Salimans-&-Chen receding front over the archived route's segments.

Source

Values ​

action_granular ​

python
action_granular: bool = False

Make every rung ONE ACTION instead of one marker segment.

Restarts are minted by replaying the archived route through an emulator and snapshotting after every action (see build_action_restarts), so the front can sit at any offset into the route. This is Salimans & Chen's actual recipe -- their start point moved back a few frames at a time -- and it decouples rung size from marker spacing entirely. Measured need (2026-08-13): the walks-unaided rule (<~10 actions) held across two grids and two routes, and the ONE rung above it (the 22-action segment-7 gap on cavern 15) defeated pure PPO on three grids and two BC-pulse designs. Per-action rungs put every step of the route inside the walks-unaided range.

Wins from mid-segment fronts count for curriculum advancement and tail-delta measurement but are NOT stitched into the archive (the archived-prefix machinery is segment-aligned); segment-aligned offsets bank exactly as before, so route improvement is still observable at every marker.

Source

advance_mode ​

python
advance_mode: Literal['plateau', 'salimans_chen'] = 'plateau'

How the front decides to move.

plateau (original) -- advance when the front's win rate has stopped improving above a floor; retreat on outright failure; stall on repeated failure. Restarts are drawn from sample_window SHALLOWER rungs (easier tails behind the front) and only credit_window rungs at the front vote.

salimans_chen -- the paper's Algorithms 1 and 2, as literally as the machinery allows. Restarts are drawn uniformly from {front - sample_window, ..., front} -- the HARDER rungs ahead of the front (their D), so the next rungs are practised before they become the front. Every episode in that band votes. Success is "as good as the demo" -- a win in no more actions than the demo's own tail from that start (their return-comparison, expressed in actions since return is air-graded here). The front advances by step_back (their Delta) whenever the success fraction over window episodes reaches minimum_advance_rate (their rho); otherwise it holds. No retreat, no attempt counting, no plateau test, no stall. Adopted 2026-08-13 after the plateau rule advanced a per-action walk at 23% and the next rung collapsed to 0.

Source

clear_stall_on_load ​

python
clear_stall_on_load: bool = False

One-shot recovery switch: when restoring from a checkpoint, clear a stalled curriculum (stall flag, advance attempts, spent-pulse depths) so the walk resumes and pulses may refire. Use to resume a run whose stall has since been diagnosed and fixed; remove once resumed.

Source

credit_window ​

python
credit_window: int = Field(default=1, ge=1)

How many segments from the front may VOTE on whether the depth is mastered. Keep at 1.

Sampling and crediting are deliberately separate. Crediting the whole sample window lets shallower, easier restarts carry the front's win rate, so the curriculum advances on competence the policy does not have at the depth it is actually leaving. Measured on cavern 15 (2026-08-09): front 38 read 28-46% -- essentially all of it from segment 40, two from the exit and ~3 actions long -- then advancing to 37 dropped segment 40 out of the band and the true rate appeared as 0.0% with no wins at all. Three retreats and a stalled curriculum followed. Widening this hides exactly the signal the front exists to measure.

Source

enabled ​

python
enabled: bool = False

Whether to confine midpoint restarts to a receding front. Off leaves selection to the utility blend.

Source

initial_offset ​

python
initial_offset: int = Field(default=2, ge=2)

How many segments back from the exit the front starts.

Minimum 2: at 1 the front would sit on the final segment, whose end state is the win, so a restart there is a zero-action episode. At 2 the first depth is the shortest real problem there is -- on cavern 15, 3 actions. That matters more now that only the front is credited: with the old default of 4 the first credited depth was a 10-action tail learned cold, with no shallower rung to climb from. A tiny first depth also makes failure fast to see, since the front either moves within a few hundred thousand steps or it does not.

Source

max_advance_attempts ​

python
max_advance_attempts: int = Field(default=3, ge=1)

Failed attempts at one depth before the curriculum stops advancing altogether, bounding retreats.

Source

min_episodes_at_front ​

python
min_episodes_at_front: int = Field(default=5000, ge=1)

Episodes at a depth before it may be judged plateaued, walled or failed.

A GRACE PERIOD, and the rail that makes the other three usable. Immediately after an advance the policy has never seen the new depth, so its win rate is legitimately 0 -- that is acquisition, not collapse, and without this the two are indistinguishable. Measured on cavern 15 (2026-08-09): with no grace, the front advanced 38 -> 37, read 0% over its first window, retreated to 38, and repeated until max_advance_attempts stalled the whole curriculum at the depth it started from. Judge only after the policy has had a fair chance.

Source

min_initial_tail_actions ​

python
min_initial_tail_actions: int = Field(default=0, ge=0)

Deepen the initial front until the reference tail is at least this many actions (0 disables).

Folds trivial terminal segments into the first depth instead of training them alone. On the y/8 cavern-15 route the offset-2 first depth was a 2-action fall that a random policy wins 100% of the time -- it taught nothing while rewarding anything, and the policy it produced could not explore the next depth (the 2026-08-12 entropy trap). initial_offset remains the shallowest allowed front; this walks it deeper until the tail is a real problem. Pick a value whose depth a RANDOM policy wins ~1-10% of the time, measured from the seed's restores: high enough that real wins flow from episode one, rare enough that they carry signal (100% teaches nothing, ~0% amplifies nothing).

Source

min_tail_budget ​

python
min_tail_budget: int = Field(default=30, ge=1)

Floor for the action budget, so the shallowest depths still get room to manoeuvre.

Source

minimum_advance_rate ​

python
minimum_advance_rate: float = Field(default=0.5, ge=0.0, le=1.0)

Win rate a plateaued depth must reach before the front is allowed past it.

A FLOOR, not a target. An absolute target cannot work -- the front recedes but the tail grows, so the same number demands ~99% per-segment reliability by the deep end and becomes unreachable in every game. Advancement is therefore on a plateau (see plateau_patience); this is only the bar below which a plateau is read as a wall rather than as mastery, at which point the curriculum stops instead of forcing a policy that cannot solve the shorter problem to attempt a longer one.

Source

over_budget_penalty ​

python
over_budget_penalty: float = Field(default=-1.01, ge=-10.0, le=0.0)

Reward applied when an episode exceeds its budget, in post-normalisation units.

Must be negative. Truncation is folded into done and fed to GAE with no bootstrap, so a capped episode is treated as terminal with zero future value -- and the corner it is stuck in is genuinely worth about -5.0. Ending at zero would therefore make the trap look better than it is. Defaults to the death penalty, which is honest here: in this game taking too long really is death by air.

Source

plateau_delta ​

python
plateau_delta: float = Field(default=0.02, ge=0.0, le=1.0)

Win-rate improvement that counts as progress; anything smaller is noise, not learning.

Source

plateau_patience ​

python
plateau_patience: int = Field(default=2000, ge=1)

Episodes at one depth without a new best win rate before the depth counts as plateaued.

Source

pulse_bc_weight ​

python
pulse_bc_weight: float = Field(default=0.0, ge=0.0)

BC loss weight applied while a just-in-time pulse is teaching a stuck depth (0 disables pulses).

The pulse is CONDITIONAL and SUFFIX-SCOPED: it fires only when a depth has PLATEAUED below pulse_trigger_rate after its grace period (most depths acquire unaided and get zero BC exposure -- the strongest over-imprint guard there is), and while active every archive BC slice starts at the curriculum front, so only the tail being learned is demonstrated. The dose is TIME-BOXED, not effect-gated (v2, 2026-08-12): the suffix from a deep front is the whole remaining route, and the smoothed imprint caps every one of its decisions at ~p=0.79, so win probability under an ACTIVE imprint is ~0.79^tail ~ 0 -- an effect gate waits for a signal the medicine itself suppresses (run 6 measured this deadlock: BC at its exact 0.85 floor, wins flat 0 for 15k episodes at the wall). So: hold for pulse_hold_episodes, fade unconditionally over pulse_fade_episodes (liftoff at pulse_liftoff_rate is an EARLY exit only), then reset the depth's judgement counters so post-pulse PPO gets a fresh grace period to re-sharpen the tail around the hint -- the profile of the manual pulse that cleared the same wall (~2.3M steps trigger-to-clear). bc_label_smoothing bounds the imprint as usual; over-imprint instruments: the saturation probe per checkpoint and Best/Total_Tail_Delta pinned at 0.

Source

pulse_fade_episodes ​

python
pulse_fade_episodes: int = Field(default=2000, ge=1)

Front episodes over which the pulse weight fades to zero after the hold (or early liftoff).

Source

pulse_hold_episodes ​

python
pulse_hold_episodes: int = Field(default=3000, ge=1)

Front episodes the pulse teaches at full weight before fading unconditionally.

Source

pulse_liftoff_rate ​

python
pulse_liftoff_rate: float = Field(default=0.15, ge=0.0, le=1.0)

Front win rate that ends the hold early -- the depth lifted off before the teaching budget ran out.

Source

pulse_trigger_rate ​

python
pulse_trigger_rate: float = Field(default=0.1, ge=0.0, le=1.0)

Front win rate below which a PLATEAUED depth counts as stuck (the plateau test stops the pulse firing during normal-but-slow acquisition, which is improving and needs no help).

Source

retreat_threshold ​

python
retreat_threshold: float = Field(default=0.0, ge=0.0, le=1.0)

Win rate at or below which a depth has failed outright and the front steps back toward the exit.

Fires only on catastrophic failure, which is not the same as the 2026-06-19 reverse curriculum oscillating around its threshold. Without it a bad advance is terminal: at 0% the returns are uniform, Explained_Variance goes negative and stays there, and no gradient leads back.

Source

sample_window ​

python
sample_window: int = Field(default=3, ge=1)

How many segments from the front restarts are DRAWN from.

A difficulty gradient, and the reason acquisition works at all: at front 38 a window of 3 admits restarts two segments from the exit -- near-free wins the policy bootstraps from. Narrowing it to 1 took the first depth's win rate from 10% to 2% and the run never recovered (2026-08-09).

Source

step_back ​

python
step_back: int = Field(default=1, ge=1)

Segments the front moves per advance or retreat.

Source

tail_budget_multiplier ​

python
tail_budget_multiplier: float = Field(default=0.0, ge=0.0)

Action budget for a restart, as a multiple of the reference route's remaining actions from that segment. 0 disables the cap.

Restarting near the exit shortens the solution but not the search: with no budget the agent still has the whole air supply to wander in, so a restart 25 actions from the exit was a thousand-action problem. Measured on cavern 15 (2026-08-09): average episode length 239 against a 25-action reference tail, with the policy parked in an unreachable bottom-left corner where every action has the same value and the advantage signal is zero. ~94% of collected transitions were that dead region, which swamps the buffer and dilutes the rare win. Salimans & Chen bound the episode on restart; this is that bound, scaled per depth so it stays tight near the exit and loosens as the front recedes.

Source

window ​

python
window: int = Field(default=50, ge=1)

Episodes at the front that must accumulate before mastery is assessed.

Source