OakEmu-RL v0.1.0

Appearance

RestartPoolConfiguration ​

Restart points harvested from the episodes a run plays, so a policy episode can start part way through the game instead of always from the beginning.

Deliberately small. Each confirmed recording marker carries everything a restart needs -- a machine snapshot, the observation frame stack at that moment, and the marker's captured values ; the pool is just a bucketed collection of those, with no trajectory store, repository, route paths, replay minting, or checkpointing behind it. It is derived state that refills within a minute of play, so it lives and dies with the run.

Requires recording.frame_stacks: true in the game's training configuration: without the frame stack a restart shows the policy a stack of identical frames, that is a world in which nothing moves.

Source

Methods ​

validate_probability ​

python
validate_probability(value: Any) -> Schedule

Accept a bare number or a schedule, and require it to stay within [0, 1].

Source

Values ​

advance_window ​

python
advance_window: int = Field(default=100, gt=0)

Restart episodes per bucket over which the bucket's advance rate is measured.

The advance rate is the fraction of episodes restarted from a bucket that go on to reach a deeper one, and it is the learning signal for restarts: a bucket is only ever compared with itself, so it does not matter that one section of the game has more in the way than another.

Source

checkpoint_points_per_key ​

python
checkpoint_points_per_key: int = Field(default=0, ge=0)

How many of each bucket's newest points to save when save_in_checkpoints is on. 0 saves the whole bucket.

A handful per bucket is enough to seed every band on resume -- the pool refills around them within minutes -- and keeps the file small (a Green Beret point is ~300 KB before compression). Saving whole buckets is right only when the bucket contents themselves matter; at Green Beret's 8192 per bucket that is ~14 GB encoded per checkpoint, written by the background checkpoint worker.

Source

key ​

python
key: list[str] = Field(min_length=1)

Marker value labels forming a restart point's bucket key, for example ["level_number", "player_map_x / 128"]. These must be captured values on the recording's markers, which for a visit trigger means its keys. Sampling is uniform over non-empty buckets, so the key is what decides how restarts spread over the game.

Source

max_per_key ​

python
max_per_key: int = Field(default=8, gt=0)

Restart points kept per bucket. Newest wins: a bucket is a ring, so it always holds what the current policy most recently managed to reach rather than a frozen early sample.

Source

max_timeouts_per_point ​

python
max_timeouts_per_point: int = Field(default=3, gt=0)

How many times a single restart point may produce an episode that runs out of actions before it is dropped from the pool.

Restoring a snapshot resumes whatever state it holds, and a game can have states it cannot play on from -- Green Beret's out-of-bounds truss top leaves the player alive, unreachable and immobile, so the episode never ends. Left in the pool such a point is served again every time its bucket is drawn, and each environment that draws one is out of action until its budget expires. Run 8 (2026-08-20) lost all 64 environments this way: 11M steps with zero completed episodes.

Eviction is deliberately blind to WHY a point fails, because filters only catch the breakage you predicted and there are likely other game bugs. A point that cannot produce a finished episode is dropped whatever the reason.

Only timeouts count, never losses -- a good deep point loses constantly, and counting losses would evict the most valuable points first. The default of 3 separates the two cases cheaply: a broken state fails every single time, while a merely hard one has to be unlucky repeatedly.

Source

maximum_values ​

python
maximum_values: dict[str, int] = Field(default_factory=dict)

Upper bounds on marker values; see minimum_values.

Source

minimum_values ​

python
minimum_values: dict[str, int] = Field(default_factory=dict)

Lower bounds on marker values, applied when a restart point is offered to the pool. Use to skip the game's opening bucket, where a restart is barely different from a true start, and to reject nonsense values from a game that has gone wrong.

Source

probability ​

python
probability: Schedule = Field(default_factory=lambda: Schedule.starting(0.0))

Probability that one policy episode starts from a pooled restart rather than the real start the tester measures. Episodes fall back to a normal start whenever the chosen bucket is empty, so this is an upper bound early on while the pool fills.

Source

random_seed ​

python
random_seed: int | None = None

Seed for the pool's sampling, for reproducible runs. None seeds from the system.

Source

save_in_checkpoints ​

python
save_in_checkpoints: bool = False

Whether to save restart points with every checkpoint, so a resumed run starts with every bucket it had reached instead of re-laddering from the start of the game. Mirrors the DQN replay buffer's save_replay_buffer_in_checkpoints.

Only the points are saved; advance windows and timeout counts start fresh on restore, and a saved file whose key labels differ from the live configuration is skipped with a warning.

Source

selection ​

python
selection: Literal['uniform', 'failure_rate'] = 'uniform'

How the bucket to restart from is chosen.

uniform draws every non-empty bucket equally, which spreads practice evenly over the game. failure_rate weights each bucket by max(1 - advance_rate, selection_floor), so the sections the policy is failing get the most restarts and the ones it has solved get a trickle. A bucket with no advance episodes yet counts as failing, so new buckets are tried. Within a bucket the point is always drawn uniformly.

Use failure_rate once a run shows one section stuck while the rest are solved: uniform gives the stuck section 1/N of restarts whatever its rate, and at a low advance rate that is almost no positive signal per million steps. Green Beret run 14 (2026-08-21): bands 3-7 at 72-94% after 50M, band 2 at 3% for 30M.

Source

selection_floor ​

python
selection_floor: float = Field(default=0.1, gt=0.0, le=1.0)

Smallest weight a bucket keeps under failure_rate selection, as a fraction of the weight a fully failing bucket gets.

A solved bucket must still be drawn now and then, both so the policy keeps practising it and so its advance rate stays current -- a bucket never drawn never updates, and its weight would freeze. Zero is not allowed for that reason.

Source