OakEmu-RL v0.1.0

Appearance

TrainingExperimentConfiguration ​

Shared configuration for all training-based experiments.

Source

Properties ​

mixed_precision_requires_grad_scaler ​

python
mixed_precision_requires_grad_scaler: bool

Get whether mixed precision needs a gradient scaler (only float16 does).

Returns:

  • bool – True when mixed precision is enabled with the float16 dtype.

Source

Methods ​

validate_lr ​

python
validate_lr(value: Any) -> Any

Convert the learning rate to a Schedule if it's a scalar value.

Parameters:

  • value (Any) – The learning rate value (scalar or Schedule).

Returns:

  • Any – A Schedule object representing the learning rate.

Source

validate_schedules ​

python
validate_schedules(data: Any) -> Any

Ensure default values are set for the inference batch size fields before validation.

Parameters:

  • data (Any) – The raw configuration data dictionary.

Returns:

  • Any – The configuration data with defaults applied.

Source

Values ​

actor_device ​

python
actor_device: Literal['cpu', 'cuda'] = 'cpu'

The device (CPU or CUDA) on which to run the actor model.

Source

actor_inference_batch_size ​

python
actor_inference_batch_size: int | None = Field(default=None, gt=0)

Number of environment results to collect before running a batched forward pass for actor action selection. Defaults to number_of_environments. Must be <= number_of_environments to prevent stalling.

Source

auto_rollback ​

python
auto_rollback: MetricDropAutoRollbackConfiguration | None = None

Optional automatic rollback configuration. When set, training will automatically roll back to a previous checkpoint if the monitored metric drops.

Source

episode_outcome_window_size ​

python
episode_outcome_window_size: int = Field(default=100, gt=0)

The size of the rolling window for Win/Loss/Truncated percentage calculations.

Source

gamma ​

python
gamma: float = Field(default=0.99, ge=0.0, le=1.0)

The discount factor for future rewards.

Source

grad_clip ​

python
grad_clip: float | None = None

The maximum norm for gradient clipping. If None, gradient clipping is disabled.

Source

lr ​

python
lr: Schedule | float | int = Field(default=0.0001, validate_default=True)

The learning rate for the optimizer. Can be a fixed value or a schedule.

Source

max_episode_actions ​

python
max_episode_actions: int = Field(default=0, ge=0)

Actions ANY episode may take before it is abandoned as a loss. 0 leaves episodes uncapped.

An infrastructure safety net, not a shaper: set it well above a normal episode so it only ever fires on a game that has stopped making progress on its own. Some emulated states cannot end -- Green Beret's out-of-bounds truss top is alive, unreachable, and immobile -- and an environment that reaches one steps forever, taking its slot out of the pool of environments that can still finish an episode. Enough of those and episodes stop completing altogether while the learner keeps training on rollouts of nothing happening (run 8, 2026-08-20: 11M steps, zero completed episodes).

RestartPoolConfiguration.max_actions overrides this for restarted episodes, which begin part way through the game and so need less. What exceeding a budget costs lives on the session that set it (ActiveTrajectorySession.over_budget_penalty), so it cannot be left unpriced.

Source

max_materialisations_in_flight ​

python
max_materialisations_in_flight: int = Field(default=8, gt=0)

How many BC-slice materialisation replays may occupy environment slots at once. Each one replays a 16-48 action slice on a spare env to warm the replay observation cache. Too low (the old hardcoded 2) starves the learner of BC minibatches when the pending queue backs up, stalling the whole pipeline at low GPU/CPU use; too high steals env slots from rollout collection. Scale with spare CPU cores / environment headroom -- must stay well below number_of_environments.

Source

mixed_precision ​

python
mixed_precision: bool = False

Whether to use automatic mixed precision (AMP) during training.

Source

mixed_precision_dtype ​

python
mixed_precision_dtype: Literal['float16', 'bfloat16'] = 'bfloat16'

The autocast dtype used when mixed precision is enabled. bfloat16 has the same exponent range as float32 so it needs no gradient scaler; float16 keeps the scaler.

Source

number_of_environments ​

python
number_of_environments: int = Field(default=1, gt=0)

The number of parallel environments to run.

Source

number_of_runners ​

python
number_of_runners: int = Field(default=1, gt=0)

The number of threads used to update the environments.

Source

number_of_test_environments ​

python
number_of_test_environments: int = Field(default=0, ge=0)

The number of parallel environments for the tester. 0 disables the tester.

Source

number_of_test_runners ​

python
number_of_test_runners: int = Field(default=0, ge=0)

The number of threads used to update the tester environments. 0 disables the tester.

Source

restart_pool ​

python
restart_pool: RestartPoolConfiguration | None = None

Optional pool of restart points harvested from played episodes, so policy episodes can start part way through the game. Independent of trajectories: it needs only the game's recording markers and their frame stacks, so it requires recording.frame_stacks: true in the training configuration.

Source

tester_inference_batch_size ​

python
tester_inference_batch_size: int | None = Field(default=None, gt=0)

Number of environment results to collect before running a batched forward pass for tester action selection. Defaults to number_of_test_environments. Must be <= number_of_test_environments to prevent stalling.

Source

torch_compile ​

python
torch_compile: bool = False

Whether to use torch.compile() for the model forward pass. Requires PyTorch 2.0+.

Source

training_actions ​

python
training_actions: list[str] | None = None

Optional list of action names to use for training. If None, all actions are used.

Source

trajectories ​

python
trajectories: TrajectoryTrainingConfiguration | None = None

Optional RL-side configuration for trajectory replay and trajectory starts.

Source

trajectory_seed_files ​

python
trajectory_seed_files: list[str] = Field(default_factory=list)

Optional .oet trajectory files used to seed the trajectory store at the start of a fresh run (before any training), replacing in-emulator exploration with pre-generated trajectories. Paths are resolved by the experiment (typically relative to its own directory). Empty disables seeding; requires trajectories to be set. Only winning episodes are ingested, so behaviour cloning imitates wins.

Source