Training experiments

The repository includes experiments for games such as Batty, Deathchase, Green Beret and Manic Miner. Each experiment combines game configuration with a model and a learning algorithm.

Choose an algorithm

DQN learns a value for each available action and selects actions using those values. It keeps earlier transitions in a replay buffer so the learner can revisit them.

PPO learns a policy that assigns probabilities to actions. It gathers a batch of recent gameplay, then updates the policy while limiting how far it changes at once.

Both use the same emulator-backed environments. Available game and algorithm pairs are listed by the command-line help:

uv run python -m experiments.main --help

Start a run

From the repository root:

uv run python -m experiments.main train Batty ppo /path/to/training-output

The final argument is the root output directory. Choose a persistent location with room for checkpoints and metrics. Experiment configurations live under experiments/games; review their device, environment count and buffer sizes before running them.

Large configurations are intended for training hardware. For a CPU experiment, select CPU devices and reduce parallel environments and memory allocations in that game's configuration.

What the framework does

Actors gather gameplay using the current model. The learner uses that gameplay to improve it. A tester evaluates progress, while metrics and checkpoints record the run.

Schedules change settings over time, such as the learning rate or exploration probability. Reward shaping adds learning signals to the game's base reward. These settings are part of each experiment configuration.

See trajectory training for ways to reuse useful routes, and checkpoints and metrics for inspecting and exporting a run.