OakEmu-RL v0.1.0

Appearance

PotentialRewardShapingConfiguration ​

Potential-based reward shaping from a backpropagated trajectory utility (for example win_proximity).

Adds coefficient * (gamma * Phi(s') - Phis) to each step reward, where Phi is the utility's per-cell value and Phi is 0 at a terminal state. This is policy-invariant (it telescopes to -coefficient * Phi(s0) over an episode, so it cannot change the optimal policy), and it pulls the policy toward high-potential cells -- for example win_proximity is high on the winning route, low on dead-ends, so the from-start policy is nudged off off-route detours without any hand-coded map knowledge. Phi is read from a lock-free snapshot refreshed once per rollout.

Source

Values ​

cell_fields ​

python
cell_fields: list[PotentialShapingCellFieldConfiguration]

How to map per-step raw values to the utility's cell key, in node-key-field order.

Source

coefficient ​

python
coefficient: float = Field(gt=0.0)

Scale of the shaping term. The base reward is post-clip ~[-10, 10] and the potential is ~[0, 1], so keep this modest -- a nudge, not a dominant signal.

Source

gamma ​

python
gamma: float = Field(default=0.99, gt=0.0, le=1.0)

Discount used in the potential difference; should match (or be close to) the PPO gamma.

Source

utility ​

python
utility: str

Name of the backpropagated potential utility to shape from (for example win_proximity).

Source