OakEmu-RL v0.1.0

Appearance

TrajectoryGraphUtilityEngine ​

python
TrajectoryGraphUtilityEngine(configuration: TrajectoryUtilitiesConfiguration | None) -> None

Applies back-propagated reward/penalty utilities on top of local utility metrics.

Source

Methods ​

apply ​

python
apply(recordings: list[TrajectoryRecording], *, default_node_key_fields: list[str] | None) -> None

Source

cell_reward_support ​

python
cell_reward_support(utility_key: str, values: dict[str, int], *, default_node_key_fields: list[str] | None) -> int

Get how many times a backpropagated utility's reward side has been corroborated for one cell.

Parameters:

  • utility_key (str) – The backpropagated utility.
  • values (dict[str, int]) – The cell's values.
  • default_node_key_fields (list[str] | None) – Node-key fields used when the utility's graph does not set them.

Returns:

  • int – The reward-side support count, or 0 when the utility is not backpropagated.

Source

cell_value ​

python
cell_value(utility_key: str, values: dict[str, int], *, default_node_key_fields: list[str] | None) -> float

Get a backpropagated utility's net cell value (reward minus penalty) for one cell.

Parameters:

  • utility_key (str) – The backpropagated utility (for example route_quality).
  • values (dict[str, int]) – The cell's values (start or end values of a recording).
  • default_node_key_fields (list[str] | None) – Node-key fields used when the utility's graph does not set them.

Returns:

  • float – reward - penalty for the cell, or 0.0 when the utility is not backpropagated.

Source

load_state_dict ​

python
load_state_dict(state: dict[str, Any] | None) -> None

Source

local_key ​

python
local_key(metric_key: str) -> str

Source

penalty_key ​

python
penalty_key(metric_key: str) -> str

Source

reward_key ​

python
reward_key(metric_key: str) -> str

Source

snapshot_cell_values ​

python
snapshot_cell_values(utility_key: str, field_names: list[str]) -> dict[tuple[int, ...], float]

Snapshot a backpropagated utility's net cell value, keyed by the integer tuple of field_names.

Built for lock-free per-step reward shaping: the values change slowly (only as episodes are recorded), so a periodically rebuilt plain-dict snapshot lets the actor read potentials per step without locking the store. Returns an empty dict when the utility is absent or not backpropagated.

Parameters:

  • utility_key (str) – The backpropagated utility (for example win_proximity).
  • field_names (list[str]) – The cell-key fields to extract, in order (the graph node-key fields).

Returns:

  • dict[tuple[int, ...], float] – {(field values as ints): reward - penalty} for every tracked cell.

Source

state_dict ​

python
state_dict() -> dict[str, Any]

Source