OakEmu-RL v0.1.0

Appearance

TrajectoryCellStatistics ​

python
TrajectoryCellStatistics(group_by: list[str] | None, utilities: TrajectoryUtilitiesConfiguration | None) -> None

Per-cell exploration and outcome bookkeeping shared by trajectory selection and utilities.

Owns every "one entry per cell" structure the store maintains: visit/reach/successor counts (plain per-cell counters) and the competence, cost-aware, and successor-route-value tables (EMA/min/max aggregated TrajectoryGraphValueTables, built from the configured utilities).

Initialize per-cell statistics, building the tables the configured utilities need.

Parameters:

  • group_by (list[str] | None) – Cell-key fields, or None to key by every recorded value.
  • utilities (TrajectoryUtilitiesConfiguration | None) – The trajectory utilities configuration, used to decide which of the competence/cost-aware/successor-route-value tables to build.

Source

Methods ​

cell_key ​

python
cell_key(values: dict[str, int]) -> TrajectoryKey

Build the cell key for a set of recorded values.

Parameters:

  • values (dict[str, int]) – The recorded values (for example a segment's start or end values).

Returns:

  • TrajectoryKey – The cell key, restricted to group_by fields when configured.

Source

cell_successor_counts ​

python
cell_successor_counts() -> dict[TrajectoryKey, dict[TrajectoryKey, int]]

Return per-cell distinct successor counts keyed by cell.

Source

cell_successor_counts_view ​

python
cell_successor_counts_view() -> dict[TrajectoryKey, dict[TrajectoryKey, int]]

Source

cell_visit_counts ​

python
cell_visit_counts() -> dict[TrajectoryKey, int]

Source

cell_visit_counts_view ​

python
cell_visit_counts_view() -> dict[TrajectoryKey, int]

Source

competence_tables_view ​

python
competence_tables_view() -> dict[str, TrajectoryGraphValueTable]

Return the live per-utility competence tables; callers must not mutate them.

Source

load_state_dict ​

python
load_state_dict(state: dict[str, Any]) -> None

Source

midpoint_reach_counts ​

python
midpoint_reach_counts() -> dict[TrajectoryKey, dict[TrajectoryKey, int]]

Return per-start downstream reach counts keyed by midpoint cell.

Source

midpoint_reach_counts_view ​

python
midpoint_reach_counts_view() -> dict[TrajectoryKey, dict[TrajectoryKey, int]]

Source

record_cell_transitions ​

python
record_cell_transitions(cell_values: list[dict[str, int]]) -> None

Record the distinct immediate successor cells observed along one episode's cell sequence.

Each cell -> next cell transition (self-transitions ignored) is counted at most once per episode, so a cell's distinct-successor count measures how many different outgoing branches the policy has ever taken from it -- the signal branch-novelty exploration uses to find the un-taken branch at a high-traffic cell.

Parameters:

  • cell_values (list[dict[str, int]]) – The episode's cells in visit order (for example segment start then end values).

Source

record_cell_visit ​

python
record_cell_visit(values: dict[str, int]) -> None

Source

record_midpoint_reach ​

python
record_midpoint_reach(start_values: dict[str, int], downstream_values: list[dict[str, int]]) -> None

Record which downstream cells were reached from one midpoint-start cell.

Each downstream cell is counted at most once per episode so repeated revisits within the same suffix do not exaggerate one start cell's local reachability.

Parameters:

  • start_values (dict[str, int]) – The midpoint-start cell values.
  • downstream_values (list[dict[str, int]]) – The downstream segment end values reached during the episode.

Source

record_route_cost ​

python
record_route_cost(utility_key: str, cell_values: list[dict[str, int]], cost: float) -> None

Record one winning route's cost against every cell on it, for a cost-aware utility.

The table aggregates with min, so each cell remembers the cheapest winning route that has passed through it; a later, shorter winning route lowers the recorded cost and the cell's cost-aware exploration score falls automatically.

Parameters:

  • utility_key (str) – The cost-aware utility this cost feeds.
  • cell_values (list[dict[str, int]]) – The winning route's cells (deduplicated internally).
  • cost (float) – The route's cost (higher = costlier, for example air consumed or route length).

Source

record_start_outcome ​

python
record_start_outcome(utility_key: str, start_values: dict[str, int], success: bool) -> None

Record one completed episode's success outcome under its start cell.

Parameters:

  • utility_key (str) – The competence utility this outcome feeds.
  • start_values (dict[str, int]) – The episode's start cell values.
  • success (bool) – Whether the episode counted as a success for this utility.

Source

record_successor_route_value ​

python
record_successor_route_value(start_values: dict[str, int], value: float) -> None

Record the route value one episode's first hop reached from a start cell.

Parameters:

  • start_values (dict[str, int]) – The episode's start cell values.
  • value (float) – The route value of the first hop's destination cell.

Source

route_cost_tables_view ​

python
route_cost_tables_view() -> dict[str, TrajectoryGraphValueTable]

Source

start_cell_success_rate ​

python
start_cell_success_rate(utility_key: str, start_values: dict[str, int]) -> float | None

Get one competence utility's success rate for a start cell.

Parameters:

  • utility_key (str) – The competence utility.
  • start_values (dict[str, int]) – The start cell values.

Returns:

  • float | None – The EMA success rate, or None when the cell has no outcomes yet.

Source

state_dict ​

python
state_dict() -> dict[str, Any]

Source

successor_route_value ​

python
successor_route_value(start_values: dict[str, int]) -> float | None

Get the best one-hop successor route value seen from a start cell.

Parameters:

  • start_values (dict[str, int]) – The start cell values.

Returns:

  • float | None – The max first-hop route value, or None when no hop has been recorded.

Source