OakEmu-RL v0.1.0

Appearance

BCSampler ​

python
BCSampler(coordinator: TrajectoryEpisodeCoordinator, cache: ReplayObservationCache, *, competence_gate: str | None = None, min_bc_actions: int = 16, max_bc_actions: int = 48, max_pending_materialisations: int = 64, segment_sample_attempts: int = 64, rng: random.Random | None = None) -> None

Learner-side behaviour-cloning data source over cached route slices.

Samples contiguous min_bc_actions..max_bc_actions sub-segment slices of stored routes (see TrajectoryEpisodeCoordinator.sample_bc_slice), but only uses slices whose observations are already in the replay observation cache. Cache misses are queued for the actor to materialise (one cheap deterministic replay each); the slice then joins the BC pool on a later sample. Per-sample weights are optionally gated by start-cell competence so imitation pressure fades from sections the policy has already mastered.

Invariant (regression-locked by test_best_route_archived_and_bc_clones_slice and the learner integration test test_step_applies_nonzero_behavior_cloning_loss_when_sampler_supplies_a_batch): while BC is enabled and eligible routes exist, this sampler must yield multi-action slices that drive a nonzero behaviour-cloning loss. BC silently did nothing for entire runs twice before -- once when the sampler discarded under-filled batches, once when only one-action fragments were cloned. Any future drop here is meant to fail those tests loudly rather than rot unnoticed.

Initialize the sampler.

Parameters:

  • coordinator (TrajectoryEpisodeCoordinator) – The trajectory coordinator owning the store and its lock.
  • cache (ReplayObservationCache) – The shared replay observation cache.
  • competence_gate (str | None) – Optional competence utility used to down-weight mastered segments.
  • min_bc_actions (int) – Minimum number of actions in a cloned route slice.
  • max_bc_actions (int) – Maximum number of actions in a cloned route slice.
  • max_pending_materialisations (int) – Maximum outstanding materialisation requests.
  • segment_sample_attempts (int) – Slices sampled per minibatch-building round. Higher values build a larger (less noisy) BC batch but touch more distinct slices, so cache misses -- and the actor-side materialisation replays they trigger -- grow and steal actor throughput.
  • rng (Random | None) – Optional RNG for shuffling; defaults to a fresh instance.

Source

Properties ​

average_slice_actions ​

python
average_slice_actions: float

Get the mean action length of cache-hit BC slices, for the fragmentation guardrail.

Source

cache_hit_percentage ​

python
cache_hit_percentage: float

Get the lifetime cache hit percentage for sampler lookups.

Source

distinct_segments_sampled ​

python
distinct_segments_sampled: int

Get how many distinct segments have ever contributed BC minibatches.

Source

pending_materialisation_count ​

python
pending_materialisation_count: int

Get the number of outstanding materialisation requests.

Source

wins_recorded ​

python
wins_recorded: int

Get the number of winning episodes ingested into the store.

Source

Methods ​

pulse_bc_weight ​

python
pulse_bc_weight() -> float

Get the backward curriculum's current just-in-time pulse BC weight (0 when idle or disabled).

Returns:

  • float – The current pulse BC weight.

Source

sample_minibatches ​

python
sample_minibatches(num_minibatches: int, batch_size: int, step: int, device: torch.device) -> list[BehaviorCloningMinibatch]

Build behaviour-cloning minibatches from cached segments.

Parameters:

  • num_minibatches (int) – The number of minibatches wanted.
  • batch_size (int) – Steps per minibatch.
  • step (int) – The current actor step, for selection schedules.
  • device (device) – The device to build tensors on.

Returns:

  • list[BehaviorCloningMinibatch] – The minibatches; fewer (or none) when too little cached data is available.

Source

take_materialisation_requests ​

python
take_materialisation_requests(max_count: int) -> list[MaterialisationRequest]

Take up to max_count queued materialisation requests for the actor to execute.

Parameters:

  • max_count (int) – The maximum number of requests to take.

Returns:

  • list[MaterialisationRequest] – The requests, oldest first.

Source