OakEmu-RL v0.1.0

Appearance

PPOActivationMonitor ​

python
PPOActivationMonitor(*, metrics: MetricsTracker, device: torch.device, interval_rollouts: int | None, max_observations: int) -> None

Logs per-layer activation health from the learner, every N rollouts.

Green Beret run 15 lost the last hidden layer to dead ReLUs (11% dead at 130M, 90% at 297M) with no loss metric moving; the policy lost the wave as the layer's effective rank fell through ~20. This gauge makes that visible on the graph while the run is live, instead of at the post-mortem.

Initialize the monitor.

Parameters:

  • metrics (MetricsTracker) – Metrics tracker to write to.
  • device (device) – Device the model's forward passes run on.
  • interval_rollouts (int | None) – Log every this many rollouts; None disables the monitor.
  • max_observations (int) – Observations used per measurement (the first rows of the minibatch given).

Source

Properties ​

enabled ​

python
enabled: bool

Whether the monitor logs anything.

Source

Methods ​

is_due ​

python
is_due(rollouts_collected: int) -> bool

Whether a measurement is due after this many rollouts.

Parameters:

  • rollouts_collected (int) – Rollouts consumed so far, including the one just trained on.

Returns:

  • bool – True on every interval_rollouts-th rollout; always False when disabled.

Source

maybe_log ​

python
maybe_log(model: torch.nn.Module, observations: torch.Tensor, *, actor_steps: int, rollouts_collected: int) -> bool

Measure and log activation health if a measurement is due.

The caller holds the model lock; the measurement runs the model forward in eval mode with no grad and restores the training mode afterwards.

Parameters:

  • model (Module) – The learner model.
  • observations (Tensor) – Observations from the current rollout, (N, *observation_shape), in the model's input dtype.
  • actor_steps (int) – The current actor step count, used as the metric step.
  • rollouts_collected (int) – Rollouts consumed so far, including the one just trained on.

Returns:

  • bool – True if metrics were logged.

Source