OakEmu-RL v0.1.0

Appearance

CategoricalLoss ​

python
CategoricalLoss(config: DQNExperimentConfiguration, learner_model_container: ThreadSafeModel, target_model: DQNModel, categorical_projection: CategoricalProjection, apply_action_mask: Callable[[torch.Tensor], torch.Tensor]) -> None

Loss strategy for categorical (C51) DQN, selected once per Learner.

Initialize the categorical-loss strategy.

Parameters:

  • config (DQNExperimentConfiguration) – The DQN experiment configuration.
  • learner_model_container (ThreadSafeModel) – The learner model container (with lock).
  • target_model (DQNModel) – The target DQN model.
  • categorical_projection (CategoricalProjection) – The categorical (C51) target-distribution projection.
  • apply_action_mask (Callable[[Tensor], Tensor]) – Callable that masks excluded actions in Q-values.

Source

Methods ​

compute ​

python
compute(batch: LearnerBatch) -> LossResult

Compute the categorical DQN loss for a sampled batch.

Parameters:

  • batch (LearnerBatch) – The sampled learner batch.

Returns:

  • LossResult – The computed loss and diagnostics.

Source