OakEmu-RL v0.1.0

Appearance

mrkwatkins.oakemu_rl.framework.diagnostics.ActivationStatistics ​

Classes and functions ​

Constants ​

ACTIVATION_TYPES ​

python
ACTIVATION_TYPES: tuple[type[nn.Module], ...] = (*RECTIFIER_TYPES, *BOUNDED_TYPES)

The activation modules whose outputs are measured. Every one of them sits directly after the layer it reports on.

Source

BOUNDED_TYPES ​

python
BOUNDED_TYPES: tuple[type[nn.Module], ...] = (nn.Tanh, nn.Sigmoid)

Activations with no off state. They can die by saturating, but a near-dead count has no meaning for them.

Source

RECTIFIER_TYPES ​

python
RECTIFIER_TYPES: tuple[type[nn.Module], ...] = (nn.ReLU, nn.LeakyReLU, nn.GELU, nn.SiLU, nn.ELU)

Activations that are off (at or below zero) for part of their input range, so "active" means output above zero.

Source