OakEmu-RL v0.1.0

Appearance

DQNExperimentConfiguration ​

Source

Methods ​

validate_learning_start ​

python
validate_learning_start() -> DQNExperimentConfiguration

Source

Values ​

actor_batch_add_size ​

python
actor_batch_add_size: int = Field(default=16, gt=0)

The number of transitions to buffer locally in the actor before adding them to the replay buffer.

Source

categorical ​

python
categorical: CategoricalConfiguration | None = None

Configuration for Categorical DQN (C51). If None, categorical DQN is disabled.

Source

double_dqn ​

python
double_dqn: bool = False

Whether to use Double DQN.

Source

dueling ​

python
dueling: bool = False

Whether to use Duelling architecture.

Source

exploration ​

python
exploration: Annotated[EpsilonGreedyConfiguration | NoisyNetsConfiguration, Field(discriminator=type)] = EpsilonGreedyConfiguration()

Configuration for the exploration strategy.

Source

loss_function ​

python
loss_function: Literal['mse', 'huber'] = 'mse'

The loss function for standard (non-categorical) DQN. 'mse' uses mean squared error, 'huber' uses Huber (smooth L1) loss.

Source

n_step ​

python
n_step: int = Field(default=1, ge=1)

Number of steps for n-step returns.

Source

num_steps_sampled_before_learning_starts ​

python
num_steps_sampled_before_learning_starts: int = Field(default=1000, ge=0)

The number of environment steps to perform before starting the learning process.

Source

replay_buffer ​

python
replay_buffer: Annotated[UniformReplayBufferConfiguration | PrioritizedExperienceReplayBufferConfiguration, Field(discriminator=type)] = UniformReplayBufferConfiguration()

Configuration for the replay buffer.

Source

replay_writer_queue_size ​

python
replay_writer_queue_size: int = Field(default=8, gt=0)

The maximum number of replay batches that may be queued before the actor blocks.

Source

save_replay_buffer_in_checkpoints ​

python
save_replay_buffer_in_checkpoints: bool = False

Whether to save the entire replay buffer in each checkpoint (can be very large).

Source

target_network_update_freq ​

python
target_network_update_freq: int = Field(default=512, gt=0)

The frequency (in sample steps) with which to update the target network weights.

Source

tau ​

python
tau: float = Field(default=1.0, gt=0.0, le=1.0)

Interpolation factor for soft (Polyak) target network updates. tau=1.0 is a hard update (full copy), tau<1.0 blends: target = tau * learner + (1-tau) * target.

Source

train_batch_size ​

python
train_batch_size: int = Field(default=32, gt=0)

The number of transitions to sample from the replay buffer for each training iteration.

Source