evox.algorithms.so.es_variants.noise_reuse_es

Module Contents

Classes

NoiseReuseES

The implementation of the Noise-Reuse-ES algorithm.

API

class evox.algorithms.so.es_variants.noise_reuse_es.NoiseReuseES(pop_size: int, center_init: torch.Tensor, optimizer: Literal[adam] | None = None, lr: float = 0.05, sigma: float = 0.03, T: int = 100, K: int = 10, sigma_decay: float = 1.0, sigma_limit: float = 0.01, device: torch.device | None = None)[source]

Bases: evox.core.Algorithm

The implementation of the Noise-Reuse-ES algorithm.

Reference: Noise-Reuse in Online Evolution Strategies (https://arxiv.org/pdf/2304.12180.pdf)

This code has been inspired by or utilizes the algorithmic implementation from evosax. More information about evosax can be found at the following URL: GitHub Link: https://github.com/RobertTLange/evosax

Initialization

Initialize the Guided-ES algorithm with the given parameters.

Parameters:

  • pop_size – The size of the population.

  • center_init – The initial center of the population. Must be a 1D tensor.

  • optimizer – The optimizer to use. Defaults to None. Currently, only “adam” or None is supported.

  • lr – The learning rate for the optimizer. Defaults to 0.05.

  • sigma – The standard deviation of the noise. Defaults to 0.03.

  • sigma_decay – The decay factor for the standard deviation. Defaults to 1.0.

  • sigma_limit – The minimum value for the standard deviation. Defaults to 0.01.

  • T – The inner problem length. Defaults to 100.

  • K – The number of inner problems. Defaults to 10.

  • device – The device to use for the tensors. Defaults to None.

step()[source]

Take a single step of the NoiseReuseES algorithm.

This function follows the algorithm described in the reference paper. It first generates a set of perturbations for the current population. Then, it evaluates the fitness of the population with the perturbations. Afterwards, it calculates the gradient of the policy parameters using the perturbations and the fitness. Finally, it updates the policy parameters using the gradient and the learning rate.

record_step()[source]