genopt

Non-Gaussian generalisation of EnOpt: the sampling distribution adapts as well.

GenOpt

Bases: OptimizerBase

Generalized ensemble optimization with an adapting mutation distribution.

EnOpt draws its ensemble from a Gaussian whose covariance is fixed apart from an optional Hessian-driven update. GenOpt draws from the marginals of :class:~popt.ensembles.ensemble_generalized.GeneralizedEnsemble -- Beta, logistic, truncated Gaussian -- and moves the distribution itself along with the controls: theta (the marginal's shape) follows its own gradient, and the correlation matrix follows corr_adapt.

So each accepted step updates three things rather than one: the controls from jac, theta from jac_mut, and corr from corr_adapt -- which is either a :class:CMA instance, called with the ensemble the mutation gradient was built from, or any callable returning a matrix to descend along.

Examples:

ensemble = GeneralizedEnsemble(options, simulator, objective)
cma = CMA(ne=ensemble.num_samples, dim=x0.size, corr_update=True)
result = GenOpt.minimize(
    x0, ensemble.function,
    jac=ensemble.gradient, jac_mut=ensemble.mutation_gradient,
    args=(ensemble.get_theta(), ensemble.get_corr()),
    corr_adapt=cma, bounds=bounds,
)

__init__(x0, fun, jac=None, jac_mut=None, corr_adapt=None, args=(), bounds=None, callback=None, **options)

Parameters:

Name	Type	Description	Default
x0	ndarray	Initial control vector.	required
fun	callable	Objective function.	required
jac	callable	Ensemble gradient, called as jac(x, theta, corr).	None
jac_mut	callable	Mutation gradient, called as jac_mut(x, theta, corr). For a :class:CMA corr_adapt it is called with return_ensembles=True and must then also return {'gaussian': ..., 'objective': ...}.	None
corr_adapt	CMA or callable	Correlation-matrix adaptation. A :class:CMA instance is called with the ensemble; any other callable is called with no arguments and its result is descended along with step size alpha_corr. None leaves the correlation fixed.	None
args	tuple	(theta, corr): the initial marginal parameter and correlation matrix.	()
bounds	sequence	(min, max) per control.	None
callback	callable	Invoked after each accepted step.	None
**options		GenOpt configuration, plus everything :class:OptimizerBase takes. tol: objective improvement required to accept a step (default: 1e-6). alpha: initial step size for the controls (default: 0.1). alpha_theta: step size for the marginal parameter (default: 0.1). alpha_corr: step size for the correlation, for a non-CMA corr_adapt (default: 0.1). beta: momentum (default: 0.0). nesterov: evaluate the gradients at the momentum-extrapolated point (default: False). alpha_maxiter: backtracking trials per iteration (default: 5). resample: resampling attempts when backtracking fails (default: 0). normalize: scale both gradients by their inf-norm (default: True). cov_factor: shrink factor applied to theta when resampling (default: 0.5). optimizer: GD or Adam (default: GD).	{}

log_columns()

Iteration, backtracking attempts, objective, step size, and the correlation's spread.

update_step()

One GenOpt step: controls by backtracking, then theta and the correlation.