popt

Optimisation methods.

CMA

__call__(cov, step, X, J)

Performs the CMA update.

Parameters:

Name	Type	Description	Default
cov	array_like, of shape (d, d)	Current covariance or correlation matrix.	required
step	array_like, of shape (d,)	New step of control vector. Used to update the evolution path.	required
X	array_like, of shape (n, d)	Control ensemble of size n.	required
J	array_like, of shape (n,)	Objective ensemble of size n.	required

Returns:

Name	Type	Description
out	array_like, of shape (d, d)	CMA updated covariance (correlation) matrix.

__init__(ne, dim, alpha_mu=None, n_mu=None, alpha_1=None, alpha_c=None, corr_update=False, equal_weights=True)

This is a rather simple simple CMA class hansen2006.

Parameters:

Name	Type	Description	Default
ne	int	Ensemble size	required
dim	int	Dimensions of control vector	required
alpha_mu	float	Learning rate for rank-mu update. If None, value proposed in [1] is used.	None
n_mu	int, `n_mu < ne`	Number of best samples of ne, to be used for rank-mu update. Default is int(ne/2).	None
alpha_1	float	Learning rate fro rank-one update. If None, value proposed in [1] is used.	None
alpha_c	float	Parameter (inverse if backwards time horizen)for evolution path update in the rank-one update. See [1] for more info. If None, value proposed in [1] is used.	None
corr_update	bool	If True, CMA is used to update a correlation matrix. Default is False.	False
equal_weights	bool	If True, all n_mu members are assign equal weighting, w_i = 1/n_mu. If False, the weighting scheme proposed in [1], where w_i = log(n_mu + 1)-log(i), and normalized such that they sum to one. Defualt is True.	True

EnOpt

Bases: OptimizerBase

Ensemble-based optimization (EnOpt).

obj_func_values

Legacy alias for fk.

__init__(x0, fun, jac=None, hess=None, args=(), bounds=None, callback=None, **options)

Initialize an EnOpt optimizer instance.

Parameters:

Name	Type	Description	Default
x0	ndarray	Initial control/state vector.	required
fun	callable	Objective function.	required
jac	callable	Ensemble gradient function.	None
hess	callable	Ensemble Hessian function.	None
args	tuple	The first tuple element is interpreted as the initial covariance.	()
bounds	sequence	Lower and upper bounds for each state variable.	None
callback	callable	Callback invoked after successful updates.	None
**options		EnOpt configuration, plus everything :class:OptimizerBase takes. - tol: Convergence tolerance for objective improvement (default: 1e-6). Also used as ftol when given. - step_size: Initial optimizer step size. Overrides alpha when provided. - alpha: Initial optimizer step size (default: 0.1). - alpha_cov: Covariance update scaling factor (default: 0.001). - beta: Momentum parameter used in the optimizer and optional Nesterov updates (default: 0.0). - nesterov: Whether to evaluate search quantities with Nesterov momentum (default: False). - alpha_maxiter: Maximum number of backtracking trials per iteration (default: 5). - resample: Number of covariance resampling attempts if no improvement is found (default: 0). - hessian: Whether to use the Hessian in the search direction computation (default: False). - normalize: Whether to normalize the gradient or Hessian-derived search quantities (default: True). - cov_factor: Covariance shrink factor applied during resampling (default: 0.5). - optimizer: Update rule name. Supported values are GD, Adam, AdaMax, and Steihaug (default: GD).	{}

log_columns()

The row of the iteration log: iteration, backtracking attempts, objective, step size, first covariance entry.

update_step()

Perform one EnOpt step with backtracking and optional resampling.

GaussianEnsemble

Bases: EnsembleOptimizationBase

Gaussian Ensemble class for ensemble-based optimization.

Methods:

Name	Description
gradient	Ensemble gradient
hessian	Ensemble hessian
calc_ensemble_weights	Calculate weights used in sequential monte carlo optimization

__init__(options, simulator, objective)

Parameters:

Name	Type	Description	Default
options	dict	Options for the ensemble class disable_tqdm: supress tqdm progress bar for clean output in the notebook ne: number of perturbations used to compute the gradient state: name of state variables passed to the .mako file prior_: the prior information the state variables, including mean, variance and variable limits num_models: number of models (if robust optimization) (default 1) transform: transform variables to [0,1] if true (default true) natural_gradient: use natural gradient if true (default false)	required
simulator	callable	The forward simulator (e.g. flow)	required
objective	callable	The objective function (e.g. npv)	required

calc_ensemble_weights(x, *args, **kwargs)

Calculate weights used in sequential monte carlo optimization. Updated version that accommodates new base class changes.

Parameters:

Name	Type	Description	Default
x	ndarray	Control vector, shape (number of controls, )	required
args	tuple	Inflation factor, covariance (\(C_x\), shape (number of controls, number of controls)) and survival factor	()

Returns:

Type	Description
sens_matrix, best_ens, best_func : tuple	The weighted ensemble, the best ensemble member, and the best objective function value

gradient(x, *args, **kwargs)

Estimate the ensemble gradient (EnOpt) at a given state.

Parameters:

Name	Type	Description	Default
x	ndarray	Control vector, shape (number of controls, ).	required
args	tuple	First positional argument must be the covariance matrix with shape (number of controls, number of controls).	()

Returns:

Type	Description
ndarray	Ensemble gradient, shape (number of controls, ).

Raises:

Type	Description
ValueError	If required inputs are missing or have invalid shapes.

hessian(x=None, *args, **kwargs)

Ensemble-based Hessian.

Parameters:

Name	Type	Description	Default
x	ndarray	Control vector, shape (number of controls, ). If None, use the last x used in gradient. If x is not None and it does not match the last x used in gradient, recompute the gradient first.	None
args	tuple	Additional arguments passed to function	()

Returns:

Name	Type	Description
hessian	ndarray	Ensemble hessian, shape (number of controls, number of controls)

References

Zhang, Y., Stordal, A.S. & Lorentzen, R.J. A natural Hessian approximation for ensemble based optimization. Comput Geosci 27, 355–364 (2023). https://doi.org/10.1007/s10596-022-10185-z

GenOpt

Bases: OptimizerBase

Generalized ensemble optimization with an adapting mutation distribution.

EnOpt draws its ensemble from a Gaussian whose covariance is fixed apart from an optional Hessian-driven update. GenOpt draws from the marginals of :class:~popt.ensembles.ensemble_generalized.GeneralizedEnsemble -- Beta, logistic, truncated Gaussian -- and moves the distribution itself along with the controls: theta (the marginal's shape) follows its own gradient, and the correlation matrix follows corr_adapt.

So each accepted step updates three things rather than one: the controls from jac, theta from jac_mut, and corr from corr_adapt -- which is either a :class:CMA instance, called with the ensemble the mutation gradient was built from, or any callable returning a matrix to descend along.

Examples:

ensemble = GeneralizedEnsemble(options, simulator, objective)
cma = CMA(ne=ensemble.num_samples, dim=x0.size, corr_update=True)
result = GenOpt.minimize(
    x0, ensemble.function,
    jac=ensemble.gradient, jac_mut=ensemble.mutation_gradient,
    args=(ensemble.get_theta(), ensemble.get_corr()),
    corr_adapt=cma, bounds=bounds,
)

__init__(x0, fun, jac=None, jac_mut=None, corr_adapt=None, args=(), bounds=None, callback=None, **options)

Parameters:

Name	Type	Description	Default
x0	ndarray	Initial control vector.	required
fun	callable	Objective function.	required
jac	callable	Ensemble gradient, called as jac(x, theta, corr).	None
jac_mut	callable	Mutation gradient, called as jac_mut(x, theta, corr). For a :class:CMA corr_adapt it is called with return_ensembles=True and must then also return {'gaussian': ..., 'objective': ...}.	None
corr_adapt	CMA or callable	Correlation-matrix adaptation. A :class:CMA instance is called with the ensemble; any other callable is called with no arguments and its result is descended along with step size alpha_corr. None leaves the correlation fixed.	None
args	tuple	(theta, corr): the initial marginal parameter and correlation matrix.	()
bounds	sequence	(min, max) per control.	None
callback	callable	Invoked after each accepted step.	None
**options		GenOpt configuration, plus everything :class:OptimizerBase takes. tol: objective improvement required to accept a step (default: 1e-6). alpha: initial step size for the controls (default: 0.1). alpha_theta: step size for the marginal parameter (default: 0.1). alpha_corr: step size for the correlation, for a non-CMA corr_adapt (default: 0.1). beta: momentum (default: 0.0). nesterov: evaluate the gradients at the momentum-extrapolated point (default: False). alpha_maxiter: backtracking trials per iteration (default: 5). resample: resampling attempts when backtracking fails (default: 0). normalize: scale both gradients by their inf-norm (default: True). cov_factor: shrink factor applied to theta when resampling (default: 0.5). optimizer: GD or Adam (default: GD).	{}

log_columns()

Iteration, backtracking attempts, objective, step size, and the correlation's spread.

update_step()

One GenOpt step: controls by backtracking, then theta and the correlation.

GeneralizedEnsemble

Bases: EnsembleOptimizationBase

Control perturbations with a non-Gaussian marginal (beta, logistic, truncated Gaussian, or Gaussian) coupled by a Gaussian copula, and the mutation-based gradient and Hessian estimates that go with them.

Perturbations are drawn as correlated standard normals enZ mapped through the marginal's quantile function; the gradient of the expected objective follows from the score of the sampling density (gradient/hessian), and its derivative with respect to the marginal's own parameter theta (mutation_gradient/mutation_hessian) lets the distribution itself be adapted.

__init__(options, simulator, objective)

Parameters:

Name	Type	Description	Default
options	dict	Options for the ensemble class	required
simulator	callable	The forward simulator (e.g. flow). If None, no simulation is performed.	required
objective	callable	The objective function (e.g. npv)	required

get_corr()

The correlation matrix of the Gaussian copula.

get_theta()

The marginal's parameters, one row per control.

gradient(x, *args, **kwargs)

Estimate the gradient of the expected objective at x from the sampled members (enX, enZ, enF may be passed in; else sampled and evaluated). Also sets avg_hess.

hessian(x, *args, **kwargs)

The Hessian estimate from the last gradient call (recomputed when sample=True).

mutation_gradient(x, *args, **kwargs)

Gradient of the expected objective with respect to the marginal's parameter theta, for adapting the distribution. Also sets nat_hess.

With return_ensembles=True it returns (nat_grad, {'gaussian': enZ, 'objective': enF}) instead, so a caller adapting the correlation matrix -- :class:~popt.optimization_methods.subroutines.cma.CMA -- can reuse the ensemble this gradient came from rather than drawing and simulating another.

mutation_hessian(x, *args, **kwargs)

The theta Hessian estimate from the last mutation_gradient call (recomputed when sample=True).

sample(size=None)

Draw size perturbed controls: correlated normals enZ through the marginal's quantile function, clipped to the bounds. Returns (enX, enZ).

var2eps()

Half-width of the beta perturbation interval that reproduces the control variance.

LineSearch

Bases: OptimizerBase

Line-search optimizer compatible with OptimizerBase.

The class supports gradient descent, BFGS, and Newton-CG search directions, together with either Wolfe or backtracking line search. It can operate with bounds, optional state transformations, logging, result persistence, and restart checkpoints.

__init__(x0, fun, method='GD', jac=None, hess=None, args=(), bounds=None, callback=None, **options)

Initialize a line-search optimizer instance.

Parameters:

Name	Type	Description	Default
x0	ndarray	Initial parameter vector.	required
fun	callable	Objective function.	required
method	(GD, BFGS, Newton - CG)	Search-direction method.	'GD'
jac	callable	Gradient function.	None
hess	callable	Hessian function, required by Newton-CG.	None
args	tuple	Extra positional arguments passed to the wrapped callables.	()
bounds	sequence	Lower and upper bounds for each state variable.	None
callback	callable	Callback invoked after successful updates.	None
**options		Line-search configuration, plus everything :class:OptimizerBase takes. - step_size: Initial step size (default: None, auto-scaled). - step_size_max: Maximum step size (default: 1e5). - step_size_adapt: Step size adaptation strategy (0: none, 1: function-based, 2: gradient-based). Default is 1 (function-based). - c1: Armijo condition constant (default: 1e-4). - c2: Curvature condition constant (default: 0.9). - rho: Step size reduction factor for backtracking (default: 0.5). - lsmaxiter: Maximum line search iterations (default: 10). - lsmethod: Line search method (0: backtracking, 1: Wolfe, default: 1). - normalize: Whether to normalize the search direction (default: False). - recompute_jac: Number of gradient recomputation attempts on line search failure (default: 0). - hess0_inv: Initial inverse-Hessian approximation for BFGS (default: identity).	{}

log_columns()

The row of the iteration log: iteration, objective, gradient infinity norm, step length taken.

update_step()

Perform one optimization step.

The method computes a search direction, performs a line search, and commits the new iterate on success. When enabled, it can recompute the gradient and retry if the line search fails.

OptimizerBase

Bases: OptimizerRestartMixin, ABC

The iteration every optimizer shares; a subclass supplies the step.

A subclass implements :meth:update_step, committing an improving point with :meth:_commit_step and returning a :class:StepReport, and names what its log row shows in :meth:log_columns. Everything else -- the starting evaluation, the callback, recording and saving the result, the log row, the function, state and projected-gradient convergence checks, restart checkpoints and the EPF outer loop -- happens here.

NAME = 'Optimizer'

Shown in the start-of-run banner.

__init__(x0, fun, jac=None, hess=None, args=(), bounds=None, callback=None, **options)

Parameters:

Name	Type	Description	Default
x0	ndarray	Initial parameter vector.	required
fun	callable	Objective function.	required
jac	callable	Gradient function.	None
hess	callable	Hessian function.	None
args	tuple	Extra positional arguments passed to callables: fun, jac, hess.	()
bounds	sequence	Lower and upper bounds for each state variable.	None
callback	callable	Called with the optimizer after every accepted step.	None
**options		Optimizer configuration such as tolerances, logging, restart, and persistence options. - maxiter: Maximum number of iterations (default: 100) - ftol: Relative function tolerance for convergence (default: 1e-5) - xtol: Relative change in state for convergence (default: 1e-8) - gtol: Projected-gradient infinity-norm tolerance for convergence (default: 1e-5) - fun0, jac0, hess0: Initial objective, gradient and Hessian values to reuse instead of evaluating them - logit: Enable logging (default: True) - logger_name: Log file name (default: 'OPTIM.log') - restart: Enable restart from file (default: False) - restartsave: Save restart file after each iteration (default: False) - restart_file: Path for restart file (default: '{optimizer_name}_restart.pkl') - epf: Dictionary of EPF options (default: None) - r: Initial penalty factor - r_factor: Penalty factor update multiplier (default: 2) - tol_factor: Function tolerance update multiplier (default: 0.9) - conv_crit: EPF convergence criterion, compared against the mean penalty with the penalty factor divided out (default: 1e-5). The objective must write penalty into the epf dict it is handed. - transform: Enable [lb, ub] → [0, 1] transformation for optimization (default: False) - saveit: Save intermediate results after each iteration (default: False) - savefolder (or save_folder): Folder for those results (default: 'Iteration_Results')	{}

check_convergence()

Optimizer-specific criteria; by default the projected gradient against gtol.

Runs after the function and state checks. An optimizer with more criteria extends this; one without a gradient gets False.

check_epf_convergence()

Evaluate convergence of the outer EPF iteration.

The loop stops once the constraints are satisfied, measured as the mean of self.epf['penalty'] with the penalty factor r divided back out. The objective is responsible for writing penalty into the epf dict it is handed; without it there is nothing to converge on and this raises.

Returns:

Type	Description
bool	True when the EPF loop should terminate, otherwise False.

check_function_convergence()

Check convergence based on relative change in objective value.

check_state_convergence()

Check convergence based on the norm of the state update.

log_columns()

One row of the iteration log. Optimizers override to show their own quantities.

minimize(x0, fun, *args, **kwargs)

Construct the optimizer with these arguments, run it, and return its result.

The arguments are the constructor's, in the constructor's order; see the class for what each optimizer takes.

run_optimization()

Run this optimizer to completion.

Named for the job rather than the mechanism; the counterpart in pipt is AssimilationScheme.run_assimilation.

The loop handles restart restoration, the starting evaluation, optional EPF outer iterations, repeated calls to update_step(), and the shared convergence checks. When enabled, restart files are updated after successful iterations and after EPF penalty updates.

update_step()

Take one step from the current iterate.

Find a better point and make it current with :meth:_commit_step, which also keeps the previous iterate for the convergence checks; then return StepReport(True). The loop runs the callback, records and saves the result, logs a row and checks convergence -- none of that is the step's job. Return StepReport(False, why) when no acceptable step exists: the run stops and why is its message.

SmcOpt

Bases: OptimizerBase

Sequential Monte-Carlo optimizer with resampling and backtracking.

obj_func_values

Legacy alias for fk.

__init__(x0, fun, sens=None, args=(), bounds=None, callback=None, **options)

Parameters:

Name	Type	Description	Default
x0	ndarray	Initial state	required
fun	callable	objective function	required
sens	callable	Ensemble sensitivity function	None
args	tuple	Initial covariance tuple where args[0] is the covariance matrix used for sampling.	()
bounds	list	(min, max) pairs for each element in x. None is used to specify no bound.	None
callback	callable	Callback invoked after successful updates.	None
options	dict	SmcOpt configuration, plus everything :class:OptimizerBase takes (transform is forced off: SmcOpt works in physical coordinates). tol: convergence tolerance for the objective function (default 1e-6). Also used as ftol when given. alpha: weight between previous and new step (default 0.1) alpha_maxiter: maximum number of backtracking trials (default 5) resample: number indicating how many times resampling is tried if no improvement is found cov_factor: factor used to shrink the covariance for each resampling trial (default 0.5) inflation_factor: term used to weight down prior influence (default 1.0) survival_factor: fraction of surviving samples (clipped to [0.1, 1.0]) best_func: best objective value seen before this run (default: the initial objective) savefolder/save_folder: folder used when saveit is true (default './')	{}

log_columns()

The row of the iteration log: iteration, backtracking attempts, objective, best objective seen, step size.

update_step()

Perform one SMC update step with backtracking and optional resampling.

StepReport

What one call to :meth:OptimizerBase.update_step produced.

accepted says the optimizer committed a new iterate (through :meth:OptimizerBase._commit_step); the loop then does the bookkeeping every optimizer used to repeat. message is why it stopped when it did not, and becomes the result's message.

TrustRegion

Bases: OptimizerBase

Trust-region Optimizer.

The class supports exact Hessian trust-region subproblems (iterative or CG-Steihaug) and optional BFGS Hessian approximation via hess='BFGS'.

__init__(x0, fun, jac, hess, method='iterative', args=(), bounds=None, callback=None, **options)

Initialize a trust-region optimizer instance.

Parameters:

Name	Type	Description	Default
x0	ndarray	Initial parameter vector.	required
fun	callable	Objective function.	required
jac	callable	Gradient function.	required
hess	callable or {BFGS}	Hessian function, or 'BFGS' to use a quasi-Newton Hessian approximation.	required
method	(iterative, CG - Steihaug)	Trust-region subproblem solver.	'iterative'
args	tuple	Extra positional arguments passed to the wrapped callables.	()
bounds	sequence	Lower and upper bounds for each state variable.	None
callback	callable	Callback invoked after successful updates.	None
**options		Trust-region configuration, plus everything :class:OptimizerBase takes. - trust_radius: Initial trust-region radius (default: 1.0). - trust_radius_max: Maximum trust-region radius (default: 100 * trust_radius). - trust_radius_min: Minimum trust-region radius before termination (default: trust_radius / 1000). - trust_radius_cuts: Maximum number of radius reductions before rejecting a step (default: 4). - rho_tol: Minimum ratio between actual and predicted reduction for step acceptance (default: 1e-6). - eta1: Threshold for rejecting a step (default: 0.05). - eta2: Threshold for increasing the trust-region radius (default: 0.5). - gam1: Factor used to decrease the trust-region radius (default: 0.5). - gam2: Factor used to increase the trust-region radius when the boundary is hit (default: 1.5). - resample: Whether to recompute gradient and Hessian after rejected steps (default: False). - convergence_criteria: Optional callable for custom convergence checks.	{}

check_convergence()

The projected gradient, the trust-region radius, and any custom criterion.

log_columns()

The row of the iteration log: iteration, objective, trust radius, reduction ratio, whether the step hit the boundary.

update_step()

Perform one trust-region step with optional radius reductions.