ensembles

Ensemble containers for data assimilation.

Mirrors the layout of :mod:popt.ensembles.

AssimilationEnsemble

Bases: ForecastMixin, OutlierMixin, LocalAnalysisMixin, BaseEnsemble

Class for organizing/initializing misc. variables and simulator for an ensemble-based inversion run. Inherits the PET ensemble structure

RESTART_ATTRIBUTES = ('enX', 'prior_enX', 'pred_data', 'member_outputs', 'member_adjoints', 'adjoints', 'scale_data', 'Am', 'proj', 'iteration', 'sparse_data', 'scale_val')

What a resume must restore on the ensemble: what iterations change (the state, its forecast), and what construction drew or derived from a draw (the prior, the observation scaling, the scaled prior's SVD), so a resumed run continues the interrupted one whatever the random state was when the resuming process built its ensemble.

__init__(keys_da, keys_en, sim)

Parameters:

Name	Type	Description	Default
keys_da	dict	Options for the data assimilation class scheme: name of the assimilation algorithm (e.g., "esmda", "lmenrml", "gnenrml") analysis: update flavour ("approx", "full" or "subspace") energy: percent of singular values kept after SVD obsvarsave: save the observations as a file (default false) restart, restartsave, restart_file: checkpointing, read by the scheme (see pipt.update_schemes.core.restart_options); the ensemble contributes restart_state() to the checkpoint. savedata: names of scheme attributes to write to one file per iteration, assimilation_result_{i}.npz. Iteration 0 is the prior. "state" expands to one array per state variable; anything else is looked up on the scheme and then on the ensemble, so e.g. "ensemble_misfit", "pred_data", "data_misfit" and "lam" all resolve. A name that resolves nowhere is reported and skipped. Omitting the key disables the saving, so there is no separate on/off switch. (Was analysisdebug, still honoured with a warning.) savefolder (or save_folder): where run artifacts go (default Results) logit: enable run logging (default true). When false, no log file is created and self.logger(...) calls become no-ops. logger_name: log file name (default ASSIM.log) nosave: present in the config disables artifact saving entirely truedataindex: order of the simulated data (for timeseries this is points in time) obsname: unit for truedataindex (for timeseries this is days or hours or seconds, etc.) truedata: the data, e.g., provided as a .csv file assimindex: index for the data that will be used for assimilation datatype: list with the name of the datatypes staticvar: name of the static variables dynamicvar: name of the dynamic variables datavar: data variance, e.g., provided as a .csv file	required
keys_en	dict	Options for the ensemble class ne: number of perturbations used to compute the gradient state: name of state variables passed to the .mako file prior_: the prior information the state variables, including mean, variance and variable limits NB: If keys_en is empty dict, it is assumed that the prior info is contained in keys_da. The merged dict keys_da|keys_en is what is sent to the parent class.	required
sim	callable	The forward simulator (e.g. flow)	required

check_assimindex_simultaneous()

Check if assim. indices is given as a 1D list as is needed in simultaneous updating. If not, make it a 2D list with one row.

perturb_observations(vecObs)

Generate the perturbed observed data ensemble

restart_state()

What a checkpoint carries for this ensemble: RESTART_ATTRIBUTES plus the random stream's state.

restore_restart_state(state)

Overlay a checkpoint's ensemble state and mark the ensemble as resumed.

ForecastMixin

Forward simulation and predicted-data preparation.

save_folder: str | None

Folder for run artifacts, or None when saving is disabled.

save_folder is accepted too; the config boundary maps it to savefolder. Reading this creates nothing; :meth:_save_path makes the folder when something is about to be written into it.

forecast(enX)

Run forecast simulations and prepare predicted data for analysis.

Parameters:

Name	Type	Description	Default
enX		The state to predict on. Passed in rather than read off the ensemble, so a scheme can forecast a trial state without first parking it somewhere for this method to find.	required

sim_to_pred_data(pred)

Filter the simulator output to match the structure of the predicted data expected.

Parameters:

Name	Type	Description	Default
pred	Any	The raw output from the simulator, which may be a list of DataFrames or a single DataFrame.	required

Returns:

Type	Description
Any	The processed predicted data, structured to match the ensemble's expected format for analysis.

treat_modeling_error()

Shift every coarser level so each row's ensemble mean matches the finest level's.

LocalAnalysisMixin

Localized (per-parameter-neighbourhood) analysis update.

local_analysis_update()

Function for updates that can be used by all algorithms. Do this once to avoid duplicate code for local analysis.

OutlierMixin

Replacement of outlier ensemble members.

Ensemble work, like the forecast: it rewrites pred_data, sim_data and the state matrix in place. Called between forecast and scoring, so the replacement feeds into the misfit the scheme sees.

remove_outliers(enX)

Replace outlier ensemble members with resampled non-outliers.

Returns the state with outliers resampled -- the same object when there is nothing to replace. Returned rather than written back, because the caller owns the state being forecast.