ensemble

Foundation shared by pipt and popt: the base ensemble, checkpoint/restart and logging.

BaseEnsemble

Class for organizing misc. variables and simulator for an ensemble-based inversion run. Here, the forecast step and prediction runs are performed. General methods that are useful in various ensemble loops have also been implemented here.

sim_data

The full forecast as a frame (one per level), built from the member outputs on first use.

Nothing on the analysis path reads it; saving, inspection and popt's objective functions do, so it is built when one of them asks and cached until the next forecast.

state_layout: StateLayout

The state's variable layout, read off idX -- the one place the row ranges live.

__init__(keys_en, sim, redund_sim=None)

Class extends the ReadInitFile class. First the PIPT init. file is passed to the parent class for reading and parsing. Rest of the initialization uses the keywords parsed in ReadInitFile (parent) class to set up observed, predicted data and data variance dictionaries. Also, the simulator to be used in forecast and/or predictions is initialized with keywords parsed in ReadInitFile (parent) class. Lastly, the initial ensemble is generated (if it has not been inputted), and some saving of variables can be done chosen in PIPT init. file.

Parameter

init_file : str path to input file containing initiallization values

calc_prediction(enX, save_prediction=None)

Function for running the simulator over several levels. We assume that it is sufficient to provide the level integer to the setup of the forward run. This will initiate the correct simulator fidelity. The function then runs the set of state through the different simulator fidelities.

Per level: the state becomes one input dict per member (:meth:_simulator_input), the members run on one of three backends (:meth:_run_members), crashed members are replaced, adjoints are split off, and the outputs are kept as returned (member_outputs); the frame view (sim_data) is built from them on demand.

Parameters:

Name	Type	Description	Default
enX		If simulation is run stand-alone one can input any state.	required

run_on_HPC(enX, batch_size=None, **kwargs)

Run the members through the simulator's HPC queue, batch_size at a time; needs the queue hooks on the wrapper.

save()

Dump everything in self to emergency_dump_file for inspection after a failed forecast.

ForwardSimulator

Bases: Protocol

What :meth:ensemble.ensemble.BaseEnsemble.calc_prediction requires of a simulator.

Two members are required, and they are all that isinstance(sim, ForwardSimulator) checks:

input_dict The parsed simulator section of the config. The ensemble reads parallel (local workers, default 1) and hpc from it. run_fwd_sim(state, member_index) Run one realisation. state maps each state variable to that member's values; member_index is the member's position in the ensemble. Return one of

- a list with one dict per report point, keyed by data type,
- a ``pandas.DataFrame`` with report points as index and data types
  as columns,
- ``False`` when the run failed, so the member can be replaced, or
- ``(output, adjoint)`` when ``compute_adjoints`` is true.

Members the ensemble looks for with hasattr/getattr and uses only when present:

setup_fwd_run(level=...) Called once before each prediction, with the fidelity level. true_order [index_name, index_values] used to index the returned records. datatype Fallback column filter when the observed data has no columns yet. compute_adjoints Whether run_fwd_sim returns (output, adjoint). Default False.

The ensemble also assigns redund_sim (a backup simulator, or None) onto the simulator when it is constructed. The analytical models in :mod:simulator are the smallest complete examples.

run_fwd_sim(state, member_index, *args, **kwargs)

Run one member; the class docstring lists the accepted return values.

NullLogger

Callable no-op standing in for a :class:PetLogger when logging is disabled -- so callers can invoke self.logger(...) unconditionally without checking whether logging is on, and no log file is created.

info(*args, **kwargs)

No-op.

PetLogger

A custom logger that logs messages and key-value pairs in a formatted table.

Parameters: filename (str): The name of the log file. Defaults to 'PET.log'.

__call__(*args, **kwargs)

Log messages or key-value pairs in a formatted table.

Parameters: *args: Positional arguments to log as a single message. **kwargs: Keyword arguments to log in a formatted table.

Example: >>> logger = PetLogger() >>> logger('This is a log message.') 2024-06-01│12:00:00 : This is a log message. >>> >>> logger(iteration=1, fun=0.5, step_size=0.1) 2024-06-01│12:00:00 : 2024-06-01│12:00:00 : ┌────────────┬────────────┬────────────┐ 2024-06-01│12:00:00 : │ iteration │ fun │ step_size │ 2024-06-01│12:00:00 : ├────────────┼────────────┼────────────┤ 2024-06-01│12:00:00 : │ 1 │ 5.000e-01 │ 1.000e-01 │ 2024-06-01│12:00:00 : └────────────┴────────────┴────────────┘ 2024-06-01│12:00:00 :

info(*args, **kwargs)

Log as given; __call__ is the table-aware form.