API

Model <: JuMP.AbstractModel

Create an empty ApplicationDrivenLearning.Model with empty plan and assess models, missing forecast model and default settings.

PredictiveModel(networks, input_output_map, input_size, output_size)

Creates a predictive (forecast) model for the AppDrivenLearning module from Flux models and input/output information.

...

Arguments

• networks: array of Flux models to be used.
• input_output_map::Union{Vector{<:Dict{Vector{Int},<:Vector{<:Forecast}}},Nothing}: array in the same ordering as networks of mappings from input indexes to output indexes on which the models should be applied.
• output_variables::Union{Vector{<:Forecast},Nothing}: array of output variables to be used.
• input_size::Int: size of the input vector.
• output_size::Int: size of the output vector. ...

Example

julia> pred_model = PredictiveModel(
        [Flux.Dense(1 => 1), Flux.Dense(3 => 2)],
        [
            Dict([1] => [1], [1] => [2]),
            Dict([1,2,3] => [3,4], [1,4,5] => [5,6])
        ],
        5,
        6
    );

Upper(model::ApplicationDrivenLearning.Model)

Create a reference to the plan model of an application driven learning model.

Assess(model::ApplicationDrivenLearning.Model)

Create a reference to the assess model of an application driven learning model.

Policy{T}

Policy variable type that holds plan and assess variables.

Forecast{T}

Forecast variable type that holds plan and assess variables.

Options(mode; params...)

Options struct to hold optimization mode and mode parameters.

...

Example

options = Options(
    GradientMode;
    rule = Optim.RMSProp(0.01),
    epochs = 100,
    batch_size = 10,
)

...

Solution

A struct to store the result of the optimisation process with final cost and solution.

NelderMeadMode <: AbstractOptimizationMode

Used to solve the application driven learning training problem using the Nelder-Mead optimization method implementation from Optim.jl package.

...

Parameters

• initial_simplex is the initial simplex of solutions to be applied.
• parameters is the parameters to be applied to the Nelder-Mead optimization method. ...

GradientMode <: AbstractOptimizationMode

Used to solve the application driven learning training problem using the gradient optimization method

...

Parameters

• rule is the optimiser object to be used in the gradient optimization process.
• 'epochs' is the number of epochs to be used in the gradient optimization process.
• 'batch_size' is the batch size to be used in the gradient optimization process.
• 'verbose' is the flag of whether to print the training process.
• 'computecostevery' is the epoch frequency for computing the cost and evaluating best solution.
• 'time_limit' is the time limit for the training process. ...

NelderMeadMPIMode <: AbstractOptimizationMode

MPI implementation of NelderMeadMode.

GradientMPIMode <: AbstractOptimizationMode

MPI implementation of GradientMode.

BilevelMode <: AbstractOptimizationMode

Used to solve the application driven learning training problem as a bilevel optimization problem by using the BilevelJuMP.jl package.

...

Parameters

• optimizer::Function is equivalent to solver in BilevelJuMP.BilevelModel.
• silent::Bool is equivalent to silent in BilevelJuMP.BilevelModel.
• mode::Union{Nothing, BilevelJuMP.BilevelMode} is equivalent to mode in BilevelJuMP.BilevelModel. ...

Returns vector of policy variables from plan model.

Returns vector of policy variables from assess model.

Returns vector of forecast variables from plan model.

Returns vector of forecast variables from assess model.

Sets Chain, Dense or custom PredictiveModel object as forecast model.

extract_params(model)

Extract the parameters of a PredictiveModel into a single vector.

apply_params(model, θ)

Return model after fixing the parameters from an adequate vector of parameters.

extract_flux_params(model)

Extract the parameters of a Flux model (Flux.Chain or Flux.Dense) into a single vector.

fix_flux_params_single_model(model, θ)

Return model after fixing the parameters from an adequate vector of parameters.

fix_flux_params_multi_model(models, θ)

Return iterable of models after fixing the parameters from an adequate vector of parameters.

has_params(layer)

Check if a Flux layer has parameters.

apply_gradient!(model, dCdy, X, opt_state)

Apply per-sample cost gradients to the model parameters.

The optimization layer provides dCdy, the gradient of the assessment cost with respect to the forecasts, for each sample. Because the optimization step itself is not differentiable by the AD backend, these gradients are propagated through the forecast model with a linear surrogate loss whose parameter-gradient equals the chain-rule term (1/T) Σₜ (dC/dŷₜ)·(dŷₜ/dθ).

...

Arguments

• model::PredictiveModel: model to be updated.
• dCdy::AbstractMatrix{<:Real}: per-sample cost gradients, size (T, output_size), with row t aligned to sample t (row t of X).
• X::Matrix{<:Real}: input data, size (T, input_size).
• opt_state: Optimisers optimisation state. ...

compute_cost(model, X, Y, with_gradients=false)

Compute the cost function (C) based on the model predictions and the true values matrix.

...

Arguments

• model::ApplicationDrivenLearning.Model: model to evaluate.
• X::Matrix{<:Real}: input data.
• Y::Matrix{<:Real}: true values.
• with_gradients::Bool=false: flag to compute and return the cost gradients with respect to the forecasts. When set, the second returned value is the per-sample gradient matrix of size (T, output_size).
• aggregate::Bool=true: when true, the returned cost is averaged over the T samples. Only affects the cost; gradients are always per-sample. ...

train!(model, X, y, options)

Train model using given data and options.

Creates new forecast variables to plan model using MOI.Parameter and new constraint fixing to original forecast variables.

Creates new constraint to assess model that fixes policy variables.

Calls functions that set new variables and constraints that are necessary to cost computation.