Manual
Quick Start Guide
Although StateSpaceModels.jl has a lot of functionalities, different models and interfaces users usuallly just want to fit a model and analyse the residuals, components and make some forecasts. The following code is a quick start to perform these tasks
import Pkg
Pkg.add("StateSpaceModels")
using StateSpaceModels
y = randn(100)
model = LocalLevel(y)
fit!(model)
print_results(model)
forec = forecast(model, 10)
kf = kalman_filter(model)
v = get_innovations(kf)
ks = kalman_smoother(model)
alpha = get_smoothed_state(ks)
using Plots
plot(model, forec)
plotdiagnostics(kf)
Models
The package provides a variaty of pre-defined models. If there is any model that you wish was in the package, feel free to open an issue or pull request.
StateSpaceModels.UnobservedComponents
— TypeUnobservedComponents(
y::Vector{Fl};
X::Matrix{Fl} = zeros(Fl, length(y), 0),
trend::String = "local level",
seasonal::String = "no"
cycle::String = "no"
) where Fl
An unobserved components model that can have trend/level, seasonal and cycle components. Each component should be specified by strings, if the component is not desired in the model a string with "no" can be passed as keyword argument.
These models take the general form
\[\begin{gather*} \begin{aligned} y_t = \mu_t + \gamma_t + c_t + \varepsilon_t \end{aligned} \end{gather*}\]
where $y_t$ refers to the observation vector at time $t$, $\mu_t$ refers to the trend component, $\gamma_t$ refers to the seasonal component, $c_t$ refers to the cycle, and $\varepsilon_t$ is the irregular. The modeling details of these components are given below.
Trend
The trend component can be modeled in a lot of different ways, usually it is called level when there is no slope component. The modelling options can be expressed as in the example trend = "local level"
.
- Local Level
string: "local level"
\[\begin{gather*} \begin{aligned} y_{t} &= \mu_{t} + \varepsilon_{t} \quad \varepsilon_{t} \sim \mathcal{N}(0, \sigma^2_{\varepsilon})\\ \mu_{t+1} &= \mu_{t} + \eta_{t} \quad \eta_{t} \sim \mathcal{N}(0, \sigma^2_{\eta})\\ \end{aligned} \end{gather*}\]
- Random Walk
string: "random walk"
\[\begin{gather*} \begin{aligned} y_{t} &= \mu_{t}\\ \mu_{t+1} &= \mu_{t} + \eta_{t} \quad \eta_{t} \sim \mathcal{N}(0, \sigma^2_{\eta})\\ \end{aligned} \end{gather*}\]
- Local Linear Trend
string: "local linear trend"
\[\begin{gather*} \begin{aligned} y_{t} &= \mu_{t} + \varepsilon_{t} \quad &\varepsilon_{t} \sim \mathcal{N}(0, \sigma^2_{\varepsilon})\\ \mu_{t+1} &= \mu_{t} + \nu_{t} + \xi_{t} \quad &\xi_{t} \sim \mathcal{N}(0, \sigma^2_{\xi})\\ \nu_{t+1} &= \nu_{t} + \zeta_{t} \quad &\zeta_{t} \sim \mathcal{N}(0, \sigma^2_{\zeta})\\ \end{aligned} \end{gather*}\]
- Smooth Trend
string: "smooth trend"
\[\begin{gather*} \begin{aligned} y_{t} &= \mu_{t} + \varepsilon_{t} \quad &\varepsilon_{t} \sim \mathcal{N}(0, \sigma^2_{\varepsilon})\\ \mu_{t+1} &= \mu_{t} + \nu_{t}\\ \nu_{t+1} &= \nu_{t} + \zeta_{t} \quad &\zeta_{t} \sim \mathcal{N}(0, \sigma^2_{\zeta})\\ \end{aligned} \end{gather*}\]
Seasonal
The seasonal component is modeled as:
\[\begin{gather*} \begin{aligned} \gamma_t = - \sum_{j=1}^{s-1} \gamma_{t+1-j} + \omega_t \quad \omega_t \sim N(0, \sigma^2_\omega) \end{aligned} \end{gather*}\]
The periodicity (number of seasons) is s, and the defining character is that (without the error term), the seasonal components sum to zero across one complete cycle. The inclusion of an error term allows the seasonal effects to vary over time. The modelling options can be expressed in terms of "deterministic"
or "stochastic"
and the periodicity as a number in the string, i.e., seasonal = "stochastic 12"
.
Cycle
The cycle component is modeled as
\[\begin{gather*} \begin{aligned} c_{t+1} &= \rho_c \left(c_{t} \cos(\lambda_c) + c_{t}^{*} \sin(\lambda_c)\right) \quad & \tilde\omega_{t} \sim \mathcal{N}(0, \sigma^2_{\tilde\omega})\\ c_{t+1}^{*} &= \rho_c \left(-c_{t} \sin(\lambda_c) + c_{t}^{*} \sin(\lambda_c)\right) \quad &\tilde\omega^*_{t} \sim \mathcal{N}(0, \sigma^2_{\tilde\omega})\\ \end{aligned} \end{gather*}\]
The cyclical component is intended to capture cyclical effects at time frames much longer than captured by the seasonal component. The parameter $\lambda_c$ is the frequency of the cycle and it is estimated via maximum likelihood. The inclusion of error terms allows the cycle effects to vary over time. The modelling options can be expressed in terms of "deterministic"
or "stochastic"
and the damping effect as a string, i.e., cycle = "stochastic"
, cycle = "deterministic"
or cycle = "stochastic damped"
.
The UnobservedComponents model has some dedicated Plot Recipes, see Visualization
References
- Durbin, James, & Siem Jan Koopman. Time Series Analysis by State Space Methods: Second Edition. Oxford University Press, 2012
StateSpaceModels.ExponentialSmoothing
— TypeExponentialSmoothing(
y::Vector{Fl};
trend::Bool = false,
damped_trend::Bool = false,
seasonal::Int = 0
) where Fl
Linear exponential smoothing models. These models are also known as ETS in the literature. This model is estimated using the Kalman filter for linear Gaussian state space models, for this reason the possible models are the following ETS with additive errors:
- ETS(A, N, N)
- ETS(A, A, N)
- ETS(A, Ad, N)
- ETS(A, N, A)
- ETS(A, A, A)
- ETS(A, Ad, A)
Other softwares have use the augmented least squares approach and have all the possible ETS combinations. The Kalman filter approach might be slower than others but have the advantages of filtering the components.
References
- Hyndman, Rob, Anne B. Koehler, J. Keith Ord, and Ralph D. Snyder. Forecasting with exponential smoothing: the state space approach. Springer Science & Business Media, 2008.
- Hyndman, Robin John; Athanasopoulos, George. Forecasting: Principles and Practice. 2nd ed. OTexts, 2018.
StateSpaceModels.SARIMA
— TypeSARIMA(
y::Vector{Fl};
order::Tuple{Int,Int,Int} = (1, 0, 0),
seasonal_order::Tuple{Int, Int, Int, Int} = (0, 0, 0, 0),
include_mean::Bool = false,
suppress_warns::Bool = false
) where Fl
A SARIMA model (Seasonal AutoRegressive Integrated Moving Average) implemented within the state-space framework.
The SARIMA model is specified $(p, d, q) \times (P, D, Q, s)$. We can also consider a polynomial $A(t)$ to model a trend, here we only allow to add a constante term with the include_mean
keyword argument.
\[\begin{gather*} \begin{aligned} \phi_p (L) \tilde \phi_P (L^s) \Delta^d \Delta_s^D y_t = A(t) + \theta_q (L) \tilde \theta_Q (L^s) \zeta_t \end{aligned} \end{gather*}\]
In terms of a univariate structural model, this can be represented as
\[\begin{gather*} \begin{aligned} y_t & = u_t + \eta_t \\ \phi_p (L) \tilde \phi_P (L^s) \Delta^d \Delta_s^D u_t & = A(t) + \theta_q (L) \tilde \theta_Q (L^s) \zeta_t \end{aligned} \end{gather*}\]
Example
julia> model = SARIMA(rand(100); order=(1,1,1), seasonal_order=(1,2,3,12))
SARIMA(1, 1, 1)x(1, 2, 3, 12) model
See more on Airline passengers
References
- Durbin, James, & Siem Jan Koopman. Time Series Analysis by State Space Methods: Second Edition. Oxford University Press, 2012
StateSpaceModels.DAR
— TypeDAR(y::Vector{Fl}, lags::Int) where Fl
A Dynamic Autorregressive model is defined by:
\[\begin{gather*} \begin{aligned} y_{t} &= \phi_0 + \sum_{i=1}^{lags} \phi_{i, t} y_{t - i} + \varepsilon_{t} \quad \varepsilon_{t} \sim \mathcal{N}(0, \sigma^2_{\varepsilon})\\ \phi_{i, t+1} &= \phi_{i, t} + \eta{i, t} \quad \eta{i, t} \sim \mathcal{N}(0, \sigma^2_{i, \eta})\\ \end{aligned} \end{gather*}\]
!!!! note System matrices When building the system matrices we get rid of the first lags
observations in order to build all system matrices respecting the correspondent timestamps
!!! warning Forecasting The dynamic autorregressive model does not have the forecast
method implemented yet. If you wish to perform forecasts with this model please open an issue.
!!! warning Missing values The dynamic autorregressive model currently does not support missing values (NaN
observations.)
Example
julia> model = DAR(randn(100), 2)
DAR
StateSpaceModels.BasicStructural
— TypeBasicStructural(y::Vector{Fl}, s::Int) where Fl
The basic structural state-space model consists of a trend (level + slope) and a seasonal component. It is defined by:
\[\begin{gather*} \begin{aligned} y_{t} &= \mu_{t} + \gamma_{t} + \varepsilon_{t} \quad &\varepsilon_{t} \sim \mathcal{N}(0, \sigma^2_{\varepsilon})\\ \mu_{t+1} &= \mu_{t} + \nu_{t} + \xi_{t} \quad &\xi_{t} \sim \mathcal{N}(0, \sigma^2_{\xi})\\ \nu_{t+1} &= \nu_{t} + \zeta_{t} \quad &\zeta_{t} \sim \mathcal{N}(0, \sigma^2_{\zeta})\\ \gamma_{t+1} &= -\sum_{j=1}^{s-1} \gamma_{t+1-j} + \omega_{t} \quad & \omega_{t} \sim \mathcal{N}(0, \sigma^2_{\omega})\\ \end{aligned} \end{gather*}\]
Example
julia> model = BasicStructural(rand(100), 12)
BasicStructural
References
- Durbin, James, & Siem Jan Koopman. (2012). "Time Series Analysis by State Space Methods: Second Edition." Oxford University Press.
StateSpaceModels.BasicStructuralExplanatory
— TypeBasicStructuralExplanatory(y::Vector{Fl}, s::Int, X::Matrix{Fl}) where Fl
It is defined by:
\[\begin{gather*} \begin{aligned} y_{t} &= \mu_{t} + \gamma_{t} + \beta_{t, i}X_{t, i} \varepsilon_{t} \quad &\varepsilon_{t} \sim \mathcal{N}(0, \sigma^2_{\varepsilon})\\ \mu_{t+1} &= \mu_{t} + \nu_{t} + \xi_{t} \quad &\xi_{t} \sim \mathcal{N}(0, \sigma^2_{\xi})\\ \nu_{t+1} &= \nu_{t} + \zeta_{t} \quad &\zeta_{t} \sim \mathcal{N}(0, \sigma^2_{\zeta})\\ \gamma_{t+1} &= -\sum_{j=1}^{s-1} \gamma_{t+1-j} + \omega_{t} \quad & \omega_{t} \sim \mathcal{N}(0, \sigma^2_{\omega})\\ \beta_{t+1} &= \beta_{t} \end{aligned} \end{gather*}\]
Example
julia> model = BasicStructuralExplanatory(rand(100), 12, rand(100, 2))
BasicStructuralExplanatory
References
- Durbin, James, & Siem Jan Koopman. (2012). "Time Series Analysis by State Space Methods: Second Edition." Oxford University Press.
StateSpaceModels.LinearRegression
— TypeLinearRegression(X::Matrix{Fl}, y::Vector{Fl}) where Fl
The linear regression state-space model is defined by:
\[\begin{gather*} \begin{aligned} y_{t} &= X_{1,t} \cdot \beta_{1,t} + \dots + X_{n,t} \cdot \beta_{n,t} + \varepsilon_{t} \quad &\varepsilon_{t} \sim \mathcal{N}(0, \sigma^2_{\varepsilon})\\ \beta_{1,t+1} &= \beta_{1,t}\\ \dots &= \dots\\ \beta_{n,t+1} &= \beta_{n,t}\\ \end{aligned} \end{gather*}\]
Example
julia> model = LinearRegression(rand(100, 2), rand(100))
LinearRegression
StateSpaceModels.LocalLevel
— TypeLocalLevel(y::Vector{Fl}) where Fl
The local level model is defined by:
\[\begin{gather*} \begin{aligned} y_{t} &= \mu_{t} + \varepsilon_{t} \quad \varepsilon_{t} \sim \mathcal{N}(0, \sigma^2_{\varepsilon})\\ \mu_{t+1} &= \mu_{t} + \eta_{t} \quad \eta_{t} \sim \mathcal{N}(0, \sigma^2_{\eta})\\ \end{aligned} \end{gather*}\]
Example
julia> model = LocalLevel(rand(100))
LocalLevel
See more on Nile river annual flow
References
- Durbin, James, & Siem Jan Koopman. (2012). "Time Series Analysis by State Space Methods: Second Edition." Oxford University Press. pp. 9
StateSpaceModels.LocalLevelCycle
— TypeLocalLevelCycle(y::Vector{Fl}) where Fl
The local level model with a cycle component is defined by:
\[\begin{gather*} \begin{aligned} y_{t} &= \mu_{t} + c_{t} + \varepsilon_{t} \quad &\varepsilon_{t} \sim \mathcal{N}(0, \sigma^2_{\varepsilon})\\ \mu_{t+1} &= \mu_{t} + \eta_{t} \quad &\eta_{t} \sim \mathcal{N}(0, \sigma^2_{\eta})\\ c_{t+1} &= c_{t} \cos(\lambda_c) + c_{t}^{*} \sin(\lambda_c)\ \quad & \tilde\omega_{t} \sim \mathcal{N}(0, \sigma^2_{\tilde\omega})\\ c_{t+1}^{*} &= -c_{t} \sin(\lambda_c) + c_{t}^{*} \sin(\lambda_c) \quad &\tilde\omega^*_{t} \sim \mathcal{N}(0, \sigma^2_{\tilde\omega})\\ \end{aligned} \end{gather*}\]
Example
julia> model = LocalLevelCycle(rand(100))
LocalLevelCycle
See more on TODO RJ_TEMPERATURE
References
- Durbin, James, & Siem Jan Koopman. (2012). "Time Series Analysis by State Space Methods: Second Edition." Oxford University Press. pp. 48
StateSpaceModels.LocalLevelExplanatory
— TypeLocalLevelExplanatory(y::Vector{Fl}, X::Matrix{Fl}) where Fl
A local level model with explanatory variables is defined by:
\[\begin{gather*} \begin{aligned} y_{t} &= \mu_{t} + X_{1,t} \cdot \beta_{1,t} + \dots + X_{n,t} \cdot \beta_{n,t} + \varepsilon_{t} \quad &\varepsilon_{t} \sim \mathcal{N}(0, \sigma^2_{\varepsilon})\\ \mu_{t+1} &= \mu_{t} + \xi_{t} &\xi_{t} \sim \mathcal{N}(0, \sigma^2_{\xi})\\ \beta_{1,t+1} &= \beta_{1,t} &\tau_{1, t} \sim \mathcal{N}(0, \sigma^2_{\tau_{1}})\\ \dots &= \dots\\ \beta_{n,t+1} &= \beta_{n,t} &\tau_{n, t} \sim \mathcal{N}(0, \sigma^2_{\tau_{n}})\\\\ \end{aligned} \end{gather*}\]
Example
julia> model = LocalLevelExplanatory(rand(100), rand(100, 1))
LocalLevelExplanatory
StateSpaceModels.LocalLinearTrend
— TypeLocalLinearTrend(y::Vector{Fl}) where Fl
The linear trend model is defined by:
\[\begin{gather*} \begin{aligned} y_{t} &= \mu_{t} + \gamma_{t} + \varepsilon_{t} \quad &\varepsilon_{t} \sim \mathcal{N}(0, \sigma^2_{\varepsilon})\\ \mu_{t+1} &= \mu_{t} + \nu_{t} + \xi_{t} \quad &\xi_{t} \sim \mathcal{N}(0, \sigma^2_{\xi})\\ \nu_{t+1} &= \nu_{t} + \zeta_{t} \quad &\zeta_{t} \sim \mathcal{N}(0, \sigma^2_{\zeta})\\ \end{aligned} \end{gather*}\]
Example
julia> model = LocalLinearTrend(rand(100))
LocalLinearTrend
See more on Finland road traffic fatalities
References
- Durbin, James, & Siem Jan Koopman. (2012). "Time Series Analysis by State Space Methods: Second Edition." Oxford University Press. pp. 44
StateSpaceModels.MultivariateBasicStructural
— TypeMultivariateBasicStructural(y::Matrix{Fl}, s::Int) where Fl
An implementation of a non-homogeneous seemingly unrelated time series equations for basic structural state-space model consists of trend (local linear trend) and seasonal components. It is defined by:
\[\begin{gather*} \begin{aligned} y_{t} &= \mu_{t} + \gamma_{t} + \varepsilon_{t} \quad &\varepsilon_{t} \sim \mathcal{N}(0, \Sigma_{\varepsilon})\\ \mu_{t+1} &= \mu_{t} + \nu_{t} + \xi_{t} \quad &\xi_{t} \sim \mathcal{N}(0, \Sigma_{\xi})\\ \nu_{t+1} &= \nu_{t} + \zeta_{t} \quad &\zeta_{t} \sim \mathcal{N}(0, \Sigma_{\zeta})\\ \gamma_{t+1} &= -\sum_{j=1}^{s-1} \gamma_{t+1-j} + \omega_{t} \quad & \omega_{t} \sim \mathcal{N}(0, \Sigma_{\omega})\\ \end{aligned} \end{gather*}\]
Example
julia> model = MultivariateBasicStructural(rand(100, 2), 12)
MultivariateBasicStructural
References
- Durbin, James, & Siem Jan Koopman. (2012). "Time Series Analysis by State Space Methods: Second Edition." Oxford University Press.
StateSpaceModels.VehicleTracking
— TypeVehicleTracking(y::Matrix{Fl}, ρ::Fl, H::Matrix{Fl}, Q::Matrix{Fl}) where Fl
The vehicle tracking example illustrates a model where there are no hyperparameters, the user defines the parameters $\rho$, $H$ and $Q$ and the model gives the predicted and filtered speed and position. In this case, $y_t$ is a $2 \times 1$ observation vector representing the corrupted measurements of the vehicle's position on the two-dimensional plane in instant $t$.
The position and speed in each dimension compose the state of the vehicle. Let us refer to $x_t^{(d)}$ as the position on the axis $d$ and to $\dot{x}^{(d)}_t$ as the speed on the axis $d$ in instant $t$. Additionally, let $\eta^{(d)}_t$ be the input drive force on the axis $d$, which acts as state noise. For a single dimension, we can describe the vehicle dynamics as
\[\begin{equation} \begin{aligned} & x_{t+1}^{(d)} = x_t^{(d)} + \Big( 1 - \frac{\rho \Delta_t}{2} \Big) \Delta_t \dot{x}^{(d)}_t + \frac{\Delta^2_t}{2} \eta_t^{(d)}, \\ & \dot{x}^{(d)}_{t+1} = (1 - \rho) \dot{x}^{(d)}_{t} + \Delta_t \eta^{(d)}_t, \end{aligned}\label{eq_control} \end{equation}\]
We can cast the dynamical system as a state-space model in the following manner:
\[\begin{align*} y_t &= \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{bmatrix} \alpha_{t+1} + \varepsilon_t, \\ \alpha_{t+1} &= \begin{bmatrix} 1 & (1 - \tfrac{\rho \Delta_t}{2}) \Delta_t & 0 & 0 \\ 0 & (1 - \rho) & 0 & 0 \\ 0 & 0 & 1 & (1 - \tfrac{\rho \Delta_t}{2}) \\ 0 & 0 & 0 & (1 - \rho) \end{bmatrix} \alpha_{t} + \begin{bmatrix} \tfrac{\Delta^2_t}{2} & 0 \\ \Delta_t & 0 \\ 0 & \tfrac{\Delta^2_t}{2} \\ 0 & \Delta_t \end{bmatrix} \eta_{t}, \end{align*}\]
See more on Vehicle tracking
Naive models
Naive models are not state space models but are good benchmarks for forecasting, for this reason we implemented them here.
StateSpaceModels.Naive
— TypeNaive(y::Vector{<:Real})
A naive model where the h step ahead forecast is
\[y_{T+h|T} = y_T\]
References
- Hyndman, Rob J., Athanasopoulos, George. "Forecasting: Principles and Practice"
StateSpaceModels.SeasonalNaive
— TypeSeasonalNaive(y::Vector{<:Real}, seasoanl::Int)
A seasonal naive model where the h step ahead forecast is
\[y_{T+h|T} = y_{T + h - m(k+1)}\]
where m is the seasonal period and k is the integer part of (h-1)/m.
References
- Hyndman, Rob J., Athanasopoulos, George. "Forecasting: Principles and Practice"
StateSpaceModels.ExperimentalSeasonalNaive
— TypeExperimentalSeasonalNaive(y::Vector{<:Real}, seasonal::Int; S::Int = 10_000)
A seasonal naive model where the h step ahead forecast is the mean of the simulation of S scenarios
\[y_{T+h|T} = y_{T + h - m(k+1)} + \varepsilon_t\]
where m is the seasonal period, k is the integer part of (h-1)/m and $\varepsilon_t$ is a sampled error.
We call it experimental because so far we could not find a good reference and implementation. If you know something please post it as an issue.
Automatic forecasting
Some models have various parameters and modelling options. The package provides simple functions that search through different parameters to obtain the best fit for your data without a deeper understanding. The search procedures can be published in scientific papers or purely heuristic designed by the developers. In any case if the documentation explains the procedures and indicates if there are any references.
StateSpaceModels.auto_ets
— Functionauto_ets(y::Vector{Fl}; seasonal::Int = 0) where Fl
Automatically fits the best ExponentialSmoothing
model according to the best AIC between the models:
- ETS(A, N, N)
- ETS(A, A, N)
- ETS(A, Ad, N)
If the user provides the time series seasonality it will search between the models
- ETS(A, N, A)
- ETS(A, A, A)
- ETS(A, Ad, A)
References
- Hyndman, Robin John; Athanasopoulos, George.
Forecasting: Principles and Practice. 2nd ed. OTexts, 2018.
StateSpaceModels.auto_arima
— Functionauto_arima(y::Vector{Fl};
seasonal::Int = 0,
max_p::Int = 5,
max_q::Int = 5,
max_P::Int = 2,
max_Q::Int = 2,
max_d::Int = 2,
max_D::Int = 1,
max_order::Int = 5,
information_criteria::String = "aicc",
allow_mean::Bool = true,
show_trace::Bool = false,
integration_test::String = "kpss",
seasonal_integration_test::String = "seas"
) where Fl
Automatically fits the best SARIMA
model according to the best information criteria
TODO write example
References
- Hyndman, RJ and Khandakar.
Automatic time series forecasting: The forecast package for R. Journal of Statistical Software, 26(3), 2008.
Implementing a custom model
Users are able to implement any custom user-defined model.
Systems
The StateSpaceModel
matrices are represented as a StateSpaceSystem
.
StateSpaceModels.StateSpaceSystem
— TypeStateSpaceSystem
Abstract type that unifies the definition of state space models matrices such as $y, Z, d, T, c, R, H, Q$ for linear models.
StateSpaceModels.LinearUnivariateTimeInvariant
— TypeLinearUnivariateTimeInvariant{Fl}(
y::Vector{Fl},
Z::Vector{Fl},
T::Matrix{Fl},
R::Matrix{Fl},
d::Fl,
c::Vector{Fl},
H::Fl,
Q::Matrix{Fl},
) where Fl <: AbstractFloat
Definition of the system matrices $y, Z, d, T, c, R, H, Q$ for linear univariate time invariant state space models.
\[\begin{gather*} \begin{aligned} y_{t} &= Z\alpha_{t} + d + \varepsilon_{t} \quad &\varepsilon_{t} \sim \mathcal{N}(0, H)\\ \alpha_{t+1} &= T\alpha_{t} + c + R\eta_{t} \quad &\eta_{t} \sim \mathcal{N}(0, Q)\\ \end{aligned} \end{gather*}\]
where:
- $y_{t}$ is a scalar
- $Z$ is a $m \times 1$ vector
- $d$ is a scalar
- $T$ is a $m \times m$ matrix
- $c$ is a $m \times 1$ vector
- $R$ is a $m \times r$ matrix
- $H$ is a scalar
- $Q$ is a $r \times r$ matrix
StateSpaceModels.LinearUnivariateTimeVariant
— TypeLinearUnivariateTimeVariant{Fl}(
y::Vector{Fl},
Z::Vector{Vector{Fl}},
T::Vector{Matrix{Fl}},
R::Vector{Matrix{Fl}},
d::Vector{Fl},
c::Vector{Vector{Fl}},
H::Vector{Fl},
Q::Vector{Matrix{Fl}},
) where Fl <: AbstractFloat
Definition of the system matrices $y, Z, d, T, c, R, H, Q$ for linear univariate time variant state space models.
\[\begin{gather*} \begin{aligned} y_{t} &= Z_{t}\alpha_{t} + d_{t} + \varepsilon_{t} \quad &\varepsilon_{t} \sim \mathcal{N}(0, H_{t})\\ \alpha_{t+1} &= T_{t}\alpha_{t} + c_{t} + R_{t}\eta_{t} \quad &\eta_{t} \sim \mathcal{N}(0, Q_{t})\\ \end{aligned} \end{gather*}\]
where:
- $y_{t}$ is a scalar
- $Z_{t}$ is a $m \times 1$ vector
- $d_{t}$ is a scalar
- $T_{t}$ is a $m \times m$ matrix
- $c_{t}$ is a $m \times 1$ vector
- $R_{t}$ is a $m \times r$ matrix
- $H_{t}$ is a scalar
- $Q_{t}$ is a $r \times r$ matrix
StateSpaceModels.LinearMultivariateTimeInvariant
— TypeTODO
StateSpaceModels.LinearMultivariateTimeVariant
— TypeTODO
Hyperparameters
The model hyperparameters are constant (non-time-varying) parameters that are optimized when fit!
is called. The package provides some useful functions to accelerate experimentation and custom model development.
The getters are:
StateSpaceModels.get_names
— Functionget_names(model::StateSpaceModel)
Get the names of the hyperparameters registered on a StateSpaceModel
.
StateSpaceModels.number_hyperparameters
— Functionnumber_hyperparameters(model::StateSpaceModel)
Get the number of hyperparameters registered on a StateSpaceModel
.
The setters are:
StateSpaceModels.fix_hyperparameters!
— Functionfix_hyperparameters!(model::StateSpaceModel, fixed_hyperparameters::Dict)
Fixes the desired hyperparameters so that they are not considered as decision variables in the model estimation.
Example
julia> model = LocalLevel(rand(100))
LocalLevel model
julia> get_names(model)
2-element Array{String,1}:
"sigma2_ε"
"sigma2_η"
julia> fix_hyperparameters!(model, Dict("sigma2_ε" => 100.0))
LocalLevel model
julia> model.hyperparameters.fixed_constrained_values
Dict{String,Float64} with 1 entry:
"sigma2_ε" => 100.0
StateSpaceModels.set_initial_hyperparameters!
— Functionset_initial_hyperparameters!(model::StateSpaceModel,
initial_hyperparameters::Dict{String, <:Real})
Fill a model with user inputed initial points for hyperparameter optimzation.
Example
julia> model = LocalLevel(rand(100))
LocalLevel model
julia> get_names(model)
2-element Array{String,1}:
"sigma2_ε"
"sigma2_η"
julia> set_initial_hyperparameters!(model, Dict("sigma2_η" => 100.0))
LocalLevel model
julia> model.hyperparameters.constrained_values
2-element Array{Float64,1}:
NaN
100.0
Mappings:
StateSpaceModels.constrain_variance!
— Functionconstrain_variance!(model::StateSpaceModel, str::String)
Map a constrained hyperparameter $\psi \in \mathbb{R}^+$ to an unconstrained hyperparameter $\psi_* \in \mathbb{R}$.
The mapping is $\psi = \psi_*^2$
StateSpaceModels.unconstrain_variance!
— Functionunconstrain_variance!(model::StateSpaceModel, str::String)
Map an unconstrained hyperparameter $\psi_{*} \in \mathbb{R}$ to a constrained hyperparameter $\psi \in \mathbb{R}^+$.
The mapping is $\psi_{*} = \sqrt{\psi}$.
StateSpaceModels.constrain_box!
— Functionconstrain_box!(model::StateSpaceModel, str::String, lb::Fl, ub::Fl) where Fl
Map a constrained hyperparameter $\psi \in [lb, ub]$ to an unconstrained hyperparameter $\psi_* \in \mathbb{R}$.
The mapping is $\psi = lb + \frac{ub - lb}{1 + \exp(-\psi_{*})}$
StateSpaceModels.unconstrain_box!
— Functionunconstrain_box!(model::StateSpaceModel, str::String, lb::Fl, ub::Fl) where Fl
Map an unconstrained hyperparameter $\psi_* \in \mathbb{R}$ to a constrained hyperparameter $\psi \in [lb, ub]$.
The mapping is $\psi_* = -\ln \frac{ub - lb}{\psi - lb} - 1$
StateSpaceModels.constrain_identity!
— Functionconstrain_identity!(model::StateSpaceModel, str::String)
Map an constrained hyperparameter $\psi \in \mathbb{R}$ to an unconstrained hyperparameter $\psi_* \in \mathbb{R}$. This function is necessary to copy values from a location to another inside HyperParameters
The mapping is $\psi = \psi_*$
StateSpaceModels.unconstrain_identity!
— Functionunconstrain_identity!(model::StateSpaceModel, str::String)
Map an unconstrained hyperparameter $\psi_* \in \mathbb{R}$ to a constrained hyperparameter $\psi \in \mathbb{R}$. This function is necessary to copy values from a location to another inside HyperParameters
The mapping is $\psi_* = \psi$
Filters and smoothers
StateSpaceModels.jl lets users define tailor-made filters in an easy manner. TODO docs here
StateSpaceModels.UnivariateKalmanFilter
— TypeUnivariateKalmanFilter{Fl <: AbstractFloat}
A Kalman filter that is tailored to univariate systems, exploiting the fact that the dimension of the observations at any time period is 1.
TODO equations and descriptions of a1 and P1
StateSpaceModels.ScalarKalmanFilter
— TypeScalarKalmanFilter{Fl <: Real} <: KalmanFilter
Similar to the univariate Kalman filter but exploits the fact that the dimension of the state is equal to 1.
StateSpaceModels.SparseUnivariateKalmanFilter
— TypeSparseUnivariateKalmanFilter{Fl <: AbstractFloat}
A Kalman filter that is tailored to sparse univariate systems, exploiting the fact that the dimension of the observations at any time period is 1 and that Z, T and R are sparse.
TODO equations and descriptions of a1 and P1
StateSpaceModels.FilterOutput
— TypeFilterOutput{Fl<:Real}
Structure with the results of the Kalman filter:
v
: innovationsF
: variance of innovationsa
: predictive stateatt
: filtered stateP
: variance of predictive statePtt
: variance of filtered statePinf
: diffuse part of the covariance
StateSpaceModels.SmootherOutput
— TypeSmootherOutput{Fl<:Real}
Structure with the results of the smoother:
alpha
: smoothed stateV
: variance of smoothed state
StateSpaceModels.get_innovations
— Functionget_innovations
Returns the innovations v
obtained with the Kalman filter.
StateSpaceModels.get_innovations_variance
— Functionget_innovations_variance
Returns the variance F
of innovations obtained with the Kalman filter.
StateSpaceModels.get_filtered_state
— Functionget_filtered_state
Returns the filtered state att
obtained with the Kalman filter.
StateSpaceModels.get_filtered_state_variance
— Functionget_filtered_state_variance
Returns the variance Ptt
of the filtered state obtained with the Kalman filter.
StateSpaceModels.get_predictive_state
— Functionget_predictive_state
Returns the predictive state a
obtained with the Kalman filter.
StateSpaceModels.get_predictive_state_variance
— Functionget_predictive_state_variance
Returns the variance P
of the predictive state obtained with the Kalman filter.
StateSpaceModels.get_smoothed_state
— Functionget_smoothed_state
Returns the smoothed state alpha
obtained with the smoother.
StateSpaceModels.get_smoothed_state_variance
— Functionget_smoothed_state_variance
Returns the variance V
of the smoothed state obtained with the smoother.
Fitting and Optimizers
StateSpaceModels.jl has an interface for Optim.jl algorithms. The models can be estimated using different algorithms and tunned to the user needs
StateSpaceModels.fit!
— Functionfit!(
model::StateSpaceModel;
filter::KalmanFilter=default_filter(model),
optimizer::Optimizer=Optimizer(Optim.LBFGS()),
save_hyperparameter_distribution::Bool=true
)
Estimate the state-space model parameters via maximum likelihood. The resulting optimal hyperparameters and the corresponding log-likelihood are stored within the model. You can choose the desired filter method (UnivariateKalmanFilter
, ScalarKalmanFilter
, etc.) and the Optim.jl optimization algortihm.
Example
julia> model = LocalLevel(rand(100))
LocalLevel model
julia> fit!(model)
LocalLevel model
julia> model = LocalLinearTrend(LinRange(1, 100, 100) + rand(100))
LocalLinearTrend model
julia> fit!(model; optimizer = Optimizer(StateSpaceModels.Optim.NelderMead()))
LocalLinearTrend model
StateSpaceModels.Optimizer
— TypeOptimizer
An Optim.jl wrapper to make the choice of the optimizer straightforward in StateSpaceModels.jl Users can choose among all suitable Optimizers in Optim.jl using very similar syntax.
Example
julia> using Optim
# use a semicolon to avoid displaying the big log
julia> opt = Optimizer(Optim.LBFGS(), Optim.Options(show_trace = true));
Missing docstring for results
. Check Documenter's build log for details.
StateSpaceModels.has_fit_methods
— Functionhas_fit_methods(model_type::Type{<:StateSpaceModel}) -> Bool
Verify if a certain StateSpaceModel
has the necessary methods to perform fit!
`.
StateSpaceModels.isfitted
— Functionisfitted(model::StateSpaceModel) -> Bool
Verify if model
is fitted, i.e., returns false
if there is at least one NaN
entry in the hyperparameters.
Forecasting and simulating
StateSpaceModels.jl has functions to make forecasts of the predictive densities multiple steps ahead and to simulate scenarios based on those forecasts. The package also has a functions to benchmark the model forecasts using cross_validation techniques.
StateSpaceModels.forecast
— Functionforecast(model::StateSpaceModel, steps_ahead::Int; kwargs...)
forecast(model::StateSpaceModel, exogenous::Matrix{Fl}; kwargs...) where Fl
Forecast the mean and covariance for future observations from a StateSpaceModel.
StateSpaceModels.simulate_scenarios
— Functionsimulate_scenarios(
model::StateSpaceModel, steps_ahead::Int, n_scenarios::Int;
filter::KalmanFilter=default_filter(model)
) -> Array{<:AbstractFloat, 3}
Samples n_scenarios
future scenarios via Monte Carlo simulation for steps_ahead
using the desired filter
.
StateSpaceModels.cross_validation
— Functioncross_validation(model::StateSpaceModel, steps_ahead::Int, start_idx::Int;
n_scenarios::Int = 10_000,
filter::KalmanFilter=default_filter(model),
optimizer::Optimizer=default_optimizer(model)) where Fl
Makes rolling window estimating and forecasting to benchmark the forecasting skill of the model in for different time periods and different lead times. The function returns a struct with the MAE and mean CRPS per lead time. See more on CrossValidation the forecasts of a model
References
- DTU course "31761 - Renewables in electricity markets" available on youtube https://www.youtube.com/watch?v=Ffo8XilZAZw&t=556s
Visualization
Some user friendly plot recipes are defined using RecipesBase.jl. If you have any suggestions do not hesitate to post it as an issue.
using StateSpaceModels, CSV, DataFrames, Plots
air_passengers = CSV.File(StateSpaceModels.AIR_PASSENGERS) |> DataFrame
log_air_passengers = log.(air_passengers.passengers)
model = BasicStructural(log_air_passengers, 12)
fit!(model)
forec = forecast(model, 24)
plot(model, forec; legend = :topleft)
using StateSpaceModels, CSV, DataFrames, Plots
finland_fatalities = CSV.File(StateSpaceModels.VEHICLE_FATALITIES) |> DataFrame
log_finland_fatalities = log.(finland_fatalities.ff)
model = UnobservedComponents(log_finland_fatalities; trend = "local linear trend")
fit!(model)
ks = kalman_smoother(model)
plot(model, ks)
using StateSpaceModels, CSV, DataFrames, Plots
finland_fatalities = CSV.File(StateSpaceModels.NILE) |> DataFrame
model = UnobservedComponents(nile; trend = "local level", cycle = "stochastic")
fit!(model)
kf = kalman_filter(model)
plotdiagnostics(kf)
Datasets
The package provides some datasets to illustrate the funtionalities and models. These datasets are stored as csv files and the path to these files can be obtained through their names as seen below. In the examples we illustrate the datasets using DataFrames.jl and CSV.jl
StateSpaceModels.AIR_PASSENGERS
— ConstantAIR_PASSENGERS
The absolute path for the AIR_PASSENGERS
dataset stored inside StateSpaceModels.jl. This dataset provides monthly totals of a US airline passengers from 1949 to 1960.
See more on Airline passengers
References
- https://www.stata-press.com/data/r12/ts.html
StateSpaceModels.FRONT_REAR_SEAT_KSI
— ConstantFRONT_REAR_SEAT_KSI
The absolute path for the FRONT_REAR_SEAT_KSI
dataset stored inside StateSpaceModels.jl. This dataset provides the log of british people killed or serious injuried in road accidents accross UK.
References
- Commandeur, Jacques J.F. & Koopman, Siem Jan, 2007. "An Introduction to State Space Time Series Analysis," OUP Catalogue, Oxford University Press (Chapter 3)
- http://staff.feweb.vu.nl/koopman/projects/ckbook/OxCodeAll.zip
StateSpaceModels.INTERNET
— ConstantINTERNET
The absolute path for the INTERNET
dataset stored inside StateSpaceModels.jl. This dataset provides the number of users logged on to an Internet server each minute over 100 minutes.
References
- Durbin, James, & Siem Jan Koopman. (2012). "Time Series Analysis by State Space Methods: Second Edition." Oxford University Press. (Chapter 9)
StateSpaceModels.NILE
— ConstantNILE
The absolute path for the NILE
dataset stored inside StateSpaceModels.jl. This dataset provides measurements of the annual flow of the Nile river at Aswan from 1871 to 1970, in $10^8 m^3$.
See more on Nile river annual flow
References
- Durbin, James, & Siem Jan Koopman. (2012). "Time Series Analysis by State Space Methods: Second Edition." Oxford University Press. (Chapter 2)
StateSpaceModels.RJ_TEMPERATURE
— ConstantRJ_TEMPERATURE
Weekly mean temperature in Rio de Janeiro in Kelvin (K).
StateSpaceModels.VEHICLE_FATALITIES
— ConstantVEHICLE_FATALITIES
The absolute path for the VEHICLE_FATALITIES
dataset stored inside StateSpaceModels.jl. This dataset provides the number of annual road traffic fatalities in Norway and Finland.
See more on Finland road traffic fatalities
References
- Commandeur, Jacques J.F. & Koopman, Siem Jan, 2007. "An Introduction to State Space Time Series Analysis," OUP Catalogue, Oxford University Press (Chapter 3)
- http://staff.feweb.vu.nl/koopman/projects/ckbook/OxCodeAll.zip
StateSpaceModels.WHOLESALE_PRICE_INDEX
— ConstantWHOLESALE_PRICE_INDEX
TODO
StateSpaceModels.US_CHANGE
— ConstantUS_CHANGE
Percentage changes in quarterly personal consumption expenditure, personal disposable income, production, savings and the unemployment rate for the US, 1960 to 2016.
Federal Reserve Bank of St Louis.
References
- Hyndman, Rob J., Athanasopoulos, George. "Forecasting: Principles and Practice"