Continuous Ranked Probability Score - CRPS
Published:
Category: { Time Series }
Tags:
References:
- Hersbach H. Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems. Weather Forecast. 2000;15: 559–570. doi:10.1175/1520-0434(2000)015<0559:DOTCRP>2.0.CO;2
- Gouttes A, Rasul K, Koren M, Stephan J, Naghibi T. Probabilistic Time Series Forecasting with Implicit Quantile Networks. arXiv [cs.LG]. 2021. doi:10.1109/icdmw.2017.19
- Gebetsberger M, Messner JW, Mayr GJ, Zeileis A. Estimation Methods for Nonhomogeneous Regression Models: Minimum Continuous Ranked Probability Score versus Maximum Likelihood. Mon Weather Rev. 2018;146: 4323–4338. doi:10.1175/MWR-D-17-0364.1
Summary: The Continuous Ranked Probability Score, known as CRPS, is a score to measure how a proposed distribution approximates the data, without knowledge about the true distributions of the data.
Definition CRPS is defined as1
$$ CRPS(P, x_a) = \int_{-\infty}^\infty \lVert P(x) - H(x - x_a) \rVert_2 dx, $$
where
$x_a$ is the true value of $x$, P(x) is our proposed cumulative distribution for $x$, $H(x)$ is the Heaviside step function $$ H(x) = \begin{cases} 1, &\qquad x=0\\ 0, &\qquad x\leq 0\\ \end{cases} $$
$\lVert \cdot \rVert_2$ is the L2 norm. Explain it The formula looks abstract on first sight, but it becomes crystal clear once we understand it.
Pages: 4
Data Generating Processes for Time Series Data
Published:
Category: { Time Series }
Tags:
Summary:
Pages: 4
Root Relative Squared Error (RSE)
Published:
Category: { Time Series }
Tags:
Summary: The Root Relative Squared Error (RSE) is an evaluation metric in time series forecasting,1
$$ \mathrm{RSE} = \frac{ \sqrt{ \sum_{i, t} ( y^{(i)}_t - \hat y^{(i)}_t )^2 } }{ \sqrt{ \sum_{i, t} ( y^{(i)}_t - \bar y )^2 } } $$
where $y^{(i)}$ is the $i$th time series, ${} _ t$ denotes the time step $t$, and $\bar y$ is the mean of the forecasted series, i.e., $\bar y = \operatorname{mean}(y^{(i\in\{0, 1, \cdots, N\})} _ { t\in \{T _ f, T _ {f+1}, \cdots T _ {f+H}\} })$.
Lai2017 Lai G, Chang W-C, Yang Y, Liu H. Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks.
Pages: 4
Empirical Correlation Coefficient (CORR)
Published:
Category: { Time Series }
Tags:
Summary: The Empirical Correlation Coefficient (CORR) is an evaluation metric in time series forecasting,1
$$ \mathrm{CORR} = \frac{1}{N} \sum_{i=1}^N \frac{ \sum_t (y^{(i)}_t - \bar y^{(i)} ) ( \hat y^{(i)}_t -\bar{ \hat y}^{(i)} ) }{ \sqrt{ \sum_t (y^{(i)}_t - \bar y^{(i)} )^2 ( \hat y^{(i)}_t -\bar{\hat y}^{(i)} )^2 } } $$
where $y^{(i)}$ is the $i$th time series, ${} _ t$ denotes the time step $t$, and $\bar y^{(i)}$ is the mean of the $i$th forecasted series, i.e., $\bar y^{(i)} = \operatorname{mean}( y^{(i)} _ { t \in \{T _ f, T _ {f+1}, \cdots T _ {f+H}\} } )$.
Lai2017 Lai G, Chang W-C, Yang Y, Liu H.
Pages: 4