Level Set Forecaster

A level set forecaster converts point forecaster to probabilistic forecasters by constructing the [[level set]] Level Set Level set can be used in ML of the forecaster1.

Given a point forecaster $f(x_1, \cdots, x_d)$ trained on dataset $\mathcal D = {(\mathcal x_i, y_i)}$, we collect the predictions and true values and build a map, $f(x_i) \to [y_{i_1}, y_{i_2}, \cdots, y_{i_m}]$.


We go through the algorithm in Hasson20211 using a small example. To make it easier to understand, we slightly modify some of the steps.

In this small system, we have

  1. a dataset

    $$ \mathcal D = \{ (x_1, y_1), (x_2, y_2), (x_3, y_3), (x_4, y_4), (x_5, y_5), (x_6, y_6) \},$$

  2. a trained point forecaster $\hat y_i = f(x_i)$.

We collect the unique forecast from the model $f$, and build a list $[v_1, v_2, v_3, v_4]$ so that

  1. $v_i \neq v_j$ for $i\neq j$, and
  2. $v_1 \lt v_2 \lt v_3 \lt v_4$.

Examples of $v_i$

We assume a model $f$ that maps $x_2$ to $\hat y_2=f(x_2)$, and also $x_6$ to $\hat y_6=f(x_6)$.

If $\hat y_2 = \hat y_6$, we define $v_1 := \hat y_2 = \hat y_6$.

We build a map for the unique forecast values that maps back to the data points.

v_1(x_2, y_2), (x_6, y_6)
v_2(x_5, y_5)
v_3(x_1, y_1)
v_4(x_3, y_3), (x_4, y_4)

We loop through $[v_1, v_2, v_3, v_4]$, and build bins of the forecasts, with maximum bin size 3.

  1. $v_1$:
    1. create bin_1 as an empty list
    2. append the value we found in the above table to bin_1, we have bin_1 = [(x_2, y_2), (x_6, y_6)]
  2. $v_2$:
    1. Update bin_1 = [(x_2, y_2), (x_6, y_6), (x_5, y_5)], since length of bin_1 is smaller than max bin size 3.
  3. $v_3$:
    1. bin_1 is full as it reached length 3,
    2. create a new bin: bin_2 = [(x_1, y_1)]
  4. $v_4$:
    1. bin_2 = [(x_1, y_1), (x_3, y_3), (x_4, y_4)]

Finishing the above loop and we have built a new map

v_1[(x_2, y_2), (x_6, y_6), (x_5, y_5)]
v_2[(x_2, y_2), (x_6, y_6), (x_5, y_5)]
v_3[(x_1, y_1), (x_3, y_3), (x_4, y_4)]
v_4[(x_1, y_1), (x_3, y_3), (x_4, y_4)]

With the above new map, we can start predicting quantiles using the point forecaster1.

Feature Space Partition

The algorithm is grouping data point that has similar predictions from our model. The input data in these data points forms the level set of our trained model $f$.

The math for the level set formalism can be found in Hasson20211.

Planted: by ;

Additional Double Backet Links:

L Ma (2022). 'Level Set Forecaster', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/wiki/forecasting/probablistic/level-set-forecaster/.