Time Convolution

#Time Series #Forecasting #Convolution Neural Network

The temporal convolution is responsible for capturing temporal patterns in a sequence.

Dilated Temporal Convolution

Unit8 has a nice blog about temporal convolution and dilated temporal convolution¹. In this

For a convolution $$ f*h(x) = \sum_{s+t=x} f(s) h(t), $$ the dilated version of it is1 $$ f*_l h(x) = \sum_{s+t*l=x} f(s) h(t), $$ where $l$ is the dilation factor. Yu2015 Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions. arXiv [cs.CV]. 2015. Available: http://arxiv.org/abs/1511.07122 ↩︎

Inception

A good convolutional network should capture both short-term and long-term patterns in the time series data. However,

single large kernel is good for long-term pattern but not good at short-term pattern,
single small kernel is good for short-term pattern but not good at long-term pattern.

The inception strategy employs multiple dilated temporal convolution and the outputs are concatenated. By applying multiple such layers, we can extract patterns longer than any of the single dilated temporal convolutions².

$$ \operatorname{concat}( z\star f_2, z\star f_3, z\star f_6, z\star f_7). $$