Time Convolution

The temporal convolution is responsible for capturing temporal patterns in a sequence.

Dilated Temporal Convolution

Unit8 has a nice blog about temporal convolution and dilated temporal convolution1. In this

Convolutions Using Fourier Transform
Convolution and Fourier transform
Dilated Convolution
For a convolution $$ f*h(x) = \sum_{s+t=x} f(s) h(t), $$ the dilated version of it is1 $$ f*_l h(x) = \sum_{s+t*l=x} f(s) h(t), $$ where $l$ is the dilation factor. Yu2015 Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions. arXiv [cs.CV]. 2015. Available: http://arxiv.org/abs/1511.07122  ↩︎


A good convolutional network should capture both short-term and long-term patterns in the time series data. However,

  1. single large kernel is good for long-term pattern but not good at short-term pattern,
  2. single small kernel is good for short-term pattern but not good at long-term pattern.

The inception strategy employs multiple dilated temporal convolution and the outputs are concatenated. By applying multiple such layers, we can extract patterns longer than any of the single dilated temporal convolutions2.

$$ \operatorname{concat}( z\star f_2, z\star f_3, z\star f_6, z\star f_7). $$

Dilated Inception Layer

See Fig 5 in the paper2.

  1. Lässig2021 Lässig F. Temporal Convolutional Networks and Forecasting. In: Unit8 [Internet]. 6 Jul 2021 [cited 28 Nov 2022]. Available: https://unit8.com/resources/temporal-convolutional-networks-and-forecasting/  ↩︎

  2. Wu2020 Wu Z, Pan S, Long G, Jiang J, Chang X, Zhang C. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2005.11650  ↩︎

Planted: by ;

L Ma (2022). 'Time Convolution', Datumorphism, 11 April. Available at: https://datumorphism.leima.is/cards/forecasting/time-convolution/.