Artificial Neural Networks
Artificial Neural Networks
5 Layer Norm
Published:
Category: { Machine Learning }
Tags:
References:
- Ba JL, Kiros JR, Hinton GE. Layer Normalization. arXiv [stat.ML]. 2016. Available: http://arxiv.org/abs/1607.06450
- Brody S, Alon U, Yahav E. On the expressivity role of LayerNorm in Transformers’ attention. arXiv [cs.LG]. 2023. Available: http://arxiv.org/abs/2305.02582
- Xu J, Sun X, Zhang Z, Zhao G, Lin J. Understanding and Improving Layer Normalization. Advances in Neural Information Processing Systems. 2019;32. Available: https://proceedings.neurips.cc/paper_files/paper/2019/file/2f4fe03d77724a7217006e5d16728874-Paper.pdf
Summary: Layer norm
Pages: 5
4 Deep Autoregressive Network
Published:
Category: { Machine Learning }
Tags:
Summary: DARN
Pages: 5
2 A Physicist's Crash Course on Artificial Neural Network
Published:
Category: { Artificial Neural Networks }
Tags:
References:
-
Summary: A very very brief introduction to neural network for physicists
Pages: 5
1 Artificial Neural Networks
Published:
Category: { Artificial Neural Networks }
Tags:
References:
- Hassoun MH, Assistant Professor of Computer Engineering Mohamad H Hassoun. Fundamentals of Artificial Neural Networks. MIT Press; 1995. Available: https://mitpress.mit.edu/books/fundamentals-artificial-neural-networks
- Shenouda EAMA. A Quantitative Comparison of Different MLP Activation Functions in Classification. Advances in Neural Networks - ISNN 2006. Springer Berlin Heidelberg; 2006. pp. 849–857. doi:10.1007/11759966_125
- Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 359–366.
- Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals, and Systems, 2(4), 303–314.
- Freitag KJ. Neural networks and differential equations. San Jose State University. 2007. doi:10.31979/etd.h2n8-mb9r
- Tensorflow and deep learning - without a PhD by Martin Görner
- Kolmogorov, A. N. (1957). On the Representation of Continuous Functions of Several Variables by Superposition of Continuous Functions of one Variable and Addition, Doklady Akademii. Nauk USSR, 114, 679-681.
- Maxwell Stinchcombe, Halbert White (1989). Multilayer feedforward networks are universal approximators. Neural Networks, Vol 2, 5, 359-366.
- Performance Analysis of Various Activation Functions in Generalized MLP Architectures of Neural Networks
- Lippe P. Tutorial 3: Activation Functions — UvA DL Notebooks v1.1 documentation. In: UvA Deep Learning Tutorials [Internet]. [cited 23 Sep 2021]. Available: https://uvadlc-notebooks.readthedocs.io
- Srivastava RK, Greff K, Schmidhuber J. Highway Networks. arXiv [cs.LG]. 2015. Available: http://arxiv.org/abs/1505.00387
Summary: Simple artificial neural networks using multilayer perceptron
Pages: 5