# Learning Theories

### Introduction: My Knowledge Cards

# PAC: Probably Approximately Correct

Published: 2020-01-16

Category: { Machine Learning::Theories }

Tags:

Summary:

Pages: 5

# SRM: Structural Risk Minimization

Published: 2021-02-18

Category: { Machine Learning::Theories }

Tags:

References:
- Structural risk minimization @ Wikipedia
- Murphy, K. P. (2012). Probabilistic Machine Learning: An Introduction.

Summary: ERM In a learning problem The learning problem posed by Vapnik:1 Given a sample: $\{z_i\}$ in the probability space $Z$; Assuming a probability measure on the probability space $Z$; Assuming a set of functions $Q(z, \alpha)$ (e.g. loss functions), where $\alpha$ is a set of parameters; A risk functional to be minimized by tunning “the handles” $\alpha$, $R(\alpha)$. The risk functional is $$ R(\alpha) = \int Q(z, \alpha) \,\mathrm d F(z).

Pages: 5

# ERM: Empirical Risk Minimization

Published: 2021-02-18

Category: { Machine Learning::Theories }

Tags:

Summary: In a learning problem The learning problem posed by Vapnik:1 Given a sample: $\{z_i\}$ in the probability space $Z$; Assuming a probability measure on the probability space $Z$; Assuming a set of functions $Q(z, \alpha)$ (e.g. loss functions), where $\alpha$ is a set of parameters; A risk functional to be minimized by tunning “the handles” $\alpha$, $R(\alpha)$. The risk functional is $$ R(\alpha) = \int Q(z, \alpha) \,\mathrm d F(z).

Pages: 5

# The Learning Problem

Published: 2021-05-06

Category: { Machine Learning::Theories }

Tags:

References:
- Vladimir N. Vapnik. The Nature of Statistical Learning Theory. 2000. doi:10.1007/978-1-4757-3264-1

Summary: The learning problem posed by Vapnik:1
Given a sample: $\{z_i\}$ in the probability space $Z$; Assuming a probability measure on the probability space $Z$; Assuming a set of functions $Q(z, \alpha)$ (e.g. loss functions), where $\alpha$ is a set of parameters; A risk functional to be minimized by tunning “the handles” $\alpha$, $R(\alpha)$. The risk functional is
$$ R(\alpha) = \int Q(z, \alpha) \,\mathrm d F(z). $$
A learning problem is the minimization of this risk.

Pages: 5

# Cross Validation

Published: 2021-05-06

Category: { Machine Learning::Theories }

Tags:

Summary: Cross validation is a method to estimate the risk The learning problem posed by Vapnik:1 Given a sample: $\{z_i\}$ in the probability space $Z$; Assuming a probability measure on the probability space $Z$; Assuming a set of functions $Q(z, \alpha)$ (e.g. loss functions), where $\alpha$ is a set of parameters; A risk functional to be minimized by tunning “the handles” $\alpha$, $R(\alpha)$. The risk functional is $$ R(\alpha) = \int Q(z, \alpha) \,\mathrm d F(z).

Pages: 5