Neyman-Pearson Theory

The Neyman-Pearson hypothesis testing tests two hypothesis, hypothesis $H$, and an alternative hypothesis $H_A$.

Neyman-Pearson Lemma

The Neyman-Pearson Lemma is an very intuitive lemma to understand how to choose a hypothesis. The lecture notes from PennState is a very good read on this topic¹.

An example

For simplicity, we assume that there exists a test statistic $T$ and $T$ can be used to measure how likely the hypothesis $H$ is true, e.g., the hypothesis $H$ is false, corresponds to $T$ being small.

One example is the ratio of likelihood²,

$$ T \to L(H)/L(H_A), $$

where $T$ will be large if $H$ is true.

To be able to judge the hypothesis, we “loosely” define a probability

$$ p_H = P(T\leq c\vert H), $$

where $c$ is a preset critical value of $T$. Given a threshold $\epsilon$ for $p_H$, we claim the hypothesis $H$ can be rejected if $p_H<\epsilon$.