An Example of a Hypothesis Test using the Likelihood Ratio
Introduction
The Neyman-Pearson approach is to cast a hypothesis test as a decision problem, and focus on the errors. The idea is to fix \(\alpha\), the probability of a type I error, and then maximize the power for that fixed \(\alpha\). The possible rejection regions are found for the alpha we set, and then we can pick the one with the largest power.
In this chapter, we will use the Neyman-Pearson approach to analyze a hypothesis test regarding whether a coin is fair or biased towards heads. Recall the definitions of the rejection and acceptance regions:
- Rejection Region (Critical Region): The set of values for the test statistic that leads us to reject the null hypothesis (and de facto accept the alternative hypothesis).
- Acceptance Region: The complement of the rejection region (where we fail to reject \(H_0\), and thus accept it).
Example: Coin Toss
Let \(X \sim Bin(5, p)\) with our hypotheses being: \[ H_0: p = p_0 = 1/2 \text{ vs } H_1: p = p_1 = 3/4 \]
Recall that the binomial probability mass function with \(n=5\), and \(p \in \{1/2, 3/4\}\):
\[ P(X=x) = \binom{5}{x}p^k(1-p)^{5-k} \] Fill in the values for the two mass functions \(f_0(x) = P(X=x\mid H_0)\) and \(f_1(x) = P(X=x\mid H_1)\), and the likelihood ratio \(f_0 / f_1\):
| \(x\) | 0 | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|---|
| \(f_0\) | ||||||
| \(f_1\) | ||||||
| \(f_0 / f_1\) |
Check your computations
| \(x\) | 0 | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|---|
| \(f_0\) | \(1/32\) | \(5/32\) | \(10/32\) | \(10/32\) | \(5/32\) | \(1/32\) |
| \(f_1\) | \(1/1024\) | \(15/1024\) | \(90/1024\) | \(270/1024\) | \(405/1024\) | \(243/1024\) |
| \(f_0 / f_1\) | \(32/1\) | \(32/3\) | \(32/9\) | \(32/27\) | \(32/81\) | \(32/243\) |
The likelihood ratio \(f_0 / f_1\) is our tool to find the “best” critical (that is, rejection) region (the one that maximizes power for a fixed \(\alpha\)).
1. Significance level \(\alpha = \dfrac{1}{32}\)
First, let’s specify \(\alpha = 1/32\). In this case, what are the possible critical regions? Which one should we use?
We look for sets \(R\) of possible values of \(x\) such that the probability that \(P(X \in R \mid H_0) = \dfrac{1}{32}\). After you fill the table above, you will see that there are two possible candidates for \(R\), namely \(R_1 = \{0\}\) or \(R_2 = \{5\}\). The outcome \(x = 5\) has the smaller likelihood ratio (\(\dfrac{32}{243}\)), and therefore the higher power \(P(X \in R \mid H_1)\).
Note that both \(\alpha\) and power are probabilities of exactly the same event, just under different distributions.
2. Significance level \(\alpha = \dfrac{6}{32}\)
If we set \(\alpha = \dfrac{6}{32}\), we have four possible rejection regions. What are they?
Check your answer
Since \(\alpha = \dfrac{6}{32}\), we can look at all combinations of values of \(x\) that will have this probability under \(H_0\). The possible regions are:
- \(R_1 = \{0,1\}\)
- \(R_2 = \{0,4\}\)
- \(R_3 = \{1,5\}\)
- \(R_4 = \{4,5\}\)
Now you can compute the probability of each of these under \(H_1\). Compute this, and then check your answers.
Check your answer
- \(P(X \in R_1 |H_1) = \dfrac{16}{1024}\)
- \(P(X \in R_2 |H_1) = \dfrac{244}{1024}\)
- \(P(X \in R_3 |H_1) = \dfrac{258}{1024}\)
- \(P(X \in R_4 |H_1) = \dfrac{648}{1024}\)
Therefore, we will use \(R_4\) as our rejection region. Note that the outcomes in this event correspond to the lowest values of the likelihood ratios. Of course, we could still make an error.
Here is the ways in which we could be right - or wrong:
| \(H_0\) is True | \(H_1\) is True | |
|---|---|---|
| Accept \(H_0\) | Correct Decision | Type II Error (\(\beta\)) |
| Reject \(H_0\) | Type I Error (\(\alpha\)) | Correct Decision (Power) |
We usually have more severe consequences from falsely rejecting a true null, so we try to keep the type I error small. In the next lecture, we will state and discuss the Neyman-Pearson lemma which gives us the best rejection region, and therefore the best test for a specified significance level.