Hypothesis Testing | Statistical Consulting Group

Hypothesis testing allows us to evaluate a hypothesis, or compare two hypotheses. I include it here as part of the theoretical framework necessary for validation of the statistical models offered.

In hypothesis testing, there are two hypotheses.

$H_0$ is the null hypothesis. This is generally the hypothesis we are trying to disprove. We attempt to mount evidence against the null hypothesis.
$H_1$ is the alternative hypothesis. This is the hypothesis we are ready to accept if the null hypothesis is not rejected.

Types of Error – There are 2 types of error associated with hypothesis testing.

$\alpha$ is the probability of rejecting $H_0$ given $H_0$ is true.
is the probability of failing to reject given is false.
- Power of a test is given as $1-\beta$ . It is the probability of rejecting $H_0$ given $H_0$ is false.

A $p$ -value, associated with a test statistic, is the largest $\alpha$ -level at which we would still fail to reject the null hypothesis, given the data on hand. It is interpreted as the probability of seeing a sample (the data) as extreme or more extreme than what you saw, given $H_0$ (the null hypothesis) is true. We can NOT interpret the p-value as the likelihood that the null hypothesis is true. Data can not serve to prove the null hypothesis is true, data only offers evidence that it is untrue. We evaluate the strength of that evidence using the hypothesis test.

Example: Normal Hypothesis Testing, Known Variance, Unknown Mean
Let us assume that a sample of size $n$ is taken from a population with a known variance $\sigma^2$ , and unknown mean $\mu$ . We know this population to be normally distributed, so we can model the sample mean using a normal distribution without any further assumptions.

$\begin{align*} x_i : i = 1,\ldots,n &\sim N(\mu, \sigma^2)\\ \xbar = \frac{\sum_{i=1}^n x_i}{n} &\sim N(\mu,\frac{\sigma^2}{n})\\ \end{align*}$

Let’s say we have an initial guess as to the population mean, that we’re trying to disprove. We represent that with our null hypothesis. The alternative is given as the alternative hypothesis.

$H_0$ : $\mu = \mu_0$
$H_1$ : $\mu \neq \mu_0$

By nature of their definitions, we can’t calculate Power or $\beta$ levels without an actual value for $H_1$ , so I’m not going to here.

The test statistic for the normal distribution, $Z$ , is defined as $\frac{\xbar - \mu_0}{\sigma/\sqrt{n}}$ . $Z$ is normally distributed with mean 0 and variance 1. In this case, because we’re conducting a two-sided test, we calculate the rejection region of our test as being outside the range $(Z_{\frac{\alpha}{2}}, Z_{1-\frac{\alpha}{2}})$ . Depending on our intended level of significance, we go to the Z-table to get those values. If we use an $\alpha$ level of .05, then that range is approximately (-1.96, 1.96).

So if the $Z$ calculated from $Z=\frac{\xbar - \mu_0}{\sigma/\sqrt{n}}$ is greater than 1.96, or less than -1.96, then we reject the null hypothesis.

Example: Normal Hypothesis Testing, Unknown Variance
Let us assume we have a sample of size $n$ , taken from a population with an unknown variance $\sigma^2$ , and an unknown mean $\mu$ . We believe this population to be normally distributed, but since we don’t know the variance we can not use the normal model. Thus enters the Student’s T-test.

Because we don’t know the variance, we have to use an estimator of variance. The unbiased estimator of variance is $S^2 = \frac{\sum(x_i - \xbar)^2}{n-1}$ . We use $n-1$ degrees of freedom here because we subtract 1 for calculation of $\xbar$ .

Since we used an estimator of variance, we must use a different test statistic. Enter the $t$ statistic.

$\begin{equation*} T = \frac{\xbar - \mu_0}{S/\sqrt{n}} \sim t_{n-1} \end{equation*}$

The $T$ follows a student’s t distribution with $n-1$ degrees of freedom. The $t$ distribution itself asymptotically approaches the normal distribution as the degrees of freedom go up. In practice, it’s generally safe for sample sizes larger than 30 to simply use the normal distribution. However, because we’re performing calculations by computer, we can let the computer do the calculations and use the t distribution with however many degrees of freedom we have.

But anyways, we find the appropriate critical $t$ value for the given degrees of freedom, and $\alpha$ level. If $T$ as calculated by $T = \frac{\xbar - \mu_0}{S/\sqrt{n}}$ is less than $t_{\frac{\alpha}{2}, n-1}$ or greater than $t_{1-\frac{\alpha}{2},n-1}$ , then we reject the null hypothesis.