Hypothesis Testing

Hypothesis testing allows us to evaluate a hypothesis, or compare two hypotheses. I include it here as part of the theoretical framework necessary for validation of the statistical models offered.

In hypothesis testing, there are two hypotheses.

  • H_0 is the null hypothesis. This is generally the hypothesis we are trying to disprove. We attempt to mount evidence against the null hypothesis.
  • H_1 is the alternative hypothesis. This is the hypothesis we are ready to accept if the null hypothesis is not rejected.

Types of Error – There are 2 types of error associated with hypothesis testing.

  • \alpha is the probability of rejecting H_0 given H_0 is true.
  • \beta is the probability of failing to reject H_0 given H_0 is false.
    • Power of a test is given as 1-\beta. It is the probability of rejecting H_0 given H_0 is false.

A p-value, associated with a test statistic, is the largest \alpha-level at which we would still fail to reject the null hypothesis, given the data on hand. It is interpreted as the probability of seeing a sample (the data) as extreme or more extreme than what you saw, given H_0 (the null hypothesis) is true. We can NOT interpret the p-value as the likelihood that the null hypothesis is true. Data can not serve to prove the null hypothesis is true, data only offers evidence that it is untrue. We evaluate the strength of that evidence using the hypothesis test.

Example: Normal Hypothesis Testing, Known Variance, Unknown Mean
Let us assume that a sample of size n is taken from a population with a known variance \sigma^2, and unknown mean \mu. We know this population to be normally distributed, so we can model the sample mean using a normal distribution without any further assumptions.

    \begin{align*} x_i : i = 1,\ldots,n &\sim N(\mu, \sigma^2)\\ \xbar = \frac{\sum_{i=1}^n x_i}{n} &\sim N(\mu,\frac{\sigma^2}{n})\\ \end{align*}

Let’s say we have an initial guess as to the population mean, that we’re trying to disprove. We represent that with our null hypothesis. The alternative is given as the alternative hypothesis.

  • H_0: \mu = \mu_0
  • H_1: \mu \neq \mu_0

By nature of their definitions, we can’t calculate Power or \beta levels without an actual value for H_1, so I’m not going to here.

The test statistic for the normal distribution, Z, is defined as \frac{\xbar - \mu_0}{\sigma/\sqrt{n}}. Z is normally distributed with mean 0 and variance 1. In this case, because we’re conducting a two-sided test, we calculate the rejection region of our test as being outside the range (Z_{\frac{\alpha}{2}}, Z_{1-\frac{\alpha}{2}}). Depending on our intended level of significance, we go to the Z-table to get those values. If we use an \alpha level of .05, then that range is approximately (-1.96, 1.96).

So if the Z calculated from Z=\frac{\xbar - \mu_0}{\sigma/\sqrt{n}} is greater than 1.96, or less than -1.96, then we reject the null hypothesis.

Example: Normal Hypothesis Testing, Unknown Variance
Let us assume we have a sample of size n, taken from a population with an unknown variance \sigma^2, and an unknown mean \mu. We believe this population to be normally distributed, but since we don’t know the variance we can not use the normal model. Thus enters the Student’s T-test.

Because we don’t know the variance, we have to use an estimator of variance. The unbiased estimator of variance is S^2 = \frac{\sum(x_i - \xbar)^2}{n-1}. We use n-1 degrees of freedom here because we subtract 1 for calculation of \xbar.

Since we used an estimator of variance, we must use a different test statistic. Enter the t statistic.

    \begin{equation*} T = \frac{\xbar - \mu_0}{S/\sqrt{n}} \sim t_{n-1} \end{equation*}

The T follows a student’s t distribution with n-1 degrees of freedom. The t distribution itself asymptotically approaches the normal distribution as the degrees of freedom go up. In practice, it’s generally safe for sample sizes larger than 30 to simply use the normal distribution. However, because we’re performing calculations by computer, we can let the computer do the calculations and use the t distribution with however many degrees of freedom we have.

But anyways, we find the appropriate critical t value for the given degrees of freedom, and \alpha level. If T as calculated by T = \frac{\xbar - \mu_0}{S/\sqrt{n}} is less than t_{\frac{\alpha}{2}, n-1} or greater than t_{1-\frac{\alpha}{2},n-1}, then we reject the null hypothesis.