This page covers content from Sections 9.3-9.4

9.3 Probability Distribution Needed for Hypothesis Testing

9.4 Rare Events, the Sample, Decision and Conclusion

Introduction

The following table summarizes various hypothesis tests and corresponding probability distributions that will be used to conduct the test (based on the assumptions shown below):

Probability Distributions for Hypothesis Testing
Type of Hypothesis Test	Population Parameter	Estimated value (point estimate)	Probability Distribution Used
Hypothesis test for the mean, when the population standard deviation is known	Population mean 𝜇	Sample mean ̅̅̅̅̅𝑥	Normal distribution $LaTeX: \overline{X}\sim N\left(\mu_X,\frac{\sigma_X}{\sqrt{n}}\right)$
Hypothesis test for the mean, when the population standard deviation is unknown and the distribution of the sample mean is approximately normal	Population mean 𝜇	Sample mean ̅̅̅̅̅𝑥	Student’s t-distribution, with standard error $LaTeX: \frac{s}{\sqrt{n}}$
Hypothesis test for proportions	Population proportion 𝑝	Sample proportion 𝑝 ′	Normal distribution $LaTeX: P\sim N\left(p,\sqrt{\frac{p\left(1-p\right)}{n}}\right)$

What You Will Do

Make sure you read through all of the information below and watch the videos that are embedded within this page. The information written here is meant to help introduce concepts, and the videos are meant to help show examples and reinforce ideas about these concepts, so both are necessary to help you learn!

Assumptions

To summarize from our textbook:

When you perform a hypothesis test of a single population mean μ using a normal distribution (often called a z-test):

We take a simple random sample from the population
The population you are testing is normally distributed or your sample size is sufficiently large.
You know the value of the population standard deviation, which, in reality, is rarely known.

When you perform a hypothesis test of a single population mean μ using a Student's t-distribution (often called a t-test):

There are fundamental assumptions that need to be met in order for the test to work properly.
Your data should be a simple random sample that comes from a population that is approximately normally distributed.
You use the sample standard deviation to approximate the population standard deviation.
Note that if the sample size is sufficiently large, a t-test will work even if the population is not approximately normally distributed.

When you perform a hypothesis test of a single population proportion p:

You take a simple random sample from the population.
You must meet the conditions for a binomial distribution: there are a certain number n of independent trials, the outcomes of any trial are success or failure, and each trial has the same probability of a success p.
The shape of the binomial distribution needs to be similar to the shape of the normal distribution. To ensure this, the quantities np and nq must both be greater than five (𝑛⁢𝑝 >5 and 𝑛⁢𝑞 >5 ). Then the binomial distribution of a sample (estimated) proportion can be approximated by the normal distribution with $LaTeX: \mu=p$ and 𝜎 = $LaTeX: \sqrt{\frac{p\left(1-p\right)}{n}}$

What is a p-value?

When we conduct a hypothesis test, we will end up getting a p-value, which is a probability, and comparing to $LaTeX: \alpha$ , the probability of a Type I error, also known as the significance level.

The p-value is the probability that, if the null hypothesis is true, the results from another randomly selected sample will be as extreme (or more extreme) as the results obtained from the given sample. In other words, if a p-value is very large, then it is very likely to obtain similar results from other samples, and your sample is not considered "extreme". If a p-value is very small, then it is very unlikely to obtain similar results from other samples, and your sample is considered "extreme."

But what is the cut off for what is "very large" or "very small"?

We use the significant level, $LaTeX: \alpha$ , as that cut off value.

How do we calculate the p-value?

The p-value comes from the probability "outside" of a value called a test statistic, in our distribution. We will learn how to calculate test statistics later, but for now, we are focusing on comparing the p-value to the significance level! (Foreshadowing: A test statistic is a z-score or a t-score of a sample mean or sample proportion in relation to the population distribution, and then we find the probability in the tails related to those values.)

We have three different types of tests:

Left-tailed test
Right-tailed test
Two-tailed test

Example of left-tailed tail:

Assume we are running a hypothesis test for the hypotheses $LaTeX: H_0:\mu=5$ and $LaTeX: H_a:\mu<5$

H_a tells you the test is left-tailed because it uses a "<" sign. The picture of the p-value is as follows:

Normal distribution curve of a single population mean with a value of 5 on the x-axis and the p-value points to the area on the left tail of the curve.

Example of right-tailed test:

Assume our hypotheses are H₀: p ≤ 0.2 and H_a: p > 0.2

H_a tells you the test is right-tailed because it uses the ">" sign. The picture of the p-value is as follows:

Normal distribution curve of a single population proportion with the value of 0.2 on the x-axis. The p-value points to the area on the right tail of the curve.

Example of a two-tailed test:

Assume our hypotheses are H₀: p = 50 and H_a: p ≠ 50

H_a tells you the test is two-tailed because of the "≠" sign. The picture of the p-value is as follows:

Normal distribution curve of a single population mean with a value of 50 on the x-axis. The p-value formulas, 1/2(p-value), for a two-tailed test is shown for the areas on the left and right tails of the curve.

Now you!

Watch this Khan Academy video to get an idea of what a p-value is in a "smaller" type of experiment:

What is a test-statistic and how do we find it?

Let's use an example to illustrate:

Prompt:

Suppose a baker claims that his bread height is more than 15 cm, on average. Several of his customers do not believe him. To persuade his customers that he is right, the baker decides to do a hypothesis test using a significance level of 0.05 (that is, $LaTeX: \alpha=0.05$ ). (Note, the baker knows from baking hundreds of loaves of bread that the distribution of heights is normal.)

He uses a null hypothesis that the mean height of his bread is less than or equal to 15 cm, and the alternative hypothesis that the mean height of his bread is more than 15 cm, as he advertises.

Hypotheses:

$LaTeX: H_0:\mu\le15$

$LaTeX: H_a:\mu>15$

Calculating the test statistic:

For a sample, he bakes 10 loaves of bread. The mean height of the sample loaves is 15.7 cm, and assume a population standard deviation of 0.5 cm. That is, $LaTeX: \overline{x}=15.7$ and $LaTeX: \sigma=0.5$ .

By the Central Limit Theorem, we know the distribution of sample means for samples of 10 loaves is normal with an assumed mean of $LaTeX: \mu=15$ (based on our null hypothesis) and a standard deviation (or standard error) of $LaTeX: \frac{\sigma}{\sqrt{n}}=\frac{0.5}{\sqrt{10}}=0.16$ .

The test statistic in this hypothesis test is the z-score associated with our sample mean in relation to our overall distribution. In other words, we want to know how far away from the mean of 15 cm is our sample mean of 15.7 cm?

Remember to calculate a z-score, we take $LaTeX: \overline{x}-\mu$ and divide by the standard deviation/error. So, the test statistic for this sample mean is found by $LaTeX: \frac{\left(15.7-15\right)}{0.16}=4.375$ .

So, z=4.375 is our test statistic. In other words, for this sample mean of 15.7 cm, it is 4.375 standard deviations above the assumed mean of 15 cm (that is pretty far away, even though 15.7 and 15 seem like they are pretty close to each other!).

Test Statistic for Each Type of Hypothesis Test
Type of Hypothesis Test	Population Parameter	Estimated value (point estimate)	Probability Distribution Used	Test Statistic
Hypothesis test for the mean, when the population standard deviation is known	Population mean 𝜇	Sample mean ̅̅̅̅̅𝑥	Normal distribution $LaTeX: \overline{X}\sim N\left(\mu_X,\frac{\sigma_X}{\sqrt{n}}\right)$	$LaTeX: z=\frac{\overline{x}-\mu}{\left(\frac{\sigma}{\sqrt{n}}\right)}$
Hypothesis test for the mean, when the population standard deviation is unknown and the distribution of the sample mean is approximately normal	Population mean 𝜇	Sample mean ̅̅̅̅̅𝑥	Student’s t-distribution, with standard error $LaTeX: \frac{s}{\sqrt{n}}$	$LaTeX: z=\frac{\overline{x}-\mu}{\left(\frac{s}{\sqrt{n}}\right)}$
Hypothesis test for proportions	Population proportion 𝑝	Sample proportion 𝑝 ′	Normal distribution $LaTeX: P\sim N\left(p,\sqrt{\frac{p\left(1-p\right)}{n}}\right)$	$LaTeX: z=\frac{\left(\hat{p}-p\right)}{\sqrt{\frac{p\left(1-p\right)}{n}}}$

Calculating the p-value:

To find the p-value, we need to identify what type of test we are using here. By our alternative hypothesis, we see this is a right tailed test. So, we need to find the probability to the right of our test statistics (or to the right of our sample mean of 17.

Since we are using a normal distribution, we can use our function in Excel "=1-NORM.S.DIST(4.375, 1)" to find the area to the right of this value or use a z-score table to find an estimated value.

When I do this in Excel, we get p $LaTeX: \approx$ 0.00000607. This means the probability we get another sample that is more extreme our sample mean of 15.7 cm is very low!

To reject the null or to not reject the null? That is the question.

If the p-value is greater than (or equal to) the significance level, then we believe that the probability to obtain similar results is high enough that the sample we have does not provide a strong argument in any direction. That is, if p-value $LaTeX: \ge$ $LaTeX: \alpha$ , then there is not enough evidence to reject our null hypothesis, so we fail to reject and there is not sufficient evidence to conclude that the alternative hypothesis is correct. We can also say there is not enough evidence to support the alternative hypothesis (but we never "accept" a hypothesis!).

If the p-value is less than the significance level, then we believe that the probability to obtain similar results is very low, meaning the sample we have provides a strong argument that the null is in fact incorrect. That is, if p-value $LaTeX: \alpha$ , then there is enough evidence to reject our null hypothesis, so we reject and there is sufficient evidence to conclude that the alternative hypothesis may be correct. We can also say there is enough evidence to support the alternative hypothesis (but we never "accept" a hypothesis!).

Example Continued:

Reminder of prompt: Suppose a baker claims that his bread height is more than 15 cm, on average. Several of his customers do not believe him. To persuade his customers that he is right, the baker decides to do a hypothesis test using a significance level of 0.05 (that is, $LaTeX: \alpha=0.05$ ). (Note, the baker knows from baking hundreds of loaves of bread that the distribution of heights is normal.)

Hypotheses:

$LaTeX: H_0:\mu\le15$

$LaTeX: H_a:\mu>15$

Test statistic is z=4.375, which as a p-value of 0.00000607.

Our significance level was 0.05.

Decision for our Example:

Since our p-value $LaTeX: \alpha$ , then there is enough evidence to reject our null hypothesis. So we reject and there is sufficient evidence to conclude that the alternative hypothesis may be correct (or in other words, there is enough evidence to support the alternative, but do not ever actually accept it"!).

How do we interpret our decision?

We need to write a meaningful conclusion for hypothesis tests. Simply saying "we reject the null" does not mean much when you are trying to communicate what your study was about!

Let's use our example from above to practice making a decision and writing a conclusion.

Conclusion for our Example:

We reject the null hypothesis. Since we reject the null hypothesis that the bread is less than or equal to 15 cm in height, we have reason to believe the true mean is actually larger than 15 cm as the baker claims! That means that there is sufficient evidence to believe that the baker is indeed correct!

Now you!

Watch this Khan Academy video about p-values and significant tests/hypothesis tests:

Watch this Khan Academy video about comparing p-values and significance levels:

Make Connections in our Textbook

Read through Sections 9.3 and 9.4 and the examples provided to help your understanding!

Read about Assumptions to use Probability Distributions in our text

Read about the Sample, Decision, and Conclusion in our text

What's Next

You will complete a Progress Check and CYUs for the material so far.