9.1 Two Population Means with Unknown Standard Deviations
SLO 4
Demonstrate an understanding of, and ability to use, basic ideas of statistical processes, including hypothesis tests and confidence interval estimation.
This is where hypothesis testing stops being about one population and starts being about a comparison. The random variable becomes a difference of two sample means, and the standard error has to carry the variation of both samples at once — the same test logic, rebuilt on a parameter with two moving parts.
SLO 5
Identify appropriate statistical techniques and use technology-based statistical analysis to describe, interpret, and communicate results.
The Aspin–Welch degrees of freedom are deliberately not computed by hand: the section reads them off a calculator or rāSHio and asks you to recognize a non-integer df as correct. It also separates significance from size, using Cohen's d to report how large a difference actually is.
Learning Objectives
By the end of this section, you will be able to:
- state the conditions that let you compare the means of two independent samples;
- compute the standard error of a difference of two sample means from the two sample standard deviations;
- carry out an Aspin–Welch t-test and read its degrees of freedom off technology rather than a formula;
- write the conclusion of a two-sample test in the context of the claim being made;
- measure how large a difference is with Cohen's \(d\), separately from whether it is significant.
Every test in Chapter 8 asked about one population: is this mean 65, is that proportion 0.30. The far more common question is comparative. Do the students in the online section score differently from the students in the room? Do graduates of one college leave with more math courses than graduates of another? Nothing about the logic of hypothesis testing changes when you move to two samples — you still write a pair of hypotheses, still standardize, still compare a p-value to \(\alpha\). What changes is the random variable. It is no longer a single sample mean but the difference of two, and the difference carries the variability of both samples at once. That is the whole content of this section: how to put a standard error under a difference, and what distribution the result follows.
9.1.1 Comparing Two Independent Samples
Two samples are independent when the observations in one are drawn separately from the observations in the other, so that knowing a value in the first sample tells you nothing about any value in the second. Each must be a simple random sample from its own distinct population.
Definition 9.1.1 — Independent samples: no link between the two groups.
When comparing two population means, the random variable is the difference of the two sample means, \(\overline{X}_{1} - \overline{X}_{2}\). Its hypothesized value comes from the null hypothesis, which almost always claims the two population means are equal and so sets \(\mu_{1} - \mu_{2} = 0\).
Definition 9.1.2 — The difference of two sample means is itself random.
For the two distinct populations, the shape requirement scales with the sample size:
- if the sample sizes are small, the distributions matter, and each population should be normal;
- if the sample sizes are large, the distributions matter much less, and the populations need not be normal.
That asymmetry is the Central Limit Theorem doing its work from Chapter 6. With enough data the sampling distribution of each mean is close to normal whatever the population looks like, so the difference of the two means is close to normal as well. With very little data you have no such protection, and the normality of the populations is carrying the test.
No calculation on your data can tell you whether the two samples were drawn independently — you have to know how they were collected. Two samples measured on the same subjects (before and after a treatment, left hand and right hand) are not independent, and they get their own method in §9.4. Reaching for a two-sample test on paired data is the single most common way to get a correct-looking answer to the wrong question.
A difference between two samples depends on both the means and the standard deviations. Very different sample means can occur purely by chance when there is a great deal of variation among the individual observations — which is exactly why the difference alone is never the answer. To judge it, you divide it by the standard error, and standardizing in that way is what turns a raw gap between two averages into a test statistic you can look up.
A two-sample test rests on two conditions. Read them as a pair before any arithmetic happens, because the second one behaves differently depending on how much data you have.
Two samples are shown in the table below. Both populations have normal distributions. The means for the two populations are thought to be the same. Is there a difference in the means? Test at the 5% level of significance.
| Sample Size | Sample Mean | Sample Standard Deviation | |
|---|---|---|---|
| Population A | 25 | 5 | 1 |
| Population B | 16 | 4.7 | 1.2 |
Solution — comparing two populations at the 5% level
Step 1 — Identify the setting. Two independent samples, two population means, and no population standard deviations given — only the two sample standard deviations. That combination is the Student's t case of this section.
Step 2 — Write the hypotheses. "Thought to be the same" puts the equality in the null, and there is no direction anywhere in the wording, so the test is two-tailed:
$$ H_{0}: \mu_{A} = \mu_{B} \qquad H_{a}: \mu_{A} \neq \mu_{B} $$Step 3 — Standardize the difference. The standard error is
$$ \sqrt{\frac{1^{2}}{25} + \frac{1.2^{2}}{16}} = \sqrt{0.04 + 0.09} = 0.3606 $$so the test statistic is \(t = \frac{5 - 4.7}{0.3606} = 0.832\), with \(df \approx 27.86\) from technology.
Step 4 — Compare and decide. The two-tailed p-value is 0.4125. Since \(0.4125 > 0.05\), do not reject \(H_{0}\).
Answer: At the 5% level of significance there is not sufficient evidence to conclude that the two population means are different. The gap of 0.3 is comfortably within what this much sampling variation produces on its own.
The average amount of time boys and girls aged seven to 11 spend playing sports each day is believed to be the same. Karim Haddad, a kinesiology researcher at the county parks department, runs a study and collects the data in the table below. Each population has a normal distribution. Is there a difference in the mean amount of time boys and girls aged seven to 11 play sports each day? Test at the 5% level of significance.
| Sample Size | Average Number of Hours Playing Sports Per Day | Sample Standard Deviation | |
|---|---|---|---|
| Girls | 9 | 2 | 0.866 |
| Boys | 16 | 3.2 | 1.00 |
Solution — a two-tailed test on hours of sport
Step 1 — Name the parameters. The population standard deviations are not known. Let \(g\) be the subscript for girls and \(b\) for boys, so \(\mu_{g}\) is the population mean for girls and \(\mu_{b}\) the population mean for boys. This is a test of two independent groups, two population means.
Step 2 — Name the random variable. \(\overline{X}_{g} - \overline{X}_{b}\), the difference in the sample mean amount of time girls and boys play sports each day.
Step 3 — Write the hypotheses. The words "the same" tell you \(H_{0}\) carries the equality. There are no other words indicating a direction, so read \(H_{a}\) as "is different" — a two-tailed test:
$$ H_{0}: \mu_{g} = \mu_{b} \quad \text{or} \quad \mu_{g} - \mu_{b} = 0 $$ $$ H_{a}: \mu_{g} \neq \mu_{b} \quad \text{or} \quad \mu_{g} - \mu_{b} \neq 0 $$Step 4 — Choose the distribution. Use \(t_{df}\), with \(df\) from the formula for independent groups with unknown standard deviations. Technology gives \(df \approx 18.8462\). Do not pool the variances.
Step 5 — Compute. With \(s_{g} = 0.866\) and \(s_{b} = 1\), the observed difference is \(\overline{x}_{g} - \overline{x}_{b} = 2 - 3.2 = -1.2\). The standard error is 0.3819, giving \(t = -3.14\). The two-tailed p-value is 0.0054: half of it lies below \(-1.2\) and half above \(1.2\).
Step 6 — Decide. Since \(\alpha = 0.05 > 0.0054\), reject \(H_{0}\). This means you reject \(\mu_{g} = \mu_{b}\); the means are different.
Answer: At the 5% level of significance, the sample data show there is sufficient evidence to conclude that the mean number of hours that girls and boys aged seven to 11 play sports per day is different.
On a TI-83/83+/84/84+: press STAT, arrow over to TESTS, and press 4:2-SampTTest. Arrow over to Stats and press ENTER. Enter 2 for the first sample mean, .866 for Sx1, 9 for n1, 3.2 for the second sample mean, 1 for Sx2, and 16 for n2. Arrow down to \(\mu_{1}\): and arrow to the \(\neq \mu_{2}\) option. Press ENTER, arrow down to Pooled: and select No, then arrow down to Calculate and press ENTER. The calculator returns p = 0.0054, \(df \approx 18.8462\), and the test statistic \(t = -3.14\). Running the same instructions but choosing Draw instead of Calculate produces the shaded graph.
9.1.2 The Standard Error of a Difference
The standard error of \(\overline{X}_{1} - \overline{X}_{2}\) is
$$ \sqrt{\frac{(s_{1})^{2}}{n_{1}} + \frac{(s_{2})^{2}}{n_{2}}} $$Definition 9.1.3 — Standard error of a difference: the two pieces add.
The test statistic for comparing two population means with unknown standard deviations is
$$ t = \frac{(\overline{x}_{1} - \overline{x}_{2}) - (\mu_{1} - \mu_{2})}{\sqrt{\frac{(s_{1})^{2}}{n_{1}} + \frac{(s_{2})^{2}}{n_{2}}}} $$Definition 9.1.4 — The t statistic: the observed gap, counted in standard-error widths.
The test statistic above is approximated by the Student's t-distribution with degrees of freedom
$$ df = \frac{\left(\frac{(s_{1})^{2}}{n_{1}} + \frac{(s_{2})^{2}}{n_{2}}\right)^{2}}{\left(\frac{1}{n_{1} - 1}\right)\left(\frac{(s_{1})^{2}}{n_{1}}\right)^{2} + \left(\frac{1}{n_{2} - 1}\right)\left(\frac{(s_{2})^{2}}{n_{2}}\right)^{2}} $$When \(n_{1} + n_{2} > 30\) you can use the normal distribution to approximate the Student's t. Use the Student's t-distribution anyway, whenever possible. The approximation was a convenience for the era of printed tables; with technology in front of you it buys nothing and costs a little accuracy.
where \(s_{1}\) and \(s_{2}\) are the sample standard deviations, used as estimates of the unknown population standard deviations \(\sigma_{1}\) and \(\sigma_{2}\).
Notice that the two contributions add under the radical. That is worth pausing on, because the null hypothesis subtracts the means. Variances of independent quantities add whether you are summing or differencing them, so taking a difference does not cancel any uncertainty — it accumulates it from both samples. A difference of two means is always less precisely known than either mean on its own.
where \(\overline{x}_{1}\) and \(\overline{x}_{2}\) are the sample means and \(\mu_{1}\) and \(\mu_{2}\) are the population means. Under a null hypothesis of equal means the term \((\mu_{1} - \mu_{2})\) is zero and the numerator is just the observed difference.
The structure is the same one you have used since Chapter 6: subtract what the null expects, divide by the standard error. Only the two pieces have grown.
The test comparing two independent population means with unknown and possibly unequal population standard deviations is called the Aspin–Welch t-test, and this degrees-of-freedom formula was developed by Aspin and Welch.
Two things about this formula deserve saying plainly. First, the degrees of freedom are not usually a whole number — 18.8462 in Example 9.1.1 — and that is not a mistake to round away. Second, you are not expected to evaluate it by hand. It exists so you understand where the number on your screen came from; a calculator or computer produces it easily, and every worked example in this section reads it off technology. When both sample sizes \(n_{1}\) and \(n_{2}\) are five or larger, the Student's t approximation is very good.
Try it in rāSHio
The formula in Definition 9.1.5 is the one calculation in this section nobody performs by hand, which makes it the natural place to let a tool take over. In rāSHio choose Stats → T Stats and enter Example 9.1.1's summary values — 2 and 0.866 with 9 for the girls, 3.2 and 1 with 16 for the boys — then set the alternative to not-equal. It returns \(t = -3.14\), the non-integer \(df = 18.8462\), and the p-value 0.0054 together, so the degrees of freedom are never a number you have to compute or remember to set. If you have raw observations instead of summaries, paste them in with File → Delimited List and read the mean and standard deviation off Stats → Summary Statistics first.
Figure 9.1.1 — The rāSHio T Stats panel: Stats → T Stats. The walkthrough switches Samples to two and enters Example 9.1.1's own summary values — 2, 0.866 and 9 for the girls against 3.2, 1 and 16 for the boys — then reads the t-statistic, the p-value and the Welch-Satterthwaite degrees of freedom straight off the results card. The df comes back as 18.85, not a whole number, which is exactly the point Definition 9.1.5 is making. It leaves the alternative on the panel's two-tailed default, which is what this example wants.
Because you do not know the population standard deviations, you estimate them with the two sample standard deviations from your independent samples. The quantity you build from them is the standard error of the difference in sample means.
9.1.3 Running the Test End to End
The examples that follow work through a fixed sequence of questions. You should be able to answer all of them about any two-sample test: what kind of test it is, what distribution it uses, what the random variable is, and which tail matters. The last one is the conclusion, written in the language of the original claim.
Some software offers a "pooled" two-sample t-test that averages the two sample variances into one. This book never uses it. Pooling assumes the two populations have equal standard deviations, an assumption you almost never have grounds for, and the unpooled test is barely less powerful when the assumption does happen to hold. If a menu asks, the answer is always No.
Mateo Reyes, an HR analyst who helps run his company's LGBTQ+ employee network, is asked to determine if Company A retains its workers longer than Company B. Company A samples 15 workers, and their average time with the company is five years with a standard deviation of 1.2. Company B samples 20 workers, and their average time with the company is 4.5 years with a standard deviation of 0.8. The populations are normally distributed.
a. Are the population standard deviations known?
b. Conduct an appropriate hypothesis test. At the 5% significance level, what is your conclusion?
Solution — worker retention at two companies
a. No. Only the two sample standard deviations, 1.2 and 0.8, are given, so this is the Student's t case of this section.
b. Step 1 — Write the hypotheses. "Retains its workers longer" is a directional claim about Company A, and it is what the study set out to show, so it belongs in the alternative. With \(A\) and \(B\) for the two companies:
$$ H_{0}: \mu_{A} \leq \mu_{B} \qquad H_{a}: \mu_{A} > \mu_{B} $$This is a right-tailed test.
Step 2 — Standardize. The standard error is
$$ \sqrt{\frac{1.2^{2}}{15} + \frac{0.8^{2}}{20}} = \sqrt{0.096 + 0.032} = 0.3578 $$so \(t = \frac{5 - 4.5}{0.3578} = 1.398\), with \(df \approx 23.01\) from technology.
Step 3 — Find the p-value. The right-tail area beyond \(t = 1.398\) is 0.0878.
Step 4 — Decide. Since \(0.0878 > 0.05\), do not reject \(H_{0}\).
Answer: At the 5% level of significance, there is not sufficient evidence to conclude that Company A retains its workers longer than Company B. Note how close this is — the p-value would clear a 10% significance level. The data lean in the direction the study expected, but not far enough to meet the burden that was set in advance.
Grace Yuen, who chairs the education committee of a community group and its LGBTQ+ student outreach, runs a study in two neighboring colleges to determine which one graduates students with more math classes. College A samples 11 graduates; their average is four math classes with a standard deviation of 1.5 math classes. College B samples nine graduates; their average is 3.5 math classes with a standard deviation of one math class. The community group believes that a student who graduates from college A has taken more math classes, on average. Both populations have a normal distribution. Test at a 1% significance level, and answer the following questions.
a. Is this a test of two means or two proportions?
b. Are the population standard deviations known or unknown?
c. Which distribution do you use to perform the test?
d. What is the random variable?
e. What are the null and alternative hypotheses? Write them in words and in symbols.
f. Is this test right-, left-, or two-tailed?
g. What is the p-value?
h. Do you reject or not reject the null hypothesis?
i. Write the conclusion.
Solution — a right-tailed test at the 1% level
a. Two means. Each college is summarized by an average number of math classes, not by a proportion of successes.
b. Unknown. The problem supplies sample standard deviations only.
c. Student's t.
d. \(\overline{X}_{A} - \overline{X}_{B}\), the difference in the sample mean number of math classes taken by graduates of the two colleges.
e. In words, the null says graduates of College A take no more math classes on average than graduates of College B, and the alternative says they take more. In symbols:
$$ H_{0}: \mu_{A} \leq \mu_{B} \qquad H_{a}: \mu_{A} > \mu_{B} $$f. Right-tailed. The community group's belief points in one direction, and that direction is the alternative. The observed difference is \(\overline{x}_{A} - \overline{x}_{B} = 4 - 3.5 = 0.5\).
g. p-value = 0.1928. The standard error is 0.5618, giving \(t = 0.890\) with \(df \approx 17.40\).
h. Do not reject the null hypothesis. \(0.1928\) is far larger than \(\alpha = 0.01\).
Answer (i): At the 1% level of significance, from the sample data, there is not sufficient evidence to conclude that a student who graduates from college A has taken more math classes, on average, than a student who graduates from college B.
Two professors teach classes on the same subjects. Professor A is Wenona Little Thunder, who moved to the college with her wife two years ago; Professor B is her longtime collaborator. Scores of 10 students from each class are selected randomly. The final exam scores of the students are as follows.
Professor A:
| 97 | 62 | 73 | 58 | 84 |
|---|---|---|---|---|
| 74 | 66 | 93 | 73 | 85 |
Professor B:
| 85 | 64 | 74 | 55 | 76 |
|---|---|---|---|---|
| 67 | 72 | 84 | 71 | 98 |
Professor Little Thunder says that the mean score of her class is more than the mean of Professor B's class. Is she correct? Test at a 5% significance level, and answer the following questions.
a. Is this a test of two means or two proportions?
b. Are the population standard deviations known or unknown?
c. Which distribution do you use to perform the test?
d. What is the random variable?
e. What are the null and alternative hypotheses? Write them in words and in symbols.
f. Is this test right-, left-, or two-tailed?
g. What is the p-value?
h. Do you reject or not reject the null hypothesis?
i. Write the conclusion.
Solution — comparing two sets of exam scores
Step 0 — Summarize the raw data. Professor A's ten scores have mean 76.5 and standard deviation 12.97; Professor B's have mean 74.6 and standard deviation 12.11.
a. Two means. b. Unknown — the standard deviations come from the samples. c. Student's t.
d. \(\overline{X}_{A} - \overline{X}_{B}\), the difference in the sample mean final exam score between the two classes.
e. In words, the null says Professor A's class mean is no higher than Professor B's, and the alternative says it is higher:
$$ H_{0}: \mu_{A} \leq \mu_{B} \qquad H_{a}: \mu_{A} > \mu_{B} $$f. Right-tailed — Professor A's claim points one way.
g. The standard error is 5.612, giving \(t = \frac{76.5 - 74.6}{5.612} = 0.339\) with \(df \approx 17.92\). The right-tail p-value is 0.3694.
h. Do not reject the null hypothesis.
Answer (i): At the 5% level of significance, from the sample data, there is not sufficient evidence to conclude that the mean final exam score of Professor A's class is higher than that of Professor B's class. A gap of 1.9 points against standard deviations near 13 is nothing at all — which is the useful lesson here, because Professor A's claim is true of the samples and still unsupported as a claim about the classes.
Professor Anand Devaraj (they/them), who teaches at a large community college, wanted to determine whether there is a difference in the means of final exam scores between students who took their statistics course online and students who took their face-to-face statistics class. They believed the mean of the final exam scores for the online class would be lower than that of the face-to-face class. Were they correct? Thirty randomly selected final exam scores from each group are listed below. Test at a 5% significance level.
Online class:
| 67.6 | 41.2 | 85.3 | 55.9 | 82.4 | 91.2 | 73.5 | 94.1 | 64.7 | 64.7 |
|---|---|---|---|---|---|---|---|---|---|
| 70.6 | 38.2 | 61.8 | 88.2 | 70.6 | 58.8 | 91.2 | 73.5 | 82.4 | 35.5 |
| 94.1 | 88.2 | 64.7 | 55.9 | 88.2 | 97.1 | 85.3 | 61.8 | 79.4 | 79.4 |
Face-to-face class:
| 77.9 | 95.3 | 81.2 | 74.1 | 98.8 | 88.2 | 85.9 | 92.9 | 87.1 | 88.2 |
|---|---|---|---|---|---|---|---|---|---|
| 69.4 | 57.6 | 69.4 | 67.1 | 97.6 | 85.9 | 88.2 | 91.8 | 78.8 | 71.8 |
| 98.8 | 61.2 | 92.9 | 90.6 | 97.6 | 100 | 95.3 | 83.5 | 92.9 | 89.4 |
Solution — a left-tailed test from raw data
Step 0 — Summarize. The online scores have mean 72.85 with standard deviation 16.92; the face-to-face scores have mean 84.98 with standard deviation 11.71.
a. Two means. b. Unknown. c. Student's t.
d. \(\overline{X}_{1} - \overline{X}_{2}\), the difference in the sample mean final exam score between the online and face-to-face classes.
e. The null hypothesis says the means of the final exam scores are equal for the online and face-to-face statistics classes; the alternative says the mean for the online class is less:
$$ H_{0}: \mu_{1} = \mu_{2} \qquad H_{a}: \mu_{1} < \mu_{2} $$f. Left-tailed. The professor's belief — that the online mean is lower — is the alternative, and it points to the left.
g. p-value = 0.0011, from \(t = -3.229\) with \(df \approx 51.61\).
h. Reject the null hypothesis. \(0.0011\) is well below \(\alpha = 0.05\).
Answer (i): At the 5% level of significance, from the sample data, there is sufficient evidence to conclude that the mean of the final exam scores for the online class is less than the mean of the final exam scores of the face-to-face class. Professor Devaraj was correct.
On a TI-83/83+/84/84+: first put the data for each group into two lists, such as L1 and L2. Press STAT, arrow over to TESTS, and press 4:2-SampTTest. Make sure Data is highlighted and press ENTER. Enter L1 for the first list and L2 for the second. Arrow down to \(\mu_{1}\): and arrow to the \(<\mu_{2}\) option. Press ENTER, arrow down to Pooled: and select No, then arrow down to Calculate and press ENTER. Be careful not to mix up the information for Group 1 and Group 2 — with a one-tailed test, swapping them silently reverses which tail you are asking about.
Try it in rāSHio
Example 9.1.3 hands you sixty raw scores instead of summaries, and retyping them into a summary panel is where the mistakes happen. In rāSHio, paste each class's list into its own column with File → Delimited List — the scores go in one per line, exactly as printed — then read the summaries off Stats → Summary Statistics to confirm 72.85 and 16.92 for the online class against 84.98 and 11.71 for the face-to-face class. Run Stats → T Stats on those same two columns with the alternative set to less-than, and it returns \(t = -3.229\), \(df \approx 51.61\) and the p-value 0.0011 without a single number being copied by hand.
9.1.4 Measuring the Size of a Difference
Cohen's \(d\) is a measure of effect size based on the difference between two means, named for the United States statistician Jacob Cohen. It measures the relative strength of the difference between the means of two populations using sample data:
$$ d = \frac{\overline{x}_{1} - \overline{x}_{2}}{s_{\text{pooled}}} \qquad \text{where} \qquad s_{\text{pooled}} = \sqrt{\frac{(n_{1} - 1)s_{1}^{2} + (n_{2} - 1)s_{2}^{2}}{n_{1} + n_{2} - 2}} $$Definition 9.1.6 — Cohen's d: three real effect sizes on one scale.
The calculated value is then compared to Cohen's standards for small, medium, and large effect sizes.
| Size of effect | \(d\) |
|---|---|
| Small | 0.2 |
| Medium | 0.5 |
| Large | 0.8 |
Read the formula and you can see what \(d\) is doing: it expresses the gap between the means in units of a typical individual's spread. A \(d\) of 0.8 says the two means sit about eight-tenths of a standard deviation apart, whether the measurement is hours, dollars, or exam points. That is why \(d\) is comparable across studies and a p-value is not — the p-value depends on your sample sizes, and \(d\) does not.
Note that \(s_{\text{pooled}}\) does appear here, even though the test in this section deliberately refuses to pool. There is no contradiction: pooling is a bad assumption to build a test on, but a perfectly reasonable way to pick a single yardstick when all you want is a common scale to report the gap in.
A small p-value says a difference is hard to explain by chance. It says nothing at all about whether the difference is large. With big enough samples a trivial gap becomes statistically significant, and with small samples an important gap can fail to reach significance — as Try It Now 9.1.2 just showed. Effect size is the separate question, and it has its own measure.
Calculate Cohen's \(d\) for Try It Now 9.1.2. Is the size of the effect small, medium, or large? Explain what the size of the effect means for this problem.
Solution — effect size for worker retention
Step 1 — Collect the pieces. \(\overline{x}_{A} = 5\), \(s_{A} = 1.2\), \(n_{A} = 15\); \(\overline{x}_{B} = 4.5\), \(s_{B} = 0.8\), \(n_{B} = 20\).
Step 2 — Pool.
$$ s_{\text{pooled}} = \sqrt{\frac{(15 - 1)(1.2)^{2} + (20 - 1)(0.8)^{2}}{15 + 20 - 2}} = \sqrt{\frac{20.16 + 12.16}{33}} = 0.990 $$Step 3 — Divide. \(d = \frac{5 - 4.5}{0.990} = 0.505\).
Answer: The effect is medium, sitting just above Cohen's 0.5. This is the pairing worth remembering: the hypothesis test in Try It Now 9.1.2 did not reject the null, yet the effect size is medium. The test failed to reach significance because 15 and 20 observations are not many, not because the difference is unimportant. A larger study of the same two companies would very likely find it significant.
Calculate Cohen's \(d\) for Example 9.1.2. Is the size of the effect small, medium, or large? Explain what the size of the effect means for this problem.
Solution — a small effect
Step 1 — Collect the pieces. From Example 9.1.2, \(\overline{x}_{1} = 4\), \(s_{1} = 1.5\), \(n_{1} = 11\) for College A, and \(\overline{x}_{2} = 3.5\), \(s_{2} = 1\), \(n_{2} = 9\) for College B.
Step 2 — Pool the standard deviations.
$$ s_{\text{pooled}} = \sqrt{\frac{(11 - 1)(1.5)^{2} + (9 - 1)(1)^{2}}{11 + 9 - 2}} = \sqrt{\frac{22.5 + 8}{18}} = 1.302 $$Step 3 — Divide. \(d = \frac{4 - 3.5}{1.302} = 0.384\).
Answer: The effect is small, because 0.384 falls between Cohen's value of 0.2 for a small effect size and 0.5 for a medium one. The difference between the mean number of math classes at the two colleges is small, which agrees with the test's failure to find a significant difference.
Robin Ashcroft (they/them), who manages a small fund with their wife, tracks weighted alpha — a measure of risk-adjusted performance of stocks over a period of a year. A high positive weighted alpha signifies a stock whose price has risen, while a small positive weighted alpha indicates an unchanged stock price during the time period. Weighted alpha is used to identify companies with strong upward or downward trends. The weighted alphas for the top 30 stocks of banks in the northeast and in the west, as identified by Nasdaq on May 24, 2013, are listed below.
Northeast:
| 94.2 | 75.2 | 69.6 | 52.0 | 48.0 | 41.9 | 36.4 | 33.4 | 31.5 | 27.6 |
|---|---|---|---|---|---|---|---|---|---|
| 77.3 | 71.9 | 67.5 | 50.6 | 46.2 | 38.4 | 35.2 | 33.0 | 28.7 | 26.5 |
| 76.3 | 71.7 | 56.3 | 48.7 | 43.2 | 37.6 | 33.7 | 31.8 | 28.5 | 26.0 |
West:
| 126.0 | 70.6 | 65.2 | 51.4 | 45.5 | 37.0 | 33.0 | 29.6 | 23.7 | 22.6 |
|---|---|---|---|---|---|---|---|---|---|
| 116.1 | 70.6 | 58.2 | 51.2 | 43.2 | 36.0 | 31.4 | 28.7 | 23.5 | 21.6 |
| 78.2 | 68.2 | 55.6 | 50.3 | 39.0 | 34.1 | 31.0 | 25.3 | 23.4 | 21.5 |
Is there a difference in the weighted alpha of the top 30 stocks of banks in the northeast and in the west? Test at a 5% significance level, and answer the following questions.
a. Is this a test of two means or two proportions?
b. Are the population standard deviations known or unknown?
c. Which distribution do you use to perform the test?
d. What is the random variable?
e. What are the null and alternative hypotheses? Write them in words and in symbols.
f. Is this test right-, left-, or two-tailed?
g. What is the p-value?
h. Do you reject or not reject the null hypothesis?
i. Write the conclusion.
j. Calculate Cohen's \(d\) and interpret it.
Solution — weighted alpha in two regions
Step 0 — Summarize. Northeast: mean 47.96, standard deviation 18.99, \(n = 30\). West: mean 47.06, standard deviation 26.23, \(n = 30\).
a. Two means. b. Unknown. c. Student's t.
d. \(\overline{X}_{1} - \overline{X}_{2}\), the difference in the sample mean weighted alpha between northeastern and western bank stocks.
e. The null says the mean weighted alpha is the same in the two regions; the alternative says it differs:
$$ H_{0}: \mu_{1} = \mu_{2} \qquad H_{a}: \mu_{1} \neq \mu_{2} $$f. Two-tailed — "is there a difference" gives no direction.
g. The standard error is 5.913, so \(t = \frac{47.96 - 47.06}{5.913} = 0.153\) with \(df \approx 52.86\). The two-tailed p-value is 0.8787.
h. Do not reject the null hypothesis.
i. At the 5% level of significance, from the sample data, there is not sufficient evidence to conclude that the mean weighted alpha of the top 30 bank stocks differs between the northeast and the west.
j. \(s_{\text{pooled}} = 22.90\), so \(d = \frac{47.96 - 47.06}{22.90} = 0.04\). The effect is far below Cohen's 0.2 for a small effect — essentially no effect at all. Here the large p-value and the negligible effect size tell the same story, which is the ordinary case; the interesting situations are the ones in Try It Now 9.1.4 where they disagree.
Note the difference in the two standard deviations, 18.99 against 26.23. The west's larger spread comes mostly from its two extreme values, 126.0 and 116.1. This is exactly the circumstance the Aspin–Welch test is built for: it never assumes the two populations have the same spread, which is why the unequal standard deviations here cause no trouble.
Calculate Cohen's \(d\) for Example 9.1.3. Is the size of the effect small, medium, or large? Explain what the size of the effect means for this problem.
Solution — a large effect
Step 1 — Collect the pieces. Online: \(\overline{x}_{1} = 72.85\), \(s_{1} = 16.92\), \(n_{1} = 30\). Face-to-face: \(\overline{x}_{2} = 84.98\), \(s_{2} = 11.71\), \(n_{2} = 30\).
Step 2 — Pool. \(s_{\text{pooled}} = 14.55\).
Step 3 — Divide. \(d = \frac{72.85 - 84.98}{14.55} = -0.834\), so \(|d| = 0.834\).
Answer: The effect is large, because 0.834 is greater than Cohen's 0.8. The difference between the mean final exam scores of the online students and the face-to-face students is large, indicating a substantial difference — here the significant p-value and the large effect size agree, and the negative sign simply records that the online mean is the lower of the two.
Problem Set 9.1
Use the following information to answer the next 15 exercises. Indicate whether the hypothesis test is for
a. independent group means, population standard deviations and/or variances known
b. independent group means, population standard deviations and/or variances unknown
c. matched or paired samples
d. single mean
e. two proportions
f. single proportion
Problem 1. It is believed that 70% of men pass their drivers test in the first attempt, while 65% of women pass the test in the first attempt. Of interest is whether the proportions are in fact equal.
Problem 1 Solution
Step 1 — Note what's being measured: passing a driving test on the first attempt is a yes/no outcome for each person, so each group is summarized by a proportion, not a mean.
Step 2 — Count the groups: men and women are two independent groups, and the claim compares their pass rates directly against each other.
Answer: e. two proportions.
Problem 2. A new laundry detergent is tested on consumers. Of interest is the proportion of consumers who prefer the new brand over the leading competitor. A study is done to test this.
Problem 2 Solution
Step 1 — Count the groups: only one group is sampled, consumers in general, and the study asks what fraction of that single group prefers the new brand.
Answer: f. single proportion.
Problem 3. A new windshield treatment claims to repel water more effectively. Ten windshields are tested by simulating rain without the new treatment. The same windshields are then treated, and the experiment is run again. A hypothesis test is conducted.
Problem 3 Solution
Step 1 — Same subjects, twice: the same ten windshields are measured once without the treatment and again after it, so the two sets of measurements are linked pair by pair rather than drawn from separate populations.
Answer: c. matched or paired samples.
Problem 4. The known standard deviation in salary for all mid-level professionals in the financial industry is $11,000. Company A and Company B are in the financial industry. Suppose samples are taken of mid-level professionals from Company A and from Company B. The sample mean salary for mid-level professionals in Company A is $80,000. The sample mean salary for mid-level professionals in Company B is $96,000. Company A and Company B management want to know if their mid-level professionals are paid differently, on average.
Problem 4 Solution
Step 1 — Identify what's unknown: the standard deviation for mid-level salaries, $11,000, is stated directly as a population value for the financial industry, not estimated separately from either company's sample.
Step 2 — Count the groups: Company A and Company B are two independent samples, each summarized by a mean, and the same known standard deviation applies to both.
Answer: a. independent group means, population standard deviations and/or variances known.
Problem 5. The average worker in Germany gets eight weeks of paid vacation.
Problem 5 Solution
Step 1 — Note what's being measured: one number is in question, the average vacation time of German workers, checked against a single sample with no comparison group anywhere in the wording.
Answer: d. single mean.
Problem 6. According to a television commercial, 80% of dentists agree that Ultrafresh toothpaste is the best on the market.
Problem 6 Solution
Step 1 — Count the groups: dentists form one group reporting one percentage figure; there is no second group to compare it against.
Answer: f. single proportion.
Problem 7. It is believed that the average grade on an English essay in a particular school system for women is higher than for men. A random sample of 31 women had a mean score of 82 with a standard deviation of three, and a random sample of 25 men had a mean score of 76 with a standard deviation of four.
Problem 7 Solution
Step 1 — Identify what's unknown: the women's and men's essay scores come with sample standard deviations only, three and four points, so the population standard deviations are unknown.
Step 2 — Count the groups: two independent random samples, each summarized by a mean, support a comparison of two population means.
Answer: b. independent group means, population standard deviations and/or variances unknown.
Problem 8. The league mean batting average is 0.280 with a known standard deviation of 0.06. The Rattlers and the Vikings belong to the league. The mean batting average for a sample of eight Rattlers is 0.210, and the mean batting average for a sample of eight Vikings is 0.260. There are 24 players on the Rattlers and 19 players on the Vikings. Are the batting averages of the Rattlers and Vikings statistically different?
Problem 8 Solution
Step 1 — Identify what's given: the league supplies a known standard deviation, 0.06, that applies league-wide rather than one estimated separately for each team.
Step 2 — Count the groups: the Rattlers and the Vikings are two independent samples, so this compares their mean batting averages using that shared known standard deviation.
Answer: a. independent group means, population standard deviations and/or variances known.
Problem 9. In a random sample of 100 forests in the United States, 56 were coniferous or contained conifers. In a random sample of 80 forests in Mexico, 40 were coniferous or contained conifers. Is the proportion of conifers in the United States statistically more than the proportion of conifers in Mexico?
Problem 9 Solution
Step 1 — Note what's being measured: whether a forest "contains conifers" is a yes/no outcome, so each country is summarized by a proportion of its sampled forests.
Answer: e. two proportions.
Problem 10. A new medicine is said to help improve sleep. Eight subjects are picked at random and given the medicine. The mean hours slept for each person were recorded before starting the medication and after.
Problem 10 Solution
Step 1 — Same subjects, twice: the same eight subjects contribute an hours-slept measurement before the medication and another after, so the two sets of numbers are linked by subject rather than independent.
Answer: c. matched or paired samples.
Problem 11. It is thought that teenagers sleep more than adults on average. A study is done to verify this. A sample of 16 teenagers has a mean of 8.9 hours slept and a standard deviation of 1.2. A sample of 12 adults has a mean of 6.9 hours slept and a standard deviation of 0.6.
Problem 11 Solution
Step 1 — Identify what's unknown: teenagers and adults are each reported with a sample standard deviation only, 1.2 and 0.6, so the population standard deviations are unknown.
Step 2 — Count the groups: two independent samples support a comparison of two means.
Answer: b. independent group means, population standard deviations and/or variances unknown.
Problem 12. Varsity athletes practice five times a week, on average.
Problem 12 Solution
Step 1 — Note what's being measured: practice frequency for varsity athletes is one number checked against a single population, with no second group in the claim.
Answer: d. single mean.
Problem 13. A sample of 12 in-state graduate school programs at school A has a mean tuition of $64,000 with a standard deviation of $8,000. At school B, a sample of 16 in-state graduate programs has a mean of $80,000 with a standard deviation of $6,000. On average, are the mean tuitions different?
Problem 13 Solution
Step 1 — Identify what's unknown: school A and school B's tuition figures come with sample standard deviations, $8,000 and $6,000, not population values.
Step 2 — Count the groups: the two schools are independent samples, so this is a comparison of two means.
Answer: b. independent group means, population standard deviations and/or variances unknown.
Problem 14. A new WiFi range booster is being offered to consumers. A researcher tests the native range of 12 different routers under the same conditions. The ranges are recorded. Then the researcher uses the new WiFi range booster and records the new ranges. Does the new WiFi range booster do a better job?
Problem 14 Solution
Step 1 — Same subjects, twice: the same 12 routers are measured under their native range and again with the booster added, so the ranges are matched by router, not independent.
Answer: c. matched or paired samples.
Problem 15. A high school principal claims that 30% of student athletes drive themselves to school, while 4% of non-athletes drive themselves to school. In a sample of 20 student athletes, 45% drive themselves to school. In a sample of 35 non-athlete students, 6% drive themselves to school. Is the percent of student athletes who drive themselves to school more than the percent of non-athletes?
Problem 15 Solution
Step 1 — Note what's being measured: driving oneself to school is a yes/no outcome, so athletes and non-athletes are each summarized by a proportion, not a mean.
Answer: e. two proportions.
Use the following information to answer the next three exercises. Amara Nwosu's food-science team runs a study to determine which of two soft drinks has more sugar. There are 13 cans of Beverage A in a sample and six cans of Beverage B. The mean amount of sugar in Beverage A is 36 grams with a standard deviation of 0.6 grams. The mean amount of sugar in Beverage B is 38 grams with a standard deviation of 0.8 grams. The researchers believe that Beverage B has more sugar than Beverage A, on average. Both populations have normal distributions.
Problem 16. Are the standard deviations known or unknown?
Problem 16 Solution
Step 1 — Read what's given: each beverage's sugar content is reported with a standard deviation computed from its own sample, 0.6 grams for Beverage A and 0.8 grams for Beverage B; no population value is stated for either drink.
Answer: Unknown. Both standard deviations are sample values, used as estimates of the unknown population standard deviations, exactly as Definition 9.1.3 describes.
Problem 17. What is the random variable?
Problem 17 Solution
Step 1 — Name what's being compared: the study sets the two beverages' average sugar content against each other, which makes the random variable a difference of two sample means.
Answer: \(\overline{X}_{B} - \overline{X}_{A}\) — the difference between the mean amounts of sugar in the two soft drinks.
Problem 18. Is this a one-tailed or two-tailed test?
Problem 18 Solution
Step 1 — Read the direction of the claim: the researchers believe Beverage B has more sugar than Beverage A, on average — a claim that points one way, not "the two differ."
Answer: One-tailed. Written as \(\mu_{B} - \mu_{A}\), the alternative hypothesis is \(H_{a}: \mu_{B} > \mu_{A}\), a right-tailed test.
Use the following information to answer the next 12 exercises. The U.S. Center for Disease Control reports that the mean life expectancy was 47.6 years for White people born in 1900 and 33.0 years for non-White people. Suppose that you randomly survey death records for people born in 1900 in a certain county. Of the 124 White people, the mean life span was 45.3 years with a standard deviation of 12.7 years. Of the 82 non-White people, the mean life span was 34.1 years with a standard deviation of 15.6 years. Conduct a hypothesis test to see if the mean life spans in the county were the same for White and non-White people.
Problem 19. Is this a test of means or proportions?
Problem 19 Solution
Step 1 — Note what's being measured: life span is a numeric quantity recorded for each person, and the study compares average life spans between two groups rather than a yes/no outcome.
Answer: Means.
Problem 20. State the null and alternative hypotheses.
a) \(H_{0}\): \(\underline{\hspace{6em}}\)
b) \(H_{a}\): \(\underline{\hspace{6em}}\)
Problem 20 Solution
Step 1 — Set the equality in the null: the question asks whether life spans "were the same" for the two groups, which puts equality in \(H_{0}\) and leaves no stated direction for \(H_{a}\).
a) \(H_{0}: \mu_{W} = \mu_{NW}\)
b) \(H_{a}: \mu_{W} \neq \mu_{NW}\)
Answer: \(H_{0}: \mu_{W} = \mu_{NW} \qquad H_{a}: \mu_{W} \neq \mu_{NW}\), where \(W\) denotes White people and \(NW\) denotes non-White people born in 1900.
Problem 21. Is this a right-tailed, left-tailed, or two-tailed test?
Problem 21 Solution
Step 1 — Look for a stated direction: the prompt asks only whether the life spans "were the same," with no word like "more" or "less" pointing the alternative one way.
Answer: Two-tailed.
Problem 22. In symbols, what is the random variable of interest for this test?
Problem 22 Solution
Step 1 — Write the difference of sample means: the random variable of a two-sample test of means is always the difference \(\overline{X}_{1} - \overline{X}_{2}\), here applied to the two race groups.
Answer: \(\overline{X}_{W} - \overline{X}_{NW}\).
Problem 23. In words, define the random variable of interest for this test.
Problem 23 Solution
Step 1 — Translate the symbol into words: \(\overline{X}_{W} - \overline{X}_{NW}\) is the gap between the two sample averages being compared.
Answer: The random variable is the difference between the mean life spans of White and non-White people.
Problem 24. Which distribution (normal or Student's t) would you use for this hypothesis test?
Problem 24 Solution
Step 1 — Check what standard deviations are available: only the two sample standard deviations, 12.7 and 15.6 years, are given; no population standard deviation is stated for either group.
Answer: Student's t.
Problem 25. Explain why you chose the distribution you did for Exercise 9.1.24.
Problem 25 Solution
Step 1 — Match the setting to the definition: two independent samples, two population means, and standard deviations estimated from the samples rather than known in advance — that combination is exactly Definition 9.1.5's Aspin–Welch t-test.
Answer: This is a comparison of two population means with unknown population standard deviations.
Problem 26. Calculate the test statistic and p-value.
Problem 26 Solution
Step 1 — Standardize the difference: the standard error is
$$ \sqrt{\frac{12.7^{2}}{124} + \frac{15.6^{2}}{82}} = \sqrt{1.3007 + 2.9678} = \sqrt{4.2685} = 2.0660 $$Step 2 — Compute the test statistic: with \(\overline{x}_{W} - \overline{x}_{NW} = 45.3 - 34.1 = 11.2\),
$$ t = \frac{11.2}{2.0660} = 5.421 $$with \(df \approx 148.74\) from the Aspin–Welch formula in Definition 9.1.5.
Step 3 — Read off the p-value: for a two-tailed test with \(t = 5.421\) and \(df \approx 148.74\), the p-value is about \(2.34 \times 10^{-7}\) — well under 0.0001.
Answer: \(t \approx 5.42\), \(df \approx 148.74\), p-value \(\approx 2.34 \times 10^{-7}\) (\(p < 0.0001\)).
Problem 27. Sketch a graph of the situation. Label the horizontal axis. Mark the hypothesized difference and the sample difference. Shade the area corresponding to the p-value.
Problem 27 Solution
Step 1 — Describe the curve and its center: sketch a \(t\)-distribution centered at the hypothesized difference, 0, with the horizontal axis labeled as the difference in sample mean life spans (or the standardized \(t\) value).
Step 2 — Mark the observed value and shade: mark the sample difference, 11.2 years (\(t \approx 5.42\)), far out in the right tail, mark its mirror image at \(-11.2\) in the left tail, and shade both tails beyond those points — that shaded area is the p-value.
Answer: Answers may vary. A correct sketch shows a \(t\)-curve centered at 0, the observed difference marked well out in the right tail with its mirror point in the left tail, and both tails shaded to represent the two-tailed p-value.
Problem 28. Find the p-value.
Problem 28 Solution
Step 1 — Carry the test statistic forward: using \(t \approx 5.421\) with \(df \approx 148.74\) from Exercise 9.1.26, read the two-tailed area beyond \(|t| = 5.421\) off technology.
Answer: p-value \(\approx 2.34 \times 10^{-7}\) (\(p < 0.0001\)).
Problem 29. At a pre-conceived \(\alpha = 0.05\), what is your:
a) Decision:
b) Reason for the decision:
c) Conclusion (write out in a complete sentence):
Problem 29 Solution
Step 1 — Compare the p-value to \(\alpha\): the p-value, about \(2.34 \times 10^{-7}\), is far smaller than \(\alpha = 0.05\).
a) Decision: Reject the null hypothesis.
b) Reason for the decision: The p-value is less than \(\alpha = 0.05\).
c) Conclusion: At the 5% level of significance, the evidence supports the claim that life expectancy in the 1900s was different between White people and non-White people.
Answer: Reject \(H_{0}\) because the p-value is less than 0.05; at the 5% level of significance, the evidence supports the claim that life expectancy in the 1900s was different between White people and non-White people.
Problem 30. Does it appear that the means are the same? Why or why not?
Problem 30 Solution
Step 1 — Weigh the evidence: the test rejected \(H_{0}\) with a p-value near two parts in ten million, and the observed gap of 11.2 years is more than five standard errors wide — that is not a result small samples produce by chance.
Answer: No, the means do not appear to be the same. The hypothesis test found sufficient evidence that the mean life spans of White and non-White people in the county differed, consistent with the historical CDC figures of 47.6 years versus 33.0 years that opened this block.
Key Terms
independent samples — two samples in which the observations of one are drawn separately from those of the other, so that no value in one sample is linked to any value in the other.
difference of two sample means — written \(\overline{X}_{1} - \overline{X}_{2}\), the random variable of a two-sample test of means.
standard error of a difference — \(\sqrt{\frac{(s_{1})^{2}}{n_{1}} + \frac{(s_{2})^{2}}{n_{2}}}\), the estimated standard deviation of \(\overline{X}_{1} - \overline{X}_{2}\); the two contributions add rather than cancel.
Aspin–Welch t-test — the test comparing two independent population means when the population standard deviations are unknown and possibly unequal; its degrees of freedom need not be a whole number.
pooled variance — a single variance estimate formed by combining two sample variances; deliberately not used in the test of this section, though it does appear in Cohen's \(d\).
Cohen's \(d\) — the difference between two means divided by the pooled standard deviation, a measure of effect size compared against the standards 0.2 (small), 0.5 (medium), and 0.8 (large).
effect size — how large a difference is, expressed on a scale that does not depend on the sample sizes; a separate question from whether the difference is statistically significant.