10.8 Lab 2: Chi-Square Test of Independence

Step 1 — State the two questions. The first question: how many populations are being sampled? The second question: how many questions (variables) are being asked of each person?

Step 2 — Apply the first question to this lab. One population is being sampled — the class itself. Nobody drew separate samples with fixed sizes from different groups.

Step 3 — Apply the second question to this lab. Two questions are asked of each person: their favorite type of snack, and their gender.

Step 4 — Match against the three chi-square designs. One population and two categorical questions per subject is a test of independence; the data form a contingency table. It is not goodness-of-fit (that involves one variable), and it is not homogeneity, because the row totals were not fixed in advance — the numbers of men and women are facts about the class, not design choices.

Answer: The two questions are "how many populations are sampled?" and "how many variables are recorded per person?" This lab has one population and two variables, so it calls for a chi-square test of independence.

Step 1 — Set up the expected-count formula. Each expected count is \(E = \frac{(\text{row total})(\text{column total})}{\text{total surveyed}}\). Here the column total is 6 and the grand total is \(n = 28\).

Step 2 — Compute the expected count for the men row.

Step 3 — Compute the expected count for the women row.

Step 4 — Check against the condition. Both expected counts must be at least five. Since \(3.21 < 5\) and \(2.79 < 5\), neither cell satisfies the condition.

Answer: The expected counts are approximately 3.21 (men) and 2.79 (women). Both fall below five, so the condition fails and categories would need to be combined before testing.

Step 1 — Recall what the chi-square statistic assumes. The test compares each observed count to an expected count computed under the assumption that the factors are independent, and the sampling distribution of \(\chi^{2}\) that makes p-values meaningful was derived under that model.

Step 2 — Explain why observed counts cannot be constrained. Observed counts are data — they are whatever the sample produced, and the researcher has no control over them. A requirement placed on observed counts would be a requirement on chance outcomes.

Step 3 — Explain why expected counts are the right target. Expected counts are determined by the margins and the sample size, so they measure how much information the design actually supplies to each cell. When an expected count is small, the approximation behind the chi-square distribution breaks down there — large relative fluctuations are possible even under independence, inflating the statistic.

Step 4 — Note the practical consequence. Because expected counts depend only on the totals, the researcher can repair small expected counts by combining categories (changing the column structure) or gathering more data (raising every margin), whereas nothing can be done directly about an individual observed count.

Answer: The condition constrains expected counts because the validity of the chi-square approximation depends on what the cells would hold under independence, which is governed by the margins and sample size. Observed counts are random data beyond the researcher's control; expected counts reflect the design and can be repaired by merging categories or enlarging the sample.

Step 1 — Compute the degrees of freedom before merging. For a 2-by-4 table,

Step 2 — Determine the table size after merging. Combining two columns into one removes one column, so the merged table is 2-by-3.

Step 3 — Compute the degrees of freedom after merging.

Step 4 — Confirm the cost rule. Each merge costs exactly one degree of freedom, and indeed \(3 - 1 = 2\).

Answer: Before merging, \(df = 3\); after merging the two columns into one neighbor, the table is 2-by-3 and \(df = 2\).

Step 1 — Identify what is wrong with the student's statement. A test of independence asks whether the two factors are related at all. The alternative hypothesis must be the logical complement of the null: if \(H_0\) says the factors are independent, then \(H_a\) must say they are dependent. "Men prefer chips more than women do" asserts a specific direction and a specific category, which is a stronger claim than mere dependence.

Step 2 — Note the structural problem. The chi-square test of independence is inherently right-tailed and symmetric across all cells: it pools deviations from every cell into one statistic and cannot test a directional claim about one particular cell. A directional comparison would call for a different procedure (for example, a two-proportion z-test comparing chips preference between men and women).

Step 3 — Write a correct alternative hypothesis. \(H_a\): Favorite type of snack and gender are dependent.

Answer: The statement is invalid because it specifies a direction and a particular category, while the alternative in a test of independence must simply be the negation of independence. The correct alternative is \(H_a\): favorite type of snack and gender are dependent.

Step 1 — Compare the p-value to the significance level. With \(\alpha = 0.05\) and p-value \(= 0.043\),

Since the p-value is smaller than \(\alpha\), reject \(H_0\).

Step 2 — Write the conclusion in context. At a 5% level of significance, from the data collected, there is sufficient evidence to conclude that favorite type of snack and gender are dependent.

Step 3 — Sanity-check against the statistic. For \(df = 1\), the critical value at \(\alpha = 0.05\) is about 3.84, and \(\chi^{2} = 4.10 > 3.84\), which agrees with the decision to reject.

Answer: Reject \(H_0\). At the 5% level of significance, there is sufficient evidence to conclude that favorite type of snack and gender are dependent.

Step 1 — Name the first forbidden claim. You may not identify which snack category drives the dependence. The chi-square statistic pools all cells into a single number, so rejecting \(H_0\) establishes that some relationship exists, not that any particular cell (such as men preferring chips) is responsible. Comparing observed and expected counts can suggest where the relationship lies, but that is a follow-up question, not something this test proved.

Step 2 — Name the second forbidden claim. You may not claim causation. Dependence means the two characteristics vary together in this sample; it says nothing about one causing the other. There may be confounding factors, and neither variable was assigned experimentally.

Step 3 — Add the generalization caveat. The sample is a single class, not a random sample of students generally, so the conclusion describes this class rather than a larger population.

Answer: You may not claim which category drives the relationship (the test pools all cells into one statistic), and you may not claim that one factor causes the other — dependence is association, not causation. Additionally, because the sample is one class, the result should not be generalized to all students.

Step 1 — Identify the practice. Choosing among possible merges based on which one yields the largest test statistic is selecting the analysis after seeing the results — a form of data snooping.

Step 2 — Explain why it biases the test. The chi-square test's p-values are calibrated for a test whose hypotheses and category structure were fixed before the data were examined. Searching over merges and keeping the one that maximizes \(\chi^{2}\) cherry-picks the most extreme outcome out of many possibilities, so the reported p-value no longer reflects the true probability of such an extreme result under \(H_0\). The actual Type I error rate becomes much larger than the nominal \(\alpha\).

Step 3 — Contrast with the legitimate use of merging. Merging is justified only as a repair for the expected-count condition, decided by looking at the expected counts (which depend on the margins, not on how dependent the data look), and ideally specified before computing the statistic.

Answer: Choosing the merge that maximizes the test statistic is data snooping: it selects the most favorable analysis after seeing the data, inflating the Type I error rate and making the reported p-value invalid. Merges should be chosen to fix small expected counts, based on the margins alone, before the statistic is computed.

Step 1 — Reason 1: sampling variability. Each class is a small sample, and small samples are noisy. Even if the true relationship (or lack of one) is identical, ordinary chance variation can push one class's \(\chi^{2}\) above its critical value and another's below. A p-value of 0.04 versus 0.06 represents nearly the same evidence but opposite decisions.

Step 2 — Reason 2: different class compositions. The two classes may genuinely differ — in size, in gender balance, or in cultural or regional snack preferences. Different margins change both the expected counts and the degrees of freedom available after repairing the expected-count condition, so the classes may effectively be testing slightly different tables.

Step 3 — Note why neither is an error. Both computations can be perfectly correct; the disagreement reflects the uncertainty inherent in small samples and the fact that each class tests its own members, not a common population.

Answer: Opposite conclusions can arise from (1) ordinary sampling variability — small samples are noisy, and p-values near \(\alpha\) can land on either side by chance — and (2) genuine differences between the classes in size, composition, or preferences, which change the margins and even the post-merge table being tested. Neither requires a computational mistake.

Step 1 — Define what a random sample requires. A random sample of a larger population requires that every member of that population have a known, equal (or at least specifiable) chance of appearing in the sample. Members of a single class were selected into it by scheduling, major requirements, and registration choices — mechanisms unrelated to random selection.

Step 2 — Explain the bias this introduces. A class shares characteristics — age range, field of study, institution, often region — that may be associated with the variables being studied. So the class systematically differs from the broader population in ways that could distort the relationship between snack preference and gender.

Step 3 — Draw the wording consequence. Statistical conclusions apply to the population from which the sample was drawn. Since the sample is the class, the conclusion must be worded as a statement about this class ("in this class, ...") rather than about students in general. Extending it further would require a properly drawn random sample.

Answer: A single class is not a random sample because membership was determined by scheduling and enrollment rather than by random selection, and the class shares systematic characteristics that may relate to the variables studied. Therefore the conclusion should be worded as applying to this class specifically — e.g., "in this class, favorite snack type and gender appear dependent" — and not generalized to students at large without new, randomly sampled data.