1.4 Experimental Design and Ethics
Learning Objectives
- Identify the explanatory variable, response variable, treatments, and experimental units in a designed experiment.
- Explain how lurking variables can distort a study and how random assignment controls for them.
- Describe how placebos, blinding, and double-blind designs counter the power of suggestion.
- Recognize when a question cannot be answered by a randomized experiment and must be studied observationally.
- Distinguish a confounding variable from the explanatory and response variables it is tangled with.
- Apply the core ethical obligations of research on human subjects: informed consent, IRB oversight, and honest data.
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer better at growing roses than another? Is driving while exhausted as dangerous as driving drunk? We answer questions like these with randomized experiments. In this section you'll learn how to design a study well — because good design is the only thing that lets us trust the data that comes out the other end. Sloppy design means unreliable answers, no matter how fancy the math looks afterward.
1.4.1 The Anatomy of an Experiment
The whole point of an experiment is to figure out how one thing affects another. When we think one variable causes a change in a second variable, we give them names. The first variable — the one we believe is doing the causing — is the explanatory variable. The second variable — the one that reacts — is the response variable.
In a randomized experiment, the researcher deliberately changes the explanatory variable and then watches what happens to the response variable. The specific settings of the explanatory variable that we hand out are called treatments, and each individual object or person we measure is an experimental unit.
Here's the key idea before we put these terms to work: in a true experiment, we control the explanatory variable. We don't just watch and record what people happen to do — we assign the treatments ourselves. That control is what separates an experiment from simply observing the world, and it's what eventually lets us claim that one thing caused another. Keep that distinction in mind as you read the first worked example, where every one of these labels gets pinned to a real aspirin study.
For each scenario, name the explanatory variable, the response variable, and the treatments.
a. A gardener tests whether plants grow taller with regular water or with water plus liquid fertilizer.
b. A teacher checks whether students score higher on a quiz after a 10-minute review or after no review.
Solution
a. The explanatory variable is the watering method. The response variable is the height the plants reach. The treatments are plain water and water plus fertilizer.
b. The explanatory variable is whether a review happened. The response variable is the quiz score. The treatments are 10-minute review and no review.
Answer: In both cases, the explanatory variable is the thing the researcher controls, the response variable is what gets measured at the end, and the treatments are the specific options handed out.
1.4.2 Lurking Variables and Random Assignment
Suppose you want to know whether vitamin E prevents disease. You round up a group of people and ask each one whether they regularly take vitamin E. You notice that the vitamin-E takers are healthier, on average, than the people who skip it. Does that prove vitamin E works?
It does not. The two groups differ in way more than just vitamin E. People who take vitamin E regularly often do all sorts of other healthy things too — they exercise, eat better, take other supplements, and avoid smoking. Any one of those habits could be the real reason they're healthier. So this study can't pin the credit on vitamin E.
These extra, sneaky variables that muddy a study are called lurking variables. To actually prove that the explanatory variable is causing the change in the response variable, we have to isolate it — we need the groups we're comparing to differ in exactly one way: the treatment we chose to give them.
How do we get that clean, single difference? By random assignment — we flip a coin (figuratively) to decide which experimental unit goes into which treatment group. When assignment is random, all the lurking variables get spread out roughly evenly across the groups. The exercisers, the smokers, the healthy eaters — they end up scattered on both sides. Once that happens, the only systematic difference left between the groups is the treatment the researcher imposed. So if the response variable comes out different, that difference must be caused by the treatment. That's how an experiment can prove cause and effect.
A study compares a new tutoring app to no tutoring. Researchers let students choose whether to use the app, then compare test scores. The app users score higher.
a. Why can't we conclude the app caused the higher scores?
b. What single change would fix the design?
Solution
a. Students who chose to use the app are probably more motivated, have more study time, or care more about grades to begin with. Those are lurking variables — any of them could explain the higher scores instead of the app.
b. Randomly assign students to use the app or not, rather than letting them choose. Random assignment spreads motivation and study habits evenly across both groups, leaving the app as the only systematic difference.
Answer: Self-selection lets lurking variables pile up in one group; random assignment is the fix.
1.4.3 The Power of Suggestion: Placebos and Blinding
The mere expectation of a result can change the result. Studies show that what a participant believes is happening can matter as much as the actual medicine. In one study of performance-enhancing drugs, researchers reported:
Results showed that believing one had taken the substance resulted in performance times almost as fast as those associated with consuming the drug itself. In contrast, taking the drug without knowledge yielded no significant performance increment.
In other words, thinking you took the drug sped people up almost as much as the real drug — and taking the real drug secretly did almost nothing. The belief was doing the heavy lifting.
This matters far beyond sports. Any medical trial — for painkillers, antidepressants, anything a person can feel — has to wrestle with the fact that hope and expectation produce real physical effects. If we don't account for that, we'll credit the pill for improvements the patient's own mind produced.
When taking part in a study triggers a physical response on its own, it gets hard to isolate the treatment's true effect. To handle this, researchers set aside one control group and give it a placebo — a fake treatment (like a sugar pill) that looks and feels real but can't actually affect the response variable. The control group lets researchers separate the effect of being in an experiment from the effect of the real treatment.
Of course, if you know you're getting the dummy pill, the power of suggestion disappears — you won't expect anything. That's why we use blinding (also called masking): a person who is blinded doesn't know whether they're getting the real treatment or the placebo. A double-blind experiment goes one step further — both the subjects and the researchers working with them are kept in the dark, so neither side's expectations can color the results.
A company tests a new energy drink against a placebo drink that looks and tastes identical. Researchers measure how long each person can run on a treadmill.
a. Why use a placebo drink instead of just comparing energy-drink users to people who drink nothing?
b. Describe how this study could be made double-blind.
Solution
a. People who know they drank an energy drink may push harder simply because they expect a boost. A look-alike, taste-alike placebo keeps the expectation the same in both groups, so any real difference comes from the drink's ingredients, not the belief.
b. Make sure the participants don't know which drink they got (subjects blinded), and make sure the staff timing the treadmill runs don't know either (researchers blinded). With both sides in the dark, the experiment is double-blind.
Answer: The placebo equalizes expectations; double-blinding hides the assignment from both subjects and the staff measuring them.
1.4.4 Putting the Pieces Together
Now let's apply every term — population, sample, experimental units, explanatory and response variables, and treatments — to real studies.
A study needs to be conducted of the effect of three medicines A, B, and C on the height of adults aged 30 to 45. 90 adults were selected randomly and divided into three equal groups. The first group was asked to take medicine A for 6 months. The second group was asked to take medicine B for 6 months. The third group was asked to take medicine C for 6 months. The average change in height in each group is calculated at the end of the study.
Identify the following values for this study: population, sample, experimental units, explanatory variable, response variable, treatments.
Solution
Population: adults aged 30 to 45.
Sample: the 90 adults selected for the study.
Experimental units: the individual adults.
Explanatory variable: the medicine taken.
Treatments: medicine A, medicine B, and medicine C.
Response variable: the change in height over the 6 months.
Answer: Three treatments, three randomly assigned groups, one measured response — change in height.
Researchers want to investigate whether taking aspirin regularly reduces the risk of heart attack. Four hundred people between the ages of 50 and 84 are recruited as participants. The people are divided randomly into two groups: one group will take aspirin, and the other group will take a placebo. Each person takes one pill each day for three years, but they don't know whether they are taking aspirin or the placebo. At the end of the study, researchers count the number of people in each group who have had heart attacks.
Identify the following values for this study: population, sample, experimental units, explanatory variable, response variable, treatments.
Figure 1.4.1 — A randomized, placebo-controlled aspirin trial: 400 recruited subjects are split at random into an aspirin group and a placebo group, then followed for three years.
Solution
Population: people aged 50 to 84.
Sample: the 400 people who participated.
Experimental units: the individual people in the study.
Explanatory variable: the oral medication (the pill type each person takes).
Treatments: aspirin and a placebo.
Response variable: whether a subject had a heart attack.
Answer: Because subjects were randomly assigned and didn't know which pill they got, the only systematic difference between groups is the treatment — so a difference in heart-attack rates can be credited to the aspirin.
The Placebo Research Group conducted a study to find the extent of placebo effects. A group of men randomly selected were asked to take a test before and after taking a pill that induces a mild headache. The pill in half of the randomly selected men was replaced with a similar pill that has no effect. For each trial, researchers recorded the change in time men took to complete the tests before and after taking the pill.
a. Describe the explanatory and response variable in this study.
b. What are the treatments?
c. Identify any lurking variables that could interfere with this study.
d. Is it possible to use blinding in this study?
Solution
a. The explanatory variable is the type of pill taken. The response variable is the change in time it takes to complete the test (before vs. after the pill).
b. The two treatments are the headache-inducing pill and the look-alike pill with no effect (the placebo).
c. Because the men were randomly assigned to the two pills, lurking variables (such as differing test-taking ability or stress levels) are spread evenly across the groups, so random assignment controls for them.
d. Yes. The pills look similar, so subjects can be blinded, and the researchers recording the times can be blinded too — making a double-blind design possible.
Answer: Pill type → change in completion time, two treatments, random assignment handles lurking variables, and blinding is possible for both sides.
The Smell & Taste Treatment and Research Foundation conducted a study to investigate whether smell can affect learning. Subjects completed mazes multiple times while wearing masks. They completed the pencil and paper mazes three times wearing floral-scented masks, and three times with unscented masks. Participants were assigned at random to wear the floral mask during the first three trials or during the last three trials. For each trial, researchers recorded the time it took to complete the maze and the subject's impression of the mask's scent: positive, negative, or neutral.
a. Describe the explanatory and response variables in this study.
b. What are the treatments?
c. Identify any lurking variables that could interfere with this study.
d. Is it possible to use blinding in this study?
Solution
a. The explanatory variable is scent, and the response variable is the time it takes to complete the maze.
b. There are two treatments: a floral-scented mask and an unscented mask.
c. All subjects experienced both treatments. The order of the treatments was randomly assigned, so there were no systematic differences between the treatment groups. Random assignment eliminates the problem of lurking variables.
d. Subjects will clearly know whether they can smell flowers or not, so the subjects cannot be blinded in this study. The researchers timing the mazes can be blinded, though — the person observing a subject won't know which mask is being worn.
Answer: Scent → maze time, two treatments, lurking variables handled by randomizing the order, and only the observers (not the subjects) can be blinded.
1.4.5 When You Can't Randomize
You are concerned about the effects of texting on driving performance. Design a study to test the response time of drivers while texting and while driving only. How many seconds does it take for a driver to respond when a leading car hits the brakes?
a. Describe the explanatory and response variables in the study.
b. What are the treatments?
c. What should you consider when selecting participants?
d. Your research partner wants to divide participants randomly into two groups: one to drive without distraction and one to text and drive simultaneously. Is this a good idea? Why or why not?
e. Identify any lurking variables that could interfere with this study.
f. How can blinding be used in this study?
Solution
a. The explanatory variable is whether the driver is texting. The response variable is the driver's reaction time (in seconds) when the lead car brakes.
b. The two treatments are driving while texting and driving without texting.
c. Choose participants who represent the drivers you care about — a mix of ages, experience levels, and driving habits — and run the test safely (ideally in a driving simulator, never in real traffic).
d. Randomly splitting participants into a texting group and a no-texting group is a good idea: random assignment spreads lurking variables (like natural reaction speed or driving experience) evenly across the groups. The major concern is safety — texting while driving is dangerous, so this should be done in a simulator.
e. Lurking variables include each driver's baseline reaction speed, experience, age, eyesight, and how comfortable they are with their phone. Random assignment helps balance these.
f. True blinding is hard here because a driver obviously knows whether they're texting. However, the people scoring the reaction times can be blinded to which condition each trial was, removing their expectations from the measurement.
Answer: Texting → reaction time, two treatments, random assignment is appropriate (with safety via a simulator), and only the scorers — not the drivers — can be blinded.
A researcher wants to study the effects of birth order on personality. Explain why this study could not be conducted as a randomized experiment. What is the main problem in a study that cannot be designed as a randomized experiment?
Solution
Step 1 — Identify the explanatory variable: The explanatory variable is birth order (firstborn, middle child, youngest, etc.).
Step 2 — Ask whether it can be assigned: You cannot randomly assign a person's birth order — it's fixed the moment they're born. There's no way to hand out "be a firstborn" as a treatment.
Step 3 — Name the consequence: Random assignment is what evens out lurking variables. When you can't assign subjects to groups at random, the groups will differ in ways beyond birth order (family size, parents' age, income, and so on), and those differences can masquerade as a birth-order effect.
Answer: The study can't be randomized because birth order can't be assigned. The main problem with any non-randomizable study is that lurking variables aren't controlled, so you can't prove cause and effect.
1.4.6 Ethics in Statistics
The widespread misuse and misrepresentation of statistics often gives the field a bad name. Some say that "numbers don't lie," but the people who use numbers to back up their claims sometimes do.
Take the case of famous social psychologist Diederik Stapel, a former professor at Tilburg University in the Netherlands. An extensive investigation across three universities where he had worked concluded that he was guilty of fraud on a colossal scale. Falsified data tainted more than 55 papers he authored and 10 Ph.D. dissertations that he supervised — and his articles were retracted from some of the world's top journals.
Stapel didn't deny that ambition drove his deceit, but he said it was more complicated than that. He insisted he loved social psychology yet was frustrated by the messiness of real experimental data, which rarely led to clean conclusions:
"It was a quest for aesthetics, for beauty — instead of the truth," he said. He described his behavior as an addiction that drove him to carry out acts of increasingly daring fraud, like a junkie seeking a bigger and better high.
The committee found that Stapel was guilty of several practices, including:
- creating datasets that conveniently confirmed his prior expectations,
- altering data in existing datasets,
- changing measuring instruments without reporting the change, and
- misrepresenting the number of experimental subjects.
Clearly, faking data the way Stapel did is never acceptable. But violations of ethics aren't always this obvious.
Stapel's fraud went undetected for years partly because his co-authors didn't know enough statistics to spot the red flags. That's the real lesson here: learning statistics isn't just about passing a class — it's the skill that lets you catch fraud, protect your own work, and avoid being fooled.
Researchers have a responsibility to verify that proper methods are being followed. The report on the Stapel investigation noted that "statistical flaws frequently revealed a lack of familiarity with elementary statistics." Many of his co-authors should have caught the irregularities — but they didn't understand the analysis well enough, and they simply trusted that he was collecting and reporting data honestly.
Many kinds of statistical fraud are hard to spot. Some researchers stop collecting data the moment they have just enough to "prove" what they were hoping to prove — they don't want to risk that a larger study would produce data contradicting their hypothesis.
Professional organizations like the American Statistical Association lay out clear expectations for researchers, and there are even federal laws governing the use of research data. When a statistical study involves human participants — as in medical studies — both ethics and the law require researchers to protect the safety of their subjects. The U.S. Department of Health and Human Services oversees federal regulations of research studies with the goal of protecting participants. Any university or research institution that runs a study must guarantee the safety of all human subjects. For this reason, institutions set up oversight committees called Institutional Review Boards (IRBs), and every planned study must be approved in advance by the IRB. Key legal protections include:
- Minimized risk: Risks to participants must be minimized and reasonable relative to the expected benefits.
- Informed consent: Participants must be told the risks clearly, must agree in writing, and researchers must keep documentation of that consent.
- Privacy: Data collected from individuals must be carefully guarded to protect their privacy.
These ideas sound basic, but they're tricky to pin down in practice. Is deleting a participant's name from the record really enough to protect privacy — or could the person still be identified from what's left? What if the study goes off the rails and unexpected risks appear? When is informed consent truly required? Suppose your doctor draws blood to check your cholesterol. Once it's tested, you assume the lab disposes of the leftover blood — at that point it's biological waste. Does a researcher have the right to grab it for a study?
These are exactly the questions students of statistics should sit with. How common is fraud in statistical studies? You might be surprised — and disappointed. There's even a website (retractionwatch.com) devoted to cataloging retractions of studies proven fraudulent, and a quick look shows the misuse of statistics is a bigger problem than most people realize. Vigilance against fraud requires knowledge: learning the basic theory of statistics empowers you to analyze studies critically.
Describe the unethical behavior, if any, in each example and explain how it could affect the reliability of the resulting data. Then explain how the problem should be corrected.
A study is commissioned to determine the favorite brand of fruit juice among teens in California.
a. The survey is commissioned by the seller of a popular brand of apple juice.
b. There are only two types of juice included in the study: apple juice and cranberry juice.
c. Researchers allow participants to see the brand of juice as samples are poured for a taste test.
d. Twenty-five percent of participants prefer Brand X, 33% prefer Brand Y, and 42% have no preference between the two brands. Brand X references the study in a commercial saying "Most teens like Brand X as much as or more than Brand Y."
Solution
a. A study paid for by a company that sells one of the products has a built-in conflict of interest — the sponsor has a stake in the outcome. The fix: disclose the funding source, and ideally use an independent party to run the study.
b. Limiting the choices to only two juices doesn't represent the full range of options teens actually like, biasing the result toward those two. The fix: include a representative variety of juice brands.
c. Letting participants see the brand lets brand loyalty (the power of suggestion) sway the taste test. The fix: blind the taste test so participants don't know which brand they're tasting.
d. The claim is technically true but deeply misleading: combining the 25% who prefer Brand X with the 42% who have no preference to imply Brand X is the favorite spins the data. The fix: report the actual percentages plainly and don't merge categories to manufacture a flattering headline.
Answer: Conflict of interest, too-narrow choices, an unblinded taste test, and misleading category-merging are all ethical problems — each fixed by transparency, representative options, blinding, and honest reporting.
Describe the unethical behavior in each example and explain how it could affect the reliability of the resulting data. Then explain how the problem should be corrected.
A researcher is collecting data in a community.
a. The researcher selects a block where they are comfortable walking because they know many of the people living on the street.
b. No one seems to be home at four houses on the route. They do not record the addresses and do not return at a later time to try to find residents at home.
c. The researcher skips four houses on the route because they are running late for an appointment. When they get home, they fill in the forms by selecting random answers from other residents in the neighborhood.
Solution
a. By choosing a convenient block they already know, the researcher is intentionally picking a sample that could be biased, and claiming it represents the whole community is misleading. The fix: select areas in the community at random.
b. Intentionally leaving out the people who weren't home creates bias. If the study is about jobs and child care, for instance, ignoring people who are away may systematically miss working families — exactly the data the study needs. The fix: make every effort to reach all members of the target sample, including returning later.
c. It is never acceptable to fake data. Even though the made-up answers are "real" responses borrowed from other participants, duplicating them is fraud and biases the data. The fix: do the work — interview everyone on the route.
Answer: (a) convenience sampling → randomize the areas; (b) ignoring non-responders → return and reach everyone; (c) fabricating answers → never fake data, interview the actual route.
1.4.7 Explanatory, Response, and Confounding Variables
When we suspect one variable might causally affect another, we label the first variable the explanatory variable and the second the response variable.
For many pairs of variables, there's no hypothesized relationship at all — in those cases neither label applies. And keep in mind: simply labeling the variables this way does nothing to guarantee that a causal relationship actually exists. A formal check of whether one variable causes a change in another requires an experiment.
A confounding variable is a variable that is associated with both the explanatory and the response variables. Because of its association with both, we cannot tell whether the response is due to the explanatory variable or due to the confounding variable.
Here's a concrete one. Suppose we look at sunscreen use and skin cancer. Sun exposure is a confounding factor because it's tied to both sunscreen use and skin cancer: people who spend all day in the sun are more likely to wear sunscreen, and people who spend all day in the sun are more likely to develop skin cancer. Research tells us the skin cancer actually comes from the sun exposure — but the variables sunscreen use and sun exposure are confounded, and without that research we'd have no way of knowing which one was the true cause.
Let's nail down the vocabulary with formal definitions, then meet one more troublemaker: the confounding variable.
Naming a variable "explanatory" is like accusing it of a crime — it's only a suspect, not a convicted culprit. The label says "we think this one might be responsible," but you still need the trial (an experiment) to prove it.
A study finds that towns with more ice cream sales also have more drownings. Someone concludes that eating ice cream causes drowning.
a. Identify a confounding variable that explains the link.
b. Explain why this confounding variable makes the "ice cream causes drowning" claim unjustified.
Solution
a. Hot weather (or summer) is the confounding variable. It's associated with both: hot days drive up ice cream sales and send more people swimming, which raises drownings.
b. Because hot weather is linked to both ice cream sales and drownings, we can't tell whether the rise in drownings comes from ice cream or from the heat. The two explanations are tangled together (confounded), so blaming ice cream isn't justified — the real driver is the weather pushing both numbers up.
Answer: Hot weather is the confounder; it's tied to both variables, so the apparent ice-cream-causes-drowning link is spurious.
1.4.8 Guided Practice
Suppose an observational study tracked sunscreen use and skin cancer, and it was found that people who use sunscreen are more likely to get skin cancer than people who do not use sunscreen. Does this mean sunscreen causes skin cancer?
Solution
No. Earlier research actually tells us that sunscreen reduces skin cancer risk, so something else must explain this odd association. The missing piece is sun exposure, a confounding variable (also called a lurking variable or confounder). People who spend lots of time in the sun are more likely to both use sunscreen and develop skin cancer. The sun exposure drives both, creating a misleading link between sunscreen and skin cancer. Because this is an observational study with no random assignment, we can't isolate sunscreen as a cause — so the data does not show that sunscreen causes skin cancer.
Look back to the study in Section 1.1 where researchers were testing whether stents were effective at reducing strokes in at-risk patients. Is this an experiment? Was the study blinded? Was it double-blinded?
Solution
Yes, it's an experiment: researchers randomly assigned patients to either receive a stent or not, deliberately controlling the explanatory variable (stent vs. no stent) rather than just observing what patients chose. Because the treatment is a surgical procedure, patients generally knew whether they received a stent, so the study was not blinded for the patients — and since both the patients and the treating physicians were aware of the treatment, it was not double-blinded either. (Blinding is straightforward with pills but much harder with surgery, where the procedure itself reveals the treatment.)
Problem Set 1.4
Problem 1. In your own words, explain the difference between the explanatory variable and the response variable in an experiment.
Solution
Step 1 — Identify the role of each variable: The explanatory variable is the factor the experimenter deliberately manipulates or sets, because it is the suspected cause whose effect we want to measure.
Step 2 — Contrast with the response: The response variable is the outcome we measure on each subject, because it is the effect we expect to change when the explanatory variable changes.
Step 3 — State the relationship: In an experiment we change the explanatory variable and watch what happens to the response variable, looking for evidence that the first influences the second.
Answer: The explanatory variable is the input the researcher controls or assigns (the suspected cause); the response variable is the outcome that is measured (the suspected effect). The experiment is designed to see whether changing the explanatory variable produces a change in the response variable.
Problem 2. A study compares two fertilizers on tomato yield. Name the explanatory variable, the response variable, and the treatments.
Solution
Step 1 — Find what is manipulated: The researcher decides which fertilizer each plant receives, so the explanatory variable is the type of fertilizer (Fertilizer A vs. Fertilizer B).
Step 2 — Find what is measured: Yield is the outcome recorded for each plant, so the response variable is the tomato yield (e.g., weight or number of tomatoes per plant).
Step 3 — List the treatments: The treatments are the specific levels of the explanatory variable applied to the experimental units: Fertilizer A and Fertilizer B.
Answer: Explanatory variable: type of fertilizer; response variable: tomato yield per plant; treatments: Fertilizer A and Fertilizer B.
Problem 3. Explain what a lurking variable is and why random assignment helps control for lurking variables.
Solution
Step 1 — Define a lurking variable: A lurking variable is a variable that is not among the explanatory or response variables being studied but that influences the response (and possibly the apparent relationship), because it can create or hide an association we wrongly attribute to the explanatory variable.
Step 2 — Explain the threat: If a lurking variable is distributed unevenly across the treatment groups, differences in the response might be caused by it rather than by the treatment, because the groups differ in more than just the treatment.
Step 3 — Explain random assignment: Randomly assigning subjects to treatment groups tends to spread lurking variables evenly across all groups, because each subject has an equal chance of landing in any group regardless of its hidden characteristics.
Answer: A lurking variable is an outside variable that affects the response but is not part of the study. Random assignment helps because, on average, it balances lurking variables across the treatment groups, so the groups are comparable and any difference in the response can be attributed to the treatment rather than to a confounding background variable.
Problem 4. What is a placebo, and why do researchers include a control group that receives one?
Solution
Step 1 — Define a placebo: A placebo is a fake or inactive treatment (such as a sugar pill or saline injection) that is indistinguishable from the real treatment, given so that subjects cannot tell whether they received the active treatment.
Step 2 — Explain the placebo effect: People often respond simply to the act of being treated, a phenomenon called the placebo effect, so improvement can occur even with an inactive treatment.
Step 3 — Justify the control group: A control group that receives the placebo experiences the same psychological and procedural conditions as the treatment group, because then the only systematic difference between groups is the active ingredient.
Answer: A placebo is an inactive, look-alike treatment. Researchers give a control group a placebo so that both groups experience the placebo effect equally; this isolates the effect of the active treatment, letting them attribute any extra response in the treatment group to the treatment itself rather than to the mere act of being treated.
Problem 5. Explain the difference between a blinded experiment and a double-blind experiment.
Solution
Step 1 — Describe a blinded experiment: In a single-blind experiment, the subjects do not know which treatment (active or placebo) they are receiving, because knowing could bias their behavior or self-reported response.
Step 2 — Describe a double-blind experiment: In a double-blind experiment, neither the subjects nor the people who interact with them and measure the response know who received which treatment, because that knowledge could bias both the subjects' responses and the researchers' assessments.
Step 3 — State the difference: The two designs differ in how many parties are kept unaware: one party (subjects only) versus two parties (subjects and the researchers/evaluators).
Answer: In a blinded (single-blind) experiment only the subjects are unaware of their treatment assignment. In a double-blind experiment both the subjects and the researchers who administer treatments or measure outcomes are unaware. Double-blinding adds protection against bias introduced by the experimenters, not just the subjects.
Problem 6. A researcher wants to study whether being born in winter versus summer affects adult height. Explain why this cannot be run as a randomized experiment.
Solution
Step 1 — Recall the requirement of a randomized experiment: A randomized experiment requires that the experimenter assign the explanatory variable to subjects at random, because random assignment is what allows a causal conclusion.
Step 2 — Examine the explanatory variable here: The explanatory variable is season of birth (winter vs. summer). A person's birth season is fixed at birth and cannot be assigned by a researcher, because we cannot choose or change when someone is born.
Step 3 — Classify the study: Since the explanatory variable can only be observed, not manipulated or randomly assigned, this must be an observational study rather than an experiment.
Answer: Season of birth cannot be randomly assigned to subjects — it is determined by nature and fixed at birth. Because the researcher cannot manipulate or randomize the explanatory variable, the study can only be observational, so it cannot establish causation and is not a randomized experiment.
Problem 7. Give an example (different from the ones in this section) of two variables that are linked through a confounding variable, and identify the confounder.
Solution
Step 1 — Recall the structure of confounding: Two variables are confounded when a third variable is associated with both, because that third variable can produce an apparent link between them even if neither causes the other.
Step 2 — Construct an example: Consider the observation that towns with more firefighters per fire tend to have more fire damage. It looks as if firefighters cause damage.
Step 3 — Identify the confounder: The confounding variable is the size or severity of the fire: larger fires both attract more firefighters and cause more damage, creating the misleading association between number of firefighters and amount of damage.
Answer: Example: ice cream sales and drowning deaths are positively associated, but neither causes the other — the confounding variable is hot weather (summer), which independently increases both ice cream sales and the number of people swimming. (Equivalently: firefighters present and fire damage, confounded by fire size.) The key point is that the confounder is linked to both variables and explains away the apparent direct relationship.
Problem 8. A vitamin company funds a study showing its vitamin improves health, but lets participants choose whether to take the vitamin. List two separate problems with this design.
Solution
Step 1 — Spot the self-selection problem: Participants choose whether to take the vitamin, so this is not random assignment. People who opt in may differ systematically (e.g., be more health-conscious) from those who opt out, which confounds the comparison — any health difference could be due to those background traits rather than the vitamin.
Step 2 — Spot the conflict-of-interest problem: The study is funded by the vitamin company, which has a financial stake in a positive result. This creates a conflict of interest that can bias the design, analysis, or reporting toward favorable findings.
Step 3 — Confirm the two are distinct: The first is a design flaw (lack of randomization / self-selection bias); the second is a source-of-funding/bias issue. They are separate weaknesses.
Answer: Two separate problems: (1) Self-selection — because participants choose whether to take the vitamin, the groups are not randomized and likely differ in lurking variables such as overall health-consciousness, so the vitamin's effect is confounded. (2) Conflict of interest — the funding company benefits from a positive result, which can bias how the study is designed, analyzed, or reported.
Problem 9. Describe the three key legal protections that an Institutional Review Board (IRB) is responsible for ensuring in studies with human participants.
Solution
Step 1 — Recall the role of an IRB: An Institutional Review Board reviews research involving human participants to ensure it is conducted ethically and legally before it begins.
Step 2 — List the three protections: The IRB ensures (a) the study's potential gains in knowledge are worth the risks to participants — a favorable risk/benefit balance; (b) participants give informed consent — they are told the risks and benefits and agree voluntarily to take part; and (c) participants' identities and data are kept confidential.
Step 3 — Explain why each matters: Each protection guards a participant's legal and ethical rights: avoiding undue harm, preserving autonomy through consent, and protecting privacy.
Answer: An IRB is responsible for ensuring that (1) the knowledge gained justifies the risk to subjects (risk/benefit assessment), (2) participants give informed consent after being told the study's risks and benefits, and (3) participants' personal information and data remain confidential.
Problem 10. Explain why "stopping data collection as soon as you have enough to prove your point" is an ethical problem, even when no data is faked.
Solution
Step 1 — Describe the practice: "Stopping as soon as the data prove your point" means watching the results accumulate and halting collection at the moment they happen to favor the desired conclusion.
Step 2 — Explain why it distorts the evidence: Random data naturally wander, so by chance the results will at some point show an apparent effect even when none exists. Choosing to stop exactly at that favorable moment cherry-picks a misleading snapshot, inflating the chance of a false positive far above the stated significance level.
Step 3 — Note that no fabrication is required: Every recorded value can be genuine, yet the selection rule for when to stop biases the analysis, because the decision to stop depends on the data turning out the way the researcher wants.
Answer: It is unethical because the stopping decision is based on the results themselves: random fluctuations guarantee that the data will eventually look favorable by chance, and stopping at that point systematically overstates the evidence and produces misleading, irreproducible conclusions. The dishonesty lies in the biased data-collection rule, not in faking any individual measurement.
Problem 11. Design an experiment. Identify the explanatory and response variables. Describe the population being studied and the experimental units. Explain the treatments that will be used and how they will be assigned to the experimental units. Describe how blinding and placebos may be used to counter the power of suggestion.
Solution
Step 1 — Choose a question and define the variables: (Open-ended — answers will vary. A model response follows.) Suppose we test whether a new fertilizer increases tomato yield. The explanatory variable is the fertilizer (new vs. standard); the response variable is the tomato yield (kg per plant).
Step 2 — Describe the population and experimental units: The population is all tomato plants of the chosen variety grown under similar conditions; the experimental units are the individual tomato plants used in the study.
Step 3 — Define and assign treatments: Two treatments: the new fertilizer and the standard fertilizer (control). Plants are randomly assigned to the two groups so other factors are balanced.
Step 4 — Blinding and placebos: A "placebo" fertilizer identical in appearance ensures the researchers measuring yield do not know which plants received the new product (single/double blinding), preventing the power of suggestion from biasing measurement.
Answer: Answers will vary. A complete response names an explanatory and a response variable, identifies the population and the experimental units, specifies the treatments and a random method for assigning them, and explains how blinding and a placebo guard against the power of suggestion. (Model: fertilizer type as explanatory, yield as response, tomato plants as units, random assignment to new vs. placebo fertilizer with blinded measurement.)
Problem 12. Discuss potential violations of the rule requiring informed consent.
a) People in a correctional facility are offered good behavior credit in return for participation in a study.
b) A research study is designed to investigate a new children's allergy medication.
c) Participants in a study are told that the new medication being tested is highly promising, but they are not told that only a small portion of participants will receive the new medication. Others will receive placebo treatments and traditional treatments.
Solution
Step 1 — Recall the informed-consent requirement: Participants must voluntarily agree to take part with full, undistorted knowledge of the study's nature and risks, free of coercion.
Step 2 — Evaluate each scenario:
a) Offering inmates good-behavior credit for participating is coercive — the incentive pressures a vulnerable population, undermining truly voluntary consent.
b) Studying a children's medication raises the consent-capacity issue: children cannot legally give informed consent, so consent must be obtained from a parent or legal guardian (with the child's assent where possible).
c) Telling participants the drug is "highly promising" while hiding that many will receive placebo or traditional treatment withholds material information and is misleading, so consent is not fully informed.
Answer: a) Coercion — good-behavior credit pressures a captive population, so consent is not voluntary. b) Children cannot give informed consent themselves; parental/guardian consent (and child assent) is required. c) Consent is not truly informed because participants are misled about the drug's promise and not told they may receive a placebo or traditional treatment.
Problem 13. How does sleep deprivation affect your ability to drive? A recent study measured the effects on 19 professional drivers. Each driver participated in two experimental sessions: one after normal sleep and one after 27 hours of total sleep deprivation. The treatments were assigned in random order. In each session, performance was measured on a variety of tasks including a driving simulation. Use key terms from this module to describe the design of this experiment.
Solution
Step 1 — Identify the experimental units and treatments: The 19 professional drivers are the experimental units (subjects). Each is observed under two conditions — normal sleep and 27 hours of total sleep deprivation — which are the two treatments (levels of the explanatory variable, sleep condition).
Step 2 — Identify the design type: Because every driver experiences both treatments, each subject serves as their own control: this is a matched-pairs (repeated-measures) design, a special case of a randomized block design where each subject is a block.
Step 3 — Identify the randomization and response: The order of the two treatments is assigned in random order, which controls for order/learning effects. The response variables are the performance measures, including the driving-simulation results.
Answer: This is a matched-pairs (repeated-measures) experiment: the 19 drivers are the subjects; the two treatments are normal sleep vs. 27 hours of sleep deprivation; the treatment order is randomized to control order effects; each driver acts as their own control; and the response variables are the task/driving-simulation performance measures.
Problem 14. An advertisement for Acme Investments displays the two graphs in Figure 1.4.2 to show the value of Acme's product in comparison with the Other Guy's product. Describe the potentially misleading visual effect of these comparison graphs. How can this be corrected?
Figure 1.4.2 — Acme Investments advertisement: two side-by-side comparison graphs of Acme vs. the Other Guys.
Solution
Step 1 — Describe the misleading effect: Comparison graphs become misleading when the vertical (value) axis does not start at zero or uses an inconsistent/compressed scale. Truncating or stretching the axis exaggerates small differences, making Acme's product appear to outperform the "Other Guy's" by far more than the actual figures justify.
Step 2 — Explain the correction: Redraw both graphs on the same vertical scale starting at zero, with equal axis intervals and identical units. With a common, zero-based axis the true (much smaller) difference between the two products is shown honestly.
Answer: The graphs mislead by using a non-zero or unequal/compressed vertical scale that magnifies the apparent gap between the products. To correct it, plot both on the same scale beginning at zero with consistent intervals, so the real difference is shown accurately.
Problem 15. The graph in Figure 1.4.3 shows the number of complaints for six different airlines as reported to the US Department of Transportation in February 2013. Alaska, Pinnacle, and Airtran Airlines have far fewer complaints reported than American, Delta, and United. Can we conclude that American, Delta, and United are the worst airline carriers since they have the most complaints?
Figure 1.4.3 — Number of complaints for six airlines reported to the US DOT, February 2013.
Solution
Step 1 — Spot the missing denominator: Complaint counts alone cannot rank airlines, because larger airlines serve far more passengers and flights and so naturally accumulate more complaints. The graph reports raw totals, not a rate.
Step 2 — Identify the fair comparison: To compare fairly we need a rate — complaints per passenger (or per flight, or per 100,000 enplanements). American, Delta, and United are large carriers, so their higher raw counts may simply reflect their size.
Answer: No. The graph shows raw complaint counts, which favor large carriers. Without adjusting for the number of passengers or flights (a complaints-per-passenger rate), we cannot conclude American, Delta, and United are the worst airlines.
Key Terms
explanatory variable — the variable a researcher suspects is causing a change; its values are deliberately set in an experiment.
response variable — the variable that is measured to see how it reacts to the explanatory variable.
treatment — a specific value or setting of the explanatory variable that is assigned to an experimental unit.
experimental unit — a single object or individual measured in a study.
lurking variable — an outside variable that can cloud a study by offering an alternative explanation for the results.
random assignment — assigning experimental units to treatment groups by chance, which spreads lurking variables evenly across the groups.
control group — a group given a placebo so researchers can separate the treatment's effect from the effect of simply being in the study.
placebo — a fake treatment that looks real but cannot actually affect the response variable.
blinding (masking) — keeping a person in a study from knowing whether they received the real treatment or the placebo.
double-blind experiment — a study in which both the subjects and the researchers working with them are blinded.
confounding variable — a variable associated with both the explanatory and response variables, making it impossible to tell which one caused the response.
Institutional Review Board (IRB) — an oversight committee that must approve studies in advance to protect the safety and rights of human subjects.
informed consent — a participant's documented, written agreement to take part after the risks have been clearly explained.