1.2 Data, Sampling, and Variation in Data and Sampling
Learning Objectives
- Sort data into qualitative (categorical) and quantitative (numerical) types.
- Tell quantitative discrete data (from counting) apart from quantitative continuous data (from measuring).
- Read and choose between pie charts, bar graphs, and Pareto charts for qualitative data.
- Describe the main random sampling methods: simple random, stratified, cluster, and systematic.
- Recognize convenience sampling, sampling bias, and the difference between sampling and nonsampling errors.
- Explain why two samples from the same population vary, and why bigger (unbiased) samples are better.
In §1.1 we pinned down the vocabulary — population, sample, parameter, statistic, variable, data. Now we put that vocabulary to work. First we sort the data itself into types, because the type of data decides which graphs and which math even make sense. Then we look at how to actually get a sample that fairly represents a population, and we close by facing a fact of life in statistics: no two samples are ever exactly alike.
1.2.1 Kinds of Data
Qualitative data (also called categorical data) are the result of categorizing or describing attributes of a population. They are generally described by words or letters rather than numbers.
Hair color, blood type, ethnic group, the car a person drives, and the street a person lives on are all qualitative. Hair color might be black, dark brown, light brown, blonde, gray, or red; blood type might be AB+, O−, or B+. Researchers often prefer quantitative data because numbers lend themselves to mathematical analysis — it simply does not make sense to find an average hair color or an average blood type.
Quantitative data are always numbers. They are the result of counting or measuring attributes of a population.
Amount of money, pulse rate, weight, the number of people living in your town, and the number of students who take statistics are all quantitative. Quantitative data split one level further, into discrete and continuous.
Quantitative discrete data are the result of counting. They take on only certain numerical values — usually whole numbers.
If you count the number of phone calls you receive on each day of the week, you might get values such as 0, 1, 2, or 3. You will never get 2.5 phone calls, so the data are discrete.
Quantitative continuous data are the result of measuring. They are not limited to counting numbers — they may include fractions, decimals, or irrational numbers.
Continuous data usually come from measurements like lengths, weights, or times. A list of the lengths in minutes of all the phone calls you make in a week — numbers like 2.4, 7.5, or 11.0 — would be quantitative continuous data.
Why fuss over discrete versus continuous? Because it changes the math downstream. Discrete data are natural to display as counts and whole-number bars; continuous data get grouped into intervals and graphed as histograms. Tagging the data type now saves you from picking the wrong graph or the wrong formula three chapters from now. The examples below walk through each type one at a time so the distinction sticks before you have to choose a tool under pressure.
Data may come from a population or from a sample. We usually write data values with lowercase letters like \(x\) or \(y\). Almost every piece of data you will ever meet falls into one of two big buckets:
- Qualitative (categorical)
- Quantitative (numerical)
Figure 1.2.1 — Every data value lands in one bucket: qualitative, quantitative discrete, or quantitative continuous.
The data are the number of machines in a gym. You sample five gyms. One gym has 12 machines, one has 15, one has ten, one has 22, and the last has 20 machines. What type of data is this?
Solution
We counted machines, and you cannot have a fractional machine — the values are whole numbers.
Answer: quantitative discrete data.
The data are the number of books students carry in their backpacks. You sample five students. Two students carry three books, one student carries four books, one student carries two books, and one student carries one book. The numbers of books — three, four, two, and one — are the data.
What type of data is this?
Solution
Step 1 — counting or measuring? We counted books. You cannot carry 2.5 books, so the values are whole numbers only.
Answer: quantitative discrete data.
The data are the areas of lawns in square feet. You sample five houses. The areas of the lawns are 144, 160, 190, 180, and 210 square feet. What type of data is this?
Solution
Area is measured, not counted, and could in principle take any value in a range.
Answer: quantitative continuous data.
The data are the weights of backpacks with books in them. You sample the same five students. The weights (in pounds) of their backpacks are 6.2, 7, 6.8, 9.1, and 4.3. Notice that two backpacks carrying three books can still have different weights. What type of data is this?
Solution
Step 1 — counting or measuring? We measured weight on a scale, and weight can land at any value — 6.2, 6.8, and so on.
Answer: quantitative continuous data.
A purchasing manager bought the following materials for a company:
- Two types of nails (2 kg box nails, 3 kg roofing nails)
- One type of oil (4 L machine oil)
- Four types of screws (3 kg wood screws, 5 kg machine screws, 1 kg set screws, 2 kg socket screws)
Name data sets that are quantitative discrete, quantitative continuous, and qualitative.
Solution
- Quantitative discrete: the counts of each item — two types of nails, one type of oil, four types of screws.
- Quantitative continuous: the weights and volumes — 2 kg, 3 kg, 4 L, 3 kg, 5 kg, 1 kg, 2 kg — because they are measured.
- Qualitative: the kinds of materials — box nails, roofing nails, machine oil, wood screws, machine screws, set screws, socket screws.
You go to the supermarket and buy three cans of soup (19-ounce tomato bisque, 14.1-ounce lentil, and 19-ounce Italian wedding), two packages of nuts (walnuts and peanuts), four kinds of vegetable (broccoli, cauliflower, spinach, and carrots), and two desserts (16-ounce pistachio ice cream and 32-ounce chocolate chip cookies).
Name data sets that are quantitative discrete, quantitative continuous, and qualitative.
Solution
One possible solution:
- Quantitative discrete: the three cans of soup, two packages of nuts, four kinds of vegetables, and two desserts — because you count them.
- Quantitative continuous: the weights of the soups (19 ounces, 14.1 ounces, 19 ounces) — because you measure weight as precisely as possible.
- Qualitative: the types of soups, nuts, vegetables, and desserts — because they are categories.
Try to identify additional data sets in this example on your own.
The data are the colors of houses. You sample five houses. The colors are white, yellow, white, red, and white. What type of data is this?
Solution
The values are color names — categories, not numbers.
Answer: qualitative (categorical) data.
You may collect data as numbers and then report it categorically. For example, quiz scores are recorded as numbers throughout the term, but at the end the scores are reported as A, B, C, D, or F. The reporting form (letter grades) is qualitative even though the raw data were quantitative.
The data are the colors of backpacks. Again, you sample the same five students. One student has a red backpack, two have black backpacks, one has a green backpack, and one has a gray backpack. The colors red, black, black, green, and gray are the data. What type of data is this?
Solution
The values are color names, not numbers, and averaging colors is nonsense.
Answer: qualitative (categorical) data.
Determine the correct data type for the number of cars in a parking lot. If it is quantitative, say whether it is continuous or discrete.
Solution
We count cars, and the values are whole numbers.
Answer: quantitative discrete data.
Work collaboratively to determine the correct data type (quantitative or qualitative). For quantitative data, also say whether it is continuous or discrete. Hint: data that are discrete often start with the words "the number of."
- the number of pairs of shoes you own
- the type of car you drive
- the distance from your home to the nearest grocery store
- the number of classes you take per school year
- the type of calculator you use
- weights of dogs at an animal shelter
- number of correct answers on a quiz
- IQ scores (this may cause some discussion)
Solution
- Quantitative discrete (counted): a, d, and g.
- Quantitative continuous (measured): c, f, and h.
- Qualitative (categorical): b and e.
The registrar at State University keeps records of the number of credit hours students complete each semester, summarized in the histogram in Figure 1.2.2. The class boundaries are 10 to less than 13, 13 to less than 16, 16 to less than 19, 19 to less than 22, and 22 to less than 25. What type of data does this graph show?
Figure 1.2.2 — Histogram of the number of credit hours completed per student, grouped into class intervals from 10 up to 25.
Solution
Credit hours are measured on a scale and grouped into intervals — the histogram is built for continuous data.
Answer: quantitative continuous data.
A statistics professor records each student's classification — first-year student, sophomore, junior, or senior — and summarizes it in the pie chart in Figure 1.2.3. What type of data does this graph show?
Figure 1.2.3 — Pie chart of statistics students' class standing: first-year, sophomore, junior, or senior.
Solution
The categories are years in school — first-year, sophomore, junior, senior. These are groups, not numbers you would average.
Answer: qualitative (categorical) data.
1.2.2 Displaying Qualitative Data
Once data are sorted by type, the next job is to show them. We start with qualitative data. Below are tables comparing the number of part-time and full-time students at De Anza College and Foothill College for the most recent spring quarter. The tables show counts (frequencies) and percentages or proportions (relative frequencies).
The percent columns make it easy to compare the same category across the two colleges. Showing percentages alongside the raw counts is always helpful, but it matters most when the totals differ — as they do here, since the two colleges have very different total enrollments. Notice how much larger the part-time percentage is at Foothill College than at De Anza College.
| Status | De Anza — Number | De Anza — Percent | Foothill — Number | Foothill — Percent |
|---|---|---|---|---|
| Full-time | 9,200 | 40.9% | 4,059 | 28.6% |
| Part-time | 13,296 | 59.1% | 10,124 | 71.4% |
| Total | 22,496 | 100% | 14,183 | 100% |
Tables are a good way to organize and display data, but graphs can make the patterns even easier to see. There are no strict rules about which graph to use. Three graphs are commonly used for qualitative data:
Pie Charts
In a pie chart, categories of data are shown as wedges of a circle, and each wedge is sized in proportion to the percent of individuals in that category.
Bar Graphs
In a bar graph, the length of each bar is proportional to the number or percent of individuals in that category. Bars may run vertically or horizontally.
Pareto Charts
A Pareto chart is a bar graph whose bars are sorted by category size, from largest to smallest. Sorting makes the biggest categories jump out immediately.
Look at the pie charts in Figure 1.2.4 and the bar graph in Figure 1.2.5 and decide which one you think shows the comparison better. It is a good idea to look at several graphs and pick whichever is most helpful for the data and the context — and that choice also depends on what you are using the data for.
Figure 1.2.4 — Side-by-side pie charts of full-time vs. part-time enrollment at De Anza College and Foothill College.
Figure 1.2.5 — Bar graph comparing full-time and part-time enrollment at De Anza and Foothill Colleges.
When Percentages Add to More or Less Than 100%
Sometimes the percentages add up to more than 100% (or less than 100%). In the data below, the percentages add to more than 100% because a student can fall into more than one category at once. When that happens, a bar graph is the right tool to compare the relative sizes of the categories — a pie chart cannot be used, because pie wedges must add to exactly 100%. (A pie chart also fails when the percentages add to less than 100%.)
| Characteristic / Category | Percent |
|---|---|
| Full-Time Students | 40.9% |
| Students who intend to transfer to a 4-year institution | 48.6% |
| Students under age 25 | 61.0% |
| Total | 150.5% |
Figure 1.2.6 — Bar graph of overlapping De Anza student characteristics, whose percentages sum to more than 100%.
When a Category Is Missing
The next table displays the ethnicity of students but leaves out the "Other/Unknown" category — people who did not feel they fit any listed category or who declined to respond. Notice that the frequencies do not add up to the total number of students. In this situation, use a bar graph, not a pie chart, because the parts no longer make a whole.
| Ethnicity | Frequency | Percent |
|---|---|---|
| Asian | 8,794 | 36.1% |
| Black | 1,412 | 5.8% |
| Filipino | 1,298 | 5.3% |
| Hispanic/Latino | 4,180 | 17.1% |
| Native American | 146 | 0.6% |
| Pacific Islander | 236 | 1.0% |
| White | 5,978 | 24.5% |
| Total | 22,044 out of 24,382 | 90.4% out of 100% |
Figure 1.2.7 — Bar graph of student ethnicity at De Anza College with the Other/Unknown category omitted.
The next graph puts the "Other/Unknown" percent (9.6%) back in, so the categories now account for all the students. That category turns out to be large compared with some others (Native American at 0.6%, Pacific Islander at 1.0%), which is important to know when we think about what the data are telling us.
Listing the categories in alphabetical order — as in the bar graph above — makes the bars jump up and down, which is hard to read. A Pareto chart fixes that: its bars are sorted from largest to smallest, so the ranking is obvious at a glance (Figure 1.2.8).
Figure 1.2.8 — Pareto chart of student ethnicity with bars sorted from largest to smallest.
When the percentages do add to 100% (Other/Unknown included), a pie chart works again. The pie chart in Figure 1.2.9(b) is organized by wedge size, which makes it more informative than the unsorted, alphabetical version in Figure 1.2.9(a).
Figure 1.2.9 — Two pie charts of student ethnicity (Other/Unknown included): (a) alphabetical order, (b) sorted by wedge size.
A survey reports the percent of students at a college who play each of several intramural sports, and a single student is allowed to sign up for more than one sport. The reported percentages add up to 130%. Should this be displayed as a pie chart or a bar graph? Why?
Solution
Because a student can be in more than one category, the percentages add to more than 100%. A pie chart's wedges must total exactly 100%, so a pie chart cannot represent this data honestly.
Answer: use a bar graph, not a pie chart.
1.2.3 Sampling Methods
In a simple random sample, any group of individuals is equally likely to be chosen as any other group of the same size. In other words, every possible sample of a given size has an equal chance of being selected.
Figure 1.2.10 — A simple random sample gives every group of the same size an equal chance of selection.
Suppose Lisa wants to form a four-person study group (herself plus three others) from her pre-calculus class, which has 31 other members. To choose a simple random sample of size three, Lisa could write all 31 names on slips, put them in a hat, shake it, close her eyes, and pick three. A more technological route is to list everyone's last name beside a two-digit ID, as in Table 1.2.4:
| ID | Name | ID | Name | ID | Name |
|---|---|---|---|---|---|
| 00 | Anselmo | 11 | King | 21 | Roquero |
| 01 | Bautista | 12 | Legeny | 22 | Roth |
| 02 | Bayani | 13 | Lundquist | 23 | Rowell |
| 03 | Cheng | 14 | Macierz | 24 | Salangsang |
| 04 | Cuarismo | 15 | Motogawa | 25 | Slade |
| 05 | Cuningham | 16 | Okimoto | 26 | Stratcher |
| 06 | Fontecha | 17 | Patel | 27 | Tallai |
| 07 | Hong | 18 | Price | 28 | Tran |
| 08 | Hoobler | 19 | Quizon | 29 | Wai |
| 09 | Jiao | 20 | Reyes | 30 | Wood |
| 10 | Khan |
Lisa can use a table of random numbers, a calculator, or a computer to generate random numbers. Suppose her calculator produces:
0.94360; 0.99832; 0.14669; 0.51470; 0.40581; 0.73381; 0.04399
She reads each value in two-digit groups until she has three class members. For example, she reads 0.94360 as the groups 94, 43, 36, 60. Each random number may contribute only one class member.
The first two numbers, 0.94360 and 0.99832, contain no usable two-digit ID (nothing between 00 and 30). The third, 0.14669, contains 14; the fifth, 0.40581, contains 05; and the seventh, 0.04399, contains 04. ID 14 is Macierz, 05 is Cuningham, and 04 is Cuarismo. So besides herself, Lisa's group is Macierz, Cuningham, and Cuarismo.
Using raSHio
To generate random numbers:
- Open raSHio.
- Choose File → Random Numbers….
- Set Min to
0, Max to30, and How many to3. - Check No repeats so no ID appears twice, then click Generate — all three sample IDs appear at once.
Figure 1.2.11 — Drawing Lisa's three sample IDs in raSHio.
Besides simple random sampling, several other methods also use a chance process. The best-known are the stratified sample, the cluster sample, and the systematic sample.
To choose a stratified sample, divide the population into groups called strata and then take a proportionate random sample from each stratum.
Figure 1.2.12 — Stratified sampling draws a few members from every stratum.
For example, you could stratify your college population by department, then take a proportionate simple random sample from each department. Number each member of the first department and use simple random sampling to pick a proportionate number; repeat for every department. All the names picked across all departments make up the stratified sample.
To choose a cluster sample, divide the population into clusters (groups), randomly select some of the clusters, and include every member of the chosen clusters.
Figure 1.2.13 — Cluster sampling keeps every member of the randomly chosen clusters.
For example, divide your college faculty by department — the departments are the clusters. Number the departments, use simple random sampling to choose four of them, and every faculty member in those four departments is in the cluster sample.
To choose a systematic sample, randomly select a starting point and then take every \(k\)-th piece of data from a list of the population.
Figure 1.2.14 — Systematic sampling takes a random start, then every \(k\)-th individual.
For example, suppose you must do a phone survey from a phone book with 20,000 residential listings, and you need 400 names. Number the population 1 through 20,000, use a simple random sample to pick the first name, then take every fiftieth name after that until you have 400 (wrapping back to the start of the list if needed). Systematic sampling is popular because it is so simple.
A convenience sample is a non-random sample that uses whatever results are readily available.
Figure 1.2.15 — A convenience sample takes whoever is easiest to reach — and misses everyone else.
Sampling carelessly can wreck a study before a single number is crunched. Mailed surveys that people return on their own are notoriously biased, because the people who bother to reply are rarely a fair cross-section of everyone. Whenever possible, the person running the survey should choose the respondents rather than letting respondents choose themselves. Keep this in mind as we sort the next batch of scenarios — the trap in each one is almost always who got left out.
For example, a software store runs a marketing study by interviewing whoever happens to be browsing in the store that day. Convenience sampling can give very good results in some cases and highly biased results (results that favor certain outcomes) in others.
There is also a subtle choice about whether a chosen member can be picked again. Sampling with replacement is truly random: once a member is picked, they go back into the population and could be chosen more than once. For practical reasons, though, most real surveys use sampling without replacement — a member can be chosen only once. When samples are small compared with a large population, sampling without replacement is approximately the same as sampling with replacement, because the chance of drawing the same person twice is tiny. The distinction only becomes a real mathematical issue when the population is small.
When you analyze data, watch for two kinds of error. A sampling error is caused by the sampling process itself — for example, the sample may not be large enough. A nonsampling error comes from factors unrelated to sampling — for example, a defective counting device. A sample will never be exactly representative of its population, so some sampling error is always present; as a rule, the larger the sample, the smaller the sampling error.
Finally, a sampling bias occurs when some members of the population are less likely to be chosen than others (remember, every member should have an equally likely chance). When bias creeps in, the conclusions drawn about the population can be flat wrong.
Gathering information about an entire population usually costs too much or is simply impossible. Instead, we study a sample of the population. A good sample has the same characteristics as the population it represents, and statisticians use various methods of random sampling to try to achieve that. In every form of random sampling, each member of the population starts out with an equal chance of being selected. Each method has its own pros and cons.
Determine the type of sampling used (simple random, stratified, systematic, cluster, or convenience): a high school principal polls 50 first-year students, 50 sophomores, 50 juniors, and 50 seniors regarding policy changes for after-school activities.
Solution
The population is divided into groups (the four grade levels), and a sample is taken from each group.
Answer: stratified sampling.
1.2.4 Evaluating Studies Critically
We should read statistical studies critically and analyze them before accepting their results. Watch for these common problems:
Problems with the Sample
A sample must be representative of the population. A sample that is not representative is biased, and biased samples give results that are inaccurate and not valid.
Self-Selected Samples
Responses only from people who choose to respond — such as call-in surveys — are often unreliable.
Sample-Size Issues
Samples that are too small may be unreliable; larger samples are better when possible. Sometimes small samples are unavoidable and still useful — for example, crash-testing cars or medical testing for rare conditions.
Undue Influence
Collecting data or asking questions in a way that influences the response.
Non-Response or Refusal to Participate
When many subjects do not respond, the collected responses may no longer represent the population. People with strong positive or negative opinions are often the ones who answer, which skews the results.
Causality
A relationship between two variables does not mean one causes the other. They may be related (correlated) only because of a third, lurking variable.
Self-Funded or Self-Interest Studies
A study run by a person or organization to support their own claim. Ask whether it is impartial. Do not automatically assume it is bad — or good — but evaluate it on the merits of the work done.
Misleading Use of Data
Improperly displayed graphs, incomplete data, or missing context.
Confounding
When the effects of several factors on a response cannot be separated, the factors are confounded, and it becomes difficult or impossible to draw valid conclusions about any single factor.
As a class, decide whether each of the following samples is representative. If it is not, discuss why.
1. To find the average GPA of all students at a university, use all honor students as the sample.
2. To find the most popular cereal among children under ten, stand outside a large supermarket for three hours and speak to every twentieth child under ten who enters.
3. To find the average annual income of all U.S. adults, sample U.S. Representatives: treat each state as a stratum, use simple random sampling to select states, then survey every Representative in the chosen states.
4. To find the proportion of people who take public transportation to work, sit on a bench in Central Park and interview the next 20 people who sit beside you.
5. To find the average cost of a two-day hospital stay in Massachusetts, survey 100 hospitals across the state using simple random sampling.
A pharmaceutical company runs and pays for a study concluding that its own new drug is highly effective, and the study surveys only patients who chose to stay enrolled to the end. Name two of the critical-evaluation problems above that this scenario raises.
Solution
- Self-funded / self-interest study: the company stands to gain from a favorable result, so the study may not be impartial — read it carefully on its merits.
- Non-response / self-selection: surveying only patients who chose to stay enrolled drops anyone who quit (perhaps because the drug did not work or caused side effects), so the remaining responses may overstate effectiveness.
Answer: self-funded/self-interest bias and non-response (self-selection) bias.
Before we move on to the worked examples that drill these sampling types, it helps to name the move you will make each time. For every scenario, ask two questions in order: Was chance used at all? If not, it is convenience sampling. If chance was used, what got divided and what got selected? If you split the population into groups and sampled within every group, it is stratified; if you picked a few whole groups and took everyone inside them, it is cluster; if you walked down a list grabbing every \(k\)-th name, it is systematic; and if every individual had an equal, independent shot, it is simple random. The next two examples are pure practice at running that checklist quickly.
A study determines the average tuition that San Jose State undergraduates pay per semester. Each student in the samples below is asked how much tuition they paid for the Fall semester. What is the type of sampling in each case?
- A sample of 100 undergraduates is taken by organizing students' names by classification (first-year, sophomore, junior, senior) and then selecting 25 students from each group.
- A random number generator selects one student from the alphabetical list of all undergraduates. Starting there, every 50th student is chosen until 75 students are included.
- A completely random method selects 75 students; each undergraduate has the same probability of being chosen at any stage.
- The first-year, sophomore, junior, and senior classes are numbered 1, 2, 3, 4. A random number generator picks two of those years, and all students in those two years are in the sample.
- An administrative assistant stands in front of the library one Wednesday and asks the first 100 undergraduates he meets what they paid in tuition.
Solution
- a. stratified — split into groups (years) and sampled within each.
- b. systematic — random start, then every 50th student.
- c. simple random — every student equally likely at every stage.
- d. cluster — picked whole groups (two years) and took everyone in them.
- e. convenience — whoever happened to be there.
Answer: a. stratified; b. systematic; c. simple random; d. cluster; e. convenience.
You will use a random number generator to draw different types of samples from the quiz-score data below. The table shows six sets of quiz scores (each quiz is worth 10 points) for an elementary statistics class.
| #1 | #2 | #3 | #4 | #5 | #6 |
|---|---|---|---|---|---|
| 5 | 7 | 10 | 9 | 8 | 3 |
| 10 | 5 | 9 | 8 | 7 | 6 |
| 9 | 10 | 8 | 6 | 7 | 9 |
| 9 | 10 | 10 | 9 | 8 | 9 |
| 7 | 8 | 9 | 5 | 7 | 4 |
| 9 | 9 | 9 | 10 | 8 | 7 |
| 7 | 7 | 10 | 9 | 8 | 8 |
| 8 | 8 | 9 | 10 | 8 | 8 |
| 9 | 7 | 8 | 7 | 7 | 8 |
| 8 | 8 | 10 | 9 | 8 | 7 |
Use the random number generator to pick samples:
- Stratified sample by column. Pick three quiz scores randomly from each column. Number each row 1 through 10. On your calculator, press
MATH, arrow toPRB, press5:randInt(, enter1,10), and pressENTER. Record the number, pressENTERtwice more (even repeats), and record those numbers. Record the three quiz scores in that column matching the three row numbers. Repeat for columns 2 through 6. These 18 scores are a stratified sample. - Cluster sample. Pick two of the columns. Press
MATH, arrow toPRB, press5:randInt(, enter1,6), pressENTER, record the number, pressENTERagain, and record that number. The quiz scores (20 of them) in those two columns are the cluster sample. - Simple random sample of 15 scores. Number the scores 1 through 60. Press
MATH, arrow toPRB, press5:randInt(, enter1, 60). PressENTER15 times and record the numbers. Record the matching quiz scores. These 15 scores are the simple random sample. - Systematic sample of 12 scores. Number the scores 1 through 60. Press
MATH, arrow toPRB, press5:randInt(, enter1, 60), pressENTER, and record the number and its quiz score. From that number, count ten scores and record that score; keep counting ten and recording until you have 12 scores. You may wrap around to the beginning.
Solution
This is a hands-on calculator activity, so answers depend on the random numbers your calculator produces. The point is the method, not a single right list:
- Stratified: three scores per column gives \(3 \times 6 = 18\) scores.
- Cluster: two whole columns gives \(2 \times 10 = 20\) scores.
- Simple random: 15 scores drawn from the full pool of 60.
- Systematic: a random start, then every tenth score, for 12 scores.
Each method should pull from the table as described above.
Determine the type of sampling used (simple random, stratified, systematic, cluster, or convenience).
- A soccer coach selects six players from boys aged 8–10, seven from boys aged 11–12, and three from boys aged 13–14 to form a team.
- A pollster interviews all human-resource personnel in five different high-tech companies.
- An education researcher interviews 50 public high school teachers and 50 private high school teachers.
- A medical researcher interviews every third cancer patient from a list of cancer patients at a local hospital.
- A high school counselor uses a computer to generate 50 random numbers and picks the students whose names match those numbers.
- A student interviews classmates in their algebra class to find the average number of pairs of jeans a student owns.
Solution
- a. stratified — sampled within each age group.
- b. cluster — picked whole groups (companies) and took everyone in them.
- c. stratified — sampled within each group (public, private).
- d. systematic — every third patient on the list.
- e. simple random — computer-generated random selection.
- f. convenience — whoever was in their own class.
Answer: a. stratified; b. cluster; c. stratified; d. systematic; e. simple random; f. convenience.
1.2.5 Variation in Data and Samples
If we examined two samples from the same population — even using random sampling for both — they would not come out exactly the same. Just as there is variation in data, there is variation in samples. As you get used to sampling, this variability will start to feel natural.
In the Confidence Intervals chapter, you will meet sample-size formulas that tell you how big a sample to take. The required size depends on the precision you want, not on the size of the population. It can feel counterintuitive, but a sample of 1,000 can adequately represent a population of 100,000 or even 1,000,000, as long as you want the same level of precision. When you reach those formulas, notice that population size never appears in them.
A local radio station has a fan base of 20,000 listeners and wants to know whether its audience prefers more music or more talk shows. Asking all 20,000 is nearly impossible, so the station uses convenience sampling and surveys the first 200 people it meets at one of its music concert events. Of those, 24 prefer more talk shows and 176 prefer more music. Is this sample representative of the entire 20,000-listener population?
Solution
No. The sample was gathered at a music concert, so it over-represents people who already like music — exactly the preference being measured. People who would rather hear talk shows are unlikely to be at a music event, so they are under-represented. This is a biased convenience sample.
Answer: the sample is not representative; it is biased toward music fans.
Suppose ABC College has 10,000 part-time students (the population), and we want the average amount a part-time student spends on books in the fall term. Asking all 10,000 students is nearly impossible, so we take samples.
First, we use convenience sampling and survey ten students from a first-term organic chemistry class. Many also take first-term calculus. They spend:
$128; $87; $173; $116; $130; $204; $147; $189; $93; $153
Second, using a list of senior citizens who take P.E. classes, we take every fifth senior citizen, for ten people. They spend:
$50; $40; $36; $15; $50; $100; $40; $53; $22; $22
It is unlikely any student is in both samples.
- Is either sample representative of the entire 10,000 part-time student population?
- Since these samples are not representative, is it wise to use them to describe the whole population?
- Now suppose we take a third sample: ten part-time students, one chosen by simple random sampling from each of ten disciplines (chemistry, math, English, psychology, sociology, history, nursing, physical education, art, and early childhood development), assuming equal enrollment across disciplines. They spend:
$180; $50; $150; $85; $260; $75; $180; $200; $200; $150
Is this third sample biased?
Solution
a. No. The first sample is mostly science-oriented students whose books (chemistry, calculus) tend to be expensive, so they likely pay more than the average part-time student. The second sample is senior citizens taking courses for health and interest, who likely pay less. Both are biased, and in both cases not all students had a chance to be chosen.
b. No. In these samples, each member of the population did not have an equally likely chance of being chosen, so the results should not be used to describe the whole population.
c. The third sample is unbiased, but a larger sample would be recommended to increase the chance that it closely represents the population. (For a biased technique, even a large sample risks being unrepresentative.) Students often ask whether sampling is "good enough" instead of surveying everyone. If the survey is done well, the answer is yes.
Variation in Data
Variation is present in any set of data. For example, 16-ounce cans of a beverage may hold a little more or a little less than 16 ounces. In one study, eight 16-ounce cans were measured and produced these amounts (in ounces):
15.8; 16.1; 15.2; 14.8; 15.8; 15.9; 16.0; 15.5
The measured amounts vary because different people make the measurements, or because exactly 16 ounces was never put in the cans in the first place. Manufacturers regularly run tests to check whether the amount in a 16-ounce can falls within the desired range.
Be aware that your data may differ a bit from data someone else collects for the same purpose — that is completely natural. But if two people collecting the same data get very different results, it is time to reevaluate the data-taking methods and accuracy.
Variation in Samples
As noted, two or more random samples from the same population will likely differ from each other. Suppose Doreen and Jung both study the average nightly sleep of students at their college, each taking samples of 500 students. Doreen uses systematic sampling and Jung uses cluster sampling, so their samples will differ. Even if they used the same method, their samples would almost certainly still differ — and neither would be wrong.
Think about what makes Doreen's and Jung's samples different. If they took larger samples, their results (the average sleep time) might land closer to the true population average — but the two samples would still, in all likelihood, differ from each other. This variability in samples cannot be stressed enough.
Two students each take a random sample of 40 classmates and compute the average height. They get slightly different averages. Does this mean one of them made a mistake?
Solution
No. Two random samples from the same population almost never produce identical results — that is normal sampling variation, not an error. Neither student is wrong; their averages simply reflect two different subsets of the same population.
Answer: no mistake — this is ordinary variation between samples.
1.2.6 Size of a Sample
The size of a sample (often called the number of observations) matters. The examples in this book so far have been small. Samples of a few hundred observations — or even fewer — are enough for many purposes. In polling, samples of about 1,200 to 1,500 observations are considered large and good enough, if the survey is random and well done. You will learn why when you study confidence intervals.
Be aware that many large samples are still biased. Call-in surveys, for example, are invariably biased because people choose whether or not to respond.
Divide into groups of two, three, or four. Your instructor will give each group one six-sided die. Try this experiment twice: roll one fair die 20 times and record how many ones, twos, threes, fours, fives, and sixes you get in Table 1.2.6 and Table 1.2.7 ("frequency" is the number of times a particular face appears).
| Face on Die | Frequency |
|---|---|
| 1 | |
| 2 | |
| 3 | |
| 4 | |
| 5 | |
| 6 |
| Face on Die | Frequency |
|---|---|
| 1 | |
| 2 | |
| 3 | |
| 4 | |
| 5 | |
| 6 |
Did the two experiments give the same results? Probably not. If you ran the experiment a third time, would you expect results identical to the first or second? Why or why not? Which experiment had the "correct" results? They both did — the job of the statistician is to see through the variability and draw appropriate conclusions.
A pollster says a national survey of 1,500 randomly chosen, well-surveyed adults is "large enough" to be reliable, but a call-in survey with 50,000 responses is not. How can the smaller sample be more trustworthy than the bigger one?
Solution
Sheer size does not fix bias. The call-in survey is self-selected — only people motivated to call in respond — so even 50,000 responses can be wildly unrepresentative. The 1,500-person survey is random, so every adult had an equal chance of being chosen, which keeps it representative. A smaller unbiased sample beats a huge biased one.
Answer: the 1,500-person random survey is more reliable because it is unbiased; the call-in survey, despite its size, is biased by self-selection.
Problem Set 1.2
For problems 1 through 5, use the following information.
A study was done to determine the age, number of times per week, and the duration (amount of time) of residents using a local park in San Antonio, Texas. The first house in the neighborhood around the park was selected randomly, and then the resident of every eighth house in the neighborhood around the park was interviewed.
Problem 1. "Number of times per week" is what type of data?
a) qualitative (categorical)
b) quantitative discrete
c) quantitative continuous
Solution
Step 1 — Classify the variable: "Number of times per week" is a count, so it is quantitative (numeric), not categorical.
Step 2 — Discrete or continuous? A count can only take whole-number values (0, 1, 2, …); you cannot use the park 2.5 times in a counting sense, so it is discrete.
Answer: b) quantitative discrete.
Problem 2. The sampling method was
a) simple random
b) systematic
c) stratified
d) cluster
Solution
Step 1 — Identify the selection rule: Starting from one randomly chosen first house, the interviewer then takes every eighth house — a fixed interval down the list.
Step 2 — Match to a sampling method: Choosing every \(k\)th element from an ordered list (here \(k = 8\)) is the definition of systematic sampling.
Answer: b) systematic.
Problem 3. "Duration (amount of time)" is what type of data?
a) qualitative (categorical)
b) quantitative discrete
c) quantitative continuous
Solution
Step 1 — Classify the variable: "Duration (amount of time)" is measured numerically, so it is quantitative.
Step 2 — Discrete or continuous? Time can take any value in a range (e.g., 12.4 minutes, 12.43 minutes); measurements are not restricted to whole numbers, so it is continuous.
Answer: c) quantitative continuous.
Problem 4. The colors of the houses around the park are what kind of data?
a) qualitative (categorical)
b) quantitative discrete
c) quantitative continuous
Solution
Step 1 — Classify the variable: A house color (red, blue, white, …) is a label/category, not a number.
Step 2 — Confirm the type: Since the values are names rather than measurements or counts, the data are qualitative.
Answer: a) qualitative (categorical).
Problem 5. The population is ____________.
Solution
Step 1 — Recall the definition: The population is every individual the study aims to describe.
Step 2 — Apply it to the park study: The researchers want to characterize the people who use this San Antonio park, so the population is all of those users.
Answer: The population is all the people (residents) who use that local park in San Antonio.
For problems 6 through 9, determine the type of sampling used (simple random, stratified, systematic, cluster, or convenience).
Problem 6. A group of test subjects is divided into twelve groups; then four of the groups are chosen at random.
Solution
Step 1 — Identify the selection rule: Twelve groups are formed, then 4 entire groups are chosen at random and everyone in those groups is used.
Step 2 — Match to a sampling method: Sampling whole pre-existing groups (clusters) rather than individuals is cluster sampling.
Answer: Cluster sampling.
Problem 7. A market researcher polls every tenth person who walks into a store.
Solution
Step 1 — Identify the selection rule: The researcher polls every tenth person entering the store — a fixed interval applied to an ordered stream of people.
Step 2 — Match to a sampling method: Selecting every \(k\)th element (\(k = 10\)) is systematic sampling.
Answer: Systematic sampling.
Problem 8. The first 50 people who walk into a sporting event are polled on their television preferences.
Solution
Step 1 — Identify the selection rule: The first 50 people through the door are polled simply because they are easiest to reach.
Step 2 — Match to a sampling method: Choosing whoever is most readily available, with no random mechanism, is convenience sampling.
Answer: Convenience sampling.
Problem 9. A computer generates 100 random numbers, and 100 people whose names correspond with the numbers on the list are chosen.
Solution
Step 1 — Identify the selection rule: A computer produces 100 random numbers, and the 100 people matching those numbers are selected; every person on the list has an equal chance.
Step 2 — Match to a sampling method: Equal-probability selection of individuals by a random device is simple random sampling.
Answer: Simple random sampling.
For problems 10 through 17, use the following Researcher A and Researcher B respiratory-virus data sets (40 patients each).
Studies are often done by pharmaceutical companies to determine the effectiveness of a treatment program. Suppose that a new drug is currently under study to address a respiratory virus. It is given to patients once the patient exhibits symptoms of the virus. Of interest is the average (mean) length of time in days from the time the patient starts the treatment until the symptoms are alleviated. Two researchers each follow a different set of 40 patients with the respiratory virus from the start of treatment until the symptoms are alleviated. The following data (in days) are collected.
Researcher A: 3; 4; 11; 15; 16; 17; 22; 44; 37; 16; 14; 24; 25; 15; 26; 27; 33; 29; 35; 44; 13; 21; 22; 10; 12; 8; 40; 32; 26; 27; 31; 34; 29; 17; 8; 24; 18; 47; 33; 34
Researcher B: 3; 14; 11; 5; 16; 17; 28; 41; 31; 18; 14; 14; 26; 25; 21; 22; 31; 2; 35; 44; 23; 21; 21; 16; 12; 18; 41; 22; 16; 25; 33; 34; 29; 13; 18; 24; 23; 42; 33; 29
Problem 10. Complete the tables using the data provided.
| Survival Length (in days) | Frequency | Relative Frequency | Cumulative Relative Frequency |
|---|---|---|---|
| 0.5–6.5 | |||
| 6.5–12.5 | |||
| 12.5–18.5 | |||
| 18.5–24.5 | |||
| 24.5–30.5 | |||
| 30.5–36.5 | |||
| 36.5–42.5 | |||
| 42.5–48.5 |
| Survival Length (in days) | Frequency | Relative Frequency | Cumulative Relative Frequency |
|---|---|---|---|
| 0.5–6.5 | |||
| 6.5–12.5 | |||
| 12.5–18.5 | |||
| 18.5–24.5 | |||
| 24.5–30.5 | |||
| 30.5–36.5 | |||
| 36.5–45.5 |
Solution
Step 1 — Set up the bins: Each row counts how many of the \(n = 40\) survival times fall in that interval (lower bound exclusive, upper bound inclusive). Relative frequency \(= \dfrac{\text{frequency}}{40}\), and cumulative relative frequency is the running total of relative frequencies.
Step 2 — Tally Researcher A (Table 1.2.8): Counting Researcher A's 40 values into each bin gives frequencies \(2, 5, 9, 5, 7, 7, 2, 3\) (sum \(= 40\)).
| Survival Length | Frequency | Rel. Freq. | Cum. Rel. Freq. |
|---|---|---|---|
| 0.5–6.5 | 2 | 0.050 | 0.050 |
| 6.5–12.5 | 5 | 0.125 | 0.175 |
| 12.5–18.5 | 9 | 0.225 | 0.400 |
| 18.5–24.5 | 5 | 0.125 | 0.525 |
| 24.5–30.5 | 7 | 0.175 | 0.700 |
| 30.5–36.5 | 7 | 0.175 | 0.875 |
| 36.5–42.5 | 2 | 0.050 | 0.925 |
| 42.5–48.5 | 3 | 0.075 | 1.000 |
Step 3 — Tally Researcher B (Table 1.2.9): Counting Researcher B's 40 values gives frequencies \(3, 2, 11, 8, 6, 6, 4\) (sum \(= 40\)).
| Survival Length | Frequency | Rel. Freq. | Cum. Rel. Freq. |
|---|---|---|---|
| 0.5–6.5 | 3 | 0.075 | 0.075 |
| 6.5–12.5 | 2 | 0.050 | 0.125 |
| 12.5–18.5 | 11 | 0.275 | 0.400 |
| 18.5–24.5 | 8 | 0.200 | 0.600 |
| 24.5–30.5 | 6 | 0.150 | 0.750 |
| 30.5–36.5 | 6 | 0.150 | 0.900 |
| 36.5–45.5 | 4 | 0.100 | 1.000 |
Step 4 — Check: Each frequency column sums to 40 and each final cumulative relative frequency equals 1.000, confirming the tables are complete.
Answer: The completed Researcher A and Researcher B frequency tables are shown above; frequencies are \(2,5,9,5,7,7,2,3\) for A and \(3,2,11,8,6,6,4\) for B, both summing to 40 with cumulative relative frequency reaching 1.000.
Problem 11. Determine what the key term data refers to in the above example for Researcher A.
Solution
Step 1 — Recall the definition: Data are the actual recorded values of the variable for the members of the sample.
Step 2 — Apply it to Researcher A: For Researcher A, the data are the 40 measured survival lengths (3, 4, 11, 15, … , 34 days).
Answer: The data are the set of 40 recorded survival times (in days) for Researcher A's patients.
Problem 12. List two reasons why the data may differ.
Solution
Step 1 — Think about the source of variation: The two researchers followed different sets of 40 patients, and individual patients respond differently to the drug.
Step 2 — List two reasons: (1) The two samples consist of different individuals, so natural patient-to-patient variation produces different values; (2) differences in conditions or methodology — for example, how the patients were selected, regional or demographic differences, severity of illness at start, or measurement/recording differences — can shift the results.
Answer: The data may differ because (1) each researcher sampled a different group of patients (sampling variability), and (2) factors such as how patients were chosen, their differing health conditions/severity, or measurement differences vary between the two studies.
Problem 13. Can you tell if one researcher is correct and the other one is incorrect? Why?
Solution
Step 1 — Interpret "correct": Each researcher honestly recorded the survival times for their own randomly chosen sample.
Step 2 — Explain why neither is wrong: Two different random samples from the same population are expected to give different results; that difference is sampling variability, not error. Neither set of data is "correct" or "incorrect" — both are valid samples.
Answer: No. Both researchers can be reporting accurate data; the differences simply reflect natural sampling variability between two different groups of patients, so one cannot be declared right and the other wrong.
Problem 14. Would you expect the data to be identical? Why or why not?
Solution
Step 1 — Consider what identical data would require: Identical data would mean both researchers happened to draw the same 40 individuals (or the same exact values), which is essentially impossible for two separate random samples.
Step 2 — Explain: Because the samples contain different patients and patients vary in how they respond, the recorded values will differ.
Answer: No, we would not expect the data to be identical. The two researchers sampled different patients, and individual responses vary, so the two data sets will naturally differ even though they describe the same population.
Problem 15. Suggest at least two methods the researchers might use to gather random data.
Solution
Step 1 — Recall what "random" requires: A random method gives every patient a known, fair chance of being selected, removing the researcher's personal choice.
Step 2 — Suggest two methods: (1) Simple random sampling — assign every eligible patient a number and use a random number generator or table to pick 40; (2) systematic sampling — order all eligible patients and select every \(k\)th one starting from a random start. (Stratified or cluster random sampling are also acceptable.)
Answer: Two workable methods are simple random sampling (number every patient and draw 40 with a random number generator) and systematic sampling (pick every \(k\)th patient from an ordered list after a random start).
Problem 16. Suppose that the first researcher conducted his survey by randomly choosing one state in the nation and then randomly picking 40 patients from that state. What sampling method would that researcher have used?
Solution
Step 1 — Identify the two stages: First a state is chosen at random; then 40 patients are chosen at random from within that one state.
Step 2 — Match to a sampling method: Selecting whole geographic group(s) at random and then sampling within them is cluster sampling (the state is the cluster).
Answer: Cluster sampling.
Problem 17. Suppose that the second researcher conducted his survey by choosing 40 patients he knew. What sampling method would that researcher have used? What concerns would you have about this data set, based upon the data collection method?
Solution
Step 1 — Identify the selection rule: The researcher used 40 patients he already knew, choosing them because they were easy to access rather than by any random mechanism.
Step 2 — Match to a sampling method: Selecting readily available, familiar subjects is convenience sampling.
Step 3 — State the concern: A convenience sample is likely biased and not representative of the whole population — patients the researcher knows may share characteristics (location, health habits, demographics) that differ from typical patients, so conclusions drawn from this sample may not generalize.
Answer: Convenience sampling. The main concern is bias: because the patients were not chosen randomly but because the researcher knew them, the sample may not represent the broader population, making the results unreliable for general conclusions.
For problems 18 through 22, two researchers are gathering data on hours of video games played by school-aged children and young adults. They each randomly sample different groups of 150 students from the same school. They collect the following data.
| Hours Played per Week | Frequency | Relative Frequency | Cumulative Relative Frequency |
|---|---|---|---|
| 0–2 | 26 | 0.17 | 0.17 |
| 2–4 | 30 | 0.20 | 0.37 |
| 4–6 | 49 | 0.33 | 0.70 |
| 6–8 | 25 | 0.17 | 0.87 |
| 8–10 | 12 | 0.08 | 0.95 |
| 10–12 | 8 | 0.05 | 1 |
| Hours Played per Week | Frequency | Relative Frequency | Cumulative Relative Frequency |
|---|---|---|---|
| 0–2 | 48 | 0.32 | 0.32 |
| 2–4 | 51 | 0.34 | 0.66 |
| 4–6 | 24 | 0.16 | 0.82 |
| 6–8 | 12 | 0.08 | 0.90 |
| 8–10 | 11 | 0.07 | 0.97 |
| 10–12 | 4 | 0.03 |
Problem 18. Give a reason why the data may differ.
Solution
Step 1 — Recognize the source of variability: Two researchers each surveyed a different set of respondents, so the two data sets are independent samples drawn from the population. Random sampling means no two samples will be identical.
Step 2 — Identify a concrete reason: Differences arise from sampling variability — each sample contains different individuals whose video-game habits differ. Other contributing factors include how the question was worded, when and where each survey was administered, and ordinary measurement/reporting error.
Answer: The data differ because each researcher surveyed a different random sample of people; sampling variability (plus possible differences in question wording, timing, and respondent honesty) produces different results even when both studies are run correctly.
Problem 19. Would the sample size be large enough if the population is the students in the school?
Solution
Step 1 — Compare sample size to population size: "The students in the school" is a relatively small, well-defined population. A survey covering a sizeable fraction of that population gives a representative picture.
Step 2 — Judge adequacy: Because the population is limited to one school, even a few hundred respondents would represent a large share of the whole, so the sample is large enough to draw reliable conclusions about that school.
Answer: Yes. For a population as small and specific as the students in one school, the sample is large enough to be reliable.
Problem 20. Would the sample size be large enough if the population is school-aged children and young adults in the United States?
Solution
Step 1 — Reconsider the population size: "School-aged children and young adults in the United States" is an enormous and highly diverse population numbering in the tens of millions.
Step 2 — Judge adequacy: A sample taken at a single school cannot capture the geographic, economic, and cultural diversity of the whole country, and it is a tiny fraction of that population, so it is neither large enough nor representative.
Answer: No. The sample is far too small and not representative enough to support conclusions about all school-aged children and young adults in the United States.
Problem 21. Researcher A concludes that most students play video games between four and six hours each week. Researcher B concludes that most students play video games between two and four hours each week. Who is correct?
Solution
Step 1 — Examine what each researcher is claiming: Each conclusion describes the typical play time for that researcher's own sample. The two samples are different sets of people.
Step 2 — Decide whether one must be wrong: Because the two samples differ, both conclusions can be valid descriptions of their respective samples. Neither is necessarily "correct" for the whole population — the difference reflects sampling variability, not error.
Answer: Neither is provably correct. Each conclusion accurately describes that researcher's own sample; the discrepancy is due to sampling variability, so we cannot say one researcher is right and the other wrong.
Problem 22. As part of a way to reward students for participating in the survey, the researchers gave each student a gift card to a video game store. Would this affect the data if students knew about the award before the study?
Solution
Step 1 — Consider how an incentive changes behavior: If students knew about the gift card in advance, the reward could influence who chose to participate and how they responded.
Step 2 — Identify the bias: Advance knowledge of a prize attracts self-selected participants (people interested in video games / gift cards) and may encourage rushed or exaggerated answers, biasing the data.
Answer: Yes. Knowing about the reward beforehand could bias the data through self-selection and influenced responses; a reward given only after participation, with no prior knowledge, would be far less likely to affect the results.
For problems 23 through 27, a pair of studies was performed to measure the effectiveness of a new software program designed to help stroke patients regain their problem-solving skills. Patients were asked to use the software program twice a day, once in the morning and once in the evening. The studies observed 200 stroke patients recovering over a period of several weeks. The first study collected the data in Table 1.2.12. The second study collected the data in Table 1.2.13.
| Group | Showed improvement | No improvement | Deterioration |
|---|---|---|---|
| Used program | 142 | 43 | 15 |
| Did not use program | 72 | 110 | 18 |
| Group | Showed improvement | No improvement | Deterioration |
|---|---|---|---|
| Used program | 105 | 74 | 19 |
| Did not use program | 89 | 99 | 4 |
Problem 23. Given what you know, which study is correct?
Solution
Step 1 — Recall that independent samples can differ: The two studies followed different groups of 200 stroke patients, so different counts are expected from sampling variability alone.
Step 2 — Decide whether one is "correct": Neither study is inherently right or wrong. Both are legitimate observations of different samples; without more information we cannot label one correct.
Answer: Neither can be declared correct. Both are valid studies of different patient groups, and the differing results reflect sampling variability rather than one study being wrong.
Problem 24. The first study was performed by the company that designed the software program. The second study was performed by the American Medical Association. Which study is more reliable?
Solution
Step 1 — Look for a conflict of interest: The first study was run by the company that makes the software, giving it a financial stake in a favorable result. The second was run by the American Medical Association, an independent professional body.
Step 2 — Judge reliability: An independent organization with no financial interest in the outcome is less prone to bias, so its findings carry more credibility.
Answer: The second study (American Medical Association) is more reliable, because it was conducted by an independent party without a financial stake in the result, whereas the company-run study has a conflict of interest.
Problem 25. Both groups that performed the study concluded that the software works. Is this accurate?
Solution
Step 1 — Compare the two data sets: In the first study, 142 of 200 program users improved (71%) versus 72 of 200 non-users (36%). In the second study, 105 of 200 users improved (52.5%) versus 89 of 200 non-users (44.5%).
Step 2 — Assess the shared conclusion: Both studies show users improving at a higher rate than non-users, so the qualitative claim "the software is associated with improvement" is consistent across both, even though the size of the effect differs.
Answer: It is accurate to say both studies found a higher improvement rate among software users than non-users. However, the second (independent) study shows a much smaller advantage, so the strength of the effect is uncertain — the direction of the conclusion is consistent, but the magnitude is not.
Problem 26. The company takes the two studies as proof that their software causes mental improvement in stroke patients. Is this a fair statement?
Solution
Step 1 — Separate association from causation: The studies are observational: patients were grouped by whether they used the software, not randomly assigned. Observed improvement could stem from confounding factors rather than the software itself.
Step 2 — Apply the causation rule: A correlation between software use and improvement does not by itself prove the software causes improvement. Only a properly controlled, randomized experiment can establish causation.
Answer: No. The studies are observational and show only an association, not causation. Without a randomized controlled experiment, the company cannot fairly claim the software causes mental improvement.
Problem 27. Patients who used the software were also a part of an exercise program whereas patients who did not use the software were not. Does this change the validity of the conclusions from Problem 25?
Solution
Step 1 — Identify the new confounding variable: Program users were also in an exercise program while non-users were not. Exercise itself can aid stroke recovery, so it is confounded with software use.
Step 2 — Reassess the conclusion from 1.2.25: Because the two groups differ in more than just software use, any improvement could be due to the exercise rather than the software. The groups are no longer comparable.
Answer: Yes, it changes the validity. Exercise is a confounding variable that differs between the groups, so the observed improvement cannot be attributed to the software alone; the conclusions from Problem 25 are weakened.
Problem 28. Is a sample size of 1,000 a reliable measure for a population of 5,000?
Solution
Step 1 — Compute the sampling fraction: The sample is \(1{,}000\) out of a population of \(5{,}000\): $$\frac{1{,}000}{5{,}000} = 0.20 = 20%.$$
Step 2 — Judge reliability: A randomly chosen sample of 1,000 is a large absolute size and covers 20% of the population, which is more than adequate to estimate population characteristics reliably (provided it is randomly selected).
Answer: Yes. A sample of 1,000 (20% of the population of 5,000), if randomly selected, is large enough to be a reliable measure.
Problem 29. Is a sample of 500 volunteers a reliable measure for a population of 2,500?
Solution
Step 1 — Compute the sampling fraction: The sample is \(500\) out of a population of \(2{,}500\): $$\frac{500}{2{,}500} = 0.20 = 20%.$$
Step 2 — Account for self-selection: Twenty percent is a substantial fraction, but these are volunteers. Volunteer (self-selected) samples are prone to bias because volunteers may differ systematically from the general population.
Answer: The size (500, or 20% of 2,500) is adequate, but because the participants are self-selected volunteers, the sample may be biased and is therefore not necessarily reliable. A randomly selected sample of the same size would be preferable.
Problem 30. A question on a survey reads: "Do you prefer the delicious taste of Brand X or the taste of Brand Y?" Is this a fair question?
Solution
Step 1 — Read the wording critically: The question describes Brand X as having a "delicious taste" while describing Brand Y's taste neutrally. The adjective primes the respondent toward Brand X.
Step 2 — Identify the flaw: This is a leading question — its phrasing biases the answer rather than letting the respondent choose freely.
Answer: No, it is not a fair question. The word "delicious" applied only to Brand X makes it a leading (biased) question; a fair version would describe both brands neutrally, e.g., "Do you prefer the taste of Brand X or the taste of Brand Y?"
Problem 31. Is a sample size of two representative of a population of five?
Solution
Step 1 — Compute the sampling fraction: A sample of \(2\) from a population of \(5\): $$\frac{2}{5} = 0.40 = 40%.$$
Step 2 — Judge representativeness: Although 40% is a large fraction, the absolute sample size of two is far too small to capture the variation in the population; just one unusual individual would dominate the result.
Answer: No. A sample of size two is too small to be representative of a population of five, despite being 40% of it — there is not enough data to reflect the population's variability.
Problem 32. Is it possible for two experiments to be well run with similar sample sizes to get different data?
Solution
Step 1 — Recall the role of randomness: Even with careful design and equal sample sizes, two experiments draw different random samples from the population.
Step 2 — Conclude: Random sampling guarantees that two independent, well-run experiments will almost always produce somewhat different data. This is expected sampling variability, not a sign of error.
Answer: Yes. Two well-run experiments with similar sample sizes can easily produce different data because they sample different individuals; this is normal sampling variability.
For problems 33 through 42, identify the type of data that would be used to describe a response (quantitative discrete, quantitative continuous, or qualitative), and give an example of the data.
Problem 33. number of tickets sold to a concert
Solution
Step 1 — Decide qualitative vs. quantitative: Number of tickets sold is a count, so it is numerical — quantitative.
Step 2 — Decide discrete vs. continuous: Counts take only whole-number values (you cannot sell 412.5 tickets), so the data are quantitative discrete.
Answer: Quantitative discrete. Example: 412 tickets sold to a particular concert.
Problem 34. percent of body fat
Solution
Step 1 — Decide qualitative vs. quantitative: Percent of body fat is a measured numerical quantity, so it is quantitative.
Step 2 — Decide discrete vs. continuous: It is measured on a scale and can take any value in a range (including fractions), so the data are quantitative continuous.
Answer: Quantitative continuous. Example: a body-fat measurement of 18.6%.
Problem 35. favorite baseball team
Solution
Step 1 — Decide qualitative vs. quantitative: A favorite baseball team is a name or category, not a number, so it is qualitative (categorical).
Step 2 — Confirm: There is no meaningful arithmetic on team names, which confirms the data are qualitative.
Answer: Qualitative. Example: the San Francisco Giants.
Problem 36. time in line to buy groceries
Solution
Step 1 — Decide qualitative vs. quantitative: Time waiting in line is a measured numerical quantity, so it is quantitative.
Step 2 — Decide discrete vs. continuous: Time can take any value in a range (e.g., 4.3 minutes), so the data are quantitative continuous.
Answer: Quantitative continuous. Example: 4.3 minutes spent in line.
Problem 37. number of students enrolled at Evergreen Valley College
Solution
Step 1 — Decide qualitative vs. quantitative: Number of students enrolled is a count, so it is numerical — quantitative.
Step 2 — Decide discrete vs. continuous: Counts are whole numbers (you cannot enroll a fraction of a student), so the data are quantitative discrete.
Answer: Quantitative discrete. Example: 9,800 students enrolled.
Problem 38. most-watched television show
Solution
Step 1 — Decide qualitative vs. quantitative: A most-watched television show is a title/category, not a number, so it is qualitative (categorical).
Step 2 — Confirm: Show titles cannot be averaged or otherwise computed with, confirming the data are qualitative.
Answer: Qualitative. Example: the most-watched show being Jeopardy!
Problem 39. brand of toothpaste
Solution
Step 1 — Decide qualitative vs. quantitative: A brand of toothpaste is a name or category, not a number, so it is qualitative (categorical).
Step 2 — Confirm: Brand names support no arithmetic, confirming the data are qualitative.
Answer: Qualitative. Example: the brand Colgate.
Problem 40. distance to the closest movie theatre
Solution
Step 1 — Decide qualitative vs. quantitative: Distance is a measured numerical quantity, so it is quantitative.
Step 2 — Decide discrete vs. continuous: Distance can take any value in a range (e.g., 2.7 miles), so the data are quantitative continuous.
Answer: Quantitative continuous. Example: 2.7 miles to the closest movie theatre.
Problem 41. age of executives in Fortune 500 companies
Solution
Step 1 — Decide qualitative vs. quantitative: Age is a measured numerical quantity, so it is quantitative.
Step 2 — Decide discrete vs. continuous: Age is measured on a continuous scale (it can take any value, e.g., 52.4 years), so the data are quantitative continuous.
Answer: Quantitative continuous. Example: an executive aged 52.4 years.
Problem 42. number of competing computer spreadsheet software packages
Solution
Step 1 — Decide qualitative vs. quantitative: Number of competing software packages is a count, so it is numerical — quantitative.
Step 2 — Decide discrete vs. continuous: Counts take only whole-number values, so the data are quantitative discrete.
Answer: Quantitative discrete. Example: 5 competing spreadsheet packages.
For problems 43 and 44, a study was done to determine the age, number of times per week, and the duration (amount of time) of resident use of a local park in San Jose. The first house in the neighborhood around the park was selected randomly and then every 8th house in the neighborhood around the park was interviewed.
Problem 43. "Number of times per week" is what type of data?
a) qualitative
b) quantitative discrete
c) quantitative continuous
Solution
Step 1 — Classify the variable: "Number of times per week" a resident uses the park is a count of visits.
Step 2 — Decide the data type: A count takes only whole-number values, so it is numerical and discrete — quantitative discrete. It is not qualitative (it is a number) and not continuous (you cannot use the park 3.5 times in the counting sense).
Answer: b) quantitative discrete.
Problem 44. "Duration (amount of time)" is what type of data?
a) qualitative
b) quantitative discrete
c) quantitative continuous
Solution
Step 1 — Classify the variable: "Duration (amount of time)" measures how long each resident uses the park.
Step 2 — Decide the data type: Time is measured on a continuous scale and can take any value in a range (e.g., 1.75 hours), so it is quantitative continuous. It is not qualitative (it is a number) and not discrete (it is not a whole-number count).
Answer: c) quantitative continuous.
Problem 45. Airline companies are interested in the consistency of the number of babies on each flight, so that they have adequate safety equipment. Suppose an airline conducts a survey. Over Thanksgiving weekend, it surveys six flights from Boston to Salt Lake City to determine the number of babies on the flights. It determines the amount of safety equipment needed by the result of that study.
a) Using complete sentences, list three things wrong with the way the survey was conducted.
b) Using complete sentences, list three ways that you would improve the survey if it were to be repeated.
Solution
Step 1 — Examine how the survey was conducted: The airline samples only six flights, all on one route (Boston to Salt Lake City), all over a single Thanksgiving weekend, then uses that to set safety equipment for all flights.
Step 2 — (a) List three flaws: (1) The sample is far too small — six flights cannot represent the airline's entire schedule. (2) The timing is biased: Thanksgiving weekend is a peak family-travel period, so the number of babies will be unusually high and unrepresentative of typical flights. (3) The single route (Boston–Salt Lake City) is not representative of all routes, which differ in destination, length, and passenger makeup.
Step 3 — (b) List three improvements: (1) Greatly increase the sample size — survey many more flights. (2) Spread the survey across the whole year (and many days of the week) instead of one holiday weekend, so it reflects typical travel. (3) Sample a variety of routes (different origins, destinations, and flight lengths) rather than one city pair.
Answer: (a) Three flaws — sample too small (only 6 flights), biased timing (Thanksgiving overstates the number of babies), and a single unrepresentative route. (b) Three improvements — survey far more flights, spread the survey across the full year and all weekdays, and include many different routes.
Problem 46. Suppose you want to determine the mean number of students per statistics class in your state. Describe a possible sampling method in three to five complete sentences. Make the description detailed.
Solution
Step 1 — Define the goal and population: The goal is the mean number of students per statistics class across all statistics classes in your state; the population is every statistics class offered in the state.
Step 2 — Build a frame and choose a method: Obtain a list of all colleges and universities in the state that offer statistics, and from each obtain a roster of its statistics class sections. This list of sections is the sampling frame.
Step 3 — Draw the sample and measure: Use stratified random sampling — group the schools by type (e.g., community colleges vs. universities) so each type is represented, then randomly select several statistics sections from each group. Record the enrollment of each selected class and average those counts to estimate the statewide mean class size.
Answer: A reasonable method: list all statistics sections at all schools in the state, stratify by school type, randomly select sections within each stratum, record each selected class's enrollment, and average them to estimate the mean number of students per statistics class. (Answers will vary — any clearly described, representative sampling plan is acceptable.)
Problem 47. Suppose you want to determine the mean number of cans of soda drunk each month by students in their twenties at your school. Describe a possible sampling method in three to five complete sentences. Make the description detailed.
Solution
Step 1 — Define the goal and population: The goal is the mean number of cans of soda drunk per month by students in their twenties at your school; the population is all students at your school who are in their twenties.
Step 2 — Build a frame and choose a method: Use the registrar's enrollment list to identify students aged 20–29. This list is the sampling frame.
Step 3 — Draw the sample and measure: Use simple random sampling — assign each eligible student a number and randomly select a sizable subset. Ask each selected student how many cans of soda they drank in the past month, then average those counts to estimate the monthly mean.
Answer: A reasonable method: get the registrar's list of students in their twenties, randomly select a representative sample, ask each how many cans of soda they drink per month, and average the responses to estimate the mean. (Answers will vary — any clearly described, representative sampling plan is acceptable.)
Problem 48. List some practical difficulties involved in getting accurate results from a telephone survey.
Solution
Step 1 — Think about who can be reached by phone: Coverage is incomplete — not everyone has a phone, some have only cell phones (and numbers may be unlisted or change), so the call list may miss whole groups, causing undercoverage bias.
Step 2 — Think about who responds: Many people do not answer unknown numbers, screen calls, or hang up, producing high nonresponse; those who do answer may differ systematically (e.g., people at home during the day), biasing results.
Step 3 — Think about answer quality: Respondents may give hurried or untruthful answers, misunderstand questions, or be influenced by the interviewer's tone, and language barriers can distort responses.
Answer: Practical difficulties include undercoverage (people without phones or with unlisted/cell-only numbers), high nonresponse (unanswered or screened calls and hang-ups), nonresponse bias (those reachable/willing differ from those not), and inaccurate answers (rushed, untruthful, or misunderstood responses, plus interviewer or language effects). (Answers will vary.)
Problem 49. List some practical difficulties involved in getting accurate results from a mailed survey.
Solution
Step 1 — Think about delivery and coverage: You need current, correct addresses; out-of-date or missing addresses mean some people never receive the survey, causing undercoverage.
Step 2 — Think about who returns it: Mailed surveys have very low response rates — many are discarded unopened — and those who bother to return them often hold stronger opinions, producing nonresponse bias.
Step 3 — Think about answer quality and timing: There is no interviewer to clarify questions, so respondents may misread or skip items; you cannot verify who actually filled it out; and replies trickle in slowly, delaying results.
Answer: Practical difficulties include undercoverage from wrong or missing addresses, very low response rates with nonresponse bias (returners differ from non-returners), no way to clarify confusing questions or verify who answered, and slow, delayed returns. (Answers will vary.)
Problem 50. With your classmates, brainstorm some ways you could overcome these problems if you needed to conduct a phone or mail survey.
Solution
Step 1 — Identify the obstacles to phone/mail surveys: Phone and mail surveys suffer from low response rates, unanswered or screened calls, undeliverable mail, out-of-date contact lists, and self-selection (only motivated people respond). Naming the problems first tells us what to overcome.
Step 2 — Brainstorm countermeasures: Possible strategies include offering a small incentive for completion, sending advance notices and follow-up reminders, keeping the survey short, calling at varied times of day, providing multiple response channels (web link, prepaid return envelope), training callers to build rapport, and using up-to-date, broadly representative contact lists rather than a single source.
Answer: This is an open-ended discussion problem. A good answer names concrete tactics — incentives, follow-up reminders, short well-worded questions, varied call times, prepaid return postage, and current representative contact lists — that directly counter low response rates and self-selection bias.
Problem 51. An instructor takes a sample by gathering data on five randomly selected students from each Lake Tahoe Community College math class. The type of sampling used is
a) cluster sampling
b) stratified sampling
c) simple random sampling
d) convenience sampling
Solution
Step 1 — Identify the structure of the sampling: The instructor goes to each math class and selects five students at random from within every class. The classes act as naturally occurring groups.
Step 2 — Match to the definition: Selecting a random sample of individuals from within every group is stratified sampling (the strata are the math classes). Cluster sampling would instead pick whole classes and survey everyone in them; here only five per class are taken, from all classes.
Answer: b) stratified sampling
Problem 52. A study was done to determine the age, number of times per week, and the duration (amount of time) of residents using a local park in San Jose. The first house in the neighborhood around the park was selected randomly and then every eighth house in the neighborhood around the park was interviewed. The sampling method was:
a) simple random
b) systematic
c) stratified
d) cluster
Solution
Step 1 — Identify the selection rule: The first house is chosen at random, then every eighth house thereafter is interviewed — selection follows a fixed interval through an ordered list.
Step 2 — Match to the definition: Choosing a random starting point and then taking every \(k\)th element is systematic sampling.
Answer: b) systematic
Problem 53. Name the sampling method used in each of the following situations:
a) A person in the airport is handing out questionnaires to travelers asking them to evaluate the airport's service. The person does not ask travelers who are hurrying through the airport with their hands full of luggage but instead asks all travelers who are sitting near gates and not taking naps while they wait.
b) A teacher wants to know if her students are doing homework, so they randomly select rows two and five and then call on all students in row two and all students in row five to present the solutions to homework problems to the class.
c) The marketing manager for an electronics chain store wants information about the ages of its customers. Over the next two weeks, at each store location, 100 randomly selected customers are given questionnaires to fill out asking for information about age, as well as about other variables of interest.
d) The librarian at a public library wants to determine what proportion of the library users are children. The librarian has a tally sheet on which they mark whether books are checked out by an adult or a child. The librarian records this data for every fourth patron who checks out books.
e) A political party wants to know the reaction of voters to a debate between the candidates. The day after the debate, the party's polling staff calls 1,200 randomly selected phone numbers. If a registered voter answers the phone or is available to come to the phone, that registered voter is asked whom they intend to vote for and whether the debate changed their opinion of the candidates.
Solution
Step 1 — Classify part (a): The surveyor approaches only the travelers who are easy to reach (seated, awake, not rushing). Taking whoever is conveniently available is convenience sampling.
Step 2 — Classify part (b): Two rows are chosen at random and every student in those rows is called on. Selecting whole naturally occurring groups at random is cluster sampling.
Step 3 — Classify part (c): At each store, 100 customers are selected at random. Each store is a stratum and a random sample is drawn within every one, so this is stratified sampling.
Step 4 — Classify part (d): The librarian records data for every fourth patron. A fixed interval through the stream of patrons is systematic sampling.
Step 5 — Classify part (e): 1,200 phone numbers are chosen at random with no grouping or interval, so every voter has an equal chance — simple random sampling.
Answer: a) convenience; b) cluster; c) stratified; d) systematic; e) simple random.
Key Terms
qualitative (categorical) data — data that describe a category, recorded as words or letters (e.g., hair color, blood type).
quantitative (numerical) data — data that are numbers, from counting or measuring.
quantitative discrete data — numerical data from counting; only certain (usually whole-number) values.
quantitative continuous data — numerical data from measuring; may take any value in a range.
frequency — the count of how many times a value or category occurs.
relative frequency — a frequency expressed as a percent or proportion of the total.
pie chart — a circular graph whose wedges are sized in proportion to each category's percent; requires percentages summing to 100%.
bar graph — a graph whose bar lengths are proportional to each category's count or percent; bars may be vertical or horizontal.
Pareto chart — a bar graph with bars sorted from largest to smallest.
simple random sample — a sample in which every group of a given size is equally likely to be chosen.
stratified sample — divide the population into strata, then take a proportionate random sample from each.
cluster sample — randomly choose whole groups (clusters) and include every member of the chosen groups.
systematic sample — pick a random starting point, then select every \(k\)-th member of the list.
convenience sample — a non-random sample using whoever or whatever is readily available.
sampling with replacement — a chosen member is returned to the population and may be picked again.
sampling without replacement — a member may be chosen only once.
sampling error — error caused by the sampling process itself (e.g., a too-small sample).
nonsampling error — error from factors unrelated to sampling (e.g., a defective measuring device).
sampling bias — occurs when some population members are less likely to be chosen than others.
confounding — when the effects of several factors on a response cannot be separated.
associated / independent — two variables are associated if one gives information about the other, and independent if there is no evident relationship.
Appendix — Additional Examples and Practice
The following items were carried over from a second source text covering the same material. They are kept here as additional practice rather than folded into the main flow.
Is \(\mu\) a parameter or a statistic? What about \(\hat{p}\)?
Solution
\(\mu\) is a parameter because it refers to the average of the entire population. \(\hat{p}\) is a statistic because it is calculated from a sample.
A systematic sample is not the same as a simple random sample. Give an example of a sample that can come from a simple random sample but not from a systematic random sample.
Solution
Answers can vary. If we take a sample of size 3, it is possible to select players numbered 1, 2, and 3 in a simple random sample. Such a sample would be impossible from a systematic sample, because systematic sampling spaces selections evenly across the list. The "every group of the same size is equally likely" property of simple random samples does not hold for other types of random samples.
Figure 1.2.16 — Two panels comparing sampling methods: (top) simple random sampling selects 18 cases at random; (bottom) systematic random sampling selects every 7th individual.
Sometimes a variable is known to be associated with the quantity we want to estimate. In that case a stratified random sample may be best. Stratified sampling is a divide-and-conquer strategy: the population is split into strata so that similar cases are grouped together, and a sampling method (usually simple random sampling) selects a certain number or proportion within each stratum. In a baseball-salary example, the 30 teams could be the strata — some teams have far more money than others.
How and why should randomization be incorporated into a matched pairs design?
Solution
Randomization should decide, within each matched pair, which member receives which treatment. This guards against hidden bias: if the experimenter always assigned the treatment to (say) the first member of every pair, any difference between "first" and "second" members would get confounded with the treatment effect. Randomizing the assignment within each pair ensures that lurking differences are, on average, balanced across the two treatment groups, so the observed difference can be attributed to the treatment rather than to how subjects were ordered.
Figure 1.2.17 — Matched-pairs design: similar subjects are paired, then a coin flip decides which member of each pair receives which treatment.
This example examines the relationship between a county's population change from 2010 to 2017 and its median household income, shown as a scatterplot. Are these variables associated?
Solution
The larger the median household income for a county, the higher its observed population growth. The trend is not true for every single county, but it is clearly visible in the plot. Because there is some relationship between the variables, they are associated — here, positively associated, since higher income tends to come with higher population growth. (By contrast, when counties with more units in multi-unit structures tend to have lower homeownership, those variables are negatively associated.)
If two variables show no evident relationship, they are independent.
Association Does Not Imply Causation
Labeling variables as explanatory and response does not guarantee the relationship is actually causal, even when an association is identified. We use these labels only to track which variable we suspect affects the other.
In many cases the relationship is complex or unknown. It may be unclear whether variable \(A\) explains variable \(B\) or vice versa. For example, a protein called REST is much depleted in people with Alzheimer's disease. This raises hopes for a treatment, but it is still unknown whether the lack of the protein causes brain deterioration, whether brain deterioration causes the depletion, or whether some third variable causes both. We simply do not know whether the lack of the protein is an explanatory variable or a response variable — perhaps it is both.
For the baseball example, briefly explain how to select a stratified random sample of size \(n = 120\).
Solution
Let each team serve as a stratum. Take a simple random sample of 4 players from each of the 30 teams, yielding \(4 \times 30 = 120\) players. Stratified sampling differs from simple random sampling: this approach makes it impossible for an entire single team (say, the Yankees) to be selected as the whole sample.
Association ≠ Causation
In general, association does not imply causation, and causation can only be inferred from a randomized experiment.
We can easily access ratings for products, sellers, and companies online. These ratings come only from people who go out of their way to provide one. If 50% of online reviews for a product are negative, does this mean that 50% of buyers are dissatisfied?
Solution
No. Reviewers are self-selected — people with strong opinions (especially negative ones) are far more likely to bother writing a review than satisfied customers who quietly move on. So the 50% figure describes the reviewers, not all buyers. The reviews are a biased, non-representative sample of the buyer population.
Watch Out For Bias
Undercoverage bias, non-response bias, and response bias can all exist within a random sample. Always determine how a sample was chosen, ask what proportion of people failed to respond, and critically examine the wording of the questions.
When there is no bias, increasing the sample size tends to increase the precision and reliability of the estimate. When a sample is biased, it may be impossible to extract helpful information from the data, even if the sample is very large.
A researcher mails questionnaires to 50 randomly selected households asking whether they support adding a traffic light in their neighborhood. Only 20% of the questionnaires are returned, so she mails questionnaires to 50 more randomly selected households in the same neighborhood. Comment on the usefulness of this approach.
Solution
Mailing more questionnaires does not fix the core problem: non-response bias. The 20% who returned the first round are likely the people with the strongest opinions (for or against the traffic light), so their responses may not represent the neighborhood. Sending 50 more questionnaires will probably yield the same low return rate and the same self-selected, opinionated respondents. To improve usefulness, she should focus on raising the response rate — follow-up reminders, in-person or phone contact — rather than simply mailing out more surveys.