Introduction to Statistics · Chapter 1 · Sampling and Data
The first question a statistician asks is not "what is the answer?" but "what kind of data is this?" — because the answer decides everything you are allowed to do with it.
bookSHelf · Introduction to Statistics · §1.2 · a self-paced section
Learning objectives — by the end of this section you will be able to
§1.2.1 — the first question a statistician asks
You cannot average a list of hair colors, and you cannot make a pie chart out of measured weights. Getting the type right up front saves you from an analysis that looks fine and means nothing.
Almost all data fall into one of two big families: qualitative and quantitative.
§1.2.1 — the family of categories and labels
Definition 1.2.1 — Qualitative data
Qualitative data are the result of categorizing or describing attributes of a population. They are generally described by words or letters rather than numbers.
Hair color
Blood type
Ethnic group
Hair color, blood type, ethnic group, the car a person drives — these are all qualitative. It does not make sense to find an average hair color or an average blood type.
Context Pause — the same data, two lives
Quiz scores are recorded all term as numbers; at the end of the term they are reported as A, B, C, D, or F. The same underlying reality can be quantitative when gathered and qualitative when published — always ask which version you are holding.
A variable's type is not a fixed property of the thing being measured. It depends on how you record it.
§1.2.1 — the family of numbers
Definition 1.2.2 — Quantitative data
Quantitative data are always numbers. They are the result of counting or measuring attributes of a population.
Amount of money
Pulse rate
Weight
Amount of money, pulse rate, weight, the number of people in your town — all quantitative. They split further into two types, depending on whether you counted or measured.
Insight Note — the one question that decides
Counting gives you whole things. Measuring gives you a reading on a scale.
"How many?" is discrete. "How much?" is continuous. You can have 3 phone calls, never 3.4. But a call lasting 3.4 minutes is perfectly ordinary.
§1.2.1 — counting lands on certain values
Definition 1.2.3 — Quantitative discrete data
All data that are the result of counting are called quantitative discrete data. These data take on only certain numerical values.
Definition 1.2.3: counting gives whole numbers — 0, 1, 2, 3 — never 2.4.
If you count phone calls per day, you might get 0, 1, 2, or 3 — never 2.4. The gaps between possible values are real.
§1.2.1 — measuring can land anywhere
Definition 1.2.4 — Quantitative continuous data
Data that are not only made up of counting numbers, but that may include fractions, decimals, or irrational numbers, are called quantitative continuous data.
Definition 1.2.4: continuous data come from measurements — lengths, weights, times.
A list of call lengths in minutes — 2.4, 7.5, 11.0 — is continuous. With a more precise timer you could write 2.437 or 7.512. The values can be refined without limit.
The whole classification in one picture
Figure 1.2.1: data-types classification tree — qualitative vs. quantitative, and quantitative split into discrete and continuous.
Every data set you meet from here on lands in exactly one of these three leaves. The tree is your first diagnostic tool.
Your turn — commit to an answer before the reveal
Try It Now 1.2.1 — counting the machines
The data are the number of machines in a gym. You sample five gyms. One gym has 12 machines, one has 15, one has ten, one has 22, and the other has 20. What type of data is this?
Quantitative discrete.
For each gym we counted the machines. A gym has 12 machines or 13 machines — there is no such thing as 12.5 machines, so only whole numbers can occur.
Worked example — counting books
Example 1.2.1 — Data sample of quantitative discrete data
The data are the number of books students carry in their backpacks. You sample five students. Two students carry three books, one carries four, one carries two, and one carries one book. What type of data is this?
Quantitative discrete.
For each student we wrote down how many books — a count. A backpack holds 3 books or 4 books, never 3.4 books. The values are whole numbers with real gaps between them.
Your turn — commit to an answer before the reveal
Try It Now 1.2.2 — measuring the lawns
The data are the areas of lawns in square feet. You sample five houses. The areas of the lawns are 144, 160, 190, 180, and 210 square feet. What type of data is this?
Quantitative continuous.
Area is obtained by measuring, not by counting. The listed areas happen to be whole numbers, but a lawn could just as easily measure 160.5 or 160.47 square feet — landing on whole numbers here is rounding, not a restriction.
Worked example — weighing backpacks
Example 1.2.2 — Data sample of quantitative continuous data
The data are the weights of backpacks with books in them. You sample the same five students. The weights (in pounds) are 6.2, 7, 6.8, 9.1, 4.3. What type of data is this?
Quantitative continuous.
Each backpack was put on a scale — a measurement, not a count. Decimals appear, and with a more precise scale we could write 6.24 or 6.238. Two backpacks carrying three books each can still weigh different amounts, because the books themselves differ.
Your turn — all three types in one purchase order
Try It Now 1.2.3 — a purchase manager's order
Grant Halloway must describe this order to his accountant. Name data sets that are quantitative discrete, quantitative continuous, and qualitative.
Worked example — all three types in one trip
Example 1.2.3 — A shopping trip with all three types
You buy three cans of soup (19 oz tomato bisque, 14.1 oz lentil, 19 oz Italian wedding), two packages of nuts (walnuts, peanuts), four kinds of vegetable (broccoli, cauliflower, spinach, carrots), and two desserts (16 oz pistachio ice cream, 32 oz chocolate chip cookies). Name data sets that are quantitative discrete, quantitative continuous, and qualitative.
Your turn — commit to an answer before the reveal
Try It Now 1.2.4 — walking the block
Sarah Whitfield walks her block and records the colors of five houses. The colors are white, yellow, white, red, and white. What type of data is this?
Qualitative (categorical).
Each observation is a color name — a category, not a measurement or a count. Notice you can count how many houses are white (three); that count is a summary of the qualitative data, not the data itself.
Worked example — colors of backpacks
Example 1.2.4 — Data sample of qualitative data
The data are the colors of backpacks. You sample five students. One has a red backpack, two have black, one has green, and one has gray. What type of data is this?
Qualitative (categorical).
For each student we wrote down a color name, not a number. Colors describe an attribute — they place each backpack in a category. "The average of red and green" is meaningless.
Your turn — commit to an answer before the reveal
Try It Now 1.2.5 — reading the phrasing
Determine the correct data type for the number of cars in a parking lot. If it is quantitative, say whether it is discrete or continuous.
Quantitative discrete.
“The number of” is the classic signal of a count. A lot holds 40 cars or 41 cars, never 40.7.
Worked example — sorting a mixed list
Determine the correct data type for each item. Hint: data that are discrete often start with "the number of."
Items
Answers
Discrete: a, d, g — counts of shoes, classes, correct answers.
Continuous: c, f, h — distance, weight, IQ (measured on a continuous scale).
Qualitative: b, e — car make, calculator make.
Your turn — reading the data type off a histogram
Try It Now 1.2.6 — credit hours at State University
The registrar keeps records of the number of credit hours students complete each semester, summarized in the histogram. The class boundaries are 10 to less than 13, 13 to less than 16, 16 to less than 19, 19 to less than 22, and 22 to less than 25. What type of data does this graph show?
Figure 1.2.2: credit hours completed per student, grouped into five intervals.
Quantitative discrete.
Credit hours sit on the horizontal axis, so the data are numbers, not categories — and they are counted in whole units. The bars are grouped into intervals to make the picture readable; the grouping does not turn the counts into measurements.
Worked example — reading the data type off a graph
Example 1.2.6 — Reading the data type off a graph
A professor collects information about the classification of her students as first-year, sophomore, junior, or senior. The data are summarized in a pie chart. What type of data does this graph show?
Figure 1.2.3: pie chart of student classification — first-year, sophomore, junior, senior.
Qualitative (categorical).
Each wedge is a class standing — a category name. The percentages describe how many students fall in each category; they summarize the data, they are not the data themselves.
Your turn — commit to an answer before the reveal
Try It Now 1.2.7 — variable and parameter
Priya Raman studies every apprentice enrolled in a statewide electrician programme and records, for each one, the number of hours logged before certification. Is that variable numerical (quantitative) or categorical (qualitative)? What is the parameter of interest?
Numerical — and the parameter is the population mean.
Hours are measured on a continuous scale, so the variable is numerical. A parameter describes the whole population: here that is the mean hours to certification across every apprentice in the programme, written μ.
§1.2.2 — displaying what you cannot average
Qualitative data need their own kind of display. Three charts do the job — and each one answers a different question.
A pie chart promises that every slice is a piece of one indivisible pie. A bar graph makes no such claim.
§1.2.2 — categories as wedges of one whole
Definition 1.2.5 — Pie chart
In a pie chart, categories of data are represented by wedges in a circle, and the wedges are proportional in size to the percent of individuals in each category.
Definition 1.2.5: each wedge is a piece of one indivisible whole.
A pie chart works only when every individual belongs to exactly one category and all categories are shown. Overlapping categories or missing data break the promise.
Insight Note — the fundamental difference
A pie is a whole. A bar is a ruler.
A pie chart promises that every slice is a piece of one indivisible pie — each individual belongs to exactly one wedge. Overlapping categories break that promise. Bars make no such claim; each bar is just a measurement against the same ruler.
§1.2.2 — each category measured against one ruler
Definition 1.2.6 — Bar graph
In a bar graph, the length of the bar for each category is proportional to the number or percent of individuals in that category. Bars may be vertical or horizontal.
Definition 1.2.6: each bar is measured independently against the same scale.
A bar graph does not require the categories to add to 100%. That makes it the honest choice when categories overlap or data are missing.
§1.2.2 — bars sorted by size
Definition 1.2.7 — Pareto chart
A Pareto chart consists of bars that are sorted into order by category size, from largest to smallest.
Definition 1.2.7: the same bars, sorted so the biggest categories stand out.
Sorting by size does not change the data — it changes what you notice first. The Pareto chart answers "which categories matter most?" at a glance.
§1.2.2 — the pies or the bars: which makes the comparison clearer?
Figure 1.2.4: De Anza, as one pie.
Figure 1.2.5: Foothill, as one pie.
Figure 1.2.6: both colleges, side by side.
Each display answers a different question. A single pie answers “how is this college split?” The bar graph answers “how do the two compare?” — and it shows sizes while hiding percentages, exactly the trade-off the pies make in reverse.
Pies vs. bars — which tells the clearer story?
| Status | De Anza # | De Anza % | Foothill # | Foothill % |
|---|---|---|---|---|
| Full-time | 9,200 | 40.9% | 4,059 | 28.6% |
| Part-time | 13,296 | 59.1% | 10,124 | 71.4% |
| Total | 22,496 | 100% | 14,183 | 100% |
Table 1.2.1: full-time and part-time enrollment at De Anza College and Foothill College.
A single pie answers "how is this college split?" The bar graph answers "how do the two compare?" — and reveals that De Anza is the bigger school, which the percentages alone hide.
Your turn — commit to an answer before the reveal
Try It Now 1.2.8 — reading Table 1.2.1 two ways
Using Table 1.2.1, explain why comparing the percent of part-time students at the two colleges tells a different story than comparing the number of part-time students. Which comparison would you use to argue that Foothill serves a mostly part-time student body?
Use the percentages — 71.4%.
By raw count De Anza has more part-time students (13,296 to 10,124). By share, Foothill is far more part-time (71.4% against 59.1%). The colleges have different totals, so counts are pulled around by the size of the school; percentages remove that effect and describe the make-up of each student body.
§1.2.3 — when the wedges cannot add up
Sometimes percentages add to more than 100% — or less. When categories overlap, a pie chart is not just unhelpful, it is dishonest.
A student can be full-time, under 25, and intending to transfer — one person counted in three categories.
When one person counts in multiple categories
| Characteristic | % |
|---|---|
| Full-time | 40.9% |
| Intend to transfer | 48.6% |
| Under age 25 | 61.0% |
| TOTAL | 150.5% |
Table 1.2.2: categories overlap, so percentages sum to more than 100%.
Figure 1.2.7: a bar graph handles overlapping categories honestly.
A pie chart cannot represent overlapping categories. A bar graph can — each bar is measured independently against the same 0–100% scale.
Your turn — commit to an answer before the reveal
Try It Now 1.2.9 — would a pie chart be honest?
Katherine Bruce runs a campus survey and reports that 55% of students work at least part-time, 62% commute more than 20 minutes, and 30% are parents. She wants to put all three numbers on one pie chart. Would that be an honest display? Explain, and say what she should use instead.
No — use a bar graph.
55%+62%+30%=147%, well over 100%, because the categories overlap: one student can work part-time, commute 25 minutes, and be a parent. A pie chart requires each individual to land in exactly one wedge. A bar graph measures each bar independently against the same 0–100% scale.
§1.2.4 — what happens when a category is left out
Nearly one student in ten is invisible in the first graph. Omitting a category does not make the people in it disappear — it makes them invisible to the reader.
"Other/Unknown" is bigger than Native American and Pacific Islander combined, and then some.
§1.2.4 — the frequencies do not add up to the total
| Ethnicity | Frequency | Percent |
|---|---|---|
| Asian | 8,794 | 36.1% |
| Black | 1,412 | 5.8% |
| Filipino | 1,298 | 5.3% |
| Hispanic/Latino | 4,180 | 17.1% |
| Native American | 146 | 0.6% |
| Pacific Islander | 236 | 1.0% |
| White | 5,978 | 24.5% |
| TOTAL | 22,044 of 24,382 | 90.4% |
Table 1.2.3: ethnicity of De Anza students, most recent fall term, with “Other/Unknown” omitted.
9.6% of students are not on this table
The omitted “Other/Unknown” category holds students who did not feel they fit the listed categories, or who declined to respond. The frequencies stop 2,338 short of the total enrollment.
When the categories do not account for everyone, build a bar graph and not a pie chart — a pie would silently rescale the listed groups so they appear to be the whole college.
Three bar views of the same data — each tells a different story
Fig 1.2.8: Other/Unknown omitted.
Fig 1.2.9: Other/Unknown included.
Fig 1.2.10: Pareto chart, sorted.
The omitted category (9.6%) is larger than Native American (0.6%) and Pacific Islander (1.0%) combined. A Pareto chart makes the ranking obvious at a glance.
Same data, same chart type — different arrangement
Figure 1.2.11: wedges in alphabetical order.
Figure 1.2.12: wedges sorted by size — the more informative arrangement.
The data are identical. Only the order changed — and the second chart is far easier to read. Sorting by size is a design choice, not a data change.
Your turn — commit to an answer before the reveal
Try It Now 1.2.10 — what is wrong with the pie?
Club president Camila Reyes reports the majors of her members as Business 40%, Nursing 25%, Engineering 15%. The club has 200 members but only 160 are accounted for in those three categories. What is wrong with drawing a pie chart of the three reported percentages, and how would you fix the display?
The three wedges cover only 80% of the club.
40%+25%+15%=80%, and 200−160=40 members — the same 20% gap. A pie chart claims its wedges fill the circle, so drawing only these three would rescale them and hide 40 people. Either add an “Other/Undeclared” wedge of 20%, or use a bar graph, which never promises the categories are exhaustive.
§1.2.5 — the practical heart of statistics
Gathering information about an entire population often costs too much or is impossible. Instead, we use a sample. A sample should have the same characteristics as the population it represents.
How do you pick a subset that speaks for the whole?
§1.2.5 — the gold standard
Definition 1.2.8 — Simple random sample
In a simple random sample, any group of n individuals is equally likely to be chosen as any other group of n individuals. Each sample of the same size has an equal chance of being selected.
Definition 1.2.8: every individual had an equal chance of being chosen.
Lisa wants a four-person study group from her class of 31. She puts all 31 names in a hat, shakes it, and picks three. That is a simple random sample — the simplest version of the gold standard.
Try it in rāSHio — no graphing calculator needed
Open rāSHio and choose File → Random Numbers… Set the range to 0 through 30 — the two-digit IDs in the class roster — and ask for 3 numbers. Check No repeats so no ID is drawn twice, then click Generate.
Figure 1.2.13: drawing Lisa's three sample IDs in rāSHio.
§1.2.5 — a little from every group
Definition 1.2.9 — Stratified sample
To choose a stratified sample, divide the population into groups called strata and then take a proportionate number from each stratum.
Definition 1.2.9: split into strata, draw from every one.
Example: stratify your college by department, then take a simple random sample from each department. Every department is represented.
§1.2.5 — whole groups at random
Definition 1.2.10 — Cluster sample
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from the selected clusters are in the sample.
Definition 1.2.10: whole groups chosen at random, every member taken.
Example: randomly pick four departments from your college and survey every faculty member in those four. Entire departments are left out — that is what clustering does.
Insight Note — the difference in one sentence
Strata slice, clusters scoop.
Stratifying takes a little from every group, so every group is represented. Clustering takes everything from a few groups, so entire groups are left out. Both are random — they just randomize different things.
§1.2.5 — every kth item
Definition 1.2.11 — Systematic sample
To choose a systematic sample, randomly select a starting point and take every kth piece of data from a listing of the population.
Definition 1.2.11: random start, then every kth item.
Example: a phone book of 20,000 listings, need 400 names. Pick a random start between 1 and 50, then take every 50th name. Simple and fast — but watch for hidden patterns in the list.
§1.2.5 — the non-random method
Definition 1.2.12 — Convenience sample
A type of sampling that is non-random is convenience sampling, which involves using results that are readily available.
Definition 1.2.12: only the readily available members are taken.
Example: a software store interviews customers who happen to be browsing. The results may be very good in some cases and highly biased in others. Convenience sampling is the most common — and the most dangerous.
§1.2.5 — can the same person be chosen twice?
Definition 1.2.13 — Sampling with replacement
Once a member is picked, that member goes back into the population and may be chosen more than once.
Definition 1.2.14 — Sampling without replacement
A member of the population may be chosen only once.
Definitions 1.2.13–1.2.14: with replacement (top) and without (bottom).
In practice, most surveys sample without replacement. When the population is large and the sample is small, the two are nearly equivalent — the chance of picking the same person twice is very low either way.
Context Pause — a fact that feels wrong
A sample of 1,000 can serve a population of 100,000 or 100,000,000 equally well.
In Confidence Intervals you will meet formulas that set sample size from the precision you want — not from the size of the population. It feels wrong, and it is true.
§1.2.5 — two kinds of error, one fixable
Definition 1.2.15 — Sampling error
An error caused by the actual process of sampling — for example, the sample may not be large enough.
Definition 1.2.16 — Nonsampling error
An error caused by factors not related to the sampling process — for example, a defective counting device.
Definitions 1.2.15–1.2.16: sampling error scatters; nonsampling error shifts everything one way.
Sampling error shrinks with a larger sample. Nonsampling error does not — a biased scale or a flawed question stays wrong no matter how many people you ask.
§1.2.5 — the most dangerous kind of error
Definition 1.2.17 — Sampling bias
A sampling bias is created when a sample is collected from a population and some members of the population are not as likely to be chosen as others.
Definition 1.2.17: one group has no path into the sample.
Unlike sampling error, more data will not fix bias. A biased method run at ten times the scale produces a bigger, more confident wrong answer.
Context Pause — why we hand the choice to chance
Left to ourselves, we pick the convenient, the nearby, the willing — and those people are not a cross-section of anybody. Handing the choice to chance removes the researcher's thumb from the scale, which is exactly what lets us generalize from a few hundred people to a few million.
Your turn — draw four different samples from the same 60 scores
Table 1.2.5 holds six sets of quiz scores. Every draw below is already done for you — read the scores straight off the table, then compare the four samples.
| #1 | #2 | #3 | #4 | #5 | #6 |
|---|---|---|---|---|---|
| 5 | 7 | 10 | 9 | 8 | 3 |
| 10 | 5 | 9 | 8 | 7 | 6 |
| 9 | 10 | 8 | 6 | 7 | 9 |
| 9 | 10 | 10 | 9 | 8 | 9 |
| 7 | 8 | 9 | 5 | 7 | 4 |
| 9 | 9 | 9 | 10 | 8 | 7 |
| 7 | 7 | 10 | 9 | 8 | 8 |
| 8 | 8 | 9 | 10 | 8 | 8 |
| 9 | 7 | 8 | 7 | 7 | 8 |
| 8 | 8 | 10 | 9 | 8 | 7 |
Table 1.2.5: six sets of quiz scores, ten rows.
The four draws
1 · Stratified by column — rows 2, 5, 9 from #1; 1, 4, 8 from #2; 3, 3, 7 from #3; 2, 6, 10 from #4; 1, 5, 9 from #5; 4, 7, 8 from #6.
2 · Cluster — take whole columns #2 and #5.
3 · Simple random — number the scores 1–60 down the columns; take 3, 7, 11, 14, 19, 22, 27, 31, 36, 40, 44, 49, 53, 57, 60.
4 · Systematic — random start 4, then every tenth score, wrapping past 60.
18, 20, 15, 12 — and none of them match a classmate's.
Stratified = 3×6=18 scores, every column represented. Cluster = 2×10=20 scores, four columns entirely absent. Simple random = 15 scores, columns over-represented purely by chance. Systematic = 12 scores — and with six columns of ten, a step of ten lands in a predictable pattern, a reminder that a systematic sample can sync up with hidden structure in the list.
Worked example — name that method
A study determines the average tuition San Jose State undergraduates pay. What type of sampling in each case?
Scenarios
Answers
1. Stratified — split by class, draw from each.
2. Systematic — random start, fixed step.
3. Simple random — every group equally likely.
4. Cluster — whole groups chosen, everyone inside taken.
5. Convenience — whoever happens to walk past.
Your turn — commit to an answer before the reveal
Try It Now 1.2.12 — name the sampling method
Principal Diane Kessler polls 50 first-year students, 50 sophomores, 50 juniors, and 50 seniors regarding policy changes for after-school activities. She wants every class year represented. Determine the type of sampling used.
Stratified, with class standing as the strata.
The four class standings divide the student body into non-overlapping groups, and a fixed number is taken from every one. Had she randomly picked two of the four classes and polled everyone in them, it would have been a cluster sample instead.
Worked example — six more studies to name
Determine the type of sampling used — simple random, stratified, systematic, cluster, or convenience.
Scenarios
Answers
a. Stratified — age brackets are the groups, some from each.
b. Cluster — five whole companies, everyone inside taken.
c. Stratified — two groups, 50 from each.
d. Systematic — a fixed step down an ordered list.
e. Simple random — random numbers, no grouping.
f. Convenience — the most readily available people.
§1.2.6 — the most important skill in this course
A study can have a perfect sampling method and still produce worthless conclusions. The question is not just "was it random?" but "was it representative?"
A bigger self-selected sample is a worse one, not a better one.
In class — are these samples representative?
Collaborative Exercise — determine whether each sample is representative
As a class, determine whether or not the following samples are representative. If they are not, name the kind of bias present.
Your turn — commit to an answer before the reveal
Try It Now 1.2.13 — name three problems
Héctor Ramos, a spokesperson for a shampoo company, presents a study his own company funded: 4,000 volunteers recruited from the company mailing list, of whom 92% saw improvement. Name at least three items from the list that his study trips over, and say whether the large sample size fixes any of them.
Self-funded, self-selected, and confounded — and 4,000 fixes none of it.
The company has a stake in the answer. Volunteers from its own mailing list are both self-selected and a convenience sample. “Saw improvement” is self-reported with no control group, so any change could be confounded with season or expectation. Sample size fixes sampling error, not sampling bias: a biased method run on 4,000 people gives a more precise estimate of the wrong quantity.
Your turn — commit to an answer before the reveal
Try It Now 1.2.14 — what is left for chance to decide?
In a matched pairs design, what is still left for chance to decide — and what goes wrong if the researcher decides it instead?
Randomize the assignment within each pair.
Pairing removes the person-to-person differences, but something still has to decide which member of each pair gets which treatment, or which treatment comes first. If the researcher decides, that choice can track something else — the healthier twin gets the new drug, the well-rested trial goes first. Flipping a coin inside each pair breaks any link between the treatment and hidden characteristics, including order effects such as practice or fatigue.
§1.2.7 — data vary. That is the point.
If every can of soda held exactly 12 ounces, there would be no need for statistics. Variation is not a mistake — it is the reason the field exists.
Two honest samples from the same population will give different results. That is expected, not an error.
Your turn — ordinary variation, or a broken machine?
Try It Now 1.2.15 — eight 16-ounce cans, measured
Quality inspector Lucía Herrera reviews eight measured cans (in ounces):
15.8 16.1 15.2 14.8 15.8 15.9 16.0 15.5
Explain in one or two sentences why she would not conclude that her filling machine is broken. What would make you suspect it really was?
Scatter on both sides of the target is normal. A pattern is not.
The values run from 14.8 to 16.1, landing above and below 16 by small amounts — the fingerprint of ordinary variation in both the filling and the measuring. A broken machine would show every can consistently under 16 (a systematic shortfall), or a spread so wide that some held 12 ounces and others 19.
Try it in rāSHio — putting a number on the spread
Open rāSHio, choose File → Delimited List… and paste the eight can amounts (e.g. 11.9, 12.1, 11.8, 12.0, 12.2, 11.9, 12.1, 12.0). Then choose Stats → Summary Statistics to read off the mean and standard deviation — the standard deviation is the number that measures how much the values spread out from the center.
Figure 1.2.14: Stats → Summary Statistics in rāSHio.
§1.2.8 — samples vary, and that is okay
If you and a classmate each take a simple random sample of the same size from the same population, your two sample means will almost certainly differ. Neither of you made a mistake.
The question is not whether the samples differ. It is whether the difference is small enough to ignore.
Your turn — commit to an answer before the reveal
Try It Now 1.2.16 — did somebody miscount?
Doreen's 500 students average 6.8 hours of sleep; Jung's 500 students average 7.1 hours. A classmate, Sam Whitlock, says one of them must have made a mistake. Respond to Sam, and explain what would happen to the gap if each of them sampled 5,000 students instead.
Neither made a mistake — that is sampling variability.
Doreen and Jung surveyed different students, and two different random samples from the same population almost never produce the same average. The gap is 0.3 hours — about 18 minutes — on a quantity that varies by hours from student to student. With 5,000 each, both means would sit closer to the population average and therefore closer to each other, but they would still not be identical.
§1.2.9 — bigger is better, but not in the way you think
A larger sample reduces sampling error — but it does nothing for bias. A biased sample of 10,000 people is just a very confident wrong answer.
Sample size determines precision. Sampling method determines accuracy. Never confuse the two.
Your turn — commit to an answer before the reveal
Try It Now 1.2.17 — is this sample representative?
Station manager Rosa Villalobos wants to know whether her 20,000 listeners prefer more music or more talk. She surveys the first 200 people she meets at one of the station's music concerts: 176 say more music, 24 say more talk. Is this sample representative of the whole fan base?
No — the venue picked the answer.
Attending a music concert is evidence that you like music, so listeners who prefer talk shows had almost no chance of being surveyed. 88% preferring music is exactly what a music-concert crowd would say: the result describes concertgoers, not the fan base. Rosa needs a random sample of the whole listener list.
In groups — see variation with your own hands
Collaborative Exercise — two experiments, 20 rolls each
Divide into groups of two, three, or four. Your instructor will give each group one six-sided die. Roll it 20 times and record each face value. Then roll it another 20 times and record those values separately. Compare the two experiments — are the means the same? The frequency of each face?
Two honest rolls of the same die, same number of rolls, different results. That is sampling variation — and it is the reason we need probability to tell us when a difference is meaningful.
Record your rolls — one table per experiment
| Face on die | Frequency |
|---|---|
| 1 | |
| 2 | |
| 3 | |
| 4 | |
| 5 | |
| 6 |
Table 1.2.6: first experiment (20 rolls).
| Face on die | Frequency |
|---|---|
| 1 | |
| 2 | |
| 3 | |
| 4 | |
| 5 | |
| 6 |
Table 1.2.7: second experiment (20 rolls).
Which experiment had the correct results? They both did. The job of the statistician is to see through the variability and draw appropriate conclusions.
Try it in rāSHio — count the faces
Once both experiments are rolled, type your 40 face values into a column in rāSHio. Then choose Graph → Frequency Table with Discrete values checked to count how many times each face appeared. Compare the two experiments' frequency tables — the variation between them is sampling variation in action.
Figure 1.2.15: counting each die face in rāSHio.
Worked example — 10,000 part-time students, three attempts to describe them
Quinn Marsden wants the average amount a part-time student at ABC College spends on books.
Sample 1 — convenience
Ten students from a first-term organic chemistry class, many also in calculus.
$128 $87 $173 $116 $130 $204 $147 $189 $93 $153
Sample 2 — systematic
Every fifth senior citizen on a list of those taking P.E. classes.
$50 $40 $36 $15 $50 $100 $40 $53 $22 $22
Sample 3 — stratified by discipline
One student drawn at random from each of ten disciplines.
$180 $50 $150 $85 $260 $75 $180 $200 $200 $150
The first two are biased; the third is not.
Sample 1 is science students buying expensive course books — they pay more than the average part-time student. Sample 2 is senior citizens taking courses for interest — they pay far less. In both, not every student had a chance to be chosen, so neither result describes the population. Sample 3 represents every discipline and draws at random within each, so no group is systematically favored — though a larger sample would still be better. Note the asymmetry: with a biased technique, even a large sample risks not being representative.
§1.2.10 — after you have chosen it
Even a perfectly drawn sample can fail. The people you cannot reach, and the people who volunteer, both break the randomness that made the sample trustworthy in the first place.
A flawless draw can still return a non-random sample.
§1.2.10 — the two post-sampling failures
Definition 1.2.18 — Nonresponse bias
The people who do not respond are systematically different from those who do — and the sample no longer represents the population.
Definition 1.2.19 — Voluntary response bias
Voluntary response is nonresponse bias with the dial turned to maximum: nobody was drawn — people chose themselves. The people with the strongest opinions are the most likely to respond.
Definitions 1.2.18–1.2.19: nonresponse and voluntary response bias.
The practical defence is unglamorous: report your response rate. A study that says "72% of those contacted responded" is far more trustworthy than one that does not say.
Context Pause — the counterintuitive truth
With sampling error, more data helps. With bias, more data makes the problem worse.
A self-selected poll that gets 10,000 responses is not more accurate than one that gets 100. It is just a more confident picture of the people who care enough to click — which is not the same as the population.
Your turn — name the failure, and say who is over-represented
For each situation, say whether the main problem is nonresponse bias, voluntary response bias, or neither.
Situations
Answers
a. Nonresponse bias. The draw was random, but only 18% replied — skewing toward alumni with a stronger attachment and more time. Recent graduates and the very busy are under-represented.
b. Voluntary response bias. Nobody was sampled. Viewers who disliked the show enough to stop watching never saw the pop-up, which quietly removes the harshest ratings.
c. Neither, or very little. A 95.5% response rate with deliberate callbacks is about as good as field work gets — persistence, not a larger initial sample.
d. Voluntary response bias. Angry callers dial at far higher rates, so 78% measures intensity of feeling among motivated listeners, not opinion among listeners.
The headline result of §1.2
Sample size buys precision. Only the sampling method buys accuracy.
Sampling error shrinks as the sample grows. Sampling bias does not — it is a property of how the sample was chosen, and it survives any amount of extra data.
This is why a convenience sample of 10,000 is worse than a random sample of 500, and why every study you read should be judged on how it selected people before it is judged on how many it reached.
† A caution for the rest of the course. Every confidence interval and hypothesis test in later chapters computes precision from the sample size. Not one of them can detect that the sample was drawn badly — that check is yours, and it happens before the arithmetic starts.
§1.2 — conclusions
The core idea
The type of data decides what you can do with it. The sampling method decides whether you can trust the result. Variation is not a bug — it is the reason statistics exists. A sample that is not representative is not saved by being large.
The failure case
A convenience sample of 10,000 people, a self-selected online poll with 50,000 responses, a survey that reaches only landline phones — every one produces a perfectly precise number that may be badly wrong. The arithmetic is fine. The conclusion is not.
Next: §1.3 — Frequency, Frequency Tables, and Levels of Measurement, where the data you have classified gets organized into tables and the measurement scale determines what "average" even means.