1.6 Sampling Experiment

1.6.1 Stats Lab: Sampling Experiment

Class Time: ____________ Names: ____________

Student Learning Outcomes

In this lab, you will be asked to pick several random samples of restaurants. In each case, describe your procedure briefly — including how you might have used a random number generator — and then list the restaurants in the sample you obtained.

The following table contains restaurants stratified by city into columns and grouped horizontally by entree cost (clusters). Use it as your population for every sample you build below.

Table 1.6.1 — Restaurants stratified by city (columns) and entree cost (rows). This is the population for the lab.
CityUnder \$10\$10 to under \$15\$15 to under \$20Over \$20
San JoseEl Abuelo Taq, Pasta Mia, Emma's Express, Bamboo HutEmperor's Guard, Creekside InnAgenda, Gervais, Miro'sBlake's, Eulipia, Hayes Mansion, Germania
Palo AltoSenor Taco, Olive Garden, Taxi'sMing's, P.A. Joe's, Stickney'sScott's Seafood, Poolside Grill, Fish MarketSundance Mine, Maddalena's, Spago's
Los GatosMary's Patio, Mount Everest, Sweet Pea's, Andele TaqueriaLindsey's, Willow StreetToll HouseCharter House, La Maison Du Cafe
Mountain ViewMaharaja, New Ma's, Thai-Rific, Garden FreshAmber Indian, La Fiesta, Fiesta del Mar, DawitAustin's, Shiva's, MazehLe Petit Bistro
CupertinoHobees, Hung Fu, Samrat, Panda ExpressSanta Barb. Grill, Mand. Gourmet, Bombay Oven, Kathmandu WestFontana's, Blue PheasantHamasushi, Helios
SunnyvaleChekijababi, Taj India, Full Throttle, Tia Juana, Lemon GrassPacific Fresh, Charley Brown's, Cafe Cameroon, Faz, Aruba'sLion & Compass, The Palace, Beau Sejour
Santa ClaraRangoli, Armadillo Willy's, Thai Pepper, PasandArthur's, Katie's Cafe, Pedro's, La GalleriaBirk's, Truya Sushi, Valley PlazaLakeside, Mariani's
Try It Now 1.6.1

You want to choose 15 restaurants from the population in Table 1.6.1 using a simple random sample. Each restaurant has been given a number from 1 to 70. Describe, in two or three sentences, how a random number generator turns those 70 labels into your sample of 15.

Solution

Use a random number generator to produce integers from 1 to 70. Read the numbers it gives you and match each one to the restaurant that holds that label. Keep generating numbers — ignoring any repeats, since we are sampling without replacement — until you have 15 different restaurants. Those 15 labeled restaurants are your simple random sample. Every restaurant had an equal chance of being picked, which is exactly what "simple random" means.

A Simple Random Sample

Pick a simple random sample of 15 restaurants.

  1. Describe your procedure.
  2. Complete the table with your sample.
Table 1.6.2 — Your simple random sample of 15 restaurants.
1.6.11.
2.7.12.
3.8.13.
4.9.14.
5.10.15.
Try It Now 1.6.2

For a systematic sample of 15 restaurants out of 70, what is the skip value \(k\)? Then explain how you decide where to start.

Solution

The skip value is the population size divided by the sample size: $$ k = \frac{70}{15} \approx 4.67 \to 4. $$ Round to \(k = 4\). Pick a random starting point between 1 and 4 — say a random number generator gives you 3. Then take restaurant 3, 7, 11, 15, and so on, choosing every 4th restaurant down the list. If you reach the end of the list before you have 15, loop back to the top and keep counting until your sample is complete.

A Systematic Sample

Pick a systematic sample of 15 restaurants.

  1. Describe your procedure.
  2. Complete the table with your sample.
Table 1.6.3 — Your systematic sample of 15 restaurants.
1.6.11.
2.7.12.
3.8.13.
4.9.14.
5.10.15.
Try It Now 1.6.3

In a stratified sample by city, you take 25% of the restaurants from each city. San Jose has 13 restaurants in Table 1.6.1. How many should you draw from San Jose, and how do you draw them?

Solution

Take 25% of San Jose's 13 restaurants: $$ 0.25 \times 13 = 3.25 \to 3. $$ Round to 3 restaurants. To choose them, run a simple random sample within the San Jose column only — number San Jose's restaurants and use a random number generator to pick 3 of them. Repeat the same "25%, round, then simple-random-within-the-stratum" recipe for every city. Stratifying guarantees each city is represented in proportion to its size.

A Stratified Sample (by City)

Pick a stratified sample, by city, of 20 restaurants. Use 25% of the restaurants from each stratum. Round to the nearest whole number.

  1. Describe your procedure.
  2. Complete the table with your sample.
Table 1.6.4 — Your stratified sample (by city) of 20 restaurants.
1.6.11.16.
2.7.12.17.
3.8.13.18.
4.9.14.19.
5.10.15.20.
Try It Now 1.6.4

When you stratify by entree cost instead of by city, what changes about which restaurants can land in the same stratum together? Use Table 1.6.1 to explain.

Solution

Stratifying by cost groups restaurants by the rows of Table 1.6.1 (Under $10, $10 to under $15, $15 to under $20, Over $20) instead of by the columns (cities). So a stratum can now hold restaurants from many different cities, as long as they share a price range — for example, the "Under $10" stratum mixes San Jose, Palo Alto, Los Gatos, and every other city. You still take 25% of each stratum and pick them with a simple random sample within the stratum, but the thing each stratum keeps balanced is price, not city.

A Stratified Sample (by Entree Cost)

Pick a stratified sample, by entree cost, of 21 restaurants. Use 25% of the restaurants from each stratum. Round to the nearest whole number.

  1. Describe your procedure.
  2. Complete the table with your sample.
Table 1.6.5 — Your stratified sample (by entree cost) of 21 restaurants.
1.6.11.16.
2.7.12.17.
3.8.13.18.
4.9.14.19.
5.10.15.20.
21.
Try It Now 1.6.5

A cluster sample here means picking two whole cities at random and taking every restaurant in them. How is that different from a stratified sample by city, where you also organize by city?

Solution

In a stratified sample by city you reach into every city and pull a few restaurants from each. In a cluster sample you randomly choose a small number of cities (here, two) and then take all the restaurants in just those cities — the other cities contribute nothing. Stratified spreads your sample across the whole population for balance; cluster concentrates your effort on a few groups, which is cheaper to carry out but risks missing whatever is special about the cities you did not pick.

A Cluster Sample

Pick a cluster sample of restaurants from two cities. The number of restaurants will vary.

  1. Describe your procedure.
  2. Complete the table with your sample.
Table 1.6.6 — Your cluster sample of restaurants from two cities.
1.6.11.16.21.
2.7.12.17.22.
3.8.13.18.23.
4.9.14.19.24.
5.10.15.20.25.

1.6.2 Chapter 1 Review

Here is the big picture from each section of the chapter, in plain English. Use it as a refresher before you tackle the problem set.

1.1 Definitions of Statistics, Probability, and Key Terms

The math behind statistics is much easier to learn once you know the language. Section 1.1 introduced the core vocabulary — population, sample, parameter, statistic, variable, and data — that gets used everywhere else in the book.

Try It Now 1.6.6

A coffee chain wants to know the mean number of drinks its customers buy per visit. They watch 200 customers at one busy downtown store. Match each term to the right piece of this study: population, sample, parameter, statistic, variable, data.

Solution
  • Population: all of the chain's customers.
  • Sample: the 200 customers they actually watched.
  • Variable: the number of drinks one customer buys per visit.
  • Data: the individual drink counts they recorded (2, 1, 3, ...).
  • Parameter: the true mean drinks-per-visit for all customers (usually unknown).
  • Statistic: the mean drinks-per-visit computed from the 200 watched customers, used to estimate the parameter.

1.2 Data, Sampling, and Variation in Data and Sampling

Data are individual pieces of information that come from a population or a sample. Data can be qualitative (categorical), quantitative continuous, or quantitative discrete.

Because it is almost never practical to measure an entire population, researchers study samples that stand in for the population. A random sample is a representative group chosen by a method that gives every individual an equal chance of being included. The random sampling methods are simple random, stratified, cluster, and systematic sampling. Convenience sampling is a nonrandom shortcut that often produces biased data.

Different individuals in a sample produce different data — even when the samples are well chosen and representative. When samples are selected properly, larger samples model the population more closely than smaller ones. Many things can quietly undermine the reliability of a sample, so statistical data should always be analyzed critically, never just accepted at face value.

Try It Now 1.6.7

A grocery store interviews the very first 40 shoppers who walk in when the doors open at 7 a.m. Name the sampling method, and give one reason this sample might not represent all of the store's shoppers.

Solution

This is convenience sampling — the store simply grabbed whoever was easiest to reach. It is not random, so it can be biased: the 7 a.m. crowd (early risers, commuters grabbing coffee) probably differs from afternoon or evening shoppers, so their answers may not reflect the whole shopper population.

1.3 Frequency, Frequency Tables, and Levels of Measurement

Some calculations spit out numbers that are artificially precise. There is no point reporting a value to eight decimal places when the measurements behind it were only accurate to the nearest tenth. The rule of thumb: round your final answer to one more decimal place than was present in the original data. So if your data were measured to the nearest tenth, report the final statistic to the nearest hundredth.

Beyond rounding, you can classify data by four levels of measurement:

When organizing data, it helps to know how many times each value appears. Frequency, relative frequency, and cumulative relative frequency are the measures that answer questions like "How many students studied five or more hours?"

Try It Now 1.6.8

Classify each by level of measurement (nominal, ordinal, interval, or ratio): (a) jersey numbers on a soccer team, (b) finishing places in a race — 1st, 2nd, 3rd, (c) temperatures in degrees Fahrenheit, (d) weights of packages in kilograms.

Solution
  • (a) Jersey numbers — nominal. They are just labels; you cannot order or do arithmetic on them.
  • (b) Finishing places — ordinal. They have a clear order, but the gap between 1st and 2nd is not a measurable, equal quantity.
  • (c) Fahrenheit temperatures — interval. Differences are meaningful, but \(0^\circ\)F is not a true zero, so ratios ("twice as hot") do not make sense.
  • (d) Weights in kilograms — ratio. There is a true zero, so differences and ratios are meaningful (8 kg really is twice 4 kg).

1.4 Experimental Design and Ethics

A poorly designed study will not produce reliable data. To rule out lurking variables, subjects must be assigned to treatment groups randomly. One group acts as a control group, showing what happens when the active treatment is not applied. To counter the power of suggestion, the control group receives an inactive placebo that looks exactly like the real treatments but cannot directly affect the response variable. To protect the placebo's cover, both researchers and subjects may be blinded. When a study is designed properly, the only difference between groups is the one the researcher imposes — so if the groups respond differently, that difference must be due to the explanatory variable.

"An ethics problem arises when you are considering an action that benefits you or some cause you support, hurts or reduces benefits to others, and violates some rule." (Andrew Gelman, Open Data and Open Methods, Ethics and Statistics.) Ethical violations in statistics are not always easy to spot. Professional associations and federal agencies post guidelines for proper conduct, so learning the basic procedures lets you recognize when data analysis is being done honestly.

Try It Now 1.6.9

A study tests a new headache pill. Identify the control group, the placebo, and what "double-blind" would mean here.

Solution
  • Control group: the patients who do not get the active pill — they show what happens without the real treatment.
  • Placebo: a fake pill (say, a sugar tablet) that looks identical to the real one, given to the control group so the power of suggestion is the same for everyone.
  • Double-blind: neither the patients nor the researchers interacting with them know who got the real pill and who got the placebo, which stops either side from nudging the results.

Problem Set 1.6 — Chapter 1 Review

Problem 1. A "random survey" was conducted of 3,274 people of the "microprocessor generation" (people born since 1971, the year the microprocessor was invented). It was reported that 48% of those individuals surveyed stated that if they had $2,000 to spend, they would use it for computer equipment. Also, 66% of those surveyed considered themselves relatively savvy computer users.

a) Do you consider the sample size large enough for a study of this type? Why or why not?

b) Based on your "gut feeling," do you believe the percents accurately reflect the U.S. population for those individuals born since 1971? If not, do you think the percents of the population are actually higher or lower than the sample statistics? Why?

Additional information: The survey, reported by Intel Corporation, was filled out by individuals who visited the Los Angeles Convention Center to see the Smithsonian Institute's road show called "America's Smithsonian."

c) With this additional information, do you feel that all demographic and ethnic groups were equally represented at the event? Why or why not?

d) With the additional information, comment on how accurately you think the sample statistics reflect the population parameters.

Solution

Step 1 — Part (a), judge the sample size: A sample of 3,274 is reasonably large for a survey of this kind; large samples reduce sampling variability. So yes, the size itself is adequate — but size alone does not guarantee a representative sample.

Step 2 — Part (b), gut-check the percents: A "gut feeling" answer is acceptable here. Many students reason that 48% wanting to spend $2,000 on computer equipment seems high for the whole population, suggesting the reported percents are probably higher than the true population values, because the people surveyed are not typical.

Step 3 — Part (c), assess representativeness given the venue: The respondents were self-selected visitors to a Smithsonian technology road show at a convention center. Such visitors are disproportionately tech-interested, affluent, and able to travel to the event, so demographic and ethnic groups were not equally represented.

Step 4 — Part (d), comment on accuracy: Because the sample is self-selected and drawn from an unrepresentative, tech-enthusiast crowd, the statistics likely overstate computer interest and savviness in the full population. The sample statistics should not be trusted as accurate estimates of the population parameters despite the large \(n\).

Answer: (a) The size (3,274) is large enough, but size does not fix bias. (b) The percents are likely too high. (c) No — convention attendees self-selected and skew tech-interested and affluent, so groups were not equally represented. (d) The statistics probably overstate the true population parameters because of self-selection/sampling bias.

Problem 2. The Well-Being Index is a survey that follows trends of U.S. residents on a regular basis. There are six areas of health and wellness covered in the survey: Life Evaluation, Emotional Health, Physical Health, Healthy Behavior, Work Environment, and Basic Access. Some of the questions used to measure the Index are listed below. Identify the type of data obtained from each question used in this survey: qualitative, quantitative discrete, or quantitative continuous.

a) Do you have any health problems that prevent you from doing any of the things people your age can normally do?

b) During the past 30 days, for about how many days did poor health keep you from doing your usual activities?

c) In the last seven days, on how many days did you exercise for 30 minutes or more?

d) Do you have health insurance coverage?

Solution

Step 1 — Recall the three data types: Qualitative (categorical) data describe a category; quantitative discrete data are counts; quantitative continuous data are measurements that can take any value in an interval (often times or amounts measured).

Step 2 — Classify (a): "Do you have any health problems that prevent you…?" is a yes/no category response — qualitative.

Step 3 — Classify (b): "About how many days did poor health keep you from your usual activities?" is a count of days — quantitative discrete.

Step 4 — Classify (c): "On how many days did you exercise 30 minutes or more?" is again a count of days — quantitative discrete.

Step 5 — Classify (d): "Do you have health insurance coverage?" is a yes/no category — qualitative.

Answer: a) qualitative; b) quantitative discrete; c) quantitative discrete; d) qualitative.

Problem 3. In advance of the 1936 Presidential Election, a magazine titled Literary Digest released the results of an opinion poll predicting that the republican candidate Alf Landon would win by a large margin. The magazine sent post cards to approximately 10,000,000 prospective voters. These prospective voters were selected from the subscription list of the magazine, from automobile registration lists, from phone lists, and from club membership lists. Approximately 2,300,000 people returned the postcards.

a) Think about the state of the United States in 1936. Explain why a sample chosen from magazine subscription lists, automobile registration lists, phone books, and club membership lists was not representative of the population of the United States at that time.

b) What effect does the low response rate have on the reliability of the sample?

c) Are these problems examples of sampling error or nonsampling error?

d) During the same year, George Gallup conducted his own poll of 30,000 prospective voters. These researchers used a method they called "quota sampling" to obtain survey answers from specific subsets of the population. Quota sampling is an example of which sampling method described in this module?

Solution

Step 1 — Part (a), why the frame was biased: In 1936, telephones, automobiles, and magazine subscriptions and club memberships were luxuries owned mainly by wealthier households. A sampling frame built from those lists systematically excluded poorer voters, so it was not representative of the U.S. electorate (this is undercoverage / sampling-frame bias).

Step 2 — Part (b), effect of low response: About 2,300,000 of 10,000,000 responded — roughly a 23% response rate. The people who chose to mail back postcards were self-selected and likely differed in opinion from non-responders, introducing nonresponse bias that makes the sample unreliable.

Step 3 — Part (c), classify the errors: A biased frame and self-selected responders are flaws in how the sample was obtained, not random chance from sampling, so these are nonsampling errors.

Step 4 — Part (d), name Gallup's method: Filling fixed quotas from specific subgroups of the population matches stratified sampling (selecting set numbers from defined strata).

Answer: (a) The lists covered mostly wealthier households, excluding poorer voters (undercoverage). (b) The ~23% response rate creates nonresponse bias, undermining reliability. (c) Nonsampling error. (d) Quota sampling corresponds to stratified sampling.

Problem 4. Crime-related and demographic statistics for 47 US states in 1960 were collected from government agencies, including the FBI's Uniform Crime Report. One analysis of this data found a strong connection between education and crime indicating that higher levels of education in a community correspond to higher crime rates. Which of the potential problems with samples discussed in 1.2 Data, Sampling, and Variation in Data and Sampling could explain this connection?

Solution

Step 1 — Read the claim: The data show a correlation — communities with more education also have higher reported crime rates. The problem asks which sampling/data pitfall could explain this.

Step 2 — Identify the likely cause: This is a case where correlation does not imply causation, driven by a confounding (lurking) variable. For example, more urban or more populous communities may have both better-funded schools (higher education) and more thorough crime reporting/higher crime, so a third variable creates the apparent link rather than education causing crime.

Answer: The connection is best explained by a confounding variable (and the correlation-is-not-causation caution): an outside factor such as urbanization or reporting practices is associated with both higher education levels and higher recorded crime, producing a misleading association.

Problem 5. Imagine you work for a polling company and a member of your team has proposed the following survey question:

"Do you feel happy paying your taxes while some politicians are allowed to use loopholes and avoid paying their fair share of taxes?"

As part of preliminary data collection, 11 people responded to this question. Each participant answered "NO!" Which of the potential problems with samples discussed in this module could explain this connection?

Solution

Step 1 — Examine the question wording: The question — "Do you feel happy paying your taxes while some politicians are allowed to use loopholes and avoid paying their fair share…?" — is emotionally loaded and pushes the respondent toward a negative answer.

Step 2 — Identify the pitfall: A leading or loaded question introduces response bias (a form of nonsampling error). Because the wording prompts a particular answer, all 11 people responding "NO!" reflects the biased instrument, not genuine independent opinion. The tiny sample size of 11 also limits any conclusion.

Answer: The uniform "NO!" responses are explained by response bias from the leading, loaded wording of the question (a nonsampling error); the very small sample size compounds the problem.

Problem 6. A scholarly article about response rates begins with the following quote: "Declining contact and cooperation rates in random digit dial (RDD) national telephone surveys raise serious concerns about the validity of estimates drawn from such research."

The Pew Research Center for People and the Press admits: "The percentage of people we interview – out of all we try to interview – has been declining over the past decade or more."

a) What are some reasons for the decline in response rate over the past decade?

b) Explain why researchers are concerned with the impact of the declining response rate on public opinion polls.

Solution

Step 1 — Part (a), reasons for declining response rates: Possible reasons include caller-ID and call screening, the spread of cell phones (vs. listed landlines), distrust of telemarketers and scams, "survey fatigue," busier lifestyles, and concerns about privacy. Any well-reasoned causes are acceptable.

Step 2 — Part (b), why researchers worry: As fewer contacted people respond, the responders become more self-selected and may differ systematically from non-responders. This nonresponse bias threatens the validity of the estimates — the poll may no longer reflect the broader public, no matter how the numbers are computed.

Answer: (a) Reasons include call screening/caller ID, cell-phone-only households, distrust and privacy concerns, and survey fatigue. (b) Lower response rates raise the risk of nonresponse bias, so the responding sample may not represent the population and the poll's conclusions can become invalid.

Problem 7. Seven hundred and seventy-one distance learning students at Long Beach City College responded to surveys in a specific academic year. Highlights of the summary report are listed in Table 1.6.7.

Table 1.6.7 — LBCC distance learning survey results.
Have computer at home96%
Unable to come to campus for classes65%
Age 41 or over24%
Would like LBCC to offer more DL courses95%
Took DL classes due to a disability17%
Live at least 16 miles from campus13%
Took DL courses to fulfill transfer requirements71%

a) What percent of the students surveyed do not have a computer at home?

b) About how many students in the survey live at least 16 miles from campus?

c) If the same survey were done at Great Basin College in Elko, Nevada, do you think the percentages would be the same? Why?

Solution

Step 1 — Part (a), percent without a home computer: 96% have a computer at home, so the complement is \(100\%-96\%=4\%\).

Step 2 — Part (b), number living at least 16 miles away: 13% of the 771 respondents: \(0.13 \times 771 \approx 100.2\), so about 100 students.

Step 3 — Part (c), would percentages transfer to Great Basin College? Probably not. Long Beach City College is in a dense urban area, while Great Basin College in Elko, Nevada serves a rural region. Differences in internet access, commuting distances, and student demographics would likely change percentages such as "live at least 16 miles from campus" and "unable to come to campus."

Answer: (a) 4%. (b) about 100 students. (c) No — the two colleges serve different (urban vs. rural) populations, so distances, access, and demographics differ, and the percentages would likely not be the same.

Problem 8. Several online textbook retailers advertise that they have lower prices than on-campus bookstores. However, an important factor is whether the Internet retailers actually have the textbooks that students need in stock. Students need to be able to get textbooks promptly at the beginning of the college term. If the book is not available, then a student would not be able to get the textbook at all, or might get a delayed delivery if the book is back ordered.

Martin, a college newspaper reporter, is investigating textbook availability at online retailers. He decides to investigate one textbook for each of the following seven subjects: calculus, biology, chemistry, physics, statistics, geology, and general engineering. He consults textbook industry sales data and selects the most popular nationally used textbook in each of these subjects. He visits websites for a random sample of major online textbook sellers and looks up each of these seven textbooks to see if they are available in stock for quick delivery through these retailers. Based on his investigation, Martin writes an article in which he draws conclusions about the overall availability of all college textbooks through online textbook retailers.

Write an analysis of his study that addresses the following issues: Is his sample representative of the population of all college textbooks? Explain why or why not. Describe some possible sources of bias in this study, and how it might affect the results of the study. Give some suggestions about what could be done to improve the study.

Solution

Step 1 — Assess representativeness of the sample of textbooks: Martin draws conclusions about all college textbooks but examines only seven books — the single most popular title in each of seven subjects. This sample is far too small and is not representative: popular, high-demand bestsellers are exactly the books most likely to be stocked, so they don't reflect the availability of the thousands of niche, low-volume, or specialized textbooks that make up most of the population.

Step 2 — Identify sources of bias: (1) Selection bias toward best-sellers, which are kept in stock more reliably than typical or obscure titles. (2) Undercoverage — only seven subjects are covered, omitting the many fields and editions students actually need. (3) Sampling only major online retailers may overstate availability compared with the full range of sellers. These biases would make availability look better than it truly is.

Step 3 — Suggest improvements: Use a much larger sample of textbooks drawn randomly across many subjects, difficulty levels, and popularity tiers (including niche and lower-demand titles) and across multiple editions. Sample a broader, representative set of online retailers, and check availability at the actual start-of-term rush when demand peaks.

Answer: The sample is not representative: seven best-selling titles cannot speak for all college textbooks, and best-sellers are the most likely to be in stock. Main biases are selection bias toward popular books, undercoverage of subjects/editions, and using only major retailers — all of which overstate availability. Improvements: take a large random sample of textbooks spanning many subjects, popularity levels, and editions; sample a representative range of retailers; and test availability during the beginning-of-term period.

Key Terms

average — also called the mean; a number that describes the central tendency of the data.

blinding — not telling participants which treatment a subject is receiving.

categorical variable — a variable that takes on values that are names or labels.

cluster sampling — divide the population into groups (clusters), use simple random sampling to select a set of clusters, and include every individual in the chosen clusters.

continuous random variable — a random variable whose outcomes are measured (for example, the height of trees in a forest).

control group — a group in a randomized experiment that receives an inactive treatment but is otherwise managed exactly like the other groups.

convenience sampling — a nonrandom method that selects easily accessible individuals and may produce biased data.

cumulative relative frequency — for an ordered data set, the sum of the relative frequencies for all values less than or equal to the given value.

data — a set of observations; usually grouped as qualitative (label-valued) or quantitative (number-valued), and quantitative data split into discrete (counting) and continuous (measuring).

double-blind experiment — an experiment in which both the subjects and the researchers working with them are blinded.

experimental unit — any individual or object to be measured.

explanatory variable — the independent variable in an experiment; the value controlled by researchers.

frequency — the number of times a value of the data occurs.

informed consent — a human subject must understand the risks, costs, and benefits of a study, and consent must be given freely by an informed, fit participant.

institutional review board — a committee that oversees research programs involving human subjects.

lurking variable — a variable that affects a study even though it is neither the explanatory nor the response variable.

nonsampling error — any reliability problem other than natural variation, including poor design, biased methods, inaccurate participant information, data-entry errors, and poor analysis.

numerical variable — a variable that takes on values indicated by numbers.

parameter — a number used to represent a population characteristic; generally hard to determine.

placebo — an inactive treatment that cannot directly affect the response variable, used to counter the power of suggestion.

population — all individuals, objects, or measurements whose properties are being studied.

probability — a number between zero and one, inclusive, giving the likelihood that a specific event occurs.

proportion — the number of successes divided by the total number in the sample.

qualitative data — observations whose value is indicated by a label.

quantitative data — observations whose value is indicated by a number; split into discrete (counting) and continuous (measuring).

random assignment — organizing experimental units into treatment groups using random methods.

random sampling — a method that gives every member of the population an equal chance of being selected.

relative frequency — the ratio of the number of times a value occurs to the total number of outcomes.

representative sample — a subset of the population that has the same characteristics as the population.

response variable — the dependent variable in an experiment; the value measured for change at the end.

sample — a subset of the population studied.

sampling bias — not all members of the population are equally likely to be selected.

sampling error — the natural variation from selecting a sample; it decreases as sample size increases.

sampling with replacement — a selected member is returned to the population before the next selection.

sampling without replacement — a member may be chosen only once and is not returned before the next selection.

simple random sampling — give each member a number, then use a random number generator to select the labels that identify your sample.

statistic — a numerical characteristic of the sample that estimates the corresponding population parameter.

stratified sampling — divide the population into strata, then use simple random sampling to pick a proportionate number from each stratum.

systematic sampling — list the population, randomly pick a starting point, set \(k = (\text{population size}) / (\text{sample size})\), then choose every \(k\)th individual, wrapping to the start of the list if needed.

treatments — the different values or components of the explanatory variable applied in an experiment.

variable — a characteristic of interest for each person or object in a population.

Advanced Sampling Techniques Require Advanced Methods

The methods of inference covered in this book generally only apply to simple random samples. More advanced analysis techniques are required for systematic, stratified, cluster, and multistage random sampling.