11.3 The Regression Equation
SLO 2
Identify appropriate graphs and summary statistics for variables and relationships between them and correctly interpret information from graphs and summary statistics.
SLO 4
Demonstrate an understanding of, and ability to use, basic ideas of statistical processes, including hypothesis tests and confidence interval estimation.
Learning Objectives
By the end of this section, you will be able to:
- explain what a residual is and what its sign tells you about the line;
- state the least-squares criterion and say why the errors are squared before adding;
- write the regression equation \(\hat{y} = a + bx\) and identify \(a\) and \(b\) from calculator output;
- interpret the slope of a best-fit line in the context of the data;
- explain why predictions outside the range of the sample x-values are not justified;
- read a residuals plot, and interpret the correlation coefficient \(r\) and the coefficient of determination \(r^{2}\).
Data rarely fit a straight line exactly. Usually, you must be satisfied with rough predictions. Typically, you have a set of data whose scatter plot appears to "fit" a straight line — and Section 11.2 gave you the tools to decide whether it does. This section answers the question that follows: which line? If ten people drew a line through the same scatter plot by eye, you would get ten different lines. What is needed is a rule that picks exactly one, for reasons that can be stated.
11.3.1 The Line of Best Fit
A Line of Best Fit, also called a Least-Squares Line, is the single straight line that a stated criterion selects as the best summary of a set of paired data.
Throughout this section one data set will carry the explanation. A random sample of 11 statistics students produced the data in Table 11.3.1, where \(x\) is the third exam score out of 80, and \(y\) is the final exam score out of 200. Can you predict the final exam score of a random student if you know the third exam score?
| x (third exam score) | y (final exam score) |
|---|---|
| 65 | 175 |
| 67 | 133 |
| 71 | 185 |
| 71 | 163 |
| 66 | 126 |
| 75 | 198 |
| 67 | 153 |
| 70 | 163 |
| 71 | 159 |
| 69 | 151 |
| 69 | 159 |
The third exam score, \(x\), is the independent variable and the final exam score, \(y\), is the dependent variable. If each of you were to fit a line "by eye," you would draw different lines. We can use what is called a least-squares regression line to obtain the best fit line.
Each point of data is of the form \((x, y)\) and each point of the line of best fit using least-squares linear regression has the form \((x, \hat{y})\).
Definition 11.3.1 — Line of Best Fit: A scatter plot of the eleven paired scores from Table 11.3.1 (third exam score x out of 80 on the horizontal axis, final exam score y out of 200....
The symbol \(\hat{y}\) is read "y hat" and is the estimated value of \(y\). It is the value of \(y\) obtained using the regression line. It is not generally equal to \(y\) from the data.
That last sentence is the whole reason the hat exists. For any \(x\) in the data there is a real observed \(y\) and a predicted \(\hat{y}\), and they almost never agree. Keeping the two symbols apart keeps the two ideas apart.
Definition 11.3.2 — y-hat: Table 11.3.1 data with ŷ = −173.51 + 4.83x — the hat marks the prediction, not the observation.
If you know a person's pinky finger length, do you think you could predict that person's height? Suppose you collect pinky length in inches from your class, take \(x\) = pinky finger length and \(y\) = height, and draw a line "by eye" that appears to fit the data. Would everyone in the class get the same equation? What does that tell you about why a rule is needed?
Solution — why "by eye" is not enough
Step 1 — Consider what "by eye" involves. Each person judges the line's position and steepness visually, picks two convenient points on their own line, computes a slope, and extends the line to read off a y-intercept.
Step 2 — Ask whether the results would agree. No. Different people place the line differently, so they get different slopes and different y-intercepts, and therefore different equations.
Step 3 — Say why that matters. Different equations give different predicted heights for the same pinky length. If one classmate predicts one height at 2.5 inches and another predicts a different one, the data alone cannot say who is right.
Step 4 — State what is needed. A criterion that anyone can apply to the same data and get the same line. That criterion is least squares, and it is the subject of the next two subsections.
Answer: Everyone would get a different equation, which is exactly why a stated rule for choosing the line is required rather than visual judgment.
11.3.2 Residuals: The Vertical Distances
The term \(y_{0} - \hat{y}_{0} = \varepsilon_{0}\) is called the "error" or residual. It is not an error in the sense of a mistake. The absolute value of a residual measures the vertical distance between the actual value of \(y\) and the estimated value of \(y\) — in other words, the vertical distance between the actual data point and the predicted point on the line.
Here \(\varepsilon\) is the Greek letter epsilon.
The sign carries information:
- If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for \(y\).
- If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for \(y\).
For each data point, you can calculate the residuals \(y_{i} - \hat{y}_{i} = \varepsilon_{i}\) for \(i = 1, 2, 3, \ldots, 11\). Each \(|\varepsilon|\) is a vertical distance.
The nearest point on a line is reached perpendicular to it, which looks like the more natural measure. Least-squares regression uses the vertical gap instead, and the reason is what the line is FOR: predicting \(y\) from \(x\). At a given \(x\), the line offers \(\hat{y}\) and the data hold \(y\); the vertical gap is exactly the amount by which the prediction missed. A perpendicular distance would mix an error in \(y\) together with a displacement in \(x\), and \(x\) is the value you were given, not something being predicted.
Definition 11.3.3 — Residual: Same x = 69, same prediction ŷ = 158.4, two different residuals — from Table 11.3.1 and Try It Now 11.3.2.
A regression line predicts \(\hat{y} = 158.4\) for a student whose third exam score is 69. One such student actually scored 151 on the final, and another scored 159. Compute both residuals, state the sign of each, and say what each sign means about the line at that point.
Solution — two residuals at the same x
Step 1 — Compute the first residual.
$$ \varepsilon = y - \hat{y} = 151 - 158.4 = -7.4 $$Step 2 — Interpret its sign. The residual is negative, so this data point lies below the line and the line overestimated that student's final exam score by 7.4 points.
Step 3 — Compute the second residual.
$$ \varepsilon = y - \hat{y} = 159 - 158.4 = 0.6 $$Step 4 — Interpret its sign. The residual is positive, so this point lies above the line and the line underestimated that student's score by 0.6 points.
Step 5 — Note what the pair shows. Both students had the same third exam score and therefore the same prediction, but different final scores. A single \(x\) can carry several different \(y\) values, and the line can only offer one \(\hat{y}\) for all of them.
Answer: Residuals \(-7.4\) (below the line, an overestimate) and \(+0.6\) (above the line, an underestimate).
A data point lies well above the regression line. Without any numbers, state the sign of its residual, say whether the line under- or overestimates there, and explain whether a large residual means the data point is wrong.
Solution — sign, direction, and what a residual is not
Step 1 — Determine the sign. The point lies above the line, so \(y > \hat{y}\) and the residual \(y - \hat{y}\) is positive.
Step 2 — Say what the line did. A positive residual means the line's prediction fell short of the observed value, so the line underestimates the actual \(y\) at that \(x\).
Step 3 — Address the "wrong" question. A residual is not an error in the sense of a mistake. It measures how far the observed value sits from the prediction, and real data scatter about any line.
Step 4 — Say what a large residual DOES signal. It marks a point the line describes poorly. That is worth noticing — it may be an unusual observation worth investigating — but it is not by itself evidence that the measurement was recorded incorrectly.
Answer: The residual is positive and the line underestimates \(y\) there. A large residual identifies a poorly-predicted point, not a mistaken one.
11.3.3 The Least-Squares Criterion
The Sum of Squared Errors (SSE) is the total obtained by squaring every residual and adding the results.
Definition 11.3.4 — Sum of Squared Errors (SSE): Definition 11.3.4 — SSE: square every residual, then add the results.
The process of fitting the best-fit line is called linear regression. The criterion for the best fit line is that the sum of the squared errors (SSE) is minimized — made as small as possible. Any other line you might choose would have a higher SSE than the best fit line. This best fit line is called the least-squares regression line.
Two reasons, and both matter. First, residuals come in both signs, so adding them raw lets a large overestimate cancel a large underestimate — a terrible line could total zero. Squaring removes the signs so that every miss counts as a miss. Second, squaring penalises a big miss far more than several small ones: being off by 10 contributes 100, while being off by 2 five times contributes only 20. The criterion therefore prefers a line that avoids large errors, even at the cost of more small ones.
Definition 11.3.5 — Least-Squares Regression Line: A scatter plot of the eleven students from Table 11.3.1, third exam score on the horizontal axis and final exam score on the vertical axis.
For the example about the third exam scores and the final exam scores for the 11 statistics students, there are 11 data points, so there are 11 \(\varepsilon\) values. If you square each \(\varepsilon\) and add, you get
$$ (\varepsilon_{1})^{2} + (\varepsilon_{2})^{2} + \ldots + (\varepsilon_{11})^{2} = \sum_{i=1}^{11} \varepsilon^{2} $$A student proposes choosing the line that minimises the sum of the residuals themselves, \(\sum \varepsilon\), rather than the sum of their squares. Explain why that criterion fails, using a concrete pair of residuals.
Solution — a criterion that a bad line can satisfy
Step 1 — Consider what the proposed sum does with signs. Residuals are positive above the line and negative below it, so the two kinds cancel when added.
Step 2 — Give a concrete failure. Suppose a line misses one point by \(+50\) and another by \(-50\). Then \(\sum \varepsilon = 50 + (-50) = 0\), the smallest possible total, even though the line is 50 units away from both points.
Step 3 — Compare with squaring. Under least squares the same two misses contribute \(50^{2} + (-50)^{2} = 5000\), a large penalty that correctly reports a poor fit.
Step 4 — State the general point. Minimising the raw sum rewards cancellation rather than accuracy, and many wildly different lines could all achieve a total of zero.
Answer: The signed residuals cancel, so a line that misses badly in both directions can score a perfect zero. Squaring removes the signs so that every miss counts.
11.3.4 The Regression Equation
Using calculus, you can determine the values of \(a\) and \(b\) that make the SSE a minimum. When you make the SSE a minimum, you have determined the points that are on the line of best fit. It turns out that the line of best fit has the equation
$$ \hat{y} = a + bx $$where
$$ a = \overline{y} - b\overline{x} \qquad \text{and} \qquad b = \frac{\Sigma(x - \overline{x})(y - \overline{y})}{\Sigma(x - \overline{x})^{2}} $$The sample means of the \(x\) values and the \(y\) values are \(\overline{x}\) and \(\overline{y}\), respectively.
Two facts about this line are worth committing to memory:
- The best fit line always passes through the point \((\overline{x}, \overline{y})\). The formula for \(a\) is just that statement rearranged.
- The slope can also be written \(b = r\left(\frac{s_{y}}{s_{x}}\right)\), where \(s_{y}\) is the standard deviation of the \(y\) values, \(s_{x}\) is the standard deviation of the \(x\) values, and \(r\) is the correlation coefficient discussed later in this section.
Computer spreadsheets, statistical software, and many calculators can quickly calculate the best-fit line and create the graphs. The calculations tend to be tedious if done by hand. What you are responsible for is reading the output correctly, interpreting the slope in context, and knowing when the line should not be used at all — none of which the calculator does for you.
For the third-exam/final-exam data, the least squares regression line has the equation
$$ \hat{y} = -173.51 + 4.83x $$Using the TI-83, 83+, 84, 84+ Calculator
Using the Linear Regression T Test, LinRegTTest:
- In the
STATlist editor, enter the X data in listL1and the Y data in listL2, paired so that the corresponding \((x,y)\) values are next to each other in the lists. If a particular pair of values is repeated, enter it as many times as it appears in the data. - On the
STAT TESTSmenu, scroll down to selectLinRegTTest. Be careful to selectLinRegTTest, as some calculators may also have a different item calledLinRegTInt. - On the
LinRegTTestinput screen enter:Xlist: L1;Ylist: L2;Freq: 1. - On the next line, at the prompt \(\beta\) or \(\rho\), highlight "\(\neq 0\)" and press
ENTER. - Leave the line for
RegEq:blank. - Highlight
Calculateand pressENTER.
The output screen contains a lot of information. The second line says \(y = a + bx\); scroll down to find \(a = -173.513\) and \(b = 4.8273\), giving \(\hat{y} = -173.51 + 4.83x\). The two items at the bottom are \(r^{2} = 0.43969\) and \(r = 0.663\), which the last two subsections explain.
To graph the scatterplot and the regression line, create the scatter plot as in Section 11.2, then press Y = and type the equation -173.5 + 4.83X into Y1, and press ZOOM 9 again to graph it.
A calculator's LinRegTTest output reads \(a = -173.513363\), \(b = 4.827394209\), \(r = 0.663093591\). Write the regression equation rounded sensibly, and use it to predict the final exam score for a student who scored 73 on the third exam.
Solution — reading output and making one prediction
Step 1 — Identify the two constants. From the output, \(a = -173.513363\) and \(b = 4.827394209\).
Step 2 — Write the equation. Rounding to two decimal places,
$$ \hat{y} = -173.51 + 4.83x $$Step 3 — Substitute \(x = 73\).
$$ \hat{y} = -173.51 + 4.83(73) = -173.51 + 352.59 = 179.08 $$Step 4 — Check the prediction is legitimate. The sample's third exam scores run from 65 to 75, and 73 falls inside that range, so the prediction is within the domain of the data.
Answer: \(\hat{y} = -173.51 + 4.83x\), predicting a final exam score of about 179 for a third exam score of 73.
SCUBA divers have maximum dive times they cannot exceed when going to different depths. The data in Table 11.3.2 show different depths with the maximum dive times in minutes. Find the least squares regression line, and consider what it predicts for a depth of 110 feet.
| X (depth in feet) | Y (maximum dive time) |
|---|---|
| 50 | 80 |
| 60 | 55 |
| 70 | 45 |
| 80 | 35 |
| 90 | 25 |
| 100 | 22 |
Solution — a negative slope, and a prediction to be careful with
Step 1 — Enter the data and run the regression. Using a calculator gives
$$ \hat{y} = 127.24 - 1.11x $$Step 2 — Check the slope's sign against the data. The slope is negative, which matches the table: deeper dives allow less time. That agreement is a quick sanity check on the arithmetic.
Step 3 — Substitute \(x = 110\).
$$ \hat{y} = 127.24 - 1.11(110) \approx 4.7 \text{ minutes} $$Step 4 — Check whether the prediction is legitimate. It is not. The sample depths run from 50 to 100 feet, and 110 lies outside that range. The line was fitted to what happens between 50 and 100 feet and carries no evidence about 110.
Step 5 — Say what the number is worth. About 4.7 minutes is what the equation returns, but it is an extrapolation. Real dive tables do not continue in a straight line, and a slightly deeper extrapolation would return a negative time, which is impossible.
Answer: \(\hat{y} = 127.24 - 1.11x\). It returns roughly 4.7 minutes at 110 feet, but 110 is outside the sample range and the prediction is not justified by these data.
11.3.5 Interpreting the Slope
The slope of the best-fit line tells us how the dependent variable \((y)\) changes for every one unit increase in the independent \((x)\) variable, on average.
The line does not claim that any particular student who scores one point higher will score exactly 4.83 points higher on the final. Look back at the data: two students both scored 69 and finished with 151 and 159. The slope describes the average tendency across the whole sample, and dropping "on average" turns a statement about a trend into a promise about an individual.
Definition 11.3.6 — Interpretation of the Slope: Definition 11.3.6 — The slope b tells how y changes per one-unit increase in x, on average.
For the third-exam/final-exam example, the slope of the line is \(b = 4.83\).
Interpretation: For a one-point increase in the score on the third exam, the final exam score increases by 4.83 points, on average.
The slope of the line, \(b\), describes how changes in the variables are related. It is important to interpret the slope of the line in the context of the situation represented by the data. You should be able to write a sentence interpreting the slope in plain English.
For the dive-time regression \(\hat{y} = 127.24 - 1.11x\), interpret the slope in a complete sentence, naming the units. Then explain what the negative sign means in this context.
Solution — interpreting a negative slope in context
Step 1 — Identify the units. \(x\) is depth in feet and \(y\) is maximum dive time in minutes.
Step 2 — Apply the definition. The slope tells how \(y\) changes for a one-unit increase in \(x\), on average. Here a one-unit increase in \(x\) is one additional foot of depth.
Step 3 — Write the sentence. For each additional foot of depth, the maximum dive time decreases by about 1.11 minutes, on average.
Step 4 — Explain the sign. The negative slope means the two variables move in opposite directions: as depth increases, allowable time decreases. It says nothing about the relationship being weak.
Answer: For every additional foot of depth, maximum dive time falls by about 1.11 minutes on average; the negative sign records that greater depth goes with less time.
A student writes: "The slope is 4.83, which means the line goes up by 4.83." Explain what is missing and write a correct interpretation.
Solution — naming the variables and their units
Step 1 — Identify what the sentence contains. It restates the number and describes the line's geometry. Nothing in it could not have been written without reading the problem.
Step 2 — Identify what is missing. Three things: what \(x\) counts, what \(y\) measures, and the phrase "on average".
Step 3 — Write a correct version. For a one-point increase in the score on the third exam, the final exam score increases by 4.83 points, on average.
Step 4 — State the test to apply. If your interpretation would read identically for a completely different data set, it is not an interpretation — it is a restatement of the slope.
Answer: The student's sentence omits both variables, their units, and "on average". A correct interpretation names all three.
11.3.6 Predicting Only Within the Data's Range
It is always important to plot a scatter diagram first. If the scatter plot indicates that there is a linear relationship between the variables, then it is reasonable to use a best fit line to make predictions for \(y\) given \(x\) within the domain of x-values in the sample data, but not necessarily for x-values outside that domain. You could use the line to predict the final exam score for a student who earned a grade of 73 on the third exam. You should NOT use the line to predict the final exam score for a student who earned a grade of 50 on the third exam, because 50 is not within the domain of the x-values in the sample data, which are between 65 and 75.
For the third-exam/final-exam line \(\hat{y} = -173.51 + 4.83x\), compute the prediction at \(x = 30\) and explain what the result reveals about extrapolation.
Solution — an impossible prediction from a reasonable line
Step 1 — Substitute \(x = 30\).
$$ \hat{y} = -173.51 + 4.83(30) = -173.51 + 144.90 = -28.61 $$Step 2 — Judge the answer against reality. A final exam score of \(-28.61\) is impossible; the exam is scored out of 200 and cannot go below zero.
Step 3 — Explain why the line produced it. The line was fitted to third exam scores between 65 and 75. Nothing in the data describes what happens at 30, and a straight line extended far enough will eventually leave the range of possible values entirely.
Step 4 — Connect to the y-intercept. The intercept \(a = -173.51\) is the predicted final score at \(x = 0\) — also impossible, and for the same reason. In many regressions the intercept has no meaningful interpretation because \(x = 0\) is nowhere near the data.
Answer: The line predicts \(-28.61\), an impossible score. Extrapolating far outside the sample range can produce results that are not merely uncertain but meaningless.
Using the third-exam/final-exam data, whose x-values run from 65 to 75, decide whether each prediction is justified: (a) \(x = 68\), (b) \(x = 75\), (c) \(x = 80\).
Solution — checking each against the domain
Step 1 — State the domain. The sample x-values are between 65 and 75.
Step 2 — Check (a). 68 lies inside the range, so the prediction is justified.
Step 3 — Check (b). 75 is the largest observed value, so it is at the boundary of the domain and the prediction is still justified.
Step 4 — Check (c). 80 is outside the range. The data contain no information about third exam scores that high, so the prediction is not justified — even though the equation will happily return a number.
Answer: (a) and (b) are justified; (c) is not, because 80 lies outside the domain of the sample x-values.
11.3.7 Residuals Plots
A residuals plot can be used to help determine if a set of \((x, y)\) data is linearly correlated. For each data point used to create the correlation line, a residual \(y - \hat{y}\) can be calculated. A residuals plot shows the explanatory variable \(x\) on the horizontal axis and the residual for that value on the vertical axis, and is often shown together with a scatter plot of the data.
While a scatter plot of the data should resemble a straight line, a residuals plot should appear random, with no pattern and no outliers. It should also show constant error variance, meaning the residuals should not consistently increase or decrease as the explanatory variable \(x\) increases.
A residuals plot can be created using StatCrunch or a TI calculator. A box plot of the residuals is also helpful to verify that there are no outliers in the data. By observing the scatter plot of the data, the residuals plot, and the box plot of residuals, together with the linear correlation coefficient, we can usually determine if it is reasonable to conclude that the data are linearly correlated.
A shop owner uses a straight-line regression to estimate the number of ice cream cones sold in a day from the temperature at noon, using the nine days in Table 11.3.3. The regression line is \(\hat{y} = -567.48 + 9.78x\). Compute the residual for the 61°F day and for the 75°F day, and say what a residuals plot of all nine should look like if the line is appropriate.
| Temperature °F | Ice cream cones sold |
|---|---|
| 70 | 105 |
| 85 | 240 |
| 65 | 49 |
| 72 | 147 |
| 80 | 231 |
| 61 | 38 |
| 75 | 193 |
| 78 | 196 |
| 68 | 89 |
Solution — two residuals, and what the plot should show
Step 1 — Predict at 61°F.
$$ \hat{y} = -567.48 + 9.78(61) \approx 29.0 $$Step 2 — Compute that residual. The observed value is 38, so
$$ \varepsilon = 38 - 29.0 = 9.0 $$a positive residual, meaning the line underestimates sales on that day.
Step 3 — Predict at 75°F.
$$ \hat{y} = -567.48 + 9.78(75) \approx 165.9 $$Step 4 — Compute that residual. The observed value is 193, so
$$ \varepsilon = 193 - 165.9 = 27.1 $$also positive, and the largest miss of the nine.
Step 5 — Say what the residuals plot should look like. If a straight line is appropriate, the nine residuals plotted against temperature should appear random — no rising or falling trend, no curve, no outliers, and roughly constant spread across the temperature range.
Answer: Residuals of about \(+9.0\) at 61°F and \(+27.1\) at 75°F. An appropriate line produces a residuals plot with no pattern and constant spread.
One residuals plot shows points scattered with no pattern and about the same spread all the way across. Another shows residuals that are small on the left and fan out steadily to the right. What does each say about the regression?
Solution — reading pattern and spread separately
Step 1 — Read the first plot. No pattern and constant spread is exactly what a residuals plot should look like. It gives no reason to doubt that a straight line is appropriate.
Step 2 — Read the second plot's pattern. There is no systematic curve, so the shape of the relationship is not obviously wrong.
Step 3 — Read the second plot's spread. The residuals fan out as \(x\) increases, so the error variance is not constant — the requirement that residuals "should not consistently increase as \(x\) increases" fails.
Step 4 — State the consequence. The line may still describe the average trend, but its predictions are far less reliable at large \(x\) than at small \(x\), and any single summary of prediction error will understate the uncertainty at the high end.
Answer: The first supports the linear fit. The second violates constant error variance, so predictions are much less trustworthy at large values of \(x\).
11.3.8 The Correlation Coefficient r
The correlation coefficient, \(r\), developed by Karl Pearson in the early 1900s, is numerical and provides a measure of strength and direction of the linear association between the independent variable \(x\) and the dependent variable \(y\). It is calculated as
$$ r = \frac{n\Sigma(xy) - (\Sigma x)(\Sigma y)}{\sqrt{[n\Sigma x^{2} - (\Sigma x)^{2}][n\Sigma y^{2} - (\Sigma y)^{2}]}} $$where \(n\) = the number of data points.
What the VALUE of \(r\) tells us:
- The value of \(r\) is always between \(-1\) and \(+1\): \(-1 \leq r \leq 1\).
- The size of the correlation \(r\) indicates the strength of the linear relationship between \(x\) and \(y\). Values of \(r\) close to \(-1\) or to \(+1\) indicate a stronger linear relationship.
- If \(r = 0\) there is likely no linear correlation. It is important to view the scatterplot, however, because data that exhibit a curved or horizontal pattern may have a correlation of 0.
- If \(r = 1\), there is perfect positive correlation. If \(r = -1\), there is perfect negative correlation. In both these cases, all of the original data points lie on a straight line. Of course, in the real world, this will not generally happen.
What the SIGN of \(r\) tells us:
- A positive value of \(r\) means that when \(x\) increases, \(y\) tends to increase and when \(x\) decreases, \(y\) tends to decrease (positive correlation).
- A negative value of \(r\) means that when \(x\) increases, \(y\) tends to decrease and when \(x\) decreases, \(y\) tends to increase (negative correlation).
- The sign of \(r\) is the same as the sign of the slope, \(b\), of the best-fit line.
We say "correlation does not imply causation." The ice cream data are a fair example: warm days and cone sales travel together, and it is easy to tell a causal story. But the same arithmetic would report a strong correlation between cone sales and sunburn cases, with neither causing the other. \(r\) measures how tightly two columns of numbers move together, and nothing more.
Definition 11.3.7 — Correlation Coefficient: Three side-by-side scatterplots illustrate the correlation coefficient.
Besides looking at the scatter plot and seeing that a line seems reasonable, how can you tell if the line is a good predictor? Use the correlation coefficient as another indicator, besides the scatterplot, of the strength of the relationship between \(x\) and \(y\).
For the third-exam/final-exam data, \(n = 11\), \(\Sigma x = 761\), \(\Sigma y = 1765\), \(\Sigma xy = 122{,}500\), \(\Sigma x^{2} = 52{,}729\) and \(\Sigma y^{2} = 287{,}529\). Compute \(r\), then state what its value and its sign tell you.
Solution — substituting into the correlation formula
Step 1 — Compute the numerator.
$$ n\Sigma(xy) - (\Sigma x)(\Sigma y) = 11(122{,}500) - (761)(1765) = 1{,}347{,}500 - 1{,}343{,}165 = 4335 $$Step 2 — Compute the first bracket.
$$ n\Sigma x^{2} - (\Sigma x)^{2} = 11(52{,}729) - 761^{2} = 580{,}019 - 579{,}121 = 898 $$Step 3 — Compute the second bracket.
$$ n\Sigma y^{2} - (\Sigma y)^{2} = 11(287{,}529) - 1765^{2} = 3{,}162{,}819 - 3{,}115{,}225 = 47{,}594 $$Step 4 — Combine.
$$ r = \frac{4335}{\sqrt{(898)(47{,}594)}} = \frac{4335}{\sqrt{42{,}739{,}412}} \approx \frac{4335}{6537.5} \approx 0.663 $$Step 5 — Interpret the sign. Positive, so as third exam scores increase, final exam scores tend to increase. This matches the positive slope \(b = 4.83\), as it always must.
Step 6 — Interpret the value. At 0.663, \(r\) sits between 0 and 1 but not especially close to 1, indicating a moderate positive linear relationship rather than a strong one.
Answer: \(r \approx 0.663\) — a moderate positive linear relationship, with a sign matching the positive slope.
A data set produces \(r = 0\). A student concludes that \(x\) and \(y\) are unrelated. Explain why that conclusion may be wrong.
Solution — r measures LINEAR association only
Step 1 — State what \(r = 0\) actually reports. There is likely no linear correlation between \(x\) and \(y\).
Step 2 — Give the case that breaks the conclusion. Data exhibiting a curved pattern may have a correlation of 0. A relationship that rises and then falls symmetrically has no net linear trend, so \(r\) can come out near zero while the two variables are tightly related.
Step 3 — Name the second case. A horizontal pattern also gives \(r = 0\), which — as Section 11.2 established — genuinely does mean no relationship. So \(r = 0\) covers two very different situations.
Step 4 — State the remedy. It is important to view the scatterplot. Only the plot distinguishes a curve from a formless cloud, and \(r\) alone cannot.
Answer: \(r = 0\) rules out a linear relationship, not every relationship. A strong curved association can produce \(r = 0\), which is why the scatter plot must always be examined.
11.3.9 The Coefficient of Determination
The variable \(r^{2}\) is called the coefficient of determination and is the square of the correlation coefficient, but is usually stated as a percent rather than in decimal form.
It has an interpretation in the context of the data:
- \(r^{2}\), when expressed as a percent, represents the percent of variation in the dependent (predicted) variable \(y\) that can be explained by variation in the independent (explanatory) variable \(x\) using the regression (best-fit) line.
- \(1 - r^{2}\), when expressed as a percentage, represents the percent of variation in \(y\) that is NOT explained by variation in \(x\) using the regression line. This can be seen as the scattering of the observed data points about the regression line.
Consider the third exam/final exam example:
- The line of best fit is \(\hat{y} = -173.51 + 4.83x\)
- The correlation coefficient is \(r = 0.6631\)
- The coefficient of determination is \(r^{2} = 0.6631^{2} = 0.4397\)
Interpretation of \(r^{2}\) in the context of this example:
- Approximately 44% of the variation (0.4397 is approximately 0.44) in the final-exam grades can be explained by the variation in the grades on the third exam, using the best-fit regression line.
- Therefore, approximately 56% of the variation (\(1 - 0.44 = 0.56\)) in the final exam grades can NOT be explained by the variation in the grades on the third exam, using the best-fit regression line. This is seen as the scattering of the points about the line.
An \(r\) of 0.663 sounds like a solid relationship until it is squared: 0.44. Fewer than half the differences in final exam scores are accounted for by differences in third exam scores, and the majority of the variation comes from everything else — study time, illness, how the particular questions fell. This is why \(r^{2}\) is often the more honest number to report. A correlation of 0.7 explains only about half the variation, and a correlation of 0.5 explains a quarter.
Definition 11.3.8 — Coefficient of Determination: Left panel: a scatter plot of the eleven pairs from Table 11.3.1 (third exam score versus final exam score) with the regression line ŷ = −173.51 +....
A regression of monthly heating cost on outside temperature gives \(r = -0.82\). Compute \(r^{2}\), interpret it as a percentage in context, and state what the remaining percentage represents.
Solution — interpreting r-squared in context
Step 1 — Square the correlation coefficient.
$$ r^{2} = (-0.82)^{2} = 0.6724 $$Step 2 — Express it as a percent. About 67%.
Step 3 — Interpret it in context. Approximately 67% of the variation in monthly heating cost can be explained by the variation in outside temperature, using the best-fit regression line.
Step 4 — Interpret the remainder. \(1 - 0.6724 = 0.3276\), so approximately 33% of the variation in heating cost is NOT explained by outside temperature. This shows up as the scattering of the observed points about the line, and reflects everything else that affects heating cost — house insulation, thermostat settings, how many people are home.
Step 5 — Note the sign disappeared. Squaring removed the negative sign, so \(r^{2}\) says nothing about direction. Only \(r\) or the slope tells you that heating cost falls as temperature rises.
Answer: \(r^{2} = 0.6724\); about 67% of the variation in heating cost is explained by temperature and about 33% is not.
Regression A has \(r = 0.9\) and regression B has \(r = 0.45\). Compute each \(r^{2}\) and state how much better A explains its data than B does.
Solution — why doubling r more than doubles the explanation
Step 1 — Compute both coefficients of determination.
$$ r^{2}_{A} = 0.9^{2} = 0.81 \qquad r^{2}_{B} = 0.45^{2} = 0.2025 $$Step 2 — Express as percentages. A explains about 81% of the variation in its \(y\); B explains about 20%.
Step 3 — Compare. A's correlation is exactly twice B's, but A explains four times as much of the variation, because squaring turns a doubling into a quadrupling.
Step 4 — State the unexplained portions. About 19% of A's variation is unexplained, against about 80% of B's.
Answer: \(r^{2}_{A} = 0.81\) and \(r^{2}_{B} = 0.2025\). Twice the correlation gives four times the explained variation.
Problem Set 11.3
Problem 1. Explain what \(\hat{y}\) means and how it differs from \(y\).
Solution
Step 1 — Define \(\hat{y}\): The symbol \(\hat{y}\), read "y hat," is the estimated value of \(y\) obtained by substituting an \(x\)-value into the regression line. It is the value the line predicts.
Step 2 — Contrast it with \(y\): The symbol \(y\) is the observed value of \(y\) from the actual data. For any \(x\) in the data set there is a real observed \(y\) and a predicted \(\hat{y}\), and they almost never agree.
Step 3 — Say why the distinction matters: Keeping the two symbols apart keeps two ideas apart: what actually happened (the data point) versus what the line claims (the point on the line directly above or below it). Their difference is precisely the residual.
Answer: \(\hat{y}\) is the predicted value of \(y\) from the regression line; \(y\) is the observed data value. They are generally not equal.
Problem 2. Define a residual and state what a positive residual says about the line at that point.
Solution
Step 1 — Define a residual: A residual (or "error"), denoted \(\varepsilon = y - \hat{y}\), is the difference between the observed value of \(y\) and the value predicted by the regression line at that same \(x\). Its absolute value measures the vertical distance between the actual data point and the predicted point on the line.
Step 2 — State what a positive residual means: A positive residual means \(y > \hat{y}\), so the observed data point lies above the line. In that case the line underestimates the actual value of \(y\) at that point.
Step 3 — Note what a residual is not: It is not an error in the sense of a mistake; real data scatter about any line, and a residual simply records how far one observation sits from the prediction.
Answer: A residual is \(y - \hat{y}\), the vertical distance between the observed point and the line. A positive residual means the point lies above the line and the line underestimates \(y\) there.
Problem 3. State the least-squares criterion, and explain why the residuals are squared before being added.
Solution
Step 1 — State the criterion: The least-squares criterion says the best-fit line is the one for which the sum of the squared errors (SSE),
$$ \sum_{i=1}^{n} \varepsilon_{i}^{2} = \sum_{i=1}^{n} (y_i - \hat{y}_i)^{2}, $$is minimized — made as small as possible. Any other line would have a higher SSE.
Step 2 — Explain why squaring matters (signs): Residuals come in both signs — positive above the line, negative below it. Adding them raw lets a large overestimate cancel a large underestimate, so a terrible line could total zero. Squaring removes the signs so every miss counts as a miss.
Step 3 — Explain why squaring matters (penalties): Squaring penalizes a big miss far more than several small ones: being off by 10 contributes \(10^2 = 100\), while being off by 2 five times contributes only \(5(2^2) = 20\). The criterion therefore prefers a line that avoids large errors even at the cost of more small ones.
Answer: The best-fit line minimizes the sum of squared residuals. Squaring removes cancellation between positive and negative errors and heavily penalizes large misses.
Problem 4. Write the regression equation and the formulas for \(a\) and \(b\).
Solution
Step 1 — Write the regression equation:
$$ \hat{y} = a + bx $$where \(a\) is the y-intercept and \(b\) is the slope.
Step 2 — Write the formula for the slope:
$$ b = \frac{\Sigma(x - \overline{x})(y - \overline{y})}{\Sigma(x - \overline{x})^{2}} $$where \(\overline{x}\) and \(\overline{y}\) are the sample means. Equivalently, \(b = r\left(\frac{s_y}{s_x}\right)\).
Step 3 — Write the formula for the intercept:
$$ a = \overline{y} - b\overline{x} $$This rearrangement expresses the fact that the best-fit line always passes through the point \((\overline{x}, \overline{y})\).
Answer: \(\hat{y} = a + bx\), with \(b = \dfrac{\Sigma(x - \overline{x})(y - \overline{y})}{\Sigma(x - \overline{x})^{2}}\) and \(a = \overline{y} - b\overline{x}\).
Problem 5. A regression line is \(\hat{y} = 12 + 3x\). Find the residual for the data point \((5, 30)\) and state its sign.
Solution
Step 1 — Compute the predicted value at \(x = 5\):
$$ \hat{y} = 12 + 3(5) = 12 + 15 = 27 $$Step 2 — Compute the residual:
$$ \varepsilon = y - \hat{y} = 30 - 27 = 3 $$Step 3 — State the sign and its meaning: The residual is positive, so the data point lies above the line and the line underestimates the actual \(y\)-value by 3 units.
Answer: The residual is \(+3\); the point lies above the line, which underestimated \(y\).
Problem 6. State the point through which every best-fit line must pass.
Solution
Step 1 — State the point: Every least-squares best-fit line passes through the point
$$ (\overline{x}, \overline{y}), $$the mean of the x-values paired with the mean of the y-values.
Step 2 — Say why: This follows directly from the intercept formula \(a = \overline{y} - b\overline{x}\). Substituting \(x = \overline{x}\) into the equation gives \(\hat{y} = a + b\overline{x} = \overline{y} - b\overline{x} + b\overline{x} = \overline{y}\).
Answer: Every best-fit line passes through \((\overline{x}, \overline{y})\).
Problem 7. A regression of weight on height gives a slope of 5.2 pounds per inch. Interpret the slope in a complete sentence.
Solution
Step 1 — Identify the variables and units: \(x\) is height in inches and \(y\) is weight in pounds; the slope is 5.2 pounds per inch.
Step 2 — Apply the definition of slope interpretation: The slope tells how the dependent variable changes for every one-unit increase in the independent variable, on average.
Step 3 — Write the sentence: For each additional inch of height, weight increases by about 5.2 pounds, on average.
Step 4 — Note the role of "on average": The phrase is essential — the line describes the average tendency across the sample, not a promise about any particular individual.
Answer: For each additional inch of height, weight increases by about 5.2 pounds, on average.
Problem 8. A sample's x-values run from 20 to 45. Explain whether the line should be used to predict \(y\) at \(x = 50\), and why.
Solution
Step 1 — State the domain: The sample x-values run from 20 to 45, so predictions are justified only within that range.
Step 2 — Compare \(x = 50\) to the domain: Since 50 is greater than 45, it lies outside the domain of the sample x-values.
Step 3 — Explain why the prediction should not be made: The line was fitted to what happens between 20 and 45, and the data contain no evidence about how the relationship behaves beyond 45. Extrapolation assumes the linear pattern continues unchanged, which the data cannot confirm — and a straight line extended far enough can produce impossible values.
Answer: No. Because 50 lies outside the sample's x-range of 20 to 45, using the line there is extrapolation and is not justified by these data.
Problem 9. Describe what a residuals plot should look like when a straight line is appropriate.
Solution
Step 1 — Describe the pattern: When a straight line is appropriate, the residuals plot should appear random, with no pattern — no rising or falling trend, no curve — and no outliers.
Step 2 — Describe the spread: It should also show constant error variance: the residuals should not consistently increase or decrease as the explanatory variable \(x\) increases, but should have roughly the same vertical spread all the way across.
Step 3 — Note supporting checks: A box plot of the residuals helps verify there are no outliers, and together with the scatter plot and correlation coefficient it supports the conclusion that the data are linearly correlated.
Answer: A good residuals plot looks random: no pattern, no outliers, and constant spread across all values of \(x\).
Problem 10. A regression gives \(r = -0.6\). State the sign of the slope, compute \(r^{2}\), and interpret it as a percentage.
Solution
Step 1 — State the sign of the slope: The sign of \(r\) always matches the sign of the slope \(b\). Since \(r = -0.6\) is negative, the slope is negative too.
Step 2 — Compute \(r^{2}\):
$$ r^{2} = (-0.6)^{2} = 0.36 $$Step 3 — Interpret as a percentage: Approximately 36% of the variation in the dependent variable \(y\) can be explained by variation in the independent variable \(x\) using the regression line.
Step 4 — Interpret the remainder: About \(1 - 0.36 = 64\%\) of the variation in \(y\) is NOT explained by \(x\); this appears as the scattering of points about the line. Note also that squaring removed the sign, so only \(r\) (or the slope) reveals the direction of the relationship.
Answer: The slope is negative; \(r^{2} = 0.36\), meaning about 36% of the variation in \(y\) is explained by \(x\) (and about 64% is not).
Problem 11. A data set has \(r = 0\) but a clearly curved scatter plot. Explain what \(r = 0\) does and does not tell you here.
Solution
Step 1 — State what \(r = 0\) does tell you: There is likely no linear correlation between \(x\) and \(y\). That is all \(r\) measures — strength and direction of a straight-line association.
Step 2 — State what it does not tell you: It does not rule out other kinds of relationships. Data exhibiting a curved pattern may have a correlation of 0: a relationship that rises and then falls symmetrically has no net linear trend, so \(r\) comes out near zero while the variables are tightly related.
Step 3 — Name the remedy: This is exactly why the scatter plot must be viewed alongside \(r\). Here the clearly curved plot shows a strong nonlinear relationship that \(r = 0\) completely fails to capture.
Answer: \(r = 0\) rules out a linear relationship only. With a curved scatter plot, the variables are strongly related non-linearly, and \(r\) alone gives a misleading picture.
Problem 12. Explain the difference between what \(r\) reports and what \(r^{2}\) reports.
Solution
Step 1 — State what \(r\) reports: The correlation coefficient \(r\) measures the strength and direction of the linear association between \(x\) and \(y\). Its value lies between \(-1\) and \(+1\), values near \(\pm 1\) indicate stronger linear relationships, and its sign matches the sign of the slope.
Step 2 — State what \(r^{2}\) reports: The coefficient of determination \(r^{2}\), expressed as a percent, represents the percent of the variation in \(y\) that is explained by variation in \(x\) using the regression line. Correspondingly, \(1 - r^{2}\) is the percent of variation in \(y\) not explained.
Step 3 — Highlight the practical difference: Squaring makes moderate correlations look considerably weaker: an \(r\) of 0.663 corresponds to \(r^{2} = 0.44\), so fewer than half the differences in \(y\) are accounted for by \(x\). Also, squaring removes the sign, so \(r^{2}\) says nothing about direction — only \(r\) does.
Answer: \(r\) reports the strength and direction of the linear relationship; \(r^{2}\) reports the percentage of variation in \(y\) explained by \(x\) through the regression line.
Key Terms
line of best fit (least-squares line) — the single line selected by minimising the sum of squared errors.
\(\hat{y}\) ("y hat") — the estimated value of \(y\) obtained from the regression line; not generally equal to the observed \(y\).
residual (\(\varepsilon\)) — \(y - \hat{y}\), the vertical distance between an observed point and the line; positive above the line (the line underestimates), negative below it (the line overestimates).
sum of squared errors (SSE) — the total obtained by squaring every residual and adding; the quantity least-squares regression minimises.
linear regression — the process of fitting the best-fit line.
interpretation of the slope — how \(y\) changes for every one-unit increase in \(x\), on average.
domain of the sample x-values — the range of observed \(x\); predictions outside it are not justified by the data.
residuals plot — residuals against \(x\); should appear random, with no pattern, no outliers, and constant error variance.
correlation coefficient (\(r\)) — a measure of the strength and direction of linear association, always between \(-1\) and \(1\); its sign matches the sign of the slope.
coefficient of determination (\(r^{2}\)) — the percent of variation in \(y\) explained by variation in \(x\) using the regression line; \(1 - r^{2}\) is the percent not explained.