📖 最小二乘法（OLS）

Quantitative Methods — Simple Linear Regression · Lesson 2

I. Where This Lesson Fits

🎯 L138 is the bridge from "reading the model" to "computing the model yourself" — last lesson you read a table, this lesson you do the math.

II. What Problem Are We Solving?

From last lesson: a regression line is Y = b0 + b1X, but we don't yet know the actual values of b0 and b1.

Core task of L138: given a set of data points (X1,Y1), (X2,Y2), ..., (Xn,Yn), find the "best" straight line.

Y ↑
 |      ·        ·      ← observed points
 |   ·      ·  ·   
 | ·    ╱      ← the line we're looking for
 |   ╱  ╱
 | ╱ ╱
 +──────────────→ X

What does "best" mean? — Make the overall vertical distances from all points to the line as small as possible.

III. Ordinary Least Squares (OLS)

3.1 The Core Idea

OLS finds the line that minimizes the sum of squared residuals.

• Residual = Actual value − Fitted value = Yi − Ŷi = Yi − (b0 + b1Xi)
• Sum of Squared Errors (SSE / SSR) = Σ(Yi − Ŷi)² = Σ ε̂i²

OLS objective function:

$$\min_{b_0, b_1} \sum_{i=1}^{n} \left(Y_i - b_0 - b_1 X_i\right)^2$$

3.2 Why Square?

Why squaring works: ① removes the sign; ② large errors get penalized more; ③ mathematically differentiable so a unique solution exists.

3.3 Where Does "Ordinary" Come From?

CFA Level 1 only tests OLS. Remember: "ordinary" = all observations treated equally.

IV. OLS Formulas (🔑 Must Memorize)

Setting the derivatives of SSE with respect to b0 and b1 to zero yields:

4.1 Slope b₁

$$b_1 = \frac{\sum(X_i - \bar{X})(Y_i - \bar{Y})}{\sum(X_i - \bar{X})^2} = \frac{\text{Cov}(X,Y)}{\text{Var}(X)}$$

4.2 Intercept b₀

$$b_0 = \bar{Y} - b_1 \bar{X}$$

Notation:

4.3 Two Key Properties

1. The regression line always passes through the mean point (X̄, Ȳ) — a direct consequence of the b₀ formula.
2. Residuals sum to zero: Σε̂i = 0 (OLS guarantees positive and negative residuals exactly balance).

📌 Memory aid: b₁ = covariance ÷ variance; b₀ = Ȳ − b₁X̄.

V. Complete Worked Example (Step by Step)

Data: Use 5 months of advertising spend (X, in $10k) to predict sales (Y, in $10k)

5.1 Compute the Means

• X̄ = (1+2+3+4+5)/5 = 15/5 = 3
• Ȳ = (2+4+5+4+10)/5 = 25/5 = 5

5.2 Compute Deviations

5.3 Plug Into the Formulas

$$b_1 = \frac{16}{10} = 1.6$$

$$b_0 = \bar{Y} - b_1\bar{X} = 5 - 1.6 \times 3 = 5 - 4.8 = 0.2$$

5.4 Result

$$\hat{Y} = 0.2 + 1.6X$$

Interpretation: each additional $10k of advertising is expected to raise sales by 1.6 ($10k); with zero advertising, baseline sales ≈ 0.2 ($10k).

VI. Residuals and Fitted Values (Understanding the Model's "Error")

Using Ŷ = 0.2 + 1.6X, compute each fitted value:

Verification: ① residuals sum to 0 ✅; ② SSE = 10.4, which is the smallest possible squared sum among all candidate lines — guaranteed by OLS.

VII. Standard Error of Estimate (SEE)

Measures the typical size of the model's prediction error:

$$SEE = \sqrt{\frac{SSE}{n - k - 1}} = \sqrt{\frac{\sum \hat{\varepsilon}_i^2}{n - 2}}$$

In simple linear regression the degrees of freedom = n − 2, because we already used the data to estimate two parameters (b0, b1).

Smaller SEE → points closer to the line → better model fit.

VIII. Practice Questions (10 Questions)

Basic Concepts (Q1–Q5)

Q1. OLS minimizes: A. Sum of residuals Σεi B. Sum of squared residuals Σεi² C. Sum of dependent variable ΣYi D. Sum of independent variable ΣXi

Q2. A residual is defined as: A. Fitted value minus actual value B. Actual value minus fitted value C. Actual value minus the mean D. Fitted value minus the mean

Q3. The OLS slope b₁ is computed as: A. Var(X)/Cov(X,Y) B. Cov(X,Y)/Var(X) C. Cov(X,Y)/Var(Y) D. Var(Y)/Cov(X,Y)

Q4. Which is always true for an OLS regression line? A. It passes through the origin (0,0) B. It passes through the mean point (X̄, Ȳ) C. The sum of squared residuals equals 0 D. The slope is always positive

Q5. "Ordinary" in OLS means: A. The model has only one independent variable B. All observations are weighted equally C. No assumptions are required D. The result is always significant

Computation & Application (Q6–Q8)

Q6. Given X̄=10, Ȳ=20, b₁=0.5, the intercept b₀ equals: A. 10 B. 15 C. 20 D. 25

Q7. A sample has Σ(Xi−X̄)(Yi−Ȳ)=40 and Σ(Xi−X̄)²=20. Then b₁ equals: A. 0.5 B. 2.0 C. 20 D. 40

Q8. A regression has SSE=30 with sample size n=8. The SEE for simple linear regression is: A. √(30/6) B. √(30/7) C. √(30/8) D. 30/6

Integrated Reasoning (Q9–Q10)

Q9. For the model Ŷ = 3 + 2X, when X=4 the actual Y=12. The residual is: A. −1 B. 1 C. 2 D. 11

Q10. Which statement about OLS residuals is correct? A. Residuals always sum to a positive value B. The sum of squared residuals is minimized, but residuals may not sum to 0 C. Residuals always sum to 0, and the sum of squared residuals is minimized D. Residuals always sum to n

IX. Answers & Explanations

X. CFA Level 1 Key Takeaways

📌 Next lesson L139: R² and the F-Test — how much of Y's variation does this line actually explain? And how do we test whether the model is useful at all?