📖 检验统计量、p 值与显著性水平

Quantitative Methods — Hypothesis Testing · Lesson 2

1. Recap from L131: From Hypotheses to Decisions

In L131, we learned how to set up hypotheses:

H₀: μ = μ₀ (status quo hypothesis, includes equality)
Hₐ: μ ≠ μ₀ / μ > μ₀ / μ < μ₀ (research claim, no equality)

But setting hypotheses is only the first step. The real question is:

🎯 How far must the sample data deviate from H₀ before we can say "this is not coincidence — H₀ really has a problem"?

This lesson answers that question through three core concepts: 1. Test Statistic — quantifies the "degree of deviation" 2. p-value — quantifies the "probability of coincidence" 3. Significance Level α — the "threshold" standard

2. Test Statistic

2.1 Definition

A test statistic is a value calculated from sample data that measures how much the sample result differs from what H₀ predicts.

2.2 Intuitive Understanding

Think of archery:

2.3 General Formula

Test Statistic = (Sample Statistic − Hypothesized Value) / Standard Error

2.4 Common Test Statistics

📌 CFA Level I focuses on z-test and t-test. χ² and F require only basic conceptual understanding.

2.5 Core Intuition

Test Statistic ≈ Signal / Noise

Signal = x̄ − μ₀ (how far the sample mean is from H₀)
Noise = Standard Error (normal range of sampling variation)

3. Significance Level α

3.1 Definition

The significance level α is the maximum probability of erroneously rejecting H₀ that the researcher is willing to tolerate, set before the test. It is the threshold for "strong enough evidence."

3.2 Common Values

3.3 α Defines the Rejection Region

Two-tailed test (Hₐ: μ ≠ μ₀):
    ┌──────────┬──────────┬──────────┐
    │ Reject   │  Do not  │  Reject  │
    │  α/2     │  reject  │   α/2    │
    └──────────┴──────────┴──────────┘
          −z_crit              +z_crit

Right-tailed test (Hₐ: μ > μ₀):
    ┌────────────────────┬──────────┐
    │   Do not reject H₀ │  Reject  │
    │                    │    α     │
    └────────────────────┴──────────┘
                         +z_crit

3.4 Critical Values (High-Frequency CFA Topic)

🔥 Memorize these three z-values — they are NOT provided on the exam!

4. p-value

4.1 Definition

The p-value is the probability of observing the current sample result (or a more extreme one), given that H₀ is true.

4.2 Plain-English Translation

p-value = "If H₀ is actually correct, what is the probability that we get a sample this extreme purely by chance?"

4.3 Decision Rule

p < α  →  Reject H₀ ✅ (statistically significant)
p ≥ α  →  Do not reject H₀ (not statistically significant)

🎯 In one line: The smaller the p-value, the stronger the evidence against H₀.

4.4 p-value ≠ Probability that H₀ is true!

This is the CFA exam's favorite trap:

❌ Wrong: "p = 0.03 means there is a 3% probability H₀ is true"
✅ Right: "p = 0.03 means: if H₀ is true, there is only a 3% chance of seeing data like this"

❌ Wrong: "p = 0.03 means we are 97% confident H₀ is false"
✅ Right: "The data contradicts H₀ to a degree of 0.03, below 0.05, so we reject H₀"

🔑 Core difference: p-value = P(Data | H₀), NOT P(H₀ | Data). These are mathematically different!

5. Computing p-values for Three Test Types

5.1 Two-Tailed Test (Hₐ: μ ≠ μ₀)

p-value = 2 × P(Z > |z_obs|)

Sum of both tail areas beyond the absolute value of the test statistic

Example: z = 2.1, table gives P(Z > 2.1) = 0.0179 → p = 2 × 0.0179 = 0.0358

5.2 Right-Tailed Test (Hₐ: μ > μ₀)

p-value = P(Z > z_obs)

Example: z = 2.1 → p = P(Z > 2.1) = 0.0179

5.3 Left-Tailed Test (Hₐ: μ < μ₀)

p-value = P(Z < z_obs)  (take left tail when z_obs is negative)

Example: z = −2.1 → p = P(Z < −2.1) = 0.0179

📌 Note: For the same |z| = 2.1, the two-tailed p-value is twice the one-tailed p-value!

6. Real-World Case: Portfolio Performance Test

Background

A fund manager claims their portfolio generates positive alpha (excess return). Given: - 36 months of historical strategy data - Sample mean monthly alpha = 0.25% - Sample standard deviation s = 0.60% - n = 36 - α = 0.05 (5% significance level)

Step 1: Set Hypotheses

H₀: alpha ≤ 0 (no positive excess return)
Hₐ: alpha > 0 (positive excess return exists) → right-tailed test

Step 2: Compute Test Statistic (t-statistic, σ unknown)

SE = s / √n = 0.60% / √36 = 0.60% / 6 = 0.10%

t = (x̄ − μ₀) / SE = (0.25% − 0) / 0.10% = 2.50

Step 3: Compute p-value

df = n − 1 = 35
t = 2.50 with df = 35
→ one-tailed p ≈ 0.0087

Step 4: Make Decision

p = 0.0087 < α = 0.05
→ Reject H₀ ✅
→ Conclusion: Sufficient evidence of positive alpha

Step 5: Investment Interpretation

7. Common p-value Misconceptions (High-Frequency CFA Topic)

Misconception 1: Small p-value = Large effect size ❌

p-value is affected by sample size. With a large enough sample, even trivial differences can produce small p-values.

💡 Is 0.01% alpha economically meaningful? Statistical significance ≠ Economic significance!

Misconception 2: p > 0.05 means H₀ is true ❌

p > 0.05 only means "insufficient evidence to overturn H₀" — not that H₀ is true. The sample might just be too small (insufficient test power).

Misconception 3: p < 0.05 means the result is important ❌

Statistical significance ≠ practical importance. A 0.001% monthly return could be statistically significant (with huge n) but economically meaningless.

Misconception 4: p-value = Probability that H₀ is true ❌

This is the classic confusion between frequentist and Bayesian thinking. p-value = P(Data | H₀), not P(H₀ | Data).

8. Choosing α: Rigor vs. Practicality

📌 Type I / Type II errors will be discussed in detail in L133.

Industry Conventions

9. Decision Flowchart

      Formulate H₀ / Hₐ
              ↓
        Choose α (e.g., 0.05)
              ↓
        Collect sample data
              ↓
    Compute test statistic (z or t)
              ↓
         Compute p-value
              ↓
       ┌──────────────┐
       │   p < α ?    │
       └──────┬───────┘
         ↙        ↘
       YES         NO
        ↓           ↓
    Reject H₀   Do not reject H₀
   (significant) (not significant)
        ↓           ↓
    Support Hₐ   Insufficient evidence

10. Practice Questions

📝 Part A: Core Concepts

Q1. The role of a test statistic is to:

A. Directly determine whether H₀ is true B. Quantify the degree of deviation between the sample result and H₀ C. Determine the significance level α D. Calculate the sample mean

Q2. Which of the following is correct about the p-value?

A. The p-value is the probability that H₀ is true B. The p-value is the probability that Hₐ is true C. The p-value is the probability of observing the current (or more extreme) sample result, given that H₀ is true D. The larger the p-value, the stronger the evidence to reject H₀

Q3. With α = 0.05, which p-value leads to rejection of H₀?

A. p = 0.10 B. p = 0.06 C. p = 0.049 D. p = 0.50

📝 Part B: Calculations

Q4. Given H₀: μ = 100, x̄ = 106, σ = 15, n = 25. Compute the z-statistic:

A. z = 1.00 B. z = 1.50 C. z = 2.00 D. z = 2.50

Q5. Continuing from Q4, the two-tailed p-value is closest to:

A. 0.16 B. 0.05 C. 0.025 D. 0.01

Q6. At α = 0.05, the two-tailed critical z-value is:

A. ±1.28 B. ±1.645 C. ±1.96 D. ±2.576

📝 Part C: Integrated Judgment

Q7. An analyst obtains p = 0.02. Which statement is correct?

A. There is a 2% probability that H₀ is true B. If H₀ is true, there is only a 2% chance of seeing such data C. There is a 98% probability that Hₐ is true D. We should accept H₀

Q8. Which statement about α is WRONG?

A. α is the maximum tolerable probability of wrongly rejecting H₀, set before the test B. The smaller α is, the harder it is to reject H₀ C. α should be chosen after observing the data D. α = 0.05 means accepting at most a 5% Type I error rate

Q9. In a t-test, as sample size increases:

A. The standard error increases B. The t-distribution approaches the standard normal distribution C. The degrees of freedom decrease D. The p-value always decreases

Q10. A backtest yields p = 0.001 for a trading strategy. The most reasonable interpretation is:

A. The strategy is 99.9% certain to be effective B. The strategy's returns are very substantial C. If the strategy is truly ineffective, the probability of observing such an extreme result is only 0.1% D. You should immediately invest all your capital

11. Answers and Explanations

Part A: Core Concepts

Q1. Answer: B

Test statistic = (sample statistic − H₀ value) / SE. It quantifies deviation, not decision. Decisions require p-value or critical value comparison.

Q2. Answer: C

p-value = P(observed or more extreme result | H₀ true). A and B reverse the conditional probability. D reverses the direction (smaller p-value → stronger evidence).

Q3. Answer: C

p = 0.049 < α = 0.05 → reject H₀. p = 0.06 > 0.05 → do not reject.

Part B: Calculations

Q4. Answer: C

z = (106 − 100) / (15/√25) = 6 / (15/5) = 6 / 3 = 2.00

Q5. Answer: B (~0.05)

z = 2.00, P(Z > 2.00) = 0.0228. Two-tailed p = 2 × 0.0228 = 0.0456 ≈ 0.05.

Q6. Answer: C

α = 0.05, two-tailed → each tail α/2 = 0.025 → critical z = ±1.96 (must memorize!).

Part C: Integrated Judgment

Q7. Answer: B

p-value = P(Data | H₀), NOT P(H₀ | Data). p = 0.02 = if H₀ true, 2% chance of seeing such data.

Q8. Answer: C

α must be set before the test, not after observing the data. This is a fundamental principle of hypothesis testing — prevents "significance level arbitrage."

Q9. Answer: B

SE = s/√n, as n ↑ → SE ↓, not ↑ (A wrong). t-distribution → normal as df increases (B right). df = n−1 increases (C wrong). p-value may not decrease; depends on deviation (D wrong).

Q10. Answer: C

p = 0.001 = if the strategy is truly ineffective, only 0.1% chance of such an extreme result. A confuses p-value with P(H₀). B confuses statistical and economic significance. D is too absolute; significance does not guarantee profits.

12. Key Takeaways

13. Next Lesson: L133

Next up: Type I Error, Type II Error & Power of a Test — "What two kinds of mistakes can we make? How do we balance them? And what is test power?"

📊 Study Load: ~25 min reading + 10 practice questions

🏆 Congratulations! You've completed L132. See you tomorrow for L133!

CFA Level I · Quantitative Methods · Hypothesis Testing · Lesson 2 Generated for Ivan on 2026-08-07