Foundations for Inference

OpenIntro Statistics · Ch. 5 · openintro.org/book/os

Statistical inference draws conclusions about a population from a sample. Two main tools: confidence intervals estimate a parameter with a margin of error, and hypothesis tests evaluate whether evidence supports a claim. Both rest on the sampling distribution.

Keep the population fixed and watch how its sample means vary. Read the guided introduction →

One sample becomes one mean

Population shapeObservations per sample

The population stays this shape

Dashed line: population mean = 20.3.

Read the chart as a table

The population stays this shape
Value range	Count
0.0–2.5	40
2.5–5.0	8
5.0–7.5	5
7.5–10.0	4
10.0–12.5	3
12.5–15.0	3
15.0–17.5	2
17.5–20.0	2
20.0–22.5	2
22.5–25.0	2
25.0–27.5	2
27.5–30.0	2
30.0–32.5	1
32.5–35.0	1
35.0–37.5	2
37.5–40.0	1
40.0–42.5	1
42.5–45.0	1
45.0–47.5	2
47.5–50.0	1
50.0–52.5	1
52.5–55.0	1
55.0–57.5	1
57.5–60.0	1
60.0–62.5	0
62.5–65.0	1
65.0–67.5	1
67.5–70.0	1
70.0–72.5	1
72.5–75.0	1
75.0–77.5	0
77.5–80.0	1
80.0–82.5	1
82.5–85.0	1
85.0–87.5	0
87.5–90.0	1
90.0–92.5	1
92.5–95.0	0
95.0–97.5	1
97.5–100.0	1

Each range includes its lower end and excludes its upper end, except the last range, which includes both.

Draw independently with replacement: each observation uses the same population again. Average the values, and place that one mean in the lower chart.

0 sample means recorded.

Means of samples of 1

Dashed line: population mean = 20.3.

Read the chart as a table

Means of samples of 1
Value range	Count
0.0–1.3	0
1.3–2.5	0
2.5–3.8	0
3.8–5.0	0
5.0–6.3	0
6.3–7.5	0
7.5–8.8	0
8.8–10.0	0
10.0–11.3	0
11.3–12.5	0
12.5–13.8	0
13.8–15.0	0
15.0–16.3	0
16.3–17.5	0
17.5–18.8	0
18.8–20.0	0
20.0–21.3	0
21.3–22.5	0
22.5–23.8	0
23.8–25.0	0
25.0–26.3	0
26.3–27.5	0
27.5–28.8	0
28.8–30.0	0
30.0–31.3	0
31.3–32.5	0
32.5–33.8	0
33.8–35.0	0
35.0–36.3	0
36.3–37.5	0
37.5–38.8	0
38.8–40.0	0
40.0–41.3	0
41.3–42.5	0
42.5–43.8	0
43.8–45.0	0
45.0–46.3	0
46.3–47.5	0
47.5–48.8	0
48.8–50.0	0
50.0–51.3	0
51.3–52.5	0
52.5–53.8	0
53.8–55.0	0
55.0–56.3	0
56.3–57.5	0
57.5–58.8	0
58.8–60.0	0
60.0–61.3	0
61.3–62.5	0
62.5–63.8	0
63.8–65.0	0
65.0–66.3	0
66.3–67.5	0
67.5–68.8	0
68.8–70.0	0
70.0–71.3	0
71.3–72.5	0
72.5–73.8	0
73.8–75.0	0
75.0–76.3	0
76.3–77.5	0
77.5–78.8	0
78.8–80.0	0
80.0–81.3	0
81.3–82.5	0
82.5–83.8	0
83.8–85.0	0
85.0–86.3	0
86.3–87.5	0
87.5–88.8	0
88.8–90.0	0
90.0–91.3	0
91.3–92.5	0
92.5–93.8	0
93.8–95.0	0
95.0–96.3	0
96.3–97.5	0
97.5–98.8	0
98.8–100.0	0

Each range includes its lower end and excludes its upper end, except the last range, which includes both.

All horizontal axes stay at 0–100. Each vertical axis counts its own observations. At most the latest 10,000 means are shown. Changing sample size clears the current means; changing population clears both runs.

Put a number on the narrowing

The population’s standard deviation is 27.17. For samples of 1, the standard error of the mean is 27.17 ÷ √1 = 27.17. The population’s spread stays the same; the estimates become steadier.

Point estimates and the sampling distribution

A point estimate is a single number computed from data (like the sample mean). Different samples give different estimates. The distribution of all possible estimates is the sampling distribution. By the 🎰 Central Limit Theorem, means of independent observations from the same distribution with finite, positive variance are approximately normal for sufficiently large n. The population itself need not be normal.

Scheme

; Point estimate: sample mean as estimate of population mean
; Standard error: SD / sqrt(n)

(define sample '(67 72 74 68 71 69 73 70 75 66))

(define (mean lst)
  (* 1.0 (/ (apply + lst) (length lst))))

(define (std lst)
  (let ((m (mean lst)))
    (sqrt (/ (apply + (map (lambda (x) (* (- x m) (- x m))) lst))
             (- (length lst) 1)))))

(define x-bar (mean sample))
(define s (std sample))
(define n (length sample))
(define se (/ s (sqrt n)))

(display "Sample mean: ") (display x-bar) (newline)
(display "Sample SD:   ") (display s) (newline)
(display "Sample size: ") (display n) (newline)
(display "Std error:   ") (display se)

Confidence intervals

A confidence interval combines a point estimate with a margin of error. For independent observations from a normal population with unknown standard deviation, a 95% t interval is x̄ ± t* × s/√n. Here n = 10, so there are 9 degrees of freedom and t* ≈ 2.262. Under those assumptions, the procedure covers the population mean in 95% of repeated samples. See the interval experiment for the coverage idea.

Scheme

; 95% confidence interval for the mean
; CI = x-bar +/- t* * SE
; Independent normal observations; sigma unknown. Keep n = 10 for this cutoff.

(define sample '(67 72 74 68 71 69 73 70 75 66))

(define (mean lst)
  (* 1.0 (/ (apply + lst) (length lst))))
(define (std lst)
  (let ((m (mean lst)))
    (sqrt (/ (apply + (map (lambda (x) (* (- x m) (- x m))) lst))
             (- (length lst) 1)))))

(define x-bar (mean sample))
(define se (/ (std sample) (sqrt (length sample))))
(define t-star 2.262157)  ; 95% confidence, df = 9 (n = 10)

(define margin (* t-star se))
(define ci-lower (- x-bar margin))
(define ci-upper (+ x-bar margin))

(display "Point estimate: ") (display x-bar) (newline)
(display "Margin of error: ") (display margin) (newline)
(display "95% CI: (")
(display ci-lower) (display ", ")
(display ci-upper) (display ")") (newline)
(newline)
(display "We are 95% confident the population mean") (newline)
(display "falls between ") (display ci-lower)
(display " and ") (display ci-upper)

; 95% confidence interval for the mean
; CI = x-bar +/- t* * SE
; Independent normal observations; sigma unknown. Keep n = 10 for this cutoff.

(define sample '(67 72 74 68 71 69 73 70 75 66))

(define (mean lst)
  (* 1.0 (/ (apply + lst) (length lst))))
(define (std lst)
  (let ((m (mean lst)))
    (sqrt (/ (apply + (map (lambda (x) (* (- x m) (- x m))) lst))
             (- (length lst) 1)))))

(define x-bar (mean sample))
(define se (/ (std sample) (sqrt (length sample))))
(define t-star 2.262157)  ; 95% confidence, df = 9 (n = 10)

(define margin (* t-star se))
(define ci-lower (- x-bar margin))
(define ci-upper (+ x-bar margin))

(display "Point estimate: ") (display x-bar) (newline)
(display "Margin of error: ") (display margin) (newline)
(display "95% CI: (")
(display ci-lower) (display ", ")
(display ci-upper) (display ")") (newline)
(newline)
(display "We are 95% confident the population mean") (newline)
(display "falls between ") (display ci-lower)
(display " and ") (display ci-upper)

Hypothesis testing and p-values

Start with a null hypothesis H₀, such as a specified population mean. The p-value is the probability, under the null model, of a test statistic at least as extreme as observed according to the chosen test. Set a decision threshold α before seeing the data. The example below uses a two-sided t test with 9 degrees of freedom and independent normal observations; its critical value assumes the ten observations shown.

Scheme

; Hypothesis test: is the population mean = 70?
; H0: mu = 70
; HA: mu != 70 (two-tailed)

(define sample '(67 72 74 68 71 69 73 70 75 66))
(define mu-0 70)  ; null hypothesis value

(define (mean lst)
  (* 1.0 (/ (apply + lst) (length lst))))
(define (std lst)
  (let ((m (mean lst)))
    (sqrt (/ (apply + (map (lambda (x) (* (- x m) (- x m))) lst))
             (- (length lst) 1)))))

(define x-bar (mean sample))
(define se (/ (std sample) (sqrt (length sample))))

; Test statistic: t = (x-bar - mu0) / SE
(define t-stat (/ (- x-bar mu-0) se))

(display "Sample mean:    ") (display x-bar) (newline)
(display "Null value:     ") (display mu-0) (newline)
(display "Std error:      ") (display se) (newline)
(display "t-statistic:    ") (display t-stat) (newline)
(newline)

; Decision at alpha = 0.05
; Reject if |t| > 2.262157
(define alpha 0.05)
(display "Critical value: 2.262157") (newline)
(display "|t| = ") (display (abs t-stat)) (newline)
(display "Reject H0? ")
(display (if (> (abs t-stat) 2.262157) "Yes" "No"))

; Hypothesis test: is the population mean = 70?
; H0: mu = 70
; HA: mu != 70 (two-tailed)

(define sample '(67 72 74 68 71 69 73 70 75 66))
(define mu-0 70)  ; null hypothesis value

(define x-bar (mean sample))
(define se (/ (std sample) (sqrt (length sample))))

; Test statistic: t = (x-bar - mu0) / SE
(define t-stat (/ (- x-bar mu-0) se))

(display "Sample mean:    ") (display x-bar) (newline)
(display "Null value:     ") (display mu-0) (newline)
(display "Std error:      ") (display se) (newline)
(display "t-statistic:    ") (display t-stat) (newline)
(newline)

; Decision at alpha = 0.05
; Reject if |t| > 2.262157
(define alpha 0.05)
(display "Critical value: 2.262157") (newline)
(display "|t| = ") (display (abs t-stat)) (newline)
(display "Reject H0? ")
(display (if (> (abs t-stat) 2.262157) "Yes" "No"))

Significance level and errors

The significance level α is the threshold for rejecting H₀. Type I error: rejecting H₀ when it is true (probability at most α for a valid level-α test). Type II error: failing to reject H₀ when it is false (probability = β). Power = 1 - β: the probability of correctly detecting a real effect.

Scheme

; Error types in hypothesis testing

;                 H0 true       H0 false
; Reject H0:     Type I (a)    Correct (power)
; Fail to reject: Correct      Type II (b)

(define alpha 0.05)
(display "Significance level (alpha): ") (display alpha) (newline)
(display "Type I error is controlled at alpha = ") (display alpha) (newline)
(newline)

; If the true effect is small, power is low
; Power increases with: larger sample, larger effect, larger alpha
(define beta 0.20)  ; typical target
(define power (- 1 beta))
(display "P(Type II error) = beta = ") (display beta) (newline)
(display "Power = 1 - beta = ") (display power) (newline)
(newline)

; Trade-off: lowering alpha reduces Type I errors
; but increases Type II errors (lowers power)
(display "Lower alpha -> fewer false positives,") (newline)
(display "             -> more false negatives")

Neighbors

Foundations (Wikipedia)

① Confidence interval
① P-value
① Hypothesis testing
① Central Limit Theorem
🔬 Fisher 1935 — the inventor of significance testing and p-values
🔬 Ioannidis 2005 — why the tools in this chapter are often misused
🤖 ML Ch.1 — statistical learning as inference from data

Translation notes

OpenIntro builds inference from the sampling distribution of the sample proportion, then generalizes. We use the sample mean for clarity. The original covers decision errors with medical testing analogies and uses simulation to build intuition for sampling distributions. With many degrees of freedom, t critical values approach normal critical values. The examples here use the t-distribution because the population standard deviation is estimated from ten observations; Ch. 7 develops this method further.

Want the full treatment? Read OpenIntro Statistics, Ch. 5.

← Distributions by june.kim Inference for Proportions →