Probability

OpenIntro Statistics · Ch. 3 · openintro.org/book/os

New to statistics? Start with Probability and Events, a short chapter you can experiment with.

Probability quantifies uncertainty. The rules are simple: probabilities are between 0 and 1, they add up for disjoint events, and conditional probability lets you update beliefs with new evidence. Bayes' theorem ties it all together.

Probability rules

For any event A: 0 ≤ P(A) ≤ 1. For disjoint events: P(A or B) = P(A) + P(B). For non-disjoint events: P(A or B) = P(A) + P(B) - P(A and B). The complement rule: P(not A) = 1 - P(A).

Scheme

; Probability rules
; P(A or B) = P(A) + P(B) - P(A and B)

(define p-a 0.4)    ; P(rain)
(define p-b 0.3)    ; P(traffic)
(define p-ab 0.12)  ; P(rain AND traffic)

; Addition rule (general)
(define p-a-or-b (+ p-a p-b (- p-ab)))
(display "P(rain OR traffic) = ")
(display p-a-or-b) (newline)

; Complement rule
(define p-not-a (- 1 p-a))
(display "P(no rain) = ")
(display p-not-a) (newline)

; If disjoint (mutually exclusive), P(A and B) = 0
(define p-heads 0.5)
(define p-tails 0.5)
(display "P(heads OR tails) = ")
(display (+ p-heads p-tails))

What does a warning tell you?

A fictional factory scanner flags 90% of defective items. It also flags 10% of good items. If an item gets flagged, is it probably defective? Predict first, then change how common defects are.

Defective items in the factory

These are expected counts per 10,000 items, not a simulated batch. The scanner’s two conditional rates stay fixed.

Items	Flagged	Passed
Defective	90	10
Good	990	8910

Defective among flagged: 90 ÷ 1080 = 8.3%.

“Given” changes the denominator. At a 1% defect rate, 90 defective and 990 good items are flagged. Only 90 of those 1,080 flagged items are defective, despite catching 90% of all defects.

The idea in one line

P(A | B) = P(A and B) / P(B), provided P(B) is positive. Keep only cases in B; among those, count the ones also in A. Reversing A and B usually changes the question and the answer.

A moment to think

A spam filter catches 90% of spam. Does that mean 90% of flagged messages are spam?

Yes; those are the same percentage.No; we also need the spam rate and the rate of flagging legitimate messages.

Conditional probability

P(A | B) = P(A and B) / P(B). The probability of A, given that B happened. This is how we update probabilities with evidence.

Scheme

; Conditional probability: P(A|B) = P(A and B) / P(B)

(define p-disease 0.01)         ; prevalence
(define p-positive-given-disease 0.95)  ; sensitivity
(define p-positive-given-healthy 0.05)  ; false positive rate

; P(positive) by total probability
(define p-healthy (- 1 p-disease))
(define p-positive
  (+ (* p-positive-given-disease p-disease)
     (* p-positive-given-healthy p-healthy)))

(display "P(positive test) = ")
(display p-positive) (newline)

; P(disease | positive) by Bayes' theorem
(define p-disease-given-positive
  (/ (* p-positive-given-disease p-disease) p-positive))

(display "P(disease | positive) = ")
(display (* 1.0 p-disease-given-positive)) (newline)

; Even with a 95% accurate test, a positive result
; only means ~16% chance of disease when prevalence is 1%
(display "Base rate matters.")

Independence

Events A and B are independent if P(A | B) = P(A), equivalently P(A and B) = P(A) * P(B). Coin flips are independent. Test results from the same patient are not.

Scheme

; Independence check: P(A and B) =? P(A) * P(B)

; Two coin flips: independent
(define p-h1 0.5)  ; first heads
(define p-h2 0.5)  ; second heads
(define p-both-heads (* p-h1 p-h2))
(display "P(both heads) = ")
(display p-both-heads) (newline)
(display "P(H1) * P(H2) = ")
(display (* p-h1 p-h2)) (newline)
(display "Independent? ")
(display (= p-both-heads (* p-h1 p-h2))) (newline)
(newline)

; Rain and umbrella: NOT independent
(define p-rain 0.3)
(define p-umbrella 0.4)
(define p-rain-and-umbrella 0.25)  ; higher than product
(display "P(rain) * P(umbrella) = ")
(display (* p-rain p-umbrella)) (newline)
(display "P(rain AND umbrella)  = ")
(display p-rain-and-umbrella) (newline)
(display "Independent? ")
(display (= p-rain-and-umbrella (* p-rain p-umbrella)))

Neighbors

Related chapters

🎰 Grinstead Ch.4 — conditional probability with combinatorial foundations
📡 Shannon Ch.3 — probability as the basis for entropy and information

Foundations (Wikipedia)

Translation notes

OpenIntro introduces probability with contingency tables and tree diagrams. We focus on the computational rules. The Bayes' theorem example (disease screening) is a standard illustration of base rate neglect. The original also covers probability trees and more complex multi-step scenarios. For rigorous probability axioms, see 🎰 Grinstead Ch. 1.

Want the full treatment? Read OpenIntro Statistics, Ch. 3.

← Summarizing Data by june.kim Distributions →