Introduction to Data

OpenIntro Statistics · Ch. 1 · openintro.org/book/os

New to statistics? Start with Probability and Events, a short chapter you can experiment with.

Statistics begins with data. Every dataset is a collection of observations (rows) described by variables (columns). The type of variable and the way data was collected determine which methods apply.

Observations and variables

An observation (or case) is a single unit in the dataset: one person, one transaction, one experiment. A variable is a characteristic measured on each observation. Variables are either numerical (quantitative) or categorical (qualitative).

Scheme

; Variable types
; Numerical: discrete (counts) or continuous (measurements)
; Categorical: nominal (no order) or ordinal (ordered)

(define dataset
  '(("Alice" 28 "Engineer" 72000)
    ("Bob"   35 "Teacher"  55000)
    ("Carol" 42 "Engineer" 88000)
    ("Dave"  31 "Designer" 61000)))

; Each row is an observation
(display "Observations: ")
(display (length dataset)) (newline)

; Variables: name (categorical), age (numerical),
;            job (categorical), salary (numerical)
(display "Variables: name, age, job, salary") (newline)

; Numerical variable: compute mean age
(define ages (map cadr dataset))
(define (mean lst)
  (/ (apply + lst) (length lst)))
(display "Mean age: ")
(display (* 1.0 (mean ages)))

Study design: observational vs. experimental

An observational study records data without intervening. An experiment assigns treatments to subjects. Random assignment supports causal inference by preventing systematic differences in who receives a treatment, provided the study is carried out appropriately. Observational causal claims need additional design and assumptions to address confounding. The distinction matters: if you observe that coffee drinkers live longer, that does not mean coffee causes longevity.

Scheme

; Observational: measure what exists
; Experimental: assign treatment, measure outcome

; Simulate an experiment: randomize to treatment/control
(define subjects '(1 2 3 4 5 6 7 8 9 10))

; Independent random assignment: group sizes can differ
(define (assign-group id)
  (if (= (random-integer 2) 0) "treatment" "control"))

(for-each
  (lambda (id)
    (display "Subject ")
    (display id)
    (display ": ")
    (display (assign-group id))
    (newline))
  subjects)

; Key question: was treatment randomly assigned?
; Random assignment supports causal attribution under study assumptions.
; It does not make these subjects representative of everyone.

Try two ways of choosing residents before comparing sampling strategies. Read the guided introduction →

A fictional town of 200 adults

People to sample

Sample without replacement: nobody is measured twice in a single sample.

Each dot is a resident. Filled dots mark the town-wide sample. Thick outlined dots mark other residents in the basketball-court subgroup.

No sample yet. What do you think the town’s average height will be?

Population mean: 169.4 cm. Measuring all 40 court residents removes uncertainty about that subgroup, but it still does not represent the whole town.

Sampling strategies

The population is everyone you want to study. The sample is who you actually measure. Good sampling avoids bias: simple random sampling, stratified sampling (split into groups first), and cluster sampling (randomly select entire groups). Convenience samples are cheap but unreliable.

Scheme

; Simple random sample: every member equally likely
(define population '(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20))

 ; Choose a uniform index from the remaining list, then remove that case.
(define (remove-at lst index)
  (if (= index 0) (cdr lst)
    (cons (car lst) (remove-at (cdr lst) (- index 1)))))
(define (simple-random-sample pop size)
  (let loop ((remaining pop) (selected '()) (count 0))
    (if (or (= count size) (null? remaining))
      (reverse selected)
      (let ((index (random-integer (length remaining))))
        (loop (remove-at remaining index)
              (cons (list-ref remaining index) selected)
              (+ count 1))))))

(display "Population: ")
(display population) (newline)

(define sample (simple-random-sample population 5))
(display "Random sample of 5: ")
(display sample) (newline)
(display "Sample size: ")
(display (length sample))

Neighbors

Related chapters

🎰 Grinstead Ch.1 — discrete probability spaces, the mathematical foundation beneath sampling

Foundations (Wikipedia)

Translation notes

OpenIntro introduces data with the county and email datasets. We use small inline datasets instead to keep examples self-contained. The original covers data matrices, scatterplots, and data collection pitfalls in more detail. The sampling code chooses uniformly from the remaining cases without replacement; rerunning it can give a different sample.

Want the full treatment? Read OpenIntro Statistics, Ch. 1.

by june.kim Summarizing Data →