Wrong Questions

A Type III error is an exact answer to the wrong question. From the inside it feels identical to knowing: the work is done correctly, the evidence is vivid and real, and all of it answers a question nobody should have asked. You cannot catch it by doubting harder, because doubt loses to vivid evidence every time. You catch it with a checklist, run before you scale.

Every Type III error is the same move underneath: the real question gets silently swapped for a more answerable one. So there is one question to run on any claim before you commit to it.

What has to be silently substituted for this answer to feel responsive?

The eleven entries below are the places the swap hides, each named by the part of the question it replaces. For each: the substitution, the tell (how you catch it from inside the work, since the feeling will not), and the fix. Every entry has a tell, and the tell is the alarm. The fixes split two ways. A reflex is a mechanical check you run before belief forms. A demotion is a narrower claim you settle for after the evidence is in. Reflexes change the procedure; demotions change the claim.

Accident: the source. You answer an easier version of the task because you took a summary for the thing. Tell: your understanding traces to a summary, not the source. Fix (reflex): read the cold source before you scale.

Incentive: the reward. You ask the question the field rewards instead of the one that holds. Tell: you can state why the result will be rewarded more easily than you can state what would make it false. Fix (demote): run the check that could kill it; keep what survives.

Category: the kind. The thing you are studying is not the kind of thing the question is about, as when a repair is called a discovery. Tell: your evidence proves performance, frequency, or correlation, but your claim uses a word like discovery, meaning, legitimacy, or harm. Fix (demote): check the verb of your claim against the verb your evidence can support; ask what kind of thing this is before asking what it does.

Proxy: the metric. You swap the target for a measurable stand-in and forget you did. Tell: the claim is only true with “as measured by” silently appended. Fix (reflex): write the forbidden clause out loud: “I am treating X as evidence for Y because.” If the because is weak, demote; if strong, keep it but name the proxy as a proxy.

Scope: the grain. You ask at the wrong level: aggregate when the effect is local, individual when it is systemic. Tell: the answer changes sign or meaning when sliced along a boundary you can already name. Fix (reflex): re-slice before believing it; demote to the level where it holds.

Baseline: the counterfactual. You ask whether it works against the wrong “compared to what.” Tell: it impresses only because the alternative is naive, passive, or unnamed; your control is not what a competent person would actually do. Fix (reflex): name the strongest boring alternative. If it dies there, demote.

Selection: the population. You answer for the cases your evidence can reach, then claim the population that matters. Tell: your claim covers cases that had no route into your sample, archive, dataset, or memory. Fix (reflex): name the inclusion rule, then demote to the population that could actually have appeared.

Temporality: the clock. You ask a static question about a process that adapts. Tell: your claim requires the system not to react to the measurement, the intervention, or the rule. Fix (demote): ask for the second move; lower “solves” to “works until adapted.”

Normativity: the value. You ask an empirical question whose real content is evaluative. Tell: two people could accept all your facts and still rationally disagree, unless they share a value premise you never defended. Fix (demote): write the value premise out loud. Measurement informs it; it never settles it.

Resolution: the vocabulary. Your terms are too coarse to carry the distinction that matters; the answer is exact and the words have already destroyed the phenomenon. Tell: the dependency you want to bracket as “it depends” is exactly the distinction your conclusion needs. Fix (reflex): split the noun before answering the predicate.

Direction: the arrow. You ask whether A causes B when A and B co-produce each other. Tell: you cannot state the mechanism without using the outcome as part of the cause. Fix (demote): lower the one-way claim; model the loop.

Not separate kinds

The catalog is organized by which part of the question gets substituted, not by the motive behind it. That is what keeps the entries from collapsing into one another. Category and Resolution are the easiest to confuse: Category is the wrong kind of thing, Resolution the right kind named too coarsely. Other near-misses are existing entries in disguise. Laziness is Accident (skipped the source) or Proxy (took the cheap stand-in). Fashion is Incentive. Ideology is Incentive (the rewarded answer) or Normativity (the hidden value). Overgeneralization is Scope or Selection. Ambiguity is Resolution (terms too coarse) or Category (wrong kind named). A new kind earns its place only when it names a substitution site none of the eleven already cover.

Casting it

Run the master question on any claim before you scale it: what has to be silently substituted for this answer to feel responsive? Then walk the eleven sites looking for the swap. Six answer to a reflex you can make automatic: read the source, write the “as measured by” clause, re-slice, name the baseline, name the inclusion rule, split the noun. The other five give you a tell but no automatic step, because the felt sense of evidence is the same whether the question is right or wrong. For those the defense is to refuse to ship a claim until it stops depending on a substitution someone can name. You write it until it stops moving.