How do you turn a question into a testable experiment?

For: Readers planning a first behavioural study or trying to understand a paper’s design.

A useful experiment begins with a question narrow enough to make a comparison possible. “Does distraction harm learning?” leaves several choices hidden. In an original planning example, compare studying the same word list with phone notifications enabled or disabled, then test recall after a specified delay. Name the participants, conditions, outcome and time point. This does not assume that the answer will favour either condition.

The first lecture helps with one part of that translation: turning a verbal claim into a population quantity. A difference in average recall is a possible target, but so is the proportion reaching a defined threshold. These are different questions. Write the target in ordinary language before choosing a statistical test. The videos on variable types and sampling statistics explain why the recorded data and the target quantity must fit together. NPTEL course and ordered lectures.

An experiment changes a condition; an observational comparison records differences that already exist. In the notification example, simply comparing people who habitually silence their phones with people who do not would leave other explanations open. The Experiment Basics chapter explains manipulation and confounding. Specify what should be held comparable, including study material, delay and instructions, and identify anything that the manipulation might unintentionally change.

Next decide whether each participant experiences one condition or both. Separate groups avoid learning the same material twice, but people differ before the study begins. A repeated-measures design compares each person with themselves, while introducing practice, fatigue and carryover. Random allocation concerns assignment to conditions; random sampling concerns who enters the study. Neither term is a substitute for describing the actual procedure. The design chapter explains these distinctions and the role of counterbalancing. Randomisation does not promise identical groups in a particular small sample.

Now work forwards to the result. What difference would matter? How noisy might recall scores be? What precision or power would be useful under explicit assumptions? Tangirala’s power examples let you vary these ingredients, rather than treating a sample-size calculation as a guarantee. Plan how missing observations and exclusions will be handled, and distinguish a planned analysis from an exploration suggested by the results.

For the exercise, write a one-page protocol without recruiting anyone. Give two plausible reasons, other than the proposed learning mechanism, for a difference between conditions. Revise the comparison to address one of them and state which uncertainty remains. Finish with a mock results sentence that includes the estimated difference and its interval. This reveals whether the proposed observations can answer the question you actually asked. It is a design rehearsal; research involving participants also needs appropriate ethical review and consent.

Next step

Take the proposed outcome to the measurement guide, then use the power lectures to vary at least two planning assumptions.

Limits of this route

The video descriptions were checked against identified official-transcript excerpts, not complete viewing. An accessible official media endpoint is not evidence that every frame played. Teaching examples are not independent validation of the underlying scientific or clinical claims. The protocol example is original and hypothetical; the sources do not test this notification intervention.