Evaluating statistical claims

These questions describe a study and ask whether its conclusion is justified. Two gates decide it, and they are independent of each other.

The first gate is random selection, which governs generalisation. If participants were chosen at random from a population, results extend to that population — and to no wider one. A random sample of one school supports claims about that school only.

The second gate is random assignment, which governs causation. Only when researchers assign the treatment at random can a difference in outcomes be attributed to the treatment rather than to how the groups differed beforehand.

So an observational study showing that people who exercise sleep better cannot establish that exercise causes better sleep, however large the sample. Something else may drive both.

The correct answer is usually the most cautious statement that the design actually supports — and the trap is always the stronger, more interesting claim.

Researchers randomly assign $400$ randomly selected students to two study methods and find one scores higher.
What can be concluded?
Both gates are passed: random selection allows generalising to that student population, and random assignment allows attributing the difference to the method.