Science Practice 2: Research Methods and Design (Skills 2.A-2.D)

Science Practice 2: Research Methods and Design (Skills 2.A–2.D)

A psychological claim is only as strong as the study design that produced it: “students who sleep more score higher” does not tell you whether sleep caused the scores, whether the students were unusual, or whether “sleep” and “scores higher” were measured accurately.

Science Practice 2: Research Methods and Design means evaluating qualitative and quantitative research methods and study designs. You use it to identify how a study was built, determine what its evidence can support, and judge whether researchers protected participants.

The research-design decision tree

Start with four questions:

1. What method was used? Experiment, correlational study, survey, naturalistic observation, case study, or another method?
2. Who was studied? Identify the population, the larger group the researchers want to understand, and the sample, the people actually studied.
3. What was manipulated or measured? Identify the independent variable, dependent variable, operational definitions, control procedures, and possible confounds.
4. What conclusion is justified? Decide whether the design supports description, correlation, or a cautious causal claim—and whether ethical protections were adequate.

A crucial rule prevents a common error: if a study does not manipulate an independent variable and does not randomly assign participants to groups, it is not an experiment. A questionnaire can appear inside an experiment, but the method is experimental only when the study includes the defining experimental features.

Skill 2.A — Identify the research method used in a study

Skill 2.A requires you to identify the research method used in a psychological study. Do not classify a study by its topic or by the presence of numbers. Classify it by what the researchers actually did.

An experiment manipulates an independent variable, measures a dependent variable, and uses random assignment to place participants into conditions. Random assignment helps distribute preexisting differences across groups, allowing the researcher to test whether the manipulation produced a difference.

A correlational study measures variables as they naturally occur and examines their relationship. It can show that two variables move together, but it cannot establish that one caused the other. A survey collects self-reported information; a naturalistic observation records behaviour in its usual setting; and a case study examines one person, group, or unusual case in depth.

Worked scenario: classify before interpreting

Researchers recruit eighty teenagers from a community center. Each teenager reports average nightly sleep and completes a standardized attention task. The researchers find that students reporting more sleep tend to perform better.

Reasoning: The researchers measured sleep and attention; they did not assign participants to sleep conditions or manipulate sleep. Therefore, this is a correlational study, not an experiment. The result supports an association between reported sleep and attention, but it does not prove that increasing sleep caused the better performance. A third variable—such as school schedule, stress, health, or caffeine use—could influence both.

Skill 2.B — Explain how research design relates to conclusions

Skill 2.B requires you to connect the design to the conclusion it permits. Identify the sample, the population, the sampling method, the groups, the variables, and the procedures before deciding what the findings mean.

A population is the full group to which researchers hope to generalize. A sample is the smaller group actually observed. Random sampling gives members of the population a chance of being selected and can improve representativeness. Convenience sampling uses people who are easy to reach and may produce sampling bias.

Do not confuse random sampling with random assignment. Random sampling concerns who enters the study and affects generalizability. Random assignment concerns which condition a participant enters and strengthens causal inference within the study.

An operational definition states exactly how a variable is measured or manipulated. For example, “stress” might be operationalized as a score on a specified stress questionnaire, while “attention” might be operationalized as the number of correctly detected target symbols in five minutes. Precise operational definitions make replication and evaluation possible.

Worked scenario: design, sample, and conclusion

A university invites volunteers from its psychology department to test a new ten-minute breathing exercise. Forty volunteers are randomly assigned either to complete the exercise or to sit quietly for ten minutes. Afterward, both groups complete the same attention test. The breathing group scores higher.

Reasoning: The independent variable is the assigned activity: breathing exercise versus quiet sitting. The dependent variable is attention, operationalized as the attention-test score. Random assignment and a comparison condition support a causal conclusion that, under these procedures, the exercise contributed to the difference.

However, the sample is a convenience sample of psychology volunteers from one university. The result may not generalize to all teenagers, older adults, people with different educational backgrounds, or people who do not volunteer for psychology studies. The design supports stronger causal inference than the sleep example, but its generalizability remains limited.

Skill 2.C — Evaluate the validity of a study’s conclusions

Skill 2.C requires you to evaluate whether a study’s conclusions are valid—whether the evidence genuinely supports the claim being made. Ask whether the researchers used manipulation, random assignment, controlled procedures, valid measurement, and an appropriate sample.

A causal inference is strongest when researchers manipulate the independent variable, randomly assign participants, control procedures across conditions, and measure the dependent variable validly and reliably. If the measure does not represent the intended construct, a statistically clear difference may still fail to demonstrate the claimed psychological effect.

Causal inference has limits even in a well-designed experiment. A study may establish that the manipulation caused a difference under the study’s conditions without proving that the effect occurs everywhere or for everyone. Generalizability depends on the sample, setting, cultural context, and similarity between the study’s operational definitions and real-world behaviour.

Misconception check: “An experiment always proves a universal cause.” Not so. An experiment can support a causal explanation when its design is sound, but control, measurement, sampling, and context determine how far that explanation can be extended.

Skill 2.D — Explain how ethical guidelines apply

Skill 2.D requires you to explain how ethical guidelines apply to a psychological study, not merely name the word “ethics.” Evaluate the protections built into the procedure and connect each protection to the participant’s rights and welfare.

Key protections include:

• Informed consent: participants receive relevant information about procedures, risks, and participation before agreeing.
• Protection from harm: researchers minimize physical and psychological risk and stop procedures when risk becomes unacceptable.
• Confidentiality and privacy: researchers protect personal information and avoid exposing identities or sensitive responses.
• Deception and debriefing: deception must have a strong scientific justification, avoid unreasonable harm, and be followed by an explanation of the study’s true purpose.
• Right to withdraw: participants can leave the study without unfair penalty.
• Ethical review: an institutional review process evaluates risks, benefits, consent, privacy, and vulnerable populations before research begins.

Suppose researchers tell participants that a computer task measures concentration, although the real purpose is to observe reactions to unexpected negative feedback. If revealing the true purpose beforehand would change behaviour, limited deception may be defensible—but only if the risk is minimal, participation is voluntary, withdrawal is allowed, and researchers debrief participants afterward. Deception is not automatically ethical merely because it improves experimental control.

Retrieval check

A researcher randomly selects adults from a city registry, randomly assigns them to a mindfulness or reading condition, measures anxiety with a named questionnaire, and protects identities through coded records. What supports causal inference, and what supports generalization?

Answer: Random assignment, manipulation of the condition, a comparison group, controlled procedures, and a defined anxiety measure support causal inference. Random selection from the city registry supports generalization to that city’s adult population, although the conclusion may not extend beyond that population or beyond the study’s measurement and conditions.