Designing a Study You Can Defend
Participants do not always receive or follow interventions as assigned. Understand how nonadherence changes the treatment comparison, why excluding noncompliant participants can introduce bias, and how to design and analyze studies that preserve credible causal conclusions.
Read Guide →
Designing a Study You Can Defend
Missing observations can distort research findings when the data that remain systematically differ from the information that was not collected. Understand how missing-data mechanisms affect internal validity, which analytical approaches may help, and why preventing missingness is often more valuable than correcting it afterward.
Read Guide →
Designing a Study You Can Defend
Research findings do not automatically apply beyond the participants studied. Learn how sampling, setting, eligibility, treatment conditions, and methodology shape generalizability, external validity, transportability, and qualitative transferability.
Read Guide →
Designing a Study You Can Defend
Internal validity concerns whether a study supports a credible inference within the conditions studied, while external validity concerns whether that inference extends to other populations, settings, or circumstances. Strong research considers both, but their importance depends on the claim being made.
Read Guide →
Designing a Study You Can Defend
Some design choices that strengthen control can narrow the populations and conditions represented by a study, but internal validity and generalizability are not inherently opposing goals. The real question is which design choices improve one inference while restricting another.
Read Guide →
Designing a Study You Can Defend
Strict eligibility criteria may improve intervention consistency and participant safety while excluding people who would use the intervention in practice. Learn when restrictions limit generalizability, which exclusions are justified, and how to design studies that remain methodologically credible and relevant to real-world populations.
Read Guide →
Designing a Study You Can Defend
A tightly controlled experiment may identify an intervention's causal effect convincingly while providing limited evidence about its performance in ordinary settings. Understand why internal and external validity can diverge, when experimental control becomes restrictive, and how to design research that supports credible and applicable conclusions.
Read Guide →
Designing a Study You Can Defend
Conducting research at multiple institutions may strengthen evidence about how findings apply across settings, but the number of sites alone does not guarantee generalizability. Learn how site selection, contextual variation, and treatment-effect heterogeneity determine what multisite research can establish.
Read Guide →
Designing a Study You Can Defend
There is no universal minimum number of research sites needed for generalizability. Learn how to determine site requirements based on contextual diversity, research objectives, statistical precision, and the evidence needed to understand variation across settings.
Read Guide →
Designing a Study You Can Defend
Convenient research sites may make a study feasible, but they may exclude settings essential to understanding how findings apply elsewhere. Learn when contextual diversity matters, how to select informative sites, and why purposive diversity and statistical representativeness serve different research purposes.
Read Guide →
Designing a Study You Can Defend
Readers cannot judge whether research findings apply to their setting without sufficient information about participants, context, intervention delivery, and outcomes. Learn how to build applicability into study design and reporting rather than treating it as an afterthought.
Read Guide →
Designing a Study You Can Defend
Your research question may name one construct while your data capture something narrower or different. Learn how measurement decisions can quietly change the empirical question your study actually answers.
Read Guide →
Designing a Study You Can Defend
Reliability asks whether measurement is sufficiently consistent; validity asks whether the evidence supports the interpretation you want to make from it. A measure may be highly reliable without measuring the intended construct well.
Read Guide →
Designing a Study You Can Defend
Construct, content, criterion, and face validity describe different questions researchers may ask about measurement, but they should not be treated as four independent certificates of validity. Modern validity frameworks emphasize the evidence supporting a particular interpretation and use of scores.
Read Guide →
Designing a Study You Can Defend
A measure can produce highly consistent results and still fail to support the interpretation a researcher wants to make from them. Reliability is important, but consistency alone cannot establish validity.
Read Guide →
Designing a Study You Can Defend
Using an instrument with strong validity evidence can strengthen one part of your methodology, but it does not validate the study as a whole. Sampling, design, implementation, analysis, and interpretation still determine whether your conclusions are defensible.
Read Guide →
Designing a Study You Can Defend
A study can implement an intervention correctly and collect reliable measurements while failing to investigate the construct it claims to address. Learn how to examine the relationship between theoretical concepts, intervention components, operational definitions, and evidence supporting construct interpretation.
Read Guide →
Designing a Study You Can Defend
Complex constructs such as engagement, critical thinking, and self-regulated learning cannot always be represented adequately by one convenient indicator. Learn when narrow operationalization threatens construct validity, how to distinguish focused measurement from underrepresentation, and what evidence supports a defensible interpretation.
Read Guide →
Designing a Study You Can Defend
Using multiple measures can strengthen construct validity when they provide complementary evidence about the intended construct. Learn how to select appropriate indicators, evaluate agreement and disagreement between measures, and avoid assuming that more instruments automatically produce better measurement.
Read Guide →
Designing a Study You Can Defend
A study cannot establish that it tested its intended construct through statistical significance, instrument reliability, or intervention labels alone. Learn what theoretical, procedural, measurement, and analytical evidence is needed to support defensible construct claims.
Read Guide →
Designing a Study You Can Defend
Selection bias can originate before researchers approach a single participant. Learn how decisions about study populations, sampling frames, research sites, and eligibility can shape who becomes observable, when selection threatens causal inference, and how to identify these risks during research design.
Read Guide →
Designing a Study You Can Defend
Eligibility criteria help define a research population, but some restrictions can distort associations or causal comparisons. Learn when inclusion and exclusion rules create selection bias, when they primarily limit generalizability, and how to justify eligibility decisions before recruitment.
Read Guide →
Designing a Study You Can Defend
Recruitment channels determine who encounters a research invitation and who has an opportunity to participate. Learn how different channels can systematically exclude eligible people, when this threatens population estimates or causal comparisons, and how to evaluate recruitment coverage before data collection.
Read Guide →
Designing a Study You Can Defend
Informed consent protects participants' autonomy, but the way consent is presented and obtained can influence who enters a study. Learn how consent procedures may shape sample composition, when consent-related nonparticipation threatens inference, and how researchers can improve accessibility without weakening ethical protections.
Read Guide →
Designing a Study You Can Defend
A comparison group must represent a credible alternative to the exposure or intervention being studied. Learn how to establish common eligibility, timing, and outcome measurement, distinguish confounding from selection bias, and avoid comparison groups that create misleading causal estimates.
Read Guide →
