03 · What You Need to Know
What a Control Group Adds to a Before-and-After Study
What an Uncontrolled Before-and-After Study Can Establish
An uncontrolled before-and-after study measures outcomes before and after an intervention without observing a separate group or condition that provides a direct comparison.
For example, researchers may measure teachers' confidence in using educational technology before a professional development program and again after its completion.
If average confidence increases, the study can establish that the measured outcome changed among the observed participants. With appropriate data, researchers can estimate the magnitude of that change, examine individual differences, and describe its statistical uncertainty.
What the design cannot readily establish is how much of the change would have occurred without the program.
Participants may gain confidence through ordinary teaching experience. A new institutional policy might encourage technology use. Repeated exposure to the confidence questionnaire may also influence responses. These are among the alternative explanations for changes over time that an uncontrolled design may struggle to distinguish.
Accordingly, an uncontrolled design can be appropriate for a descriptive research question even when it is inadequate for a strong causal claim.
What a Control Group Actually Does
A control group provides observations under a comparison condition. Depending on the study, that condition might involve no intervention, usual practice, an alternative program, or another relevant exposure.
In a controlled before-and-after study, researchers typically measure outcomes in both the intervention group and the comparison group before and after implementation.
The comparison group's change helps estimate what might have occurred in the intervention group without the intervention, provided the groups are sufficiently comparable for the intended analysis.
This is an attempt to approximate the counterfactual: the outcome that would have occurred under the alternative condition.
A control group does not directly reveal the intervention participants' unobserved counterfactual outcomes. Rather, it provides evidence that may support an estimate under particular design assumptions.
When a Control Group Is Especially Important
A comparison becomes particularly valuable when the primary research question concerns intervention effectiveness rather than merely observed change.
Consider the distinction between these questions:
Descriptive question
How did students' research-literacy scores change after participating in the program?
Causal question
How much did the program improve research-literacy scores compared with what would have happened without it?
The first question can be answered descriptively using a properly conducted single-group pretest-posttest study. The second requires evidence about the alternative outcome trajectory.
A credible comparison is particularly important when participants are expected to improve naturally, when external conditions are changing, or when the intervention is introduced in response to unusually poor performance.
Does the Comparison Group Have to Receive No Intervention?
No. The appropriate comparator depends on the causal question.
If researchers want to know whether a new learning platform improves outcomes relative to existing teaching practices, students receiving the usual instruction may provide a more meaningful comparison than students receiving no instruction.
Similarly, comparing a new treatment against established care may be more ethically and scientifically appropriate than withholding care.
The comparator defines the effect being estimated. An intervention compared with no treatment answers a different question from the same intervention compared with an established alternative.
Researchers should therefore select a comparison condition that represents the decision they are trying to inform, not simply the condition that is easiest to arrange.
Does Adding a Control Group Automatically Establish Causation?
No. A comparison group can strengthen a design, but its presence alone does not guarantee valid causal inference.
Suppose a university introduces a new instructional program in its highest-performing department and compares outcomes with a department serving students with substantially different academic preparation.
Differences in student characteristics, instructional resources, academic culture, or pre-existing performance trends could influence the results.
These differences create potential selection bias and confounding. If the groups would have changed differently even without the intervention, their observed changes may not provide an unbiased estimate of the intervention effect.
This is why choosing an appropriate comparison group is a substantive methodological decision rather than an administrative formality.
Randomized and Nonrandomized Comparison Groups
In a randomized controlled design, eligible units are assigned to intervention conditions through a random mechanism. Proper random assignment helps reduce systematic baseline differences between groups, although chance imbalances can still occur.
In a nonrandomized controlled before-and-after design, groups may be formed through existing classes, departments, clinics, schools, or administrative decisions.
Nonrandomized groups can be useful, especially when random assignment is impractical. However, researchers must consider why particular units received the intervention and whether those reasons are related to the outcomes.
Baseline measurements, information about relevant covariates, and observations from earlier periods can strengthen the assessment of comparability. None of these features automatically eliminates unmeasured confounding.
What Difference-in-Differences Adds
One common approach to analyzing controlled before-and-after data is difference-in-differences.
Rather than comparing only post-intervention outcomes, this approach compares how outcomes changed in the intervention group with how they changed in the comparison group.
The 7-point contrast represents the additional observed change in the intervention group relative to the comparison group.
Interpreting that contrast as a causal effect requires assumptions. A central assumption is that, without the intervention, the groups would have followed parallel trends in their expected outcomes. This assumption concerns the unobserved counterfactual and cannot be proven merely because baseline scores are similar.
Multiple pre-intervention measurements can help assess whether the groups previously followed comparable trends, although historical agreement does not guarantee that the assumption will hold after implementation.
Other threats include changes in group composition, differential measurement, anticipation of the intervention, and events affecting one group but not the other.
When a Separate Control Group May Be Impractical
Some interventions are implemented simultaneously across an entire institution or population. Others cannot reasonably be withheld from eligible participants.
In these situations, researchers may consider alternative designs rather than treating a single pretest and posttest as the only available option.
| Design Option |
Potential Contribution |
Important Limitation |
| Single-group pretest-posttest |
Documents observed change among participants. |
Provides limited evidence about what would have happened without the intervention. |
| Controlled before-and-after |
Compares changes across intervention and comparison groups. |
Nonrandomized groups may have different counterfactual trends. |
| Randomized controlled design |
Uses random assignment to strengthen causal identification. |
May be infeasible or inappropriate in some settings. |
| Interrupted time series |
Examines changes in outcome level or trend around an intervention point. |
Concurrent events may still explain an apparent interruption. |
| Controlled interrupted time series |
Combines longitudinal trends with a comparison series. |
Requires appropriate comparison data and credible assumptions. |
An interrupted time-series design may be particularly useful when routinely collected outcomes are available over many periods before and after implementation.
When no credible comparison or time-series approach is possible, the study may still provide valuable descriptive evidence. Researchers should then avoid presenting the observed change as a demonstrated causal effect.
Can the Same Participants Serve as Their Own Controls?
Researchers sometimes describe a pretest-posttest study as one in which participants serve as their own controls.
This terminology reflects the fact that each participant's earlier outcome is compared with their later outcome. Such within-person comparisons remove some stable differences between individuals from the calculation of change.
However, the earlier observation is not necessarily a valid estimate of the participant's later outcome without the intervention.
Participants can mature, experience external events, or respond differently because they have already completed the assessment. A within-person comparison therefore does not automatically substitute for an independent control condition.
How Much Does the Purpose of the Study Matter?
The need for a control group depends partly on the intended inference.
A feasibility study might investigate recruitment, adherence, acceptability, data completeness, or whether an intervention can be delivered as planned. A single-group design may be sufficient for some of these objectives.
An implementation evaluation may focus on how an intervention was adopted, what resources it required, and whether participants engaged with it. Again, a separate control group may not be essential.
By contrast, a study intended to support a consequential policy decision about comparative effectiveness generally needs stronger counterfactual evidence.
The methodological standard should follow the research claim. A descriptive question should not be artificially presented as causal, and a causal question should not be answered solely by documenting change.
04 · A Practical Example
Evaluating a Digital Learning Intervention With and Without a Comparison Group
Hypothetical Example
Two Classes, One New Learning Platform
A university introduces an adaptive learning platform in one undergraduate research-methods class. Another class continues using the established learning environment. Both classes complete comparable assessments before and after a six-week instructional period.
For simplicity, assume that the same students complete both assessments in each class and that the outcome is measured on a 100-point scale.
| Group |
Before |
After |
Change |
| Adaptive learning platform |
60 |
78 |
+18 |
| Established learning environment |
62 |
72 |
+10 |
If researchers examined only the adaptive learning class, they would report an 18-point improvement. That is an accurate description of the observed change, but it would provide limited evidence about the platform's effect.
The comparison class also improved by 10 points, despite not receiving the adaptive platform. This suggests that at least some improvement may have been possible through ordinary instruction, accumulated experience, or other shared influences.
The 8-point estimate is more informative for evaluating the intervention than the 18-point within-group improvement alone.
However, suppose the adaptive learning class was taught by a more experienced instructor, or its students were more motivated to improve. Those differences could influence the outcome independently of the platform.
Alternatively, the comparison class might have had a different pattern of achievement growth before the study began. Similar baseline scores would not resolve that concern.
To strengthen the evaluation, researchers could examine earlier assessment results, document instructor differences, investigate concurrent educational activities, and consider whether the classes are sufficiently comparable.
If students or classes could be randomly assigned to conditions under appropriate safeguards, randomization could further strengthen the design.
The appropriate conclusion from the hypothetical nonrandomized results is that the adaptive learning class experienced 8 points more improvement than the comparison class. Whether this represents a causal platform effect depends on the design's assumptions and the available evidence.
06 · What This Means for You
How to Decide Whether Your Study Needs a Control Group
Begin with the exact conclusion you want the evidence to support. If your objective is to document what changed after implementation, an uncontrolled design may be adequate. If your objective is to estimate an intervention effect, identify how you will approximate the relevant counterfactual.
Next, consider whether a comparison group can be formed without introducing serious ethical or methodological problems. A convenient group is not necessarily a credible group.
Finally, evaluate whether alternative sources of comparison, such as historical outcome trends or staggered implementation, could support the intended inference.
A simple decision framework
If your question concerns observed change, feasibility, or implementation
A single-group before-and-after design may be sufficient, provided its descriptive purpose and limitations are explicit.
If your question concerns comparative effectiveness
Prioritize a credible comparison condition and define the intervention effect you intend to estimate.
If random assignment is feasible and appropriate
If comparison groups are naturally occurring
Examine baseline characteristics, prior trends, intervention selection, and potential confounding before choosing the analysis.
If no separate comparison group is available
Assess whether repeated observations, alternative timing arrangements, or other quasi-experimental approaches could strengthen the evaluation.
If withholding the intervention would be unethical
What Should You Report If You Have No Control Group?
Report the design accurately, including the absence of a separate comparison condition. Specify the outcome, measurement schedule, intervention procedures, participant eligibility, and methods used to handle missing observations.
Present the observed change with appropriate measures of uncertainty. Describe plausible alternative explanations, especially those arising from natural development, concurrent events, repeated testing, and regression to the mean.
Avoid treating the limitations section as a substitute for careful interpretation. If the design supports only descriptive change, use language such as "scores increased following implementation" rather than asserting that the intervention produced the increase.
Watch Out
Adding a poorly chosen comparison group can create a misleading impression of causal rigor. A defensible comparison requires substantive justification of the groups' relationship, not merely the presence of two columns in a results table.
When the study is intended to inform policy or institutional investment, the costs of an incorrect causal conclusion may justify additional effort to construct a credible comparison. When the purpose is preliminary description or feasibility, a simpler design may be proportionate to the decision being made.
