03 · What You Need to Know
Understanding, Defining, and Applying the Smallest Effect Size of Interest
What Exactly Does the SESOI Represent?
An effect size describes the magnitude of a relationship, difference, or other quantity of interest. The SESOI identifies a boundary below which an effect would not be sufficiently important for the purpose under investigation.
Suppose an intervention is intended to improve examination performance. A difference of one point might be too small to influence teaching decisions, while a difference of five points could justify further investment. If five points is the smallest improvement considered worthwhile, it may serve as the SESOI.
The threshold is not necessarily universal. An effect considered negligible for a costly institutional intervention could be valuable for an inexpensive intervention delivered to a large population. Likewise, a small effect might be theoretically important even when it has limited immediate practical value.
Meaningfulness must therefore be evaluated in relation to the research question and the consequences of the effect.
Is the SESOI the Same as the Minimum Important Difference?
The concepts are closely related but are not always interchangeable.
A minimum important difference, often discussed in clinical and patient-reported outcome research, typically refers to a change or difference considered important to patients or other relevant stakeholders. Its precise definition and estimation method can vary.
The SESOI is broader. It may be defined using practical consequences, theoretical predictions, measurement characteristics, or other defensible scientific criteria.
For example, a researcher studying memory may specify a threshold based on a theory's quantitative prediction rather than on an intervention's practical usefulness.
SESOI
The smallest effect considered important for the stated research objective.
Minimum important difference
A threshold of importance, commonly defined for changes or differences in clinical or other outcome measures.
How Can Researchers Determine a Defensible SESOI?
There is no single procedure suitable for every discipline. Lakens, Scheel, and Isager (2018) discuss approaches that connect effect-size thresholds to theoretical predictions, practical consequences, and existing evidence.
The central requirement is that the threshold must have a reason for existing beyond its convenience in a statistical calculation.
Use Practical or Policy Consequences
When research informs a decision, the SESOI can reflect the minimum benefit that would justify action.
Consider an institution evaluating an instructional intervention. The minimum worthwhile improvement might depend on implementation costs, faculty workload, student benefits, and the availability of alternatives.
A modest gain could be meaningful if the intervention is inexpensive and accessible. The same gain might not justify replacing an established instructional system if implementation requires substantial investment.
Stakeholder judgments can inform this threshold, although they should be collected and interpreted systematically where possible.
Use Quantitative Theoretical Predictions
Some theories imply that effects must exceed a particular magnitude to support a proposed mechanism.
For example, if a theoretical account depends on a perceptual change being noticeable, an empirically established detection threshold may help define the smallest effect relevant to that theory.
This approach can make a theoretical claim more falsifiable because it specifies what magnitude of effect the explanation requires.
However, not all theories provide quantitative predictions. Researchers should not invent a numerical boundary and present it as a theoretical requirement when the theory does not support one.
Use Empirical Evidence About Meaningful Changes
Previous studies may provide information about changes associated with outcomes that stakeholders recognize as important.
In health research, anchor-based approaches can relate measured changes to external judgments of improvement. Other approaches may use empirical distributions or established performance criteria.
These methods require careful interpretation. A statistically derived threshold is not necessarily meaningful merely because it can be calculated.
When adopting an existing threshold, examine whether the population, measurement instrument, baseline conditions, and intended decisions are sufficiently comparable.
Use Resource-Based Thresholds Transparently When Necessary
Sometimes researchers cannot establish a strong substantive SESOI. Lakens, Scheel, and Isager (2018) discuss approaches that can define bounds using the sensitivity of previous research or available resources.
Such bounds may be useful for a particular inferential purpose, but their interpretation must remain limited. A threshold determined by what a study can detect is not automatically a threshold of practical importance.
If the proposed SESOI is primarily resource-based, describe it accordingly rather than claiming that it represents a universally meaningful difference.
Should the SESOI Be Expressed in Raw or Standardized Units?
Whenever possible, begin with units that have a clear substantive interpretation.
For an achievement test, the SESOI might be four score points. For a proportion, it might be a five-percentage-point difference. For a time-based outcome, it could be a reduction of ten minutes.
Standardized effect sizes are useful when measurement scales differ or when effects need to be compared across contexts. However, standardization introduces dependence on the variability of the outcome.
For a standardized mean difference, a common expression is:
Raw and standardized thresholds can therefore answer subtly different questions. When the practical decision depends on a particular number of score points, specifying the threshold in raw units may preserve its intended meaning more clearly.
Why Is the SESOI Useful for Sample-Size Planning?
Sample-size planning requires researchers to explain what information the proposed study should provide.
In a conventional superiority test, the SESOI may serve as the alternative effect used in an a priori power calculation. A study powered for that magnitude is designed to reject the zero-effect null hypothesis with a specified probability if the true effect equals the SESOI and the planning assumptions hold.
This approach is particularly relevant when the expected effect size is uncertain but the smallest worthwhile effect can be justified.
The distinction between the expected effect and the SESOI matters because the former is a prediction, whereas the latter defines a boundary of substantive interest.
Neither quantity should be chosen merely to make the required sample size convenient.
How Does the SESOI Relate to Equivalence Testing?
Equivalence testing addresses a different question from conventional superiority testing.
Rather than asking whether an effect differs from zero, equivalence testing examines whether the data provide sufficient evidence that the population effect lies within prespecified bounds representing effects considered negligible.
For a signed mean difference, researchers might define equivalence bounds of −4 and +4 points if differences within that interval are considered too small to matter.
The two one-sided tests procedure, commonly called TOST, provides one established approach to equivalence testing.
A statistically nonsignificant conventional test does not establish equivalence. The data may simply be too imprecise to distinguish meaningful effects from negligible ones.
Watch Out
Powering a conventional superiority test at the SESOI does not automatically provide adequate power for equivalence testing. Equivalence-test sample-size calculations depend on the equivalence bounds, the assumed true effect, variability, and the selected error rates.
Can the SESOI Help Interpret Confidence Intervals?
Yes. A prespecified SESOI gives researchers a meaningful reference point when interpreting uncertainty around an effect estimate.
For example, suppose the SESOI is a positive improvement of four points. A confidence interval ranging from one to seven points is compatible with effects below and above that threshold. It therefore does not resolve whether the true improvement reaches the minimum worthwhile magnitude.
A confidence interval entirely above four points would provide stronger evidence that the effect exceeds the threshold, subject to the statistical model and confidence procedure.
For symmetric equivalence bounds, a confidence interval sufficiently contained within those bounds may support an equivalence conclusion under the corresponding testing procedure.
This is one reason why precision-based sample-size planning can be valuable when the main goal is to distinguish effects on opposite sides of a meaningful threshold.
What If No Defensible SESOI Can Be Identified?
Not every research question permits a convincing numerical threshold. Exploratory studies, early theoretical investigations, and research involving poorly understood outcomes may lack the evidence needed to establish one.
Researchers should acknowledge that limitation rather than manufacture a threshold.
Alternative justifications may include planning for a desired estimation precision, using a well-supported expected effect, evaluating a range of plausible effects, or explicitly adopting a resource-constrained design.
As Lakens (2022) emphasizes, sample-size justification should explain the informational value of the proposed data relative to the study's inferential objectives. It need not always take the form of a conventional power calculation.
