When Is a Pilot Study More Appropriate Than a Feasibility Study?

When Should You Conduct a Pilot Study Instead of a Feasibility Study?

You have designed a research project but are uncertain whether its procedures will work as intended. Perhaps you want to test participant recruitment, the delivery of an educational intervention, the administration of outcome measures, or the coordination of several research activities.

Should you conduct a pilot study, or would a feasibility study be more appropriate?

The distinction can be confusing because researchers sometimes describe any small preliminary investigation as a pilot study. Others use pilot and feasibility interchangeably, even when the preliminary work addresses different questions.

More importantly, the two terms are not necessarily competing alternatives. Methodological guidance, particularly in preparation for randomized controlled trials, generally treats a pilot study as a particular type of feasibility study.

The practical question is therefore not simply which label sounds more appropriate. It is whether the uncertainty you need to investigate requires a smaller-scale rehearsal of the intended definitive study.

Choose a Pilot Study When You Need to Rehearse the Proposed Study

Understanding When a Pilot Study Is the Better Preliminary Design

What Is the Relationship Between Pilot and Feasibility Studies?

A feasibility study examines whether a proposed study or intervention can be undertaken, how it might be implemented, and whether modifications are necessary before proceeding.

A pilot study goes further in a particular direction: it implements a smaller-scale version of the intended definitive study, or selected components of that study, to investigate how the procedures operate in practice.

Eldridge and colleagues developed a conceptual framework distinguishing feasibility studies that are not pilots from pilot studies conducted as preparation for randomized controlled trials. Their framework emphasizes that pilot studies are feasibility studies, but not all feasibility studies are pilots.

Although the framework was developed in the context of randomized trials, its underlying distinction can inform planning in other research settings. Researchers should nevertheless avoid assuming that terminology and procedures transfer identically across all disciplines and designs.

For a broader comparison, see Pilot Study vs. Feasibility Study: What's the Difference?.

What Makes a Pilot Study More Appropriate?

A pilot study becomes particularly useful when the uncertainty concerns how several planned procedures will function together under conditions resembling the definitive study.

For example, a researcher may know that university students can be recruited and that an online learning intervention is acceptable. However, the researcher may not know whether participants can be randomized, receive the intervention, complete scheduled assessments, and remain in the study according to the proposed timeline.

A pilot can rehearse these connected activities and identify problems that might not become apparent when individual procedures are examined separately.

The defining consideration is the need to test the research process in operation rather than merely establish that a particular activity is possible.

When Might a Broader Feasibility Study Be Sufficient?

A broader feasibility study may be more appropriate when the main uncertainty can be investigated without implementing a smaller version of the complete research design.

For example, researchers may need to determine whether a university will permit access to learning analytics, whether sufficient eligible participants exist, or whether teachers consider a proposed intervention acceptable.

These questions can sometimes be answered through stakeholder consultation, preliminary recruitment assessment, institutional records, interviews, or other focused feasibility procedures.

Conducting a full pilot merely to answer an access or acceptability question may consume unnecessary resources.

However, if several feasibility uncertainties interact, a pilot may still be useful.

How Do the Two Approaches Compare?

These distinctions are not absolute divisions between mutually exclusive designs. A pilot can examine recruitment, acceptability, cost, and other feasibility outcomes.

When Is a Pilot Especially Useful for a Randomized Trial?

A pilot randomized trial may be appropriate when researchers need to determine whether the proposed randomized design can be implemented.

Relevant uncertainties may include whether eligible participants agree to randomization, whether allocation procedures operate correctly, whether intervention delivery is practical, and whether outcome assessments can be completed.

For example, a university may plan to evaluate an AI-supported feedback intervention by randomly assigning classes to different instructional conditions.

A pilot could examine the feasibility of class recruitment, allocation procedures, intervention delivery, contamination prevention, and data collection.

It should not ordinarily be treated as a small definitive trial intended to establish intervention effectiveness.

The primary outcomes should address feasibility questions rather than hypothesis tests of treatment effects.

When Is a Pilot Useful for Nonrandomized Research?

Pilot testing can also be useful for observational, qualitative, and mixed-methods studies, although the relevant procedures and terminology may differ.

For example, a longitudinal observational study may need to test whether participants can complete repeated assessments over several weeks.

A mixed-methods study may need to rehearse the coordination of survey administration, participant recontact, interviews, and data linkage.

A qualitative project may pilot an interview guide or observational procedure to examine clarity, timing, and practical suitability.

However, testing one interview guide does not necessarily constitute a pilot of the entire qualitative study. Researchers should describe the scope of the preliminary activity accurately.

Should You Pilot the Whole Study or Only Selected Components?

The answer depends on where uncertainty lies.

If the central concern involves interactions among recruitment, allocation, intervention delivery, follow-up, and data management, a more comprehensive rehearsal may be justified.

If the uncertainty concerns one procedure, a focused pilot or pretest may be sufficient.

For example, a researcher who needs to determine whether questionnaire instructions are understandable may conduct cognitive interviews or a questionnaire pretest without rehearsing the entire study.

The preliminary design should be proportionate to the uncertainty being investigated.

See What Exactly Should a Pilot Study Test? for further consideration of the procedures that may warrant preliminary assessment.

What Questions Should a Pilot Study Answer?

A pilot study should address explicit feasibility objectives derived from the intended definitive design.

Possible questions include whether recruitment can proceed at an adequate rate, whether participants complete the intervention as planned, whether outcome assessments are acceptable, and whether the data management process functions reliably.

Researchers may also examine intervention fidelity, retention, missing data, staff workload, or the practical coordination of study activities.

Not every pilot requires all these outcomes.

The selected outcomes should correspond to the uncertainties that could prevent the definitive study from proceeding successfully.

How Should Pilot Outcomes Be Selected?

Begin by identifying the most consequential risks in the proposed definitive study.

If recruitment is uncertain, recruitment rate or eligibility yield may be a primary feasibility outcome.

If the intervention is complex, fidelity, adherence, and implementation difficulties may be more important.

If the study depends on repeated assessments, retention and data completeness may warrant particular attention.

Researchers should define outcomes operationally and establish how they will be measured.

For example, retention should specify the relevant denominator and follow-up point rather than being reported as an undefined percentage.

Should a Pilot Study Have Progression Criteria?

When a pilot informs a decision about whether to proceed to a definitive study, prespecified progression criteria can make the decision more transparent.

These criteria may identify levels of recruitment, retention, intervention delivery, or data completeness that would support proceeding, modifying the design, or reconsidering the study.

A traffic-light approach may distinguish conditions under which the study can proceed, requires modification, or should not proceed without substantial redesign.

However, progression criteria should be justified in relation to the study rather than adopted from arbitrary universal thresholds.

The decision should also consider uncertainty, contextual information, and whether identified problems are remediable.

Can a Pilot Estimate the Intervention Effect?

A pilot may produce preliminary outcome data, but such estimates are commonly too imprecise to support definitive conclusions about effectiveness.

Small pilot studies are generally not powered to test intervention effects reliably.

A nonsignificant treatment comparison does not establish that the intervention is ineffective, and a statistically significant difference does not automatically justify claims of effectiveness.

Researchers should avoid using pilot hypothesis tests as the principal basis for deciding whether a definitive study should proceed.

Leon, Davis, and Kraemer emphasize that pilot studies should focus on feasibility rather than hypothesis testing.

The limitations of preliminary effect estimates are considered further in What Shouldn't You Conclude From a Pilot Study?.

Can Pilot Data Inform Sample Size Planning?

Sometimes, but caution is necessary.

Pilot data may provide useful information about recruitment rates, retention, outcome variability, or other planning parameters.

However, estimates from small samples can be unstable.

Using a pilot treatment effect estimate as the assumed effect for definitive sample size calculation can be particularly problematic because the estimate may be highly uncertain.

Researchers should consider external evidence, clinically or educationally meaningful effects, and sensitivity analyses when planning the definitive study.

The appropriate approach depends on the design and the parameter being estimated.

What Is an External Pilot Study?

An external pilot is conducted separately from the definitive study.

Its participants and data are not automatically incorporated into the subsequent definitive trial.

Researchers use the pilot findings to evaluate feasibility and decide whether the proposed design should proceed, be modified, or be abandoned.

An external pilot may be particularly useful when substantial changes are anticipated or when the preliminary procedures require separate evaluation.

Whether pilot data can later be incorporated depends on methodological compatibility, ethical arrangements, and the final study design.

What Is an Internal Pilot Study?

An internal pilot is an initial phase incorporated into the definitive study, typically with prespecified rules for assessing feasibility and continuing the research.

When the protocol permits it and essential design features remain compatible, participants and data from the internal pilot may contribute to the final analysis.

However, an internal pilot should not be treated as permission to modify the design freely after examining unblinded treatment effects.

Changes affecting statistical validity, allocation procedures, or outcome definitions require careful methodological consideration.

Internal pilots are particularly relevant to some randomized trial designs and should be planned with appropriate statistical expertise.

How Large Should a Pilot Study Be?

There is no universal pilot sample size.

The appropriate size depends on the feasibility objectives, precision required for relevant estimates, design complexity, and available resources.

For example, a pilot intended to estimate recruitment or retention rates with useful precision may require a different sample from one primarily examining whether a technical procedure functions.

Where the pilot is cluster-randomized, the number of clusters and their characteristics may be more consequential than the total number of individuals alone.

Researchers should justify the sample size according to the pilot objectives rather than treating a small fraction of the definitive sample as an automatic rule.

When Might a Pilot Study Be Unnecessary?

A pilot may be unnecessary when the proposed procedures are sufficiently established, relevant feasibility evidence already exists, and no consequential uncertainty requires rehearsal.

For example, a straightforward survey using an established instrument and familiar recruitment procedures may not require a separate pilot of the complete study.

Nevertheless, targeted pretesting may still be valuable if the instrument is being used in a new language, population, or setting.

The absence of a pilot should be justified by the actual risk profile of the study rather than by an assumption that preliminary testing is always unnecessary.

Can a Pilot Study Reveal That the Definitive Study Should Not Proceed?

Yes. A pilot may show that recruitment is inadequate, intervention delivery is impractical, or outcome collection is unlikely to produce usable evidence.

These findings can justify substantial redesign or a decision not to proceed.

Such an outcome should not automatically be considered a failed pilot.

A pilot that identifies an unworkable research design may prevent the expenditure of considerably greater resources on a definitive study that cannot answer its intended question.

How Should the Choice Be Justified in a Proposal?

The proposal should identify the uncertainty requiring preliminary investigation and explain why a smaller-scale rehearsal is necessary.

Researchers should specify which definitive-study procedures will be piloted, the feasibility outcomes to be assessed, and how the findings will inform subsequent decisions.

Where relevant, progression criteria should be stated before the pilot begins.

The report should distinguish feasibility objectives from effectiveness objectives and avoid implying that a small pilot is intended to establish definitive treatment effects.

Choosing a Pilot for an AI-Supported Feedback Intervention

Why a Pilot Is Preferable in This Situation

The pilot is justified because the principal uncertainty concerns the implementation of the proposed trial as an integrated process.

If the only unresolved question were whether teachers considered the AI feedback system acceptable, a narrower feasibility study might be more proportionate.

Common Misconceptions About Choosing Pilot and Feasibility Studies

How to Decide Whether a Pilot Study Is Necessary

Begin by identifying the precise uncertainty that could prevent the definitive study from proceeding.

If the uncertainty concerns a single procedure, a focused feasibility assessment or pretest may be sufficient. If it concerns how several procedures function together, a pilot may be more appropriate.

The selected preliminary design should be proportionate to the risks and uncertainties of the intended research.

Its purpose is to improve decisions about the definitive study, not to produce a smaller version of every possible analysis.

Before Choosing a Pilot Study, Check:

Frequently Asked Questions About Choosing Pilot and Feasibility Studies

Choose a Pilot When You Need to Test the Study in Operation

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.