When Should You Pilot Part of the Research Plan Before Committing to the Full Study?

When is a pilot worth the extra time and effort?

You can inspect a questionnaire, calculate a sample size, map a recruitment process, rehearse an interview, and discuss a protocol with experienced colleagues. At some point, however, planning reaches questions that cannot be answered convincingly on paper.

Will eligible participants actually agree to take part? Can they complete the procedure as intended? Will a multi-step data collection workflow work in the real setting? Can the intervention be delivered consistently? Will the measurement schedule produce usable data? These are empirical questions about whether the proposed research process works.

A pilot can be valuable in such situations. But “let's pilot it first” should not become an automatic stage in every research project. Piloting costs time, participants, money, and analytical effort. It is most useful when it targets a consequential uncertainty whose answer could change whether or how the full study proceeds.

Pilot when an important uncertainty needs a real-world test

A pilot should reduce uncertainty about the future study

The terminology surrounding pilot and feasibility studies has historically been inconsistent. A useful methodological distinction comes from the conceptual framework developed by Eldridge and colleagues: feasibility is the broader concept concerned with whether something can be done, whether researchers should proceed, and, if so, how. A pilot is a particular form of feasibility work in which the future study, or part of it, is conducted on a smaller scale.

This distinction matters because not every feasibility question requires a pilot. You do not need to recruit 20 participants merely to discover whether a collaborating institution is willing to give you access to its records. A conversation with the appropriate data custodian may answer that question much more efficiently.

Start with the uncertainty, not with the idea of running a pilot

Before planning a pilot, write down exactly what you do not know about the proposed full study.

Perhaps you do not know whether recruitment is fast enough, whether participants will tolerate repeated measurements, whether an intervention can be delivered within ordinary classroom time, whether interview prompts are understood as intended, or whether several sites can implement the same procedure consistently.

Then ask whether the uncertainty matters enough to affect the full study. A pilot becomes more defensible when the answer could lead you to proceed, modify the design, conduct additional feasibility work, or stop.

Pilot when recruitment feasibility is genuinely uncertain

Recruitment plans often look much easier in spreadsheets than in real life. Eligibility may be narrower than anticipated. Gatekeepers may slow access. Potential participants may ignore invitations. Consent rates may be lower than expected.

A pilot can provide information about recruitment processes and rates in the intended setting. That information can help researchers assess whether the full study's recruitment target and timeline are plausible.

However, a tiny pilot conducted under atypical conditions may provide a poor estimate of future recruitment. Context matters. Recruitment by an enthusiastic investigator at one highly cooperative site may not generalize to a larger multi-site study.

Pilot when retention or adherence could determine whether the study works

A design may require participants to attend several sessions, complete repeated surveys, use an intervention for weeks, provide biological samples, maintain diaries, or complete follow-up assessments. Each additional demand creates opportunities for nonadherence and attrition.

If the study's validity depends on participants completing these procedures, preliminary testing can reveal whether the burden is realistic and where participants disengage.

Pilot unfamiliar or complicated procedures

Novel procedures deserve particular attention. A research team may understand each component individually but still encounter problems when those components are combined into a real workflow.

Timing can be wrong. Instructions may be ambiguous. Equipment may not integrate with data systems. Participants may perform tasks differently from what researchers anticipated. Researchers at different sites may interpret procedures inconsistently.

Running the procedure on a limited scale can expose these problems before they are multiplied across the full sample.

Pilot measurement processes when usability is uncertain

There is an important distinction between evaluating whether a data collection process works and establishing that a measure is psychometrically valid for its intended use. A small pilot cannot magically validate an instrument.

It can, however, reveal practical problems: questions that participants misunderstand, response options that do not fit common answers, excessive completion time, technical failures, floor or ceiling patterns worth investigating, missing responses, or instructions that require revision.

If you modify an established instrument, use it in a substantially different population, translate it, or create a new measure, the methodological work required may extend well beyond ordinary pilot testing.

Pilot an intervention when delivery itself is uncertain

For intervention research, feasibility can concern whether the intervention can be delivered as planned, whether participants accept it, whether providers can implement it, and whether fidelity can be assessed.

Complex interventions may contain interacting components and depend heavily on context. Preliminary work can help researchers understand implementation problems before asking whether the intervention produces the intended outcomes.

Pilot when data collection depends on several systems working together

Some studies have long chains of dependency: participant identification, consent, scheduling, data capture, record linkage, laboratory processing, follow-up, coding, and secure storage. Failure at one point may compromise everything downstream.

A small-scale run can test the chain rather than merely inspecting each link separately. This is especially valuable when the study involves multiple sites, research personnel, technologies, or data systems.

Not every preliminary test is a pilot study

You might ask colleagues to review survey items, conduct cognitive interviews, test a software form using dummy data, obtain a preliminary count of eligible records, rehearse laboratory procedures, or consult a statistician about an analysis plan. These activities can be extremely useful without constituting a pilot of the future study.

A pilot needs explicit feasibility objectives

“We will conduct a pilot study” is not an objective. The protocol should specify what the pilot is intended to learn.

Examples might include whether a recruitment strategy reaches enough eligible participants, whether participants complete a particular procedure, whether data can be collected at the planned time points, whether intervention delivery meets a defined fidelity standard, or whether a workflow can be completed within available resources.

Clear objectives determine what data the pilot needs. They also prevent a common problem: conducting a small study first and deciding afterward what it supposedly demonstrated.

Decide in advance what different pilot results would mean

A particularly useful pilot produces actionable information. Before beginning, ask what result would support proceeding unchanged, what result would require modification, and what result would make the planned full study implausible.

These are sometimes formalized as progression criteria. Depending on the project, criteria may concern recruitment, retention, adherence, intervention fidelity, data completeness, acceptability, or other feasibility outcomes.

The thresholds should be justified rather than chosen merely because they make progression likely. They may also need interpretation rather than mechanical application, particularly when several feasibility indicators point in different directions.

Do not use a pilot primarily to test whether the main hypothesis is significant

A small pilot is generally poorly suited to establishing the effectiveness of an intervention or producing a definitive estimate of an effect. Methodological literature has repeatedly cautioned against treating pilot studies as miniature hypothesis-testing trials.

The pilot's central question is about the future study: can it work, what needs to change, and should researchers proceed? Outcome data may sometimes contribute useful information, but the objectives and analysis should remain aligned with feasibility rather than quietly turning the pilot into an underpowered definitive study.

Sometimes piloting adds little

A pilot may be unnecessary when the procedure is already well established in a highly comparable context, feasibility is supported by strong evidence, and no consequential implementation uncertainty remains. It may also be inefficient when the question can be answered more directly through consultation, record review, technical testing, or another simpler feasibility method.

Conversely, if the research question itself is unstable or the design lacks a coherent rationale, piloting may be premature. The project should first resolve the uncertainties that should not be carried into the study.

A pilot that answers a decision rather than merely producing data

The team is unsure whether the problem is serious enough to justify piloting. Rather than asking vaguely whether the intervention “works,” they identify the specific uncertainties that could undermine the full study.

The value of the pilot is not that it provides an early answer to the intervention's effectiveness question. Its value is that it prevents the team from committing substantial resources to a full study whose implementation assumptions have never been tested.

Common mistakes when deciding to pilot a study

Pilot the uncertainty, not the entire project by default

The most efficient preliminary study is often narrower than researchers initially imagine. If your main uncertainty concerns recruitment, you may not need to rehearse every downstream analysis. If it concerns a complicated laboratory workflow, there may be no reason to pilot unrelated survey procedures.

Target the part of the plan where empirical uncertainty is consequential. This keeps feasibility work proportional to the decision it needs to inform.

Where feasibility depends mainly on access to institutions, datasets, equipment, or collaborators, a pilot may also be unnecessary at first. Obtaining preliminary confirmation from the people who control those resources may resolve the uncertainty more directly.

Before deciding to run a pilot

Questions about pilot and feasibility work

Run a pilot when trying the plan teaches you something consequential

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.