Should Data Completeness Be an Outcome in a Feasibility Study?

Should You Measure Whether the Study Produces Complete and Usable Data?

You have designed a feasibility study to examine whether a larger investigation can proceed. Recruitment appears manageable, participants are willing to complete the planned procedures, and the research team has selected appropriate instruments.

But will the resulting dataset contain the information required for analysis?

Participants may skip questionnaire items, miss assessments, provide incomplete responses, or encounter technical problems. Data may also become unusable because of incorrect coding, missing identifiers, or failures during transfer between systems.

Some participants may remain enrolled throughout the study while providing incomplete outcome information. Others may complete every questionnaire but generate measurements that do not adequately represent the intended construct.

Should data completeness therefore be a formal feasibility outcome? How should completeness be measured, and what can the findings legitimately tell you about readiness for the definitive study?

Measure Data Completeness When Missing or Unusable Information Could Compromise the Study

How to Measure and Interpret Data Completeness in Feasibility Research

What Is Data Completeness?

Data completeness concerns the extent to which required research information is available for its intended purpose.

Its operational meaning depends on the study.

For example, a survey may require responses to specified items, a longitudinal study may require measurements at several time points, and a mixed-methods investigation may require records that can be linked across data sources.

A dataset can be complete for one analysis but incomplete for another.

Researchers should therefore define completeness in relation to the research question, planned analysis, and required variables rather than assume that a single percentage captures every aspect of data availability.

Why Is Completeness Important in a Feasibility Study?

A definitive study requires sufficient usable information to answer its research question.

Even when recruitment and intervention delivery proceed successfully, missing or unusable data may reduce statistical precision or prevent the planned analysis.

For example, a study may enroll 200 students but obtain primary outcome scores from only 140.

Whether this creates serious problems depends on the design, reasons for missingness, and analytic assumptions.

A feasibility study can identify where data are lost and whether the proposed collection procedures require modification.

The purpose is not merely to achieve a high completeness percentage, but to establish whether the study can produce interpretable evidence.

How Is Data Completeness Different From Participant Retention?

Retention concerns continued participation or availability for follow-up. Data completeness concerns whether required information is available.

A participant may remain enrolled but skip the final questionnaire. Another may discontinue intervention activities yet provide the primary outcome measurement.

Researchers should not use retention as a substitute for data completeness when the definitive analysis depends on particular observations.

The related distinction is examined in participant retention as a feasibility outcome.

What Levels of Data Completeness Can Be Measured?

Completeness may be assessed at the participant, assessment, variable, item, or dataset level.

These indicators should not be combined indiscriminately. A high percentage of answered questionnaire items may coexist with substantial missingness in the primary outcome.

How Do You Calculate Data Completeness?

A basic completeness proportion compares available usable observations with the observations expected under a defined requirement.

This result describes availability of the specified primary outcome under the observed conditions.

It does not establish that the measurement is valid, that missing outcomes are unrelated to the results, or that the definitive study will achieve the same completeness.

Should Completeness Be Defined for the Primary Outcome?

Often, yes. The primary outcome is central to the definitive study's principal research question and frequently influences sample size and analysis planning.

For example, a randomized educational trial may require a final achievement score for each participant.

A feasibility study can measure the proportion of enrolled participants for whom a usable score is available.

However, researchers should distinguish a score that is merely present from one that satisfies the measurement and scoring requirements.

A technically recorded value may still be unusable if the assessment was administered incorrectly or the score cannot be interpreted.

Primary outcome completeness should therefore be defined according to the intended analysis.

Should Secondary Outcome Completeness Also Be Measured?

Secondary outcomes may warrant attention when their availability is consequential to the definitive study.

For example, an intervention evaluation may measure academic achievement as the primary outcome and student engagement as a secondary outcome.

If engagement questionnaires are frequently incomplete, the researchers may need to revise administration procedures.

However, the feasibility study need not assess every secondary variable with equal intensity.

Priorities should reflect the importance of the information and the uncertainty surrounding its collection.

What Is the Difference Between Missing Data and Unusable Data?

Missing data are observations that are not available in the dataset for the relevant measurement.

Unusable data may be present but unsuitable for the intended analysis because of technical errors, invalid administration, corrupted files, or other problems.

For example, a student may submit an online assessment, but the system may fail to preserve the score correctly.

The record exists, yet the primary outcome may be unusable.

Researchers should define usability criteria and distinguish unavailable observations from records excluded for documented reasons.

These distinctions can reveal different procedural problems.

Should Questionnaire Item Missingness Be Evaluated?

Yes, when questionnaires are important to the definitive analysis.

Item-level missingness may reveal confusing wording, unsuitable response options, sensitive content, or participant fatigue.

For example, a questionnaire may be submitted by nearly every participant while several items concerning institutional support remain unanswered.

The overall questionnaire completion rate may therefore conceal a specific measurement problem.

Researchers should examine which items are missing and investigate plausible reasons.

However, missing responses should not automatically be interpreted as evidence of defective wording. Some questions may be legitimately inapplicable or participants may choose not to answer sensitive items.

How Should Structural Missingness Be Handled?

Structural missingness occurs when a value is not expected because of the study design or questionnaire logic.

For example, students who report never using a particular technology may appropriately skip questions about the frequency of that technology's use.

Those skipped items should not automatically count as incomplete responses.

Researchers should distinguish information that was required but not obtained from information that was never applicable.

Otherwise, the completeness rate may underestimate the adequacy of data collection.

The data dictionary should document the relevant rules.

How Should Data Completeness Be Evaluated Over Time?

Longitudinal studies may experience increasing missingness across repeated assessments.

For example, baseline questionnaires may be nearly complete while final assessments have substantially more missing records.

Researchers should report completeness at each relevant time point rather than rely exclusively on an overall percentage.

This can reveal whether missingness is associated with assessment scheduling, follow-up procedures, or particular stages of the study.

However, observed patterns do not necessarily establish the underlying missing-data mechanism.

Additional information may be required to understand why observations become unavailable.

Can Data Completeness Be High While Data Quality Is Poor?

Yes. Completeness is only one dimension of data quality.

A dataset may contain values for every variable while including incorrect coding, inconsistent measurements, duplicate records, or invalid interpretations.

For example, an online questionnaire may require responses to every item, producing no blank fields.

Participants may nevertheless misunderstand the questions or select arbitrary answers to proceed.

Researchers should therefore distinguish completeness from accuracy, consistency, reliability, and validity.

A complete dataset is not necessarily an informative dataset.

Does High Completeness Establish Measurement Validity?

No. Measurement validity concerns the evidence and theory supporting the intended interpretation and use of scores.

Completeness concerns whether the required observations are available.

For example, every student may complete an engagement questionnaire, but the instrument may not adequately represent engagement in asynchronous learning.

Successful completion does not resolve that conceptual problem.

Researchers should assess measurement suitability independently of data availability.

The distinction is especially important when newly developed or adapted instruments are used.

Can Missing Data Introduce Bias?

Potentially. Missing data may introduce bias when the probability of missingness is related to relevant variables or outcomes in ways not adequately addressed by the analysis.

For example, students experiencing difficulty with an intervention may be less likely to complete follow-up assessments.

If their outcomes are systematically unavailable, the observed results may overrepresent participants with more favorable experiences.

However, the percentage missing does not by itself determine the direction or magnitude of bias.

Researchers should consider the missingness process and the assumptions required by the planned analysis.

Can a Feasibility Study Determine Whether Data Are Missing at Random?

Generally, not conclusively.

Statistical terminology distinguishes missing completely at random, missing at random conditional on observed information, and missing not at random.

These categories involve assumptions about the relationship between missingness and observed or unobserved information.

A feasibility study may identify observed associations or practical reasons for missingness, but it usually cannot establish the full mechanism.

In particular, the missing-at-random assumption cannot generally be verified from observed data alone.

Researchers should avoid claiming that missingness is random simply because no statistically significant differences were detected in a small pilot.

Should Data Completeness Be Measured Separately by Group?

In comparative or randomized studies, group-specific completeness may be informative.

For example, students receiving a demanding intervention may be less likely to complete follow-up assessments than students in the comparison condition.

Researchers may report the observed proportions by group and investigate procedural differences.

However, small feasibility samples may produce imprecise comparisons.

Failure to detect a statistically significant difference does not establish equivalent completeness across groups.

The objective is to identify potential risks rather than make definitive claims about differential missingness.

How Should Completeness Be Evaluated in Cluster Studies?

Cluster studies may experience missingness at both the cluster and individual levels.

For example, a class may participate in the intervention while some students fail to provide outcome data.

Alternatively, an entire institution may withdraw, resulting in the loss of many observations.

These events have different consequences for the analysis.

Researchers should therefore examine completeness according to the relevant unit and data structure.

An overall individual-level percentage may conceal the loss of important clusters.

What About Qualitative Data Completeness?

Qualitative research requires a different interpretation of completeness.

It may involve whether interviews were recorded successfully, transcripts preserve relevant material, field notes are available, or necessary contextual information was documented.

However, qualitative completeness should not be reduced to whether every participant answered every interview question.

Many qualitative approaches permit participants to introduce unexpected topics or decline particular questions.

The adequacy of qualitative material depends on the methodology, research question, and analytic purpose.

A feasibility study may assess technical and procedural completeness without imposing inappropriate standardization on qualitative inquiry.

How Should Completeness Be Evaluated in Mixed-Methods Research?

Mixed-methods research may require information from different strands to be connected.

For example, an explanatory sequential study may use survey results to select participants for follow-up interviews.

Even if the survey and interview datasets are individually complete, integration may be compromised if the required linkage information is missing.

Researchers should determine whether the data needed for integration can be collected and preserved appropriately.

Completeness may therefore include the availability of required case identifiers, contextual information, or analytic categories.

However, technical linkage does not automatically establish the quality of mixed-methods interpretation.

Should Data Completeness Be Tested Using Synthetic Data?

Synthetic records can help verify whether the data collection and management systems correctly handle missing values, invalid entries, and incomplete records.

For example, researchers may construct test cases containing blank questionnaire responses, unavailable follow-up scores, or intentionally missing identifiers.

These tests can reveal coding or export errors.

However, synthetic data cannot establish how frequently real participants will omit information.

Technical verification and empirical feasibility assessment provide different forms of evidence.

The broader workflow is addressed in testing the analysis pipeline with dummy or pilot data.

Can Mandatory Questionnaire Responses Improve Completeness?

Mandatory fields may reduce blank responses in electronic questionnaires, but they are not always appropriate.

Participants may have legitimate reasons to decline sensitive questions or may lack a suitable response option.

Forcing an answer can produce inaccurate information or encourage withdrawal.

Researchers should consider whether an item is essential, whether a not-applicable option is appropriate, and whether participants must retain the ability to decline particular responses.

Higher apparent completeness should not be achieved by compromising response quality or ethical safeguards.

Should Data Completeness Be a Primary Feasibility Outcome?

It may be primary when uncertainty about obtaining usable information represents the central threat to the definitive study.

For example, a longitudinal investigation may have reliable recruitment procedures but uncertain follow-up assessment completion.

In that situation, primary outcome completeness may be the most consequential feasibility indicator.

However, completeness need not be primary when other uncertainties dominate.

The outcome hierarchy should reflect the study's specific objectives.

What Completeness Rate Is Acceptable?

There is no universal percentage that establishes adequate data completeness.

The appropriate threshold depends on the research design, importance of the missing variables, planned analysis, and consequences of unavailable observations.

For example, 95% overall item completion may be insufficient if the missing 5% concerns the only primary outcome.

Conversely, missing values in optional descriptive variables may have less serious consequences.

Researchers should justify thresholds according to the information required for the definitive study rather than adopt an arbitrary convention.

How Should Completeness Progression Criteria Be Established?

Progression criteria should connect completeness findings to the feasibility of the definitive analysis.

For example, researchers may identify the minimum proportion of usable primary outcome records required under the planned sample size and analytic assumptions.

However, the percentage alone may be insufficient.

Systematic missingness concentrated in one study group or participant subgroup may be consequential even when overall completeness appears favorable.

Researchers should consider uncertainty, missingness patterns, and the feasibility of corrective changes.

The role of prespecified progression criteria is to support transparent decisions, not replace methodological judgment.

How Large Should a Data Completeness Feasibility Study Be?

There is no universal sample size.

A small pilot may identify obvious technical failures or confusing questionnaire items.

Estimating completeness proportions with useful precision may require more observations.

Longitudinal studies may also need an observation period long enough to examine relevant follow-up procedures.

Researchers should justify the sample and duration according to the completeness questions and decisions being investigated.

They should avoid treating an imprecise preliminary percentage as a reliable prediction of definitive-study data availability.

Can Data Completeness Inform Sample Size Planning?

Yes. Expected outcome availability may influence the number of participants who need to be recruited.

For example, if the planned analysis requires 180 usable final outcome observations and expected completeness is 90%, a simple approximation suggests enrolling 200 participants.

However, this calculation does not account for all design-specific issues, including clustering, unequal group missingness, or bias arising from missing outcomes.

Increasing enrollment may address some losses of precision but cannot automatically correct informative missingness.

Sample size planning should reflect the definitive study's statistical requirements.

What If Completeness Is Lower Than Expected?

Researchers should identify where information becomes unavailable and why.

Missing questionnaire items may require clearer wording or more suitable response options.

Missed assessments may require revised scheduling or communication procedures.

Incorrectly linked records may require data management changes.

However, corrective actions should preserve the intended measurement and participant protections.

Substantial modifications may require further testing before the definitive study.

Should Researchers Impute Missing Pilot Data?

Imputation may be appropriate for certain analyses, but it should not be used to conceal the observed extent of missingness in a feasibility report.

The raw completeness indicators should describe what information was actually obtained.

Researchers may separately examine whether a planned missing-data analysis procedure can be implemented.

However, successful imputation does not mean that the missing observations were collected or that the assumptions underlying imputation are satisfied.

Feasibility reporting should distinguish observed data availability from statistical handling of missing values.

Does High Completeness Mean the Definitive Study Is Feasible?

Not by itself.

A study may obtain complete data while recruitment is inadequate, intervention delivery is inconsistent, or institutional resources are insufficient.

High completeness also does not establish that the measures are valid or that missing-data bias is absent.

Completeness should therefore be interpreted alongside the other feasibility outcomes necessary for the definitive design.

The objective is to determine whether the proposed research can produce credible evidence under realistic conditions.

How Should Data Completeness Findings Be Reported?

Researchers should define the required observations, identify the relevant denominators, and report usable and missing records at appropriate levels.

Where relevant, results should be presented by assessment time, study group, site, or variable.

The report should distinguish missing, structurally inapplicable, invalid, and technically unusable values.

Reasons for missingness should be described when available, without claiming more certainty than the evidence supports.

Finally, the report should explain how completeness findings influence the proposed definitive study.

Evaluating Data Completeness in a Longitudinal Educational Study

Illustrative Completeness Indicators

The findings show that different completeness indicators produce different results.

The 90% retention rate does not mean that 90% of students provided usable primary outcome data. Similarly, the proportion submitting the engagement questionnaire exceeds the proportion completing all required items.

The researchers should investigate the causes of these differences and determine whether the resulting information will support the definitive analysis.

The pilot cannot establish that the same missingness pattern will occur in the main study or that the observed data are free from bias.

Common Misconceptions About Data Completeness

How to Decide Whether Data Completeness Should Be a Feasibility Outcome

Begin by identifying the information essential to answering the definitive research question.

Then determine whether uncertainty about collecting, preserving, or using that information could compromise the planned analysis.

The relevant outcome is not simply the proportion of nonblank cells. It is whether the information necessary for meaningful analysis can be obtained under the intended research conditions.

Before Using Data Completeness as a Feasibility Outcome, Check:

Frequently Asked Questions About Data Completeness in Feasibility Studies

Measure Completeness When Missing Information Could Undermine the Definitive Analysis

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.