03 · What You Need to Know
Designing a Survey That Anticipates and Reduces Nonresponse
Understand What Survey Nonresponse Actually Means
Nonresponse occurs when information sought from eligible survey units is not obtained.
Two forms are particularly important.
Unit nonresponse
An eligible sampled or enumerated unit does not provide a usable survey response according to the study's specified disposition rules.
Item nonresponse
A participating respondent does not provide an answer to a particular applicable question or item.
The causes may differ. A teacher who never receives a survey invitation contributes to a coverage or contact problem, while a teacher who receives the invitation but declines may contribute to unit nonresponse. A teacher who completes the questionnaire but skips a question about income contributes item nonresponse for that question.
These distinctions matter because different problems require different preventive measures.
Why Nonresponse Matters Beyond the Response Rate
Nonresponse reduces the amount of observed information and may decrease statistical precision. More importantly, it can introduce bias when respondents differ systematically from nonrespondents on characteristics relevant to the research estimates.
Suppose a university surveys students about difficulties accessing online learning. Students with unstable internet connections may be less likely to complete an online questionnaire.
If those students experience greater access difficulties, estimates based on respondents may understate the problem.
However, the existence and direction of bias cannot be inferred from nonresponse alone. They depend on how participation relates to the variables being estimated.
Research by Groves (2006) explains why nonresponse rates and nonresponse bias should not be treated as equivalent. A high response rate can coexist with systematic differences, while a lower response rate does not necessarily imply substantial bias for every estimate.
The goal is therefore not merely to maximize completed questionnaires. It is to obtain credible information from the intended population.
Begin with a Clearly Defined Target Population
Nonresponse prevention starts with identifying exactly who is eligible to participate.
Researchers should specify inclusion and exclusion criteria before recruitment. These may involve institutional affiliation, employment status, enrollment period, professional role, or other characteristics relevant to the research question.
Next, evaluate the sampling frame or population list.
An outdated directory may contain incorrect contact details, duplicate entries, or people who are no longer eligible. It may also omit newly eligible members.
These problems can complicate recruitment and distort participation statistics.
Where possible, verify contact information and eligibility before launching the survey. Maintain clear procedures for classifying ineligible, unreachable, and eligible nonresponding units.
The choice between a census and a sample survey does not remove these responsibilities. Inviting everyone does not guarantee that everyone can be contacted or will respond.
Identify Which Groups May Be Least Likely to Participate
Participation barriers are rarely distributed uniformly across a population.
Some respondents may have limited internet access, demanding work schedules, language barriers, accessibility needs, or concerns about confidentiality.
Others may consider the topic irrelevant or distrust the organization requesting information.
Before fieldwork, researchers should consider whether these barriers are associated with characteristics central to the study.
For example, if the research concerns workload among healthcare workers, those experiencing the heaviest workloads may have the least time to participate.
If the study concerns digital exclusion, an online-only survey may be particularly problematic.
Researchers can consult relevant literature, population information, stakeholder representatives, and pilot participants to identify plausible barriers.
Such assessments should inform the recruitment strategy without assuming that all members of a demographic group have identical participation preferences.
Choose an Administration Mode That Does Not Unnecessarily Exclude Respondents
The administration mode determines how participants encounter and complete the questionnaire.
An online survey may be efficient for a digitally connected population. Paper questionnaires, telephone administration, or interviewer assistance may be more appropriate for some other groups.
Researchers should evaluate available survey administration modes according to population access, respondent capabilities, privacy, measurement demands, and resources.
Mixed-mode administration may help reach people who would otherwise be excluded, but it also introduces potential differences in measurement and respondent selection.
Consequently, the decision should consider how administration mode can affect participation and answers, not simply which approach appears likely to produce more completed forms.
Reduce Unnecessary Questionnaire Burden
Respondents may decline or discontinue a survey when its expected demands exceed the time or effort they are willing or able to contribute.
Questionnaire length is one consideration, but burden also depends on question complexity, recall requirements, sensitive content, navigation, and the administration environment.
Before fieldwork, review every question against the research objectives.
Remove unnecessary items, simplify confusing instructions, and use appropriate routing to avoid asking respondents questions that do not apply to them.
However, reducing burden should not involve removing essential measures or arbitrarily shortening established scales.
The objective is to balance questionnaire length and response quality while retaining adequate evidence for the intended analysis.
Write Invitations That Explain the Survey Clearly
The invitation is often the first substantive interaction between the researcher and a potential respondent.
It should communicate enough information for an informed participation decision without overwhelming the recipient.
Depending on the study and applicable ethics requirements, an invitation may need to identify:
- The responsible researcher or institution.
- The purpose of the study.
- Why the recipient has been invited.
- What participation involves.
- The realistic estimated completion time.
- Whether participation is voluntary and whether questions may be skipped.
- How responses and personal information will be protected.
- How participants can ask questions or verify the invitation's legitimacy.
Invitations should avoid exaggerated claims about anonymity, confidentiality, or the benefits of participation.
For example, a survey using unique links that permit the research team to connect responses with participants should not be described as anonymous merely because results will be reported in aggregate.
Trust is better supported through accurate communication than through assurances the research procedures cannot substantiate.
Plan Contact Attempts and Reminders Before Launch
A single invitation may not reach respondents at a convenient time. Some recipients overlook messages, postpone participation, or intend to respond but forget.
A planned contact strategy may include an initial invitation, appropriate reminders, and alternative contact procedures where justified.
The number and timing of contacts should reflect the population, survey duration, ethical requirements, and communication context.
There is no universally optimal reminder schedule.
Researchers should avoid excessive contact that creates annoyance, perceived pressure, or privacy risks.
Where individual response status can be tracked lawfully and appropriately, reminders can be directed to nonrespondents rather than repeatedly contacting everyone.
For anonymous surveys without individual tracking, researchers may need to send general reminders while clearly asking those who have already participated to disregard them.
Reminder procedures should be documented and applied consistently with the study protocol.
Consider Whether Incentives Are Appropriate
Incentives may encourage participation by recognizing respondents' time and effort.
They may take monetary or nonmonetary forms, depending on the population, research context, available resources, and ethical requirements.
Research syntheses, including work by Singer and Ye (2013), indicate that incentives can affect survey participation, although their effects vary across designs and populations.
Researchers should not assume that a particular incentive will work equally well for all groups.
Incentives also require consideration of voluntariness, fairness, institutional policy, and whether the amount or form could create inappropriate pressure.
Where incentives are used, the eligibility rules and conditions for receiving them should be communicated accurately.
Increasing participation is useful, but an incentive does not automatically remove nonresponse bias.
Make Participation Accessible
Accessibility is a methodological issue because inaccessible instruments may systematically exclude eligible respondents.
Online questionnaires should be evaluated for compatibility with assistive technologies, keyboard navigation, readable layouts, and appropriate display on different devices.
Language should be suitable for the intended population, and translations should be evaluated for conceptual and linguistic equivalence when necessary.
Paper, telephone, or interviewer-assisted alternatives may be appropriate for respondents whose needs cannot be met through the primary mode.
Researchers should test accessibility before launching rather than relying on platform claims alone.
Providing an alternative mode can improve access, but researchers should also consider whether it changes how questions are understood or answered.
Use Pretesting to Identify Causes of Item Nonresponse
Item nonresponse may occur because respondents cannot remember the requested information, do not understand the question, consider it inapplicable, or prefer not to disclose the answer.
These causes should not be treated as interchangeable.
For example, a question about annual income may be skipped because respondents are uncertain about the relevant income definition, do not know the amount, or consider the information too sensitive.
Cognitive interviewing and pilot testing can help identify these problems.
Researchers should examine whether response options include necessary categories such as "Not applicable" or "Don't know," where conceptually appropriate.
Such options should not be added automatically to every question, because they may alter response behavior and interpretation.
Likewise, forcing answers to every item can reduce missing entries while producing arbitrary or inaccurate responses.
Item nonresponse should be addressed through appropriate measurement design, not merely software restrictions.
Protect Voluntary Participation in Hierarchical Settings
Survey recruitment can create ethical and methodological difficulties when potential respondents have dependent relationships with those requesting participation.
Examples include students recruited by their instructors, employees contacted by supervisors, and patients approached by healthcare professionals responsible for their care.
Participants may perceive that declining could have consequences, even when researchers intend participation to be voluntary.
Researchers should consider recruitment through appropriately independent channels, clear statements of voluntariness, and procedures preventing authority figures from accessing identifiable response status where unnecessary.
These safeguards may also improve response quality by reducing pressure to provide favorable answers.
Ethical approval and applicable institutional requirements should be addressed before recruitment begins.
Should You Use Personalized Survey Invitations?
Personalized invitations can sometimes improve the relevance and credibility of survey requests.
For example, an invitation explaining that a teacher was selected from a defined institutional sampling frame may be more meaningful than a generic message sent to an unrestricted mailing list.
However, personalization requires appropriate handling of personal information and should not imply that respondents are being monitored unnecessarily.
Researchers should distinguish personalization used for contact management from the collection of identifiable survey answers.
When unique survey links are used, participants should receive accurate information about whether their responses can be linked to their identities.
Plan How Response Rates Will Be Calculated
Response-rate reporting should be considered before data collection because it depends on accurate records of sample dispositions.
The American Association for Public Opinion Research provides standardized definitions for classifying outcomes and calculating response, cooperation, refusal, and contact rates.
Different response-rate formulas may be appropriate depending on the survey design and how cases of unknown eligibility are handled.
Researchers should therefore avoid calculating a response rate by dividing completed questionnaires by an arbitrary number of invitations without examining eligibility and disposition definitions.
For a straightforward survey in which all invited units are known to be eligible, the following simplified calculation may be useful:
Maintaining accurate disposition records from the beginning makes transparent reporting possible later.
Plan a Nonresponse Bias Assessment Before Collecting Data
Nonresponse bias cannot be evaluated adequately using the response rate alone.
Researchers should identify information that may be available for both respondents and nonrespondents, subject to privacy and data-access requirements.
Examples may include institutional role, academic unit, geographic location, or other frame characteristics relevant to the research question.
Comparing respondents with the target population on known characteristics can reveal some forms of imbalance.
Researchers may also consider follow-up studies of nonrespondents, comparisons across recruitment waves, weighting adjustments, or sensitivity analyses.
However, these procedures have limitations. For example, late respondents are not automatically equivalent to nonrespondents, and agreement on observed demographic characteristics does not establish agreement on unobserved outcomes.
Weighting can reduce some imbalances when appropriate auxiliary information and assumptions are available, but it cannot guarantee elimination of nonresponse bias.
The assessment strategy should therefore be connected to the survey's substantive estimates and intended inference.
Why Prevention Should Be Planned Before Fieldwork
Some nonresponse problems become difficult to address after recruitment has begun.
If the original sampling frame excludes an important population group, reminders sent to listed participants will not reach that group.
If the questionnaire contains confusing or overly demanding items, researchers may discover the problem only after substantial missing data have accumulated.
Changing questions or administration procedures midway through fieldwork may also create comparability problems.
A documented pre-fieldwork plan should therefore address population coverage, recruitment, questionnaire burden, contact procedures, accessibility, data protection, response monitoring, and the intended nonresponse analysis.
The plan should allow justified operational adjustments while preserving transparency about what changed and why.
