01 · The Question
Can the First AI Suggestion Shape What Researchers Decide Later?
A researcher asks an AI assistant how many participants a study might require. The system suggests 120. After examining the study design, the researcher recognizes that the recommendation was not based on a defensible sample-size calculation.
Nevertheless, subsequent discussions continue to revolve around whether the study needs slightly more or slightly fewer than 120 participants. Other possibilities receive less attention.
The original AI recommendation may have become an anchor: a starting point that influences later judgments even when its evidential basis is weak.
This possibility matters because AI systems frequently provide initial suggestions for research questions, methodological choices, analytical approaches, and interpretations. An early recommendation may influence not only the immediate decision but also how researchers evaluate subsequent information.
03 · What You Need to Know
How AI Suggestions Can Become Anchors in Research
What Is Anchoring Bias?
Anchoring describes the influence of an initial reference point on subsequent judgments.
Tversky and Kahneman (1974) identified adjustment from an anchor as one of the heuristics people use when making judgments under uncertainty. Their work showed how starting values can influence numerical estimates, sometimes producing systematic errors.
For example, a person asked to estimate an unfamiliar quantity may begin with a suggested number and adjust upward or downward. The final estimate can remain influenced by the starting point, even when that point has little relevance to the correct answer.
In research, an anchor might be a suggested sample size, expected effect magnitude, anticipated publication timeline, or numerical interpretation supplied by AI.
The broader concern is that an initial recommendation may influence what researchers regard as reasonable before they have independently established the appropriate criteria.
Anchoring Is Not Limited to Numerical Estimates
The classical anchoring literature provides particularly strong foundations for understanding numerical judgments. Research discussions also use anchoring to describe how initial ideas or reference points may influence subsequent evaluation.
For AI-assisted research, a numerical anchor is relatively straightforward to identify. An AI-generated estimate of 200 participants might influence later sample-size discussions.
Other possible influences are more conceptual. An AI assistant might recommend a qualitative research design, identify a particular theoretical framework, or propose an interpretation of findings. That initial suggestion could become the reference against which alternatives are considered.
However, researchers should avoid assuming that all first-suggestion effects are psychologically identical to classical numerical anchoring.
Conceptual recommendations may also influence decisions through framing, confirmation bias, perceived authority, or ordinary information updating. Establishing the mechanism requires more than observing that the first recommendation affected the final decision.
Why an Initial AI Recommendation May Receive Disproportionate Attention
An AI recommendation can become influential because it provides an immediate starting point for an otherwise uncertain decision.
Consider a researcher who has not yet determined the appropriate statistical model. An AI assistant recommends hierarchical linear modeling and explains why it may be suitable.
The researcher may then concentrate on evaluating that model rather than first identifying the full range of approaches compatible with the study design.
This is not necessarily irrational. A useful recommendation can reduce the effort required to identify promising options.
The problem arises when the initial suggestion restricts the evaluation process without sufficient justification.
For instance, the researcher might compare variations of the recommended model while overlooking a simpler approach that better matches the research question and data structure.
The potential bias concerns the narrowing of subsequent consideration, not merely the fact that an initial suggestion was influential.
What Does Research on Algorithmic Advice Suggest?
Research on advice-taking provides useful evidence about how people integrate recommendations from external sources.
A study titled Advice Taking Under Uncertainty: The Impact of Genuine Advice Versus Arbitrary Anchors on Judgment examined how advice and arbitrary anchors influenced judgments. The researchers reported that the relevance of advice to the focal judgment mattered, while arbitrary anchors could still exert influence. Their findings also indicated that identifying the source as human rather than computer did not change the observed results under the tested conditions.
This suggests that the influence of an initial recommendation cannot always be explained simply by whether the source is human or automated.
Another study, Algorithm-Based Advice Taking and Clinical Judgement: Impact of Advice Distance and Algorithm Information, examined advice-taking in a clinical judgment context. It discussed anchoring and insufficient adjustment as relevant mechanisms in how individuals integrate their own initial estimates with algorithmic advice.
These studies provide evidence relevant to AI-assisted decision-making, but their settings differ from academic research design and interpretation. Researchers should therefore avoid assuming that their findings directly quantify anchoring effects among scientists using generative AI.
How Might AI Anchoring Affect Research Design?
Research design involves decisions that are often made before all relevant information is available.
AI suggestions may become influential when researchers are still exploring possible approaches.
| Research decision |
Possible AI anchor |
Potential consequence |
| Sample-size planning |
An unsupported numerical estimate |
Later planning remains centered on the suggested number rather than a justified calculation |
| Research question development |
An initially proposed research question |
Alternative formulations receive less consideration |
| Methodological selection |
A recommended design or analytical approach |
Evaluation concentrates on the proposed method rather than the study's requirements |
| Theoretical framing |
An AI-suggested conceptual framework |
Relevant alternative theories may be overlooked |
| Data interpretation |
An initial explanation of an unexpected result |
Subsequent interpretation may remain oriented toward the first explanation |
| Manuscript evaluation |
An AI-generated assessment of a paper's strengths or weaknesses |
Later reviewers may focus disproportionately on the issues initially identified |
These are hypothetical pathways through which early AI suggestions might influence research decisions. They should not be interpreted as empirically established effects of equal magnitude across the listed activities.
Can Anchoring Occur Even When Researchers Know the First Suggestion Is Weak?
Potentially. The anchoring literature demonstrates that starting points can influence judgments under conditions in which the anchor does not provide a sound basis for the final estimate.
However, knowing that an AI recommendation is unreliable may reduce its influence, and the effect will depend on how the researcher evaluates subsequent evidence.
A researcher who receives an unsupported sample-size estimate might reject it explicitly but continue using the same number as a reference during planning.
That possibility illustrates why identifying an error and removing its influence are not necessarily equivalent.
The related problem of whether an incorrect AI recommendation continues to influence judgment after correction deserves particular attention when researchers revise consequential decisions.
How Is Anchoring Different From Automation Bias?
Anchoring and automation bias can produce superficially similar outcomes, but they concern different mechanisms.
Anchoring bias
An initial reference point influences subsequent judgment, potentially leading to insufficient adjustment.
Automation bias
Automated information or recommendations receive inappropriate priority over other relevant evidence.
A researcher may be anchored by an initial estimate supplied by another person, a published paper, or an AI system. The influence does not require a belief that automation is especially authoritative.
Conversely, automation bias
The two processes can coexist. A researcher may both overvalue an AI recommendation because of its source and use its initial numerical estimate as a reference for later decisions.
Does a Confidently Presented AI Suggestion Create a Stronger Anchor?
A confidently presented recommendation may appear more credible, potentially affecting how seriously researchers consider it.
However, confidence effects and anchoring effects should not be conflated.
The evidence that confidence cues can affect advice acceptance does not by itself establish that they increase anchoring in every research setting.
The relationship between AI confidence and researcher trust should therefore be considered separately from the influence of initial reference points.
Researchers evaluating these phenomena empirically would need designs capable of distinguishing source credibility, confidence framing, and anchoring effects.
Can AI Anchoring Be Reduced by Forming an Independent Judgment First?
Developing an initial assessment before viewing AI advice may help researchers recognize when an automated suggestion changes their judgment.
However, independent initial judgments can themselves become anchors. Research on advice-taking has examined how people sometimes give disproportionate weight to their own estimates when integrating external advice.
Therefore, the objective is not to privilege the researcher's first opinion automatically.
A stronger approach is to establish relevant decision criteria, examine independent evidence, and evaluate alternative recommendations using those criteria.
The researcher's initial judgment and the AI's recommendation should both remain open to revision.
04 · A Practical Example
How an AI-Generated Sample Size Can Influence Research Planning
Hypothetical Example
Planning an Educational Intervention Study
A researcher plans to compare student outcomes across instructional conditions. Before specifying the expected effect, study design, allocation structure, or statistical assumptions, the researcher asks AI how many participants might be needed.
The AI suggests 150 participants and describes this as a reasonable target.
The researcher later learns that the number was not derived from a study-specific power analysis or precision-based calculation. Nevertheless, subsequent planning discussions continue to treat 150 as the expected recruitment target.
1. Initial anchor
The AI supplies a numerical estimate of 150 participants before the necessary design parameters have been established.
2. Subsequent adjustment
The researcher considers recruiting 130 or 170 participants, but the discussion remains centered on the original estimate.
3. Independent assessment
The researcher specifies the study design, intended analysis, effect-size assumptions or precision objectives, significance criterion where relevant, and other necessary parameters.
4. Evidence-based planning
An appropriate sample-size justification is developed without treating the AI-generated number as an authoritative starting point.
The final sample size could be smaller than, equal to, or larger than 150. The numerical difference alone would not establish whether anchoring occurred.
The important issue is whether the researcher allowed an unsupported initial estimate to influence subsequent decisions when a more appropriate evidential basis was available.
A defensible recruitment target should follow from the study's objectives and methodological requirements rather than from the apparent reasonableness of the first number encountered.
06 · What This Means for You
How to Prevent Early AI Suggestions From Dominating Research Decisions
Researchers do not need to avoid initial recommendations altogether. They need procedures that prevent unsupported starting points from becoming the basis of consequential decisions.
A useful approach is to distinguish between generating possible options and deciding which option is justified.
A simple decision framework
If AI provides an initial numerical estimate
Identify the variables, assumptions, and calculations needed to justify the estimate independently.
If AI recommends a research design or method
Compare it with credible alternatives using criteria derived from the research question and study constraints.
If a recommendation has already been rejected
Reconsider the decision from its underlying requirements rather than merely adjusting the original suggestion.
If multiple researchers are evaluating the same AI advice
Consider collecting independent assessments before discussing the automated recommendation.
Define Decision Criteria Before Requesting Recommendations
Before asking AI to recommend a method, researchers can specify what the method must accomplish and which constraints are relevant.
For sample-size planning, this means establishing the research objective and the parameters required by the chosen justification method.
For theoretical framing, it means identifying the phenomenon under investigation and the explanatory requirements a suitable framework should satisfy.
This approach supports independent evaluation before consulting AI without requiring researchers to commit prematurely to a particular answer.
Use Independent Evidence to Reassess the Decision
When an AI recommendation appears questionable, simply asking the same system to revise it may not provide sufficient independence.
Researchers should return to the relevant data, original literature, methodological standards, or qualified expertise.
For example, an unsupported AI sample-size estimate should be replaced by an appropriate calculation or justification, not merely adjusted until it seems more reasonable.
Consider Independent Assessments Within Research Teams
When several researchers must evaluate a consequential recommendation, collecting initial assessments separately may help prevent one prominent suggestion from defining the discussion.
This is a practical risk-reduction strategy rather than a guarantee that anchoring will disappear.
Its effectiveness may depend on whether team members have access to the same evidence, whether the decision criteria are explicit, and whether participants can genuinely challenge the proposed approach.
Teams may also consider reviewing recommendations without identifying their source when the purpose is to examine the influence of AI attribution. Source-blind evaluation and anchoring mitigation address different concerns, so neither should automatically be treated as a substitute for the other.
Document Why the Final Decision Was Made
For consequential research decisions, documentation should explain the evidence and reasoning supporting the final choice.
Recording an AI recommendation can be useful for transparency, but the recommendation itself should not become the justification.
A defensible decision record should show why the selected option satisfies the research requirements, what alternatives were considered, and which assumptions remain uncertain.
