03 · What You Need to Know
Understanding When Documents Constitute Research Evidence
What Does It Mean to Treat Documents as Primary Research Data?
Documents become primary research data when they are systematically examined as evidence relevant to the study's research questions.
For example, a researcher investigating how universities define acceptable generative AI use may analyze official institutional policies to identify permitted activities, disclosure requirements, and responsibility assignments.
The policies are not merely references supporting an introduction. Their provisions, terminology, and relationships constitute the material being analyzed.
The researcher may compare documents, examine recurring patterns, identify differences, and develop interpretations grounded in their content and context.
Bowen (2009) describes document analysis as a systematic procedure for reviewing and evaluating documentary materials. The approach can support independent research or contribute to studies using additional methods.
Does Primary Data Mean the Researcher Must Collect Something New?
No. The terms primary source, primary data, and primary data collection are related but should not be treated as interchangeable.
A historical policy document may be a primary source for research about the policy's development even though it was created decades earlier.
Likewise, an existing institutional memorandum may provide direct evidence about the institution's formal communication at a particular time.
However, a study analyzing an existing dataset may conventionally be described as secondary data analysis because the data were originally collected for another purpose.
The appropriate terminology depends on the discipline, the origin of the material, and its role in the research.
Researchers should explain whether documents are original sources for the phenomenon, previously collected materials being reanalyzed, or contextual evidence supporting another data source.
When Are Documents Particularly Suitable?
Documentary evidence is particularly useful when the research concerns formal rules, institutional representations, recorded decisions, public communication, historical developments, or the language through which organizations construct meaning.
For example, a study may examine how national education policies assign responsibilities for AI-assisted assessment.
The relevant evidence may consist of policy documents, official guidelines, implementation circulars, and published institutional statements.
Interviews could provide additional perspectives, but they are not necessarily required to establish what the documents formally prescribe.
The suitability of documentary evidence depends on the alignment between the question and the information the documents can reasonably provide.
Documents Can Be Evidence of Institutional Positions
Official documents can provide evidence about what institutions formally state, require, authorize, or prohibit.
For example, an academic integrity policy may specify whether students must disclose generative AI assistance.
A researcher can analyze that provision without interviewing the policy's authors.
However, the document does not establish whether students follow the requirement, whether instructors enforce it consistently, or whether the institution has adequate resources for implementation.
These are different empirical questions.
Researchers should distinguish institutional prescriptions from observed institutional practices.
Documents Can Reveal How Concepts Are Constructed
Some studies are concerned less with what documents prescribe than with how they represent particular ideas.
For example, a researcher may examine whether university AI policies frame students primarily as responsible decision-makers, potential rule violators, or learners requiring institutional support.
Such an investigation may analyze language, categories, assumptions, and recurring forms of justification.
Documents are appropriate evidence because the representation itself is the phenomenon under investigation.
However, interpretations should be grounded in the analytical approach and documentary context rather than inferred from isolated words without considering their use.
What Types of Documents Can Be Used?
Research documents may include policy statements, administrative records, meeting minutes, institutional reports, curriculum materials, correspondence, public announcements, archival records, websites, and other recorded texts.
Digital materials can also qualify when they constitute stable or appropriately documented research objects.
The boundaries of what counts as a document for research purposes depend on the research question and conceptualization of documentary evidence.
Researchers should avoid restricting documents to printed materials while also avoiding the assumption that every accessible text is relevant data.
What Is the Difference Between Document Analysis and a Literature Review?
A literature review ordinarily examines existing scholarship to establish what is known, identify debates, and position a study within a field.
Document analysis examines selected documentary materials as evidence about a research phenomenon.
For example, reviewing published studies about AI governance is different from analyzing university AI policies to investigate how responsibility is formally allocated.
The same publication could serve different purposes in different studies. A scientific article might be background literature in one project and an object of analysis in research examining scholarly representations of educational AI.
The distinction depends on the document's analytical role, not simply its publication type.
Can Documents Be the Only Data Source?
Yes. A document-only study can be methodologically defensible when the documents provide sufficient evidence to answer the research question.
For example, a comparative study of official AI assessment policies across universities may reasonably rely on a systematically defined collection of policy documents.
The researcher can identify similarities, differences, omissions, and institutional approaches without collecting participant accounts.
However, a question about how teachers actually experience those policies would ordinarily require evidence beyond the policy texts.
The sufficiency of documents is therefore an issue of evidentiary alignment rather than a requirement to use multiple data-collection methods.
How Do You Decide Whether Documents Are Sufficient?
Begin by identifying the exact claims the study intends to make.
If the intended conclusion concerns what a document states, how language is organized, or how institutional requirements differ, documentary analysis may be sufficient.
If the conclusion concerns actual behavior, individual experiences, policy effectiveness, or causal effects, documentary evidence may be insufficient unless the documents contain appropriate records of those phenomena.
For example, a policy can establish that a university requires AI disclosure. It cannot, by itself, establish the proportion of students who comply.
Researchers should identify which claims the documents directly support and which would require additional evidence.
Documentary Evidence Is Not Necessarily Neutral
Documents are produced by people and organizations for particular purposes.
An institutional annual report may emphasize achievements, while an internal audit may document implementation problems.
Neither should automatically be treated as a complete account of organizational reality.
Atkinson and Coffey (1997) argue for understanding documents as social facts rather than transparent reflections of an independent reality.
This perspective encourages researchers to examine how documents are produced, circulated, authorized, and used.
Documentary claims should be interpreted in relation to their origins and institutional functions.
How Do You Assess Authenticity and Credibility?
Authenticity concerns whether a document is what it purports to be.
Credibility concerns the trustworthiness of its contents for the intended purpose.
A genuine institutional policy may be authentic but outdated. An official report may accurately describe the organization's stated priorities while selectively presenting its achievements.
Researchers should examine authorship, publication context, version history, dates, provenance, and the circumstances of production.
Where possible, original institutional sources should be preferred over unattributed copies.
However, authenticity and credibility must be evaluated separately. Official origin does not establish that every statement is empirically accurate.
What Does Representativeness Mean in Documentary Research?
Representativeness concerns whether the available documents adequately reflect the documentary population or phenomenon the researcher intends to investigate.
For example, publicly available AI policies may disproportionately represent institutions with developed digital governance systems.
Universities without publicly accessible policies may be absent from the corpus.
That absence does not necessarily mean those institutions have no internal guidance.
Researchers should distinguish the population of relevant documents from the documents they can access.
When defining a document corpus, inclusion and exclusion criteria should make this distinction visible.
Why the Original Purpose of a Document Matters
Documents are rarely produced solely to answer a researcher's questions.
A policy may establish obligations, a press release may communicate institutional achievements, and meeting minutes may summarize decisions without reproducing the complete discussion.
These purposes influence what information appears and what is omitted.
For example, a meeting record documenting approval of an AI policy may not explain the disagreements preceding that decision.
Researchers should consider how to analyze documents produced for different purposes rather than treating every text as equivalent evidence.
How Should Documents Be Selected?
Document selection should follow the research question and a transparent set of eligibility criteria.
Relevant criteria may include institutional origin, document type, publication period, jurisdiction, intended audience, and relationship to the phenomenon.
For example, a study of current university AI policies might include official policies in force during a defined period and exclude informal commentary or superseded drafts unless historical change is part of the research question.
Researchers should determine whether to analyze all eligible documents or select a sample.
The decision about sampling documents versus analyzing all available materials should consider the corpus, analytical purpose, and feasibility.
How Are Documents Analyzed?
Document analysis is not a single analytical technique.
Researchers may use qualitative content analysis, thematic analysis, discourse analysis, historical analysis, or quantitative coding, depending on the question and methodological orientation.
For example, a quantitative analysis might record how many institutional policies contain explicit AI disclosure provisions.
A qualitative analysis might examine how those provisions construct responsibility and academic integrity.
The same corpus can support different analyses, but the procedures and conclusions should be specified accordingly.
The choice between quantitative, qualitative, and mixed-methods content analysis should follow the intended interpretation.
Can Documentary Analysis Support Comparisons Across Institutions?
Yes, provided the comparison uses defensible criteria and accounts for differences in document availability and function.
For example, a researcher may compare whether university policies address disclosure, assessment, privacy, and institutional support.
However, one university may consolidate its requirements into a single policy, while another distributes them across several documents.
Comparing one document from each institution may therefore create misleading differences.
The researcher should determine whether the unit of analysis is the document, institution, policy provision, or another entity.
Comparative conclusions should reflect the actual documentary coverage.
What If Important Documents Are Missing?
Missing documents may reflect restricted access, incomplete archives, institutional nonpublication, document loss, or the absence of formal documentation.
These explanations have different implications.
For example, failure to locate a public AI policy does not prove that a university lacks internal AI guidance.
Researchers should document search procedures, inaccessible materials, and uncertainty about the documentary population.
The methodological treatment of missing or unavailable documents should follow the claims the study intends to make.
Missingness should not silently be converted into evidence of institutional absence.
When Is Triangulation Necessary?
Triangulation may be useful when documentary evidence provides only one perspective on a phenomenon.
For example, institutional policies may describe formal AI governance, while interviews reveal how teachers interpret the requirements.
Observation may provide evidence about actual classroom practices.
However, triangulation is not automatically required for every document-based study.
If the research question concerns the language of official policies, the documents themselves may provide adequate evidence.
The decision about triangulating document analysis with other data sources should be based on evidentiary needs rather than a universal expectation that more methods are always better.
Can Publicly Available Documents Raise Ethical Concerns?
Yes. Public availability does not eliminate every ethical consideration.
Documents may contain personal information, sensitive institutional details, or statements that could identify individuals when combined with contextual information.
Researchers should consider privacy expectations, applicable data-protection requirements, copyright, and potential harms arising from publication.
For example, an accessible disciplinary report may contain identifiable information that is unnecessary for answering the research question.
Researchers should minimize unnecessary disclosure and verify applicable institutional ethics requirements.
Access to a document does not automatically authorize every possible research use or republication.
How Do You Establish Originality in Document-Based Research?
Research originality does not require producing every item of evidence from scratch.
Originality may arise from the research question, analytical framework, comparison, interpretation, or identification of previously unexamined relationships.
For example, existing university AI policies may have been published individually, but a systematic comparative analysis of how they distribute accountability may provide a new scholarly contribution.
However, the researcher should establish the contribution through an appropriate literature review rather than assuming that a new comparison is automatically novel.
The originality lies in the defensible knowledge generated from the documents, not merely in their collection.
How Should a Document-Based Study Be Reported?
The methodology should explain the research design, documentary sources, eligibility criteria, retrieval procedures, corpus boundaries, and analytical approach.
Researchers should describe how document authenticity, credibility, and contextual relevance were evaluated.
Where versions changed over time, the relevant dates and versions should be recorded.
The report should also identify inaccessible documents and limitations affecting interpretation.
Readers should be able to understand how the corpus was assembled and how the documentary evidence supports the findings.
