03 · What You Need to Know
Why Research Designs Must Be Evaluated Within Their Disciplinary and Institutional Contexts
What Are Field-Specific Research Constraints?
Field-specific constraints are requirements or conditions arising from the discipline, research setting, participant population, technology, or professional practices involved in a study.
They may affect participant recruitment, intervention assignment, measurement, data access, research ethics, and the interpretation of findings.
For example, an intervention that can be randomly assigned in a laboratory may be difficult to assign within an established classroom schedule. A data collection procedure acceptable for anonymous adult surveys may require substantially different safeguards when involving children or identifiable educational records.
AI may recognize broad methodological principles without knowing which local requirements apply.
Researchers should therefore distinguish general design recommendations from procedures that have been verified as appropriate for their specific context.
What Types of Constraints Might AI Overlook?
| Constraint |
Illustrative issue |
Possible methodological consequence |
| Disciplinary conventions |
A qualitative tradition requires methodological commitments not addressed by a generic interview design. |
The proposed approach may lack coherence with the intended form of inquiry. |
| Institutional policies |
Existing classes cannot be reorganized for research purposes. |
Individual or classroom randomization may be infeasible. |
| Participant protections |
Research involves minors or students in dependent relationships with instructors. |
Additional consent, recruitment, and safeguarding procedures may be necessary. |
| Data governance |
Learning analytics contain identifiable student information. |
Data access, processing, storage, or transfer may require authorization and safeguards. |
| Technical infrastructure |
Students have unequal access to compatible devices or reliable internet. |
Intervention exposure and measurement may vary systematically. |
| Professional obligations |
Withholding an established educational service may disadvantage participants. |
The proposed control condition may be ethically inappropriate. |
| Research calendar |
Assessment schedules are fixed by institutional requirements. |
Measurement timing may not match the proposed design. |
These constraints can interact. For example, a platform may be technically available but unsuitable for collecting particular student information under institutional data protection rules.
Why Might a Methodologically Appropriate Experiment Be Inappropriate in a School?
Randomized experiments can provide strong evidence about intervention effects under appropriate conditions. However, their implementation may conflict with existing instructional arrangements or ethical obligations.
Suppose AI recommends randomly assigning individual students within a classroom to receive different instructional approaches.
The teacher may be unable to deliver both approaches independently, and students may share intervention materials. Such conditions create implementation and contamination concerns.
Alternatively, the institution may prohibit changes to required instruction during the semester.
These problems do not imply that educational experiments are inherently inappropriate. They indicate that the assignment unit, intervention, comparator, and institutional conditions must be examined before selecting the design.
Shadish, Cook, and Campbell (2002) provide a methodological foundation for understanding experimental and quasi-experimental designs under real-world constraints.
Can AI Overlook Ethical Restrictions on Comparison Groups?
Yes. AI may suggest a no-treatment control group without establishing whether withholding the intervention is acceptable.
For example, withholding supplementary AI tutoring may be permissible in some circumstances, while denying students an established educational accommodation or required instructional service may be inappropriate.
The ethical distinction depends on the intervention, available alternatives, participant needs, and institutional requirements.
Researchers should not assume that a control group must receive no instruction or support.
An active comparator or usual-practice condition may be more appropriate, but it must still address the research question.
The selection of comparison and control groups should therefore consider ethical obligations alongside causal identification.
How Can Student-Instructor Relationships Affect Research Design?
Educational researchers may recruit participants whom they also teach, supervise, assess, or otherwise influence.
These relationships can create concerns about voluntary participation, perceived pressure, confidentiality, and the separation of research participation from academic evaluation.
For example, students may believe that refusing to participate could affect their grades even when the researcher has no intention of penalizing them.
AI may recommend direct classroom recruitment without recognizing the power relationship involved.
Possible safeguards may include independent recruitment, separating consent information from academic assessment, and ensuring that participation decisions do not affect educational opportunities.
The appropriate arrangements depend on institutional ethics requirements and the actual relationship between researchers and participants.
Can AI Overlook Data Privacy and Governance Requirements?
Yes. Educational technology research may involve student records, learning management system logs, writing submissions, assessment results, or interactions with external AI services.
These data may contain identifiable or sensitive information requiring particular safeguards.
In the Philippines, the Data Privacy Act of 2012, Republic Act No. 10173, establishes a legal framework for personal data protection. Researchers must also consider applicable implementing rules, institutional policies, and the specific legal basis and safeguards for processing personal information.
For example, uploading identifiable student writing or institutional records to an external AI service may create data governance concerns that are not resolved simply by obtaining general research consent.
AI can help identify questions for a data management review, but it cannot independently determine whether a particular transfer or processing arrangement complies with all applicable requirements.
Why Do Commercial AI Platforms Create Additional Research Constraints?
Studies involving commercial generative AI systems may depend on platforms whose features, models, interfaces, pricing, or access conditions change over time.
These changes can affect intervention consistency and reproducibility.
For example, students assigned to an AI-supported learning condition may encounter different system behavior if the platform changes during data collection.
Researchers may need to document the model or service used, relevant settings, dates of access, intervention prompts, and changes that could affect exposure.
However, the appropriate documentation depends on the platform and research objectives. Not every system permits researchers to fix a model version or control its updates.
AI should not assume that commercial platforms provide stable, unrestricted, or institutionally approved research environments.
Can Unequal Technology Access Affect the Study Design?
Yes. Participants may differ in internet connectivity, device availability, digital literacy, or access to paid AI features.
These differences may affect recruitment, intervention adherence, exposure measurement, and outcomes.
For example, a study requiring continuous access to an AI tutoring platform may systematically disadvantage students with unstable connectivity.
Providing standardized devices or access arrangements may reduce some differences, but these safeguards require resources and may change the context to which the findings apply.
Researchers should examine whether technological access is part of the phenomenon being studied or a condition that must be standardized for the intended comparison.
Do Field-Specific Constraints Also Matter in Qualitative Research?
Yes. Qualitative research requires attention to methodological traditions, participant relationships, context, researcher positionality, and the conditions under which evidence is generated.
AI may recommend phenomenology, grounded theory, ethnography, or case study based on familiar keywords without examining the researcher's intended form of inquiry.
For example, studying teachers' experiences does not automatically justify phenomenology. The research question may instead concern institutional practices within a bounded case or the development of professional decision-making processes.
The researcher must evaluate whether the proposed design is consistent with its methodological assumptions and the actual research setting.
Field-specific appropriateness therefore extends beyond experimental feasibility.
Can Institutional Constraints Change the Research Question?
Sometimes. If a proposed design cannot be implemented, the researcher may need to reconsider the question or the type of inference being sought.
For example, an institution may prohibit random assignment of students to instructional conditions.
The researcher might investigate whether a credible nonrandomized comparison is possible. Alternatively, the study could focus on implementation experiences or associations rather than a causal intervention effect.
However, these alternatives are not scientifically equivalent. Changing the design may change what the study can establish.
Researchers should therefore revisit the alignment between the research question and design whenever contextual constraints require substantial methodological changes.
Can AI Verify That a Study Complies With Institutional Requirements?
Not independently. AI may help interpret supplied policies or identify relevant compliance questions, but institutional requirements must be checked against authoritative and current sources.
A policy may differ across universities, departments, participant populations, or types of data.
Researchers should consult the relevant ethics committee, data protection office, institutional research office, or other responsible authority when required.
AI-generated statements about institutional permission should not be treated as actual authorization.
Watch Out
Do not assume that a design is permissible simply because AI recommends it or because similar studies have been published elsewhere. Institutional permissions, participant protections, data governance requirements, and professional obligations must be verified in the actual research setting.
How Can You Conduct a Field-Specific Design Audit?
1. Identify the research context. Specify the discipline, institution, participants, technology, and relevant professional practices.
2. Describe the proposed procedures. Include recruitment, assignment, intervention delivery, measurement, data handling, and analysis.
3. Identify contextual requirements. Examine institutional policies, participant protections, technical conditions, and disciplinary conventions.
4. Verify authoritative requirements. Consult relevant institutional offices, legal guidance, methodological sources, and responsible personnel.
5. Evaluate methodological consequences. Determine how constraints affect comparison conditions, measurement, validity, and intended inference.
6. Consider defensible alternatives. Compare design modifications without assuming that they preserve the original study's conclusions.
7. Document the rationale. Explain how the final design accommodates the research setting while maintaining scientific integrity.
This audit should complement a broader feasibility assessment, particularly when the study involves educational interventions or external technologies.
