How Do You Balance Openness and Reproducibility Against Privacy and Confidentiality?

Can Research Be Reproducible Without Making Every Dataset Public?

A researcher completes a study examining how university students use an online learning platform. The analysis draws on learning management system logs, assessment records, discussion activity, and demographic information.

A journal encourages the researcher to deposit the underlying data in an accessible repository so that other scholars can reproduce the findings. Yet even after student names and identification numbers are removed, combinations of course enrollment, timestamps, grades, and activity patterns may reveal individual identities.

Publishing the complete dataset could improve opportunities for independent verification. It could also expose students to risks they did not anticipate when agreeing to participate.

Should the researcher prioritize transparency, restrict access to the data, or withhold the dataset entirely? More fundamentally, does reproducible research necessarily require unrestricted access to identifiable or potentially identifiable information?

Research Can Be Transparent Without Making Sensitive Information Public

Understanding the Ethical Limits of Open and Reproducible Research

Why Are Openness and Reproducibility Important?

Research transparency allows other scholars to examine how findings were produced. Reproducibility, although defined differently across disciplines, commonly concerns the ability to obtain consistent computational results using the same data, analytical procedures, and computational conditions.

The National Academies of Sciences, Engineering, and Medicine distinguishes computational reproducibility from replicability, which concerns obtaining consistent results across studies that collect their own data to address a scientific question.

Making relevant materials accessible can support error detection, methodological scrutiny, and cumulative knowledge development. Researchers may be able to examine analytical code, verify variable construction, assess exclusions, or identify discrepancies between reported methods and actual procedures.

However, transparency is not identical to unrestricted publication. The ethical legitimacy of sharing depends partly on the nature of the information and the consequences of making it accessible.

Why Can Open Data Conflict With Participant Privacy?

Research datasets may contain information that identifies participants directly or permits identification through combinations of variables.

Direct identifiers include names, personal identification numbers, and contact details. Indirect identifiers may include age, occupation, location, institutional affiliation, rare characteristics, or distinctive sequences of events.

Removing direct identifiers does not necessarily make a dataset anonymous. A person may still be identifiable when several apparently ordinary attributes are combined with publicly available information.

For example, a dataset describing a small university program may contain no student names but include course enrollment, assessment scores, exact submission times, and demographic characteristics. Someone with access to institutional information might be able to infer which record belongs to a particular student.

Researchers should therefore evaluate identifiability in context, including the possibility of linkage with other datasets and the capabilities of foreseeable data recipients.

What Is the Difference Between Privacy, Confidentiality, and Data Security?

These concepts overlap but address different questions. Encryption can help secure confidential information, yet encryption alone does not establish that collecting or sharing the information is ethically appropriate.

Similarly, participants may consent to one form of data use without authorizing every future disclosure.

Does FAIR Mean Research Data Must Be Open to Everyone?

No. The FAIR principles concern making data Findable, Accessible, Interoperable, and Reusable. They do not require every dataset to be downloadable by anyone without conditions.

The original FAIR principles recognize that accessibility may involve authentication and authorization procedures. Metadata can remain discoverable even when access to the underlying data is restricted.

This distinction is particularly useful for sensitive research. A repository record may describe the dataset, its variables, provenance, access conditions, and permitted uses without exposing individual-level information.

FAIRness and openness are related but not synonymous. A well-documented controlled-access dataset may follow important FAIR practices, whereas an openly downloadable dataset with inadequate metadata may be difficult to interpret or reuse.

Researchers should also distinguish technical accessibility from legal permission to reuse data. A file being downloadable does not establish that every intended use is authorized.

What Levels of Data Access Are Available?

Research data sharing is not necessarily a binary decision between publishing everything and sharing nothing.

The appropriate arrangement depends on the research question, participant agreements, sensitivity of the data, legal requirements, and available safeguards.

Researchers should explain the verification opportunities and limitations associated with their chosen arrangement rather than describe every restricted-access approach as fully reproducible.

When Is De-identification Sufficient?

De-identification involves removing or transforming information that could identify individuals. Depending on the context, techniques may include suppressing direct identifiers, generalizing dates or locations, aggregating observations, and reducing the precision of sensitive variables.

However, de-identification is not a universal guarantee of anonymity.

The United States National Institute of Standards and Technology discusses de-identification in terms of disclosure risk and the limitations of particular techniques. The effectiveness of a method depends on the dataset, available external information, and intended release conditions.

Researchers should also recognize that data protection laws may distinguish anonymous information from pseudonymized personal data. Under the European Union General Data Protection Regulation, pseudonymized information may remain personal data when attribution to an individual is possible using additional information.

In the Philippines, researchers handling personal information should examine the Data Privacy Act of 2012 and applicable National Privacy Commission guidance. Removing names does not automatically settle whether information remains personal or sensitive personal information under the relevant legal framework.

The appropriate assessment should be documented and, when necessary, reviewed by qualified privacy or information-security personnel.

Can Researchers Share Analytical Code Without Sharing Raw Data?

Often, yes. Analytical scripts, software versions, package dependencies, variable definitions, and processing documentation can be shared separately from sensitive participant records.

For computational research, these materials can substantially improve transparency. They may allow others to inspect data transformations, statistical models, exclusion rules, and visualization procedures.

However, researchers should review code and associated files for embedded credentials, file paths containing personal information, small-cell outputs, or other sensitive details.

Code sharing also has limits. Without the original data, independent researchers generally cannot confirm that the reported results are exactly reproducible, although they may be able to inspect the procedures and test them using appropriately constructed example data.

A transparent data availability statement should explain this limitation rather than imply that public code alone provides complete computational reproducibility.

How Can Controlled Access Support Independent Verification?

Controlled-access arrangements may permit qualified researchers to inspect sensitive data under agreements specifying permitted uses, security requirements, confidentiality obligations, and restrictions on further disclosure.

Some repositories or secure environments allow approved users to conduct analyses without obtaining unrestricted copies of the underlying records.

These arrangements can support independent verification while reducing certain disclosure risks. Their effectiveness depends on the access criteria, independence of reviewers, technical safeguards, and whether meaningful analytical scrutiny is permitted.

Controlled access should not be treated as an excuse to prevent legitimate criticism. Arbitrary restrictions, excessive fees, or discretionary approval processes can weaken the transparency that the arrangement is intended to support.

Researchers should select arrangements that are proportionate to the sensitivity of the information and as accessible as responsibly possible.

Does Participant Consent Authorize Public Data Sharing?

Consent may authorize specified forms of data sharing, but the scope of that authorization matters.

Participants who consented to a confidential interview have not necessarily agreed that their transcript will be deposited in a public repository. Similarly, consent to share de-identified information does not eliminate the need to assess whether the data can actually be released with acceptable disclosure risk.

Consent is also not the only possible lawful basis for processing research data under every data protection framework. Researchers must identify the applicable legal basis and satisfy relevant safeguards.

Where future sharing is anticipated, the consent process should explain the intended recipients, access arrangements, potential uses, and relevant limitations as accurately as possible.

When the proposed sharing differs materially from the original arrangements, researchers may need additional consent, ethical review, or another lawful authorization.

In some circumstances, even consented data sharing may create risks that make public release inappropriate.

What Should a Reproducibility Package Contain When Data Cannot Be Public?

A reproducibility package should provide as much information as can responsibly be disclosed about how the study was conducted and analyzed.

Depending on the research design, useful materials may include a protocol, data dictionary, instrument documentation, variable construction rules, analytical scripts, software environment specifications, exclusion criteria, and a description of missing-data handling.

Researchers may also provide aggregate outputs or synthetic examples that demonstrate the workflow without exposing participant records.

The package should clearly identify which verification steps are possible with public materials and which require authorized access to restricted data.

Where methodological details themselves create substantial risks, researchers should consider whether limited disclosure of sensitive procedures is justified.

Transparency about restrictions is preferable to implying that unavailable materials can be independently examined.

When Should Researchers Refuse Public Data Release?

Public release may be inappropriate when credible re-identification risks remain, consent or legal permissions do not authorize the disclosure, contractual obligations prohibit it, or publication could expose individuals or communities to serious harm.

Researchers should also consider whether a dataset contains information about third parties who never consented to participate.

The decision should not depend solely on whether a journal requests open data. Publishers commonly provide mechanisms for explaining justified restrictions, although the precise requirements vary and must be checked.

If a publisher's requirements cannot be reconciled with binding confidentiality obligations, researchers should seek clarification, consider an approved alternative access arrangement, or select an appropriate publication venue.

Responsible openness requires recognizing circumstances where scientific openness conflicts with other research responsibilities.

Making a Learning Analytics Study Verifiable Without Exposing Students

This arrangement does not provide the same openness as unrestricted data release. It nevertheless offers meaningful methodological transparency while avoiding an unnecessary public disclosure of potentially identifiable student information.

The researchers should periodically reassess the access arrangement when relevant risks, permissions, or technical capabilities change.

Common Misconceptions About Open Data, Reproducibility, and Privacy

How to Choose an Appropriate Research Data-Sharing Arrangement

Begin by identifying which research materials are necessary for others to understand, evaluate, or reproduce your findings.

Separate those materials according to their sensitivity and the permissions governing their use. Analytical code and documentation may be suitable for public release even when participant-level records are not.

Then assess whether de-identification, controlled access, or a secure analysis environment can support legitimate verification without creating disproportionate privacy risks.

Where transparency requirements conflict with privacy obligations, a reasoned ethical decision should examine the competing interests rather than automatically prioritize either complete openness or complete secrecy.

Document the data-sharing decision, including the applicable permissions, disclosure-risk assessment, access controls, and limitations on independent verification.

Researchers should plan these arrangements before data collection whenever possible. Retrospective attempts to create a public dataset from information collected under restrictive confidentiality assurances may be difficult or impossible to justify.

Before Sharing Data or Claiming Research Reproducibility

Frequently Asked Questions About Reproducibility and Research Privacy

Transparency Should Be Maximized Within Defensible Privacy Boundaries

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.