Should Easily Available Predictors Be Preferred Over Expensive Ones?

Is a More Accurate Prediction Model Worth Using if Its Predictors Are Difficult or Expensive to Obtain?

You are developing a prediction model and have identified several characteristics that may improve its performance. Some are readily available from routine records, while others require specialized laboratory tests, lengthy assessments, additional equipment, or substantial staff time.

Including the expensive predictors may improve discrimination or reduce prediction error. But will the improvement be meaningful enough to justify the additional cost and burden?

Alternatively, a model using only routinely available information may be easier to implement, although it could sacrifice some predictive accuracy.

The decision is not simply statistical. Predictor selection determines who can use the model, when predictions can be generated, how much implementation costs, and whether the resulting information improves decisions.

Prefer Accessible Predictors When They Provide Sufficient Information for the Intended Decision

How Predictor Availability and Measurement Cost Affect Prediction Model Usefulness

What Does Predictor Availability Mean?

Predictor availability refers to whether a characteristic can be obtained when and where the model is intended to generate predictions.

For example, age and previous diagnoses may be available in routine records. A specialized biomarker may require a laboratory test that is not routinely performed.

Availability is not simply whether a variable exists somewhere in the development dataset.

A predictor may be available in a research hospital but unavailable in a rural clinic. It may be routinely recorded in one university but absent from another institution's information system.

Researchers should therefore evaluate availability in the intended application setting rather than only in the development environment.

What Counts as Predictor Cost?

Predictor cost includes more than the monetary price of obtaining a measurement.

It may involve staff time, equipment, laboratory processing, licensing fees, participant burden, delays, maintenance, and the infrastructure needed to integrate measurements into a decision process.

For example, a questionnaire may have little direct financial cost but require considerable administration time.

A laboratory test may be inexpensive per sample but unavailable outside specialist centers.

An automatically extracted electronic record variable may appear costless while requiring substantial data integration and quality assurance.

Researchers should consider the full operational burden rather than only the purchase price of a test.

Why Does the Intended Decision Matter?

The value of a predictor depends partly on what the prediction will be used to decide.

For example, a small improvement in accuracy may be worthwhile when the model guides a high-stakes intervention with substantial consequences.

The same improvement may be difficult to justify when predictions inform a low-risk decision that can already be made adequately using routine information.

Researchers should define the target population, decision point, available actions, and consequences of incorrect predictions.

Without a clear decision context, it is difficult to determine whether additional measurement costs are justified.

How Do Routine and Expensive Predictors Compare?

These differences are contextual rather than universal. A specialized measurement may become routine over time, while a supposedly inexpensive predictor may be difficult to obtain reliably in some settings.

What Is Incremental Predictive Value?

Incremental predictive value concerns the additional predictive information contributed by a new predictor beyond information already included in a model.

For example, researchers may compare a model using routine clinical characteristics with a model that adds an expensive biomarker.

The question is whether the biomarker improves predictions sufficiently to justify its measurement.

Incremental value should be assessed using performance measures appropriate to the intended outcome and decision.

A predictor may have a strong individual association with the outcome while adding little information to an existing model.

Conversely, a predictor with a modest association may meaningfully improve predictions in a particular subgroup or risk range.

Why Is Statistical Significance Not Enough?

A predictor can have a statistically significant coefficient without producing a meaningful improvement in predictive performance.

For example, a biomarker may be associated with hospitalization after adjustment for routine characteristics, yet adding it may change predicted probabilities only slightly.

If those changes do not alter decisions or improve their consequences, the practical value of the additional measurement may be limited.

Researchers should therefore distinguish association significance from incremental predictive usefulness.

Costly predictors should not be selected solely because their regression coefficients have small p-values.

Should Researchers Compare Models With and Without the Expensive Predictor?

Yes. A useful approach is to establish a reference model containing routinely available predictors and compare it with an extended model that includes the additional measurement.

Both models should address the same prediction task and be evaluated using appropriate, comparable procedures.

Researchers should assess whether the extended model improves calibration, discrimination, prediction error, or decision-analytic usefulness.

Comparisons should account for model-development optimism and should ideally be evaluated in independent data.

The additional predictor's contribution should be assessed relative to the existing model rather than only through its standalone association with the outcome.

Why Can AUC Changes Be Misleading?

The area under the receiver operating characteristic curve measures discrimination, or the ability to rank individuals according to outcome risk.

Adding a predictor may produce only a small change in AUC even when predicted probabilities change meaningfully for individuals near an important decision threshold.

Conversely, a measurable increase in AUC may have little practical consequence if decisions remain unchanged.

Researchers should therefore avoid using AUC improvement as the sole criterion for deciding whether a costly predictor is worthwhile.

Calibration, decision thresholds, and the consequences of changed predictions also matter.

What Is Decision-Curve Analysis?

Decision-curve analysis evaluates the net benefit of using a prediction model across specified risk thresholds.

It incorporates the relative consequences of false-positive and false-negative classifications through the threshold probability.

For binary outcomes, a common net-benefit expression is:

Decision-curve analysis may help determine whether adding a predictor changes decisions beneficially, but separate economic evaluation may be required to assess whether the additional measurement is worth its cost.

Does Better Discrimination Justify a More Expensive Model?

Not automatically. Discrimination is only one aspect of model performance.

A model may achieve a higher AUC while remaining poorly calibrated or providing little additional value at relevant decision thresholds.

Researchers should consider whether improved predictions change actions in ways that produce meaningful benefits.

For example, a costly predictor may be justified if it identifies individuals who would otherwise miss an effective intervention.

However, if the model's predictions rarely alter decisions, the additional cost may be difficult to defend.

The balance depends on the intervention, outcome severity, resources, and consequences of errors.

What Is Cost-Effectiveness in Prediction Model Use?

Cost-effectiveness concerns the relationship between additional resources used and the benefits produced by an intervention or decision strategy.

For prediction models, relevant costs may include measurement, software, implementation, follow-up testing, and downstream actions triggered by predictions.

Benefits may include improved health outcomes, avoided complications, reduced unnecessary interventions, or other contextually appropriate consequences.

A model with better predictive accuracy is not automatically cost-effective.

Economic evaluation may require information about how predictions change decisions, the effectiveness of resulting actions, and the resources consumed.

Researchers should distinguish predictive performance from the economic value of implementing the model.

Can a Cheap Predictor Be More Valuable Than an Accurate Expensive Predictor?

Potentially. A routinely available predictor may provide sufficient information for a decision at minimal additional burden.

For example, a model using existing records may allow risk assessment immediately, while a specialized test may delay decisions.

If the additional test changes few decisions or provides little improvement in outcomes, its practical value may be limited.

However, a costly predictor may be worthwhile when it substantially improves risk estimation in a high-stakes setting.

The appropriate comparison concerns total decision value, not the price or predictive association of a single variable in isolation.

What Is a Two-Stage Prediction Strategy?

A two-stage strategy uses readily available predictors for an initial assessment and reserves more expensive measurements for selected individuals.

For example, a routine-data model may identify patients whose estimated risk lies near a decision threshold. A specialized test may then be used to refine predictions for those patients.

This approach can potentially reduce unnecessary testing while preserving additional information where it matters most.

However, the complete strategy requires validation.

Researchers should evaluate how individuals are selected for additional testing, whether selection creates bias, and whether the resulting decisions improve outcomes.

The two-stage approach should not be assumed cost-effective without appropriate evidence.

How Does Predictor Availability Affect Missing Data?

Predictors that are not routinely measured may be missing for substantial portions of the target population.

For example, a specialized laboratory test may be ordered primarily for patients with severe symptoms.

Its availability may therefore depend on clinical decisions related to outcome risk.

Restricting model development to individuals with available measurements can change the study population and introduce selection problems.

Imputation may be considered under appropriate assumptions, but it does not automatically solve systematic measurement or availability limitations.

Researchers should assess whether the predictor can realistically be obtained for individuals when the model is applied.

Why Does Measurement Reliability Matter Alongside Cost?

A predictor's practical value depends on whether it can be measured consistently.

An inexpensive variable recorded inconsistently across institutions may contribute unreliable information.

Conversely, an expensive measurement may be highly standardized and reproducible.

Researchers should examine measurement error, interobserver variability, instrument differences, and changes in data collection procedures.

Predictive performance estimated using highly controlled research measurements may not be reproduced when the same predictor is collected routinely.

Measurement feasibility should therefore include reliability, not merely availability.

How Can Expensive Predictors Affect Equity?

Models requiring specialized measurements may be difficult to implement in resource-limited settings.

This can restrict access to model-informed decisions or create differences in prediction availability across population groups.

For example, a model requiring advanced imaging may be impractical in facilities without the necessary equipment.

However, using only inexpensive predictors does not automatically guarantee fairness. Routine data may also reflect historical inequalities or uneven measurement practices.

Researchers should evaluate access, subgroup performance, and the consequences of decisions informed by the model.

Equity considerations should be integrated into predictor selection rather than addressed only after implementation.

When Should Expensive Predictors Be Collected During Research?

If an expensive predictor is scientifically promising but its incremental value remains uncertain, a development or validation study may justify collecting it to evaluate that value.

However, the study should specify the comparison between a routine-predictor model and an extended model.

Researchers should plan sample size, measurement procedures, and validation to support a credible comparison.

Collecting expensive information does not mean it must be retained in the final operational model.

The purpose may be to determine whether the additional measurement is worth using.

The principles of planning candidate predictors before data collection are particularly relevant when measurements are costly or burdensome.

How Should Predictor Cost Be Reported?

Researchers should describe the resources required to obtain predictors and identify whether they are routinely available in the intended setting.

Where relevant, they should report monetary costs, measurement time, equipment requirements, and missingness associated with availability.

Comparisons between models should report predictive performance and any assessed decision consequences.

When economic conclusions are made, the perspective, included costs, assumptions, and outcome measures should be specified.

Researchers should avoid labeling a model cost-effective merely because it uses fewer predictors or produces a higher AUC.

When a Costly Biomarker Adds Little Practical Value to a Prediction Model

The example does not establish that an AUC increase of 0.02 is either trivial or worthwhile in every context.

Its value depends on the outcome, decision thresholds, effectiveness of available interventions, and consequences of prediction errors.

Common Misconceptions About Predictor Availability and Cost

How to Balance Predictive Performance With Measurement Feasibility

Begin by developing or identifying a credible reference model using information routinely available at the intended prediction time.

Then evaluate whether additional measurements improve performance and decisions sufficiently to justify their cost and burden.

Report the incremental performance of expensive predictors relative to an appropriate reference model, not merely their individual statistical associations.

Where economic claims are important, evaluate costs and consequences using an appropriate decision or economic framework.

A practically valuable prediction model balances predictive information with the resources and conditions required to obtain and use that information.

Before Including Expensive Predictors in a Prediction Model

Frequently Asked Questions About Predictor Cost and Availability

Choose Predictors for Their Decision Value, Not Their Price or Statistical Significance Alone

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.