03 · What You Need to Know
How Prognostic Factor Research and Prediction Model Research Differ
What Is a Prognostic Factor Study?
A prognostic factor study investigates the relationship between one or more characteristics and a future outcome in a defined population.
The factor may be demographic, clinical, behavioral, biological, educational, or otherwise relevant to the phenomenon being studied.
For example, researchers may investigate whether baseline symptom severity is associated with recovery within six months, or whether prior academic performance is associated with subsequent program completion.
The principal interest is usually the direction, magnitude, precision, and interpretation of the association.
Researchers may estimate unadjusted associations and associations conditional on other prespecified variables.
However, adjustment does not automatically make an association causal. The selected variables and interpretation should reflect the scientific objective and plausible relationships among the factors.
What Is a Prediction Model Study?
A prediction model study develops, validates, or updates a mathematical model that combines information to estimate an outcome for individuals or groups.
For example, a model might estimate a patient's probability of hospitalization within one year or a student's probability of completing a program within the expected duration.
The principal concern is whether the predictions are sufficiently accurate, reliable, and useful for their intended purpose.
Prediction-model research therefore requires more than examining individual regression coefficients.
Researchers must evaluate performance, including calibration, discrimination where appropriate, and the consequences of using predictions in decisions.
The choice among model development, validation, and updating should be determined by whether a suitable model already exists and what evidence is needed.
How Do Their Primary Questions Differ?
| Feature |
Prognostic Factor Study |
Prediction Model Study |
| Primary question |
Is a characteristic associated with a future outcome? |
How accurately can future outcomes be predicted using available information? |
| Main focus |
Individual or conditional associations |
Combined predictive performance |
| Typical result |
Hazard ratio, risk ratio, odds ratio, or other association estimate |
Prediction equation, risk estimate, or prediction algorithm |
| Variable selection |
Guided by the factor of interest and scientifically relevant adjustment variables |
Guided by predictive information, availability, intended use, and overfitting control |
| Key evaluation |
Association magnitude, uncertainty, bias, and interpretation |
Calibration, discrimination, validation, and usefulness |
| Typical conclusion |
A factor is associated with subsequent outcomes |
A model predicts outcomes with specified performance in an evaluated population |
| Major limitation |
Association does not necessarily imply causation or predictive usefulness |
Predictive accuracy does not establish causal relationships |
The distinction concerns the objective rather than the statistical software or whether the analysis contains several variables.
Can Both Studies Use Multivariable Regression?
Yes. Logistic regression, Cox regression, and other multivariable methods can be used for both purposes.
In prognostic factor research, regression may estimate the association between a particular factor and the outcome after accounting for selected covariates.
In prediction-model research, regression coefficients may be combined to calculate predicted risks or other outcome estimates.
The same mathematical model can therefore support different analyses, but the scientific claims require different evidence.
A regression table containing several statistically significant coefficients does not automatically constitute a validated prediction model.
Likewise, a model with strong predictive performance does not establish that its predictors have independent causal effects.
Why Is Statistical Significance More Central to Some Prognostic Factor Studies?
Prognostic factor studies often examine whether the observed association is compatible with a specified null hypothesis.
However, a scientifically useful analysis should not rely solely on whether the p-value falls below 0.05.
The magnitude, direction, uncertainty, and potential bias of the association are often more informative than a binary significance label.
A small association may be statistically significant in a large sample but have limited practical importance.
Conversely, an imprecise estimate may be consistent with several substantively important effects despite failing to reach conventional statistical significance.
Researchers should therefore interpret confidence intervals and study limitations alongside the association estimate.
Why Is Statistical Significance Not a Reliable Criterion for Selecting Predictors?
Prediction models are evaluated according to their ability to generate accurate predictions, not simply whether each included variable has a statistically significant coefficient.
A predictor may improve predictive performance even when its individual coefficient is imprecisely estimated in a particular sample.
Conversely, a statistically significant predictor may add little information beyond variables already included in the model.
Selection based only on univariable p-values can exclude useful predictors and produce unstable models.
Data-driven variable selection can also introduce optimism, particularly when the number of candidate predictors is large relative to the available outcome information.
Researchers should consider substantive knowledge, predictor availability, sample size, and appropriate modeling strategies rather than treating p-value thresholds as universal selection rules.
What Does an Association Estimate Tell You?
A prognostic factor study may report a hazard ratio, odds ratio, risk ratio, mean difference, or another effect measure appropriate to the outcome.
For example, a hazard ratio greater than one may indicate a higher instantaneous event rate associated with a factor, under the specified model.
However, it does not directly provide an individual's absolute probability of experiencing the event within a particular period.
An odds ratio also should not automatically be interpreted as a risk ratio, particularly when the outcome is common.
The interpretation depends on the outcome, time horizon, adjustment variables, model assumptions, and target population.
Researchers should distinguish relative associations from absolute predictions.
What Does a Prediction Model Produce?
A prediction model typically produces an estimated probability, score, category, or time-dependent prediction for a specified outcome.
For example, a model may estimate that an individual has a 20% probability of experiencing an outcome within one year.
Such an estimate is meaningful only in relation to the outcome definition, prediction time, and population in which the model is intended to operate.
A prediction model may use regression, machine learning, or other statistical methods.
Methodological sophistication does not guarantee predictive validity. A complex algorithm can perform worse than a simpler model when data are limited or the model does not generalize.
Researchers should therefore evaluate the actual performance of the resulting predictions.
What Is Discrimination in Prediction Models?
Discrimination concerns whether a model assigns appropriately different predictions to individuals with different outcomes.
For binary outcomes, the area under the receiver operating characteristic curve is commonly used to summarize discrimination.
An AUC of 0.5 corresponds to chance-level ranking under the usual binary interpretation, while an AUC of 1 indicates perfect ranking in the evaluated data.
However, discrimination does not establish that predicted probabilities are numerically accurate.
A model can rank individuals reasonably well while systematically overestimating or underestimating their risks.
For time-to-event outcomes, discrimination measures must account appropriately for follow-up and censoring.
What Is Calibration, and Why Does It Matter?
Calibration concerns the agreement between predicted probabilities and observed outcome frequencies.
For example, among individuals assigned approximately 20% predicted risk, a well-calibrated model should produce an observed event frequency reasonably consistent with that prediction, subject to sampling uncertainty and the evaluation method.
A model that systematically predicts 40% risk when the observed risk is closer to 15% is poorly calibrated for that population and horizon.
Calibration is especially important when predictions inform decisions based on absolute risk thresholds.
Researchers should evaluate calibration using appropriate plots and quantitative measures rather than relying only on discrimination.
The relative importance of discrimination, calibration, and practical usefulness depends on the intended decision.
Why Does a Prediction Model Need Validation?
A model may perform well in the data used to develop it because it captures both genuine patterns and sample-specific variation.
This is known as overfitting when the model learns patterns that do not generalize adequately to new observations.
Internal validation, often using bootstrapping or cross-validation, helps assess optimism and model stability when performed correctly.
External validation evaluates the fixed model using data independent of development, ideally from a population relevant to intended use.
Validation should assess calibration, discrimination, and other relevant performance measures.
A model should not be described as validated merely because its regression coefficients are statistically significant or its apparent development AUC is high.
Do Prognostic Factor Studies Also Need Replication?
Yes. Prognostic associations may vary across populations because of differences in case mix, measurement, treatment, and other contextual factors.
Independent replication and systematic synthesis can help determine whether a reported association is consistent and credible.
However, replication of an association is not the same as external validation of a prediction model.
A factor may repeatedly show an association with an outcome while contributing little to individualized risk prediction.
Conversely, a prediction model may perform adequately using several modest predictors whose combined information is useful.
Researchers should match the evaluation strategy to the claim they intend to make.
How Do Adjustment Variables Differ Between the Two Approaches?
In prognostic factor research, adjustment variables are selected according to the scientific question and the interpretation of the association.
Researchers may want to establish whether a factor provides prognostic information beyond established characteristics. If a causal interpretation is intended, confounding and the underlying causal structure require particular attention.
In prediction modeling, variables are selected according to their potential contribution to prediction, practical availability, and suitability at the intended prediction time.
A variable can be useful for prediction without being a confounder or a cause of the outcome.
Similarly, a variable important for estimating a particular association may contribute little to overall prediction performance.
These differences explain why adjustment sets should not be copied mechanically between association and prediction analyses.
Can a Prediction Model Include Noncausal Variables?
Yes. Predictive usefulness does not require a predictor to cause the outcome.
For example, a laboratory measurement may summarize underlying disease severity without being a direct cause of subsequent deterioration.
It may nevertheless provide useful prognostic information.
However, predictors should be available at the intended prediction time, measured reliably, and sufficiently stable across the settings where the model will be used.
Variables that encode future information or reflect decisions made after the prediction point can create leakage.
Predictors that are highly dependent on local practices may also reduce transportability.
Why Does the Intended Decision Matter?
Researchers should ask what the results are expected to support.
If the aim is to understand whether a particular characteristic is associated with later outcomes, prognostic factor research may be sufficient.
If the aim is to identify individuals who should receive additional monitoring or support, a prediction model may be more relevant.
However, even a well-performing prediction model does not establish that acting on its predictions improves outcomes.
Implementation may require additional evidence concerning decision thresholds, resource implications, fairness, and the consequences of incorrect predictions.
Predictive accuracy and intervention effectiveness are distinct claims.
How Do Sample Size Requirements Differ?
Both approaches require adequate data, but the determinants of sample size differ.
A prognostic factor study may be planned around the precision of an association estimate, expected event frequency, and the number of parameters required for the intended analysis.
Prediction-model development must additionally consider overfitting, calibration, the number of candidate predictor parameters, and the anticipated predictive performance.
Simple rules based on a fixed number of events per variable are often insufficient for modern prediction-model planning.
Researchers should justify the number of candidate predictors and available outcome events using methods appropriate to the specific objective.
The relevant considerations are discussed further in how many candidate predictors the available data can support.
Can One Study Address Both Objectives?
Yes, provided the objectives and analyses are distinguished clearly.
Researchers might first investigate the association between a particular factor and an outcome, then develop a multivariable prediction model using suitable predictors.
However, the second analysis requires its own development strategy, validation, and performance assessment.
The same dataset does not automatically provide independent validation merely because different analyses are performed.
When both objectives are pursued, researchers should prespecify their roles and avoid presenting exploratory factor selection as confirmatory prediction-model validation.
Which Reporting Guidelines Apply?
The PROGRESS series provides a framework for distinguishing prognostic factor research from prognostic model research.
Observational prognostic factor studies may use STROBE for general reporting, supplemented by methodological guidance relevant to their specific analyses.
Prediction-model development, validation, and updating should follow applicable TRIPOD guidance, including the updated TRIPOD+AI reporting framework where relevant.
Reporting guidelines improve transparency but do not replace methodological assessment of bias, sample size, validation, or applicability.
