03 · What You Need to Know
How Firmware and Software Changes Affect Longitudinal Sensor Research
What Is the Difference Between Firmware and Software?
Firmware is software closely associated with controlling the operation of a device's hardware. In wearable and sensor systems, it may influence signal acquisition, sensor configuration, power management, and onboard processing.
Application software may manage device settings, synchronize information, display measurements, or calculate additional indicators.
Cloud-based software may also process uploaded sensor data and generate derived outcomes.
These components may be updated independently.
Consequently, a research variable may change even when the physical sensor and wearable device remain unchanged.
Distinguish Hardware Stability From Measurement Stability
Hardware stability
The physical sensing components and device configuration remain unchanged.
Processing stability
The algorithms and transformations applied to sensor signals remain sufficiently consistent.
Measurement comparability
Outputs collected under different conditions or versions support sufficiently equivalent interpretations for the intended analysis.
A device can remain physically identical while its processing software changes.
For example, the same accelerometer may produce different step counts after a revised algorithm changes how movement patterns are classified.
Researchers should therefore document both hardware and software characteristics when they influence the measurement.
Which Updates Can Affect Research Measurements?
| Update component |
Possible change |
Research consequence |
| Signal acquisition |
Modified sampling or sensor configuration |
Different temporal or signal characteristics |
| Filtering |
Revised noise or artifact removal |
Changes in retained signal information |
| Calibration |
Updated correction coefficients |
Shifted measurement values |
| Activity classification |
New step or activity detection rules |
Different derived activity estimates |
| Sleep estimation |
Revised sleep-wake or stage classification |
Changes in sleep summaries |
| Data aggregation |
Modified averaging or reporting intervals |
Reduced comparability of summaries |
| Synchronization |
Revised upload or timestamp handling |
Missing, duplicated, or shifted observations |
| Cloud processing |
Updated server-side algorithms |
Derived outputs change without a device update |
These are potential mechanisms rather than claims that every update produces such effects.
Researchers should determine which components are relevant to their particular outcome and device.
Why Can an Update Produce an Artificial Research Trend?
Longitudinal research often assumes that measurements collected at different times represent the same underlying quantity in a sufficiently consistent manner.
If a device changes its processing rules midway through the study, the resulting series may contain a measurement discontinuity.
For example, an activity monitor may begin recognizing movements that its previous algorithm did not classify as steps.
Recorded daily step counts could increase without an equivalent change in participants' walking behavior.
Researchers should investigate whether observed shifts coincide with relevant updates and whether alternative explanations remain plausible.
Temporal coincidence alone does not establish that the update caused the difference.
Can Firmware Updates Affect Raw Sensor Data?
Potentially. Firmware may control sampling frequency, sensor sensitivity, gain, filtering, and other aspects of signal acquisition.
Changes to these settings may alter the recorded raw or minimally processed signal.
However, some updates affect only derived indicators or interface functions without changing acquisition.
Researchers should distinguish updates to the measurement hardware interface from updates to downstream processing.
This distinction influences whether previously collected signals can be reprocessed into comparable outputs.
Can Cloud Algorithms Change Measurements Without Updating the Device?
Yes. Some wearable systems upload signals or intermediate measurements to external servers for processing.
A manufacturer may revise a cloud-based algorithm without changing the device firmware.
Consequently, derived values may change even when participants continue using the same device and application version.
In some systems, historical data may also be recalculated or displayed differently after processing changes.
Researchers should establish where critical variables are calculated and whether the processing version can be identified.
Device firmware alone may not provide a complete record of the measurement pipeline.
What Is Measurement Invariance Across Software Versions?
For longitudinal sensor research, the practical concern is whether measurements obtained under different versions retain sufficiently comparable meanings and measurement properties.
This may involve evaluating systematic differences, changes in error distributions, or altered relationships with an appropriate reference.
The term measurement invariance has a more specific statistical meaning in some fields, particularly latent-variable modeling.
Researchers should not assume that formal latent-variable invariance testing is necessary or suitable for every sensor output.
The appropriate equivalence assessment depends on the variable, device, and intended inference.
How Can Researchers Identify Relevant Updates?
Potential sources include manufacturer release notes, technical documentation, device logs, application version histories, and information supplied by technical support.
Researchers should record installation dates and versions where the system permits.
However, publicly available release notes may not disclose every change affecting measurement.
Some manufacturers describe updates broadly as performance improvements without identifying the algorithms modified.
When documentation is insufficient, empirical validation may be necessary.
Should Researchers Disable Automatic Updates?
Preventing unplanned updates may improve consistency when the device and software permit controlled version management.
However, disabling updates is not always possible or advisable.
Updates may address security vulnerabilities, safety issues, data loss, or serious measurement defects.
Researchers should evaluate the scientific need for version stability alongside device security, manufacturer requirements, and participant safety.
Where updates cannot be prevented, the protocol should specify how changes will be documented and assessed.
Version control should not come at the expense of essential safety or security protections.
Can Researchers Freeze a Validated Measurement Configuration?
Some research-grade systems allow investigators to maintain a specified hardware, firmware, and processing configuration during data collection.
This may reduce the risk of unplanned measurement changes.
However, configuration control should include more than the firmware version.
Relevant settings may include sampling frequency, sensor placement, calibration, algorithm version, application configuration, and data-processing parameters.
Researchers should document which components are fixed and which may change.
Any approved modifications should be recorded with their effective dates and implications for measurement.
How Can Researchers Test Whether an Update Changed Measurements?
A defensible assessment compares measurements obtained under the relevant versions using appropriate reference information or controlled conditions.
For example, a researcher may evaluate the same standardized activity protocol using devices configured with different versions.
Where possible, the comparison should account for participant characteristics, device placement, and environmental conditions.
Differences should be examined using appropriate agreement and error measures rather than correlation alone.
However, the ability to conduct direct comparisons depends on whether earlier versions remain available and whether the device supports controlled configuration.
What Is a Bridging Study?
A bridging study evaluates whether measurements obtained under different instruments, procedures, or software configurations can be interpreted comparably.
For sensor updates, this may involve collecting overlapping measurements under old and new versions or comparing both with a suitable reference.
For example, a research team may use two otherwise comparable devices running different firmware versions during standardized activities.
The analysis can examine whether differences are sufficiently small for the intended outcome or whether a validated conversion is possible.
Bridging evidence should cover relevant measurement ranges and operating conditions.
A single comparison at rest may be insufficient for a study involving vigorous activity.
How Should Agreement Across Versions Be Evaluated?
Agreement assessment should focus on whether differences between versions matter for the intended research use.
For continuous measurements, researchers may examine mean differences, absolute errors, variation across measurement ranges, and limits of agreement where appropriate.
For categorical outputs, relevant measures may include confusion matrices, sensitivity, specificity, or category-specific agreement against an appropriate reference.
Researchers should establish scientifically meaningful tolerances rather than rely solely on statistical significance.
A small average difference may conceal substantial errors among particular participants or activities.
Can Statistical Adjustment Correct Version Differences?
Sometimes, when the relationship between versions is sufficiently understood and supported by appropriate validation data.
For example, a documented, stable calibration difference may permit a justified correction.
However, an algorithm update may change errors differently across activities, signal ranges, or participant groups.
In such cases, a single adjustment factor may be inadequate.
Including firmware version as a statistical covariate does not automatically establish that measurements are comparable.
Researchers should distinguish modeling an observed version effect from correcting the underlying measurement process.
What If Different Participants Receive Updates at Different Times?
Staggered updates can create additional analytical complexity.
For example, participants may install new firmware at different points during a longitudinal study.
If version changes influence measurement, participants may contribute observations collected under different processing conditions.
The timing of updates may also be associated with participant behavior, connectivity, or device management practices.
Researchers should document version histories at the observation level where feasible.
They should examine whether version changes are associated with relevant participant characteristics or study periods.
Can Researchers Reprocess Historical Data?
Reprocessing may be possible when raw or sufficiently detailed source data are retained and compatible processing algorithms are available.
For example, acceleration recordings collected before and after an update might be processed using one documented algorithm if their acquisition characteristics remain compatible.
This can improve consistency, but it does not automatically resolve differences introduced during signal acquisition.
If the earlier firmware recorded different information or discarded essential signal features, later processing may not recover them.
Researchers should verify the comparability of the underlying source data before applying a common algorithm.
What If Only Proprietary Summary Data Are Available?
Some consumer devices provide daily scores or classifications without access to the underlying sensor signals.
If the manufacturer changes the processing algorithm, researchers may be unable to reconstruct the original outputs or apply a consistent method across periods.
In such circumstances, validation of version-specific outputs becomes especially important.
If equivalence cannot be established, researchers may need to analyze periods separately or restrict conclusions to outcomes whose definitions remain stable.
Researchers should avoid treating proprietary summary measures as unchanged simply because the displayed variable names remain the same.
Can Updates Improve Research Validity?
Yes. Updates may correct known measurement errors, improve artifact detection, or enhance calibration.
However, an improvement in accuracy may still introduce a discontinuity between earlier and later measurements.
For example, correcting a systematic overestimation may reduce reported values even when the underlying phenomenon remains unchanged.
The newer measurements may be more accurate individually while the combined longitudinal series becomes less directly comparable.
Researchers should therefore distinguish improved measurement performance from consistency across time.
How Should Version Changes Be Addressed in the Research Protocol?
The protocol should identify critical devices and software components, document the intended configuration, and specify how updates will be managed.
It should also establish procedures for recording changes, assessing their relevance, and determining whether additional validation is required.
Where appropriate, researchers may prespecify sensitivity analyses or rules for handling measurements collected under different versions.
Substantive protocol modifications may require institutional, ethics, or regulatory review depending on the study.
The goal is to prevent measurement changes from becoming undocumented analytical decisions made after results are observed.
How Should Software Changes Be Reported?
Research reports should identify relevant device models, firmware versions, software versions, algorithm configurations, and dates of important changes.
Where measurements span versions, researchers should explain how comparability was evaluated.
Validation findings, harmonization procedures, exclusions, and sensitivity analyses should be reported when they affect interpretation.
For studies involving digital health technologies, the FDA's guidance on remote data acquisition provides relevant considerations concerning the selection, verification, validation, and use of digital technologies in clinical investigations.
Researchers should apply the requirements appropriate to their particular study and jurisdiction rather than assume that clinical regulatory guidance governs all sensor research.
When Do Updates Make Longitudinal Sensor Data Unsuitable?
A longitudinal dataset may become unsuitable for the original question when critical measurements change substantially across versions and no defensible method can establish comparability.
For example, a study investigating small changes in daily activity may be compromised if a new step-counting algorithm introduces an unknown difference of comparable magnitude.
The researcher may need to restrict the analysis to stable periods, use an independently validated outcome, or revise the research question.
Data should not be pooled solely because all records come from the same physical device.
Watch Out
Do not assume that unchanged hardware guarantees unchanged measurements. Firmware, application software, and cloud algorithms can modify how signals are acquired or interpreted. A statistically significant shift after an update may reflect measurement change rather than a genuine change in the phenomenon.
