On the reliability of wearable technology: A tutorial on measuring heart rate and heart rate variability in the wild

Wearable sensors are quickly making their way into psychophysiological research, as they provide the only way to collect longitudinal and ecologically valid data. The present tutorial considers fidelity of physiological measurement with wearable sensors, focusing specifically on reliability of the measurement. We elaborate why ensuring reliability for wearables is important, and offer statistical tools for assessing wearable reliability for between participants and within-participant designs. The tools proposed in this tutorial allow estimating measurement reliability (precision) from data readily available from any wearable sensor, without referencing benchmark equipment. The framework offered here is illustrated in two studies: In Study 1, we applied reliability estimates we offer to a publicly available dataset of heart rate measured from 53 participants with 6 different sensors. In Study 2 we applied reliability calculations to data collected with one wearable sensor of cardiac biometrics (Biostrap) in the wild, outside of the laboratory. Our hope is that by systematically quantifying measurement fidelity in different circumstances, researchers will eventually be able to make informed choices about specific wearable devices and measurement procedures that meet their research goals. Translational Abstract Wearable sensors are the technological breakthrough of this decade. The promise is that devices like smart watches will soon monitor many bodily states, providing data to predict illnesses and optimize performance. While many studies examine the validity of wearable signals, much less attention is paid to their reliability. Here we argue that reliable measurements are crucial for indexing physiological states in the daily life of participants. We offer a framework and several statistical tools for assessing the reliability of sensors. The framework is tested using several brands of commercially available photoplethysmography-based wrist-worn heart rate sensors. The results show that (1) different brands of wearable sensors of heart rate have different between-participant reliability, (2) reliability of heart rate and especially heart rate variability varies along with the intensity of activity participants engage in, (3) reliability of heart rate and especially heart rate variability is higher during sleep than during wakefulness. We conclude that reliability of a wearable sensor is not fixed but varies across different contexts and circumstances. Estimating reliability is thus a useful way to quantify measurement fidelity for a particular research question, a specific new experimental procedure, or a special target population.