Estimating real-world walking speed from a single wearable device: analytical pipeline, results and lessons learnt from the Mobilise-D technical validation study

doi:10.21203/rs.3.rs-2965670/v1

Background:

Estimation of walking speed from wearable devices requires combining a set of algorithms in a single analytical pipeline. The aim of this study was to validate a pipeline for walking speed estimation and assess its performance across different factors (complexity, speed, and walking bout duration) to make recommendations on the use and validity of wearable devices for real-world mobility analysis.

Methods:

Participants with Parkinson's Disease, Multiple Sclerosis, Proximal Femoral Fracture, Chronic Obstructive Pulmonary Disease, Congestive Heart Failure, and healthy older adults (n = 97) were monitored in the laboratory and for 2.5 hours in the real-world, using a wearable device worn on the lower back. Two pipelines for estimating WS were validated across 1298 detected walking bouts, against 1365 walking bouts detected by a multi-sensor reference system.

Results:

In the laboratory, the mean absolute error (MAE) and mean absolute relative error (MARE) for estimation of walking speed ranged from − 0.06 to 0.04 m/s and 2.1–14.4% respectively, with ICCs ranged between good (0.79) and excellent (0.91). The real-world MAE ranged from − 0.04 to 0.11, and MARE from 1.3–22.7%, where ICCs showed moderate (0.57) to good (0.88) agreement. Errors were lower for cohorts with no major gait impairments, for less complex gait tasks and when considering longer walking bouts.

Conclusions:

We demonstrated that the analytical pipelines estimated walking speed with good accuracy. Accuracy was dependent upon confounding factors, highlighting the importance of undertaking a robust technical validation of wearable device-derived walking speed before clinical application.

Trial registration

ISRCTN – 12246987.

Health sciences/Biomarkers

Physical sciences/Engineering/Biomedical engineering

Health sciences/Medical research/Outcomes research

Health sciences/Medical research

Health sciences/Medical research/Translational research

real-world gait

algorithms

digital mobility outcomes

validation

wearable sensor

walking speed

walking

digital health

inertial measurement units

Mobility has been defined as the ability to move freely and easily, representing an essential component of health and quality of life, being key to physical, mental, and social well-being ¹. Sudden loss in mobility, has been associated with morbidity, falls, dementia, cognitive decline, hospitalisations, mortality and symptoms of chronic disorders ^2–6. Mobility loss translates to an inability to perform activities of daily living, which has also been defined as mobility disability ^1,7. According to the World Health Organization (WHO), it is estimated that 1 billion people currently live with mobility disability due to impairments in their respiratory, cardiovascular, musculoskeletal or neurological systems ⁸. This number Is expected to rise further due to increased longevity of the worldwide population and prolonged survival in people with chronic diseases leading to motor detriment. This will entail large public health implications and increase an already growing social and economic burden upon healthcare systems.

Efforts to mitigate the loss of mobility disability are of increasing priority and have been the focus of several recent clinical intervention trials ^9–12. Existing mobility endpoints are based upon patient self-reporting and one-off assessments of physical function, such as the timed up and go test or a 6 minutes’ walk test. While such an assessment provides useful insights on an individual’s mobility capacity (how much they can do), it lacks ecological validity as it does not reflect an individual’s mobility performance (how much they actually do) during daily life ^1,7,13,14. This incomplete assessment of mobility limits therapeutic development and clinical management ¹⁴. Therefore, valid and easy-to-use methods to accurately and reliably assess mobility performance would provide insight into how mobility disability manifests in the real-world ⁷.

Digital health technology, such as wearable devices, offer an objective, low-cost and ecologically valid approach of continuously monitoring real-world mobility performance through characterisation of Digital Mobility Outcomes (DMOs) ⁷. A single wearable device can be worn unobtrusively and comfortably on the lower back, attached to a belt or affixed to the skin ^15,16. Walking speed, remains the most widely explored DMO ¹⁷, where reduced walking speed has been associated with ageing, mortality, neurological and cardiovascular conditions, cognitive decline, and risk of falling ^2,5,18–21. Furthermore, walking speed represents a composite measure of walking ability, as it is estimated from the combination of other spatial and temporal DMOs, specifically stepping cadence and stride length. As such walking speed represents a global measure of mobility that can interpreted and meaningful to patients and clinicians alike ¹⁷.

A lack of robust technical validation of real-world walking speed measurements has prevented the adoption of walking speed derived from wearable devices as a clinical endpoint for interventional trials ^7,22,23. Technical validation requires the comparison of DMOs quantified from a wearable device with DMOs quantified by an established reference system, whilst accounting for and acknowledging a wide range of contextual factors ²⁴. The majority of algorithms to estimate walking speed have typically been validated based upon healthy adults assessed in simple laboratory tasks in standardized and supervised settings that do not represent the more challenging and variable nature of real-world environments ²⁴. Furthermore, studies often validate DMO algorithms in isolation, without considering the complexity of a comprehensive multi-stage pipeline needed to estimate walking speed ^25–28. This first requires the identification of walking activity, followed by the estimation of DMOs (e.g., steps, cadence, stride length, walking speed). Interactions between all algorithmic steps in the pipeline will influence final outputs, and errors will accumulate along the pipeline. A robust validation of walking speed therefore requires analysis of the estimate of walking speed at a walking bout (WB) level from the implementation of the full pipeline.

The aim of this study was to provide a comprehensive validation of walking speed estimated from a single inertial measurement unit based wearable device against a multi-sensor reference system integrating pressure insoles (INDIP) ^29–31, to enable a robust validation: (i) in both laboratory and real-world settings, (ii) across different clinical cohorts with a range of mobility disabilities, (iii) across gait tasks of varying complexity and (iv) accounting for confounding factors (walking bout (WB) duration and walking speed). Subsequently, we provide recommendations for the suitability of a wearable device paired with the proposed analytical pipeline as a measure of real-world mobility and suggest a framework for future studies aiming to validate DMOs for real-world monitoring.

Clinical and demographic characteristics of the participants are presented by cohort in Table 1 (Mean ranges: Age 47 to 79 years, Height 166 to 176 cm, mass 70.6 to 83.6 kg). 108 participants were recruited from the following cohorts: congestive heart failure (CHF), chronic obstructive pulmonary disease (COPD), healthy adult (HA), multiple sclerosis (MS), Parkinson’s disease (PD) and proximal femoral fracture (PFF). Eleven participants (CHF: 4, MS: 3, PD: 1, PFF: 3) were excluded from the laboratory recordings and 26 participants (CHF: 3, HA: 3, MS: 7, PD: 5, PFF: 8) from the real-world recordings due to technical difficulties with either the reference system or the wearable device.

< Insert Table 1 >

An overview of the method of validating walking speed can be viewed in Figure 1.

<Insert Figure 1 >

True Positive Evaluation

In the laboratory assessments a total of 1365 WBs were detected by the reference system and 1298 WBs by the wearable device. To be able to compare DMOs on a WB level, the analysis included WBs that were concurrently detected by both systems (true positive analysis). All WBs with a time-overlap of more than 80% of their duration were considered true positive, resulting in 692 WBs that were considered for analysis and considered a TP (see Methods and Supplementary figures 1 & 2 for more details). Based on these true positive WBs, we observed a mean error of 0.01 m/s (MRE = 5.9 %), and MAE of 0.10 m/s (MARE = 14.96 %) across all cohorts (Figure 2, left. Table 2). We found that walking speed was estimated with good reliability (ICC = 0.84) by the wearable device, with a slight overestimation compared to the INDIP reference system.

In the 2.5-hour real-world assessment, 4409 WBs were detected by the reference system and 4620 WBs by the wearable device. Of these, 1414 (30% of all WBs) were identified as true positives. Based on these WBs, we observed a mean error of 0.06 m/s (MRE = 14.48 %) and a MAE of 0.11 m/s (MARE = 20.31 %) across all cohorts (Figure 2, right, Table 2). As observed in the laboratory data, results showed a good reliability (ICC = 0.77) (Table 3), with an overestimation of walking speed by the wearable device (0.01 m/s).

< Insert Figure 2 >

< Insert Table 2 >

Combined Evaluation

To remove potential bias by focusing only on the true-positive WBs and to mimic actual use of wearable device where reference data may not be available, we performed a second evaluation for which we combined all WBs for a Laboratory test and 2.5h recording in the real world by taking the median of the calculated DMOs (see Methods). These combined values were then compared between the systems. Results from laboratory data showed a mean error over all tests of 0.01 m/s (MRE = 7.47 %) and a MAE of 0.12 m/s (MARE = 17.82 %) (Figure 3, left. Table 3). In contrast, in the real-world we observed a higher mean error over all participants of 0.11 m/s (MRE = 24.48 %) and a MAE of 0.13 m/s (MARE = 26.47 %) (Figure 3, right. Table 3). For both environments the errors were higher than those estimated from the analysis on true-positive WBs. The biggest effect was seen on the ICC during the real-world recording which dropped considerably to 0.33 (moderate) across all cohorts and showed no (PFF: 0.04) or even negative correlation (MS: -0.15). This is not surprising due to the limited number of datapoints included for this type of analysis (one datapoint per participant).

< Insert Figure 3 >

< Insert Table 3 >

Factors than can influence walking speed validity

Influence of the cohort

The MAE based on the true-positive evaluation differed by < 0.05 m/s between cohorts in both laboratory and real-world settings. In the laboratory, the COPD cohort had the lowest MAE (0.06 m/s) followed by HAs (0.08 m/s) (Table 2), whereas the PFF and CHF cohorts had the largest MAE of 0.12 m/s. In the real-world, HAs presented the lowest MAE (0.09 m/s) followed by the PFF cohort (MAE = 0.11 m/s) (Table 2). Walking speed tended to be overestimated for all cohorts apart from CHF, for which walking speed was underestimated by 0.06 m/s in the laboratory and 0.04 m/s in the real-world.

Influence of WB duration and walking speed

In the analysis based on the true positive WBs, errors decreased for longer WB durations (Figure 4). MAE across all cohorts for very short WBs < 10 s ranged between 0.09 and 0.16 m/s, compared to 0.06 to 0.11 m/s for long WBs (between 60s and 120s). However, as WB duration increased, the number of available WBs included in the validation analysis decreased as well. When looking at the combined approach (across all detected WBs) (Figure 5) the trends from the true-positive analysis were confirmed. The MAE for the very short WBs (< 10 s) was lower than for the short WBs (10-30 s), and the number of very short WBs detected by the wearable device was disproportionally higher compared to the reference system. Overall, about two thirds of all WBs were shorter than 30 s. When removing very short WBs (<10 s) from the calculation of the mean/median errors, the range in error was marginally smaller for some cohorts than the error observed at all WBs for the true positive analysis (improvement < 0.1 m/s). The median of the absolute difference of the combined analysis increased from 0.1 m/s over all WBs to 0.14 m/s for the WBs longer than 10 s.

< Insert Figure 4 >

< Insert Figure 5 >

In both environments, a clear linear negative relationship between the magnitude of the reference walking speed and the measurement errors was observed (Figure 2). Walking speed tended to be overestimated for slow and underestimated for fast WBs. For the slowest WBs (<0.6 m/s), we observed the largest absolute errors, increasing to 0.8m/s in real-worlds WBs. This trend was consistent in the true positive and combined analysis.

Influence of task complexity

As task complexity increased, so did the MAE. For instance, the most complex laboratory gait task (“simulated daily activities”) presented the highest MAE across all cohorts (0.17 m/s) and least complex task (the slow straight walking test) presented the lowest MAE (0.08 m/s) (Table 4, Figure 6). Furthermore, the influence of task complexity was cohort dependent. The largest differences between the simple and complex gait tests were observed for the MS, PD, and PFF cohorts (P1 pipeline). In the real-world, for the same cohorts, differences were observed in the errors estimated between the WBs without turns and WBs with turns. For all cohorts, mean error and MAE from real-world assessments were comparable or slightly lower than from simulated daily activities.

< Insert Figure 6 >

< Insert Table 4 here >

To our knowledge, this study is the most extensive validation of a complex comprehensive multi-stage analytical pipeline for estimation of walking speed from a single wearable device. Overall, our findings showed good to excellent validation results in the laboratory and moderate to good agreement in the real-world. We demonstrated that validity of walking speed estimation is slightly impacted by several factors including environment (laboratory vs real-world), clinical cohort, gait task complexity and other confounding factors (number of turns, WB duration, WB speed). Our results have strong implications for future research, below we provide our recommendations for future validations and on the use of wearable device-based walking speed in daily life and more broadly, digital mobility outcomes in general.

Overall Validation Results

Overall, laboratory walking speed demonstrated excellent agreement with the reference system, with the ICCs of the true positive WBs ranging from good (0.79) to excellent (0.91) and MAEs ranging from 0.06 m/s to 0.12 m/s across all cohorts. Within the combined evaluation, the ICC of walking speed were slightly lower (0.72 to 0.82), indicating that only a small difference was introduced by the true positive evaluation. Previous studies conducted across various HAs and various clinical cohorts in laboratory settings have shown lower or comparable results ^27,32–34. However, in comparison to those studies, the pipelines in this study were validated over a wider variety of more complex gait tasks, challenging the estimation of walking speed as the signals are more variable and less cyclic, in comparison to steady-state and straight path gait.

Real-world gait assessment is challenging for walking speed estimation due to more complex and non-standardized environments. Despite these challenges, we achieved good results, since agreement was found to be moderate to good (ICCs within true positive WBs ranging between 0.57 to 0.88) and MAE ranged from 0.09 to 0.13 m/s. As regards to the combined real-world WB analysis, the ICCs were lower than the ICCs from true positive WBs. The MAE remained within usable ranges ( < = 0.18 m/s), but MARE increased up to 44% primarily due to large relative errors for low gait speeds. In the combined analysis, median average walking speeds for each participant was calculated. As this resulted in one data point per participant only, this increased the impact of individual datapoints with larger errors. Furthermore, this analysis reduced the range in walking speeds, where a larger number of slow WBs were included, which further increased the estimation error. These results show that, when aggregating over the combined real-world recording, the bias of the true-positive analysis becomes relevant and the results were significantly influenced by the mismatch in WB detection. Furthermore, the combined approach aggregates real-world data into singular values, which does not reflect the entire the entire distribution of walking speed ³⁵.

Comparing to one of the few other studies that performed a real-world validation of walking speed, the work by Soltani et al, ³⁶ validated an algorithm based on a single wrist-worn sensor against a head-mounted Global Navigation Satellite System device, finding low bias [interquartile range (IQR) = -0.01, 0.00 m/s] and an accuracy expressed by root mean square error [IQR = 0.04, 0.06 m/s]. However, this validation was only performed in 30 HAs (mean age = 37 years) ³⁶. Given the promising results we report for estimation of walking speed, and the results provided for the individual algorithmic blocks previously reported ²⁶, we demonstrate that it is possible to use a single wearable device on the lower back for accurate quantification of mobility. However, it must be considered that the performance of algorithms and pipelines are dependent upon a variety of factors that should be taken into consideration during study design, future validation, and data interpretation.

Recommendation for real-world DMO validation

Validation protocol

Across all cohorts we observed larger absolute errors and lower ICCs with walking speed estimated from the real-world in comparison to laboratory assessment, showing the importance of real-world validations to obtain realistic and ecologically valid error estimates of DMOs.

Despite this, our results also show that some real-world challenges can be replicated within laboratory settings, as the errors observed during the simulated-daily activities in the laboratory were in fact higher than in the real-world. In these tasks, participants undertook short WBs containing turns, changes of direction and transitions. Scott, et al, ³⁷ compared the walking speed ranges recorded from the laboratory and 2.5 hours protocol that were adopted in the present study and found a diverse profile of walking speed ranges in the laboratory that was representative of the walking speed range observed in the real-world. Future validation studies should take into account an adequate balance between challenging tasks (short WBs, turns and transitions) and long uninterrupted walks in the laboratory protocol to properly replicate the expected error ranges from the real-world.

In general, the expected error ranges were dependent upon task complexity. Most condition specific differences were only prevalent in the real-world, which is consistent with previous research reported in HAs and people with MS and PD ^13,18,38. Our findings motivate the inclusion of complex tasks and simulated daily activities into any future laboratory validation. However, we also recommend inclusion of real-world measurements to capture the true range of gait task complexity performed in daily life as well as myriad of contextual factors, including the distribution of WB duration and walking speed.

Reference system

Utilisation of the INDIP system as a reference during both the laboratory-based and real-world protocol proved to be successful in overcoming limitations in accuracy, battery life and usability, all of which are common restrictions of real-world reference systems previously adopted in the literature (e.g., wearable camera and GNNS systems ^36,39). Specifically, the INDIP system has been validated showing excellent agreement (ICC > 0.95) and very low MAEs (simulated daily activities = ≤ 0.05 m/s) against a stereophotogrammetric system in the same cohorts and laboratory protocol as in the present study ³¹. The INDIP system can be used in both laboratory and real-world settings to enable a concurrent validation of walking speed measurements, as provided in the present study. The recording duration of 2.5 hours with the INDIP system enabled recording of a wide range of activities and walking speeds.

Data analysis

We adopted two approaches to analysing walking speed, (i) only considering WBs that were directly matched between the wearable device and reference system (true positive evaluation) and (ii) considering the median value of walking speed across all available data (combined analysis).

The true positive evaluation allowed comparison to be performed with high granularity on a WB level, allowing better understanding of the circumstances under which the wearable device performs best. However, for the true positive analysis we observed bias with regards to the overall walking speed ranges (Supplementary Figs. 1 & 2), resulting in a non-negligible impact on the real-world results. Therefore, the combined analysis is required to confirm observed error ranges and differences in the results of the two approaches should be considered and discussed. Furthermore, this type of analysis introduced the true-positive threshold as a parameter that influences the results. While we could not find a relevant effect of the selection of this parameter value on the walking speed error (Supplementary Figs. 1 & 2), this might influence other DMOs. As our results indicate, no single type of analysis can provide a definite and full picture of the error ranges. Given the lack of other established approaches to perform real-world comparisons of DMOs with a high granularity, we suggest our framework as a basis for future DMO validation studies.

Practical recommendations for the use of wearable devices for real-world walking speed measurements

Our results demonstrate that walking speed can be estimated accurately and reliably across a range of environments, cohorts, tasks and contextual factors. Based upon our promising validation results, below we provide our recommendations on the use of wearable device for real-world walking speed measurement.

Influence of pipeline

For improved understanding of the error, the impact on individual DMOs within the pipeline should be considered. In our case, given the complexity of the respective algorithms, stride length is expected to have a larger contribution to the observed walking speed error compared to cadence (Supplementary tables 1 & 2) ²⁶. This motivates further research in more robust methods for spatial parameter estimation. Furthermore, the wearable device seems to record more shorter WBs than the reference, suggesting that longer continuous bouts of walking were split into multiple shorter WBs. Based on specific investigations of such cases, this was often due to limitations of the initial contact and left-right detection. Under challenging conditions (e.g., turns or stairs), these algorithms could not provide reliable stride information leading to a separation of longer periods of walking into multiple WBs, as no valid stride was detected for multiple seconds. This could be the result of the wearable device being positioned on the lower back, where the reference system was also comprised of feet sensors, thus being more robust to quantify gait events across longer periods. The full pipeline is implemented separately for each system, so the combined estimates of all DMOs needed to meet criteria of a WB leads to heterogeneity between systems. This motivates further research in the detection of initial contacts and their laterality under challenging real-world conditions.

Walking bout duration

Real-world walking speed encapsulates a rich dataset of mobility that has been undertaken across various WBs which differ in their length, duration, and context. Each WB reflects a different profile of walking in terms of the number of turns, transitions, and periods of straight walking, which influences walking speed measurement. Therefore, it is not surprising that walking speed estimations were influenced by the WB duration, where very short WBs (< 10 seconds) presented the largest error. We speculate that very short WBs predominantly took place within confined indoor spaces such as the home environment, and that walking speed at this short duration do not reflect steady state gait activities ⁴⁰. Hence, algorithms optimised for straight walking in controlled settings had an increased likelihood of higher absolute errors at very short durations. Based on this, we would recommend using a lower cut-off (WBs > 10 s), to trade-off between the number of removed WBs and still including a minimum threshold of 401 WBs, needed to ensure reliability and validity for real-world gait monitoring in a single cohort ³⁷.

Moving toward clinical application of wearable devices and walking speed measurement, it is important to consider in which specific real-world context wearable devices can quantify mobility most accurately and reliably. Our findings demonstrated that WBs > 30 seconds provide the most accurate and reliable measurement. WBs > 30 can be characterised as medium to long in their duration. Walking speed estimated from medium length WBs (between 30 to 60 seconds), may reflect activities of daily living, such as intermittent periods of shopping or undertaking other errands in public spaces outside the home ^18,41. In contrast, longer WBs (> 60 seconds), typically capture faster walking speeds that are closer to what is already being measured in the laboratory. Thus, walking speed measured in medium length WBs reflects a balance between capturing activities of daily living, and sufficient periods of straight walking activity that enable the robust quantification of walking speed. However, for certain patients walking continuously for 30 s as our cut-off suggests might already be strenuous. Thus, we would recommend using all (WBs > 10 s) to include a balance between capturing a sufficient number of WBs for patients with a variety of condition severities, whilst ensuring walking speed can still be quantified reliably. Future clinical validation studies with a larger number of participants with severe gait impairments are required to confirm the reported error ranges for specific disease populations and can confirm the influence of WB duration upon the functional insight of mobility provided by walking speed ⁴².

Walking speed

The influence of the speed at which each WB is completed upon the validity of the walking speed is considered a confounding factor of gait analysis ⁴³. When exploring the average walking speed across all WBs, we found that walking speed at WBs undertaken at slower speeds (< 0.6 m/s) tended to be overestimated by ≤ 0.8 m/s the wearable device and WBs with faster speeds were underestimated, where moderate walking speeds provided the highest accuracy (Fig. 2). Longer WBs (> 60 seconds) were completed at faster walking speed in comparison to medium length WBs, which were undertaken at moderate speeds. Thus, this provides further justification to our recommendation that medium length WBs provide the right balance between functional relevance and accuracy. Exploration of the clinical properties of walking speed encapsulated within these WBs, will become a topic of research in further investigated in on-going clinical validation efforts ⁴². Furthermore, measurements with cohorts consisting of predominantly slow walkers will likely result in larger error ranges. This is consistent with previous research that also validated algorithms based upon a single lower back sensor ^33,44.

The algorithms used in this study were optimized and developed based on independent datasets to avoid bias. We foresee that future algorithm developed on the TVS dataset, and other similar real-world datasets can improve on the speed dependency observed here.

Real-world complexity

Aside from the WB duration, accuracy of walking speed estimation was also dependent upon the complexity of tasks/activities. The influence of complexity was cohort dependent and had the largest influence upon error for the MS, PD and PFF cohorts (estimated from P1 pipeline). We would expect those cohorts to experience more gait impairments than the CHF, COPD and HA cohorts (estimated from the P2 pipeline). However, whether the observed effect is caused by specific gait properties of the respective cohorts or by shortcomings in the selected algorithms, cannot be concluded based on the performed analysis.

Despite the challenges posed by outdoor environments (changes in terrain, weather and traffic negotiation (humans and vehicles)) ^45,46, outdoor environments capture more prolonged and uninterrupted walks in comparison to indoor environments, such as the household, which represent more confined and cluttered spaces with limited capacity for completing sequences of straight walking. Thus, we would expect error ranges from long uninterrupted outdoor walks are expected to be lower than results from confined indoor environments. Therefore, the combination of gait parameters with further contextual information might help to take this into account during data interpretation.

Limitations

While this study is one of the largest and most comprehensive validation studies for gait analysis based on wearable devices to date, the analysis of specific subgroup effects would require larger sample sizes. Potential links between the error, the condition severity and other medical comorbidities could not be established. Furthermore, the effect of walking aid use on results has not been assessed in this study. Future studies with more variable condition severity are needed to explore the influence of walking aid usage upon the validity of the analytical pipelines.

The real-world data was limited to 2.5 hours for technical reasons. However, we accept that a recording of this length may not be sufficient to capture all the variability and patterns that would be included in multiple days of consecutive assessment. Due to technical issues with the devices, we were unable to assess some participants, which reduced our dataset. Data was also collected during the COVID-19 pandemic which may have impacted on participants’ activity. As previously stated, the analytical pipelines for the wearable device data have the tendency to split longer WBs into multiple individual WBs. Hence, future research should explore whether this is caused by limitations of the initial contact and Left-Right detections, and how specific real-world contexts may influence walking speed performance. We found that error in walking speed estimation was more dependent upon stride length (spatial) estimation (MARE across all cohorts; laboratory = 14.31% and real-world 20.35%), however cadence (temporal) can be estimated with substantially lower errors (MARE across all cohorts; laboratory = 4.1% and real-world 4.8%) (Supplementary tables 1 & 2). Therefore, future researchers looking to further improve performance of walking speed estimation, should target optimisation of spatial algorithms.

Through the extensive real-world and laboratory validation across multiple cohorts, this study represents, to the best of our knowledge, the most accurate estimate of the expected error ranges of a lower-back wearable device for estimation of walking speed. The presented state-of-the-art algorithms-pipelines could reliably estimate DMOs across a wide range of scenarios, providing a solid foundation for future studies to establish their clinical meaningfulness ⁴². Our results showed that various parameters can influence DMO performance and multi-faceted analysis is crucial for understanding of the capabilities of any DMO pipeline. This motivates the capture of additional context information during real-world measurements to focus analysis on signal areas where high reliability can be expected. Furthermore, we identified clear areas where future algorithm pipelines can still improve, and we believe that the captured dataset will be vital for the development of future algorithms specifically targeting the challenges of unsupervised real-world recordings.

Participants

For the Mobilise-D technical validation study (TVS), 108 participants were recruited from five clinical cohorts; CHF, COPD, MS, PD, and PFF alongside HA. Participants were recruited at five sites: The Newcastle upon Tyne Hospitals NHS Foundation Trust, UK (Sponsor of the study) and Sheffield Teaching Hospitals NHS Foundation Trust, UK (ethics approval granted by London – Bloomsbury Research Ethics committee, 19/LO/1507); Tel Aviv Sourasky Medical Center, Israel (ethics approval granted by the Helsinki Committee, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel, 0551-19TLV), Robert Bosch Foundation for Medical Research, Germany (ethics approval granted by the ethical committee of the medical faculty of The University of Tübingen, 647/2019BO2), University of Kiel, Germany (ethics approval granted by the ethical committee of the medical faculty of Kiel University, D438/18). Informed consent was provided by all participants to take part in the study and all research was performed in accordance with the Declaration of Helsinki. Inclusion and exclusion criteria are fully described in ²⁴.

Protocol

The protocol has been extensively detailed in ²⁴. Participants were assessed in the laboratory and during a 2.5-hour real-world observation. Mobility data was collected with a wearable device (McRoberts Dynaport MM+, sampling frequency: 100 Hz, triaxial acceleration range: ±8 g / resolution: 1 mg, triaxial gyroscope range: ±2000 degrees per second (dps)/ resolution: 70 mdps), secured at the waist with a Velcro belt. Participants were also asked to wear a multisensor INDIP reference system (sampling frequency: 100 Hz) ^24,30. Specifically, two magneto-IMUs were positioned over the instep and fixed to shoelaces with clips, and a third IMU was attached the lower back with Velcro. Distance sensors were then positioned asymmetrically with Velcro (one above left ankle and another 3cm higher on the right leg). Pressure insoles were selected for each participant’s foot size and inserted into the shoe. The INDIP system has been validated in previous studies across a range of conditions and in this TVS cohorts also, showing excellent results and reliability in the qualification of mobility outcomes (MAE laboratory ≤ 0.02 m/s, simulated daily activities = 0.03 to 0.05 m/s), a complete overview of the validation results can be found in ³¹. The INDIP and the wearable device were synchronised using their timestamps (± 10 ms). Participants only performed tasks that they felt comfortable and safe to do so in both protocols.

Laboratory protocol

Participants were asked to complete seven motor tasks with increasing complexity: Straight walking (slow, normal and fast speed), Timed Up and Go, L-Test, Surface Test, Hallway Test and Simulated Daily Activities. Each task was designed to capture and assess various elements associated with real-world walking including a range of walking speeds, incline/steps, surface, path shape, turns and specific motor tasks to simulate typical real-world transitions ^24,37.

Real-world protocol

Participants were assessed for up to 2.5 hours in the real-world, as they went about their normal activities unsupervised (home/work/community/outdoor). To ensure a variety of walking activities were captured, participants were asked to perform specific tasks such as outdoor walking, walking up and down a slope and stairs and moving from one room to another, only if they felt comfortable and safe to do so ²⁴.

Calculation of Walking Speed

The evaluation of walking speed requires the combination of various algorithmic steps, including the identification of gait sequences and of initial contacts, estimation of Digital Mobility Outcomes (DMOs), i.e., cadence and stride length. The performance and selection of state-of-the-art algorithms to detect gait-sequences, estimate initial contact events, cadence and stride length within identified gait-sequences was determined in our previous work ²⁶ (Figure 1). The best performing algorithm was then used to estimate walking speed using the outputs of the stride length and cadence algorithms using eq. (1):

W (1)

< Insert Figure 7 >

Two independent analytical pipelines (P1 and P2) were identified in this process due to differences in algorithm selection for gait-sequence detection and cadence for the different conditions included in the study ²⁶. P1 provides the optimal combination of algorithms selected for HA, COPD, and CHF conditions, and P2 provides the optimal combination for PD, MS, and PFF (Figure 7).

Two additional algorithms were added to both gait analysis pipelines: turn detection algorithm ⁴⁷, and a customised algorithm to detect the laterality (left or right step) of each IC ⁴⁸. Laterality was used to interpolate the cadence, stride length, and walking speed parameters (provided as per-second values by the algorithms) to stride-level values (stride interpolation).

DMOs were evaluated on a stride level, conforming to consensus agreed definitions ⁴⁹ for WBs. Accordingly, a WB was defined as a continuous sequence containing at least two consecutive strides of both feet (e.g., R-L-R-L-R-L or L-R-L-R-L-R, being R/L the right/left foot contact with the ground); consecutive WBs were defined if a break greater than 3 s was identified between them; and, for a stride to be included in a given WB it had to have a duration between 0.2s and 3.0 s and a stride length > 0.15 m ⁵⁰. WBs compliant with this definition were generated by first filtering the list of identified strides based on the stride level definition (stride selection) and then grouped into final WBs based on breaks within the stride sequence (walking bout assembly). The same definitions were also used to define the WBs for the reference system. For both systems final DMOs were calculated as the average value over all strides within a WB.

Validation of Walking Speed

All comparisons between the wearable device and the reference system where performed based on the average walking speed within each identified WB. In addition, comparison results for cadence and stride length can be found in Supplementary tables 1 & 2.

Real-world recordings also provide new challenges during data analysis. WBs detected by the wearable device, and reference system might not match up, thus direct comparison of individual strides or WBs is not possible. One straight-forward approach is to average DMOs across all WBs before comparison. However, this reduced granularity makes it difficult to fully understand under which circumstances a wearable device works well and can “mask” the bias or error (e.g., over or underestimation under specific circumstances) that only considering single WB could be identified. We proposed a new approach for these types of data analysis, by splitting the analysis into a detailed comparison of only WBs that were identified in both systems (True positive WBs) and a traditional analysis of all data combined (Combined WBs).

True Positive Evaluation: Novel method of analysis, which directly compares the performance of the DMOs on only the WBs that were detected in both systems (true positives). This allows for the calculation of traditional of comparison metrics (e.g., inter-class-correlation and Bland-Altman plots), that require a direct comparison of individual measurement points. WBs were included in the true positive analysis, if there was an overlap of more than 80% between the two systems (details about the selection of this threshold can be found in Supplementary figure 1).
Combined Evaluation: Traditional method of analysis, where we calculated the median walking speed from all identified WBs within each laboratory task (resulting in one value per gait task per participant) or within the 2.5 hours real-world assessment per participant (resulting in one value per participant) and compared these combined values between the systems. This comparison is free of potential biases introduced by the selection of only the true-positive WB and reflects how DMOs will typically be evaluated in a research or clinical setting or when reference data may not be available.

Factors than can influence walking speed validity

A range of factors can influence walking speed, and this may impact on the algorithm performance and validity of the results. We investigated the possible sources of confounding such as: the cohort, environment (laboratory vs real-world), task complexity, walking speed and walking bout duration, and participant performance upon walking speed validity. All comparisons (unless otherwise stated) are performed using WBs identified as true positive (true positive evaluation).

Influence of the cohort and environment

We compared errors in the estimation of walking speed between each of the different clinical cohorts included in the study, alongside differences between laboratory and real-world environments.

Influence of gait task complexity

Real world walking contains complex gait sequences, which are comprised of short steps, frequent turns, or obstacle negotiation where individuals often multitask during walking. Thus, gait patterns observed in the real-world are not comparable with the straight walking tasks undertaken in controlled environments, even if we account for differences in WB duration and walking speed. To assess the effect of gait-task complexity we compared validation results of walking speed estimated from the (i) simulated daily activities (high complexity), (ii) slow straight walking (low complexity), (iii) all straight walking tasks (low complexity), and (iv) all laboratory tests with the validation results of walking speed estimated from real-world walking. We further subdivided real world walking based on the percentage of a WB that was assessed to be a turn. Based on this we defined the following levels of gait complexity: (v) “simple” straight gait (<20% covered by turns) and (vi) “complex gait” (>=60% covered by turns).

Influence of walking speed and walking bout duration

Given the impact of real-world WB durations and speeds ⁴⁰ on the adopted biomechanical strategies ⁵¹, we analysed their influence on the validity of the walking speed. For this, we assessed whether validity of walking speed estimation differed within specific WB durations bins (< 10 seconds, > 10 seconds, 10 to 30 seconds, 30 to 60 seconds, > 60 seconds and > 120 seconds). This was first performed for all true positive WBs comparing their errors across each WB threshold, and subsequently repeated for the combined analysis, by calculating the median walking speed for each participant within the respective speed bout and comparing the median values between the reference system and the wearable device. All these analyses permitted the validation of the quantification of walking speed across different walking strategies.

Validation measures

For all types of evaluations (all available WBs/aggregated values or on the respective subgroups), we calculated various statistical/comparison measures to quantify the walking speed estimation error for the sensitivity analysis:

Intra Class Correlation Coefficient (ICC_(2,1)) ⁵² was calculated to assess the association between the DMOs of the two systems. Based on ICC estimates, values < 0.5, between 0.5 and 0.75, between 0.75 and 0.9, and > 0.90 were deemed to be indicative of poor, moderate, good, and excellent reliability, respectively ⁵³.
Absolute agreement was assessed by quantifying (i) the accuracy/mean absolute error (MAE), (ii) bias/mean error and (iii) precision/limits of agreements (LoA) ⁵⁴ between walking speed estimates of both systems.
Mean relative errors (MRE) and mean absolute relative error (MARE) were estimated as the ratio between the (absolute) errors per WB and the corresponding estimates from the reference system, expressed as a percentage.

Data availability statement

Representative data from the dataset presented in this study can be found on Zenodo online repository. Permanent link and DOI will be generated upon the paper’s acceptance.

Funding

This work was supported by the Mobilise-D project that has received funding from the Innovative Medicines Initiative 2 Joint Undertaking (JU) under grant agreement No. 820820. This JU receives support from the European Union's Horizon 2020 research and innovation program and the European Federation of Pharmaceutical Industries and Associations (EFPIA). SDD, LR, AY were also supported by the Innovative Medicines Initiative 2 Joint Undertaking (IMI2 JU) project IDEA-FAST - Grant Agreement 853981. LA, LR, AY and SDD were also supported by the National Institute for Health Research (NIHR) Newcastle Biomedical Research Centre (BRC) based at The Newcastle upon Tyne Hospital NHS Foundation Trust, Newcastle University and the Cumbria, Northumberland and Tyne and Wear (CNTW) NHS Foundation Trust. LA, LR, AY and SDD were also supported by the NIHR/Wellcome Trust Clinical Research Facility (CRF) infrastructure at Newcastle upon Tyne Hospitals NHS Foundation Trust. This study was also supported by the National Institute for Health Research (NIHR) through the Sheffield Biomedical Research Centre (BRC, Grant Number IS-BRC-1215–20017). ISGlobal acknowledges support from the Spanish Ministry of Science and Innovation through the “Centro de Excelencia Severo Ochoa 2019-2023” Program (CEX2018-000806-S), and from the Generalitat de Catalunya through the CERCA Program.

All opinions are those of the authors and not the funders. The content in this publication reflects the authors’ view, and neither IMI nor the European Union, EFPIA, NHS, NIHR, DHSC, or any associated partners are responsible for any use that may be made of the information contained herein.

Acknowledgements

The authors acknowledge all the members of the Mobilise-D Work Package 2 for continuous discussion and critical input. They are particularly grateful to the participants in the study for their time and enthusiastic contribution, especially during the pandemic.

Author contributions

PIs – Project development and study design: SDD, CM, LR, AC, AM, BV. Data collection and pre-processing of the data: TB, FS, KS, LA, PB, SB, FS, MC, EB, EG, CH, LP, IA and LS. Patient recruitment and clinical oversight: BS, WM, CB, JH, IV, EH, DM, AY, LR. Algorithm development: AK, API, AS, MU, EG, FK. Data management platform and analysis: HH, DS, AK, CK, MU. Data analysis, statistical analysis: AK, EMA and CK. Figures and tables preparation: AK, CK. Data interpretation: EMA, SDD, AK, TB, LR, AC, CM. Drafting of the manuscript: AK, CK, EMA. Intellectual contribution: EMA, AK, SDD, TB, LR, AC, SB, SK, JG, CM, AC, CK, AM, HS, KA, BE, BC, AK, BV, FK, LC, MN. All authors have provided critical intellectual input during the revision of the manuscript. All authors have reviewed the manuscript and approved the submitted version.

Competing interests

A. Mueller and F. Kluge are employees of, and may hold stock in, Novartis. B. Eskofier reports consulting activities with adidas AG, Siemens AG, Siemens Healthineers AG, WSAudiology GmbH outside of the study. He is a shareholder in Portabiles HealthCare Technologies GmbH. In addition, B. Eskofier holds a patent related to gait assessment. H. Sillén is an employee of, and may hold stock in, AstraZeneca. L. Palmerini and L. Chiari are co-founders and own shares of mHealth Technologies (https://mhealthtechnologies.it/). L. Schwickert and C. Becker are consultants of Philipps Healthcare, Bosch Healthcare, Eli Lilly, Gait-up. M. Niessen is an employee of McRoberts. Jeff Hausdorff reports having submitted a patent for assessment of mobility using wearable sensors in 400 Parkinson’s disease. The intellectual property rights 401 are held by the Tel Aviv Medical Center. S. Del Din reports consultancy activity with Hoffmann-La Roche Ltd. outside of this study. The other authors have no competing interests.

Viceconti, M. et al. On the use of wearable sensors as mobility biomarkers in the marketing authorization of new drugs: A regulatory perspective. Frontiers in Medicine 9, (2022).
Jehu, D. A. et al. Risk factors for recurrent falls in older adults: A systematic review with meta-analysis. Maturitas 144, 23–28 (2021).
Walsh, J. A. et al. Gait speed and adverse outcomes following hospitalised exacerbation of COPD. Eur Respir J 58, 2004047 (2021).
Del Din, S. et al. Analysis of Free-Living Gait in Older Adults With and Without Parkinson’s Disease and With and Without a History of Falls: Identifying Generic and Disease-Specific Characteristics. J Gerontol A Biol Sci Med Sci 74, 500–506 (2019).
Cameron, M. H. & Nilsagard, Y. Balance, gait, and falls in multiple sclerosis. Handb Clin Neurol 159, 237–250 (2018).
Dyer, S. M. et al. A critical review of the long-term disability outcomes following hip fracture. BMC Geriatr 16, 158 (2016).
Rochester, L. et al. A Roadmap to Inform Development, Validation and Approval of Digital Mobility Outcomes: The Mobilise-D Approach. Digit Biomark 4, 13–27 (2020).
Disability and health. https://www.who.int/news-room/fact-sheets/detail/disability-and-health.
Ginis, P. et al. Feasibility and effects of home-based smartphone-delivered automated feedback training for gait in people with Parkinson’s disease: A pilot randomized controlled trial. Parkinsonism Relat Disord 22, 28–34 (2016).
Jonsdottir, J. et al. Intensive Multimodal Training to Improve Gait Resistance, Mobility, Balance and Cognitive Function in Persons With Multiple Sclerosis: A Pilot Randomized Controlled Trial. Frontiers in Neurology 9, (2018).
Zhang, Y.-Y., Zhang, Y.-G., Li, Z., Li, S.-H. & Xu, W.-G. Effect of Home-based Telerehabilitation on the Postoperative Rehabilitation Outcome of Hip Fracture in the Aging Population. Orthopaedic Surgery 14, 1768–1777 (2022).
Bisht, B. et al. Effects of a multimodal intervention on gait and balance of subjects with progressive multiple sclerosis: a prospective longitudinal pilot study. Degener Neurol Neuromuscul Dis 7, 79–93 (2017).
Hillel, I. et al. Is every-day walking in older adults more analogous to dual-task walking or to usual walking? Elucidating the gaps between gait performance in the lab and during 24/7 monitoring. Eur Rev Aging Phys Act 16, 6 (2019).
Warmerdam, E., Hausdorff, J. M., Atrsaei, A., Zhou, Y. & Maetzler, W. Long-term unsupervised mobility assessment in movement disorders. The Lancet Neurology (2020) doi:10.1016/S1474-4422(19)30397-7.
Keogh, A., Taraldsen, K., Caulfield, B. & Vereijken, B. It’s not about the capture, it’s about what we can learn”: a qualitative study of experts’ opinions and experiences regarding the use of wearable sensors to measure gait and physical activity. Journal of NeuroEngineering and Rehabilitation 18, 78 (2021).
Keogh, A. et al. Acceptability of wearable devices for measuring mobility remotely: Observations from the Mobilise-D technical validation study. DIGITAL HEALTH 9, 205520762211507 (2023).
Polhemus, A. et al. Walking on common ground: a cross-disciplinary scoping review on the clinical utility of digital mobility outcomes. npj Digit. Med. 4, 1–14 (2021).
Del Din, S., Godfrey, A., Galna, B., Lord, S. & Rochester, L. Free-living gait characteristics in ageing and Parkinson’s disease: impact of environment and ambulatory bout length. J NeuroEngineering Rehabil 13, 46 (2016).
Rosmaninho, I., Ribeirinho-Soares, P. & Nunes, J. P. L. Walking Speed and Mortality: An Updated Systematic Review. South Med J 114, 697–702 (2021).
Fonseca Alves, D. J. et al. Walking Speed, Risk Factors, and Cardiovascular Events in Older Adults-Systematic Review. J Strength Cond Res 31, 3235–3244 (2017).
Morris, R., Lord, S., Bunce, J., Burn, D. & Rochester, L. Gait and cognition: Mapping the global and discrete relationships in ageing and neurodegenerative disease. Neurosci Biobehav Rev 64, 326–345 (2016).
Del Din, S., Kirk, C., Yarnall, A. J., Rochester, L. & Hausdorff, J. M. Body-Worn Sensors for Remote Monitoring of Parkinson’s Disease Motor Symptoms: Vision, State of the Art, and Challenges Ahead. J Parkinsons Dis 11, S35–S47 (2021).
Viceconti, M. et al. Toward a regulatory qualification of real-world mobility performance biomarkers in parkinson’s patients using digital mobility outcomes. Sensors 20, 1–13 (2020).
Mazzà, C. et al. Technical validation of real-world monitoring of gait: a multicentric observational study. BMJ Open 11, e050785 (2021).
Pacini Panebianco, G., Bisi, M. C., Stagni, R. & Fantozzi, S. Analysis of the performance of 17 algorithms from a systematic review: Influence of sensor position, analysed variable and computational approach in gait timing estimation from IMU measurements. Gait & Posture 66, 76–82 (2018).
Micó-Amigo, M. E. et al. Assessing real-world gait with digital technology? Validation, insights and recommendations from the Mobilise-D consortium. (2022). doi:10.21203/rs.3.rs-2088115/v1.
Soltani, A. et al. Algorithms for walking speed estimation using a lower-back-worn inertial sensor: A cross-validation on speed ranges. IEEE Trans Neural Syst Rehabil Eng PP, (2021).
Paraschiv-Ionescu, A. et al. Locomotion and cadence detection using a single trunk-fixed accelerometer: validity for children with cerebral palsy in daily life-like conditions. Journal of NeuroEngineering and Rehabilitation 16, 24 (2019).
Madgwick, S. O. H., Harrison, A. J. L. & Vaidyanathan, R. Estimation of IMU and MARG orientation using a gradient descent algorithm. in 2011 IEEE International Conference on Rehabilitation Robotics 1–7 (2011). doi:10.1109/ICORR.2011.5975346.
Bertuletti, S., Della Croce, U. & Cereatti, A. A wearable solution for accurate step detection based on the direct measurement of the inter-foot distance. J Biomech 84, 274–277 (2019).
Salis, F. et al. A multi-sensor wearable system for gait assessment in real- world conditions: performance in individuals with impaired mobility. (2023). doi:10.21203/rs.3.rs-2486943/v1.
Paraschiv-Ionescu, A., Soltani, A. & Aminian, K. Real-world speed estimation using single trunk IMU: methodological challenges for impaired gait patterns*. in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC) 4596–4599 (2020). doi:10.1109/EMBC44109.2020.9176281.
Del Din, S., Godfrey, A. & Rochester, L. Validation of an Accelerometer to Quantify a Comprehensive Battery of Gait Characteristics in Healthy Older Adults and Parkinson’s Disease: Toward Clinical and at Home Use. IEEE Journal of Biomedical and Health Informatics 20, 838–847 (2016).
Byun, S. et al. Walking-speed estimation using a single inertial measurement unit for the older adults. PLoS ONE 14, (2019).
Atrsaei, A. et al. Gait speed in clinical and daily living assessments in Parkinson’s disease patients: performance versus capacity. NPJ PARKINSONS DISEASE 7, (2021).
Soltani, A., Dejnabadi, H., Savary, M. & Aminian, K. Real-World Gait Speed Estimation Using Wrist Sensor: A Personalized Approach. IEEE Journal of Biomedical and Health Informatics 24, 658–668 (2020).
Scott, K. et al. Design and validation of a multi-task, multi-phase protocol for real-world gait simulation. (2022). doi:10.21203/rs.3.rs-1908917/v1.
Shema-Shiratzky, S. et al. A wearable sensor identifies alterations in community ambulation in multiple sclerosis: contributors to real-world gait quality and physical activity. J Neurol 267, 1912–1921 (2020).
Hickey, A., Din, S. D., Rochester, L. & Godfrey, A. Detecting free-living steps and walking bouts: validating an algorithm for macro gait analysis. Physiol. Meas. 38, N1 (2016).
Rehman, R. Z. U. et al. Investigating the Impact of Environment and Data Aggregation by Walking Bout Duration on Parkinson’s Disease Classification Using Machine Learning. Frontiers in Aging Neuroscience 14, (2022).
Shah, V. V. et al. Effect of Bout Length on Gait Measures in People with and without Parkinson’s Disease during Daily Life. Sensors (Basel) 20, 5769 (2020).
Mikolaizak, A. S. et al. Connecting real-world digital mobility assessment to clinical outcomes for regulatory and clinical endorsement–the Mobilise-D study protocol. PLOS ONE 17, e0269615 (2022).
Hubble, R. P., Naughton, G. A., Silburn, P. A. & Cole, M. H. Wearable Sensor Use for Assessing Standing Balance and Walking Stability in People with Parkinson’s Disease: A Systematic Review. PLoS One 10, e0123705 (2015).
Trojaniello, D. et al. Estimation of step-by-step spatio-temporal parameters of normal and impaired gait using shank-mounted magneto-inertial sensors: application to elderly, hemiparetic, parkinsonian and choreic gait. Journal of NeuroEngineering and Rehabilitation 11, 152 (2014).
Kim, Y. & Brown, R. Effect of meteorological conditions on leisure walking: a time series analysis and the application of outdoor thermal comfort indexes. Int J Biometeorol 66, 1109–1123 (2022).
Kowalsky, D. B., Rebula, J. R., Ojeda, L. V., Adamczyk, P. G. & Kuo, A. D. Human walking in the real world: Interactions between terrain type, gait parameters, and energy expenditure. PLoS One 16, e0228682 (2021).
El-Gohary, M. et al. Continuous Monitoring of Turning in Patients with Movement Disability. Sensors 14, 356–369 (2014).
Ullrich, M. et al. Machine learning-based distinction of left and right foot contacts in lower back inertial sensor gait data. in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC) 5958–5961 (2021). doi:10.1109/EMBC46164.2021.9630653.
Kluge, F. et al. Consensus based framework for digital mobility monitoring. PLOS ONE 16, e0256541 (2021).
Bonci, T. et al. An Objective Methodology for the Selection of a Device for Continuous Mobility Assessment. Sensors 20, 6509 (2020).
Najafi, B., Helbostad, J. L., Moe-Nilssen, R., Zijlstra, W. & Aminian, K. Does walking strategy in older people change as a function of walking distance? Gait Posture 29, 261–266 (2009).
McGraw, K. O. & Wong, S. P. Forming inferences about some intraclass correlation coefficients. Psychological Methods 1, 30–46 (1996).
Koo, T. K. & Li, M. Y. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med 15, 155–163 (2016).
Giavarina, D. Understanding Bland Altman analysis. Biochem Med (Zagreb) 25, 141–151 (2015).

Table 1. Demographic and clinical characteristics of the participants included the real-world analysis. Values are presented as mean ± standard deviation.

Characteristic	HA (n=17)	CHF (n=9)	COPD (n=17)	MS (n=13)	PD (n=15)	PFF (n=11)
Age (years)	72.35 ± 6.00	68.00 ± 12.80	69.35 ± 9.10	47.23 ± 11.09	69.20 ± 7.48	79.70 ± 6.86
Height (cm)	167.00 ± 10.91	176.33 ± 8.75	168.97 ± 6.61	166.31 ± 9.11	172.73 ± 7.96	170.23 ± 9.07
Weight (kg)	74.36 ± 12.53	83.61 ± 19.56	73.71 ± 14.22	80.09 ± 22.11	79.13 ± 16.27	70.59 ± 16.86
Walking aid users (% [n])	5% [1]	22% [2]	5% [1]	30% [4]	26% [4]	45% [5]
MoCa [0 - 30]	28.18 ± 1.38	26.56 ± 3.21	24.65 ± 3.39	26.23 ± 3.49	23.93 ± 4.45	25.09 ± 4.46
Hoehn & Yahr stage (n)					H&Y I: 3 H&Y II: 7 H&Y III: 5
MDS-UPDRS III [0 - 132]					30.67 ± 13.33
EDSS [0 - 10]				3.85 ± 1.72
SPPB [0 - 12]						7.73 ± 3.10
CAT Score [0 - 40]			19.65 ± 8.95
FEV1/FVC			0.54 ± 0.12
6MWT distance (m)		348.81 ± 164.34	357.65 ± 88.52
KCCQ-12 Score [0-100]		82.39 ± 18.51

CAT: Chronic obstructive pulmonary disease (COPD) Assessment Test; EDSS: Expanded Disability Status Scale; FEV1: Forced Expiratory Volume in 1 Second; KCCQ-12: Kansas City Cardiomyopathy Questionnaire-12; MDS-UPDRS III: Movement Disorder Society Unified Parkinson’s Disease Rating Scale Part III; MoCA: Montreal Cognitive Assessment; SPPB: Short Physical Performance Battery; 6MWT: 6 Minute Walking Test; HA: healthy adults; PD: Parkinson’s disease; MS: multiple sclerosis; COPD: chronic obstructive pulmonary disease; CHF: congestive heart failure; PFF: proximal femoral fracture.

Table 2. Characterisation of relative and absolute errors, Intraclass correlation coefficients (ICC), Limits of agreement (LoA), for walking speed estimated from the true-positive walking bouts (WBs) from all Laboratory tasks combined and the real-world assessment. Values are either provided as mean and [5%, 95%] quantile or as mean and limit of agreement, if indicated by LoA.

Cohort	n-WBs [#]	Reference [m/s]	Wearable device [m/s]	Error with LOA [m/s]	Rel. Error with LOA [%]	Abs Error [m/s]	Rel. Abs. Error [%]	ICC
Laboratory
HA	103	0.86 [0.54, 1.15]	0.86 [0.54, 1.17]	-0.00 [-0.21, 0.20]	0.55 [-28.19, 29.28]	0.08 [0.01, 0.21]	10.34 [0.68, 29.22]	0.86 [0.80, 0.90]
CHF	60	0.94 [0.45, 1.52]	0.88 [0.56, 1.29]	-0.06 [-0.37, 0.26]	-2.06 [-40.71, 36.60]	0.12 [0.01, 0.30]	13.86 [1.72, 37.90]	0.82 [0.72, 0.89]
COPD	106	0.89 [0.59, 1.13]	0.90 [0.58, 1.14]	0.01 [-0.15, 0.17]	2.27 [-20.62, 25.16]	0.06 [0.01, 0.16]	7.82 [0.75, 19.35]	0.91 [0.87, 0.94]
MS	169	0.83 [0.45, 1.27]	0.85 [0.53, 1.18]	0.02 [-0.26, 0.30]	6.58 [-38.09, 51.25]	0.11 [0.01, 0.28]	15.87 [1.20, 48.71]	0.81 [0.75, 0.86]
PD	133	0.80 [0.47, 1.20]	0.82 [0.54, 1.15]	0.02 [-0.26, 0.31]	7.99 [-45.21, 61.19]	0.11 [0.01, 0.33]	17.01 [1.28, 45.27]	0.79 [0.72, 0.85]
PFF	121	0.70 [0.35, 1.34]	0.74 [0.44, 1.12]	0.04 [-0.26, 0.33]	14.43 [-50.98, 79.83]	0.12 [0.01, 0.32]	22.19 [1.19, 66.49]	0.82 [0.75, 0.87]
All	692	0.83 [0.43, 1.28]	0.84 [0.52, 1.18]	0.01 [-0.25, 0.28]	5.92 [-40.94, 52.77]	0.10 [0.01, 0.28]	14.96 [0.93, 46.15]	0.84 [0.81, 0.86]
Real-world
HA	364	0.72 [0.38, 1.22]	0.76 [0.47, 1.23]	0.04 [-0.18, 0.27]	10.53 [-35.92, 56.98]	0.09 [0.01, 0.26]	15.45 [0.83, 56.00]	0.88 [0.85, 0.90]
CHF	172	0.95 [0.46, 1.38]	0.92 [0.56, 1.33]	-0.04 [-0.35, 0.27]	1.29 [-48.12, 50.70]	0.12 [0.01, 0.32]	15.48 [1.24, 53.42]	0.82 [0.76, 0.86]
COPD	328	0.65 [0.36, 1.01]	0.76 [0.51, 1.06]	0.11 [-0.16, 0.38]	22.71 [-43.84, 89.26]	0.13 [0.01, 0.40]	24.88 [1.05, 91.11]	0.57 [0.49, 0.64]
MS	196	0.67 [0.40, 1.02]	0.76 [0.53, 1.06]	0.09 [-0.20, 0.39]	19.56 [-48.42, 87.53]	0.13 [0.01, 0.37]	23.53 [1.70, 80.64]	0.63 [0.54, 0.71]
PD	192	0.76 [0.37, 1.19]	0.84 [0.52, 1.20]	0.08 [-0.24, 0.40]	17.79 [-52.94, 88.53]	0.13 [0.01, 0.38]	22.99 [0.64, 94.50]	0.72 [0.64, 0.78]
PFF	162	0.66 [0.38, 1.04]	0.69 [0.45, 0.97]	0.04 [-0.25, 0.32]	10.63 [-50.73, 71.98]	0.11 [0.01, 0.33]	20.02 [1.24, 73.39]	0.66 [0.56, 0.74]
All	1414	0.72 [0.38, 1.21]	0.78 [0.49, 1.18]	0.06 [-0.23, 0.35]	14.48 [-47.17, 76.13]	0.11 [0.01, 0.36]	20.31 [0.95, 76.20]	0.77 [0.75, 0.79]

Table 3. Walking speed ranges and error analysis for the results combined per Laboratory test (laboratory) or over the entire real-world recording. Values are either provided as mean and [5%, 95%] quantile or as mean and limit of agreement, if indicated by the limits of agreement (LoA). The first column shows the number of datapoints (n-DPs) used to calculate the statistics.

Cohort	n-DPs [#]	Reference [m/s]	Wearable device [m/s]	Error with LoA [m/s]	Rel. Error with LoA [%]	Abs. Error [m/s]	Rel. Abs. Error [%]	ICC
Laboratory
HA	136	0.90 [0.56, 1.32]	0.85 [0.57, 1.15]	-0.05 [-0.34, 0.23]	-3.65 [-34.63, 27.33]	0.12 [0.01, 0.30]	12.72 [0.86, 32.94]	0.74 [0.66, 0.81]
CHF	58	0.92 [0.45, 1.52]	0.87 [0.52, 1.24]	-0.05 [-0.40, 0.30]	-0.54 [-41.45, 40.38]	0.13 [0.01, 0.38]	14.42 [0.90, 39.66]	0.80 [0.68, 0.87]
COPD	109	0.88 [0.47, 1.23]	0.87 [0.55, 1.13]	-0.01 [-0.34, 0.31]	1.51 [-32.13, 35.16]	0.10 [0.01, 0.27]	11.65 [0.60, 35.76]	0.72 [0.61, 0.80]
MS	129	0.81 [0.43, 1.29]	0.84 [0.52, 1.23]	0.03 [-0.26, 0.33]	9.05 [-38.46, 56.55]	0.11 [0.01, 0.28]	16.92 [0.74, 60.84]	0.81 [0.74, 0.86]
PD	132	0.79 [0.42, 1.27]	0.82 [0.53, 1.20]	0.03 [-0.25, 0.32]	9.10 [-38.67, 56.86]	0.11 [0.01, 0.31]	17.25 [1.35, 52.02]	0.82 [0.76, 0.87]
PFF	119	0.65 [0.23, 1.24]	0.72 [0.43, 1.04]	0.07 [-0.24, 0.37]	26.05 [-60.69, 112.78]	0.14 [0.01, 0.32]	32.57 [1.84, 124.21]	0.80 [0.73, 0.86]
All	683	0.81 [0.40, 1.31]	0.82 [0.51, 1.18]	0.01 [-0.31, 0.32]	7.47 [-47.71, 62.66]	0.12 [0.01, 0.30]	17.82 [0.75, 59.08]	0.80 [0.77, 0.83]
Real-world
HA	17	0.56 [0.44, 0.76]	0.61 [0.51, 0.71]	0.05 [-0.13, 0.24]	11.89 [-21.95, 45.74]	0.09 [0.03, 0.16]	17.25 [3.67, 32.29]	0.41 [-0.06, 0.73]
CHF	9	0.66 [0.42, 0.84]	0.82 [0.66, 1.04]	0.17 [-0.02, 0.35]	29.62 [-12.61, 71.86]	0.17 [0.02, 0.26]	29.80 [2.04, 61.60]	0.45 [-0.21, 0.84]
COPD	17	0.59 [0.47, 0.72]	0.66 [0.52, 0.78]	0.07 [-0.07, 0.21]	12.67 [-14.18, 39.52]	0.08 [0.01, 0.17]	14.12 [1.98, 32.21]	0.43 [-0.04, 0.75]
MS	13	0.57 [0.40, 0.74]	0.71 [0.57, 0.83]	0.13 [-0.15, 0.42]	28.91 [-34.27, 92.09]	0.16 [0.00, 0.31]	32.23 [0.28, 74.43]	-0.15 [-0.62, 0.41]
PD	15	0.54 [0.28, 0.77]	0.71 [0.51, 0.86]	0.17 [-0.07, 0.41]	44.06 [-45.40, 133.51]	0.17 [0.01, 0.36]	44.27 [1.77, 124.63]	0.33 [-0.19, 0.71]
PFF	11	0.52 [0.41, 0.64]	0.64 [0.53, 0.77]	0.13 [-0.01, 0.26]	26.03 [-11.30, 63.35]	0.13 [0.06, 0.25]	26.03 [11.65, 58.40]	0.04 [-0.53, 0.60]
All	82	0.57 [0.35, 0.82]	0.68 [0.52, 0.84]	0.11 [-0.10, 0.33]	24.48 [-32.34, 81.30]	0.13 [0.01, 0.30]	26.47 [1.62, 71.34]	0.33 [0.12, 0.51]

Table 4. Dependency on complexity for a selection of the gait tasks. The results are shown for each cohort, with limits of agreement (LoA). The “All” represents the statistics over all walking bouts (WBs) independent of the cohort.

Cohort	Error with LOA [m/s]	Rel. Error with LOA [%]	Abs Error [m/s]	Rel. Abs. Error [%]
All Straight Walks (Low Complexity)
HA	-0.03 [-0.18, 0.11]	-5.88 [-30.92, 19.16]	0.07 [0.01, 0.13]	10.83 [1.47, 24.03]
CHF	-0.12 [-0.44, 0.19]	-11.28 [-33.35, 10.78]	0.14 [0.03, 0.31]	12.99 [2.90, 29.27]
COPD	-0.03 [-0.13, 0.07]	-3.41 [-17.78, 10.95]	0.04 [0.00, 0.09]	5.80 [0.38, 14.00]
MS	-0.02 [-0.29, 0.25]	0.67 [-38.85, 40.18]	0.10 [0.02, 0.25]	14.40 [1.31, 37.39]
PD	-0.04 [-0.34, 0.26]	0.50 [-34.73, 35.72]	0.12 [0.01, 0.34]	14.86 [1.64, 33.08]
PFF	-0.03 [-0.30, 0.25]	3.38 [-40.07, 46.82]	0.11 [0.01, 0.32]	15.29 [1.19, 55.27]
All	-0.04 [-0.31, 0.23]	-0.87 [-37.34, 35.61]	0.10 [0.01, 0.33]	13.66 [1.16, 36.71]
Straight Walk Slow (Low Complexity)
HA	-0.03 [-0.18, 0.11]	-5.88 [-30.92, 19.16]	0.07 [0.01, 0.13]	10.83 [1.47, 24.03]
CHF	-0.10 [-0.26, 0.06]	-12.64 [-31.78, 6.50]	0.10 [0.02, 0.22]	13.02 [2.34, 26.89]
COPD	-0.02 [-0.11, 0.08]	-2.89 [-17.59, 11.82]	0.04 [0.00, 0.09]	5.51 [0.36, 14.06]
MS	0.02 [-0.18, 0.23]	6.15 [-38.39, 50.69]	0.08 [0.03, 0.22]	16.58 [4.01, 51.47]
PD	0.01 [-0.18, 0.19]	4.42 [-28.15, 37.00]	0.08 [0.01, 0.15]	13.55 [1.72, 35.43]
PFF	0.05 [-0.12, 0.22]	14.55 [-33.80, 62.91]	0.08 [0.01, 0.19]	19.00 [1.13, 66.73]
All	0.00 [-0.18, 0.19]	3.14 [-36.01, 42.28]	0.08 [0.01, 0.19]	14.14 [1.01, 40.68]
Simulated Daily Activities (High Complexity)
HA	0.09 [-0.18, 0.36]	16.54 [-29.66, 62.73]	0.12 [0.03, 0.30]	20.54 [3.77, 51.48]
CHF	0.13 [-0.12, 0.38]	29.99 [-28.01, 88.00]	0.15 [0.03, 0.29]	32.24 [3.41, 64.94]
COPD	0.02 [-0.25, 0.30]	15.23 [-48.13, 78.59]	0.11 [0.03, 0.22]	24.87 [4.06, 61.21]
MS	0.19 [-0.11, 0.49]	37.12 [-27.99, 102.24]	0.21 [0.06, 0.40]	39.93 [6.36, 96.34]
PD	0.16 [-0.16, 0.49]	37.65 [-56.98, 132.29]	0.18 [0.00, 0.36]	39.55 [0.47, 149.25]
PFF	0.18 [0.03, 0.33]	40.29 [-3.19, 83.78]	0.18 [0.10, 0.30]	40.29 [16.66, 77.12]
All	0.15 [-0.14, 0.43]	32.39 [-35.64, 100.43]	0.17 [0.02, 0.37]	35.14 [2.42, 82.42]
All Real World WBs
HA	0.04 [-0.18, 0.27]	10.53 [-35.92, 56.98]	0.09 [0.01, 0.26]	15.45 [0.83, 56.00]
CHF	-0.04 [-0.35, 0.27]	1.29 [-48.12, 50.70]	0.12 [0.01, 0.32]	15.48 [1.24, 53.42]
COPD	0.11 [-0.16, 0.38]	22.71 [-43.84, 89.26]	0.13 [0.01, 0.40]	24.88 [1.05, 91.11]
MS	0.09 [-0.20, 0.39]	19.56 [-48.42, 87.53]	0.13 [0.01, 0.37]	23.53 [1.70, 80.64]
PD	0.08 [-0.24, 0.40]	17.79 [-52.94, 88.53]	0.13 [0.01, 0.38]	22.99 [0.64, 94.50]
PFF	0.04 [-0.25, 0.32]	10.63 [-50.73, 71.98]	0.11 [0.01, 0.33]	20.02 [1.24, 73.39]
All	0.06 [-0.23, 0.35]	14.48 [-47.17, 76.13]	0.11 [0.01, 0.36]	20.31 [0.95, 76.20]

Competing interest reported. A. Mueller and F. Kluge are employees of, and may hold stock in, Novartis. B. Eskofier reports consulting activities with adidas AG, Siemens AG, Siemens Healthineers AG, WSAudiology GmbH outside of the study. He is a shareholder in Portabiles HealthCare Technologies GmbH. In addition, B. Eskofier holds a patent related to gait assessment. H. Sillén is an employee of, and may hold stock in, AstraZeneca. L. Palmerini and L. Chiari are co-founders and own shares of mHealth Technologies (https://mhealthtechnologies.it/). L. Schwickert and C. Becker are consultants of Philipps Healthcare, Bosch Healthcare, Eli Lilly, Gait-up. M. Niessen is an employee of McRoberts. Jeff Hausdorff reports having submitted a patent for assessment of mobility using wearable sensors in 400 Parkinson’s disease. The intellectual property rights 401 are held by the Tel Aviv Medical Center. S. Del Din reports consultancy activity with Hoffmann-La Roche Ltd. outside of this study. The other authors have no competing interests

Supplementarymaterial.docx

Estimating real-world walking speed from a single wearable device: analytical pipeline, results and lessons learnt from the Mobilise-D technical validation study

Status:

Journal Publication

Version 1

Abstract

Background:

Methods:

Results:

Conclusions:

Trial registration

Figures

Introduction

Results

Discussion

Overall Validation Results

Recommendation for real-world DMO validation

Validation protocol

Data analysis

Practical recommendations for the use of wearable devices for real-world walking speed measurements

Influence of pipeline

Walking bout duration

Walking speed

Real-world complexity

Limitations

Conclusion

Methods

Declarations

References

Tables

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1