Commentary: A systematic review of the characteristics and validity of monitoring technologies to assess Parkinson's disease

Domingos J^1,3, Godinho C^1,2,3, Ferreira JJ^1,3,4*

---

Abstract

Technologies may have implications for improving clinical diagnosis and prognosis, and for the development of therapeutic interventions, specific biomarkers, and preventive strategies. Given the amount of existing and ever-growing quantitative assessments using technology in PD, clinicians, patients and researchers are faced with the challenge of deciding which assessment tool to use in the laboratory, clinic and home environment. In order to facilitate this decision-making a systematic review was done to identify and classify the available monitoring technologies for individuals with PD over the last 2 decades. This is a commentary on the systematic review which adds on discussion on some controversial issues in the area. It tackles some of current open-to-discussion topics in the technology field, such as: which definitions to use, the heterogeneity of the clinimetric properties among technologies, standardization of a validation process, how to group different measuring technologies, and the need to conduct further studies on existing technologies before developing new ones. The strength of this comprehensive, timely and useful review is the detailed and robust approach taken by authors to classify technologies as listed, suggested, or recommended for the assessment of individuals with PD.

---

 

Commentary

The main challenges faced by the technology field include synthesizing the vast type of technologies being developed for use in Parkinson´s disease (PD), organizing the amount of raw data obtained and how to meet the needs of both clinicians and patients, and providing trends and highlights of a patient’s status.

The authors of this paper sought to provide answers to such challenges by performing a systematic review of the available monitoring technologies for individuals with PD over the last 2 decades. Given the complexity of the controversy issues in technology, the authors further discuss the challenging topics raised in the form of this commentary. These technologies may have implications for improving clinical diagnosis and prognosis, and for the development of therapeutic interventions, specific biomarkers, and preventive strategies. Given the amount of existing and ever-growing quantitative assessments using technology in PD, this paper will hopefully facilitate decision-making by clinicians, patients and researchers when deciding which assessment tool to use in the laboratory, clinic and home environment.

The authors classify technologies as listed, suggested, or recommended for the assessment of individuals with PD. These classifications are based on 3 criteria, namely: 1) used in the assessment of PD; 2) used by individuals other than the developers; and 3) have undergone successful clinimetric testing. The authors conclude with a summary of the recommended technologies that can objectively assess several PD signs (postural control, tremor, bradykinesia, freezing, dyskinesia, gait, and daily activity/physical activity) and comment on which other disease-related aspects need more attention such as disease progression markers and non-motor symptoms (cognition, sleep and dysphagia).

Controversial Issues: Clinimetric Data Validation

One of the controversial issues that are highlighted in this paper concerns the clinimetric data validation process of the technologies reviewed. The validity criteria of a system are an interesting but also complex concept. The authors argue that technologies have not undergone a formal validation process and should be assessed based on validity criteria similar to, or adapted from those used for clinical scales (for examples: Goetz, Tilley et al. 20081; Colosimo, Martinez-Martin et al. 20102; Elble, Bain et al. 20133). Given the current gap in knowledge on the proper clinimetric properties of devices applied as evaluation tools, the technology devices and systems are therefore evaluated against clinical application criteria. Recommendations were limited to the available data and specific to the motor disorder or sometimes specific metrics that where validated for that device.

Importantly, there are many types and degrees of validity. Combining all types of validity into a single yes/no binary answer may not accurately reflect the validity of a certain system. Unfortunately, limited data were available from primary sources, validation processes were dispersed, and while some focused on validating the algorithm, others focused on the metrics or on the device itself. On a positive note, the authors highlight that they only considered whether any kind of validity had been performed, and given the complexity of information in this area any validation was accepted in the yes/no binary.

Additionally, when analyzing any technology system, readers should be aware of what the "nature of the comparison-validation" was when assessing the system. For example, when comparing an accelerometer-based system to an instrumented mattress or to a three-dimensional motion capture system, it is important to ensure that systems measuring different signals were compared (acceleration vs. pressure vs. position). This has been highlighted in this review and was believed not to impact classification of the technology instruments, as the aim of the review was to report only the existing published validation data. However, another possible method to validate technology could be the COSMIN framework of measurement properties which provides taxonomy and definitions of which measurement properties are important for outcome measures4. Within this framework, a checklist was also developed containing standards for assessing the methodological quality of studies on measurement properties for patient-reported outcome measures5 and can ultimately be adapted and used in a systematic review of technologies.

Importantly, in order to truly establish a fair comparison across different technologies used in PD and reflect the state-of-the-art of that technology, we will need an in-depth review of important aspects of existing technology devices, such as sensor types and positions, data collection protocols, subject groups, feature extractors, classification methods and performance measures.

Controversial Issues: Difficulties in Standard Terminology and Grouping Technologies

One of the bottlenecks identified in the review refers to the use of terminology in the field of technologies and how the different technologies are grouped or classified. Reflecting on how technologies, technological instruments, or technology-based devices are defined currently and how this has changed over the years, might be of interest. This lack of standardization could represent a serious methodological pitfall for the evaluation and interpretation of assessment and intervention strategies using such technologies. This may not be as intuitive as it seems. For example, could a system that is based on a sensor attached on the body, yet requiring constant communication with a short range fixed-base station be considered wearable? Coming up with a definition for a wearable system might not be straight forward but it is notable that some effort was made to better clarify the use of terminologies and definitions. As such, an effort was made to positively contribute to this standardization of definitions. Wearable devices were defined as electronic technology or computers designed to be worn on the body, or embedded into watches, bracelets, clothing, and others. Less known terms such as hybrid were also carefully defined as the blend of technologies that combined wearable and non-wearable devices.

Grouping these technologies is also a challenge. Some previous studies have focused on specific motor and non-motor deficits in PD and review which wearables were used to assess them (problem-based structure)6. Other option for classifications may focus on specific construct and review which technology/ instrument can be used for measuring a specific construct (construct-based structure). Yet, the challenge of classifying technologies may be because technologies can measure multiple disease related deficits or constructs at the same time and this may ultimately complicate the assessment of reliability and validity.

In this review, a device-based structure was used focusing on which devices exist and for what use. This could ultimately transmit the image of a simple inventory of the available systems and limit the clinical applicability because PD symptoms are not intuitively highlighted or it is not totally clear which device covers which motor disorder and at what level. A device might have only been assessed for example for sway but not for tremor. The supplementary table or the main manuscript would benefit from an additional problem-based approach where a reader who is interested only in freezing of gait can identify the devices that are better recommended for this purpose. Such a problem-based structured divided between motor, non-motor problems and disease-related complications may ultimately better guide clinicians in their decisions. Importantly, care needs to be taken to avoid endless repetition of data because many devices can be used for multiple purposes (e.g., sway, gait, or tremor) but may meet criteria for only a certain subset.

Recommendations for Future Studies and need for Collaboration

Overall, this paper provides an overview of the application of clinical methodological criteria applied in other reviews of scales and assessment tools in PD, and currently available technology devices. Regardless of the limitations of the available data, some useful and clinically relevant recommendations regarding which devices should be prioritized in the clinic and in research could be made using these criteria. This could help set the grounds for thinking about developing/defining a core outcome set of measurement instruments to be used in research in PD7. Technology development is currently based on competitive and individual initiatives allowing for duplication of efforts instead of collaborative work to develop complementary technologies. Additionally, technologies may be driven by ease of development as opposed to clinical need for a technical solution. Such developments will also soon face the need for establishing business models to cover the costs and regulatory validation of their efficacy and safety.

Any new development should keep in mind some key requirements such as reliability, low cost, small size, low power consumption, and clinical relevance. Technology applications relying on mobile phones may have additional value because smart phones are broadly available and make system use and maintenance quick and cost-effective. Yet, one must keep in mind that each new brand of a phone that is developed can be considered a "new technology device”, given it has different gyro/acceleration, barometer or GPS location systems, and thus can potentially give different metrics. As such, larger sample studies using phones will need to consider a starting calibration setup, allowing for a cross validation model. Additionally, the placement of the phone is another aspect that needs to be considered. There is evidence suggesting the waist as the optimal placement given its proximity to the centre of mass8. Yet the common place people use phones would be in the bags of pockets. An additional thought of a band of the wrist (smartwatch) can be ultimately an alternative solutions, given its cost effectiveness, usability in both indoor and outdoor environments, and ability to track a patient using the GPS module in the phone that can ultimately be useful for fall events by allow timely assistance and intervention. So, alongside using technology for making assessments, it is worth focusing on how to move forward to developing devices for safety monitoring, such as detecting falls with advanced signal processing techniques to achieve high accuracy of falls detection with alarm messages to an emergency response system. Another clinical situation where technology applications may deserve particular interest is monitoring PD patients with cognitive impairment and building systems that could assist these patients with remembering daily activities, tracking medication compliance and monitoring daily behavior for early signs of deterioration, allowing such patients to live independently for longer.

Despite the tremendous progress in this field and the development of numerous new technologies, we emphasize the need for further studies on the existing recommended technologies as opposed to developing new technology devices. However, some technological systems have been used for many years in different settings, diseases and teams from different geographies. So different needs such as wanting to clinically validate a new biometric outcome to be used on a collaborative study on monitoring a particular clinical issue of PD might justify the merge old with new technologies, the development of new proximate devices, and standardized methods of clinical evaluation and data extraction. As such, present and future technologies will probably always have a very important scientific role. We thus await, with interest, future results for these and/or new recommended technologies used in clinical studies as standardized measures of disease progression, allowing for comparison between trials and representing more accurate measures of clinically efficacy.

References

1. Goetz CG, Tilley BC, Shaftman SR, Stebbins GT, Fahn S, Martinez-Martin P, et al., Movement Disorder Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS): scale presentation and clinimetric testing results. Mov Disord. 2008; 23: 2129-70. Doi: 10.1002/mds.22340.
2. Colosimo C, Martinez-Martin P, Fabbrini G, Hauser RA, Merello M, Miyasaki J, et al. Task force report on scales to assess dyskinesia in Parkinson's disease: critique and recommendations. Mov Disord. 2010; 25: 1131-42. Doi: 10.1002/mds.23072.
3. Elble R, Bain P, Forjaz MJ, Haubenberger D, Testa C, Goetz CG, et al. Task force report: scales for screening and evaluating tremor: critique and recommendations. Mov Disord. 2013; 28: 1793-800. Doi: 10.1002/mds.25648.
4. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010; 63: 737-45. Doi: 10.1016/j.jclinepi.2010.02.006.
5. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010; 19: 539-49. Doi: 10.1007/s11136-010-9606-8.
6. Maetzler W, Domingos J, Srulijes K, Ferreira JJ, Bloem BR. Quantitative wearable sensors for objective assessment of Parkinson's disease. Mov Disord. 2013; 28: 1628-37. Doi: 10.1002/mds.25628.
7. Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012; 13: 1-8. Doi: 10.1186/1745-6215-13-132.
8. Pannurat N, Thiemjarus S, Nantajeewarawat E. Automatic Fall Monitoring: A Review. Sensors. 2014; 14: 12900.