A) Descriptive Analysis on Number of Research articles published on digital health
Our search strategy yielded a total of 563 articles on Medline, with the primary screening of article title and abstract 186 articles were found to be satisfying eligibility criteria. From 2016 to 2020 the number of research articles published on digital health has been steadily increasing and the trend over the past five years is visualized in Figure1.
When the studies were stratified based on study design, Review Articles and Guidelines made up to 65% of all the studies published on digital in past five years with a maximum number of review articles published in 2020. Figure 2 visualizes the study design and year was a trend. The year on year trend presented an average of 10% increase in Review Articles, 5% increase in Letter to Editors and Correspondence,4% in Diagnostic studies and 2-3% increase in Experimental Studies, Cohort Studies, Pilot and Development studies on digital health between 2016 to 2021.
The studies were stratified based on diseases and conditions for which digital initiative was primarily targeted, across the 5 years’ timeline. Health Systems Strengthening which included initiatives such as capacity building amongst healthcare staff, geo-spatial analysis for improving access to healthcare services, using data science to improve the availability of essential medicines, was the area wherein the highest digital health initiatives were focused. Ophthalmic disorders and COVID-19 was the second and third conditions to be frequently researched. The year on year trend analysis of conditions wherein digital health initiatives were primarily targeted, Ophthalmic Disorders and Health Systems Strengthening recorded <20% increase, Cancer Screening, Cardiovascular Disorders, Mental Health recorder 4-6% increase on an average from 2016-21.
Negative year on year trend was seen for Dental Sciences, Dermatology, Drug safety, Elderly Care, HIV prevention and control, Injuries and Accidents, Kidney and Urinary tract disorders, Leprosy Management, Neurological Conditions, Nutrition, Orthotic Disorders, Sexual and Reproductive Health, Stroke, Tobacco and Alcohol Cessation, Tuberculosis Prevention and Control, Vaccination, Vector-Borne Disease Prevention and Control
The study used the FITT framework to classify digital tools on tasks, the broad classification was Health Information Exchange, Data Science, and Surveillance. Further classification and definition with examples of each digital tool is presented in Table 2
Table 2 - Types of digital tools
Task wise classification
|
Sub Classification
|
Definition
|
Example
|
Health Information Exchange (HIE)
|
Data Sharing
|
Any tool a platform has to support data sharing between heterogeneous computer systems of different organizations(15)
|
Health Information and management system
|
Information delivery
|
A tool used for delivery of information (preventive, curative) to the end-users
|
SMS’s sent for a reminder of upcoming health visit,
|
Remote Consultation
|
Consultation by remote telecommunications, generally for diagnosis or treatment of a patient at a site remote from the patient or primary physician(16)
|
Consultation using Skype, Zoom
|
Intelligent Diagnosis
|
Intelligent Diagnosis systems are capable of identifying the nature of a problem by examining the observed symptoms and possibly an explanation or justify the same(17)
|
Algorithm-based diagnosis of risk of developing Diabetes
|
Data Science
|
Patient-Generated Health Data
|
Health-related data created, gathered or inferred by or from patients and for which the patient controls data collection and data sharing(18)
|
Digital Diary
|
Predictive Analytics
|
Predictive analytics is a branch of advanced analytics that makes predictions about future outcomes using historical data combined with statistical modelling, data mining techniques and machine learning(19)
|
Predictions on chances of raining
|
Clinical Decision Support System
|
Tools designed to be direct aid to clinical decision making, in which the characteristics of an individual patient are matched to a computerized clinical knowledge base and patient-specific assessments or recommendations are then presented to the clinician for a decision(20)
|
Image-based computer-assisted screening of oral lesions for cancer screening
|
Big Data Mining
|
Big data analytics covers the integration of heterogeneous data, data quality control, analysis, modelling, interpretation and validation(21)
|
Using Air Pollution data to develop Air Quality Index
|
Surveillance
|
Risk Screening
|
Continuous risk assessment of a condition, or population through multiple screening surveys(22)
|
NA
|
Realtime data collection and visualisation
|
Ongoing collection of data and presentation of data in a pictorial or graphical format (23)
|
COVID-19 dashboards
|
Contact Tracing
|
Digitalization of identification and follow-up of persons who may have come into contact with a person infected with the infectious diseases(24)
|
NA
|
The studies on remote consultation were most commonly reported, followed by information delivery systems and clinical decision support systems as visualized in Figure 4.
Cross-tabulation of Conditions and type of digital tool used is visualized in Figure 5; Health Information Exchange has been the most common digital tool for the majority of conditions, while all the digital initiatives in Metabolic Disorders (Diabetes), Tobacco and Alcohol Cessation, HIV prevention and Control were found to of Health Information Exchange. Surveillance initiatives were commonly employed for Vector-Borne Diseases Control, Cardiovascular disorders, COVID-19 and Health Systems Strengthening. Data Science initiatives were common for Ophthalmic Disorders, Health Systems Strengthening.
The year on year trend analysis of digital tool used, presented an average annual increase in Health Information Exchange tools recorded 5% annual increase, under HIE, Remote consultation recorded a 9% average annual increase, followed by a 7% increase by Health Information Delivery. Data Science tools recorded a 0.5% annual increase, under DS tools only Clinical Decision Support Systems recorded a 2.5% average annual increase, whereas Big data Mining, Predictive Analysis and Patient-Generated Health Data tools recorded a negative year on year trend.
The Surveillance tools usage trend presented a 0.5% average annual increase for 2016-2021 with Real-time Data Collection and Visualization tools showing a 2% increase.
The Descriptive analysis highlights
- COVID-19 pandemic has positively impacted digital health research with a 40% increase in the number of studies report in 2020 when compared to 2019
- The three most common diseases and conditions wherein substantive digital health research has been focused on are Health Systems Strengthening, Ophthalmic Disorders, and COVID-19.
- Remote Consultation, Health Information delivery and Clinical Decision support systems are the top three commonly developed tools from 2016 -2021
The details of all the studies included, their classification based on year, diseases, digital tools are included in the supplementary material.
B) Assessment of Sustainability
Validation of Sustainability assessment tool
We use inter-rater operability as an indicator to measure the ease of replicability of the tool. An independent rater, assessed 30 randomly selected studies using a computer-generated list for the 87 articles assessed in section B of this study. Cohen’s Unweighted Kappa was used to determine the inter-rater operability. Overall Kappa value was 0.806 (SD ±0.088). The highest inter-rater operability was seen for Q1 (Does the study involve multisectoral team?) with Kappa value 1 (100%), lowest inter-rater operability was seen for Q3 (Does the study mention adherence to any standards of data components, data interchange, application-level support?) with Kappa value 0.54 (SD±0.036).
Kappa Value for other questions was
Q2 (Does the study involve engagement with sectors other than health?) – 0.91 (SD 0.066)
Q4 (Does the study mention stakeholder analysis/ community needs assessment/ with end-users for development of initiative?) – 0.84 (SD 0.042)
Q5 (Does the study mention scope of collecting feedback from the end-users?) – 0.71 (SD 0.06)
The interpretation of results was divided into Studies published pre COVID-19 pandemic (2016-2019) and during the COVID-19 pandemic (2020-2021). Analysis was reported for study design
Experimental Study Design
All the experimental studies pre-pandemic and during pandemic had an active involvement of IT, Data management team while developing the intervention and were either part of the team writing manuscript or dully acknowledged indicating the multisectoral team involved in developing and testing of the initiatives.
10 out of 18 experimental studies involved engagement of multiple sectors, for example, Swendenmen et, al (25) included behavioural scientist, HIV care providers, Front line health workers for the implementation of the study.
All the experimental studies mentioned adherence to data standards like WHO or ICDS classification of diseases, however, adherence to data interchange standards like HL7, was not mentioned.
All the experimental studies have either conducted a gap analysis or referred to previously published authors papers on gap analysis, community needs assessment for the development of initiatives
All the experimental studies have mentioned feedback collection from end-users, delivery providers and have mentioned changes made in digital initiative upon receiving feedback.
Overall the average sustainability of experimental studies on digital health was 80%, and there was no statistically significant difference in overall sustainability score between the studies published Pre-pandemic (85.6%) and During Pandemic (76.4%) (p-value -0.33).
Table 3 and Figure 6 presents the study wise assessment score summary and percentage of sustainability scores based on the author's judgement.
Table 3 - Assessment of Sustainability of Experimental Studies
|
Author
|
Does the study involve a multisectoral team?
|
Does the study involve engagement with sectors other than health?
|
Does the study mention adherence to any standards of data components, data interchange, application-level support?
|
Does the study mention stakeholder analysis/ community needs assessment/ with end-users for the development of the initiative?
|
Does the study mention scope of collecting feedback from the end-users?
|
During COVID-19 pandemic
|
Shekhawat et.al 2020(26)
|
Yes
|
No
|
Yes
|
Yes
|
Yes
|
Swendeman et.al 2020(25)
|
Yes
|
Yes
|
Yes
|
Yes
|
Yes
|
Johri et.al 2020(27)
|
Yes
|
No
|
Yes
|
Yes
|
Yes
|
Suryavanshi et.al 2020(28)
|
Yes
|
Can't Say
|
Yes
|
Yes
|
Yes
|
Nandita et.al 2020(29)
|
Yes
|
Yes
|
Yes
|
Yes
|
Yes
|
Pre COVID-19 Pandemic
|
Modi et.al 2019(30)
|
Yes
|
No
|
Yes
|
Yes
|
Can't Say
|
Zhilian et.al 2019(31)
|
Yes
|
Yes
|
Yes
|
Yes
|
Yes
|
Joshep et.al 2019(32)
|
Yes
|
Yes
|
Can't Say
|
Yes
|
Yes
|
Saran et.al 2019(33)
|
Yes
|
Yes
|
Yes
|
Yes
|
Yes
|
Gross et.al 2019(34)
|
Yes
|
Can't Say
|
Yes
|
Yes
|
Yes
|
Jiang et.al 2019(35)
|
Yes
|
Yes
|
Yes
|
Yes
|
Yes
|
Peiris et.al 2019(36)
|
Yes
|
Yes
|
Yes
|
Yes
|
Yes
|
Roohippor et.al 2019(37)
|
Yes
|
No
|
Yes
|
Yes
|
Yes
|
Prabhakaran et.al 2018(38)
|
Yes
|
No
|
Can't Say
|
Yes
|
Yes
|
Zhang et.al 2017(39)
|
Yes
|
Yes
|
Can't Say
|
Yes
|
Yes
|
Ajay et.al 2016(40)
|
Yes
|
No
|
Yes
|
Yes
|
Yes
|
Anand et.al 2016(41)
|
Yes
|
Yes
|
Yes
|
Yes
|
Yes
|
Sharma et.al 2016(42)
|
Yes
|
Yes
|
Yes
|
Yes
|
Yes
|
Cohort Study Design
80% of the Cohort studies had active involvement of IT, Data management team while developing the intervention and were either part of the team writing manuscript or dully acknowledged indicating the multisectoral team involved in developing and testing of the initiatives. The pattern of engagement with multisectoral was similar in studies reported before the COVID-19 pandemic and during the COVID-19 pandemic. However, in all the cohort studies assessed, engagement of multiple sectors was not seen amongst 80% of studies (needs rephrasing).
65% of Cohort studies reported adherence to WHO/ ICDS standards for the classification of diseases, however, studies have not mentioned adherence to data interchange standards like HL7.
All the cohort studies assessed have mentioned gap analysis and needs assessment for the development of the initiative. About 60% mentioned feedback collection from end-users, delivery providers and have mentioned changes made in digital initiative upon receiving feedback.
Overall the average sustainability of Cohort studies on digital health was 40%, and there was no statistically significant difference in overall sustainability score (how has this score been calculated and what is the validity of the tool may be to consider reporting in the form of yes and no) between the studies published Pre-pandemic (25.3%) and During Pandemic (35.6%) (p-value 0.45).
Table 4 and Figure 7 presents the study wise assessment score summary and percentage of sustainability scores based on the authors' judgement.
Table 4 - Assessment of Sustainability, Cohort studies
|
Author
|
Does the study involve a multisectoral team?
|
Does the study involve engagement with sectors other than health?
|
Does the study mention adherence to any standards of data components, data interchange, application-level support?
|
Does the study mention stakeholder analysis/ community needs assessment/ with end-users for the development of the initiative?
|
Does the study mention scope of collecting feedback from the end-users?
|
During COVID-19 pandemic
|
Saw et.al 2020(43)
|
Yes
|
Can't Say
|
No
|
Yes
|
Yes
|
Baroutsout et.al 2020(44)
|
Yes
|
No
|
Can't Say
|
Yes
|
Yes
|
Tyler et.al 2020(45)
|
Yes
|
No
|
Yes
|
Yes
|
Can't Say
|
Garg et.al 2020(46)
|
Yes
|
No
|
Yes
|
Yes
|
Yes
|
Mahadevan et.al 2020(47)
|
No
|
No
|
Yes
|
Yes
|
No
|
Rachmani et.al 2020(48)
|
Yes
|
No
|
Yes
|
Yes
|
Yes
|
Pre COVID-19 pandemic
|
Shah et.al 2019(49)
|
Yes
|
Yes
|
Can't Say
|
Yes
|
Yes
|
Farnham et.al 2017(50)
|
Yes
|
No
|
Yes
|
Yes
|
Yes
|
Balakrishnan et.al 2016 (17)
|
No
|
No
|
Yes
|
Yes
|
Yes
|
Cross-Sectional Study
25% of the Cross-Sectional studies had active involvement of IT, Data management team while developing the intervention and were either part of the team writing manuscript or dully acknowledged indicating the multisectoral team involved in developing and testing of the initiatives. The percentage of active involvement of multisectoral team was higher in cross-sectional studies reported during COVID-19 as compared to Pre COVID-19 reported studies. However, amongst 80% of cross-sectional studies assessed, engagement of multiple sectors was not seen.
90% of cross-sectional studies reported during the COVID-19 pandemic reported adherence to WHO/ ICDS standards for the classification of diseases, whereas only 10% of cross-sectional studies reported pre COVID-19 reported adherence to WHO/ ICDS standards for the classification of diseases.
30% of cross-sectional studies have mentioned gap analysis, needs assessment for the development of the initiative. About 40% of studies have mentioned feedback collection from end-users, delivery providers and have mentioned changes made in digital initiative upon receiving feedback.
Overall the average sustainability of Cross-Sectional studies on digital health was 40%, and there was a statistically significant difference in overall sustainability score between the studies published Pre-pandemic (45.3%) and During Pandemic (27.7%) (p-value 0.002).
Table 5 and Figure 8 presents the study wise assessment score summary and percentage of sustainability scores based on the authors' judgement.
Table 5 -- Assessment of Sustainability, Cross-Sectional Study
|
Author
|
Does the study involve a multisectoral team?
|
Does the study involve engagement with sectors other than health?
|
Does the study mention adherence to any standards of data components, data interchange, application-level support?
|
Does the study mention stakeholder analysis/ community needs assessment/ with end-users for the development of the initiative?
|
Does the study mention scope of collecting feedback from the end-users?
|
During COVID-19 pandemic
|
Ravindran et.al 2021(51)
|
No
|
No
|
Yes
|
Yes
|
Yes
|
Ward et.al 2020(52)
|
Yes
|
Yes
|
Yes
|
Yes
|
Yes
|
Huang et.al 2020(53)
|
No
|
No
|
Yes
|
No
|
Yes
|
Singh et.al 2020(54)
|
No
|
No
|
Yes
|
No
|
Yes
|
Vijayasundaram et.al 2020(55)
|
No
|
No
|
Yes
|
No
|
Yes
|
Xiong et.al 2020(56)
|
Yes
|
Yes
|
Yes
|
No
|
No
|
Das et.al 2020(57)
|
No
|
No
|
Yes
|
No
|
No
|
Shenoy et.al 2020(58)
|
No
|
No
|
No
|
Yes
|
Yes
|
Charumathi et.al 2020(59)
|
Yes
|
No
|
Yes
|
No
|
No
|
Shreshta et.al 2020(60)
|
No
|
No
|
Yes
|
No
|
No
|
Chuenphitthayavut et.al 2020(61)
|
Yes
|
No
|
Yes
|
No
|
No
|
Pre COVID-19 pandemic
|
Kogan et.al 2019(62)
|
Yes
|
No
|
No
|
No
|
No
|
Dandge et.al 2019(63)
|
No
|
No
|
No
|
No
|
No
|
Soni et.al 2018(64)
|
No
|
N0
|
No
|
No
|
No
|
Bhatt et.al 2018(65)
|
No
|
No
|
No
|
Yes
|
Yes
|
Shah et.al 2018(49)
|
Yes
|
No
|
No
|
No
|
No
|
Chahar et.al 2018(66)
|
No
|
No
|
Yes
|
No
|
No
|
Birur et.al 2018(67)
|
No
|
No
|
No
|
Can’t Say
|
Yes
|
Lee et.al 2018(68)
|
Yes
|
No
|
No
|
No
|
Yes
|
Kazi et.al 2017(69)
|
Yes
|
No
|
No
|
Yes
|
No
|
Devasahay et.al 2017 (70)
|
Yes
|
No
|
No
|
No
|
No
|
Lan Hoang et.al 2016(71)
|
Can't Say
|
No
|
No
|
No
|
No
|
Diagnostic studies
70% of the Diagnostic studies had active involvement of IT, Data management team while developing the intervention and were either part of the team writing manuscript or dully acknowledged indicating the multisectoral team involved in developing and testing of the initiatives. The percentage of active involvement of multisectoral team was higher in diagnostic studies reported Pre COVID-19 pandemic as COVID-19 reported studies. However, amongst 95% of diagnostic studies assessed, engagement of multiple sectors was not seen.
60% of Diagnostic studies reported adherence to WHO/ ICDS standards for the classification of diseases, majority of studies reported during the COVID-19 pandemic were able to provide information on adherence to standards
25% of Diagnostic studies assessed have mentioned gap analysis, needs assessment for the development of the initiative. About 30% of studies have mentioned feedback collection from end-users, delivery providers and have mentioned changes made in digital initiative upon receiving feedback.
Overall the average sustainability of Diagnostic studies on digital health was 45%, and there was no statistically significant difference in overall sustainability score between the studies published Pre-pandemic (45%) and During Pandemic (55%) (p-value 0.5)
Table 6 and Figure 8 presents the study wise assessment score summary and percentage of sustainability scores based on the authors' judgement.
Table 6- Assessment of Sustainability, Diagnostic Study
|
Author
|
Does the study involve a multisectoral team?
|
Does the study involve engagement with sectors other than health?
|
Does the study mention adherence to any standards of data components, data interchange, application-level support?
|
Does the study mention stakeholder analysis/ community needs assessment/ with end-users for the development of the initiative?
|
Does the study mention scope of collecting feedback from the end-users?
|
During COVID-19 pandemic
|
Rajvanshi et.al 2021(72)
|
Yes
|
Yes
|
Yes
|
Can’t Say
|
Yes
|
Kannure et.al 2021(73)
|
No
|
No
|
No
|
No
|
No
|
Satgunam et.al 2020(74)
|
No
|
No
|
Yes
|
No
|
No
|
Tham et.al 2021(75)
|
Yes
|
No
|
Yes
|
No
|
No
|
Praveen Raj et.al 2020(76)
|
Yes
|
No
|
Yes
|
No
|
No
|
Bulten et.al 2020(77)
|
No
|
No
|
Yes
|
Yes
|
Yes
|
Dan Milea et.al 2020(78)
|
No
|
No
|
Yes
|
Yes
|
Yes
|
Mondal et.al 2020 (80)
|
No
|
No
|
Yes
|
No
|
Yes
|
Tahsin kurc et.al 2020 (79)
|
Yes
|
No
|
Yes
|
No
|
No
|
Pre COVID-19 pandemic
|
Shantharam et.al 2019(80)
|
Yes
|
No
|
No
|
Yes
|
Yes
|
Sumsum Sunny et.al 2019(82)
|
Yes
|
No
|
No
|
No
|
No
|
Muller et.al 2019(81)
|
Yes
|
No
|
Yes
|
No
|
No
|
Vorakulpipat et.al 2019(82)
|
Yes
|
No
|
No
|
No
|
No
|
Beane et.al 2019(83)
|
Yes
|
No
|
No
|
Yes
|
Yes
|
Ramkumar et.al 2018(84)
|
Yes
|
No
|
No
|
Yes
|
Yes
|
Kumar et.al 2018(85)
|
Yes
|
No
|
Yes
|
No
|
No
|
Koesoemadinata et.al 2018(86)
|
Yes
|
No
|
Yes
|
No
|
No
|
Kimberly M et.al 2018(87)
|
Yes
|
No
|
Yes
|
Yes
|
Yes
|
Maity et.al 2017(88)
|
Yes
|
No
|
Yes
|
No
|
No
|
Malhotra et.al 2017(90)
|
No
|
No
|
No
|
Yes
|
Yes
|
Pilot Studies and Development studies
We used the definition given by Stewart PW(89) for Pilot and Development studies as “small study to test research protocols, data collection instruments, sample recruitment strategies, and other research techniques in preparation for a larger study” however pilot randomized studies were included in experimental studies.
75% of the Pilot and Development studies had active involvement of IT, Data management team while developing the intervention and were either part of the team writing manuscript or dully acknowledged indicating the multisectoral team involved in developing and testing of the initiatives. The percentage of active involvement of multisectoral team was similar for studies report Pre COVID-19 pandemic and during COVID-19 reported studies. However, amongst 95% of diagnostic studies assessed, engagement of multiple sectors was not seen.
80% of Pilot and Development studies reported adherence to WHO/ ICDS standards for the classification of diseases.
50% of Pilot and Development studies assessed have mentioned gap analysis, needs assessment for the development of the initiative. Similarly, 50% of studies have mentioned feedback collection from end-users, delivery providers and have mentioned changes made in digital initiative upon receiving feedback.
Overall the average sustainability of Pilot and Development studies on digital health was 65%, and there was no statistically significant difference in overall sustainability score between the studies published Pre-pandemic (56%) and During Pandemic (63%) (p-value 0.28)
Table 7 and Figure 9 presents the study wise assessment score summary and percentage of sustainability scores based on the authors judgement
Table 7- Assessment of Sustainability, Pilot and Development studies
|
Author
|
Does the study involve a multisectoral team?
|
Does the study involve engagement with sectors other than health?
|
Does the study mention adherence to any standards of data components, data interchange, application-level support?
|
Does the study mention stakeholder analysis/ community needs assessment/ with end-users for the development of the initiative?
|
Does the study mention scope of collecting feedback from the end-users?
|
During COVID-19 pandemic
|
Bafna et..al 2020(90)
|
No
|
No
|
Yes
|
No
|
No
|
Nikita et.al 2020(91)
|
No
|
No
|
No
|
Yes
|
Yes
|
Hegde et.al 2020(92)
|
Yes
|
No
|
No
|
No
|
No
|
AnLee et.al 2020(93)
|
Yes
|
No
|
Yes
|
No
|
No
|
Misra et.al 2020(94)
|
Yes
|
No
|
Yes
|
Yes
|
Yes
|
Thornber et.al 2020(95)
|
Yes
|
No
|
No
|
Yes
|
Yes
|
Pre COVID-19 pandemic
|
Ayyanar et.al 2019(96)
|
Yes
|
No
|
Yes
|
No
|
No
|
Ahmed et.al 2019(97)
|
No
|
No
|
Yes
|
No
|
No
|
Devraj et.al 2019(98)
|
Yes
|
No
|
Yes
|
Yes
|
Yes
|
Drusbosky et.al 2019(99)
|
Yes
|
No
|
Yes
|
No
|
No
|
Jain et.al 2019(100)
|
Yes
|
No
|
No
|
Yes
|
Yes
|
Verma et.al 2018(101)
|
No
|
Yes
|
Yes
|
No
|
No
|
Rao et.al 2018 (102)
|
Yes
|
No
|
Yes
|
Yes
|
Yes
|
Aggarwal et.al 2018(103)
|
Yes
|
No
|
Yes
|
No
|
No
|
Devraj et.al 2018(104)
|
Yes
|
No
|
Yes
|
Yes
|
Yes
|
The assessment of sustainability analysis indicates
- Experimental studies and Cohort studies had incorporated factors contributing to sustainability and involvement of teams, sectors, feedback was reported across a majority of studies irrespective of Pre COVID-19 pandemic or During the COVID-19 pandemic
- Cross-Sectional studies, conducted during pandemic improved on parameters of assessment of sustainability and the difference between sustainability assessment Pre COVID-19 Pandemic and During COVID-19 pandemic was statistically significant
- Diagnostic Studies and Pilot and Development studies had incorporated limited factors contributing to sustainability irrespective of Pre COVID-19 pandemic or During the COVID-19 pandemic