The Diagnostic Accuracy of Skin Diseases Via Teledermatology – A Single Center Study

doi:10.21203/rs.3.rs-4968038/v1

Download PDF

Article

The Diagnostic Accuracy of Skin Diseases Via Teledermatology – A Single Center Study

https://doi.org/10.21203/rs.3.rs-4968038/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background:

Teledermatology emerged as a safe tool that could fill healthcare gaps in underprivileged remote areas.

Objective:

To compare the diagnostic accuracy of teledermatologists with in-clinic dermatologists and assess various methods of photo capturing skin lesions, for better incorporation of teledermatology into clinical practice.

Methods

A single center randomized prospective trial at the American University of Beirut Medical Center involved adult participants seeking dermatologic consultation. Three sets of photographs were taken: patient-taken without guidance/unassisted, patient-taken with guidance/assisted, and resident-taken.

Results:

Diagnosis match with in-clinic visits increased from patient-unassisted (79%) to patient-assisted (84%) to resident-taken photographs (87%). Similarly, treatment match increased among the 3 modes respectively (38%,44% and 45%) and need for history decreased among the 3 modes respectively (64%, 59%, and 47%). Acne cases had higher diagnosis matches than malignant tumors and papulosquamous dermatoses among the three modes.

Conclusion:

A limitation of this study was the reliance on in-clinic expert opinions as “true diagnosis or treatment” for comparison purposes. Teledermatology’s accuracy in diagnosing common dermatoses, coupled with proper patient education on picture taking holds promise in its integration into clinical practice.

Health sciences/Medical research

Health sciences/Medical research/Outcomes research

Teledermatology

diagnosis

skin diseases

Dermatology specialty primarily relies on visual inspection to make a diagnosis ¹. However, in many cases, additional information such as the history of present illness, past medical history, physical exam and confirmatory testing such as blood tests or skin biopsies are still required. Many dermatological consultations are now conducted online, owing to the advancement of telecommunication portals. After the unprecedented COVID-19 pandemic, the importance of teledermatology has never been clearer, as it was a safe and effective alternative for self-quarantine and social distancing.

Teledermatology is defined as the practice of dermatology at a distance ². Consults come in two main forms. The first one is asynchronous or “store and forward” which allows clinicians to take digital images of skin lesions and send them to a dermatologist to review them later. The second one is synchronous or “live and interactive consultations” which allows for real-time videoconferencing with the patient ³.

While face-to-face initial encounters remain the gold standard for correct diagnosis, it is relevant to consider teledermatology as a way to transcend medical barriers. According to a systematic review, teledermatology has shown itself as a useful and convenient tool for the management of common ambulatory dermatoses like eczema or acne in the COVID-19 pandemic ². Additionally, in countries where numerous remote areas lack easy access to healthcare, teledermatology could serve as a solution to bridge this healthcare gap.

The goal of this study is to evaluate dermatologists’ diagnostic accuracy of skin problems using smartphone photographs. Most of the evidence on the diagnostic accuracy of teledermatology comes from clinicians who took photographs themselves. However, in real-world scenarios, patients may not have access to a general practitioner and may be required to take photographs themselves without instructions on proper techniques. Our plan was to assess the accuracy of teledermatologists’ diagnoses using photographs taken through a “store and forward” method and presented to them later. We then compared the diagnostic accuracy of teledermatologists with that of dermatologists who saw the patient in clinic. This comparative analysis will consider the impact of various approaches to capturing skin lesion photographs, including those taken by patients themselves, those taken by patients after training and those taken by a professional dermatology resident. Our objective is to translate these findings into practical clinical applications. By investigating the diagnostic accordance between dermatologists and clinic-based dermatologists and evaluating the most effective methods of photo-capturing, we aim to provide valuable insights into the integration of teledermatology into clinical settings.

Our study is a single-center prospective trial that was reviewed and approved by the Institutional Review Board (IRB) at the American University of Beirut (AUB) under approval number BIO-2020-0241. The research was conducted at the American University of Beirut Medical Center in Beirut, Lebanon, and all procedures were carried out in accordance with relevant guidelines and regulations. Participants, aged 18 or above, seeking initial dermatology consultation, were recruited from private clinics and the outpatient department. Exclusion criteria include patients on treatment for over a week and cases where in-clinic dermatologists disagree on the diagnosis.

All eligible patients were consented privately. After initial diagnosis, patients underwent a separate examination by another dermatologist and a resident for verification. The documented in-clinic diagnosis by all three physicians was considered the "correct diagnosis."

First, patients were asked to take pictures of their lesions using their smartphones, with lighting conditions, angles and instructions not standardized. Then another photo is taken after they receive training on proper photography techniques (Supplementary form 4). It was provided immediately in the clinic, highlighting the importance of showing the exact location of the lesion, as well as ensuring proper lighting, angle, and a clear background. A third set of standardized photographs were taken by the resident. Thus, each patient gets three sets of photographs: those taken without guidance/unassisted, those taken after training/with assistance, and those taken by the resident. The teledermatology method used is “store and forward,” where photos are stored securely, with patient information kept separately.

Teledermatology assessment involved three groups of physicians independently assessing the photos of different patients than the ones assessed initially in clinic to avoid overlapping, with each group shown only one set of images per patient. The attending dermatologists and residents in each group evaluate patient-untrained, patient-trained, and resident-taken photos for different patients. The teledermatologists made tele-diagnoses. The time to diagnosis and follow-up were recorded while assessing the images. The patient's medical history was given only if requested by the teledermatologist. Teledermatologists and residents evaluating the images were asked to rate their confidence in making a diagnosis on a scale from 1 to 10. Face-to-face diagnosis served as the benchmark or “true diagnosis”. This process aims to evaluate the reliability of teledermatology compared to in-clinic diagnosis, considering variations in picture quality across the three main categories.

Statistical Analysis

For each picture-based mode of assessment (pictures taken by patient without assistance, pictures taken by patient with assistance, and pictures taken by residents), diagnosis and treatment match with the in-person visit were recorded. In addition, information was collected on confidence in clinical assessment, time to diagnose, time to follow-up visit, and whether medical history was required.

Our primary analysis studied the variation of diagnosis and treatment match and related clinical outcomes/decisions across the three modes. Given that diagnosis and treatment match and other outcomes were repeated across the three picture-based modes of assessments for each individual patient, we used mixed-effect models with subject as a random effect to account for repeated assessments per patient. We first evaluated the intraclass correlation coefficients (ICC) of the repeated clinical outcomes across modes using mixed-effect models with subject as a random effect. The ICCs ranged from 0.124 to 0.439 and were > 0.3 for most outcomes indicating moderate correlation and clustering of the clinical outcome assessments within each subject (ICC estimations and conclusions were similar for each rater group, attendings and residents, and are presented in Supplementary Tables S1 and S2).

Our primary analysis thus used mixed effect models with subject as a random effect to estimate how the different assessment modes and rater expertise groups (i.e., attendings and residents) were related to diagnosis and treatment match and related outcomes. From these models, we reported descriptive statistics of the average percent matches and means of the clinical outcomes of interest across the three modes (descriptive statistics not accounting for repeated assessments per subject are presented in Supplementary Table S3). Diagnosis and treatment match for specific conditions of interest (Papulosquamous dermatoses, Malignant neoplasms, and Acne/ Rosacea) were also described across the three modes. We also report associations between modes and rater’s expertise and the outcomes of interest. Finally, we examined whether patients’ educational level was related to clinical matches and outcomes of interest.

In additional analyses, we assessed agreement between rater groups with regards to diagnosis and treatment match and other outcomes across the modes using kappa coefficients and ICC; results indicated relatively low agreement (kappa = 0.169 to 0.342; Supplementary Table S4) and supported including rater group in the model in the primary analysis. We also examined whether differences in clinical match and outcomes of interest across the three modes were modified by the level of expertise of raters (through a mode*rater interaction term; results showed no consistent patterns of interactions (Supplementary Table S5).

All analyses were conducted using Stata, and a significance level of p-value < 0.05 was applied to determine statistical significance (p < 0.1 was used for interaction terms).

Descriptives of the diagnosis and treatment match and clinical outcomes across the three modes (pictures taken by patient without assistance, pictures taken by patient with assistance, pictures by resident) that account for repeated assessment are presented in Table 1. The average percent of diagnosis matches with the in-person clinical diagnosis gradually increased from pictures by patient without assistance (79%), pictures taken by patient with assistance (84%), and pictures taken by resident (87%). Similarly, treatment match increased gradually across the modes (38% in the patient unassisted mode, 44% in the patient assisted mode, and 45% in the picture taken by resident mode, respectively). The need to ask for history decreased progressively across these modes respectively (64%, 59%, and 47%) with a corresponding decrease in mean time to diagnosis (18.79, 17.01, and 16.27 seconds) and an increase in mean follow-up (indicating delayed follow-up) (16.18, 18.87, and 20.39 days). The mean confidence in diagnosis also increased gradually across these modes (6.43, 6.73, and 6.91).

Table 1

Match in diagnosis and treatment, and related outcomes across the three assessment modes
	Pictures taken by patient without assistance
	Mean^a	95% CI	Mean^a	95% CI	Mean^a	95% CI
Diagnosis match ^b	0.79	0.75, 0.84	0.84	0.80, 0.88	0.87	0.83, 0.90
Treatment match ^b	0.38	0.33, 0.43	0.44	0.38, 0.50	0.45	0.39, 0.50
Asked for history	0.64	0.59, 0.69	0.59	0.53, 0.64	0.47	0.42, 0.52
Mean follow-up (in days)	16.18	13.41, 18.96	18.87	16.09, 21.65	20.39	17.62, 23.17
Time to diagnosis (in seconds)	18.79	17.61, 19.96	17.01	15.83, 18.18	16.27	15.10, 17.45
Confidence	6.43	6.23, 6.63	6.73	6.53, 6.93	6.91	6.71, 7.11
^a Means represent margin values estimated using mixed models accounting for repeated assessments across modes (i.e., repeated patient subjects), with modes of evaluation and rater as predictor.
^b Means of percent matches (ranging from 0 to 1).

Diagnosis match for specific conditions and treatment match for biopsy in the presence of malignant neoplasm are presented in Table 2. The proportion of diagnosis match increased across the modes for specific conditions. Three main categories were compared, with acne cases having a higher proportion of diagnosis matching (95%, 96%, and 98%), compared to malignant neoplasms (76%, 90%, and 86%) and followed by papulosquamous dermatoses (74%, 81%, and 83%). Treatment match for biopsy following correct diagnosis match of malignancy demonstrated an increasing proportion from 82% in pictures taken by patient unassisted and assisted to 92% in pictures taken by residents.

Table 2

Diagnosis and treatment match for specific conditions across the three assessment modes
	Pictures taken by patient without assistance
	Mean^a	95% CI	Mean^a	95% CI	Mean^a	95% CI
Diagnosis match^b
Acne, Rosacea and Hidradenitis (n = 39)	0.95	0.88, 1.01	0.96	0.91, 1.01	0.98	0.94, 1.01
Papulosquamous rashes (n = 50)	0.74	0.61, 0.86	0.81	0.70, 0.91	0.83	0.73, 0.93
Malignant neoplasm (n = 25)	0.76	0.53, 0.99	0.90	0.77, 1.02	0.86	0.69, 1.02
Treatment matchb
Biopsy match for malignanciesc (n = 17)	0.82	0.61, 1.03	0.82	0.62, 1.02	0.93	0.81, 1.05
^a Means represent margin values estimated using mixed models accounting for repeated assessments across modes (i.e., repeated patient subjects), with modes of evaluation as predictor.
^b Means of percent matches (ranging from 0 to 1).
^c Subset of matched malignancy and clinical treatment being biopsy

Table 3 presents the relative differences in diagnosis and treatment match and related clinical outcomes across modes and raters, which also showed the improved gradient from picture taken by patient unassisted to picture taken by resident. Compared to assessments based on picture taken by resident, assessments relying on pictures taken by patients unassisted had significantly lower odds of diagnosis match (OR 0.59; 95% CI: 0.45, 0.79), treatment match (OR 0.76; 95% CI: 0.59, 0.97), and higher odds of asking for patient’s history (OR 1.98; 95% CI: 1.55, 2.82). Assessments based on pictures taken by the patient unassisted had a lower mean follow-up (β -4.21; 95% CI: -7.45, -0.97) and confidence (β -0.48; 95% CI: -0.70, -0.26), and took more time for diagnosis (β 2.51; 95% CI: 1.20, 3.82) compared to those based pictures taken by residents. Matches in diagnosis and treatment and other clinical outcomes were not significantly different if based on pictures taken by patients assisted compared to pictures taken by residents, except for asking for patient’s history (OR 1.59; 95% CI: 1.25, 2.03).

Table 3

Extent of agreement in diagnosis and treatment match, as well as related medical outcomes, across the three assessment modes and raters
Categorical Outcomes	OR	95% CI	P-VALUE
Outcome: Diagnosis match
Mode
Taken by resident	Ref.
Patient, unassisted	0.59	0.45, 0.79	< 0.001*
Patient, assisted	0.83	0.62, 1.11	0.20
Raters
TD	Ref.
Resident	0.66	0.52, 0.83	< 0.001*
Outcome: Treatment match
Mode
Taken by resident	Ref.
Patient, unassisted	0.76	0.59, 0.97	0.03*
Patient, assisted	0.98	0.76, 1.25	0.85
Raters
TD	Ref.
Resident	1.35	1.10, 1.65	0.004*
Outcome: Asked for History
Mode
Taken by resident	Ref.
Patient, unassisted	1.98	1.55, 2.82	< 0.001*
Patient, assisted	1.59	1.25, 2.03	< 0.001*
Raters
TD	Ref.
Resident	1.43	1.17, 1.75	< 0.001*
Continuous outcomes	Coefficient (β)	95% CI	P-VALUE
Outcome: Mean follow-up
Mode
Taken by resident	Ref.
Patient, unassisted	-4.21	-7.45, -0.97	0.01*
Patient, assisted	-1.52	-4.76, 1.71	0.36
Raters
TD	Ref.
Resident	0.49	-2.16, 3.13	0.72
Outcome: Time to diagnosis
Mode
Taken by resident	Ref.
Patient, unassisted	2.51	1.20, 3.82	< 0.01*
Patient, assisted	0.74	-0.57, 2.05	0.27
Raters
TD	Ref.
Resident	-5.42	-6.49, -4.35	< 0.001*
Outcome: Confidence
Mode
Taken by resident	Ref.
Patient, unassisted	-0.48	-0.70, -0.26	< 0.001*
Patient, assisted	-0.17	-0.39, 0.04	0.12
Raters
TD	Ref.
Resident	-1.49	-1.66, -1.31	< 0.001*
*Significant at p-value < 0.05

Moreover, residents were less likely to match the correct diagnosis (OR 0.66; 95% CI: 0.52, 0.83) compared to attendings, but they had higher odds of matching the correct treatment (OR 1.43; 95% CI: 1.17, 1.75) and of asking for patient’s history (OR 1.43; 95% CI: 1.17, 1.75). Residents took less time to diagnose (β -5.42; 95% CI: -6.49, -4.35) and were less confident (β -1.49; 95% CI: -1.66, -1.31) compared to attendings.

Patient’s educational level was not significantly related to outcomes of interest (p > 0.05), as shown in Table 4.

Table 4

Extent of agreement in diagnosis and treatment match, as well as related medical outcomes, across the three assessment modes, medical provider roles, and patient’s education
Categorical Outcomes	OR	95% CI	P-VALUE
Outcome: Diagnosis match
Mode
Taken by resident	Ref.
Patient, unassisted	0.59	0.45, 0.79	< 0.001*
Patient, assisted	0.83	0.62, 1.11	0.204
Raters
TD	Ref.
Resident	0.66	0.52, 0.83	< 0.001*
Education
University degree	Ref.
No University degree	0.71	0.44, 1.15	0.166
Outcome: Treatment match
Mode
Taken by resident	Ref.
Patient, unassisted	0.76	0.59, 0.97	0.029*
Patient, assisted	0.98	0.76, 1.25	0.851
Raters
TD	Ref.
Resident	1.35	1.10, 1.65	0.004*
Education
University degree	Ref.
No University degree	1.15	0.76, 1.75	0.504
Outcome: Asked for History
Mode
Taken by resident	Ref.
Patient, unassisted	1.97	1.54, 2.52	< 0.001*
Patient, assisted	1.59	1.24, 2.02	< 0.001*
TD/R
Attendings	Ref.
Resident	1.43	1.18, 1.75	< 0.001*
Education
University degree	Ref.
No University degree	1.33	.90, 1.98	0.154
Continuous outcomes	Coefficient (β)	95% CI	P-VALUE
Outcome: Mean follow-up
Mode
Taken by resident	Ref.
Patient, unassisted	-4.20	-7.45, -0.96	0.011*
Patient, assisted	-1.51	-4.76, 1.74	0.363
Raters
TD	Ref.
Resident	0.48	-2.18, 3.13	0.725
Education
University degree	Ref.
No University degree	-2.49	-7.38, 2.40	0.319
Outcome: Time to diagnosis
Mode
Taken by resident	Ref.
Patient, unassisted	2.49	1.18, 3.80	< 0.001*
Patient, assisted	0.73	-0.58, 2.04	0.276
Raters
TD	Ref.
Resident	-5.40	-6.48, -4.32	< 0.001*
Education
University degree	Ref.
No University degree	1.63	-0.52, 3.77	0.137
Outcome: Confidence
Mode
Taken by resident	Ref.
Patient, unassisted	-0.47	-0.69, -0.25	< 0.001*
Patient, assisted	-0.18	-0.40, 0.04	0.107
Raters
TD	Ref.
Resident	-1.48	-1.66, -1.30	< 0.001*
Education
University degree	Ref.
No University degree	-0.29	-0.66, 0.08	0.126
*Significant at p-value < 0.05

Teledermatology stands as a transformative avenue, bridging the accessibility gap for patients encountering challenges in obtaining in-clinic dermatological services. A study conducted in Amsterdam strongly endorses the effectiveness of teledermoscopy (TDsc) in elevating the quality and efficiency of Dutch dermatologic general practitioner (GP) healthcare services ⁴. Building upon this foundation, our study delves deeper, shedding light on the pivotal role of patient education in optimizing teledermatology outcomes, accentuating the significance of image quality for accurate diagnosis and treatment alignment.

Our research unfolds a progressive trend in diagnostic and treatment matching across three modes of image capture: pictures taken by the patient independently, pictures taken with patient assistance, and pictures taken by residents. The correlation is evident – as the level of expertise and assistance in image capture increases, so does the precision of diagnoses and treatment plans (Table 1). Notably, comparing patient-taken pictures with or without assistance to reference pictures taken by residents showcases a significant difference, underscoring the need for educational resources accessible to all.

Introducing educational tools, such as instructional videos guiding users through optimal image capture, emerges as a practical approach. Following teledermatology practice guidelines, a video tutorial could enhance image clarity by guiding through recommended sequences for capturing skin lesion images. This proactive step aims to empower individuals in maximizing the benefits of teledermatology sessions⁵.

Our analysis unveils distinct patterns in outcomes based on dermatological conditions. While acne cases exhibit consistently high diagnostic matching across all modes, more complex disorders like malignant neoplasms and papulosquamous disorders showcase variations, emphasizing the need for careful consideration of image quality and context. Notably, better-quality pictures, especially those taken by residents, significantly reduce the likelihood of missing malignant cutaneous neoplasms, thereby emphasizing the clinical relevance of high-quality imagery (Table 2).

Teledermatology emerges as a powerful tool in the early detection of skin cancers, particularly cutaneous malignancies. Research underscores its efficacy in effectively screening high-risk melanoma patients, demonstrating a high agreement between face-to-face (FTF) dermatologists and teledermatologists. Prompt access to dermatological guidance is facilitated, offering a crucial advantage in cases where early detection is imperative^{6, 7}. However, the context of pigmented lesions in teledermatology remains a topic of debate. While some studies report a high agreement for pigmented lesions, others indicate lower accuracy, particularly in cases of dysplastic nevi, melanoma, and lentigines ^{6, 7}.

Residents exhibit higher odds of matching correct treatment and inquiring about patient history but are less likely to match correct diagnoses compared to attending physicians. This highlights the need for further refining residents' diagnostic capabilities in teledermatology.

Importantly, our study reveals no significant correlation between the patient's educational level and teledermatology outcomes, emphasizing the versatility of this tool across different educational backgrounds and making it accessible to a broad range of patients (Table 4).

While teledermatology presents substantial advantages, it comes with inherent limitations. The inability to conduct a comprehensive full-body examination, potential legal risks associated with diagnostic errors, and the risk of overlooking other skin lesions while focusing on a specific one are noteworthy concerns ^{4, 8}.

Our study acknowledges specific limitations, including the reliance on in-person expert opinions as the sole basis for the "true diagnosis" due to constraints on pathological confirmation. Diverse management strategies and inter-variability among experts further contribute to differences in diagnostic and treatment approaches. The study's broad recruitment strategy adds complexity, necessitating larger-scale studies to effectively categorize the utility of teledermatology for specific conditions.

In conclusion, our research amplifies the nuanced facets of teledermatology, spotlighting both its advantages and avenues for refinement. The findings underscore the considerable potential of teledermatology as a valuable resource in dermatological care, particularly after refining picture-taking methods with the “store and forward” approach. As we unravel these insights, the ongoing evolution of teledermatology continues to shape a more accessible and efficient landscape for dermatological healthcare.

Funding information: None

Conflicts of Interest: None declared.

Ethical Approval: Reviewed and approved by AUB IRB; approval # BIO-2020-0241

Ethics statement: The patients in this manuscript have given written informed consent to the publication of their case details

Author Contribution

S.S wrote the proposal and lead the research work D.K conducted the research in the institution W.A wrote the main parts of the paper and helped in conducting the work in the institutionC.S helped in writing the proposal N.C entered the data and helped in writing the manuscriptR.J entered the data and helped in writing the manuscriptM.E did the main part of the data analysis and tables M.B helped in data analysis and tables S.M helped in data annalysis and tables ALL authors reviewd the manuscript

Data availability statement:

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Manuelyan, K., Shahid, M., Vassilev, V., Drenovska, K. & Vassileva, S. Direct patient-to-physician teledermatology: Not a flash in the pan(demic). Clin. Dermatol. 39 (1), 45–51 (2021).
Loh, C. H., Chong Tam, S. Y. & Oh, C. C. Teledermatology in the COVID-19 pandemic: A systematic review. JAAD Int. 5, 54–64 (2021).
Chuchu, N. et al. Teledermatology for diagnosing skin cancer in adults. Cochrane Database Syst. Rev. 12 (12), Cd013193 (2018).
van Sinderen, F. et al. Is Teledermoscopy Improving General Practitioner Skin Cancer Care? Stud. Health Technol. Inf. 264, 1795–1796 (2019).
Koop, C. et al. A country-wide teledermatoscopy service in Estonia shows results comparable to those in experimental settings in management plan development and diagnostic accuracy: A retrospective database study. JAAD Int. 12, 81–89 (2023).
Piccolo, D. et al. Teledermoscopy–results of a multicentre study on 43 pigmented skin lesions. J. Telemed Telecare. 6 (3), 132–137 (2000).
Warshaw, E. M. et al. Teledermatology for diagnosis and management of skin conditions: a systematic review. J. Am. Acad. Dermatol. 64 (4), 759–772 (2011).
Viola, K. V. et al. Outcomes of referral to dermatology for suspicious lesions: implications for teledermatology. Arch. Dermatol. 147 (5), 556–560 (2011).

No competing interests reported.

SRSupplementarytables.docx

Download PDF

Reviews received at journal
17 Sep, 2024
Reviewers agreed at journal
15 Sep, 2024
Reviewers invited by journal
14 Sep, 2024
Editor assigned by journal
14 Sep, 2024
Editor invited by journal
06 Sep, 2024
Submission checks completed at journal
04 Sep, 2024
First submitted to journal
24 Aug, 2024

You are reading this latest preprint version

The Diagnostic Accuracy of Skin Diseases Via Teledermatology – A Single Center Study

Status:

Version 1

Abstract

Background:

Objective:

Methods

Results:

Conclusion:

Introduction

Methods

Statistical Analysis

Results

Discussion

Declarations

Author Contribution

Data availability statement:

References

Additional Declarations

Supplementary Files

Status:

Version 1