Perception and Knowledge of Telemedicine Technology in Hispanic Practicing Physicians: Instrument Validation

doi:10.21203/rs.3.rs-129467/v1

Download PDF

Research Article

Perception and Knowledge of Telemedicine Technology in Hispanic Practicing Physicians: Instrument Validation

https://doi.org/10.21203/rs.3.rs-129467/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background During the COVID-19 pandemic, multiple countries has taken measures, such as isolation and quarantine to prevent person-to-person spread of disease. This forced many physicians to adopt new techniques to continue patient care, such as telemedicine, that has proven to be useful in continued care of non-COVID 19 pathologies. Different factors influence the perception of telemedicine in medical practitioners such as security, confidentiality, cost-effectiveness issues, comfort, and the risk of malpractice.

Methods The original questionnaire included 6 domains with 40 questions, each question was measured with a five-point likert-scale ranging from very high [5] to very low [1]. Consequently, the survey was translated to spanish using machine translation (MT). The translation was reviewed independently and then a consensus was made regarding minor changes in the syntax of the survey to facilitate understanding. After the expert feedback and questionnaire indicators, the research team members proposed to reduce the instrument to 13-items in 4 domains, due similarity of questions. The sample was divided into 2 groups, randomly selected.

Results The 382 surveys were collected and separated in two random samples, S1 and S2, (198 and 184 respectively). In exploratory factor analysis (EFA) the 13-items are grouped into the four theoretical domains, item 7 presented cross loading between factors and was removed. Confirmatory factor analysis (CFA) was performed to assess scale reliability and inter scale associations; three models were tested.

Conclusions The translated instrument was clear, with adequate internal consistency, readability, and appropriate to be applied in the physician setting. This validated questionnaire made it possible to evaluate the real knowledge of telemedicine in order to increase its use, especially in the COVID-19 pandemic.

Health Economics & Outcomes Research

Infectious Diseases

Health Policy

Telemedicine

Healthcare

Questionnaire

Perceptions

The use of information and communication technologies (ICTs) in healthcare has significantly changed the physician-patient relationship (1). Telemedicine is the implementation of ICTs to deliver a distance medical care (2). During the COVID-19 pandemic, multiple countries had taken public health measures. Isolation and quarantine were the community containment plan to prevent person-to-person spread of disease (3). This unique situation forced many physicians to adopt new techniques to continue patient care (4). Telemedicine has proven to be a useful tool during the pandemic. As a result, it has been used in the assessment and triage of patients with COVID-19 reducing healthcare load. Nevertheless, it has proven to be useful in continued care of non-COVID-19 pathologies (5–7).

The adaption to new technologies is generally a process of constant evaluation of costs and benefits (8). For instance, different factors influence the perception of medical practitioners such as security, confidentiality, cost-effectiveness issues, comfort, and the risk of malpractice (9). Kuo et al. (2015) described several factors that could induce the intention to adopt use in physicians such as attitude, subjective norm, and perceived behavioral control (10). In addition, telemedicine may reduce costs due to decreased prolonged hospital stays (11).

The appropriate assessment of knowledge and perception of clinicians about telemedicine is essential. Thereby, create a reliable tool that can detect changes in physicians’ perceptions about the use of telemedicine in the future (12–13). The aim of this study was to adapt a validated Spanish version questionnaire of the perception and knowledge of telemedicine in healthcare professionals (9).

Study Design

We followed the STROBE guidelines for observational studies.

The original questionnaire included 6 domains with 40 questions, each question was measured with a five-point Likert-scale ranging from very high [5] to very low [1] (9). Yang et al. (2018) proposed a model for the adoption of machine translations (MT) (14). Consequently, the survey was translated to spanish using MT. The translation was reviewed independently by every research team member, then a consensus was made regarding minor changes in the syntax of the survey to facilitate understanding.

The translated instrument was converted into a google form survey and was implemented as pilot test among 65 physicians. The survey objective was to ensure that the redaction, vocabulary, design, and time needed to complete the survey was appropriate enough to prove its stability and comprehension (15).

After the expert feedback and questionnaire indicators, the research team members proposed to reduce the instrument to 13-items in 4 domains, due similarity of questions. In the same way were excluded questions that could not be used due to differences in administrative characteristics (budget management, conferences, trainings or guidelines).

Sample size was determined by multiplying the number of questions (13 items) by 5 (lower limit) and 20 (upper limit), as proposed by Suhr (2005) and other studies of validation in Ecuadorian population (16–18). The total sample was 65 to 250 doctors.

Eligibility criteria included physicians from private and public services with an active medical/clinical practice. The questionnaire was answered by 404 participants and 22 of them were excluded because they did not belong to the Ecuadorian population. The sample was divided into 2 groups, randomly selected. The first group consisted of 198 doctors for sample 1 (S1) to perform exploratory factor analysis (EFA). The second group included 184 doctors for sample 2 (S2) for confirmatory factor analysis (CFA).

Statistical Analysis

Descriptive Analyze

A descriptive statistical analysis was implemented for sociodemographic data. Descriptive variables were presented as frequencies and percentages, and quantitative variables as means and standard deviations. The perception of each aspect of telemedicine was represented by mean response value in the corresponding group of questions. Age, gender, educational level, and area information were used as independent variables on each analysis.

Psychometric Analysis

The Kaiser-Meyer-Olkin (KMO) measure was calculated to evaluate the sampling adequacy of each item on the anti-image of the correlation matrix (19). All sampling measure cutoff points were set at 0.5 to be considered acceptable (19). Bartlett’s test of sphericity indicated suitability for analysis being relevant to the scale (20). Commonality analysis was performed and after this procedure the questionnaire was reduced to ensure construct parsimony and usability.

The scale structure was determined by several indicators. Initially, eigenvalues were extracted. A factor structure with eigenvalues above 1 were selected as candidate for EFA. Maximum likelihood estimation (MLE) was performed. Therefore, MLE supported the reduction of the instrument in 4 domains, according to the model fit test. The chi square showed a non-significant result, supporting the MLE.

The internal consistency was calculated by Cronbach's alpha for both, the full scale and for each of the subscales found in EFA. Cronbach's alpha measures the interrelation of the items, with values ranging between 0 and 1 (21). The results were considered acceptable above the threshold of > 0.7.

Other tests were used to indicate goodness of fit such as Chi square, Roots mean square error of approximation-RMSEA (range 0–1 with a recommended result of ≤ 0.06), Normed fit index-NFI (range 0–1 recommending > 0.90), Tucker Lewis index-TLI (0–1 recommending > 0.90), Comparative fit index-CFI (range 0–1 recommending > 0.90), Parsimony ratio (range 0–1 recommending about 1), Parsimony normed fit index-PNFI (range 0–1 recommending > 0.50), and Parsimony comparative fit index-PCFI (range 0–1 recommending > 0.50) (22–23).

Readability

Fernandez-Huerta index and Crawford were used to assess readability and estimate grade level, respectively. Scores were reported (range 0-100), higher results indicated greater readability, and a result of 60–70 represented an easy understanding for the population of ~ 15 years old (24).

The 382 surveys were collected and separated in two random samples, S1 and S2, (198 and 184 respectively). EFA was performed in S1. Sociodemographic characteristics of the sample were presented in table 1 and showed that 110 (55.6%) of doctors had 20 years of experience. Additionally, 60 (30.5%) of all doctors worked in hospitals, 174 (87.9%) of them had permanent employment, and 41 (20.7%) worked in primary care (see Table 1)

TABLE 1 Sociodemgraphics characteristics of the population
TABLE 1 Sociodemgraphics characteristics of the population
Variables		S1		S2
		S1		S2
		Re-count (n)	Percentage (%)	Re-count (n)	Percentage (%)
Gender	Male	115	58,1	109	59,2
	Female	83	41,9	75	40,8
	Total	198	100	184	100
Employment type	Permanent	174	87,9	165	89,7
	Temporarily	24	12,1	19	10,3
	Total	198	100	184	100
Education level	Bachelor	1	0,5	2	1,1
	General doctor	41	20,7	44	23,9
	Medical specialist	132	66,7	123	66,8
	Master's degree	13	6,6	7	3,8
	Phd	11	5,6	8	4,3
	Total	198	100	184	100
Work experience (years)	1--5	19	9,6	16	8,7
	6--10	21	10,6	19	10,3
	11--15	23	11,6	30	16,3
	16--20	25	12,6	25	13,6
	up to 20	110	55,6	94	51,1
	Total	198	100	184	100
Workplace	Hospital	60	30,3	42	22,8
	Clinic	47	23,7	46	25
	Both	36	18,2	38	20,7
	Other	55	27,8	58	31,5
	Total	198	100	184	100

Exploratory factor analysis

In EFA the 13-items are grouped into the four theoretical domains, item 7 presented cross loading between factors and was removed. However, item removal at this stage did not improve reliability statistics (see Table 2). From the original instrument, the domains that remained were "Knowledge about telemedicine”, "Perception of the utility of telemedicine", "Perception of the disadvantages of telemedicine", and "Knowledge of the security of telemedicine”.

Global Cronbach’s alpha for internal consistency was 0.76.

TABLE 2 Factor Analysis Pattern Matrix
TABLE 2 Factor Analysis Pattern Matrix
Questions	Domains
Questions	1	2	3	4
Q1	0,91
Q2	0,983
Q3	0,824
Q4			0,786
Q5			0,825
Q6			0,79	-0,38
Q7		0,476	0,601
Q8				0,774
Q9				0,741
Q10				0,782
Q11		0,922
Q12		0,945
Q13		0,927

Confirmatory factor analysis.

CFA was performed to assess scale reliability and inter scale associations. Three models (M1, M2, M3) were tested, establishing that the questionnaire had greater reliability (see Table 3). The removal of item 7 at this stage provided the best goodness of fit measures as a result from the final model. Goodness of fit measures were recorded in table 4 (see Table 4). Figure 1 shows the results for model 3 with RMSEA (0.009) and CFI (0.999) (see Figure 1).

Table 3 Comparison of Cronbach’s alpha of the three models-Model 3 excluded item 7

SAMPLE 1	Cronbach’s Alfa per domains-Model 1	Total Cronbach’s Alfa-Model 1	SAMPLE 2	Cronbach’s Alfa per domains-Model 2	Total Cronbach’s Alfa-Model 2	SAMPLE 2	Cronbach’s Alfa per domains-Model 3	Total Cronbach’s Alfa-Model 3
Q1	0,93	0,76	Q1	0,942	0,742	Q1	0,942	0,715
Q2			Q2			Q2
Q3			Q3			Q3
Q4	0,834		Q4	0,803		Q4	0,833
Q5			Q5			Q5
Q6			Q6			Q6
Q7			Q7
Q8	0,807		Q8	0,772		Q8	0,772
Q9			Q9			Q9
Q10			Q10			Q10
Q11	0,952		Q11	0,947		Q11	0,947
Q12			Q12			Q12
Q13			Q13			Q13

TABLE 4 Goodness of fit measures for Model 1, Model 2 and Model 3
	ABSOLUTE ADJUSTMENT MEASURES		INCREMENTAL ADJUSTMENT MEASURES			PARISOMIA ADJUSTMENT MEASURES
MODELS	Chi-Square	RMSEA(IC)	NFI	TLI	CFI	PRATIO	PNFI	PCFI
M1	0,002	0.057(0.034 – 0.078)	0,939	0,97	0,976	0,782	0,735	0,763
M2	0,176	0.032 (0.000 – 0.060)	0,961	0,992	0,994	0,758	0.728.	0,753
M3	0,443	.009(0.000 – 0.049)	0,968	0,999	0,999	0,727	0,704	0,727

Readability

A readability analysis was conducted. Results showed a Fernandez-Huerta score of 52.6 and a Crawford Grade level of 6.6 (25). These results indicated that these items were likely to be understandable by a typical 14–15-year-old individual and required a minimum of six years of schooling, respectively.

The result of this validation study achieved a slim, and reliable instrument. The present results can be used to rapidly assess the perception towards a new technology backed up by data derived from a large cohort of medical practitioners. Several studies concluded that the length of the questionnaires had an inverse relation with their response rate (26–27). Although, due to the short amount of time available for physicians, the survey offered a shorter and feasible model.

The digital transformation of medicine had enabled new ways to improve communication between patients and physicians. However, they had negative perceptions about their use. Misleading a diagnosis or how their patient’s information security can be compromised were some of them (28–29). The knowledge about new technologies is an important key to identify negative. In consequence, healthcare professionals will have to learn and adapt their tool communication and technology for better results.

Evidence reported that MT can be very useful to the scientific literature for reliability measures of an instrument (30). The question syntax was simple enough to be rapidly translated to a different language with acceptable results. In addition, participant characteristics and sample size ensure that this instrument was appropriate for validation. Another strength of the validation was the heterogeneity of the sample, with different medical specialties responding to the questionnaire.

Goodness of fit measures for model 3 were better in comparison with models 1 and 2. Our instrument had readability and validity according to the CFA. In contrast, the original instrument only had correlation between the items.

The final proposed model had a positive relationship between domains 1 and 2, which was related to a positive selection in domain 2 but negatively in domain 3. Likewise, there was a lower positive relationship between domain 1 and domain 4, and a negative relationship of domain 3 in domain 1 and 2.

There were some limitations of our study that need to be discussed. First, our population covered only Ecuadorian physicians, and the sample may not be representative in other Hispanic countries. Second, we used only 4 of the 6 domains of the original version. Nevertheless, we had a robust methodology to confirm validation of our survey.

The original survey compiled 6 characteristics domains towards the population and was validated using face and content validity methods. The reliability of the original questionnaire reported a Cronbach’s alpha coefficient of 0.73 (9). Additionally, our instrument exhibited an acceptable KMO test, a significant Bartlett, and explained the variance of the 4 dimensions with a strong significant association. Furthermore, our questionnaire in comparison with the original one had some other statistical tests such as chi square, RMSEA, TLI, NFI, PNFI, CFI, and PCFI indicating goodness of fit.

Our study provided a translated and validated questionnaire to evaluate physician knowledge and perception of telemedicine. The translated instrument was clear, with adequate internal consistency, readability, and appropriate to be applied in the physician setting.

After performing the CFA, it was possible to obtain the 12 items grouped into the four theoretical domains.

It was essential to identify the difficulties medical doctors have to adapt themselves to new technologies. This validated questionnaire made it possible to evaluate the real knowledge of telemedicine in order to increase its use, especially in the COVID-19 pandemic.

CEISH

Comité de ética e Investigación en Seres Humanos, CFA:confirmatory factor analysis, CFI:Comparative fit index, EFA:exploratory factor analysis, ICTs:Information and communication technologies, KMO:Kaiser-Meyer-Olkin, MLE:maximum likelihood estimation, MT:Machine translations, NFI:Normed fit index, PCFI:Parsimony comparative fit index, PNFI:Parsimony normed fit index, RMSEA:Roots mean square error of approximation, TLI:Tucker Lewis index.

Ethical considerations

This study was approved by Comité de ética e Investigación en Seres Humanos (CEISH), ethical review board, Guayaquil-Ecuador (#HCK-CEISH-18-0060). Informed consent was obtained from every participant.

Consent for publication

Ayatollahi H. et al. gave written consent via e-mail for the use of the questionnaire.

Availability of data

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no conflict of interest.

Funding

The funds for the development of this research where given by Universidad de Especialidades Espíritu Santo.

Authors contributions

ICO and GA coordinated the study. The design, acquisition, analysis, and interpretation of data was performed with substantial contributions from all participating authors. CV, CKY and NSP drafted the manuscript. The statistical analysis was performed by GA and CV. All authors contributed to edited drafts of this manuscript. All authors read and approved the final version of the manuscript.

Acknowledgements

We want to acknowledge Ayatollahi H. et al. to allow the use of the original questionnaire. We express our gratitude to all members of HR services. We also want to acknowledge Universidad Espiritu Santo for their continuous support.

Schaper LK, Pervan GP. ICT and OTs: A model of information and communication technology acceptance and utilisation by occupational therapists. Int J Med Inf. 2007 Jun;76:S212–21.
Combi C, Pozzani G, Pozzi G. Telemedicine for Developing Countries: A Survey and Some Design Issues. Appl Clin Inform. 2016 Oct;07(04):1025–50.
Wilder-Smith A, Freedman DO. Isolation, quarantine, social distancing and community containment: pivotal role for old-style public health measures in the novel coronavirus (2019-nCoV) outbreak. J Travel Med. 2020 Mar 13;27(2):taaa020.
Wosik J, Fudim M, Cameron B, Gellad ZF, Cho A, Phinney D, et al. Telehealth transformation: COVID-19 and the rise of virtual care. J Am Med Inform Assoc. 2020 Jun 1;27(6):957–62.
Portnoy J, Waller M, Elliott T. Telemedicine in the Era of COVID-19. J Allergy Clin Immunol Pract. 2020 May;8(5):1489–91.
Smith AC, Thomas E, Snoswell CL, Haydon H, Mehrotra A, Clemensen J, et al. Telehealth for global emergencies: Implications for coronavirus disease 2019 (COVID-19). J Telemed Telecare. 2020 Mar 20;1357633 × 2091656.
Giavina Bianchi M, Santos AP, Cordioli E. The majority of skin lesions in pediatric primary care attention could be managed by Teledermatology. Picardo M, editor. PLOS ONE. 2019 Dec 2;14(12):e0225479.
Aldunate R, Nussbaum M. Teacher adoption of technology. Comput Hum Behav. 2013 May;29(3):519–24.
Ayatollahi H, Sarabi FZP. Clinicians’ Knowledge and Perception of Telemedicine Technology. 2015;16.
Kuo K-M, Talley PC, Lee C-M, Yen Y-C. The Influence of Telemedicine Experience on Physicians’ Perceptions Regarding Adoption. Telemed E-Health. 2015 May;21(5):388–94.
Hjelm NM. Benefits and drawbacks of telemedicine. J Telemed Telecare. 2005 Mar;11(2):60–70.
Alaboudi A, Atkins A, Sharp B, Balkhair A, Alzahrani M, Sunbul T. Barriers and challenges in adopting Saudi telemedicine network: The perceptions of decision makers of healthcare facilities in Saudi Arabia. J Infect Public Health. 2016 Nov;9(6):725–33.
de Souza CHA, Morbeck RA, Steinman M, Hors CP, Bracco MM, Kozasa EH, et al. Barriers and Benefits in Telemedicine Arising Between a High-Technology Hospital Service Provider and Remote Public Healthcare Units: A Qualitative Study in Brazil. Telemed E-Health. 2017 Jun;23(6):527–32.
Yang Y, Wang X. Modeling the intention to use machine translation for student translators: An extension of Technology Acceptance Model. Comput Educ. 2019 May;133:116–26.
Vogel S, Draper-Rodi J. The importance of pilot studies, how to write them and what they mean. Int J Osteopath Med. 2017 Mar;23:2–3.
Suhr D. 200 – 31: Exploratory or Confirmatory Factor Analysis?:17.
Cherrez-Ojeda I, Haddad M, Vera Paz C, Valdevila Figueira JA, Fabelo Roche J, Orellana Román C, et al. Spanish Validation Of The Revised Depression Attitude Questionnaire (R-DAQ). Psychol Res Behav Manag. 2019 Nov;Volume 12:1051–8.
Alvarado-Villa GE, Moncayo-Rizzo JD, Gallardo-Rumbea JA. Spanish validation endorsement of SDM-Q-9, a new approach. BMC Public Health. 2019 Dec;19(1):106.
Williams B, Onsman A, Brown T. Exploratory factor analysis: A five-step guide for novices. Australas J Paramed [Internet]. 2010 Aug 2 [cited 2020 Jul 24];8(3). Available from: http://ajp.paramedics.org/index.php/ajp/article/view/93
Courtney M, Gordon R. Determining the Number of Factors to Retain in EFA: Using the SPSS R-Menu v2 0 to Make More Judicious Estimations. [cited 2020 Jul 23]; Available from: https://scholarworks.umass.edu/pare/vol18/iss1/8/
Taber KS. The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Res Sci Educ. 2018 Dec;48(6):1273–96.
Sun J. Assessing Goodness of Fit in Confirmatory Factor Analysis. Meas Eval Couns Dev. 2005 Jan;37(4):240–56.
Schermelleh-Engel K, Moosbrugger H, Müller H. Evaluating the Fit of Structural Equation Models: Tests of Significance and Descriptive Goodness-of-Fit Measures. 2003;8(2):52.
AM F Lecturabilidad de Fernández Huerta; 2016. Available from: https://legible.es/blog/lecturabilidad-fernandez-huerta/.
Fernández AM. Analizador de legibilidad de texto [Internet]. Legible. 2016 [cited 2020 Jul 23]. Available from: https://legible.es/
Iglesias C, Torgerson D. Does Length of Questionnaire Matter? A Randomised Trial of Response Rates to a Mailed Questionnaire. J Health Serv Res Policy. 2000 Oct;5(4):219–21.
Rolstad S, Adler J, Rydén A. Response Burden and Questionnaire Length: Is Shorter Better? A Review and Meta-analysis. Value Health. 2011 Dec;14(8):1101–8.
Ly BA, Labonté R, Bourgeault IL. The beliefs of Senegal’s physicians toward the use of telemedicine. Pan Afr Med J [Internet]. 2019 Oct 16 [cited 2020 Aug 25];34. Available from: http://www.panafrican-med-journal.com/content/article/34/97/full/
Lin C-CC, Dievler A, Robbins C, Sripipatana A, Quinn M, Nair S. Telehealth In Health Centers: Key Adoption Factors, Barriers, And Opportunities. Health Aff (Millwood). 2018 Dec;37(12):1967–74.
Zulfiqar S, Wahab MF, Sarwar MI, Lieberwirth I. Is Machine Translation a Reliable Tool for Reading German Scientific Databases and Research Articles? J Chem Inf Model. 2018 Nov 26;58(11):2214–23.

Download PDF

Editorial decision: Major revision
21 Feb, 2021
Reviews received at journal
04 Feb, 2021
Reviewers agreed at journal
28 Jan, 2021
Reviewers agreed at journal
28 Jan, 2021
Reviewers agreed at journal
07 Jan, 2021
Reviewers invited by journal
04 Jan, 2021
Editor assigned by journal
04 Jan, 2021
Editor invited by journal
04 Jan, 2021
Submission checks completed at journal
04 Jan, 2021
First submitted to journal
15 Dec, 2020

You are reading this latest preprint version

Perception and Knowledge of Telemedicine Technology in Hispanic Practicing Physicians: Instrument Validation

Status:

Version 1

Abstract

Figures

Background

Methods

Statistical Analysis

Results

Discussion

Conclusions

List of Abbreviations

Declarations

References

Supplementary Files

Status:

Version 1