Study Design
In this study, a sequential mixed methods research design, with initial semi-structured interviews of several stakeholders in cardiovascular care followed by a self-administered online survey of a wider group, was used. Results from the interviews were used to develop the quantitative survey. This approach allowed the capturing of in-depth perspectives from key stakeholders in cardiovascular care and the corroboration of these findings with quantitative evidence from a wider community of healthcare professionals.
Qualitative Study
Participant selection and recruitment
Medical Doctors, nurses, health IT specialists, clinical scientists, medical researchers, hospital administrators, and cardiovascular AI specialists were eligible to participate in the interviews. The primary inclusion factors were:
- Knowledge of AI in cardiovascular care
- Over 3 years of experience in cardiovascular care or AI solutions in cardiovascular care.
Participant recruitment was conducted using purposive sampling. This approach was chosen to ensure that the participants could provide in-depth and informed perspectives directly relevant to the research objectives. Eligible participants were recruited and identified by the researchers and invited to participate in the research via email and LinkedIn
Data collection
A semi-structured interview guide was developed following an extensive literature review of the challenges of AI in medical care. The interview guide was tested on a demographically similar subset of the target population. Feedback from this group led to refinements in question wording and order, optimizing clarity and response efficacy. All interviews lasted 30-40 minutes and were conducted online, via video conferencing, with one interviewer in November 2023. The interviewer asked follow-up questions for clarification and to allow participants to elaborate on the responses provided. The interviews were recorded, transcribed verbatim, and inductively analyzed to identify major themes and additional sub-themes.
Data analysis
The transcribed text was subjected to inductive thematic analysis. This entailed a systematic process of coding the data in multiple rounds, allowing themes to emerge directly from the participants' narratives without the constraint of pre-existing theoretical frameworks. Initial codes were generated by reading and re-reading the transcripts, which were then grouped into potential themes reflecting the core meanings evident in the data. These themes were reviewed and refined iteratively, ensuring they accurately represented the views of participants.
Quantitative Study
Participant selection and recruitment
Medical Doctors, nurses, health IT specialists, clinical scientists, medical researchers, hospital administrators, and cardiovascular AI specialists were eligible to participate in the surveys. A mix of purposive sampling and snowball sampling was used to ensure that participants met the eligible professional criteria and to reach a larger portion of the target population. Participants were invited to participate in the survey via email and LinkedIn.
Data collection
A 10-minute online survey was created using Soscisurvey (Version 3.5.00) developed by SoSci Survey GmbH based on the key themes identified in the qualitative analysis. The questionnaire was developed in English and pretested with 7 individuals who were demographically similar to the target population. Feedback from the pretest indicated that certain questions in the survey were ambiguous and lacked clarity. Consequently, the highlighted questions were refined to be more precise and understandable. A screener question was employed to identify and filter out inattentive responses from participants. The survey link was shared with the participants and was online for 6 weeks from November to December 2023. The survey results were subsequently downloaded in Comma-Separated Values (CSV) format and subsequently analyzed.
Data analysis
The data was initially cleaned to remove incomplete responses and responses that incorrectly answered the screener. As responses were numerically coded automatically be the survey platform, additional coding was not necessary. Descriptive statistics were employed to gain insights into the characteristics of respondents and for univariate analysis. The non-parametric tests, Kruskal-Wallis and Mann-Whitney with Bonferroni correction, were used for pairwise comparisons. The analysis was done using Python libraries: Pandas (version 1.4.2), SciPy (version 1.8.1), and NumPy (version 1.23.3) due to their versatility, efficiency, and extensive capabilities for handling and analyzing large datasets. Statistical significance was set at p-value ≤ 0.05.
Qualitative Results
Description of participants
Five experts representing diverse sectors within the cardiovascular care domain were interviewed: PT1, a medical doctor and AI researcher; PT2, the CEO of an AI-focused CVD company; PT3, a clinical scientist specializing in CVD care; PT4, a digital health hospital consultant; and PT5, an interventional cardiologist with experience in AI-assisted cardiovascular procedures.
Identified Challenges
Eight major challenges were identified by participants during the interviews. The findings are summarized in Table 1 and described below.
- Data-Related Challenges: Participants identified several issues related to data that existed and needed to be addressed. According to them, data is vital for training AI algorithms in cardiovascular care and issues with data affect the quality of the algorithms developed. Challenges related to data integration and data access were highlighted by participants. Data for training and validating AI models is scarce. However, this data scarcity is artificial as participants acknowledged that data exist in different cardiovascular institutions. Issues like data fragmentation, data silos, differences in regulation, and variations in annotation make accessing this data difficult. Participants also highlighted that cardiovascular care requires multimodal data for the diagnosis of CVDs. However, existing AI solutions do not allow for the integration of multimodal data in the course of treatment, posing a challenge to their use in cardiovascular care.
- Regulatory Challenges: Participants also highlighted unclear regulations and the inflexibility of existing regulations as the primary regulatory challenges to AI in cardiovascular care. Existing AI regulations are vague and do not have specific medical or cardiovascular applications. This lack of clarity results in a long and arduous process for obtaining regulatory approval for AI solutions in cardiovascular care. Additionally, participants indicated that existing regulations require developers to freeze their algorithms to obtain regulatory approval and developers have to apply for a new approval once changes are made to the algorithms. Participants believed that this poses a challenge as AI algorithms require constant training with real-world data to be safer and more efficient. Requiring regulatory approval each time an algorithm is trained results in an overly burdensome process.
- Infrastructural Challenges: Participants also acknowledged the dearth of human and technological infrastructure for the integration of AI in cardiovascular care. Having individuals with the right knowledge to develop and maintain AI infrastructure is key to AI use in clinical practice. However, participants indicated that most healthcare institutions lack the necessary IT department, equipped with an understanding of how AI systems work. In addition, healthcare facilities largely use legacy systems that are incompatible with the latest AI technology and participants believe that this hampers the use of AI in cardiovascular care. Participants also highlighted the existence of a rural-urban divide regarding infrastructure. While healthcare institutions in large urban areas are actively researching and investing in improving their AI infrastructure, healthcare institutions in rural areas are largely not focused on AI infrastructure.
- Knowledge Challenges: Participants highlighted a bidirectional knowledge gap between healthcare professionals and developers. They noted that healthcare professionals lack sufficient understanding of AI to effectively communicate its mechanisms to patients, while developers often lack the medical expertise necessary for creating clinically relevant applications. Participants also acknowledged the lack of AI training in medical and medically-affiliated curricula as one of the primary reasons for this gap. However, they also acknowledged the existence of advanced professional courses for individuals willing to improve their knowledge of AI.
- Transparency Challenges: A lack of explainability of existing AI solutions was identified as a primary challenge of AI in cardiovascular care by participants. They indicated that the transparency of AI models is key for legal and liability protection and existing AI models in cardiovascular care do not offer sufficient explanation of their decision-making process and functionality. Explainability is also key for regulatory compliance and entry into healthcare. Hence, the existence of black-box models does not spark confidence among clinicians and regulators, which results in slower adoption in clinical practice.
- Ethical Challenges: Participants highlighted fairness in data collection, lack of accountability, and vagueness of responsibility as the primary ethical challenges faced by AI in cardiovascular care. They indicated that training datasets used in the development of AI solutions are not representative of the target populations. Hence biases are replicated in the results of these models. They also believe that questions on responsibility pose a challenge to the integration of AI in clinical care. While traditional medical care places responsibility primarily on clinicians when negative outcomes occur, this is not clearly outlined when an AI algorithm is primarily responsible for the decisions made. Hence, reluctance exists amongst clinicians on the use of AI in cardiovascular care. Participants also believe that AI developers are not transparent enough in reporting the developmental process and the steps leading to the creation of the final product. They believe that this lack of accountability diminishes the trust of clinicians and their willingness to adopt AI solutions in cardiovascular care.
- Change Management Challenges: Participants acknowledged the lack of quality change management plans and the lack of medical and economic impact analysis for AI solutions. They believe that the integration of AI in cardiovascular care is significantly slowed as healthcare institutions lack a comprehensive and coherent plan on how to integrate AI in cardiovascular care. One of the key aspects of change management plans is the impact analysis. Participants indicated the lack of in-depth impact analysis that offers genuine insights into the effect of AI solutions on clinical care and administration. This lack of coherent change management plans creates hurdles for medical institutions and governments in the integration of AI solutions in cardiovascular care.
- Acceptance Challenges: Participants acknowledged that acceptance by healthcare professionals and patients was a key factor in the integration of AI in cardiovascular care. They believed that some skepticism exists among healthcare professionals on the use of AI in cardiovascular care due to the reluctance to change established ways of working, concerns about job security, and lack of trust in AI solutions. It is important to note that participants do not acknowledge this as being the majority opinion but they acknowledged that the minority opinion still impacts the integration of AI in cardiovascular care.
[Table 1]
Quantitative Results
Description of participants
Of the 134 individuals who initiated the survey, 94 (70.1%) completed the survey in its entirety. However, three participants (3.2% of completed surveys) provided incorrect responses to the screener and were subsequently excluded from the analysis. The decision to exclude three participants who provided incorrect responses to the screener was made to maintain data integrity and ensure the reliability of the study findings. Thus, a total of 91 valid responses were obtained for analysis. Although the majority of participants (n= 55, 60.4%) were doctors, the responses also included nurses, medical researchers, health IT specialists, hospital administrators, medical assistants, and cardiovascular technologists. The distribution of participants across different job descriptions is visualized in Figure 1. The majority of participants were from Europe (n = 56, 61.5% of valid responses), followed by Africa (n = 24, 26.4%), Asia (n = 7, 7.7%), and North America (n = 6, 6.6%). Notably, no responses were received from Australia or South America.
Participants were also asked about their frequency of using AI for work. The responses indicated that 42.6% (n = 38) reported never using AI, 23.6% (n=21) reported monthly usage, 16.9% (n=15) reported weekly usage and another 16.9% (n=15) reported daily usage.
- Data-related Challenges: In terms of data access, the majority of respondents (51.5%) reported encountering difficulty (45.3% found it difficult, while 6.2% found it very difficult) in accessing cardiovascular health data for AI analysis and interpretation. 1.6% found data access to be very easy and 12.5% found it to be easy. 34.4% of respondents maintained a neutral position on data access. A pairwise comparison with Kruskal Wallis test indicated the presence of a statistically significant correlation [H(4) = 9.10, p = 0.028] between the frequency of use and the opinion regarding data access. Further exploration with post-hoc Mann-Whitney test with Bonferroni correction indicated the presence of a statistical difference (p=038) in data access views between daily AI users and monthly AI users. Figure 2 shows a box plot visualization in which daily AI users consistently rated data access as more challenging, compared to the more varied responses of monthly AI users. On the compatibility of different data sources for CVD analysis, 36.5% indicated that data sources for CVD analysis were incompatible (4.8% found them highly incompatible and 31.7% found them incompatible). 36.5% maintained a neutral position while 23.8% and 3.2% found data sources for CVD care to be compatible and highly compatible respectively.
- Regulatory Challenges: To measure the clarity of existing regulations for AI solutions in cardiovascular care, participants were asked about the transparency and comprehensibility of existing regulations. The majority of respondents (55%) indicated that existing regulations were not transparent or comprehensible (41.7% expressed disagreement and 13.3% expressed strong disagreement). 35% maintained a neutral position and 10% agreed that existing regulations were transparent and comprehensible. Notably, no participant expressed strong agreement. Participants were also asked about the adequacy of the current regulatory framework in facilitating the safe and efficient implementation of AI solutions in cardiovascular care. 40.7% and 5.1% expressed disagreement and strong disagreement with the notion that existing AI regulations were adequate. 37.3% remained neutral and 16.9% expressed agreement. Notably, there were no participants who strongly agreed.
- Infrastructural challenges: Regarding organizational readiness for dealing with the introduction of AI in cardiovascular care, 24.7% of respondents indicated that their organizations were not at all equipped. 39.3% and 30.2% of respondents considered their organizations to be slightly equipped and moderately equipped respectively. Only 5.6% indicated that their organizations were very equipped. Notably, none of the participants regarded their organizations as extremely equipped. The Kruskal-Wallis test showed a significant difference [H(3)= 7.893, p= 0.048] in organizational readiness based on geographical location. Post-hoc analysis using Mann-Whitney test with Bonferroni correction indicated a marginal difference (p= 0.063) between the respondents in Europe and Africa. The boxplot visualization in Figure 3 shows that respondents from Africa consistently indicated lower levels of organizational readiness compared to the varied responses of respondents in Europe.
- Knowledge Challenges: The question asking participants to rate their knowledge of AI showed less than optimal levels in the majority (68.2%) of participants ( 3.3% had very poor knowledge, 18.7% had below-average knowledge, and 46.2% had average knowledge). The frequency and distribution of participants’ knowledge levels are shown in Figure 4. Additionally, participants were asked if they would be willing to take courses to improve their knowledge of AI. 78% of respondents were willing to take courses to improve their knowledge of AI, 7.7% were not willing to take additional courses and 14.3% were unsure.
- Transparency Challenges: On the importance of transparency, 82.4% of respondents indicated that AI systems needed to offer some form of explainability in their decision-making process (50.5% expressed agreement and 31.9% expressed strong agreement). 12.1% maintained a neutral position while 4.4% and 1.1% expressed disagreement and strong disagreement respectively. Additionally, on the impact of transparency on user trust, the majority of respondents (62%) indicated that they needed to understand the clinical decision-making process of AI systems to trust their recommendations (36.3% agreed and 26.4% strongly agreed). Conversely, 20.9% of respondents did not perceive understanding the decision-making process of AI systems as impactful on their trust in their recommendations. Specifically, 15.4% disagreed and 5.5% agreed. 16.5% of respondents were neutral.
- Ethical Challenges: On fairness, participants were asked the importance of AI solutions considering patient diversity in cardiovascular care. The majority of respondents (52.2%) considered it extremely important for AI solutions to consider patient diversity. 18.9% considered it to be important and 18.9% maintained a neutral position. 7% and 3.3% of respondents considered it to be slightly unimportant and not important at all respectively. Subsequently, participants were asked if existing AI solutions adequately addressed the diversity of patient population. 10.2% of participants responded in the affirmative, 38.6% maintained a neutral position and 51.2% responded in the negative.
- Change Management Challenges:3% and 34.2% of respondents considered having an organizational plan for the integration of AI in cardiovascular care to be important and extremely important respectively. 19.7% of respondents took a neutral position. 10.5% and 5.3% indicated that having a plan was slightly unimportant and not important at all respectively. When asked about the quality of their organizational plans for the integration of AI in cardiovascular care, none of the respondents indicated that their organizational plans were fully optimized and comprehensive. 6.6% and 19.7% indicated that their organizational plans were well-developed and moderately developed respectively. 34.2% indicated that their organizational plans were poorly developed while 39.5% indicated that their organizational plans were non-existent.
- Acceptance Challenges: Participants were asked several questions to gauge their perspective of AI. Figure 5 contains a comprehensive overview of the distribution of responses.