We have divided our results section into prominent areas of the patient engagement continuum, namely participant demographics, the need for patient engagement, patient recruitment, timing of engagement, engagement methods, patient education/training, the overall engagement process, and evaluation of the engagement. Please see Table 1 for representative participant quotes for each section.
Table 1
Representative participant quotes by engagement theme NB: This table should be located in the Results section of this paper after: “Please see Table 1 for representative participant quotes for each section.”
| Engagement Stage | Representative Participant Quotes |
| Need for Patient Engagement | “In surveys, I also think that it's not uncommon for AI specifically, not to have a lot of patient engagement, because of the specific software and its use. So if it's diagnosing cancer and whatnot, patients wouldn't be using it, it would just be the doctor. So it's not surprising that they're not involved in that process.” “I think it's really important to have to expand the [AI] space and expand the people that are participating in it. Because with AI, as in a lot of things, it's the question of garbage in, garbage out. So if you have such a small sample [size], the information you’re basing your policies on will not be consistent with reality.” “Like, I personally, I'm not surprised, but it's very upsetting to see, you know, the majority of the participants were identified as white, because realistically, we cannot generalize that to the entire population. And it just shows that we need more race-based research focusing on black individuals, brown individuals, indigenous populations, and so on. Because at the end of the day, also, racial and ethnic minorities receive a lower quality of health care than white people. Like that's what all the studies say, right? And then that kind of ties back in with the patient engagement rollout, who are we actually reaching out to?” “And I'm just speaking out of the experience, as a patient advisor with a variety of health care settings, both in education and in hospitals, is that most of the patients participating, the patients really participating are usually very homogeneous. They have post-secondary education, they either have or have had work and are retired, they have a home, they have a certain level of income, they speak English, usually as their first language. And they're mostly white, mostly women, too. So mostly from a very, very homogeneous set of care. And it's been a struggle, because even if the organizations want to do more outreach to patients, the way the whole patient participation is set up, is sometimes not conducive to bring, for example, people who work all day and are only available in the evening, or on the weekends. Right. So there's the setup for participation also has a lot of barriers.” |
| Patient Recruitment | “I think that it should be a part of the clinics [where] every patient that comes in and is asked ‘hey, we need your input on this AI tech’. Because you need a diverse sample and people who are willingly going to want to participate in studies and focus groups they're definitely going to be similar in some way.” “And consider who is the population in Toronto, for example, there's a large majority of people whose language is not first language is not English. They we celebrate our diversity in the fact that we are in an immigrant city, that people learn English to work, they cannot read a paper. This is from my Latin American community, that's my experience, they don't know how to read or write. They don't want to rely on other people to actually take care of their health. So how are we making it fully accessible? Not only in terms of accessibility for persons with disability, which is the law, but also for those who are not health literate, who don't have the capacity to speak either official language?” |
| Timing of Engagement | “So to me, patient engagement in AI should invite the patient right at the beginning to even identify the problem: ‘what is the problem that we want to research?’ Because health care is a two sided coin, the priorities of the investigators and health care providers will be on how they can do the work best and more efficiently and more effectively for delivery. From the other side of the coin, from the patient perspective, I will be more interested in what are the barriers for me to actually access that health care, they give me the best medication prescription? Do I have the money for it? So I would like the question for research or the problem identified, to have input from people who are going to be the users.” |
| Patient Education/Training | “…having a round table with experts that are doing the actual AI, like a programmer, who does the algorithm…. Like having those high-end professions at the table as well as researchers. So if participants have a question, we get the answer right away.” “It’s up to the provider to educate, you should educate them on those things that they're not aware of, and it can take time. But that's what quality entails, you need to take more time to give good results, basically.” |
| Methods of Engagement | “There's not a lot of like qualitative perceptions. It's mostly through surveys of satisfaction and acceptability and AI interventions. And I think that if you want to engage the patient more, you need to talk to them.” “So I would think that a modular approach would be the best. And by modular I mean, invite a group for a three stage or three meetings, it could be one meeting in person, another meeting, online, etc. Just event, there's a variety of ways to participate. So you addressed a variety of communication styles and learning styles, you also have the opportunity to see familiar faces to feel that this is a safer space, I have not been harmed, I feel like impressed this group, in make it purposefully diverse.” “You could have small groups in libraries…libraries are a really good way to reach people and have computers so people who can't afford to have a phone or computer can also access it that way.” |
| Process of Engagement | “Also, I think that having funding for to say to the person, you know, if it is in person, we're going to give you lunch or dinner, we're going to pay for your transportation, we're gonna recognize your one hour commitment with this much money. So then persons may say, ‘yeah, I can skip my gig, because I can go to this place and still not lose income.” “So how are we making it fully accessible? Not only in terms of accessibility for persons with disability, which is the law, but also for those who are not health literate, who don't have the capacity to speak either official language? And so are we allowing ourselves a budget in timeframe to prepare the patient and inform the patient so then they can feel more appropriate? Because otherwise they just feel like oh, this is too educated for me this is too English for me and they self you know, self-eliminate before even participating” “But I also want to bring the concept of upstream. So, the researchers need to educate themselves, they need to also advocate with their schools, their faculties, because right now, today, all the researchers are graduating, are still being taught, from the perspective that health research is ‘how can we fix the problem?’ And when I show up at my doctor's office, I am the problem. And they want to fix me, not the disease, they want to fix me. And it's like, no, I have a chronic illness. It's never going to be fixed. So how about we both have a chat, and we learn how to manage the disease in a way that benefits my day to day living? So, they are still looking at patient equals disease equals deficit, and we need to flip that conversation. Because patients are empowered, I'm in charge of my life, not my doctor. So then how can my doctor help me with that? Right, so we need to go upstream, educate the researchers, but the researchers also need to look at their schools or an inclusive type of education.” “The second part to that, for that to be effective would be to recognize the patient's knowledge and experience as a skill. I said, my experience of disability is a skill. It took me 30 years to know everything I knew about my disease, I didn't choose this, but I have to have that knowledge. That's my skill. So when I participate in I bring that knowledge, I think it should be recognized because when I invite another professional that is going to bring their hard earned knowledge, it's usually recognized.” “I was invited to create a workshop for PhD and master students who were actually doing some research about diseases and treatments, and they have never, ever talked to a patient. So they were looking at the patient as an object, of ‘how can I fix this disease’, but they didn't know how that disease impacts the life of the person. Will the person be able to adopt and embrace your research solution? So I think the culture of inclusion of seeing the patient as a true partner in their health care, it's a learning curve.” “First, contact the general population, educate them, and then ask for the participation. So a few of those might want to participate when the research is done. So you go back to the patients who were engaged and say, ‘this is the publication, do you have anything else to add?’, if they have helped you write the paper, acknowledge them as co authors. Go back to the community and tell them listen, we did this research, this is how it's going to be published. And the next time around the community will embrace and say, yes, we want to participate, because we're seeing, they see us not just at the token, there's not an extractive process of our knowledge, but it's actually enriching it back, when they're finished, they bring back that knowledge.” “For everyone, the researchers, but also the patients that you engage, to educate them in an anti-oppression framework. So then, with my participation, I'm not just saying Ladies and gentlemen and maybe harming a person who was not binary in their identity. And that's extremely important that as a patient, my participation with all the love that I bring is not harming others in the process.” |
| Evaluation of Engagement | “I think it's a very difficult thing to do, because there's not a final product. So the engagement process is a continuum. So I would say that having the goal in mind and say, for this particular research, treated as a quality improvement project.” “I feel like it's almost impossible to know that up front, you'd have to like, incorporate the policies that were developed from the results, and then see if it improves health after a few years of actually implementing those policies.” “So different, like multiple kinds of opportunities to say and share what's happening with me as a patient. I think that's important.” “Other than an evaluation at the end of the project, to allow the patients for very honest and open feedback? Did they feel engaged? Did they feel that the knowledge of the subject in that particular project transformed? Like they have gained or learned something from it? And have they seen that the input, not just my input, but the input that the group was focusing more towards was actually considered?” “I guess a short answer: good quality patient engagement is: a) include the patient right from the start and b) make it fully accessible including time/funding to properly do it” |
Participant Demographics
We recruited 30 participants across 4 patient focus groups, ranging from 5–8 participants per focus group. Please find a detailed description of participant demographics in Table 2. In summary, 67% of our participants identified as female and the average patient age was 35 years old. We found that most participants identified as White, Black or South-Asian. Most participants resided in the Greater Toronto Area in Ontario, with a couple of participants from British Columbia. Most participants have completed college/university level degrees and self-identify as being moderately knowledgeable about AI. None of our participants expressed having worked within the AI field prior to attending the focus group. Approximately a third of participants self-identified as experiencing chronic illness and a third of participants expressed having accessibility needs.
Table 2
Participant Demographic Data NB: This table should be located in the Results section of this paper after: “Please find a detailed description of participant demographics in Table 2.”
| Participants |
| Characteristic | # (%) of Participants (N = 30) |
| Age, mean (range) | 35 (18–73) |
| Gender | |
| Male | 10 (33) |
| Female | 20 (67) |
| Trans Female/Trans Woman | 0 (0) |
| Trans Male/Trans Man | 0 (0) |
| Two-Spirit | 0 (0) |
| Race/Ethnocultural group | |
| Asian – South (India, Pakistan, Sri Lanka) | 8 (27) |
| White/European/North American | 6 (20) |
| Black – Africa and Caribbean | 6 (20) |
| Asian – East (China, Japan, Korea) | 3 (10) |
| Asian – Southeast (Vietnam, Philippines, Malaysia) | 2 (7) |
| Indigenous (Inuit, Metis, First Nations) | 0 (0) |
| Hispanic/Latin | 1 (3) |
| Middle Eastern (Iran, Egypt, Lebanon) | 3 (10) |
| Prefer not to answer | 1 (3) |
| Difficulties making ends meet at the end of the month? | |
| Yes | 7 (23) |
| No | 23 (76) |
| Highest level of education | |
| Some grade school | 0 (0) |
| Some high school | 0 (0) |
| High school | 1 (3) |
| Some college/university | 8 (27) |
| College/university | 21 (70) |
| Post-graduate | 0 (0) |
| Perceived AI knowledge | |
| Not knowledgeable at all | 2 (7) |
| Slightly knowledgeable | 7 (23) |
| Moderately knowledgeable | 16 (53) |
| Very knowledgeable | 3 (10) |
| Extremely knowledgeable | 2 (7) |
| Currently experiencing chronic illness | |
| Yes | 8 (27) |
| No | 13 (43) |
| Prefer not to answer | 9 (30) |
| Self-identified accessibility needs | |
| Yes | 8(27) |
| No | 13 (43) |
| Prefer not to answer | 9 (30) |
The Need for Patient Engagement
To start the focus group discussion, participants were presented with recent systematic review findings on the prevalence of patient engagement in AI development in health care. From these findings, participants expressed a mixture of surprise and anticipation. Some participants described surprise that although patient engagement is well-known to be beneficial, we are still so far off from doing it well in the AI space. Other participants were not surprised, yet still upset, by the lack of patient voice. Nonetheless, the majority of participants stated patient engagement is critical for inclusion in AI development processes, while highlighting their expectation of patient engagement being a new standard in all AI development, as with any other field.
A common theme when discussing the need for patient engagement was the importance of diverse patient representation across social determinants of health and background (e.g., low income, racialized, English as second language, etc) for those who are engaged, and that the lack thereof thus far contributes to health inequities and low generalizability. One participant stated: “Because with AI, as in a lot of things, it’s the question of garbage in-garbage out. So if you have such a small sample [of patients engaged], the information you’re basing your policies on will not be consistent with reality.” Another participant stated: Because at the end of the day, racial and ethnic minorities receive a lower quality of health care than White people. Like that’s what all the studies say, right? And that kind of ties back in with patient engagement roll-out, who are we actually reaching out to?”.
A secondary theme that emerged was the idea that if the AI technology is meant to serve the physician in doing their tasks, such as a diagnostic tool, then perhaps patients do not need to be engaged in those applications. However, it was also mentioned that physicians should serve as the bridge between the AI technology and patients, as they are still by proxy end-users.
When discussing the need for patient engagement in AI applications, concerns with respect to AI integration in health care were expressed. Specifically, the removal of the humanistic component of medicine, fears of data privacy and storage, the lack of consenting processes and patient notification pathways, and the worsening of health inequities through biased algorithmic design/data. However, many participants highlighted patient engagement as being a method of addressing patient and community needs in addition to it being used as a tool to foster acceptability of AI interventions. One participant highlighted this here: “I think that good patient engagement in general can help build trust, I guess, with the health care providers and just with the health care system itself. So I feel like when you're introducing something new, such as AI, people are kind of more willing to, if not accept then even just listen and kind of understand what's going on.”
Recruitment
Participants discussed two key components of where, how and who to recruit for engagement in AI application development. A recurring theme was performing recruitment in primary care clinics, rather than hospitals, as a method of engaging a large representative group of patients in addition to leveraging primary care physicians’ longitudinal relationships with their patients. Another major theme was the need for recruitment in spaces where racialized populations are located geographically, and through community organizations that patients trust, such as churches or neighborhood community centres. In order to engage intergenerational perspectives, some suggested the need for recruitment in long-term care homes to engage older adults.
When discussing how to recruit engaged patients, participants placed emphasis on having multiple recruitment avenues, including information boots in clinics and hospitals to have in-person recruitment, as well as using social media specifically to recruit younger generations and those digitally connected. The majority of participants, particularly living in Toronto, urged recruitment materials to be translated to commonly spoken languages to ensure that researchers are not excluding non-English speakers. It was particularly important in the field of AI, as this field uses complex language and terminology that patients with English proficiency as a second language may still have difficulty interpreting.
In terms of who should be engaged, participants emphasized the need for both patients and their caregivers to be recruited. Further, some participants highlighted the need for interdisciplinary collaboration with simultaneous recruitment of developers, programmers, researchers, physicians and policymakers.
Timing of Engagement
There was a resounding emphasis on the need for patient engagement from the very beginning of AI development at problem identification and prioritization stages. As one participant stated: “So to me, patient engagement in AI should invite the patient right at the beginning to even identify the problem: ‘what is the problem that we want to research?’ Because health care is a two sided coin, the priorities of the investigators and health care providers will be on how they can do the work best and more efficiently and more effectively for delivery. From the other side of the coin, from the patient perspective, I will be more interested in what are the barriers for me to actually access that health care or technology.” Some participants also highlighted that early engagement would provide the opportunity to save technological resources and prioritize future developmental iterations appropriately. A smaller minority of participants noted that the timing of engagement may be dependent on the end-user of the intervention itself, and in instances where physicians are end-users, prioritizing physician engagement during these stages and during later stages of development consulting patients. Importantly, the majority of participants agreed that providing choice to patients in terms of which stages and to what degree they would like to be engaged in the development process is essential, as every patient has a different agenda, ability, and interest.
Patient Education and Training
Participants highlighted AI education as being a critical component to patient engagement within this field so that they may meaningfully engage. Although some participants noted that patients do not need to know everything about AI, researchers and AI developers should determine which level of basic understanding is fundamental for quality patient contribution.
Few participants reported having worked or learned about AI prior to completing our patient educational module. As such, the novelty and complexities of AI pose a challenge and may have implications on participation. It was a common participant worry that highly educated patients with higher income would be more involved in AI technology than those with less education and lower income, creating a class divide in patient engagement. Another concern was that older patients may be hesitant to participate if they are not comfortable or savvy with technology. Importantly, one participant mentioned that having patients with very little AI knowledge to start is important, as it is representative of the general public.
On the topic of whose responsibility it is to educate patients on AI, one participant highlighted the role of physicians as being direct patient educators. It was also suggested to have interdisciplinary experts involved in future patient engagement team training, in order to answer patients questions about AI and further their understanding in-person, in real time. Importantly, a common theme was that patient education takes time, but “that is what quality entails, and we must take the time and energy necessary.”
Participants enjoyed our educational module and appreciated the learning. From our feedback received on the module, we found that it took participants 25–35 minutes on average to complete the module, with a global rating of there being slightly too much content. The most challenging reported sections were those on AI methodologies, namely machine learning, natural language processing, and deep learning. In contrast, users generally found the ethics section the easiest. As a whole, participants rated the difficulty of the module as neither too easy nor too difficult. Areas highlighted for future improvement include the addition of videos and enhanced case studies. In terms of strengths, participants appreciated the use of images, glossaries, and real-life examples of AI. We found that age nor educational attainment impacted participants' self-rating of AI knowledge prior to completing our educational module.
Methods of Engagement
Participants emphasized the need for starting the patient engagement process with patient partners in mind, and choosing and creating engagement methodologies based on patient needs and preferences, while balancing feasibility concerns. It was well agreed upon that regardless of the situation, there should be a core group of patients engaged in a project from start to finish with a series of longitudinal meetings and continuity at each step to gain honest feedback and develop trust amongst patient partners. Other engaged patients may be involved in specific steps of the project, such as testing out an AI prototype.
Having a variety of engagement modalities was found to be an important topic in the focus groups, with some believing that both surveys and focus groups should be implemented as ways for patients to engage with AI applications. Participants often discussed the pros and cons to focus group and survey methodologies, specifically as it concerned sample size. Some patients expressed concerns with capturing a breadth of patient perspectives and experiences through focus groups, while others favored the quality of data to be had through focus groups in contrast to surveys: “There's not a lot of like qualitative perceptions in patient engagement in AI. It's mostly through surveys of satisfaction and acceptability and AI interventions. And I think that if you want to engage the patient more, you need to talk to them [more in depth]. Others discussed that the method of engagement is contingent on the type of AI application itself, with some mentioning that focus groups may be more appropriate in the setting of trialing the intervention/product.
For the location of patient engagement, participants frequently mentioned the need for both on-line and in-person avenues for engagement. Mentioned in-person locations included sites like community centers and libraries that are easily accessible for patients, specifically as it pertains to patients without access to electronic devices. The majority of participants believed a mix of online and in-person meetings allowed teams to address a variety of communication and learning styles, and provide opportunities to have a familiar, in-person place that feels safe and comfortable for engaged patients.
For knowledge dissemination of patient engagement results, a similar emphasis on a multi-method approach was proposed. Some participants proposed researchers and AI developers send on-going updates of study progress and the usage of summary documents to be sent to all patient partners and study participants. Other participants found that town halls may assist in being able to engage not only the study partners and participants, but the larger community as a whole. Similar to patient engagement recruitment, participants suggested a mixture of formal (email) and informal (social media) pathways for knowledge dissemination, with the emphasis on accessibility and language translation, as needed.
Process of Engagement
We define the “process of engagement” as enablers for satisfactory patient engagement experiences. Participants discussed these enablers in three categories: patient-specific principles, provider-specific principles, and combined patient-provider principles. “Providers” include clinicians, researchers, and others on the AI application team.
Patient-Specific Principles
For patients, important components for satisfactory patient engagement experiences were compensation, attentiveness to competing patient commitments, and accessibility.
For compensation, participants discussed the importance of AI teams allocating sufficient funds from the start for their patient partners and participants. Specifically, funds that cover potential lost wages and transportation costs, as well as funds for their participation time and energy. Providing a meal at team meetings was another form of compensation. Additionally, participants found it important that researchers are mindful of the other commitments patients may have with respect to their work or personal lives, and how this may affect their capacity to participate in engagement.
Another common topic of discussion among the focus groups was the importance of accessibility throughout the patient engagement process. Specifically, ensuring that different mediums of engagement have factored in the accessibility needs of participants both from the perspective of patients with a physical and/or mental disability, and from a health literacy perspective. It is important for AI teams to budget in time and funds “to ensure patients do not self-eliminate before even participating.”
Provider-Specific Principles
Participants discussed important provider-specific enablers for improving the patient engagement experience. First, provider education was highlighted as an area for continuous development, specifically so that clinicians, researchers, and AI developers are educated on more upstream methods of engagement to garner representative patient sampling and the incorporation of diverse perspectives and experiences, as well as being educated on what meaningful engagement looks like. This was followed by a common theme of providers understanding how to develop and nurture community partnerships; not only looking to patients, but also to communities to assist in research problem identification, recruitment, and knowledge dissemination. Community engagement was highlighted as a way to gain trust with end users of the AI application, especially in communities that are notably marginalized or at risk of harm of AI applications. They discussed how community members who were engaged in the project also bring back a unique skillset and knowledge base that they can share with their community.
Second, many participants expressed the importance of researchers validating their patient knowledge as a skill, particularly in the environment of developing technology which seeks out to improve their lived experience of illness. “My experience of disability is a skill. It took me 30 years to know everything I knew about my disease, I didn’t choose this, but I have to have that knowledge. That’s my skill. So when I participate in I bring that knowledge, I think it should be recognized because when I invite another professional that is going to bring their hard earned knowledge, it’s usually recognized.” Another participant stated: “Some researchers have never, ever talked to a patient. So they were looking at the patient as an object, of ‘how can I fix this disease’, but they didn't know how that disease impacts the life of the person. Will the person be able to adopt and embrace your research solution? So I think the culture of inclusion of seeing the patient as a true partner in their health care, it's a learning curve.”.
Additionally, participants discussed the importance of adequate acknowledgment of contribution to the AI project of patient partners, not only in financial compensation, but also in academic authorship or recognition in reports and presentations. Participants all agreed that there was no room for tokenistic engagement where patients were included as a checkmark.
Combined Patient-Provider Principles
Participants highlighted that empathy and active listening were critical for patients and providers to work together in the engagement. Both patients and providers alike were discussed in terms of their importance in the engagement process, with providers initiating these opportunities with patients, and patients seeking out these opportunities themselves, as well. Throughout the patient engagement process, participants discussed the importance of decision-making power, such that patients being engaged feel and believe that they are able to enact change in AI development through the proposed engagement pathways. Anti-oppression frameworks were reported.
Evaluation of Patient Engagement
Participants discussed how the topic of patient engagement evaluation is challenging, given the nature of patient engagement being an improvement continuum of long-term patient health outcomes and there may not be a final product in all cases. While some participants stated evaluation should come well after the implementation of the AI application, it was important to the majority of participants that there be incorporation of patient feedback at multiple time points throughout the longitudinal project, and not just at the end.
It was also discussed that the term “successful engagement” is difficult to define, because success will look different to different patients and teams. Importantly, participants reflected on the idea that effective patient engagement can be achieved only when the values of the project align with those of the participants, namely as it concerns research transparency and authenticity throughout the process. On the matter of markers of effective patient engagement, one participant suggested using the patient's sentiments of inclusion, adequate knowledge to participate, and a sense of self-improvement and gain. Specifically, if they feel like they are being engaged well, if they feel prepared to engage and if they have learned/gained anything throughout the process. For engagement of community organizations, it was also suggested to seek feedback from these organizations to foster a long-term relationship of engagement and trust.