Extending the Population Health Workforce through Service Learning Internships during COVID

doi:10.21203/rs.3.rs-445100/v1

Download PDF

Research Article

Extending the Population Health Workforce through Service Learning Internships during COVID

https://doi.org/10.21203/rs.3.rs-445100/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

This report arises from the intersection of service learning and population health at an academic medical center. At UCSF, the Office of Population Health and Accountable Care (OPHAC) employs health care navigators to help patients access and benefit from high-value care. In early 2020, facing COVID-19, UCSF leaders asked OPHAC to help patients and employees navigate testing, treatment, tracing, and returning to work protocols. OPHAC established a COVID hotline to route callers to the appropriate resources, but needed to increase the capacity of the navigator workforce. To address this need, OPHAC turned to UCSF’s service learning program for undergraduates, the Patient Support Corps (PSC). In this program, UC Berkeley undergraduates earn academic credit in exchange for serving as unpaid patient navigators. In July 2020, OPHAC provided administrative funding for the PSC to recruit and deploy students as COVID hotline navigators. In September 2020, the PSC deployed 20 students collectively representing 2.0 full-time equivalent navigators. After training and observation, and with supervision and escalation pathways, students were able to fill half-day shifts and perform near the level of staff navigators. Key facilitators relevant to success reflected both PSC and OPHAC strengths. The PSC onboards student interns as institutional affiliates, giving them access to key information technology systems, and trains them in privacy and other regulatory requirements so they can work directly with patients. OPHAC strengths included a learning health systems culture that fosters peer mentoring and collaboration. A key challenge was that, even after training, students required around 10 hours of supervised practice before being able to take calls independently. As a result, students rolled on to the hotline in waves rather than all at once. Post-COVID, OPHAC is planning to use student navigators for outreach. Meanwhile, the PSC is collaborating with pipeline programs in hopes of offering this internship experience to more students from backgrounds that are under-represented in healthcare. Other campuses in the University of California system are interested in replicating this program. Adopters see the opportunity to increase capacity and diversity while developing the next generation of health and allied health professionals.

service learning

population health

COVID-19

telehealth

undergraduate

navigation

internship

This case study reports on a collaboration that represents the intersection of two major trends: service learning in education (1-4) and population health in health care (5, 6). Service learning programs involve students in experiential learning outside of classroom settings. Population health programs target an entire population or panel of patients and attempt to address their health and wellness in an integrated and holistic fashion. The past decade has seen a steady increase in the proportion of patients cared for under accountable care or other risk sharing programs. Such programs create alignment for all parties for the provision of high quality and affordable health care, and create opportunity for health systems to innovate with new models of care delivery.

In early 2020, population health programs faced an influx of demand from patients who were potentially exposed to coronavirus infection and who needed help with testing, treatment, and tracing services related to COVID-19 (7). Population health programs needed to expand their capacity to address this demand. At the same time, health care delivery systems were dealing with a reduction in revenue, and many had instituted hiring freezes.

Meanwhile, undergraduate institutions have launched service learning programs to ensure that students are exposed to high impact practices such as internships (8-11). Students benefit from internships and other experiential learning opportunities because they allow students to apply knowledge, gain skills, interact with role models and mentors, and work on interprofessional teams (2, 12-14). Internships also present challenges, as well as opportunities, in terms of equity and access (15).

In principle, service learning programs can extend the workforce capacity of population health programs, including during a surge in demand due to a pandemic. This case study describes one such innovative collaboration at an academic medical center where students helped increase the capacity of a COVID hotline.

The setting for this case study was the University of California, San Francisco. The organizational actors included UCSF’s Patient Support Corps (PSC) and the Office of Population Health and Accountable Care (OPHAC). The key participants included pre-health students serving in the PSC; OPHAC leaders and staff; and UCSF patients and employees exposed to coronavirus infection and concerned about obtaining testing, treatment, tracing, and return-to-work services relevant to COVID-19.

We now briefly describe the case study participants before describing the innovative collaboration that emerged to address the need for increased hotline capacity.

2.1 University of California, San Francisco

The University of California, San Francisco is a university focused exclusively on health. It includes schools of medicine, nursing, pharmacy, and dentistry; as well as UCSF Health, a health delivery system. UCSF has over 26,000 employees and sees 45,000 hospital admissions and 1.7 million outpatient visits annually.

2.2 The Office of Population Health and Accountable Care (OPHAC)

In 2015, UCSF established its Office of Population Health and Accountable Care (OPHAC) to provide high-value care to defined populations of patients. In accountable care arrangements, public and private payers agree to share financial risk with a delivery system so that incentives are better aligned to focus on quality, not volume, of care. OPHAC employs 28 licensed professionals and 15 unlicensed navigators who provide care to 100,000 covered members. Along with analysts and other staff, this team helps patients access care related to cancer screenings, chronic obstructive pulmonary disease, depression, diabetes, end-stage renal disease, heart failure, hypertension, immunizations and other acute and chronic conditions.

In early 2020, in response to the pandemic, UCSF’s Chief Medical Officer (author JA) tasked OPHAC with navigating patients and employees to established pathways for testing, tracing, treatment, and return to work. This led OPHAC to establish an employee and a patient-facing COVID telephone hotline. The volume of calls to this hotline quickly absorbed the capacity of existing staff navigators, who were redeployed from other population health assignments. Meanwhile, UCSF had implemented a hiring freeze. In hopes of expanding navigator capacity, OPHAC’s director, author GI, responded to an email from UCSF’s Chief Medical Officer (author JA), introducing OPHAC to author JB, director of the Patient Support Corps. In view of the COVID pandemic, JB had written to JA inquiring whether UCSF’s pandemic response team could make use of student interns as workforce extenders.

2.3 The Patient Support Corps (PSC)

The Patient Support Corps is a service learning program in which pre-health students serve as unpaid patient advocates, health coaches, and navigators while earning academic credit. In 2012, author JB and colleagues, championed by author JA, established the program with a competitive planning grant from the Center for Health Quality and Innovation in the University of California Office of the President. The planning grant expanded a predecessor program that trained paid post-baccalaureate student interns in the Breast Care Center to be patient advocates for one day a week as part of a job enrichment initiative (16). That predecessor program was recognized for its innovation in 2009 by the Mayo Clinic Center for Innovation (17) and by the United States Department of Health and Human Services (18). The US Agency for Healthcare Research and Quality featured it as one of the first 100 programs in its Innovations Exchange, with an evidence rating of “strong.” (19)

The 2012 expansion established an affiliation agreement between UCSF and UC Berkeley so that the Patient Support Corps could deploy undergraduate students as workforce extenders at UCSF. The affiliation agreement specified terms to address the concerns of UCSF’s clinical sites (e.g. training, risk management, insurance, security, confidentiality) and UC Berkeley’s educational enterprise (e.g. academic content, academic credit) along with other dimensions of inter-organizational cooperation. Since 2012, the Patient Support Corps has trained and deployed 137 undergraduates earning academic credit through UC Berkeley’s Undergraduate Research Apprentice Program.

Until the 2020 pandemic, UCSF’s service learning program, the Patient Support Corps, deployed student interns as patient advocates, coaches, and navigators in adult ambulatory care clinics. After originating in the Breast Care Center in 2003, in 2012 the program expanded to other adult cancer services, and from there to other adult specialty care services such as orthopedics.

In 2020, the director (author JB) began a series of discussions with OPHAC leaders and staff about adding Population Health to the roster of clinical sites that host student interns. The rationale was to increase the capacity of Population Health’s navigator workforce using students as workforce extenders.

2.4 UCSF Patients and Employees Exposed to Coronavirus Infections (Hotline callers)

The purpose of the Population Health collaboration with the Patient Support Corps was to better serve patients and employees exposed to Coronavirus infections. The institution directed patients and employees to call the COVID hotline to access testing, tracing, treatment, and return-to-work resources. OPHAC physicians authored branching logic instructions for hotline staffers to administer in guiding callers to the appropriate resources (20, 21). The instructions took the form of a base set of flowcharts, supplemented by a daily bulletin with modifications. Authors MH, HS, TT, GT, and FL made changes as new information emerged about the virus, testing, tracing, and treatment. The daily bulletin also provided guidance about specific scenarios that weren’t anticipated by the original instructions.

3.1 Program Plan

In July 2020, OPHAC signed a memorandum of understanding with PSC, providing one year of financial support for the administration of the Patient Support Corps’ recruitment, onboarding, training, deployment, and supervision of students. The PSC agreed to provide up to 15 students working half-day shifts on the COVID hotline. The agreement also foresaw that OPHAC would evaluate whether students could be deployed as workforce extenders for other population health tasks and assignments. Both sides entered the agreement hoping that the arrangement would lead to OPHAC becoming a long-term clinical site hosting PSC student interns.

Author JB articulated a program plan for the new internship. The program plan consisted of five sub-plans: the strategic direction (vision, purpose, mission, values, goals); the service delivery plan (how callers would interact with student navigators); the operational plan (what resources students would access to deliver services); the evaluation plan; and the financial plan (22).

For the service delivery plan and operational plan, we relied on two principles of lean design: prototyping and iteration (23). The prototype phase consisted of having two students act as trailblazers during summer 2020, with the plan to expand the number in Fall 2020 based on lessons learned from the summer pilot. The iteration phase included recruitment; larger scale training for the Fall rollout; competency checking and deployment of students; supervised practice; and program expansion. Next we will describe these phases of our program rollout.

3.2 Program Implementation

We now summarize, chronologically, our program implementation. In later sections, we will offer reflections on lessons learned.

3.2.1 Initial Training and Test Deployment (Pilot with 2 students)

Author HT, a Clinical Manager in OPHAC, initially trained the two trailblazing students (authors JM and SK) on July 29, 2020. Topics included: the purpose of the hotline; the multiple workflows for patients calling regarding testing and symptom evaluation; the use of Cisco Finesse and Jabber telephony systems; and the documentation of telephone encounters on ApeX (UCSF’s electronic health record) and Qualtrics (an online survey system). Author JB recorded this initial training so that future students could access it asynchronously.

3.2.1.1 Overview of Training on Service Delivery Plan and Operational Plan

The initial training addressed the overall functioning of the COVID hotline as follows. The purpose of the UCSF COVID hotline was to answer the caller’s basic questions about COVID-19, and to provide protocol-driven telephone triage and disposition for symptomatic and/or exposed UCSF patients. OPHAC created the hotline to refer callers to appropriate levels of care and direct them to resources such as the Center for Disease Control and San Francisco Department of Public Health website.

Callers called the UCSF COVID hotline and selected if they were a patient or employee. The Cisco Finesse system routed callers to a navigator who picked up the incoming call on their Cisco Jabber softphone interface.

For students to serve on the hotline as part of their service learning internships, they needed to be onboarded, trained, and badged as UCSF affiliates. The Patient Support Corps arranged student appointments as USCF affiliates and requisitioned access to various systems via UCSF’s Information Technology department. Students configured their personal laptops to comply with UCSF’s information technology security requirements, and installed enterprise software that encrypted their devices; provisioned them for remote wiping in case of theft or loss; and facilitated access to the Cisco Finesse and Jabber software applications, as well as UCSF’s electronic health record and Qualtrics data collection platform.

Using Jabber, each student was able to transfer calls; merge two calls; place a call on hold; and message colleagues in the Jabber chat room. Navigators interacted with callers and followed scripts with branching logic created by physicians as part of a larger effort to triage patients and employees at UCSF (20, 21).

Navigators accessed the scripts on a shared Box folder. Hotline leaders updated a daily bulletin notifying navigators of changes, and changed the scripts or add new branching logic as needed, for example when a large employer contracted with UCSF for employee support. Hotline leaders hosted online meetings before the hotline opened each day to make sure all navigators were aware of changes, learned about unusual issues, and gathered feedback relevant to updating the scripts and branching logic.

Navigators documented each call in the patient’s medical record by creating a telephone encounter in the APeX system. APeX is UCSF’s implementation of the Epic electronic health record. In the documentation section of the telephone encounter, the navigator documented the patient's responses to the branching logic questions and added notes.

Some outcomes of the branching logic resulted in the navigator routing the record of the telephone encounter, either to a nurse team or a scheduling team. To route to the nurse team, in the telephone encounter interface of APeX, navigators selected a pool (group of message recipients) called "P COVID POP Hotline Escalations.” This prompted members of the nurse team to review the telephone encounter and call back the patient for further triage or to place an order for covid testing.

To route to the scheduler team, the navigator selected a pool (group of message recipients) called "VACC Support", prompting members of the scheduling team to call and schedule the patient for a video visit at the Virtual Acute Care Clinic; or in person visits at the Respiratory Symptom Clinic (RSC); or for testing at UCSF sites.

In addition to documenting their notes in the caller’s electronic health record, navigators also entered data about each call into a data collection form hosted on the Qualtrics survey system. Here, navigators entered the caller's affiliation to UCSF; the main reason for calling the hotline; if the patient was adult or pediatric; if they were an employee; the patient's medical record number; the outcome of the call; and the name of the navigator completing the form. The OPHAC team used Qualtrics’ reporting capabilities to summarize data about the calls.

3.2.1.2 Hotline scripts and branching logic

The triage scripts and branching logic resulted in the following steps. First, the navigator verified the caller’s name and date of birth to open their medical record in the electronic health record. The navigator created a telephone encounter and reviewed the patient’s encounters to determine if they were eligible to be screened on the hotline. The screening qualified UCSF patients who had seen a primary care provider in the last 3 years; had seen a qualifying specialist in the past one year; or a visit with the Screening Acute Care Clinic or Medical Acute Care Clinic (urgent care clinics).

If the patient qualified, the navigator continued with the screening process and preliminary triage questions. The navigator asked the patient about their travel exposure, contact exposure, symptoms, pregnancy status, prior testing, and ability to quarantine.

Some patients had already addressed some or all of these questions in an online symptom screener within the patient portal (MyChart) before calling the hotline (21). In those cases, the navigator reviewed the disposition and routed the telephone encounter as usual to Virtual Acute Care Clinic (VACC) Support for scheduling a video visit or in-person Respiratory Symptom Clinic visit; or to the nurse team.

In increasing order of severity, the script and branching logic provided for the following dispositions:

For callers who were non-UCSF patients inquiring about coronavirus testing, navigators directed them to contact their non-UCSF primary care provider or referred them to call the new patient access line where they could establish care at UCSF.
For callers who reported no symptoms and no exposure, navigators instructed them to follow CDC guidelines and referred them to alternative testing sites.
For callers who reported that they were exposed in a clinical setting, navigators asked for the code that the clinic gave them, and ascertained whether callers (or parents, caregivers or children) were symptomatic. Navigators then followed the scripts and branching logic and routed the telephone encounter accordingly.
For callers who reported being given the code “BCHSF Essential Caregiver Group” at Benioff Children’s Hospital San Francisco. This code indicated that the caller was an essential caregiver over 18 and wanted to stay with a child overnight but had to be tested with a negative result in order for that to be approved. Navigators routed their notes about these telephone encounters to the nurse team, which called these patients back and further triaged them.
For adult callers with no symptoms and an exposure, navigators informed them that nurses would place the coronavirus test order, and that the scheduling team would contact them to schedule the coronavirus test. In the case of pediatric patients, navigators referred the patients to pediatric video visit providers or (evenings and weekends) to an after-hours pediatric provider.
For callers wishing to schedule their coronavirus test, navigators confirmed the COVID test order and referral before transferring the patient to the scheduling team.

For symptomatic patients, the script and branching logic provided for the following dispositions;

For post-vaccine callers who reported side effects, navigators selected the "Post COVID Vaccine Sx" disposition, assigned the Reason for Call to "Post COVID Vaccine Sx", and routed the caller to the nurse clinical triage team. Side effects included fever or chills, muscle aches, fatigue, headache, or local symptoms (redness, swelling, hives).
For callers who were pregnant and reported COVID-19 symptoms, early in the project, navigators directed them to call obstetrics triage. Later in the project, navigators triaged these callers directly, based on updated scripts and logic.
For callers who reported one or more qualifying symptoms (cough, fever, sore throat, sinus congestion, runny nose, vomiting, diarrhea, or pink/red eye, trouble breathing, muscle aches, loss of taste or smell), navigators routed the telephone encounter to the nurse team and instructed the patients to wait for a call back from nurses for symptom evaluation.
For patients who reported (but were not currently experiencing) high priority symptoms, navigators routed their notes about the telephone encounter to the nurse team and marked as high priority, requesting a call back within 15 minutes. The high priority symptoms included: breathing problems, unable to swallow, unable to keep fluids down, severe weakness, fever of 104 degrees farenheit for pediatric patients, choking problem, loss of consciousness, seizure, slurred speech, change in mental status, chest pain, persistent vomiting, or severe dizziness.

The most severe or urgent caller reports resulted in a warm handoff:

For callers currently experiencing shortness of breath and/or continuous severe pain or pressure in their chest, navigators immediately transferred the call to a nurse (“warm handoff’).

3.2.1.3 Students observing navigators and being observed

Having studied the above scripts and branching logic, the first two students proceeded with practical training. On July 30 and 31, 2020, JM and SK devoted 6 hour shifts to observing an experienced navigator take calls on the UCSF COVID patient hotline. On August 1 and 2, 2020, JM and SK simulated calls and protocols with each other, alternately playing the role of caller or hotliner. During the week of August 3, JM and SK fielded calls on the hotline under the observation of experienced health care navigators for 9 hours while being given feedback on health care navigator competencies.

3.2.1.4 Formal Competency Check

The formal competency check consisted of experienced navigators, authors GT and FL, observing JM and SK take calls on the hotline via Zoom. GT and FL evaluated the students based on a competency checklist which included the measures listed below. If the intern successfully met these competencies, they were cleared to take calls independently:

Demonstrates opening Box (locates Daily Bulletin, Schedule, and most recent Algorithms [scripts with branching logic], searches Daily Bulletin with Control F)
Demonstrates proper use of Jabber (Updates status w/phone number, sends message, creates group chat)
Demonstrates use of interpreting services (finds preferred language in chart, inserts SmartPhrase at top of note)
Demonstrates transferring of call to another agent
Demonstrates where to check for a patient’s UCSF Primary Care Physician or Specialist
Demonstrates placing SmartPhrases in note
Demonstrates use of Algorithm [script with branching logic] to determine disposition
Demonstrates routing notes about telephone encounter to Schedulers with appropriate routing comments
Demonstrates appropriate use of Qualtrics for documenting the call.

After passing the competency check, JM and SK began fielding calls independently on the hotline on August 11, 2020, completing over 100 hours on the hotline through August 25, 2020.

3.2.2 Recruitment

After JM and SK implemented the summer test of student hotliners, authors JB and TW proceeded with the plan to scale up in the Fall. This required some lead time planning. Since 2013, the Patient Support Corps has recruited students in partnership with two UC Berkeley organizations. The first partner is UC Berkeley’s Undergraduate Research Apprentice Program (URAP). Each semester, URAP allows researchers at UC Berkeley and other sites to post internship opportunities on a website portal accessible to UC Berkeley students. Students then apply through the portal, submitting essays and transcripts, and requesting to be interviewed. Those accepted then sign a learning contract and register for up to three units of academic credit in the course Undergraduate Interdisciplinary Studies 192. At the end of each semester, author JB assigns pass/fail grades based on student participation and performance in the program and in written and verbal critical reflections. Thus URAP is a crucial partner in the PSC’s overall service learning endeavor, as it provides a mechanism for soliciting student applications, and registering students to earn academic credit via coursework.

The second partner in PSC recruiting is a student organization. In 2013, as part of the original program planning for the Patient Support Corps, author JB encouraged the first student participants to form an official student group. The purpose of the proposed student group would be to facilitate student participation in this and other activities related to patient advocacy and support. The first PSC students participating via URAP therefore formed UC Berkeley’s Patient Advocacy Student Group (PASG). Like other student groups, it operates within a UC Berkeley framework requiring a charter and student officers as leaders. The group now qualifies for funding and leaders submit a budget proposal each Fall. The budget is used to help students cover financial costs of participating in internship programs such as the Patient Support Corps. Pre-pandemic, these funds were often used to reimburse students for the public transit costs of commuting to work in person onsite at UCSF. During the pandemic, the group reimbursed students for the purchase of headsets and other supplies needed for telecommuting.

From the PSC’s point of view, a key function of the student group is to facilitate student access to the PSC by helping with recruitment. In that regard, each semester, the student group leaders organize the following tasks: they advertise the URAP application process as widely as possible; screen applicants; interview finalists; and recommend a slate of students for consideration by PSC leaders JB and TW.

The Fall 2020 recruitment cycle kicked off with a Virtual Calapalooza presentation on August 16. Normally the Calapalooza recruitment fair takes place in person at a central location on campus. This presentation took place via the Zoom, with follow up questions and answers over time on a social media platform called LoopChat. JM and SK publicized this presentation, and the overall PSC opportunity, via social media (e.g. Facebook and Instagram posts) and other campus networks. Over 150 students registered and attended the presentation and participated in the questions and answers session afterward. The presentation described the Patient Support Corps internship opportunities, including the most recent COVID hotline partnership.

The URAP deadline was August 31, 2020, at which point 269 applicants submitted their essays and transcripts for consideration. Author TM sent these 269 applicants a supplementary application form, which 251 completed. On September 1, 2020, JM and SK divided these 251 confirmed applications among 12 leaders of the student group with each therefore having approximately 20 applications to screen.

In consultation with JB and TW, JM and SK instructed the application reviewers to search for applicants who demonstrated the following competencies in action:

Patient centered competencies:

true humanism - caring about people as people, regardless of how different they may be;
ability to remain neutral and focused on serving and advancing the patient agenda;
ability to interact verbally with crystal clear volume and enunciation so that older or hard of hearing or non-English speaking patients have the best chance of understanding;
ability to paraphrase and summarize complex information in a linear fashion (e.g. without rambling or losing the thread).

Teamwork competencies:

ability to follow complex protocols (read carefully and follow instructions in detail);
ability to discuss and recover from inevitable errors without covering up or worrying about losing face (continuous improvement orientation);
ability to coordinate with other team members in pursuit of the mission, without worrying about who gets credit or who is noticed;
possibility of multi-year commitment (as we put so much effort into training students)

Pre-requisites included facility and flexibility with technology; and ability to type 40+ words per minute.

Application reviewers used a Google Sheets document to record their notes and rankings of applications. JM and SK identified 93 finalists to interview based on the application rankings. The reviewers gathered for a Zoom call on September 3, 2020 to agree on the finalists and create a plan for interviews.

The volume of finalists created logistical challenges for allocating interview slots. One of the student leaders collected interviewer availability through an online poll, and then assigned each interviewer a series of 3-hour shifts, comprised of fifteen minute interview slots, based on availability. Then the student leader created a new Google calendar, and used Google’s built-in Appointment functionality (available in Gsuite for Education) so that students could sign up for slots. The student leader invited each interviewer to the 15 minute slots in their shifts. Interviewers added their personal Zoom link to the calendar slot.

The student leader then emailed the calendar link to finalists on September 2, 2020, giving them 12 hours to respond. Out of 93 finalists, 92 claimed interview slots on a first-come, first-served basis before the deadline. This process resulted in some workload imbalance across interviewers, so student leaders manually reassigned a few applicants to different interviewers. Interviewers completed interviews between September 4 and September 7, 2020, again recording their notes and ratings in a Google Sheet. On the evening of September 4, the interviewers convened online to discuss their impressions and ratings. After facilitating the process to consensus, JM and SK loaded the top-ranked applicants into 40 internship positions based on matching each candidate’s availability with the internship shifts that needed to be filled. JM and SK assigned 11 of the 40 applicants into COVID hotline shifts. Nine returning student interns also slotted into COVID hotline shifts. Overall this process resulted in two students being slotted into morning or afternoon shifts for every day of the week, representing a capacity of 2.0 full-time equivalent navigators being added to the COVID hotline.

On September 8 and 9, PSC program leaders JB and TW called the applicants recommended by the student leaders to verify their suitability, offer them positions, and advise on next steps for accepting the positions. TW sent each new offeree, and all returning student interns from the previous year, the URAP learning contract and UCSF affiliate agreement for signature.

The learning contract specified the conditions for student participation, including the terms of the affiliation agreement between UCSF and UC Berkeley. Broadly, the students agreed to devote 11 hours a week to their internship responsibilities, while author JB assured them of service learning opportunities on a career ladder ranging from entry-level tasks (e.g. clerical duties) to health coaching tasks (e.g. motivational interviewing). As the faculty member in charge of the program, author JB also assured the academic component, consisting of weekly written and verbal reflections. Students learned to use the Critical Incident Technique to reflect on the lessons they were learning. Specifically, they reflected on six competencies defined by the Accreditation Council of Graduate Medical Education. These competencies are patient care and procedural skills; interpersonal skills and communication; medical knowledge; systems-based practice; practice-based learning; and professionalism.

The affiliate agreement included a Student Responsibility Statement that all students must sign, in which they agreed to follow all applicable regulations and policies, including those governing privacy and confidentiality (e.g. protecting patient information) and information technology security (e.g. using UCSF-encrypted devices for all program purposes.)

By September 11, all the offerees had accepted and signed their learning contracts and affiliate agreements. We then embarked on the task of training an additional 18 students for deployment on the COVID hotline.

3.2.3 Larger Scale Training

JM and SK led initial COVID hotline training sessions for 20 students on September 19th and September 26th 2020 via Zoom. Trainees first listened to the recording of the orientation given by HT about the purpose of the hotline, technology used, and workflows.

JM and SK then demonstrated how to field incoming calls on Cisco Finesse, use Cisco Jabber call functions, create and route a telephone encounter on the electronic health record system (ApeX), and track calls on Qualtrics.

Next, JM and SK reviewed the patient hotline workflows including the “COVID-19 Patient Triage” workflow and associated disposition outcomes, the MyChart Symptom Screener workflow, and the warm handoff protocol.

JM and SK also demonstrated how navigators use the Cisco Jabber chat room to communicate in real-time with the nurse escalations team and consult with the hotline lead navigators.

On the second training day, JM and SK explained the various types of calls received on the hotline including non-UCSF patients inquiring about testing; patients calling to schedule their COVID test appointment; patients misdirected to the hotline; and patients exposed to COVID-19 positive individuals.

Students role-played the standard workflows and practiced documenting telephone encounters and using smart phrases with pre-populated questions in a test patient record. Finally, JM and SK assigned the interns to practice the workflows with a partner and submit a recording for evaluation.

Meanwhile, JM and SK expanded from their regular one shift (e.g. week of 10/5) to two half-day shifts per week on the hotline on 10/12, 10/19, and 11/2. They invited each new student to sign up for hourlong slots within these shifts. During these slots, JM and SK opened a Zoom session on their computer, sharing their sound and screen. In this way, the trainees could observe and hear JM and SK interacting with callers. Each new student observed JM and SK for a total of 1 hour during the weeks of 10/5 and 10/12. Then JM and SK used Zoom to observe the trainees taking calls, for a total of 2 hours each during the weeks of 10/19 and 11/2. JM and SK instructed trainees to place the patients on hold when necessary in order to consult with JM and SK, who by this point were experienced enough to help address most questions. In complex cases, JM and SK could escalate questions to the Jabber chat room or message lead navigators.

3.2.4 Competency Checking and Deployment

As experienced navigators, Authors FL and GT administered competency checks to determine whether trainees were ready to field calls on the hotline independently. The competency checks were the same as administered earlier to the first two trainees JM and SK (see above).

FL and GT competency checked nine trainees the week of October 26, 2020, and approved six for independent work the week of 11/2. Since the 12 remaining trainees were not ready to field calls independently, they used their pre-assigned shifts time to practice alongside FL and GT or other experienced navigators on the hotline, either observing them or being observed. The way this worked is that trainees would post a message in the hotline navigator chat room at the beginning of their shift and ask who was available to shadow. Available navigators responded with their Zoom links, which the interns then joined to share screens and audio.

3.2.4.1 Supervised Practice during Suspension of Competency Checks

Between November 16 and December 10, 2020, lead navigators FL and GT suspended competency checks as they felt students needed more time observing and practicing. They suggested that students should shadow or be observed for 10 hours before being competency checked. JM and SK arranged for students to practice under the supervision of the existing competency checked students, and to shadow the more experienced navigators on the hotline. Interns continued this training, spending approximately five hours per week shadowing and precepting until the resumption of competency checks.

3.2.4.2 Resumption of Competency Checks

FL and GT resumed competency checking on December 11, 2020, and by December 31, 2020, 14 of the 20 interns were competency checked and able to take calls independently. By end of February, 2021, all 20 interns were taking calls independently. Through February, 2021, program records indicate that 20 interns worked 1,240 hours on the hotline after being competency checked.

3.2.5 Program Expansion

3.2.5.1 Occupational Health

In mid-December, 2020, author MH identified the need to create a system for responding to employee questions about COVID exposure, testing, tracing, treatment, and return to work. This responsibility fell under the Occupational Health realm of the Office of Population Health and Accountable Care.

Because the holidays were approaching, MH needed workforce capacity to help set up this Occupational Health response system. Author JB recruited 5 students from the COVID hotline to work with MH in Occupational Health. These students devoted 270 hours to Occupational Health between December 23, 2020 and January 21, 2021, a period corresponding to their winter break from school.

Under the leadership of MH, the students helped set up a process whereby employees could direct a voicemail or email message to Occupational Health reporting contacts and exposures; positive test results; symptoms; adverse reactions to being vaccinated; or the need for return to work orders. Occupational Health navigators (including the 5 students) would then follow scripts and branching logic to either forward the emails or route notes about the voicemails to the appropriate Occupational Health staff who could address the employee questions.

3.2.5.2 Population Health Outreach

In late 2020, the Office of Population Health and Accountable Care shifted some navigator capacity from the COVID hotline back to regular care management activities, including outreach to patients with chronic conditions. Author JB contacted the Outreach Manager to see if students might help as workforce extenders in this arena. The rationale was that the need for workforce extenders on the COVID hotline would eventually subside, so it made sense to anticipate where else students might be able to contribute. JB asked author SK to recruit one other student and work with the Outreach Manager to define a role for students in Population Health Outreach.

From January 7-11, 2021, the Outreach Manager and navigators oriented the two students to Population Health Outreach initiatives surrounding diabetes, child wellness, and hypertension. Then, between January 11 and January 27, 2021, the students engaged in supervised practice with more experienced navigators. The students learned to conduct chart reviews to identify patients due for health maintenance exams, or who might be suffering from care gaps such as missing eye exams, blood pressure checks, and blood tests (e.g. Hemoglobin A1C lab). The students also learned how to schedule patients for appointments with a primary care doctor for diabetes follow up appointments. They learned how to pend orders for blood pressure cuff, HbA1C lab, and microalbumin lab and how to pend referrals to ophthalmology for diabetic eye exam.

Between January 27 and March 3, 2021, the students worked on outreach tasks relevant to patients with hypertension. They learn how to case-find and then contact patients for hypertension follow up, documenting their outreach encounter on ApeX (the electronic health record system) and scheduling patients for appointment in the next 2 weeks with their primary care physician. They learned how to instruct patients to bring their blood pressure medications, take blood pressure readings on their BP cuff and bring their BP cuff to their appointment.

On March 3, 2021, author SK began independently performing chart review and outreaching to patients with diabetes, and the next week SK worked on screening and depression.

4.1 Lessons learned from Test Deployment

Our goal in launching with just two students was to test our program plan over the summer before scaling it up in the fall. This worked well. The students were able to practice, reflect, and provide mutual support as they mapped out how to fulfill as student navigator role on the COVID hotline. Key success factors included starting with two students who were experienced and were co-Presidents of a student group dedicated to patient advocacy. Having two students meant they could pair up for training, reflection, and peer support; and having prior experience meant they were already used to following complex protocols and interacting in real time with patients. The fact that they were also leaders of the student group meant that they had a long-term view of what success would mean in terms of creating opportunities for future students.

As well as our test phase worked, we did not anticipate some key ways in which the Fall would differ from the Summer. First, using experienced students as trail-blazers meant that we underestimated the difficulty that less experienced students would experience in learning new tasks in a new role. Second, hotline leaders had more bandwidth in the Summer to train two students than they did in the Fall to train 18 more.

4.2 Lessons learned from Recruitment

Our goal in the recruitment phase was to find 11 new students who would be immediately effective in joining 9 returning students, all learning to be COVID hotline navigators. Our strategy involved interviewing 93 finalists to assess their competencies. Then we offered positions to qualified finalists based on their availability to fill half-day shifts throughout the week. This worked reasonably well but was extremely time consuming. On reflection, because we were looking to staff specific shifts, we could have reversed the sequence. Next time, we might advertise 10 half-day shifts, and invite candidates to apply for specific shifts. We could then save time by simply conducting interviews until we filled each shift. This search strategy is known as “satisficing” – spending resources (e.g. interviews) until a “good enough” solution is found, rather than interviewing all applicants to find the best possible candidates.

4.3 Lessons learned from Larger-Scale Training

Our goal in the larger-scale training was to get 18 students who were new to the COVID hotline up to speed without unduly burdening existing staff navigators. The hotline was experiencing very high call volumes and staff navigators were needed on the hotline. Therefore, we had to launch the navigators in waves rather than all at once. In some ways, this was an artifact of starting a new program. Now that the program is launched, each year we will have returning students who can serve as peer mentors and train new students, reducing the training burden placed on staff navigators. In addition, we plan in the future to institute a call recording feature. This will allow us to review calls in slow motion with trainees. With other clinical sites, the Patient Support Corps has used call recordings to good effect as a way of accelerating training (16). For example, we have trainees transcribe calls, which is a way of having them absorb dialogue in slow motion, through their fingertips. They can then code the transcripts for fidelity to the original training, and therefore learn through repetition and reinforcement.

4.4 Lessons learned from Competency-Checking and Deployment

Our goal with competency-checking and deployment was to ensure that only students who were well-qualified would be cleared to work independently on the COVID hotline. The COVID hotline had already implemented a competency-checking process, and this worked equally well with student interns as with staff. A key lesson here was that we attempted competency checking too early for some students who needed more time observing navigators, and practicing under supervision, before being competency checked. Experienced navigators suggested that in the future we ensure that each student have at least 10 hours of supervised practice before attempting to be competency checked.

4.5 Lessons learned from Program Expansion

Our goal in the program expansion phase was to explore whether students could contribute to OPHAC in other internship roles, post-COVID. One of our explorations was particularly relevant to post-COVID contributions. This was a test we conducted with the Outreach team, in which two students again blazed the trail by learning new and practicing new roles. From this test, we learned that students could contribute to outreach tasks, starting immediately with easier tasks (such as case-finding) and progressing to more challenging tasks (such as coaching patients to close key care gaps.) Overall, we concluded that the COVID hotline was an unusually intense way to launch a new program, as students had to be interacting in real time with patients in the stressful environment of a pandemic response. Deploying students to work on outreach tasks should be more straightforward. As author MH, Population Health Manager, said on a debriefing call, “We've proven that we can do this in a crisis. Now that we have time to actually develop something, we could do it 20 times better!”

4.6 Implications for future service learning collaborations with Population Health programs

4.6.1 Next steps at UCSF

We have identified some key next steps at UCSF. First, we are renewing and extending the memorandum of understanding for the collaboration between OPHAC and PSC. OPHAC will provide 3 years of funding to support part-time effort by the PSC faculty director and program coordinator. In turn, the Patient Support Corps will recruit, train, and supervise 20-40 students per year working to extend OPHAC’s navigator capacity. We expect to bring on new students in their first or second year of college, and retain most students for three or four years.

Second, we are joining efforts to strengthen the pipeline of students from under-represented backgrounds. To that end, we have received funding from a foundation to work with local community colleges and state universities. We expect to use some of this support to fund paid Population Health Equity Internships for students from under-resourced backgrounds, alongside our existing academic (for-credit) internships.

As we bring on new student interns, we expect to formalize an internship career ladder, with rungs corresponding to stages of development. Our current thinking is that students will enter at level 1, side-by-side practice. At this level, they will observe experienced students or existing staff. Even at this entry level, students will increase navigator capacity. Navigators will delegate clerical tasks such as asking the level 1 interns to look up information or enter data.

At level 2, students will engage in practice under observation. They will take on navigator level tasks, such as coaching or scheduling patients, while being observed by more experienced students or existing staff.

In level 3, students will engage in supervised practice. They will perform navigator-level tasks independently. Experienced students or existing staff will audit their performance, or review recordings, to provide feedback. Within this level, students can expand their scope over time. They may start working within one narrow field of practice, such as scheduling patients to address care gaps (e.g. bringing patients in for cervical cancer screening). Over time, they may be cross-trained to provide health coaching for patients spanning multiple health conditions.

4.6.2 Opportunities for other Population Health programs

A natural pathway for expanding this work arises through the University of California (UC) system. Each of the five medical centers in the UC system has a Population Health program, and their leaders participate in a system-wide population health council. As of publication date, leaders at UC Davis and UCLA have expressed interest in replicating a population health internship program. As is the case at UCSF, these programs are interested in increasing the capacity and diversity of their population health workforce. Student interns can provide an immediate boost, while also creating a pipeline of qualified candidates for future employment in these programs. Other academic medical centers outside of the UC system could likely benefit from similar internship programs, as could population health programs in community settings. The health care workforce must continue to evolve in order to reflect the diversity of our patient populations and to accommodate the novel ways in which patients will interact with the health care system. This program provides important training and a potential pipeline of future health care staff and professionals.

We have described the launch, under crisis conditions, of a novel internship program. This program increased the navigator capacity in our institution’s Population Health program from 15 to 17 full-time equivalents – an increase of 13%. The strengths of our report relate to how we have documented our program design for others to adopt and adapt. A limitation was that our measures of quality for student performance were limited to competency checks. In future iterations of the program, we hope to record calls and survey callers. Another limitation was that we relied on weekly group calls and ad hoc individual communications to monitor student growth. By these accounts, consistent with a prior study (24), students reported tremendous growth in core competencies including medical knowledge, patient care, systems-based practice, practice-based learning and improvement, interpersonal skills and communication, and professionalism. However, in the future we hope to observe and survey students to assess their growth.

In conclusion, our report provides a detailed description of a successful service learning internship program that contributed to our institution’s COVID response. We are now investing in the continued expansion of this program. We anticipate that population health internships will contribute to increased capacity for delivering high-value care to patients; and that these internships can also help train more students from under-represented backgrounds in health care.

1 Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

2 Author Contributions

Author JB outlined and drafted the manuscript. TW, JM, and SK reflected on programmatic adaptations and other lessons learned from the Patient Support Corps perspective and contributed to corresponding sections of the manuscript. HT, TT, FL, GT, MH, and GI reflected on programmatic adaptations and other lessons learned from the Population Health perspective and contributed to corresponding sections of the manuscript. JA reflected on lessons learned from the institutional leadership perspective and contributed to the introduction and discussion.

3 Funding

The underlying collaboration was funded by UCSF’s Office of Population Health and Accountable Care. Authors JB and TW also drew salary for administering the Patient Support Corps from funds provided by the Chavez Family Foundation and individual donors making charitable donations to the program.

4 Acknowledgments

The authors wish to acknowledge the data and comments provided by students on the COVID Hotline: Zahra Anwar, Kaveh Boostanpour, Emily Chang, Sasha Demeulenare, Azeen Keihani, Tasmiah Khan, Vinson Mak, Marissa McPhaul, Jacqueline Nguyen, Eric Nguyen, Lisa Qiu, Lydia Sayad, Mahima Sinha, Saffanat Sumra, Isabelle Thapar, Albert Tran, Angela Wei, Rayann Yao, Iris Yen.

The authors also wish to acknowledge the contributions of other student leaders: Elissa Lyn Apiag, Prarthana Bhattacharya, Mira Cheng, Aditya Garg, Yee May Lwin, Prachi Saldhi, Amritha Somasekar, Rosey Stone, Andrea Yeung, Josephine Widjaja.

The following employees of the Office of Population Health and Accountable Care were especially helpful in training or supervising student interns through early 2021: Jessica Bianconi, Dianne Garcia, Kristin Gagliardi, Andrea Reategui.

Thanks to Lisa Lingham at UCSF’s Institute for Health Policy Studies for her help onboarding student interns as affiliates; David Hernandez, Chris Carrasquillo, Monica Oseguera-Alvarado, Natalie Liu, Maria Gonzalez, and Nauman Ahmad at UCSF Medical Center for assistance configuring information technology systems for student intern access; and Stefanie Ebeling of UC Berkeley for her leadership of the Undergraduate Research Apprentice Program. Thanks to Ian Shore of UCSF’s University Development & Alumni Relations for assistance with fundraising for the Patient Support Corps.

5 Data Availability Statement

The data for this report consist of field notes and recordings collected by author JB and available upon request.

Rhodes TL, Hein KD, Petersen DJ, Harver A. Integrative Learning in US Undergraduate Public Health Education: Effective High-Impact Practices. Frontiers in Public Health (2019) 7:383.
Hora MT, Wolfgram M, Thompson S. What do we know about the impact of internships on student outcomes? Results from a preliminary review of the scholarly and practitioner literatures. Center for Research on College-Workforce Transitions Research Brief (2017) 2:1-20.
Hora MT, Parrott E, Her P. How do students conceptualise the college internship experience? Towards a student-centred approach to designing and implementing internships. Journal of Education and Work (2020) 33(1):48-66. doi: 10.1080/13639080.2019.1708869.
Chickering AW, Gamson ZF. Seven principles for good practice in undergraduate education. AAHE bulletin (1987) 3:7.
National Academies of Sciences E, Medicine. Integrating social care into the delivery of health care: Moving upstream to improve the nation's health. (2019).
Gourevitch MN, Curtis LH, Durkin MS, Fagerlin A, Gelijns AC, Platt R, et al. The Emergence of Population Health in US Academic Medicine: A Qualitative Assessment. JAMA Network Open (2019) 2(4):e192200-e. doi: 10.1001/jamanetworkopen.2019.2200.
Lai L, Wittbold KA, Dadabhoy FZ, Sato R, Landman AB, Schwamm LH, et al., editors. Digital triage: Novel strategies for population health management in response to the COVID-19 pandemic. Healthcare; 2020: Elsevier.
Carter JK. College-community internship program: Collaborative efforts to develop local opportunities. Journal of Career Development (1998) 25(2):135-40.
Bell P. Experiential learning through a health professional shadowing program. Journal of Cooperative Education & Internships (2010) 44(1):34-41.
Silva P, Lopes B, Costa M, Seabra D, Melo AI, Brito E, et al. Stairway to employment? Internships in higher education. Higher Education (2016) 72(6):703-21.
Chorazy ML, Klinedinst KS. Learn by Doing: A Model for Incorporating High-Impact Experiential Learning Into an Undergraduate Public Health Curriculum. Frontiers in Public Health (2019) 7(31). doi: 10.3389/fpubh.2019.00031.
Callanan G, Benzing C. Assessing the role of internships in the career‐oriented employment of graduating college students. Education+ Training (2004).
Knouse SB, Fontenot G. Benefits of the business college internship: A research review. Journal of employment counseling (2008) 45(2):61-6.
D'abate CP, Youndt MA, Wenzel KE. Making the most of an internship: An empirical study of internship satisfaction. Academy of Management Learning & Education (2009) 8(4):527-39.
Hora M, Chen Z, Parrott E, Her P. Problematizing college internships: Exploring issues with access, program design and developmental outcomes. International Journal of Work-Integrated Learning (2020) 21(3):235.
Belkora J, Volz S, Loth M, Teng A, Zarin-Pass M, Moore D, et al. Coaching patients in the use of decision and communication aids: RE-AIM evaluation of a patient support program. BMC Health Serv Res (2015) 15:209. Epub 2015/05/29. doi: 10.1186/s12913-015-0872-610.1186/s12913-015-0872-6 [pii]. PubMed PMID: 26017564; PubMed Central PMCID: PMC4446845.
Mayo Clinic Center for Innovation. Innovation Contest at the TRANSFORM Conference (2009) [cited 2021 January 26]. Available from: https://www.youtube.com/watch?v=-BIJ8r-wbZQ&t=2s.
Clancy C, Munier W, Crosson K, Moy E, Ho K, Freeman W, et al. 2010 National Healthcare Quality & Disparities Reports(2011).
Agency for Healthcare Research and Quality. Innovation Profile: Personalized Support Improves Patient-Physician Communication and Enhances Decision Making for Breast Cancer Patients.: AHRQ Health Care Innovations Exchange [Web site]. (2008) [cited 2021 January 26]. Available from: https://innovations.ahrq.gov/profiles/personalized-support-improves-patient-physician-communication-and-enhances-decisionmaking.
Judson TJ, Odisho AY, Young JJ, Bigazzi O, Steuer D, Gonzales R, et al. Implementation of a digital chatbot to screen health system employees during the COVID-19 pandemic. J Am Med Inform Assoc (2020) 27(9):1450-5.
Judson TJ, Odisho AY, Neinstein AB, Chao J, Williams A, Miller C, et al. Rapid design and implementation of an integrated patient self-triage and self-scheduling tool for COVID-19. J Am Med Inform Assoc (2020) 27(6):860-6.
Belkora J. DEAL! Discovery, Engagement, and Leverage for Professionals. San Francisco, CA: Guidesmith (2015). 238 p.
Ries E. The lean startup : how today's entrepreneurs use continuous innovation to create radically successful businesses. 1st ed. New York: Crown Business (2011). 320 p. p.
Zarin-Pass M, Belkora J, Volz S, Esserman L. Making better doctors: a survey of premedical interns working as health coaches. J Cancer Educ (2014) 29(1):167-74. Epub 2013/10/29. doi: 10.1007/s13187-013-0565-y. PubMed PMID: 24163016.

Download PDF

Version 1

posted

You are reading this latest preprint version

Extending the Population Health Workforce through Service Learning Internships during COVID

Status:

Version 1

Abstract

1 Introduction: Description Of The Nature Of The Problem Being Addressed And Rationale For The Proposed Innovation

2 Context (Setting And Population) In Which The Innovation Occurs

2.1 University of California, San Francisco

2.2 The Office of Population Health and Accountable Care (OPHAC)

2.3 The Patient Support Corps (PSC)

2.4 UCSF Patients and Employees Exposed to Coronavirus Infections (Hotline callers)

3 Programmatic Elements

3.1 Program Plan

3.2 Program Implementation

3.2.1 Initial Training and Test Deployment (Pilot with 2 students)

3.2.1.1 Overview of Training on Service Delivery Plan and Operational Plan

3.2.1.2 Hotline scripts and branching logic

3.2.1.3 Students observing navigators and being observed

3.2.1.4 Formal Competency Check

3.2.2 Recruitment

3.2.3 Larger Scale Training

3.2.4 Competency Checking and Deployment

3.2.4.1 Supervised Practice during Suspension of Competency Checks

3.2.4.2 Resumption of Competency Checks

3.2.5 Program Expansion

3.2.5.1 Occupational Health

3.2.5.2 Population Health Outreach

4 Discussion Section That Shares Practical Implications, Lessons Learned For Future Applications

4.1 Lessons learned from Test Deployment

4.2 Lessons learned from Recruitment

4.3 Lessons learned from Larger-Scale Training

4.4 Lessons learned from Competency-Checking and Deployment

4.5 Lessons learned from Program Expansion

4.6 Implications for future service learning collaborations with Population Health programs

4.6.1 Next steps at UCSF

4.6.2 Opportunities for other Population Health programs

Strengths, Limitations, And Conclusions

Declarations

1 Conflict of Interest

2 Author Contributions

3 Funding

4 Acknowledgments

5 Data Availability Statement

References

Status:

Version 1