Study setting
North Carolina (NC) is the 10th largest state in the US with a population over 10 million and a racial and ethnic composition of 71% White, 22% Black, and 9.6% Latino, respectively.(29) The burden of unhealthy alcohol use in NC remains large, accounting for an estimated 4,000 deaths annually as well as substantial morbidity.(30) Approximately 17% of adults in North Carolina engage in unhealthy alcohol use and 1 in 11 deaths among working-age adults (20-64 years old) in North Carolina are attributable to alcohol.(31, 32) The most common cause of alcohol-related deaths in NC is alcohol-impaired driving fatalities.(33)
About 7,400 physicians and 2,000 advanced practice providers work mainly as primary care providers in North Carolina, according to 2019 NC licensure data in the North Carolina Health Professions Data System.(34) Practice size estimates vary according to practice definition (tax ID# vs. single geographic location), but based on provider to practice ratios, we estimate that there are approximately 2,200 primary care practices in NC. Thirty to forty percent of these remain unaffiliated with larger health systems, and almost all have fully implemented EHRs. The practices eligible for enrollment in STUN are skewed toward rural areas.
Research design
STUN Alcohol Use Now is an adaptive, randomized, controlled trial to evaluate the effect of primary care practice facilitation and the effect of using telehealth services on evidence-based screening, counseling, and pharmacotherapy for AUD (Figure 1).
The design is considered an adaptive trial design because inclusion in the randomized phase of the study is based on practice performance during the initial 6 months of the intervention. All enrolled practices will receive the practice facilitation intervention.
After practices have received the practice facilitation intervention for 6 months, those in the lower 50th percentile of performance will be randomized (block randomized with 1:1 allocation, block sizes of 2) to continued practice facilitation or to using telehealth services plus continued practice facilitation for the next 6 months. The random allocation sequence will be generated using Microsoft excel by a data analyst who is not involved in recruitment or practice facilitation. The random allocation sequence will be stored in a password protected electronic file and concealed from all other team members involved in recruitment and practice facilitation until after a practice has been determined to be in the lower 50th percentile of performance and has received the practice facilitation intervention for 6 months. The performance assessment over the initial 6 months will be based on the percentage of patients screened for unhealthy alcohol use and the percentage of patients who receive brief counseling when it is indicated (after screening indicates unhealthy alcohol use). Randomizing the lower 50th percentile will allow us to assess (1) whether provision of telehealth services accelerates uptake for practices with slower uptake and (2) whether “staying the course” with ongoing practice facilitation is an effective strategy for those with slower uptake (comparing whether they catch up to the upper 50th percentile). Practices in the upper 50th percentile over the initial 6 months will continue to receive the practice facilitation intervention for another 6 months.
Practice Recruitment
Practice recruitment will be conducted by the practice facilitators (i.e., practice coaches). Due to COVID-19-related constraints recruitment will be largely conducted on a virtual basis using phone calls, emails, and video conferencing. For the purpose of planning, guiding recruitment activities, and refining recruitment strategies the research team will create and provide informational materials, schedule informational webinars with potential practice representatives, and participate in regular educational sessions with the practice facilitators. We initially aimed to enroll up to 135 small to medium-sized primary care practices although our sample size calculations required fewer practices (with the rationale of trying to help as many practices as might be feasible); but competing demands and constraints due to the pandemic tempered enrollment. Practices are eligible for enrollment if 10 or fewer providers occupy a single location and do not receive facilitation services specifically related to unhealthy alcohol use. Enrolling practices agree to the following: 1) work with practice facilitators for 4 to 8 hours a month to implement an evidence-based screening process as well as a process for counseling and/or referring patients with unhealthy alcohol use; 2) participate in webinars conducted by project personnel about the screening and brief counseling process as well as how and when to prescribe medications for AUD; 3) respond to surveys about the practice environment and the improvement process; and, 4) collect implementation effectiveness data on a monthly basis with help from practice facilitators.
Potential barriers to practice recruitment include fear of financial risk, dedicating practice resources to other new activities such as Medicaid reform or Accountable Care Organization participation to the exclusion of participation in this study, as well as concerns related to staffing issues, competing priorities, and lack of time. These barriers have been compounded during the COVID-19 pandemic as practices struggle to meet the demands of caring for patients in a changed environment and contend with new financial pressures arising from reduced patient visits. Practice facilitation will attempt to mitigate these barriers by helping practices adapt — for example, by streamlining workflows. During recruitment we will also emphasize current evidence that the increase in social isolation, household pressures, and economic stress during the pandemic has been associated with an increase in unhealthy alcohol use.(35-37)
Practice support intervention
The NC Area Health Education Centers (AHEC) program has permanent statewide infrastructure and highly trained personnel to support and deliver practice facilitation services and has developed strong relationships with primary care practices.(38) Participating practices will receive direct practice facilitation over the 12-month intervention period from NC AHEC personnel. The facilitator will ensure that key drivers for improvement are identified and that the practice is comfortable with implementing the improvement with rapid-cycle tests of change. Practices will receive 1-2 hours of direct practice facilitation services per month and be expected to apply tests of change using a Plan-Do-Study-Act (PDSA) approach(39) fairly independently, confirmed and coached by a member of the facilitation team. The facilitator will ensure that the practice has established specific workflows for the unhealthy alcohol use measures and conduct periodic data checks to ascertain progress, sharing the results of these data checks with the practice (in an audit and feedback fashion). Webinars and video recordings will serve as additional tools that facilitators can use to educate practices on best practices for alcohol screening, counseling, and interventions. Expert consultation with physician faculty will be available, primarily virtually, to supplement facilitation efforts. While practice facilitation was initially planned as a combination of face-to-face meetings, phone calls, web-based video meetings, and email communication, remote communications will be the primary communication modality until risks from the pandemic have subsided. Facilitation meetings will emphasize implementation of evidence-based protocols and use of clinical algorithms by:
- Forming clinical QI teams to engage the practice (or its participating clinicians) in a high standard of care delivery, including use of standing orders, EHR templates and clinical decision support tools.
- Establishing human workflows including team-based roles to use these practice tools
- Optimizing the use of the EHR to perform monthly data pulls to guide and evaluate the progress of the screening process, counselling, pharmacologic treatment, and associated referrals.
- Developing spreadsheet registries and other electronic tools for practices lacking the EHR capabilities to develop the data resources described above.
- Assisting the practices in optimizing billing for reimbursable SBI services.
- Working with practices to develop proactive assigned roles and responsibilities to prepare the clinical team to develop needed care and engage patients.
- Providing lists of available counseling and referral resources by region and county, to be used when primary care clinicians encounter patients whose unhealthy alcohol use, AUD, or comorbid behavior health conditions exceed the clinician’s comfort level or expertise.
- For practices randomized to the telehealth group, protocols for scheduling and utilizing telehealth services will be developed.
Evaluation framework
Implementation of practice facilitation to support uptake of an evidence-based screening, brief counseling intervention, and referral to alcohol treatment represents a major innovation for addressing the substantial health burden of unhealthy alcohol use. Briefly, effective implementation is a function of the implementation support the practice receives and the policies and practices it employs to support innovation use.
Organized QI effort and capability will be the key driver of improvement, creating an environment in which primary care practices can embrace implementation of the Chronic Care Model.(40) The Chronic Care Model emphasizes that practices use clinical decision support, clinical information systems, optimal delivery system design, self-management support, and community linkages to create prepared and proactive care teams and informed and motivated patients, leading to improved health outcomes.(41) The elements of the Model fit well with implementation of SBI for unhealthy alcohol use and pharmacotherapy for AUD.
Our primary hypotheses, that practice facilitation (and, in the case of slow-uptake practices, the further addition of telehealth) will improve processes of care for unhealthy alcohol use are motivated by adult learning theory and social cognitive theory (SCT).(42-44) Adult learning theory posits that people prefer to learn based on real-life problems, by setting realistic goals, listening to their peers, and by experiencing success when they experiment with improvement efforts. STUN Alcohol Use Now’s practice facilitation adheres to the preferences of the targeted adult learners (clinicians and practice staff) in primary care practices (Table 1).
Table 1. Theory motivating STUN Alcohol Use Now’s hypothesized influence
Adult learners learn…
|
STUN Alcohol Use Now Approach
|
Based on real life problems
|
Practice facilitation, expert consultation, and training modules will incorporate real problems
|
By setting realistic goals
|
Practice facilitation, expert consultation, and training modules will emphasize realistic goals
|
By listening to their peers
|
Practice facilitation, expert consultation, and training modules will incorporate peers in their stories
|
By experiencing success
|
Use of EHR reports, run charts, and positive reinforcement by facilitators
|
Data and Safety Monitoring
Our Data Safety and Monitoring Plan (DSMP), commensurate with the low degree of risk involved in participation, will focus on monitoring and minimizing risks to participating practices, careful monitoring of the study’s progress, protecting the confidential medical and personal information of subjects, and ensuring the validity and integrity of the collected data. More information related to privacy and data security can be found in the Protection of Human Subjects section of the grant application.
Toward this end, we propose that the principal investigator, DEJ, shall be responsible for carrying out the DSMP. Key personnel involved with the logistics of practice enrollment, participation, and follow-up (e.g., the overall project manager, AHEC project manager, and data manager) will meet regularly (at least quarterly) to review, among other things, progress on accrual, implementation, data collection, and adverse events. Many of these individuals, along with the study investigators, will meet weekly to address ongoing study issues including, among other things, progress on accrual (practice recruitment and retention), follow-up, and adverse events. The project manager will then prepare the needed reports (Table 2) for the PI.
Annually, the PI will prepare a written report addressing the study’s progress and safety, which will be provided to the IRB as part of the annual renewal submission.
Table 2. Data and Safety Monitoring Plan reporting: Type of data and frequency of review
Data Type and Description
|
Frequency of Review
|
Practice recruitment and retention
|
Weekly, with monthly summaries for PI including graphic of projected vs. actual
|
Study performance (via Practice Facilitator contact logs, we will capture number and types of personnel working with practices to support implementation, number and type of interactions between project staff/consultants and practices, number of practices reached by the implementation, number of clinicians engaged, and percentage of PF contacts with a practice that are in-person)
|
Monthly summary to the PI
All significant protocol deviations will be reported to the IRB during the annual review process
|
Implementation effectiveness (assessed via required measures.)
|
A monthly summary, including run charts, will be prepared from uploaded encrypted and secure data submitted from the practices to our database
|
Risks to participating practices (practice facilitators will be trained to monitor the time commitment of the practice to the project and to note any disruption in practice workflow or functions)
|
Monthly summary to the PI
|
Adverse events (AEs), such as breach of confidentiality
|
Practice facilitators will be trained to identify and report AEs as they occur to the AHEC project manger and overall project manager. The PI will be responsible for reporting AEs, as they occur, to the IRB, using UNC IRB definitions, standards, and forms
|
Stopping rules regarding benefits and harms
|
Not applicable to the study
|
Stopping rules regarding statistical power
|
Not applicable to the study
|
Constructs and Measures
Implementation Support. Using practice facilitation contact logs, we will capture number and type of interactions between project staff/consultants and practices, number of practices reached by the implementation, and number of clinicians engaged.
Practice Capacity for QI encompasses 2 constructs, change process capability and adaptive reserve. We will assess the former using the 32-item Change Process Capacity Questionnaire (45) and the latter using 3-item adaptive reserve (i.e., capacity for change) scale.(46, 47) Both the CPCQ and the adaptive reserve scale exhibit reliability and known-groups validity.(45, 48)
Organizational Readiness for Change refers to the extent to which organizational members are psychologically and behaviorally prepared to implement organizational change.(49-51) We will measure readiness using the 12-item Organizational Readiness for Implementing Change (ORIC) scale.(49) ORIC has demonstrated reliability, content validity, structural validity, structural invariance, and known-groups validity.(49)
Implementation Policies and Practices are the strategies that an organization puts into place to support innovation use.(52, 53) We will measure these with the Key Driver Implementation Scale (KDIS), which uses a 5-point, behaviorally-anchored, ordinal scale that covers multiple fundamental drivers of improvement.
Implementation Climate refers to organizational members’ “shared summary perception of the extent to which their use of a specific innovation is rewarded, supported, and expected within their organization.”(54) We will measure implementation climate using a 6-item scale with demonstrated reliability, structural validity, structural invariance, known-groups validity, and predictive validity.
Implementation Effectiveness refers to the consistency and quality of innovation use.(52) Implementation effectiveness is conceived here as an organization-level construct describing organizational members’ pooled consistency and quality of innovation use (i.e., evidence-based screening, counseling, referrals, and pharmacotherapy).(52, 55, 56) We will assess implementation effectiveness using the measures in Table 3.
Table 3: Implementation Effectiveness Measures
Measure
|
Description
|
# and % of patients screened
|
Number and percentage of adult patients with documentation of screening for unhealthy alcohol use with the screening questions recommended by NIAAA*
|
# and % of patients with a positive screen
|
Number and percentage of adult patients with a positive initial screen
|
# and % of patients completing the AUDIT
|
Number and percentage of those with a positive initial screen* who go on to complete the 10-question AUDIT (the next step in assessment after an initial positive screen)
|
# and % of patients who receive brief counseling
|
Number and percentage of adult patients with documentation of brief counseling for risky drinking, when indicated/appropriate
|
# and % of patients who have AUD
|
Number and percentage of adult patients with documented ICD diagnoses of AUD
|
# and % of patients who receive pharmacotherapy for AUD
|
Number and percentage of adult patients with AUD who receive evidence-based pharmacotherapy with naltrexone, acamprosate, disulfiram, or topiramate†
|
# and % of patients referred to specialty care for AUD
|
Number and percentage of adult patients with AUD referred to specialty care (e.g., psychiatry, CBT, motivational enhancement therapy, 12-step programs)
|
* Do you sometimes drink beer, wine, or other alcoholic beverages? (If yes) How many times in the past year have you had 5 or more (for men 64 and younger)/4 or more (for women of any age, and men 65 and older) drinks in a day?(6) Response of 1 or more is a positive initial screen.
† Topiramate is not FDA approved for AUD, but it has been shown to be beneficial (e.g., for reducing heavy drinking days).(14)
AUD = Alcohol Use Disorder; AUDIT = Alcohol Use Disorders Identification Test; CBT = Cognitive Behavioral Therapy; ICD = International Classification of Diseases; NIAAA = National Institute on Alcohol Abuse and Alcoholism.
Telehealth acceptability for those practices provided with telehealth services will be measured using a brief survey based on the Technology Acceptance Model.(57, 58).
Data collection
Table 4 outlines data collection, measures, sources, and timing for each construct.
Table 4. Theoretical constructs, measures, data sources, and data collection timing
Construct
|
Measure
|
Source
|
Timing
|
Implementation Support
|
Frequency, duration, mode, and purpose of practice contacts
|
PF contact logs
|
I
|
Practice capacity for QI
|
Change Process Capacity Questionnaire (CPCQ)
Adaptive Reserve Questionnaire
|
Provider/Staff Survey
|
B, E, F
|
Organizational Readiness
|
Organizational Readiness for Change Questionnaire (ORIC)
|
Provider/Staff Survey
|
B
|
Implementation Policies and Practices
|
Key Driver Implementation Scale (KDIS) and type and quantity of strategies implemented
|
PF ratings
|
I
|
Implementation Climate
|
Implementation Climate Questionnaire
|
Provider/Staff Survey
|
E, F
|
Implementation Effectiveness
|
Number and % of patients in target population who are screened for unhealthy alcohol use, screen positive, receive brief counseling, have AUD, receive pharmacotherapy, referred for AUD
|
Data forms on website, chart review, or direct from EHR
|
B, I, E, F
|
Contextual Factors
|
Practice characteristics, patient population, EHR capabilities
|
Provider/Staff Survey
|
B, I, E, F
|
Telehealth acceptability
|
Acceptability to practice; satisfaction with workflow and quality
|
Provider/Staff survey
|
E, F
|
PF = practice facilitator; QI = quality improvement; EHR = electronic health record; AUD = alcohol use disorder; Timing: B = Baseline; I = Intervention (weekly/monthly); E = End of intervention; F = 6-month post intervention
Practice facilitator contact logs
To capture implementation support, practice facilitators will record the date, duration, and mode of each practice contact. We will use these data to measure the level of implementation support provided to each practice.
Provider/staff survey
We will survey providers and staff to assess practice capacity for QI at baseline, end-of-intervention, and 6 months post-intervention; organizational readiness to implement change at baseline; and implementation climate at end-of-intervention and 6 months post-intervention. To document contextual factors, surveys will also capture relevant practice characteristics, general patient population and demographics, and EHR capabilities. The survey sample at each time point will consist of 3–5 providers and staff members depending on practice size (N = 400–675 survey respondents).
Practice facilitator ratings of practices’ progress
Practice facilitators will assess practices’ progress in implementing clinical practice and organizational changes to support improvement in screening, brief intervention, and referral to treatment (SBIRT) (implementation policies and practices) at monthly intervals. Ratings will focus on the extent to which practices have implemented multiple key drivers of improved SBIRT provision and level of leadership and team engagement.
Chart Review
To assess screening rates at baseline, practice facilitators will assist practice staff in querying their EHR or conducting a chart review to determine the number and proportion of patients who have had evidence-based screening over the previous 2 years.
EHR/Informatics
Practices will obtain performance data on the implementation effectiveness (uptake) from their EHRs or by recording the data in a registry, the creation of which the practice facilitators and project team will guide. These data will be entered as aggregate counts and percentages for each practice (no protected health information [PHI] will be included) in a dedicated, online tool that feeds into the project database. The implementation effectiveness measures will be collected at baseline, during the intervention, at the end of the intervention, and post-intervention (6 follow-up).
Data Analysis
Aim 1: Evaluate the effect of primary care practice facilitation on uptake of evidence-based SBI. Our primary hypotheses are that practice facilitation will increase the number and percentage of patients in a practice who are: (a) screened for unhealthy alcohol use, (b) identified to have unhealthy alcohol use, and (c) provided with brief counseling. Our secondary hypothesis is that practice capacity for QI, organizational readiness to implement change, implementation climate, and contextual factors will moderate the effect of primary care practice facilitation on use of evidence-based SBI for unhealthy alcohol use.
Statistical Analysis. For each practice, percent screened for unhealthy alcohol use will be computed using all data available from the three months up to baseline (henceforth “baseline”) as well as in each of the two quarters up to 6 months. 95% percent confidence intervals will be calculated with the percentages compared using repeated measures ANOVA for an overall test of change following practice facilitation, as well as for differences in percentages for each of the post-facilitation quarters relative to pre-baseline. The changes in percent screened will be evaluated with paired t-tests looking at differences between each pair of quarters at alpha=.05. The primary analysis will be based on a comparison of the baseline versus the 2nd quarter following practice facilitation, where the impact of practice facilitation is likely to be greatest. The repeated measures ANOVA will allow us to potentially detect a delay in effectiveness. Similar analyses will be conducted using percent identified with unhealthy alcohol use and for the percent provided with counseling, based on the data from baseline and up to 6 months after practice facilitation initiation. The analyses will be redone adjusting for calendar time period to control for potential changes occurring naturally over time. As a secondary analysis we will examine proportion trajectories using a mixed modeling approach, iteratively testing linear, quadratic, and cubic time functions (Level 1), interacting those time functions with intervention condition (Level 2) and controlling for key covariates (e.g., clinic size). Relative model fit statistics (e.g., Akaike information criterion [AIC]) will be used to inform selection of the final model.
We will examine the effects of practice characteristics, such as organizational readiness and implementation environment on the impact of practice facilitation. Descriptive statistics will be calculated for practice level variables. Linear regression models will be fit to practice level change scores based on differences in the two quarters after practice facilitation compared to the baseline percentage. Generalized estimating equations will be used to fit models simultaneously using the two change scores. Each moderating variable will be screened in univariate models, with significant variables entered in a multivariable model. AIC will be used to choose the final model.
Secondary analyses will consider quarterly data collected after 6 months to the end of follow-up at 12 months, and to the post-intervention follow-up, to evaluate the effect of practice facilitation as well as the effect of telehealth combined with practice facilitation. This will involve analyses similar to those above in which all practices that are randomized to no telehealth are grouped together, while higher performing practices at 6 months randomized to telehealth (but not receiving telehealth) and lower performing practices randomized to telehealth constitute a second group. Change scores will be calculated from baseline. Average percent improvements from baseline between the two groups will be compared at each quarter after 6 months using two sample t-tests, with an overall test based on repeated measures ANOVA. Adjustment for practice level characteristics will be explored using GEE. Survey data collected from multiple clinic staff/health care providers will be analyzed using a mixed modeling approach, with person (Level 1) nested within clinic (Level 2)—where appropriate, three level models with repeated measures (Level 1) within person (Level 2) within clinic (Level 3) will be employed when considering time effects for these data.
Patterns of missing data will be assessed descriptively. Associations between practice level covariates and missingness will be explored, with differences formally tested using logistic regression models for binary missingness variables. Inverse weighting based on missingness probabilities will be employed to adjust for the effects of practice level covariates on missingness, where appropriate.
The main outcomes are being measured at the practice level (e.g., did the practice increase their rates of screening for unhealthy alcohol use), not at the individual patient level, and the unit of inference is the practice (not the individual patient). Therefore, the trial was not designed as a cluster trial and there is not a need to account for clustering; trials focused on outcomes measured at the group/practice level can be regarded as standard clinical trials with respect to estimation of sample size and analysis approach.(59, 60)
Statistical Power
We estimate that a very low percentage (<5%) currently receive recommended screening. On average, based on our experience and prior evaluations we expect about 45% of patients within each practice to receive screening following practice facilitation, by 6 months. For unhealthy alcohol use, preliminary data suggest that about 25% of those screened will have unhealthy alcohol use identified (roughly 7.5% of all adults served by the practice). Conservatively, if there is a 10% increase in the percentage (of either screening or detection of unhealthy alcohol use from baseline to 6 months) on average across practices, with a standard deviation in the percentage improvement of 10%, then a one sample t-test at level 0.05 has at least 80% power to detect that improvement with sample size 34. Thus, the proposed design has very good power to detect realistic effect sizes for improved screening and detection of unhealthy alcohol use, even with considerable dropout and/or missing data (even more than 30% missingness).
Aim 2: Evaluate the effect of practice facilitation on uptake of evidence-based counseling and pharmacotherapy for AUD.
Our hypotheses are that practice facilitation will increase the number and percentage of patients in a practice who are: (a) identified to have AUD, (b) provided with pharmacotherapy for AUD, and (c) referred to specialty care for AUD.
Statistical Analysis. As in Aim 1, for each practice, percent identified with AUD will be computed using all data available. 95% percent confidence intervals will be calculated with the percentages compared using repeated measures ANOVA for an overall test of change following practice facilitation, with paired t-tests looking at differences between each pair of quarters. The primary analysis will be based on a comparison of the baseline and the 2nd quarter following practice facilitation. Similar analyses will be conducted using percent provided pharmacotherapy and referred to specialty care for AUD, based on the data from baseline and up to 6 months after practice facilitation initiation. Additional secondary analyses will consider quarterly data collected after 6 months from baseline to the end of follow-up at 12 months, and to the post-intervention follow-up, using the intent to treat framework described for Aim 1. Missingness for Aim 2 endpoints will be assessed as in Aim 1. Mixed modeling will be used to compute trajectory models as described for Aim 1.
Statistical Power. Detection of previously unknown AUD via screening in primary care is relatively rare, on the order of 1% among adults with no history of AUD. Assuming 45% of adults are screened over 6 months, we would expect an increase of roughly 0.45% from baseline following practice facilitation. Among those with AUD, we anticipate a relatively large increase in pharmacotherapy for AUD, from 0 to 33% on average. To detect an increase of 0.45% in AUD detection on average with standard deviation 1% using a one sample t-test at level 0.05, sample sizes of 39 and 52 give 80% and 90% power to detect such differences. The increase in pharmacotherapy for AUD for those identified as having AUD is quite large and the one sample t-tests have very large power to detect such improvements.
Aim 3: For practices with slower uptake, evaluate the effect of telehealth services on use of evidence-based (a) SBI for unhealthy alcohol use and (b) counseling and pharmacotherapy for AUD.
Our primary hypotheses are that, compared with continued practice facilitation, practices randomized to the provision of telehealth services will increase the number and percentage of patients who are: (a) provided with brief counseling for unhealthy alcohol use, (b) provided with pharmacotherapy for AUD, and (c) referred to specialty care for AUD. Our secondary hypothesis is that telehealth services for counseling will be acceptable to small to medium-size primary care practices.
Statistical Analysis. Practices in the lower 50th percentile at 6 months will be randomized to telehealth or not. The primary analysis will be an ITT analysis based on a comparison of changes in counseling, pharmacotherapy, and specialty care from 6 months to 12 months (end-of-intervention). Such changes will be calculated based on a comparison of quarterly percentages, where 3-6 months serves as baseline and 6-9 and 9-12 months as the post randomization quarters. Note that for this aim the 6-month baseline differs from the baseline in Aims 1 and 2. Average percent changes and 95% confidence intervals will be computed for SBI and pharmacotherapy in the two intervention arms for 6-9 and 9-12 months. The two arms will be compared using repeated measures ANOVA and two sample t-tests at each quarter at level .05. The main focus is on the changes from 3-6 months to 9-12 months, where the impact of telehealth is likely to be greatest. The repeated measures ANOVA will allow us to potentially detect increased impact over time. Secondary analyses will assess the post-intervention follow-up (focusing on change from 3-6 to 15-18 months) and acceptability of telehealth. Acceptability percentage will be calculated across practices, along with 95% confidence intervals. Mixed modeling will be used to assess the impact of practice level characteristics on acceptability on the part of staff/healthcare providers.
Statistical Power. The focus is on detecting differences in the changes from 6 to 12 months for practice facilitation + telehealth versus practice facilitation without telehealth. Differences will be tested at level .05 using two sample t-tests based on an ITT analysis of percentage changes in the two randomized groups. Assuming that the standard deviation of the percent changes is .15 in each group (and a between group difference of 20%), a sample size of 9 per group gives 80% power and sample size of 12 gives 90% power. The resulting effect sizes are moderate in size and consistent with previous results. Thus, the proposed randomization of 68 slower uptake practices should have adequate power.
Aim 4: Evaluate the effect of practice facilitation on the implementation of clinical practice and office systems changes to improve evidence-based SBI and pharmacotherapy.
Our primary hypothesis is that practice facilitation will increase implementation of clinical practice and office systems changes to improve evidence-based SBI and pharmacotherapy. Our secondary hypotheses are that (a) practice capacity for QI, organizational readiness to implement change, and contextual factors will moderate the effect of practice facilitation on the implementation of clinical practice and office systems changes and (b) provision of telehealth services will increase implementation of clinical practice and office systems changes among practices with slower uptake.
Statistical Analysis. This aim is structured to explore the impact of practice facilitation and telehealth on the implementation of clinical practice and office systems changes. The analysis plan mirrors that in Aims 1 and 2. The first set of analyses is based on outcomes from the first two quarters from baseline to 6 months in which the effect of practice facilitation is explored, independently of telehealth. The second set of analyses compares the effect of practice facilitation with and without telehealth using quarterly data from 6 to 12 months. Moderating effects of practice level characteristics will be explored in both sets of analyses.
Statistical Power. These power calculations are similar to those for Aims 1 and 2 which focused on the impact of practice facilitation. Here, the primary endpoint is detecting a difference in KDIS score from baseline, which is evaluated using a one sample t-test. A meaningful improvement would be 1 unit. With standard deviation of KDIS differences equaling 3 (a conservative assumption), sample sizes of 70 and 95 give 80% and 90% power based on a .05 level test.
Dissemination Plan
The overall goal of our dissemination plan is to inform all stakeholders including patients, providers, payers, and government agencies about the process and findings of STUN Alcohol Use Now. The main message will be that small to medium-size primary care practices are partnering with NC AHEC to rapidly improve implementation of SBI and MAT for unhealthy alcohol use. These efforts are expected to prevent deaths from unhealthy alcohol use as well as morbidity from the many adverse health consequences. We will commit to working with AHRQ and its contractors in disseminating information about the project, such as through the AHRQ annual meeting and AHRQ publications. We will fully inform AHRQ and its designated contractors regarding implementation results and status of outcomes over time. All presentations and publications derived from STUN Alcohol Use Now will also be made available to AHRQ and its contractors.
By involving our stakeholders, we have already enlisted key people and organizations with the wherewithal and enthusiasm to support dissemination of this project. We have begun to work closely with the NC AHEC Practice Support Program, the NCHQA, the NC Academy of Family Practice, Carolina Partners in Mental HealthCare, UNC’s Alcohol and Substance Abuse Program, UNC’s Virtual Care Center, and UNC-CH’s practice-based research network, NCNet (see Letters of Support). AHEC has already informed their networks about the possibility of this work and the vast potential for reducing unhealthy alcohol use and improving outcomes using this proposal as a nidus. Additional specific organizations interested in research for unhealthy alcohol use that we intend to engage include: the nascent accountable care organizations in NC and beyond, the NC Chapter of the American College of Physicians, the American Public Health Association, BCBS NC, the Society of General Internal Medicine, the National AHEC Organization, and Academy Health.
Once informed of award, NC AHEC will be able to activate many associated practices before the actual funding using teams that already touch hundreds of primary care practices. The Steering Committee, that will include patient advocates yet to be named, will be regularly briefed on process and results during every quarterly meeting. As we document evidence of process improvement, practice acceptance, and improved outcomes, we will provide the committee members with materials to distribute to their constituents without delay. The AHEC Practice Support Program remains closely connected to the NC Office of Health Benefits and Division of Public Health and often shares improvement results of ongoing efforts with these state agencies.
As we have done in the past, we will use community forums, local media outlets, scientific meetings and publications to disseminate findings. AHEC is developing a presence on social media that could be used to disseminate unhealthy alcohol use awareness to new audiences. The practices involved in the actual study represent only a fraction of AHEC practices. The larger network provides a ripe environment for dissemination.