Over the course of one year, we enrolled 40 participants in the SAFER program. Participants mostly had type 2 diabetes, and we had a near-even split of men and women (Table 1), despite the fact that there tend to be more men than women who experience homelessness in Calgary. (28) There was a high rate of anxiety and/or depression in our sample (28/40).
The screening provided by our program during the initial nurse visit yielded that our population consisted of individuals whose glycemia was largely not within the target range (60% with A1C > 7%) and that many participants had evidence of diabetic kidney disease based on the presence of albuminuria (37% with ACR > 3mg/mmol). Some degree of loss of protective sensation in the lower extremity was present in 31/40 participants, and 8/34 had evidence of retinopathy (Table 2).
Given that the qualitative elements of our RE-AIM evaluation are documented elsewhere; this manuscript presents results only regarding components of Reach, Effectiveness, and Implementation. The following sections discuss the primary objective along with the remaining RE-AIM domain findings that were considered either facilitators or barriers to the sustainability of this program.
Primary Objective - Effectiveness
Of the 40 participants in the study, none were up-to-date with all of their microvascular screening (at least once in the past two years) and glycemic monitoring (at least once within the past 6 months) at the time of enrollment. However, after having gone through the program, 29/40 had completed all the tests. One participant reported nausea post pupil dilation in the screening clinic. The study nurse kept the participant a bit longer for monitoring, made the host site staff aware, asked the medical staff to monitor the participant for the remainder of the day and instructed staff to arrange for transport to the nearest hospital if nausea progressed to vomiting or if any visual disturbances increased. No further follow up was required. The proportions test showed that this was a statistically significant difference. With respect to the individual screening tests (Fig. 1), we documented statistically significant increases in screening for: diabetic neuropathy (2.9-fold increase: 10/40 à 39/40, p = < 0.0001); diabetic retinopathy (1.7-fold increase: 11/40 à 30/40, p = < 0.0001); diabetic nephropathy (90% increase: 20/40 à 38/40, p = < 0.0001); and glycemic monitoring over the past 6 months (73% increase: 23/40 à40/40, p = < 0.0001).
The second aspect of the effectiveness domain was with respect to the program’s ability to connect participants with specialists when microvascular complications were identified or there was a need for major glycemic improvement. One in ten screened for foot health were referred to a footcare specialist (partnering podiatrist or foot surgeon) for high or urgent needs that were identified. Unfortunately, only one out of the four referrals made was seen by the foot care specialty team. Nearly one quarter (7/31) of those screened for retinal disease were referred to retinal specialists for repeat screening, fluorescein angiograms, or for treatment. Fewer than half (3/7) were ultimately seen by the retina specialist. Despite a high prevalence of albuminuria, most were mild-to-moderate, so only 2/38 were flagged for needing specialty follow-up, one of whom was seen. With respect to glycemia, 8/40 participants were referred for specialist follow-up with a diabetes educator or medical specialist due to elevated A1C. Only half were seen, two by a pharmacist who was immediately available after the nurse had completed the follow-up visit, and two were seen by the principal investigator/partnering endocrinologist during outreach clinics. We were unable to ascertain whether the remaining four participants were seen by specialists at some point after their initial visit.
Finally, we also assessed effectiveness with respect to patient-reported ease of completion of the screening tests (Table 3). At baseline, pre-SAFER, when asked about the ease of completing their regular microvascular screening, approximately one-quarter stated that these were difficult to access: blood and urine tests (9/40), retinal screening (11/40), and foot exams (13/40). Comparing the patient-reported assessment of ease demonstrated that there was a statistically significant change (to increased ease) after completing the SAFER program for both foot assessment and eye assessments. There was a non-significant trend towards increased ease for laboratory assessment.
Table 3
Patient-reported ease of completing screening tests
| Pre-SAFER assessment, n | Post-SAFER assessment, n | Change, n | p-value* |
“It is easy for me to have my annual screening foot exam.” (n = 39) | Strongly Disagree | 2 | 0 | More difficult | 4 | 0.0073 |
Disagree | 11 | 0 | Easier | 15 |
Agree | 15 | 17 | No change | 12 |
Strongly Agree | 11 | 14 | | |
“It is easy for me to have my annual screening eye exam completed (dilated eye exam with an eye doctor).” (n = 40) | Strongly Disagree | 1 | 1 | More difficult | 3 | 0.0163 |
Disagree | 10 | 0 | Easier | 12 |
Agree | 18 | 18 | No change | 16 |
Strongly Agree | 11 | 13 | | |
“It is easy for me to have my blood work and urine tests completed” (n = 40) | Strongly Disagree | 1 | 1 | More difficult | 6 | 0.0919 |
Disagree | 8 | 0 | Easier | 13 |
Agree | 19 | 19 | No change | 14 |
Strongly Agree | 12 | 13 | | |
* using the Wilcoxon signed-rank test |
Our secondary objectives were to assess quantitative aspects related to reach and implementation. Reach is concerned with whether there was interest in the program under evaluation.(40) In this regard, 40/49 (82%) of the potential participants who were approached by the partner sites and who signed the consent to contact form attended the initial visit. Reasons documented for five of the nine who did not attend included: hospitalization (n = 2); change in their housing situation (n = 2); and a family member that did not trust the clinic, forbidding the potential participant from attending (n = 1). We had a high rate of return visits, with 33/40 (82.5%) of those who attended the initial visit also attended the follow-up visit. Reasons for not attending the second visit were: individual was barred from the site where the clinic was offered (n = 2); person had extended hospital stays after the initial visit (n = 2); participant moved out of town (n = 1); and death (of unrelated causes) days after the initial visit (n = 1).
There are several dimensions of implementation within the RE-AIM framework.(22) One of which is fidelity – which questions whether the intervention was delivered consistently and as it was intended in the study protocol. We found that generally, the SAFER intervention was delivered as intended, with 72% of participants receiving all four of the offered tests, and another 25% received three out of the four tests. There were various reasons that some did not complete all the screening tests. One individual had bilateral below-the-knee amputations, making foot assessment impossible. One-third of participants did not have their retinal screening completed for various reasons: technical problems/equipment failure (n = 2); refusal on the basis of not wanting eyes to be dilated (n = 3); no place to store contact lenses during the assessment (n = 1); inability to capture satisfactory images (n = 6); and already being followed by an eye care specialist regularly (n = 3). For nephropathy screening using urinary albumin quantification, two individuals declined to provide a urine sample for analysis.
Another domain of implementation pertains to the feasibility of the intervention and whether enough resources had been allocated to the program. The initial visit was intended to last 75 minutes. In practice, initial visits lasted 84 minutes, on average. This included the informed consent process and completion of the research surveys. The follow-up visit was intended to be 45 minutes in length. The average length of the follow-up visit was 60 minutes. We originally planned to see four participants in each four-hour clinic. However, given that each visit took longer than anticipated, this was never achieved. Oftentimes there were last-minute cancellations or no-shows, therefore, we were able to spend extra time as needed with each participant who attended.
The final domain of implementation we assessed was with respect to costs and resources. Throughout the program, a nurse was employed two days a week to implement the clinic. A research associate (RA) was hired one day a week to obtain informed consent, assist with questionnaires, conduct the interviews, and assist with other needs within the clinic as they arose, costing $43,680 for staff time to cover the 37 clinic dates offered. In preparation for implementing SAFER, the staff received distress and non-violent crisis intervention training. The nurse also received footcare and cardiopulmonary resuscitation for healthcare providers training, as well as training specific to each screening tool, costing a total of $5,430. Total supply costs for the program equalled $12,636. This value does not include the cost of the retinal camera or the point-of-care biospecimen analyzer as both were provided in kind to the project by our partners. We determined that it cost $846 per visit, or $1544 per participant screened (Table 4).
Table 4
Description | Costs |
Salary of one nurse (2 days per week) and one RA (1 day per week) x 52 weeks) | $43,680 |
Staff Training | $5430 |
Total cost of supplies (not including supplies gifted in kind to the program) | $12,636 |
Total cost of implementing SAFER pilot for one year | $61,746 |
Cost per clinic visit (total of 73 visits) ($61,746/73) | $845.83 |
Cost per patient (with 2 visits available to them) ($61,746/40) | $1543.65 |