Patient characteristics and recruitment
During November and December 2019, 14 patients were screened and asked to participate in the study. Four patients did not want to participate due to a lack of motivation (n=4, 29%, median age: 27 years IQR 26-30) and one patient was excluded due to limited German language skills (n=1, 7%). Nine patients were enrolled, resulting in a recruitment rate of 64%. Median age was 35 years (IQR 29-56), whereby more females (n=6, 67%) than males (n=3, 33%) were included in the study. Five patients were diagnosed with AA, three with PNH and one with overlapping AA/PNH. Median disease duration was 9 years (IQR 6-13). Further patient characteristics are listed in Table 2.
Patient adherence
A total of 234 weekly reminders were sent and 168 questionnaires completed, resulting in a 72% questionnaire adherence rate. Figure 1 illustrates the monthly adherence rate, showing a decreasing rate from 91% (month 1) to 53% (month 6) over the total study period. One patient completed all questionnaires over the 6-month study period. Besides the weekly questionnaires, an additional questionnaire was filled out in 12 instances.
Symptom reporting and triggered actions
Figure 2 demonstrates all reported symptoms for AA and PNH respectively. A total of 331 symptoms were reported, of which 154 (47%) were classified as mild, 95 (29%) as moderate and 82 (25%) as severe. The most common reported symptom was fatigue in AA (44 entries) and in PNH (43 entries). Symptoms of severe grade were most often reported as bleeding in AA (14 entries) and fatigue in PNH (21 entries). Eighty-two symptoms were graded as severe according to the predefined algorithm, 28 due to symptom severity, 23 due to interference with daily activity and 31 due to both reasons equally. Severe symptoms led to 36 alerts being sent out to the treating physicians. These alerts did not lead to additional physician contacts or admissions of the patients, as patients either managed the symptoms directly at home or presented themselves directly to their medical care team already before reporting these symptoms within the application. In addition to the predefined symptom questions, additional symptoms reported were nausea, cough, itching and chest pressure.
Besides symptom-monitoring, vital parameters were documented 81 times by 6 patients. Three patients did not record any vital parameters, resulting in an overall adherence of 35%. Blood values were documented by 5 patients with a total of 41 entries. The timepoint of eculizumab infusion was documented by one of the three PNH patients receiving eculizumab.
User experience and technical performance
All patients, two nurses and five physicians reported on their experience with the ePRO application, showing no significantly different opinions on the usability and efficiency of the tool after 3 weeks, 3 months and 6 months.
Technical problems were reported in 3 cases, all of which were due to difficulties with a password reset (two nurses and one physician). Patients did not report any technical problems or concerns on data security. The application was mostly accessed by smartphone (n = 7, 78%). Overall, the usability of the application was rated as “easy to use” while the symptom questionnaire was classified as “easy to understand” by all patients.
One patient mentioned difficulties in grading the severity of symptoms (“I had trouble deciding if my pain was minor or moderate”). Of the five patients who reported severe symptoms, two stated that the warnings and recommendations to contact a physician were in line with their personal experience and symptom management. Three patients reported that the warnings were only sometimes in agreement with their personal experience, of which all three stated that the warning for “fatigue” was triggered too soon. Four patients never triggered an automatic warning.
Reasons for non-adherence were forgetting (n=4, 44%), being too busy (n=4, 44%), the lack of any symptoms (n= 1, 11%) and not wanting to be reminded of the disease (n=1, 11%). While some patients (n = 3, 33%) deemed the self-instructions for symptom care useful, almost half of the patients (n = 4, 44%) regarded the self-instructions as “not very useful” since they were already trained in disease management after having lived with the disease for many years (median duration 9 years). They suggested that less experienced and newly diagnosed patients would benefit the most from this application. All patients rated the time expenditure as “appropriate”. While 6 patients (67%) would like to continue to use the app weekly, 2 patients (22%) would prefer to use it less often (every two weeks), and 1 patient (11%) more often (every 3 days). The integration of the app into daily life was rated as “very easy” by all patients.
Overall, all patients reported to be satisfied with the application. While 7 patients (78%) would continue to use the app, 2 patients (22%) would stop using it. All patients would recommend the application to other AA or PNH patients. Recommendations for improvement focused on the automatic integration of vital parameters, blood values and medication into the app, a calendar function for medication and hospital visits, a possibility to ask the care-team questions and voluntary social functions.
All members of the medical team described the application as “easy to use”. While more than half of the medical team members (n = 4, 57%) reported a benefit from the tool, others (n = 3, 43%) did not. Both nurses felt confident in their ability to screen and manage the reported symptoms. Though not everybody saw a personal benefit from the tool, all caretakers would continue to use the application due to the likely benefit provided to patients. All recommended that the tool should be integrated into the electronic hospital records and clinical work flow. To improve data interpretation, it was suggested to integrate data on therapies into the application. Another suggestion was an alarm function for patients directly integrated into the start page of the application. Data extraction from the interviews can be found in the supplemental materials (Table 5, 6 and 7).