Descriptive statistics
Demographic characteristics are shown in Table 1. Fifteen clients formed the analysis cohort (4 of these also provided data for the pre-intervention period). Mean participant age was 44.5 years (SD = ± 28.85) with a range of 3 to 84 years. Sixty per cent of participants were male. Participants were drawn from three sites; the majority were receiving services from CHAMP (73%), whilst the remainder were receiving services from CF.
Table 1
Demographic Characteristics (N = 15)
Characteristic | Participant Sample |
Age (Years) | |
Mean | 44.53 (SD = 28.85) |
Range | 3–84 |
Gender | |
Male | 9 (60.0%) |
Female | 6 (40.0%) |
Service | |
ICS | 11 (73.3%) |
CF | 4 (26.7%) |
Health Status | |
Type 2 diabetes | 6 (17.6%) |
Hypertension | 2 (5.9%) |
CHD | 3 (8.8%) |
PVD | 1 (2.9%) |
CVD | 2 (5.9%) |
COPD | 1 (2.9%) |
Mental Health | 7 (20.6) |
Other | 8 (23.5) |
Developmental Delay | 4 (11.8) |
CF Children and Families, CHD Coronary Heart Disease, PVD Peripheral vascular disease, COPD Chronic Obstructive Pulmonary Disease, CVD Cardiovascular disease, ICS Intermediate Care Services
Health-related quality of life (HRQoL) and Occupational Performance
The mean and standard deviation of HRQoL outcome scores, as assessed by the participants at baseline and 12 months follow up, are presented in Table 2. At baseline (pre-intervention), data were available for 4 participants, whilst complete data were available at follow up (post-intervention) for 15 participants. EQ-5D-5L scores improved by an average of 0.171 per person from 0.422 (standard deviation = 0.565) at baseline to 0.593 (standard deviation = 0.264) at the post-intervention period. This improvement was not statistically significant (p = 0.379). Clients had a clinically significant improvement (an increase of ≥ 2 points) in both their COPM performance and satisfaction change scores (4.25), respectively.
Table 2
Outcome Measures at Pre and Post Intervention
Measure | n | Mean | SD | Range (Max, Min) |
Pre-intervention Measures | | | | |
EQ5D pre intervention | 4 | 0.422 | 0.565 | 0.646 (1.0,-0.35) |
EQ5D Visual analogue Scale pre-intervention | 3 | | | |
COPM Performance Scale pre intervention | 2 | 3.5 | 0.71 | 1.0 (4,3) |
COPM Satisfaction Scale pre-intervention | 2 | 4.0 | 0.71 | 1.0 (4.5,3.5) |
Post-intervention Measures | | | | |
EQ5D post intervention | 9 | 0.593 | 0.264 | 0.781 (1.0, 0.219) |
EQ5D Visual analogue Scale post intervention | 9 | 63.53 | 19.01 | 66 (100, 34) |
COPM Performance Scale post intervention | 2 | 7.75 | 1.06 | 1.5 (8.5, 7) |
COPM Satisfaction Scale post intervention | 2 | 8.25 | 1.06 | 2.5 (9, 7.5) |
Change Scores | | | | |
EQ5D Incremental Change Score | - | 0.171 | 0.19 | P value = 0.379 |
EQ5D Visual analogue Scale Change Score | - | | | |
COPM Performance Scale Change Score | - | 4.25 | 0.35 | 0.5 (4.5, 4) |
COPM Satisfaction Scale Change Score | - | 4.25 | 0.35 | 0.5 (4.5, 4) |
COPM Canadian Occupational Performance Measure, EQ-5D EuroQoL-5 Dimension
Incremental costs and outcomes
Base case analysis
At 12 months follow-up (Table 3 ), mean per participant total costs were lower during the intervention than before the intervention was introduced (by $689 per patient). This difference was not statistically significant (95% CI: -$2,401 to $1,023, p-value = 0.431). The intervention was more effective than pre-intervention in terms of EQ-5D 5L-based QALYs. It was associated with 0.008 more QALYs gained per patient, this was also not statistically significant (95% CI: -0.353 to 0.370, p-value = 0.964) (Table 4 ). The resulting ICER was negative.
Table 3
Mean costs in 2021 Australian dollars ($) per patient presented
Variable | n | Intervention Mean (Standard error) | n | Pre-intervention Mean (Standard error) | Difference (Bootstrapped 95% CI) | p-value |
Total costs at baseline | 15 | 0 (0 | 15 | 0 (0) | 0 (0, 0) | 1.000 |
Total costs at 12 months | 15 | 1,984 (569) | 15 | 2,672 (1,194) | -689 (-2,537, 1,160) | 0.465 |
Incremental (baseline to 12 months) | | | | | | |
Total costs | 15 | 1,984 (1,031) | 15 | 2,672 (1,094) | -689 (-2,401, 1,023) | 0.431 |
Table 4
Mean outcomes (utility scores and quality-adjusted life years (QALYs) gained) per patient
Variable | n | Intervention Mean (Standard error) | n | Pre-intervention Mean (Standard error) | Difference (Bootstrapped 95% CI) | p-value |
EQ5D-5L Scores | | | | | | |
EQ5D-5L Scores at baseline | 4 | 0.441 (0.065) | 15 | 0.560 (0.027) | -0.119 (-0.261, 0.022) | 0.099 |
EQ5D-5L Scores at 12 months | 15 | 0.593 (0.039) | 4 | 0.441 (0.086) | 0.152 (-0.008, 0.313) | 0.063 |
EQ5D-5L gains (baseline to 12 months) | | | | | | |
EQ5D-5L Scores | 4 | 0.152 (0.111) | 4 | -0.119 (0.111) | 0.272 (0.014, 0.529) | 0.039 |
QALYs gains (baseline to 12 months) | | | | | | |
Adjusted QALY gains | 4 | 0.507 (0.264) | 4 | 0.498 (0.111) | 0.008 (-0.353, 0.370) | 0.964 |
a EQ5D-5L = EuroQoL 5 dimensions 5 levels; QALY = Quality Adjusted Life Years calculated using EQ5D-5L responses. QALY gains have been adjusted for differences between the two groups in the EQ5D-5L scores at baseline |
In the cost-effectiveness plane (CEPs) (Fig. 1), the bootstrapped paired estimates of mean differences in costs and QALY scores appear in all four quadrants. Suggesting considerable uncertainty in these economic evaluation results, which is further illustrated in the CEAC.
The CEAC presented in Fig. 2 depicts the probability of the intervention being the cost-effective option compared to usual care. At 12 months, the probability of the intervention being the cost-effective option compared to pre-intervention at a WTP threshold of $50,000 per QALY. QALY gained was about 86%. This probability rose to about 90% at a WTP threshold of $100,000 per QALY gained. (See also Table 4)
Telehealth Survey
Sixteen clients completed the survey, of which 14 received in-person services for their current problem prior to moving to telehealth services. Clients felt comfortable with the use of technology in everyday life (mean (SD) of 7.06 (2.46) on scale of 1–10), see Table 5), and all clients used a device, such as a mobile phone, and internet several times a week. Level of experience using telehealth technology was average (mean (SD) of 5.71 (3.27) on scale of 1–10) with some clients having no experience at all, and others being very experienced (see Table 5)
Table 5
Client’s responses in telehealth survey
Experience with technology |
In general, how confident are you using technology in day-to-day life? Scale 1 (never) to 10 (extremely confident), mean (SD), range | 7.06 (2.46), 1–10 |
How often do you use: Every day / several times a week, n (%) | |
Mobile phone | 14 (57.5) |
Computer (Desktop and/or laptop) | 10 (66.7) |
Email | 11 (68.8) |
Tablet (e.g. iPad) | 6 (40.0) |
Internet for any purpose | 13 (92.9) |
Online video conferencing platforms (eg. skype, facetime, zoom) | 2 (14.2) |
Social Media (e.g. Facebook, Twitter) | 11 (73.3) |
How experienced were you with telehealth technology prior to starting your telehealth consultations? Scale 1 (no experience at all) to 10 (extremely experienced), mean (SD), range | 5.71 (3.27), 1–10 |
Experience with telehealth consultations |
Have you had any telehealth consultations with any health professional in the past? Yes, n (%) | 13 (81.3) |
Have you had any telehealth consultations with current health professional in the past? Yes, n (%) | 11 (68.8) |
How effective did you find the therapy care received by telehealth? Scale 1 (not at all effective) to 10 (extremely effective), mean (SD), range | 6.44 (2.78), 1–10 |
How satisfied were you with the telehealth consultations you had with your therapist? Scale 1 (extremely dissatisfied) to 10 (extremely satisfied), mean (SD), range | 6.81 (2.66), 1–10 |
How satisfied were you with the care you received from the therapist? Scale 1 (extremely dissatisfied) to 10 (extremely satisfied), mean (SD), range | 7.63 (2.71), 1–10 |
How much confidence in your therapist’s ability to manage your problem via telehealth? Scale 1 (not at all confident) to 10 (extremely confident), mean (SD), range | 7.25 (2.84), 1–10 |
How likely would you be to recommend therapy via telehealth to someone else? Scale 1 (extremely unlikely) to 10 (extremely likely), mean (SD), range | 6.13 (2.63), 1–10 |
How likely to consult with therapist via telehealth after COVID-19 pandemic? Scale 1 (extremely unlikely) to 10 (extremely likely), mean (SD), range | 6.27 (3.15), 1–10 |
How easy was it using technology required for your telehealth consultations? Scale 1 (extremely difficult) to 10 (extremely easy), mean (SD), range | 8.63 (1.50), 5–10 |
Telehealth consultations compared to in-person consultations |
Have you had any telehealth consultations with current health professional in the past? Yes, n (%) | 7 (43.8) |
Prior to current telehealth consultations how did you rate the quality of a telehealth service compared to an in-person service? Scale 1 (telehealth much worse) to 7 (telehealth much better), mean (SD), range | 4.27 (1.85), 1–7 |
Before starting telehealth consultations did you have any in-person visits for your current problem/condition(s)? Yes, n (%) | 14 (87.5) |
How do you rate the quality of the telehealth service compared to an in-person service for your current problem/condition(s)? Scale 1 (telehealth much worse) to 7 (telehealth much better), mean (SD), range | 3.38 (1.96), 1–7 |
Your care team for your current problem/condition(s) |
Did your therapist provide any of the following resources to support management via telehealth? Yes, n (%) | |
Text message reminders | 11 (68.8) |
Follow up phone calls | 10 (62.5) |
Exercise apps for my smart phone tablet | 1 (6.3) |
Educational material about my condition | 8 (50.0) |
Written exercise instructions, diagrams or booklets | 8 (50.0) |
Exercise videos | 0 (0.0) |
Suggested websites for further information | 5 (31.3) |
Logbooks and diaries | 1 (6.3) |
Provision/purchase of exercise equipment | 1 (6.3) |
Provision/purchase of other equipment or devices | 3 (18.8) |
Other | 1 (6.3) |
Did you receive telehealth consultations from more than one health professional? Yes, n (%) | 12 (75.0) |
How would you rate your team’s ability on each of the following Scale 1 (terrible) to 7 (excellent), mean (SD), range | |
Use a team approach to assess your/your child’s health situation | 5.22 (1.20), 4–7 |
Use a team approach to coordinate all aspects of your care | 5.50 (1.08), 4–7 |
Include you in decision-making about your care | 6.20 (1.03) 4–7 |
Actively listen to your perspectives | 6.20 (1.03) 4–7 |
Thirteen out of 16 clients had telehealth consultations previously and at the time of completing the survey, 11 of those had a previous telehealth consultation with their current health professional. The results demonstrated a spread of experiences with regards to effectiveness, satisfaction, and confidence in the therapist’s ability to manage their health problem via telehealth, with ratings ranging from 1 (worst score) to 10 (best score), described in Table 5. All clients found using the telehealth technology required for their consultation sessions easy (mean (SD) 8.63 (1.50) on a scale of 1–10). (See Table 5)
Of the eleven clients who had previously had telehealth consultations with their current health professional prior to their current telehealth consultations, three (19%) reported the telehealth service was worse than an in-person service, four (25%) clients felt it was equivalent, and four clients (25%) rated the quality of a telehealth service better than an in-person service. Amongst all 16 clients, when comparing the telehealth service received for their current problem, to their prior in-person service received for the same condition, eight (50%) felt that the quality of the telehealth service was worse, four (25%) felt it was equivalent, and four (25%) felt the quality of the telehealth service was better than of the in-person service, refer to Table 5.
Fifteen (93.8%) of the 16 clients, indicated they were provided with additional resources to support the management of their health condition via the telehealth service, with 12/15 (80.0%) using more than one supporting resource. Text message reminders, 11 (68.8%), follow up phone calls, 10 (62.5%), educational material, 8 (50.0%) and written instructions or booklets 8 (50.0%) were commonly used.
Twelve (75%) clients received telehealth services from more than one health professional. Team approaches were rated fair to excellent with regards to assessment, coordination of care, inclusion of clients in decision-making processes and felt being actively listened to (See Table 5).
Qualitative results
Seven participants completed semi-structured interviews. Twenty staff participated in one of five focus groups conducted to gather perspectives on ICP and telehealth. The interviews averaged 25 minutes (range 15 to 42 minutes) and the focus groups 52 minutes (range 48–70 minutes).
Three themes were identified in interviews and focus groups: 1. Focus on client-centred care and teamwork; 2. Adjustments to communication; and 3. Value of face-to-face.
Indicative quotes under each theme are found in Table 6.
Table 6
Indicative Quotes for qualitative results
Quote Reference | Indicative quote |
Theme 1: Focus on client-centred care and teamwork |
6.1.1 | “They worked very well because they talked to each other and they talked to me.” (interviewee #15) |
6.1.2 | “Lots of times they had that information, obviously had done homework.” (interviewee #20) |
6.1.3 | “It’s kind of like saving patient time as well, so they see X first and their goal is to be able to walk around the shops and they see them after and I ask the same goal and they’re like getting that goal thing again. Whereas if I just read X’s note … I can say I noticed last week, your goal was to walk around the shops and they’ll be like yes, I did. And then it shows that we actually care, we collaborate, we work together.” (focus group 1 participant #F3) |
6.1.4 | “From my experience I haven’t found it to be different to face-to-face as to how we collaborate inter-professionally. We still do that really well no matter whether I see someone face-to-face, phone or video…. I would collaborate with you guys no matter what.” (focus group 1 participant #F1) |
6.1.5 | “But I wonder if that’s because we are all together, so let’s say we were working from home during COViD, that might have been a very different challenge” (focus group 1 participant #M2) |
6.1.6 | “I think that for me it was better because not…having to travel and not having to danger ourselves. But it was also very good for me, because she was a huge support for me mentally as well. So if I was really struggling with some of his behaviours…I could call her and she could call me back. Whereas face-to-face you have to make an appointment, you have to wait for that appointment” (interviewee #39) |
6.1.7 | “If someone was unwell or unable to come in, they could still participate in that…so both clinicians could still run that session with them and it could be a joint session as well.” (focus group 1 participant #F2) |
Theme 2: Adjustments to communication |
6.2.1 | “…there was just sort of one phone appointment…she was supposed to call me I think three times and failed to do so. And I never heard from her again, I was never given any information from her or follow ups. She never followed up with Social Work either.” (interviewee #39) |
6.2.2 | “I think it only works as good as the documentation of it, so if you’re not working together at the same time you’re just relying on reading each other’s comments.” (focus group 1 participant #M1) |
6.2.3 | “It’s just more the right information in there and that you can find, because everyone writes notes differently so it’s just about making sure that people are reporting the information and that they have an assessment and they have a plan and it’s clearly written out rather than just a paragraph.” (focus group 2 participant #F3) |
6.2.4 | “We reviewed how well we are getting clients to engage in setting their own goals and pathways. And we found that we weren’t very good at documenting that, so in that whole process we’ve now got a bit more of a structured case note template that we use, so as a team you can see each other’s information clearly and you can see what the clients are aiming for really clearly despite which individual disciplines talk with the individual.” (focus group participant #M2) |
6.2.5 | “It’s really good to hear how someone else looks at that and talks about it to a family.” (focus group 3 participant F5) |
6.2.6 | “For example you can ring a parent tomorrow, that I was going to do today but I ran out of time so I’ll do it tomorrow and give them the results of their assessment.” (focus group 4 participant F2) |
6.2.7 | “I guess probably the shadowing but that’s not just observing, that’s about wanting people to actually reflect on what they have seen and ask questions to gain more knowledge.” (focus group 2 participant #F1) |
Theme 3. Value of face-to-face |
6.3.1 | “It’s very well to have a phone conversation with somebody but if they haven’t met you they can’t really visualise, you know.” (interviewee #15) |
6.3.2 | “Pre-COVID the ideal was to do a phone assessment to get just general information and then do a face-to-face so that we can do the physical walk test and things like that. Over COVID we couldn’t do any of those walk tests, we couldn’t assess their mobility or their endurance and so we just had to prescribe exercises by just what they told us over the phone which was very difficult; but we just had to do the best we could.” (focus group 1 participant #F3) |
6.3.3 | “And when Teams was used it wasn’t a very effective replacement for meetings you had because they were shorter and they didn’t really have the conversation.” (focus group 2 participant #F4) |
6.3.4 | “I think it dropped out twice and in the end I just picked up the phone and rang.” (focus group 4 participant F1) |
6.3.5 | “Because when you’ve got three different people on there there’s always the lag.” (focus group 5 participant F3) |
6.3.6 | “A session with more than one clinician was rare. (Telehealth) was more helpful than two separate appointments.” (interviewee #11) |
6.3.7 | “When doing face-to-face it is a little more siloed.” (focus group 3 participant F1) |
1. Focus on client-centred care and teamwork
Participants, both staff and clients, generally felt that client-centred care and teamwork continued in the ICP telehealth context, due to COVID-19. Clients/carers were aware of the continued ICP and were happy with the impact on their care (6.1.1, 6.1.2).
Staff were conscious of the client’s perception of continuity of care (6.1.3). Staff highlighted that their physical co-location enabled them to enact ICP as they were together, while patient location did not matter. However, they thought that if staff had been working from home, this may have impacted overall quality of ICP(6.1.4, 6.1.5). Additional benefits were also identified through telehealth delivery in comparison to face-to-face appointments, with staff noting that fewer clients failed to attend their appointments (6.1.6, 6.1.7).
2. Adjustments to communication
Staff and clients noted different strategies were required when using telehealth. Where strategies to manage communication were implemented in the new context, sessions were more effective. Negative experiences frequently related to communication issues (6.2.1, 6.2.2, 6.2.3).
Staff reported improved communication processes due to working via telehealth (6.2.4, 6.2.5, 6.2.6). However, they also reported barriers to learning how to work in an ICP framework when they were unable to apply usual strategies to upskill new staff (6.2.7). 3. Value of face-to-face
Whilst staff and clients recognised the need for telehealth during COVID-19 lockdown, face- to- face opportunities, particularly for initial appointments and assessments were missed (6.3.1, 6.3.2).
Staff reported that face-to-face meetings were easier than online meetings (6.3.3).
Issues with technology were also raised, with staff believing that clients with poorer quality internet received poorer experiences (6.3.4, 6.3.5). However, it was acknowledged that face-to-face appointments could be limiting for ICP (6.3.6, 6.3.7)