This section first describes the intervention, and how it was designed and modified. In the second section, the feasibility study methodology is described.
The overall approach followed the UK Medical Research Council (MRC) framework (Skivington et al., 2021) and integrated existing evidence and the expertise of healthcare professionals.
Alongside the UK MRC framework, newly developed interventions were to be assessed for feasibility including the intervention and evaluation design (Skivington et al., 2021).
Phase 1 Intervention development
The design phase included the development of the technical system and the embedment into an overall study intervention. The technical development was guided by the user-centered, co-creative approach (Farao et al., 2020). This concept actively includes users in the development of an e-health intervention to prioritize their needs and finally obtain a tool adapted to the requirements of end users. The concept is divided into three overlapping and iteratively interconnected development stages. First, technical user requirements and relevant environmental factors of the application are revealed (relevance cycle). This leads to the creation of a technical prototype, which is evaluated by user involvement (design cycle). After reaching a certain level of readiness for use in the application context, it is deployed by the users with the aim of making further relevant adjustments from this application scenario (precision cycle). Our iterative intervention design process comprised the following steps (and methods):
- Identify utility needs on the VR -based home assessment (expert workshop with professionals involved in discharge planning processes).
- Identify usability problems of the technical prototype of a VR-based home assessment tool to increase user-friendliness (pre-pilot testing).
- Identify usability preferences of therapists to increase user-friendliness of ReTHo and VR-home representation (usability testing groups with OTs and PTs).
- Embedding the VR-based home assessment system into clinical processes (expert interviews with key persons involved in discharge planning processes and document analysis of clinical SOPs for patient flow in rehabilitation and professional assessment and discharge planning)
The development steps are presented in Figure 1: Steps and methods of intervention development.
Data analysis:
Qualitative data from expert workshops, pretesting, literature and usability testing groups were analyzed regarding aspects of functionality and usability according to the definitions of the concept of the “perceived usefulness” and “perceived ease of use” of the “Technology acceptance model” (Venkatesh & Hillol, 2008). The analysis approach taken to the dataset was both inductive and deductive. The high-level themes were set a priori by the targets of the analysis: useful functionality and ease of use in the user context. These yielded a deductive frame for analysis. Subthemes were closely linked to the data (incorporating the views on iterative versions of the technical prototype) and were analyzed in an inductive manner.
Datasets were produced and analyzed subsequently with each iteration step of the development phase.
Furthermore, the qualitative data from expert interviews and data analysis of discharge processes within the study setting were analyzed with the same technique, guided by the categories of the updated Consolidated Framework for Implementation Research (Damschroder et al., 2022) to identify context factors.
The characteristics of all the HCPs involved in the development steps are shown in detail in Additional File 1.
Expert workshop
The initial prototype concept (mock-up) comprised the following technical components: produce a 3D home scan (within the home environment), process data and organize data in backend (browser-based), and represent and use data (within the clinical setting).
First, a scientist (OT) and an IT-developer, who was also responsible for the subsequent realization of the user requirements for the design, led an online expert workshop. The topics of the workshop were: type and quality of home information needed for the respective duties in the discharge process, discussion of technical tools currently known from the literature to support the performance of pre-discharge home visits, presentation and discussion of the initial concept of the prototype (mock-up stage) together with the exploration of specific technical requirements for the latter, and the prioritization of demands. The workshop was audio-recorded and transcribed verbatim.
As the use of VR is not very common yet and none of the workshop participants had ever tried VR goggles, we sent VR goggles to three HCPs and asked them to try several applications to become accustomed to using them. In each online session with the IT-developer and one researcher, we discussed views on usability and ideas for functionality regarding the planned home assessment tool.
The initial and overarching user demands on functionality to perform a VR PDHA while the patient stays in the clinic were:
- To gain a general impression of the home environment.
- To measure the home and objects within it.
- To take notes within the system.
- All generated data (surveys and notes) should be exportable for further use.
In the next stage, the IT-expert developed a prototype to conduct virtual home visits at the patient`s real home by using the virtual twin of this home. The prototype system comprised the production of a 3D model of the home, the data processing of this model and the representation of the model in VR and 3D on the screen as well as a user desktop application with an interface to operate the data from each dataset.
Software (ReTHo) was developed to access, navigate and measure the virtual twin of the home on screen interface and in VR using VR goggles. Furthermore, we implemented features to note measurements by default and any other information as free access options into each “customer” account on the therapist´s user interface.
During the development phase from November 2020 to September 2021, a new smartphone model (I-Phone 12pro) with LIDAR technology and various, partly freely available LIDAR apps (e.g. Toolbox AI) for scanning the rooms could be integrated into our technical system and ReTHo was interconnected with a Data Management System to process and organize 3D scans of patients´ homes in a data-backend.
Pretesting
Once the initial system was conceptualized and the initial software was set up, one research team member pretested each form of technology in the system and gave feedback to the IT-developer about the perspectives for usability and usefulness of the technologies: 1) conducting the scan, 2) operating the data management system 3) operating ReTHo and 4) operating and using representations via VR goggles and on the 3D desktop application.
During this pretesting phase, different options of scan-apps, designs of the user interface and functions for measuring and moving in VR and 3D models were tested. Additionally, from the researcher´s view, evidence from studies about usability aspects in virtual pre-discharge home assessment tools was included in the feedback. The main usability problems were the calibration of the immersive VR environment (leading to motion sickness) and the different performance qualities in mobile LIDAR-apps for scanning different textures and surfaces (e.g. reflecting surfaces like tiles in bathrooms and kitchens could not be scanned in good quality and these rooms are important for participation at home).
Usability testing groups
The goal was to identify any problems in functionality and usability regarding operating data representation, measurement functions and the use of ReTHo. Therefore, we conducted six user-testing groups. Field notes were taken during usability tests. The IT-developer iteratively solved problems and implemented users´ preferences.
Expert interviews
To achieve the best possible fit of the study intervention to the specific implementation context, the study intervention was adapted to the ward setting. Interviews with two leading therapists and one social worker from an inpatient rehabilitation clinic were conducted by telephone. Strategies to implement the intervention in clinical processes were discussed. These interviews were underlined by SOP documents reflecting clinical discharge preparation processes.
Results of intervention development
A total of 45 individual user demands for necessary functions or aspects of user friendliness were derived from pretesting, from the expert workshop and usability testing groups. The main themes in user demands are displayed in an additional file (Additional File 2). All technologies in the system, IT-infrastructures and implemented functions are shown in Table 1 Technical infrastructure and functions of the VR-based PDHA-tool. Figure 2 illustrates the concept and user scenario of the tool.
Table 1 Technical infrastructure and functions of the system
System technology
|
Hardware/software/IT-infrastructure
|
Functions
|
Produce a 3D home scan
|
- Smartphone with LIDAR-technology
- Smartphone application to produce 3D scans
|
- Scan room by room
- Export anonymous 3D data to cloud
|
Backend to process and organize data (technical writer role)
|
- Ability to transmit data digitally to a content management system
- Ability to store data (on a server from the Martin-Luther-University)
- Ability to perform technical writing (secured with password)
- Own data model to present data of each patient and to collect newly generated data while using the end-use application
- Interface to retrieve data from server and collect newly generated data from end-use application
|
- Log in with password for technical writer role
-Creating a new patient folder with folder ID-number
-Apply individual number of spaces for room data to the folder
-Import scan data from cloud to backend
-Assign scan data of rooms to each space and name the room
|
ReTHo desktop application
|
- External WLAN router (insufficient internet resources in clinic)
- Laptop, mouse
- User interface for organizing data, operable on the screen
|
- Downloading data at clinic
- Selecting subject account with respective list of scanned rooms
- Automatic appearance of measurement data, listed in the currently named room-based organizational interface
- Manual labelling and commenting of the list’s measurement data
- Select relevant hazards from a clinic-based pre-configured checklist and add to the room-based data list.
|
ReTHo application to represent 3D models on a screen or in VR
|
- Additional large screen
|
- Show rooms as 3D model and as floor plan on a screen
- Navigate through the rooms on the screen by moving the 3D model on the screen in three dimensions
- Measure with virtual folding rule in 3D model and floor plan
|
- VR goggles with external sensors
- Use of controllers
|
- Immersive VR- representation of rooms
- Navigate through the rooms in VR (physically move on the playing ground or "beam" in the VR room with controllers)
- Measure with two different measuring tools: distance meter ("laser meter") and virtual folding rule
|
Implementation strategies of the study intervention
Clinical outcome measures and routine data collection were checked for suitability for study data collection and integrated, if possible, to keep the burden for study therapists low. There were two therapists formally appointed as study therapists responsible for data collection on site. There were no formally appointed “intervention therapists”. Prior to the start of the study, one researcher informed all the therapeutic staff about the intervention and trained them in its implementation (n=29). Implementation strategies that had been applied before and during the pilot study were evaluated (an overview of the strategies is displayed in Additional File 3).
Intervention procedure
The home scan was performed in the patients´ homes by the first author or carers/family in the presence of the author. After completing the scan, the data were uploaded into the backend. The therapists downloaded 3D data and performed a virtual pre-discharge home visit during the rehabilitation period with patients and other participants or alone. The therapists could perform the home assessment once or repeat it, if necessary.
During the virtual home visit, the therapists took individual, relevant home measurements. Notes could be entered as free text on the user interface. Every data input item (measure or hazard or individual miscellaneous note) was required to be commented with a corresponding recommendation (e.g., for home modifications /aid installation or elimination of hazard or activity adaptation). Data could be exported as Excel spreadsheets with room-based lists of measurements and/or detected home hazards and/ or individual miscellaneous notes with corresponding recommendations.
The intervention is described in detail according to the TIDieR checklist (T. C. Hoffmann et al., 2014) and presented in the Additional File 4.
To focus process evaluation, a process-oriented logic intervention model was developed in accordance with the guideline by Rohwer et al. (Rohwer et al., 2016). The logic model of the intervention (figure 3) displays the a priori mechanisms of intervention activities that influence the intervention outcomes.
Phase 2 Feasibility testing
Design
The study was designed as an exploratory single-arm feasibility study to evaluate the acceptance and suitability of the underlying logic intervention model as well as the implementation of the intervention. Furthermore, we wanted to test the feasibility of the study procedures (including outcome measures, data collection procedures and recruitment process) to prepare a larger-scale study and to improve the intervention design.
The evaluation of the study processes was based on the MRC framework as an overarching research approach (Moore et al., 2015; Skivington et al., 2021). Since intervention stakeholders were therapists and patients, this process evaluation focused on the experiences and perspectives of both. Barriers and facilitators regarding delivering and receiving the intervention were analyzed following the Consolidated Framework for advancing Implementation Research (CFIR) (Damschroder et al., 2022).
Participants and Setting
Recruitment and implementation of the intervention took place in an inpatient rehabilitation clinic in a rural area in Germany with a large catchment area of up to 400 km. Recruitment was carried out at MEDIAN Saale Klinik Bad Kösen II, in the Departments of Neurology and Geriatrics, each of which was managed and staffed independently.
Eligibility Criteria
Patients were included in the trial according to the following criteria: (i) age at least 18 years, (ii) neurological diagnosis or geriatric syndrome (defined as “multimorbidity with impending risk of loss of autonomy in activities of daily living”), (iii) anticipated persistent functional limitation(s) associated with increased risk of falls in the home environment and/or need for environmental adaptations, and (iv) discharge destination home or uncertain.
Patients were excluded according to the following criteria: (i) not able to understand the goal of the intervention (e.g., due to severe aphasia or severe cognitive impairment), (ii) not able to give consent and their legal care was not taken over by relatives, (iii) lethal course of disease, (iv) living outside a catchment area of 100 km distance from the rehabilitation facility, (v) patient or relatives did not consent.
To increase the number of eligible patients four weeks after the start of recruitment, the eligibility criteria were adapted to (iv) living outside the defined catchment area of 150 km distance from the rehabilitation facility. Furthermore, “geriatric diagnoses or stroke only” was changed to “geriatric diagnoses or any neurological diagnoses”.
Identification and recruitment
After admission, all patients were screened for eligibility by the study therapists. After the eligible participants had been provided with comprehensive information by the study therapists, they could consent to participate. If other persons lived in the same home with the patient, a consent from this person was necessary for the study intervention and had to be obtained. The recruitment of patients and their relatives who were willing to participate in a problem-centered interview for process evaluation took place at the time when the research team conducted the standardized telephone follow-up, two weeks after discharge (T3).
For process evaluation, HCPs were interviewed who either had implemented the intervention or were present during the intervention. Furthermore, HCPs were interviewed who potentially could have used the intervention or could have been affected by the output of the intervention. All the recruited patients were invited to participate in an interview for process evaluation.
Data collection procedures
An overview of the research questions, data collection instruments and the analysis plan is displayed in Additional File 5.
Pre-screening list
To specify future inclusion criteria for an RCT, study therapists in each department recorded reasons for exclusion, using a standardized pre-screening list for every patient after admission.
Participant description
To describe the participants, baseline data collection from routine documentation was recorded, including: sociodemographic data, comorbidities, functional status (ability to walk; Functional Ambulation Categories (FAC) (Martin & Cameron, 1996; Mehrholz et al., 2007; Park & An, 2016) fall events, paralysis; Motricity Index (Collin & Wade, 1990), ADL status; Barthel Index (BI) (Collin et al., 1988) ) and patient assistive devices.
Process outcomes
The number of recommendations and identified issues/hazards was collected, as well as the rates of installed aids/adaptations and elimination of issues/hazards after discharge.
Feasibility and acceptance
Fidelity and dosage in delivering the intervention were collected using standardized process data on the use of technology during the virtual assessment as well as during interviews. Therapists rated satisfaction in usability and usefulness of the tool by answering four questions at the time point of performing the VR assessment (NRS 0-10). Satisfaction with the tool was explored regarding feeling supported by technology when discussing home issues with the patient and pursuing therapeutic goals. Furthermore, the perceived overall effort in using the tool as well as the level of enjoyment when using the technology was rated. Problem-centered semi-structured interviews were used to obtain the participants' experiences of the intervention as well as factors affecting the delivery/reception of the intervention as well as the process and patient outcomes. Interview guidelines were agreed on by the research team prior to use. Interviews with HCPs were conducted face to face and interviews with patients via telephone, one time each, and were carried out exclusively by one researcher (UKH) who was trained and involved in the project. Further information about the researcher´s personal characteristics according to CORREQ (Tong et al., 2007) is provided in the Additional File 6. The duration of each interview should not exceed 30 minutes taking the time constraints of the participants into consideration. Interviews were audio-recorded and transcribed verbatim.
(Guidelines can be requested from the authors.)
Patient outcome measures
In preparation for a subsequent effectiveness study, survey procedures and instruments were already tested to measure the effectiveness of the intervention. To assess performance and satisfaction with performance in client-identified daily activities in the individual home environment, we used the Canadian Occupational Performance Measure (COPM) (Law et al., 1990). Furthermore, the number of falls and fear of falling were measured.
Quantitative data were collected after admission (t0), at the time of intervention (t1), at discharge (t2) and two weeks after discharge (t3), by telephone or with self-administered questionnaires, respectively.
Sample size
As this was a feasibility study, there was no formal sample size calculation. The target number of participants was based on clinical estimates of the likely number of patients who fit the inclusion criteria and could be enrolled during the enrollment period.
The feasibility study was planned with a sample of 30 participants (15 each in the Department of Geriatrics and Department of Neurology).
Data analysis
As part of the feasibility study, quantitative data were analyzed and presented descriptively. Continuous characteristics are presented as the mean and standard deviation, and categorical variables are presented as absolute and percentage frequencies.
Qualitative data from the problem-centered interviews and group discussions were analyzed using a mixed deductive-inductive approach based on the structured approach of directed content analysis (Malterud, 2012). For this purpose, audio interviews were pseudonymously transcribed. Transcripts were not returned to the participants before analysis. MAXQDA software and Excel were used for the analysis. The themes and categories of the coding guideline were based on the mechanisms of impact and acceptance outcomes derived from the logical model defined a priori, also allowing for emerging themes.
Two members of the research team coded interviews independently in the first two stages of the analysis. Codes were condensed and synthesized by one researcher alone under the supervision of another researcher from the team.
To evaluate the study processes, the implementation and fit of the intervention within the setting and to develop the intervention further, context factors were extracted and structured according to the Consolidated Framework for Implementation Research based on user feedback (Damschroder et al., 2022). Possible recommendations for future intervention designs were derived from the lessons learned here.