A multicenter web-based survey with post-ICU patients was conducted. It was realized as a multi-centre quality assurance measure and quantitative data of potential users were collected as an empirical evaluation procedure. The survey was positively assessed by the ethics committee of the University Hospital RWTH Aachen (EK 269/22, August 2022) and the participating study centers. Participants received no payment or other compensation for their participation in the survey. The reporting complied to the CONSORT-EHEALTH checklist (V.1.6.1) by Eysenbach (15) for evaluation reports of web-based and mobile health interventions. As far as appropriate to the survey-study design, the items on the CONSORT-EHEALTH checklist were worked through.
Recruitment
Potential future app users were recruited to take part in this survey study. Patients, who were treated on ICU for at least 72 hours or who were mechanically ventilated for more than 24 hours, could participate. All patients had to be at least 18 years old. Pregnant women and patients who were unable to answer the questionnaire themselves were excluded. Patients were recruited from four different hospitals in Germany: University Hospital RWTH Aachen, University Hospital Jena, University Hospital Leipzig and St. Franziskus-Hospital Münster.
Web-based survey-study material
A clickable prototype of the developed PICOS app was built with the prototyping tool Adobe XD. The prototype was a way to make the planned design tangible and give potential users an impression of the planned functionality and design. The prototype included a main page where the users were able to track their registered vital values like blood glucose level, heart rate, blood pressure and weight as well as their activity in form of step tracking. The PHQ-4 questionnaire (patient health questionnaire-4) (16) with a 4-item scale asking for depression symptoms and anxiety was also included. No real data capture occurred when using this prototype.
In the PICOS app, the users would receive reminders about the completeness of their entries. From the main page, users were able to access an interface to manage their values directly: Update them or indicate that they are not relevant. From the main page, users could also access their medication plan, view their therapy plan, create an overview of caregivers and relatives, save documents or manage their doctor and hospital visits (see Fig. 1).
Goals of the design was an attractive und accessible interface with simple and intuitive operations. The user interface followed the principles of usability engineering for interactive systems formulated in Part 110 of DIN EN ISO 9241 as seven interaction principles for interactive systems: task appropriateness, self-descriptiveness, conformity to expectations, learnability, controllability, fault tolerance, customisability. Task appropriateness was achieved by limiting to appropriate functionality, minimising unnecessary interactions. Large buttons and displays were used to simplify the orientation. The design contained more but less cluttered screens for the individual functionalities, so users could focus on the essentials. Interaction aids and progress scales increased comprehensibility in order to achieve self-descriptiveness and learnability. Through the use of familiar design concepts and style guides, the design should conform to the expectations of the user. The user was able to control the content, course, direction and speed of their interactions due to controllability of the system. Fault tolerance was achieved through as few free entries as possible and mostly single- and multiple-choice questions, numerical ratings and visual analogue scales. The app could be adapted to individual needs by switching functionalities on and off.
<<< Fig. 1 should be placed here >>>
Left: Main page where users can register their vital values and activity, Middle: Graphical overview of blood sugar and heart rate over one week, Right: Access to store information such as medication plan, contacts of treating physicians/caregivers and relatives and additional documents as well as an appointment reminder
Questionnaires
The questionnaires covered the demographic data of the participants, their interaction with technology and their perception of the click-dummy of the developed PICOS app. Due to the vulnerability of the target group, the survey was kept as short as possible. The survey was evaluated through an expert panel with four researchers focussed on the relevance, flow and clarity of the chosen measurement instruments to construct an effective survey questionnaire following the recommendations of Ikart (17). All of the used items were already validated and used in other surveys, therefore the question structure, objectivity and look as well as feel of the questions were not further discussed (17).
For demographics, the participants were asked about their age, their education level and their gender identity. In addition, participants were asked if they own or use a smartphone and if they would like support (e.g., from family members or caregivers) when using the PICOS app.
The interaction with technology was operationalised by the German version of the Affinity for Technology Interaction (ATI) scale developed by Franke et al. (18). This scale consists of nine items and uses a 6-point Likert scale from 1 (completely disagree) to 6 (completely agree) and is based on the need for cognition-theory (19). Need for cognition is described as one’s personality trait to engage in and enjoy cognitively demanding tasks out of an intrinsic motivation. The ATI is conceptualised as a domain-specific variant of the need-for-cognition-construct. The construct validity, dimensionality and reliability of the ATI were tested with good to excellent results in five different studies (18) with samples of students of different study programs, activity tracker users, school students an US American online sample and a German quota sample. In the mentioned studies Cronbach’s alpha coefficients ranged in the samples between .83 and .92. The ATI is a recommended tool for investigating user diversity in usability testing and evaluation studies.
The perception of the PICOS app was measured by the scale by Krishnan et al. (20) which addresses the intention to use CHI (consumer health informatics) applications. The questionnaire was developed based on the Theory of Reasoned Action by Ajzen and Fishbein (21), the Technology Acceptance Model by Davis (22) and the Extended Unified Theory of Acceptance and Use of Technology by Lewis et al. (23). Because representatives of the target group quickly showed signs of exhaustion in the pre-testing the number of items had to be reduced by the expert panel. Therefore, only the following dimensions of the scale were used: Hedonic motivation consisting of 7 items, perceived ease of use consisting of 6 items and performance expectancy consisting of 5 items. These dimensions showed a significant linear relationship with the adoption behaviour of CHI-Applications (20, 24–27), so we assume the overall value for all these dimensions as the intention to use the PICOS app. The participants rated their approval of the items on a scale from 1 (completely disagree) to 7 (completely agree).
Hypotheses
The questionnaires used are intended to analyse the characteristics of the target group and the relationship between these characteristics. It is to be expected that subjects with different personality traits have different perceptions and requirements of the app, based on existing research: The study results of Franke et al.(18) showed significant gender differences in Affinity for Technology Interaction with woman having a significantly lower ATI-score than men and a significant weak negative correlation between age and ATI-score. Other studies indicate significant age and gender differences in perception and acceptance of mobile applications (28–30). In this study, similar effects are expected in terms of age and gender. Hence, the following hypotheses are to be tested:
H1: There is a significant relationship between the affinity for technology interaction and
-
…the age of the participants.
-
…the gender identity of the participants.
-
…the possession and usage of a smartphone.
-
…the interest in support (e.g. from family members or caregivers) when using the PICOS app.
H2: There is a significant relationship between the perception of the PICOS app (hedonic motivation, perceived ease of use, performance expectancy, overall intention to use the app) and
-
…the age of the participants.
-
…the gender identity of the participants.
-
…the possession and usage of a smartphone.
-
…the interest in support (e.g. from family members or caregivers) when using the PICOS app.
H3: There is a significant relationship between the affinity for technology interaction and the perception of the PICOS app.
Procedure
The web-based survey was conducted between 01.09.2022 and 01.09.2023. It was implemented as an online survey via LimeSurvey, but participants completed the questionnaires with the help of a study nurse. At the beginning of the evaluation, all participants received a briefing about the background of the research project and the questionnaires, the duration and scope of the survey and information regarding the data protection and anonymity measures. The survey started after informed consent. First, the participants filled in demographic information and the ATI. After that, they got the opportunity to look at the mock-ups and tried out the click-dummy of the PICOS app prototype. Subsequently, the participants gave their impression of the app and evaluated it through the scale by Krishnan et al. (20). The entire survey did not take longer than 15 minutes for the participants.
Statistical Analysis
The collected data were analysed with the IBM SPSS Statistics 28 analysis software. The collected data were analysed descriptively, and the frequency, mean values and standard distributions are reported. Normal distribution was tested by the Shapiro-Wilk test because it is the more powerful than the Kolmogorov-Smirnov test (31). Correlations were tested through Spearman’s ρ for ordinal variables or rejected normal distribution. For evaluation of the Spearman’s ρ effect sizes, the classification according to Cohen (32) was chosen with 0.10–0.29 as a small/weak effect 0.30–0.49 as a medium/moderate effect and > = 0.50 as a large/strong effect. Since the sample was sufficiently large (n > 30 for both groups) (33) and because the t-Test is considered robust to a violation of the normal distribution (34), the results of the test for normal distribution could be ignored when performing t-Tests. Levene’s Test was used to test for homogeneity of variances and depending on the results unpaired t-Test for homogeneous or heterogeneous variances (Welch Test) with a significance level of .05.