Over a year, nearly 80% of the respondents to the French PE survey left a comment after their hospitalisation in the HCL. The analysis of the surveys containing comments allowed to create a patient journey with a very high number of precise sub-themes. More than 15% of these were prioritised using the present methodology and are described in operational sheets to help institutions, departments, and staff conduct quality improvements regarding four key categories: relationship between patient and staff; environment; surgery and pain management; information and care coordination.
The results obtained herein reflect those found in the existing literature, particularly the fact that more than half of the themes mentioned in patient comments were not present in the survey’s closed-ended questions [16] and that patient narratives can improve healthcare quality beyond what standardized survey scores can accomplish [10,17]. The most commented themes and sub-themes identified herein also confirm those reported in the literature, especially the importance of patient-staff interaction to improve patient journey [18], and the need to improve information in the aftercare [19]. Some expectations reported by the patients are specific to the present analysis as they concern the specific institution commented on (for example, the orientation issues) or are specific to French hospitals (for example, the incomprehension of patients who find themselves in a shared room despite their request for a single one and despite the fact that they pay their insurance company to cover this particular service).This approach places the patient's view at the heart of the institutional strategy while combining it with action levers that are directed at the department and staff level. It also encourages the engagement of patients by bringing patient comments to light. Communication of the results and the actions defined should also increase the overall engagement of patients in the process. Identifying such actions at different levels and on different time scales allows to rapidly implement targeted actions at the department and staff level while developing long-term institutional strategies, thus enabling to keep a balance between the improvements needed and the daily functioning of an academic hospital federation. For example, a first step consisted in valuing staff for their work by sharing the practices commented on by patients and the good quantitative and qualitative feedbacks. To this end, the modelling of the patient journey represented a good pedagogical tool when sharing the results to the staff. Another type of rapid action concerns practical improvements (e.g., orientation issues, information before admission, birth registration process) implemented by support teams, thus avoiding the burdening of front-line professionals. In a second and longer step, improvements such as the need to improve aftercare information, became part of the institutional strategy which aimed to engage staff at different moments of the patient journey (e.g.: after care, during preparation for discharge, follow-up after discharge, etc.). Some of the improvements needed (such as pain management and behaviours during visits) are not limited to the institutions included as such issues can be addressed during the initial training of the professionals (schools or universities). In this specific context of an academic hospital federation, the results will be shared with the actors involved in initial training with the aim to raise awareness on patients’ expectations as early as possible.
The methodology applied herein is particularly relevant to reflect PE and conduct changes at the staff, department, and institution level. For instance, another analysis conducted on the same material but using a different methodology (word and comment filtering and non-negative matrix factorisation algorithm), allowed to identify about twice as less themes compared to the present methodology [20]. That specific analysis, conducted by the French National Authority for Health (Haute Autorité de Santé, HAS), differed however, as it aimed to identify frequently raised themes and included more than two million comments. Another difference relates to the choice to analyse themes and sub-themes commonly, whether they are mentioned in the positive or negative question. This approach, which was applied herein, allows to measure how often a theme or a sub-theme is mentioned (positively or negatively), to determine the mean overall satisfaction regarding the theme/sub-theme, and to measure the proportion of positive and negative speech. This therefore allows to identify the reasons of dissatisfaction but also the practices valued by the patients on a same theme. Furthermore, when sharing the results with the staff, it contributes to conveying the results in a positive and engaging manner. The constraint of a joint approach, however, lies in the additional analytical time required, as it is necessary to differentiate positive, neutral, and negative speech. The present analysis also proposes to consider the syntax of the comments, allowing to measure speech engagement [14], to go beyond the mere recurrence of themes in order to prioritise improvement actions, and to highlight sub-themes that are rarely mentioned. In both approaches, the quantification of the identified themes allowed to transform qualitative material into measurable and comparable data that could allow comparisons in the long-term, given that the expression mode of patients remains stable over time. This expression mode depends on the case mix (age, sex, type of disease), the educational, and socio-economic level of patients [21–22]. The selection of an analytical method must thus consider the number of respondents, the expected outcomes, the time constraints, and the technical challenges. A manual analysis, which can be time consuming, enables to overcome certain challenges met by natural language processing (NLP) as, according to Greaves et al., “machines struggle to read and understand comments accurately; software finds comments preceded by negatives difficult to interpret”9. Furthermore, the use of sarcasm and irony – a feature of British and French cultures- is hard to process by machines [9]. The proposed approach enables to analyse many patient comments, allowing a comparability of results between institutions and over time, which is difficult to obtain on small samples using methods such as qualitative interviews or focus groups.
The present study has certain limitations. The use of comments from open-ended questions that are placed after closed-ended questions may bias the content of the comments. However, one study tested the variation in narrative content and quantitative scores as a function of the placement of open-ended questions on the Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey. The results showed that the relative placement of closed and open-ended questions had little impact on CAHPS narratives or scores [23]. The use of the e-Satis survey implies a bias in the selection of respondents, as it implies needing an e-mail address, answering the online survey, etc. This bias, however, is limited by the manual intervention which enables to identify specific sub-themes raised by only a few respondents and the use of patient characteristics that allows to target specific populations.
The next step in our research will consist in involving the staff to collect and analyse their perceptions regarding their interactions with patients in order to balance the present patient-centred results. This would allow to refine improvement actions to remain as close as possible to the staffs’ specific needs and engage staff in a dynamic process for improving their practice and patient care.