Overall, some data were collected from all the 12 inaugural members and the 3 replacement members, although a complete dataset was not available for some. There were missing responses from two members and a further two members did not complete the introductory questionnaires; three of these four members were men.
Three CMO configurations were identified from the data, as shown in Table 1. All of them showed that using the portfolio facilitated growth in forum members’ personal development, but only where the members valued using the portfolio.
Table 1
CMO configurations from the realist evaluation.
Configuration | Title | Details |
1 | Valuing the portfolio as a tool to record and evidence activities | When members value the portfolio as a record of achievement (C), personal motivation to record and map transferable skills (M), facilitated enhanced skillset, especially in newly acquired skills (O) |
2 | Valuing the portfolio as a tool to facilitate reflective practice | When female members had previous experience of using reflective practice in a professional capacity (C), the personal motivation to adopt reflective practice to supports ones’ personal development (M) led to enhanced self-confidence and self-awareness (O) |
3 | Valuing portfolio as a guiding framework | When participants had a positive perception of the portfolio as a guiding framework (C) there was a willingness to use the portfolio (C). This ultimately enabled reflective practice (M) which enhanced reflective experiential practice, critical thinking and improved communication skills(O) |
Configuration 1 – Valuing the portfolio as a tool to record and evidence activities
This configuration suggests that, when members valued the portfolio as a record of achievement (C), personal motivation to record and map transferable skills (M), it facilitated an enhanced skillset, especially in newly acquired skills (O). The enabling mechanism was a positive perception that their portfolio would be useful in helping them obtain additional public service roles and or employment. Many of those in the public forum were keen to use their involvement as a stepping stone to membership of other public involvement groups. Thus the portfolio was useful as a document that summarised members’ progression throughout their involvement in the public forum. It was seen as providing members with a record and proof of what they had learnt and the skills they had developed. It was valued for use in applications and at interviews as means of demonstrating their employability and or a broader commitment to career development or societal advancement. For some, primarily male forum members, the portfolio acted as a living document which showcased how involvement in a public forum had made an important contribution to gaining practical skills that they could use in real world settings.
“If I were to apply for a role on another Health related forum or committee for example, the PIP [personal involvement portfolio] will show: - a desire to improve my knowledge and willingness to learn and develop”. (Member A, Male)
Two individual level contextual factors were identified as facilitating this mechanism. The first was a perception that the portfolio could act as a framework for continuing professional development i.e. that it could be employed as a tool in which the attainment of personal skills could be mapped against the requirement for external professional standards.
The portfolio will be really useful for my career development because it’s a record, of what I have done in my own time to improve my skills. This will enhance my employability because it demonstrates my commitment to learning new skills and ongoing development, things that are important in a professional development capacity. (Member C, Male)
The second was that the portfolio helped members identify gaps in their knowledge and this awareness encouraged some members to undertake additional study to improve their learning.
The PIP [personal involvement portfolio] has been important because I’m keeping a record of all my learning and development that’s specific to GM CHC and it has helped me spot where I'm a bit lacking in knowledge and so I have tried to address that so for example I have completed a MOOC about Antibiotic Resistance and know so much more now. (Member B, Male)
This subsequent additional learning frequently enhanced the members’ self-confidence and inspired greater involvement in public engagement events and activities within the GM CHC project.
However, not all the members valued the portfolio as a tool to record and evidence activities. The face to face consultations and exit interviews identified that the oldest members of the forum were less enthusiastic about the portfolio than the other members. These members were all retired and this may have influenced their attitude towards this aspect of the portfolio. They felt they had little reason to catalogue their activities within the broader context of this being useful in obtaining employment or additional public service roles.
Exit interviews with the three members who left the forum early provided a rich data source in which to explore how initial involvement in a public forum influenced members’ personal development. For these members, the portfolio played only a small role in enhancing their personal development but these skills had contributed to an improved sense of how to work effectively in a team or sharpened their reflective aptitude.
“I don’t think I really had the time to pick up new skills but writing down my personal statement did help me to appreciate that I had become a better listener and also that using the PIP [personal involvement portfolio] was helping me hone my reflective skills”. (Early exit member1, Female)
Many members (N = 7, five of whom were women), however, thought the portfolio was very useful for documenting their self-development. The portfolio worked well because it facilitated the development of personal skills, provided members with a hard-copy record of involvement and was an effective way of learning and developing both individually and for the programme. However this appeared to be linked to gender and previous experience of using reflection in a professional capacity. Male forum members primarily valued the portfolio for its utility as a record of achievement.
“The PIP [personal involvement portfolio] is a useful document for anyone wishing to apply for work or any voluntary role. It is a good document to support any application as it shows commitment, willingness to learn and contribute as well to self-development”. (Member D, Male)
Female forum members, however, also valued the portfolio because it facilitated personal learning and development, particularly those who had previous experience of using reflection in a professional capacity.
I have experience of evaluating my learning in a number of ways. For example in the courses I have undertaken in [name of company], maintaining a reflective journal was an essential learning tool and contributory element. Self-awareness has been engendered into both my formal learning and operational work. So writing it down is not new to me. I am also an organised person and like to review what I have done, reflect, so I can identify learning points to fulfil my drive to develop myself. (Member E, Female)
The degree to which gender may affect members’ relationship to diary based approaches to learning and development will be considered in the discussion.
The single most important outcome reported by eight forum members was increased self-confidence. This was typically described as a growing sense of belief that they could take part and be effective in public engagement activities. During the first year of the forum, members did not feel confident enough to help out at public engagement events. In the final six months of the project, their developing self-confidence saw some members participate in public engagement events, engaging in discussions with the public, running activities with school children and co-designing and delivering workshops for young people.
Even though I am used to leading, facilitating and generating discussion amongst groups and individuals when we started I didn’t feel confident about going out and getting involved in the public engagement activities. I've grown in confidence loads though and I have done loads of different activities and events this year and loved doing them. (Member J, Female)
Data from the face to face consultation sessions revealed that female members were particularly effusive about the affect the portfolio had on their transferable skills and the broader application of the newly acquired skills to other aspects of their lives. For some this meant taking part in digital activities which they had not previously come into contact with or did not have the confidence or expertise to tackle. For others it also encouraged health and wellbeing related behaviour change.
“It has encouraged me to apply knowledge acquirement to things I could not do before – taking part in webinars, using doodle poll e.g. joining the on-line [discussion] group for the [public engagement activity] was very self-motivating and helped me improve my fitness”. (Member F, Female)
Overall, it may be that this very pragmatic aspect of a member’s involvement in a public forum can help to demonstrate how the acquisition of tangible skills contributes to personal development.
Configuration 2 – Valuing the portfolio as a tool to facilitate reflective practice
This configuration suggests that, when female forum members had previous experience of using reflective practice in a professional capacity (C), the personal motivation to adopt reflective practice to supports ones’ personal development (M) led to enhanced self-confidence and self-awareness (O).As noted in configuration 1, previous experience of using reflective practice to support professional and personal development appeared to be the most important enabler of using the portfolio as more than a tool to record and evidence activities .
Whilst many members used reflection in some capacity, e.g. to demonstrate lessons learnt, only certain members used the portfolio as a mechanism that enabled and structured reflection and reflective learning. All three younger members were either studying, had recently completed their studies or were going back to studying; reflection formed or had formed part of their learning programme. However, it was notable that the two young men used reflection at a superficial level, primarily to help them identify skills gaps, as part of their broader commitment to enhancing their employability. The younger woman, however, engaged in critical reflection as a means of supporting and facilitating her ability to engage in challenging PEI pursuits and gain meaningful insights.
The data suggested that it was only female forum members, especially those who had previously used reflective practice (as quoted above), who actively valued reflection and pursued it to the point when they became capable of critical reflection. When they had attained this level of reflection, they described how this process had enhanced their personal development.
I was used to being reflective in my professional life but I am retired so it had been a while since I had used these skills. I always knew that it was just a case of working to reignite my skills. I have done that and acquired new skills as well. (Early exit member 2, Female)
Acquiring active listening skills was an important personal outcome for members for a number of reasons. It helped them build and maintain relationships with fellow public forum members, had a positive effect on their behaviour in forum meetings, workshops and public engagement activities and allowed them to see how much they could learn from others if they actively listened to and focused on the speakers’ message and point of view. This inspired members to be more considered in their response to others, a skill used to good effect when discussing use of health data with members of the public at public engagement events.
Collaborative working skills was an equally important outcome i.e. having learnt to employ their active listening skills, members communicated their point of view more effectively, accepted and promoted compromise, made collective decisions, became more open-minded, and successfully built and maintained functioning relationships with others. On a personal level, members felt that they also had a greater appreciation of the importance of embracing diversity of opinion, being respectful of views inconsistent with their own and valuing the skills, experiences, and contributions of others. These were all skills that they felt they could use to good effect in other public service roles.
The PIP [personal involvement portfolio] really helped me get to grips with reflecting on my own behaviour, especially in meetings. I realised that I never truly listened I was just waiting for my opportunity to speak but that meant I wasn’t learning from others. That penny dropping was important because when I actually started to listen I realised how much I could learn from the other members of the forum. (Member E, Female)
Configuration 3- Valuing the portfolio as a guiding framework
When members had a positive perception of the portfolio as a guiding framework (C) there was a willingness to use the portfolio (C). This ultimately enabled reflective practice (M) which enhanced reflective experiential practice, critical thinking and improved communication skills (O). The enabling mechanism was that personal motivation to adopt reflective practice in order to support personal development
Self- awareness is a skill which requires work to develop and I have been encouraged and supported to develop the skills for many years. However I believe it is like any other skill – if you stop doing it or don’t keep up the practice the skill becomes lazy and less well honed. So the forum has provided me with the medium to practice my skills, particularly circumspection. I haven’t always got it right but reflection allows me to recognise that so I can make adjustments. (Replacement Member 1, Female)
One, individual level, contextual factor identified as facilitating this mechanism was a perception that reflective practice could be harnessed to improve or attain certain skills which could feedback into society. The desire to make a positive contribution to research, public health or society motivated many to work through the challenges of developing critical reflection skills.
I want to give something back and getting involved in this project is a great way to do that. I also think it’s very important for the voice of the public to be included…and so although it has been time consuming and very hard at times I'm glad that I didn’t give up on the portfolio and my attempts to be more adept at reflection!(Member H, Male)
Ten members (seven of whom were women) thought that the portfolio had been helpful in encouraging such reflection. Face to face consultation sessions and exit interviews facilitated our understanding that being more reflective was part and parcel of how most members began to appreciate that the diary approach underpinning the portfolio enabled their personal development. By becoming more reflective, members became more aware of aspects of their personal development that they wished to enhance and made a strong commitment to learning in order to progress as an individual and as part of the team. However, it was notable that for others, primarily the men, the portfolio was used predominantly as a diary as a record of achievement.
For me personally I've not put that much effort in or looked at it outside of the meeting s(Member I, Male)
Internalised motivation appeared to underpin the key enabling mechanisms. Five members’ data identified motivation for personal development achieved through reflection as the primary catalyst for continued use of their portfolio. For all these member, motivated use of their portfolio related to using their new skills and learning for the benefit of the project and the broader public good. For those who were retired, the motivated use also related to building on their former professional practice in order to continue to develop as a person outside of the workplace. For those who were not retired, it also related to their personal and professional development.
Whilst these individual level contextual factors played an important role as enabling mechanisms, societal level contextual factors also played a part. The female forum members appeared to find reflection easier than their male counterparts. This was underpinned in part by previous experience of using reflective practice in a professional capacity, as described earlier. Analysis of all the strands of data revealed that female forum members were more motivated to use the portfolio and this was enabled by prior use of reflection in a professional capacity, which meant that they valued reflective practice. This prior use accelerated their competence in undertaking critical reflection and the subsequent enhancement of their personal development.
Members who used the portfolio as a framework to guide their learning and development demonstrated increased self-confidence, notably in their ability to undertake tasks that were previously unfamiliar to them. In particular, these were activities associated with digital technology (N = 5) and where members felt more able to assert themselves in public and professional arenas (N = 4). It was in these aligned domains of personal development that members recorded the greatest impact. One member cared for her mother and was responsible for assisting with her interactions with health professionals. Previously, meetings with her mother’s consultants had left the member feeling powerless and her voice ignored. Her enhanced self-confidence, greater knowledge and understanding of healthcare and the sense of authority to assert herself, enabled her to challenge her mother’s consultant about her treatment. As a result, valuable new information was discussed with the consultant, who listened to and acted on the information. This led to an improved health outcome for her mother.
I definitely feel able to challenge health professionals now. In fact I have done so on a number of occasions now and they have listened to me! (Member G, Female)
Three women identified the portfolio as the most successful element of their involvement in the public forum as it had enabled them to develop personal and professional insights and encouraged additional and improved learning behaviours. This may have been achieved because the very nature of writing and critically reflecting on one’s thoughts in a journal facilitates the latter’s ability to understand how theory and practice inform each other, in effect acting as bridging concept or connection between theory and practice.
“Playing devil’s advocate to my own theories, considering how my behaviours impacted on others. I could do this is an oral way but that only captures the moment and I need to be able to recall and revisit learning. Learning evolves and therefore needs to be reviewed. Actions implemented, theories tested, amended adopted or rejected and changes made. I can only make my learning meaningful and capable of meeting these objectives if I keep a written record”. (Member G, Female)
Nearly half of the members (N = 6, including four women) also reported that the portfolio was a useful mechanism for cascading elements of what they learnt to other public forum members and members of the broader public.
I am always talking to other people about what I have learned and I find that most people are really receptive to what I tell them and quite a few would like to learn more (Member D, Male)
However, six members thought that the portfolio was ‘not at all useful’ for cascading elements of what was learnt to these others. This was reported regardless of demographic characteristics such as gender, age and employment status. It may be that the diary method which is predicated on a very personal and private recollection of one’s inner most thoughts is not conducive to the more collective communication of learning to others.