What are the most important unanswered research questions in trial retention? A James Lind Alliance Priority Setting Partnership – The PRioRiTy II (Prioritising Retention in Randomised Trials) Study

doi:10.21203/rs.2.439/v1

Download PDF

Methodology

What are the most important unanswered research questions in trial retention? A James Lind Alliance Priority Setting Partnership – The PRioRiTy II (Prioritising Retention in Randomised Trials) Study

https://doi.org/10.21203/rs.2.439/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 15 Oct, 2019

Read the published version in Trials →

You are reading this latest preprint version

Background

One of the top three research priorities for the UK clinical trial community is to address the gap in evidence-based approaches to improving participant retention in randomised trials. Despite this, there is little evidence supporting methods to improve retention. This paper reports the PRioRiTy II project, a Priority Setting Partnership (PSP) that identified and prioritised unanswered questions and uncertainties around trial retention in collaboration with key stakeholders.

Methods

This PSP was conducted in collaboration with the James Lind Alliance, a non-profit making initiative to support key stakeholders (researchers, patients and public) in jointly identifying and agreeing priority research questions.

There were three stages

(i) An initial online survey consisting of six open-ended questions about retention in randomised trials. Responses were coded into thematic groups to create a longlist of questions. The longlist of questions was checked against existing evidence to ensure they had not been answered by existing research. (ii) An interim stage, which involved a further online survey where stakeholders were asked to select questions of key importance from the longlist.. (iii) A face-to-face consensus meeting, where key stakeholder representatives agreed an ordered list of 21 unanswered research questions for methods of improving retention in randomised trials.

Results

456 respondents yielded 2,431 answers to six open ended questions, from which 372 questions specifically about retention were identified. Further analysis included thematically grouping all data items within answers and merging questions in consultation with the steering group. This produced 27 questions for further rating during the interim survey. The top 21 questions from the interim online survey were brought to a face-to-face consensus meeting, in which key stakeholder representatives prioritised the order. The Top 10 of these is reported in this paper. The number one ranked question was “What motivates a participant’s decision to complete a clinical trial?” The entire list will be available at www.priorityresearch.ie.

Conclusion

The Top 10 list can inform the direction of future research on trial methods and be used by funders to guide projects aiming to address and improve retention in randomised trials.

Internal Medicine Specialties

Trials methodology

Retention challenges

Participation in randomised trials

Participant retention

Priority setting partnership

James Lind Alliance

PPI

Randomised trials are essential for evidence-based health and social care, though many struggle to recruit and retain participants. This poses significant problems for the overall reliability and generalisability of results when recruitment goals are missed, or participants are lost to follow-up. One of the top three priorities for research into randomised trials, as set out by the UK clinical trials community, is to address the gap in evidence-based approaches to improving retention in randomised trials [1]. This priority setting partnership (PSP) addresses this by investigating unanswered research questions for how to improve retention in randomised trials. Many trial participants drop out before its completion, sometimes in excess of 20%, and 50% of trials in the UK have a loss to follow-up of over 11% [2]. As retention within a trial decreases, so too does its credibility and our ability to say with confidence the results are accurate. This also affects the trial’s ability to contribute towards changes in clinical practice. Without any improvements to retention, trials may be hindered in both their applicability and ability to guide evidence-based care [3].

Despite this, there is currently little evidence other than anecdotal accounts about how to improve retention in randomised trials, as opposed to improvements to recruitment for which there is substantial evidence. Similarly, the Cochrane review on trial retention at present only has evidence relating to financial incentives to return postal questionnaires and use of second-class versus first class postage. It also does not include interventions designed specifically to address an individual’s reasons for dropping out of a trial [4]. Current strategies to improve retention in randomised trials could be bolstered by this collaborative priority setting partnership, as current guidance is limited to primary care trials only [5] or has been developed without equal patient involvement [6, 7]. To address this, further investigation was necessary with a wider range of stakeholders involved to identify research uncertainties.

These uncertainties about how to retain participants mean that at present sample sizes of trials tend to be increased in anticipation of a loss to follow-up. As well as this, many clinical trial units and Chief Investigators use strategies to compensate for missing data with little evidence to support their effectiveness [7]. Around 50% of trials fail to reach their recruitment targets [2], and the average cost of a clinical trial in the UK per participant is around £8,500 [8]. Countering loss to follow-up through increases to sample size uses resources that could otherwise be put to better use. Collecting insufficient data to address the aims of a trial reduces our ability to make meaningful use of the generosity of trial participants who enter trials in good faith. It is therefore not only financially prudent to improve trial retention, but important from an ethical perspective in order to reduce research waste and provide better care [9].

For these reasons, there is an urgent need to conduct further research into trial retention. Moreover, this research should be focussed on the areas considered essential to stakeholders involved directly in trial retention. This paper outlines the areas of research that key stakeholders believe should be the focus of future efforts to improve retention in randomised trials. We have defined non-retention as instances in which participants are prematurely “off-study”, such as withdrawal of consent or loss to follow-up, and therefore outcome data cannot be obtained. This is in line with the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidelines [10]. This research builds on the methodological framework used in the PRioRiTy I project [11], which used a James Lind Alliance (JLA) priority setting partnership to identify the top priorities for research into participant recruitment to randomised trials. The JLA method of bringing together relevant stakeholders to decide on research priorities has been used extensively for setting clinical research priorities [12, 13]. However, the application of the JLA method to methodological research is a recent development first utilised in PRioRiTy I [11].

PRioRiTy II viewed the involvement of patient partners in the governance of the project as an integral and essential part of our whole approach. This built and expanded on the PRioRiTY I study by adding more members to the Steering Group. The patient partners included individuals who were new to research methodology as well as some experienced representatives. The impact of this involvement will be discussed in separate publications.

This PRioRiTy II PSP followed the same overall method to that used during the PRioRiTy I PSP [11]. The initial stage consisted of data collection and analysis to generate a list of unanswered questions. This led onto an interim stage that used a ‘back-translation’ process to generate an indicative question list for use in the interim survey. The project then culminated in bringing 21 questions from the indicative list to a final consensus meeting to agree the top priorities for future research into trial retention. These priorities are uncertainties raised by the stakeholders and judged to be unanswered by existing evidence. All three stages of this project were open to anyone over the age of 18 years who had been involved in randomised trials in the U.K. and Ireland. For better precision during data collection, seven categories were given as options to describe the role of the stakeholders during the initial survey. For the remainder of the project, we combined stakeholder roles and organised them into four groups, listed below.

· Patient or member of the public involved in a trial (as a participant or parent/carer of a participant, or as a contributor to design/delivery of trial)

· Frontline staff or other staff involved in trial retention (e.g. Research Nurse, Trial Manager, regulatory or oversight role such as Sponsor or Research Director)

· Investigator (e.g. Chief Investigator, Principal Investigator, Co-investigator)

· Trial Methodologist

This PSP did not consider uncertainties relating to adherence to trial interventions. The objectives of the PSP were to:

· Bring the public, clinicians and researchers together to identify unanswered questions around retention in randomised trials;

· Agree by consensus a prioritised list of those unanswered questions which will be used to inform future research.

Steering Group

We established a Steering Group to oversee the PSP in accordance with JLA guidance and held the first meeting in January 2018. Membership reflected the range of stakeholders with whom we wished to engage during the PSP. Drawing on members’ expertise and networks, the Steering Group helped identify and recruit stakeholders during each stage. We also held regular meetings to ensure the work proceeded to agreed timetables and to continue engagement and momentum. The JLA was also represented on the Steering group to ensure the process adhered to JLA principles.

Initial Online Survey

Identification of Stakeholders and Development of Initial Survey

Steering Group members, including the patient partners, identified and engaged a wide range of appropriate potential stakeholders through their networks of contacts. We developed an eight-question online survey in SurveyMonkey to gather uncertainties for our initial stage. This was open for four weeks. We also made a paper copy of the survey with pre-paid return envelopes available if required. We set no formal target sample size for the number of responses. The eight questions included six open-ended questions [Appendix 1] that were modelled on those used for PRioRITy I, and an additional two demographic questions about the respondents to help monitor the geographic spread and roles of people responding to the survey. We then distributed a link to the survey to the four stakeholder groups (see Methods section for an explanation of the reduction of stakeholder groups between phases), as well as promoting the survey through social media channels and twitter hashtags.

We also asked respondents if they would consider attending the final consensus meeting. The initial survey questions aimed to identify unanswered questions and general comments relating to retention in randomised trials that stakeholders would like to see answered. The Steering Group designed the survey and conducted a pilot with a small sample of stakeholders (n=6).

Coding and Analysing Responses

The initial survey was hosted by the JLA, who provided the Steering Group with regular updates and the compiled answers once completed. We used samples of responses as they were returned to us to identify key themes and questions. This allowed us to generate a representative series of thematic groups efficiently once the survey closed. The JLA collated the survey responses into a single Excel spreadsheet, and we coded the responses using a process of constant comparison analysis [14] into a thematic group where appropriate. We repeated this process of comparison until the range and number of thematic groups was a true reflection of the whole data set. The determination of thematic groups was an iterative process and generated through discussion amongst the Aberdeen team members, who also conducted the subsequent analysis (DB, HG, KG, and ST). Where an item did not fit into an existing thematic group, we either expanded a thematic group or created a new one. Each stakeholder response could contain numerous items and multiple themes may have arisen. We therefore subdivided responses into constituent parts during coding to allow their mapping across different thematic groups. We did not assign responses that were out-of-scope to a thematic code, rather, we categorised these responses separately for potential future use. All responses that did not refer to a process (such as recruitment) were assumed to be about retention as per the survey questions and hence were considered to be within scope. This process also involved regular group discussion and consultation to ensure consistency in approach and accuracy of the coding.

Once coded, we analysed the data to determine the initial sub-questions and broader main questions present within each theme as well as how often they occurred over the course of four weeks. To guide this process, we used word for word responses as a framework for developing the sub-questions, which grew iteratively as the data were analysed. We compiled the broader questions from each theme together and conducted a check to ensure they remained representative of their respective sub-questions. To evaluate reliability, once we connected all coded data items to sub-questions, a 10% sample was selected at random from each team member’s analysis to compare findings. We held group discussions to identify discrepancies and resolve disagreements. Alongside this, we conducted a check with the questions identified in the initial scoping survey against existing sources of evidence reporting trial retention research. This ensured questions raised for the interim stage were unanswered by research. The evidence sources used for checking were:

A. The Cochrane review of interventions to improve retention in trials, with the 2012 search updated by members of the PRioRITy II team (October 2017) and screened [4];

B. A qualitative synthesis of barriers and facilitators to participant retention in trials [15];

C. A systematic review of non-randomised evaluations of interventions to improve retention in trials [16].

Together with the Steering Group, we grouped and merged the longlist of broad questions where appropriate and removed duplicates to create a shortlist of questions in advance of the interim stage. Through consultation with the Steering Group, we discussed and sometimes revised the terminology to improve clarity of the original meaning of the questions whilst ensuring the items remained true to the voices of respondents.

Interim Priority Setting Stage

Development of the indicative question list and interim survey

In the interim stage, we conducted a ‘back-translation’ on the initial stage shortlist in which we asked for feedback and comments from stakeholders who were not involved in the project on their understandings of the questions and to provide examples of the types of activities or interventions they would expect to see covered by each question. We created a list of indicative questions for the interim survey using further comments from the Steering Group along with the results of the ‘back-translation’.

For the interim survey, we used SurveyMonkey to ask stakeholders to choose up to 10 of the questions that they believed were the most important. This survey was open for six weeks and we made paper copies of the survey available if required. Invitations to this were open to anyone, and not restricted to the participants from the initial survey. As with the initial survey, no formal target sample size was set. However, the number of respondents within each reported group was checked weekly. This allowed us to target groups with lower representation during the ongoing dissemination of the survey.

We distributed the survey link through email, institution websites, blogs, newsletters, and social media. The Health Services Research Unit (HSRU) at the University of Aberdeen also issued a press release and coordinated promotion alongside the JLA.

Voting and ranking interim survey items

The online survey included a drop-down menu showing the indicative question list from which no more than 10 could be selected. This generated a total score for each question to represent the overall number of times the question was selected. We also used ranked weighted scores to decide which of the interim survey research questions would be taken forward to the final consensus meeting, following the standard JLA approach as described in the JLA Guidebook [17] (www.jla.nihr.ac.uk/jla-guidebook/):

Each time a question was chosen, we assigned it one point. To ensure equal influence, points for each stakeholder group were tallied separately, generating separate total scores for each group for the questions. Within each of the four stakeholder groups, the scores for each question were put into order from highest to lowest. We then gave these a new score according to their position, from 27 for the most popular question down to one for the least popular. This resulted in the lowest ranked question receiving the lowest total score, through to the highest ranked question receiving the highest. The list was then ordered by score from highest to lowest and presented to the Steering Group. In cases when the questions had the same total, we ranked these in joint place. This gave the overall interim ranking to the research questions and the rankings for each of the stakeholder groups, whilst minimising bias owing to numbers of responses from each stakeholder group.

Consensus Meeting

The final prioritisation consensus meeting was held in Birmingham, UK, in October 2018 to identify and agree a ‘Top 10’ list of research questions. We brought together representatives from the key stakeholder groups (in roughly equal numbers) to agree the Top 10 list of priorities from the top 21 questions from the interim survey. The consensus meeting followed the standard approach described in the JLA Guidebook, namely using small and whole group discussions in a face-to-face meeting with a particular emphasis on the Top 10 [17] (www.jla.nihr.ac.uk/jla-guidebook/). We remunerated patient participants for their time according to INVOLVE UK guidance, and travel expenses for all attendees were reimbursed. Members of the Aberdeen team planned and organised the event alongside members of the JLA.

The consensus meeting was a full day of plenary and small group discussion, chaired by a JLA Senior Adviser. All attendees were provided with the list of 21 questions in advance of the meeting, to allow time to familiarise themselves with the questions and consider their thoughts on the importance of each one. A JLA facilitator led each of three small groups, which consisted of even representation of the stakeholder groups. The JLA facilitators acted as neutral guides for the process and ensured equal participation. The small groups were asked to discuss and prioritise all the listed questions. To support the discussions, individual question cards were used with example quotes from related initial survey responses to provide context. Tri-colour segmented tables were used (red, amber, and green) to represent areas of increasing importance, with red meaning less important and green meaning most important. These small groups were mixed part way through the day to ensure exposure to a range of ideas.

Initial Survey

The initial survey was completed by 456 respondents with 454 (99%) answering at least one open-ended feedback question. Only three people requested a paper copy of the initial scoping survey. Completion of the initial survey questions is shown in Table 1.

Table 1 Completion of initial survey

Demographic Questions	Number	Completed (%)
Consent to participate (yes)	456	100%
Age (range)	452	99%
Respondent’s role in trials	450	99%
Where respondent lives	452	99%
Specific open-ended feedback questions
Based on your experience, what questions or comments do you have (if any) about why people stay involved in a trial?	454	99%
Based on your experience, what questions or comments do you have (if any) about the planning of study data collection?	415	91%
Based on your experience, what questions or comments do you have (if any) about how trials collect follow up data from participants?	397	87%
Based on your experience, what questions or comments do you have (if any) about the information people are given about follow up data collection procedures for a trial?	376	82%
Based on your experience, what questions or comments do you have (if any) about trial staff who are involved in collecting follow up data from trial participants?	343	75%
Do you have any other questions or comments about how people are encouraged to stay involved in trials?	261	57%

Demographic information – initial survey

The most frequently reported role amongst initial respondents was a researcher involved in aspects of trials other than retention (22%). The proportion of respondents within each stakeholder group and the geographical spread can be seen in Table 2.

Table 2 Initial survey respondent roles

Which one of the following best describes your main role in a randomised trial?	Number	Percentage
A researcher involved in aspects of the trial other than retention	96	22
A principal investigator	95	21
A person invited to take part in a trial	70	15
A trial methodologist (someone who specialises in the methods of how trials are designed, run, analysed and reported)	70	15
A researcher involved in encouraging people to stay involved in trials	65	14
A patient, carer or public contributor to the design or running of trials	22	5
Other (please describe)	19	4
A parent or carer of a person invited to take part in a trial	10	2
No response	9	2
Total	456	100
Where do you usually live?
England	281	62
Scotland	61	14
Republic of Ireland	51	11
Wales	33	7
Other	15	3
Northern Ireland	11	2
No Response	4	1
Total	456	100

Initial stage, collating themes and merging questions

Thematic grouping was used to sort and separate data from the 2,431 answers from 456 responses into categories, resulting in 3,256 individual coded items. Within this there were 372 specific questions about retention that served as templates to create a series of sub-questions which would later be grouped within and constitute the main questions, depending on how often they appeared within the data. This allowed each coded item to be represented with a question and led to the creation of a longlist of 105 total questions through combining overlapping main and sub-questions. Through review and discussion with the Steering Group, we merged questions from this list where there was substantial overlap to a shortlist of 33 questions to be carried over to the interim stage.

Interim Stage

Completion of interim survey

With further consultation and the ‘back-translation’ process, we reduced the initial question shortlist to an indicative list of 27 questions to be used in the interim survey. The survey received 886 responses overall. Of these only one was on paper and 16 were not answered fully d. The results below therefore report data from 870 responses. Of these, 100% made a selection of up to 10 of the 27 questions, and 864 respondents gave information describing their gender, with 650 (75%) selecting female and 205 (24%) selecting male, one using their own term and eight (1%) preferring not to say.

Demographic information

The spread of stakeholder groups and their geographical spread can be seen in table 3.

Table 3 Interim survey respondent roles

Which one of the following best describes your main role in a randomised trial?	Number	Percentage
Frontline staff or other staff involved or invested in trial retention (e.g. Research Nurse, Trial Manager, regulatory or oversight role such as Sponsor or Research Director)	403	46
Investigator (e.g. Chief Investigator, Principal Investigator, Co-investigator)	225	26
Patient or public member involved in a trial (as a participant or parent/carer of a participant, or as a contributor to design/delivery of trial)	174	20
Trial Methodologist	68	8
Total	870	100
Where do you usually live?
England	590	68
Scotland	102	12
Wales	72	8
The Republic of Ireland	51	6
Other	30	3
Northern Ireland	19	2
Total	864	100

Interim survey ranking progress

Following the standard JLA approach, a ranked weighted score across all stakeholder groups was used to select the top 21 questions to be taken to the consensus meeting. We made this decision as experience from the JLA suggests between 20 and 25 questions are optimal for discussion. We selected 21 rather than 20 because there was a tie between the rankings, where two questions received the same score.

Consensus Meeting

The consensus meeting consisted of 30 stakeholders across the four groups, comprising 12 patients, nine clinicians, and nine total researchers and other staff. Some of the patient partners from the Steering Group attended the meeting as observers. Three JLA facilitators, including one who acted as chair, were present at the meeting in addition to 10 observers from the Steering Group. It culminated in a plenary session involving all the stakeholders. This finalised the ordering of the questions, created the top 10 list shown in table 4, and ranked the remaining questions (11-21). All of this is available online at www.priorityresearch.ie.

Table 4 Top 10 research questions prioritised

Overall Ranking	Research Question
1	What motivates a participant’s decision to complete a clinical trial?
2	How can trials make better use of routine clinical care and/or existing data collection to improve retention?
3	How can trials be designed to minimise burden on staff and participants and how does this affect retention?
4	What are the best ways to encourage trial participants to complete the tasks (e.g. attend follow-up visits, complete questionnaires) required by the trial?
5	How does involvement of patients/the public in planning and running trials improve retention?
6	How could technology be best used in trial follow-up processes?
7	What are the most effective ways of collecting information from participants during a trial to improve retention?
8	How does a participant’s ongoing experience of the trial affect retention?
9	What information should trial teams communicate to potential trial participants to improve trial retention?
10	How should people who run trials plan for retention during their funding application and creation of the trial (protocol development)?

The process of this PSP from data collection to the final priority question list is illustrated in Figure 1.

Figure 1 PRioRiTy II priority setting partnership process

Link to PRioRiTy I

During the course of this PSP, there were many similarities between our findings and those from PRioRiTY I. Of our final top 10 questions generated at the consensus meeting, five share a thematic area with questions within the PRioRiTy I PSP’s top 10 questions for recruitment. These have been highlighted in table 5 alongside their respective ranking in each PSP.

Table 5 Question overlap across PRioRiTy PSPs

	PRioRiTy I Ranking	PRioRiTy II Ranking	Research Question
A	6		What are the key motivators influencing members of the public’s decisions to take part in a randomised trial?
		1	What motivates a participant’s decision to complete a clinical trial?
B	1		How can randomised trials become part of routine care and best utilise current clinical care pathways?
		2	How can trials make better use of routine clinical care and/or existing data collection to improve retention?
C	3		Does patient/public involvement in planning a randomised trial improve recruitment?
		5	How does involvement of patients/the public in planning and running trials improve retention?
D	10		What are the advantages and disadvantages to using technology during the recruitment process?
		6	How could technology be best used in trial follow-up processes?
E	2		What information should trialists communicate to members of the public who are being invited to take part in a randomised trial in order to improve recruitment to the trial?
		9	What information should trial teams communicate to potential trial participants to improve trial retention?

Availability of the research question list

The entire list of prioritised questions from the PRioRiTY II project will be available online, alongside the results from the PRioRiTY I project into recruitment, on a dedicated website (www.priorityresearch.ie). This was created initially by the Health Research Board-Trials Methodology Research Network Ireland (HRB-TMRN) and expanded to include the results of this PSP. Questions can be viewed by their ranked importance or by thematic category. Teams undertaking research in the areas of these questions are asked to submit details of their work to the HRB-TMRN through this website.

Alongside the PRioRiTy I project, we believe the results of this PSP can contribute towards future efforts to reduce research waste and help ensure the voice of key stakeholders required to improve retention is reflected in the direction of future research. Participation was restricted to within the U.K. and Ireland to ensure that the results were directly applicable in those countries. Therefore, responses from other countries were not used for the final analysis, though international collaboration could help identify common retention issues across randomised trials in different contexts.

Challenges Encountered

An early challenge was the volume of data generated using the six open-ended questions in the initial survey. As we thematically coded and sorted responses during analysis, where a response could contain multiple thematic items and questions within a single answer, we were inevitably faced with the total number of data items being an order of magnitude larger than the total of initial responses. This required extensive work by the Aberdeen team to ensure both a true representation of the data and to conduct the necessary checks and discussions where there was disagreement, and to ensure the methodological reliability and integrity of the data. We sorted the data so that the connection between individual data items and the final question list was maintained and could be easily viewed and traced. This allows us to demonstrate how initial survey responses were developed into the final priority question list through each stage of this project.

Another challenge faced was the possibility that including examples within a question might influence how the question was interpreted. We sought a balance between ensuring that the questions were clearly explained and trying to avoid unduly influencing their meaning and interpretation. For this reason, and in specific instances, we decided, in discussion with the Steering Group, to give brief examples within questions that might otherwise be hard to understand. . However, it was clear from observations of the small group discussions during the consensus meeting that groups in some instances focused on the examples more than the category the examples represented. In future, if examples are included within a question, it should be made clear that they are not intended to cover the whole scope of the question, and should not encourage direct answers from the consensus meeting participants. Although, where appropriate, the JLA facilitators and some group participants did point out the difference, it is possible that some people chose to interpret and therefore rank the questions on that basis. In the final ordered question list, the questions ranked in the bottom six positions (16 – 21) all featured examples. This calls into question whether these examples limited the perceived potential scope of the research question and affected stakeholders’ understanding of their comparative importance.

A concern during this project was the potential for one stakeholder group to unduly influence another due to having more extensive knowledge and experience of trial methodology. This could be exacerbated by the difficulties in advertising the project to patients and carers who are unfamiliar with research terminology. At each stage, we acted to ensure researchers and trial methodologists, who only made up a majority of responders during the initial and interim phases, did not overshadow the input of patients and members of the public. This required regular checking of survey response totals to inform our audience targeting, as well as utilising a weighted system of point allocation during the interim survey ranking to provide a balanced perspective of respondent views. We also took steps to ensure the research remained easily accessible and understandable to all stakeholders during the course of the project. These steps included working closely with the Steering Group during the initial stage to combine similar questions, and conducting the ‘back-translation’ on the terminology of research questions that were generated at the interim stage. We also deliberately omitted titles and job descriptions from name badges at the final consensus meeting to promote equity between the stakeholder groups. Further, the guidance of experienced JLA facilitators promoted a respectful discussion with equal participation. For this entire process the input and contribution of our patient partners was invaluable, as they provided guidance and feedback throughout the PSP. The involvement of patient partners enriched the methods and results of this project, and should be considered essential for future PSPs.

During the final consensus meeting, one issue encountered was maintaining the topic of discussion focussed on retention rather than branching out onto issues of recruitment, which overlap considerably. The experience of the JLA facilitators was indispensable to direct conversation back to the topic and question at hand. This allowed members of the Steering Group and research team to act as impartial observers unless asked to clarify a point by the JLA facilitator, although clarifications were rarely required. Through these combined efforts, we believe we were able to accurately and fairly represent the views of stakeholders, taking due consideration for patient and public contributors, and create an equal opportunity for all to affect the project outcome. Our experiences of communicating and involving members of the public as stakeholders for this meeting are in line with existing research [11, 18-20], in that a flexible approach and quick-response was essential.

Implications

Through our approach, this PSP has successfully identified uncertainties and unanswered research questions on retention in randomised trials. This now provides a platform for future research projects into retention in trials to build upon, and create a credible methodological approach to the selection of these research topics. This can provide a higher level of certainty for both funders and research organisations that the questions identified in this project are critical to address in any effort to improve retention in trials and reduce research waste [9]. When viewed with PRioRiTy I, we are able to give direction to future research into two of the top three research priorities identified by the UK clinical trials community [1]. Future research to address the third priority, choosing appropriate outcomes to measure, would benefit from a similar approach as reported here such as the COMET (Core Outcome Measures in Effectiveness Trials) Initiative, which uses a similar method of Delphi survey and consensus meeting to develop agreed standardised sets of outcomes for clinical trials, known as ‘core outcome sets’ [21]. Furthermore, by viewing the results of both of these PSPs thematically, we can observe the overlap between uncertainties across retention and recruitment to randomised trials, as well as specific topic areas stakeholders view as important.

This PRioRiTy II PSP found that the key stakeholders involved in randomised trials such as staff, researchers, and patients/public believe future research on improvements to retention should focus primarily on individual motivation to complete trials, how trials can better use routine clinical care and existing data collection pathways, and how burden to participants can be minimised through trial design. Addressing these concerns is central to any sincere effort to investigate retention within trials as well as more efficiently provide a benefit to patients and others who use our health services. The complete 21 question list will be hosted online at www.priorityresearch.ie and will be further grouped into the thematic comparisons generated by the previous PRioRiTy I PSP on recruitment.

Researchers are encouraged to build proposals addressing the questions raised. We also encourage funders to incorporate these research priorities into their current strategies to address issues with randomised trial retention. We also highly valued having patients and carers as our research partners throughout this project and advocate their contribution as essential for future research into trial retention and trial methodology.

Table 6 Initial Survey question list

Specific open-ended feedback question

1	Based on your experience, what questions or comments do you have (if any) about why people stay involved in a trial? Motivators might be cash incentives, additional doctor appointments, career benefit, to improve healthcare for themselves and/or others, hope, commitment etc.
2	Based on your experience, what questions or comments do you have (if any) about the planning of study data collection? Planning trial follow up includes deciding what should be considered as important to measure, when they should be measured, how they should be measured, where participants should complete follow up (e.g. at clinic or at home), how they should complete follow up (e.g. questionnaire), how many people are needed to complete follow up, what should happen to data for those who don't complete follow up, why don't people complete follow up, what is the impact on others (e.g., work, dependents etc.).
3	Based on your experience, what questions or comments do you have (if any) about how trials collect follow up data from participants? This might include things like who contacts participants, what type of follow up is appropriate, what types of consequences should be considered (e.g. a clinic visit might require travel), what equipment is needed, how those taking part will know they have to complete follow up procedures etc.
4	Based on your experience, what questions or comments do you have (if any) about the information people are given about follow up data collection procedures for a trial? This might include information from the trial team that is provided to help people decide whether or not to take part in the trial, it might be the questionnaires that are sent to trial participants, or a letter with an appointment for a clinic visit. There may also be information sent as a way of helping participants to return the questionnaire or attend the clinic, a prompt or reminder such as a telephone call, a letter, an email or a text message.
5	Based on your experience, what questions or comments do you have (if any) about trial staff who are involved in collecting follow up data from trial participants? The trial staff might be a family doctor, a research nurse, a consultant, or other clinicians involved with the trial, or a person employed by a University or other organisation tasked with conducting the research.
6	Do you have any other questions or comments about how people are encouraged to stay involved in trials?

Ethics Approval and Consent to Participate

This project was confirmed by the Sponsor (University of Aberdeen) and local Research Ethics Service (North of Scotland Research Ethics Service) to be service evaluation (in this context the service being trials) and development. Therefore, this work is deemed to not require ethical approval.

Stakeholders were identified and invited to participate in the research through their virtue of being part of national and/or international organisations. No participants were identified and invited directly through the NHS, therefore this work was deemed not to require Research and Development approvals.

Consent for Publication

Not Applicable.

Availability of Data and Material

The datasets generated and/or analysed during the current study are not publicly available due to privacy protections but are available from the corresponding author on reasonable request.

The full results of this study are available at www.priorityresearch.ie

Competing interests

Prof Shaun Treweek is an Editor-in -Chief for Trials.

Funding

This research was funded through contributions from various sources. Primarily the work was supported through a Catalytic Grant Award from Chief Scientist Office of the Scottish Government Health and Social Care Directorate (CGA/17/49). Additional sources of funding included: support from a Medical Research Council Methodology Fellowship (K Gillies MR/L01193X/1); and funding from the Health Research Board -Trials Methodology Research Network (HRB-TMRN). The Health Services Research Unit, University of Aberdeen, receives core funding from the Chief Scientist Office of the Scottish Government Health and Social Care Directorate.

Authors’ contributions

KG, ST, DD, CG and VB conceived the study. All authors were involved in designing the study. KG led the study and DB oversaw delivery of study components. KG, DB, HG and ST were responsible for survey development, tracking survey responses and data analysis in both surveys. ST was responsible for website development. KG was responsible for evidence checking of the literature. DB and KG wrote the first draft of the manuscript. DB, CW and BM organised the face-to-face prioritisation meeting. DB facilitated reimbursement and remuneration to attendees. DB and CW were responsible for communication to all study stakeholders. All authors contributed to the conduct of the study, including survey promotion, interpretation of the results, manuscript review and input. All authors read and approved the final manuscript.

Acknowledgements

The authors would like to thank all stakeholders who took part, all respondents to the initial and interim surveys, and the attendees at the consensus meeting. We would also like to thank staff members at the HSRU who screened abstracts, and specifically Jane Brady and Beverly Smith for their insurmountable work. We would also like to thank Katherine Cowan from the JLA for her work as the lead facilitator at the consensus meeting.

PRioRiTY – Prioritising recruitment/retention in randomised trials PSP – Priority setting partnership JLA – James Lind Alliance HRB-TMRN - Health Research Board Trials Methodology Research Network HSRU – Health Services Research Unit

1. Tudur Smith C, Hickey H, Clarke M, Blazeby J, Williamson P. The trials methodological research agenda: results from a priority setting exercise. Trials. 2014; 15:32. https://doi.org/10.1186/1745-6215-15-32.

2. Walsh M, Sackett D, Deveraux PJ. When RCT participants are lost to follow up. Why even a few can matter. Clin Trials. 2015; 12:537–9.

3. Gillies, K., Bower, P., Elliott, J., MacLennan, G., Newlands, RSN., Ogden, M., Treweek, SP., Wells, M., Witham, MD., Young, B. & Francis, JJ. 2018. 'Systematic Techniques to Enhance rEtention in Randomised controlled trials: the STEER study protocol'. Trials, vol. 19, 197.

4. Brueton VC, Tierney J, Stenning S, Harding S, Meredith S, Nazareth I, Rait G. Strategies to improve retention in randomised trials. Cochrane Database Syst Rev. 2013; 12:MR000032. https://doi.org/10.1002/14651858.MR000032. pub2.

5. Brueton VC, Stevenson F, Vale C, Tierney J, Meredith S, Harding S, Stenning S, Nazareth I, Rait G. (2014) Use of strategies to improve retention in primary care randomised trials: a qualitative study with in-depth interviews. BMJ Open 2014;4:1 e003835.doi:10.1136/bmjopen-2013-003835. http://bmjopen.bmj.com/content/4/1/e003835.full

6. Brueton V. Stenning S. Stevenson S. Tierney j. Rait G. Best practice guidance for the use of strategies to improve retention in randomised trials developed from two consensus workshops. Journal of Clinical Epidemiology 88 (2017) 122-132.

7. Kearney A, Daykin A, Shaw AR, Lane AJ, Blazeby JM, Clarke M, Williamson P, Gamble C. Identifying research priorities for effective retention strategies in clinical trials. Trials. 2017 Dec;18(1):406.

8. NIHR CRN Annual Performance Report 2014/2015.

9. Chalmers I, Bracken M, Djulbegovic B, Garattini S, Grant J, Gülmezoglu M, Howells D, Ioannidis J, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet 2014; 383: 156–65.

10. Standard Protocol Items: Recommendation For Interventional Trials (SPIRIT) http://www.spirit-statement.org/. Accessed 13 June 2017.

11. Healy, P., Galvin, S., Williamson, P., Treweek, S., Whiting, C., Maeso, B., Bray, C., Brocklehurst, P., Moloney, M., Douiri, A., Gamble, C., Gardner, H., Mitchell, D., Stewart, D., Jordan, J., O’Donnell, M., Clarke, M., Pavitt, S., Guegan, E., Blatch-Jones, A., Smith, V., Reay, H. and Devane, D. Identifying trial recruitment uncertainties using a James Lind Alliance Priority Setting Partnership – the PRioRiTy (Prioritising Recruitment in Randomised Trials) study. Trials. 2018; 19(1).

12. Duley L, Uhm S, Oliver S. Top 15 UK research priorities for preterm birth. Lancet 2014; 383: 2041–2042.

13. Batchelor JM, Ridd MJ, Clarke T, Ahmed A, Cox M, Crowe S, Howard M, Lawton S, McPhee M, Rani A, Ravenscroft JC, Roberts A, Thomas KS. The Eczema Priority Setting Partnership: a collaboration between patients, carers, clinicians and researchers to identify and prioritize important research indicative questions for the treatment of eczema. British Journal of Dermatology 2013; 168: 577–582.

14. Lewis-Beck, M., Bryman, A. and Liao, T. “Constant Comparison” in The Sage encyclopedia of social science research methods. Thousand Oaks, California: Sage. 2004.

15. Skea, Z, Newlands, R, Gillies, K. What influences non-retention in clinical trials? A meta-ethnographic synthesis of studies reporting participant reasons for drop out. PLoS Med. 2017.

16. El Feky, A., Gillies, K., Gardner, H., Fraser, C. and Treweek, S. A protocol for a systematic review of non-randomised evaluations of strategies to increase participant retention to randomised controlled trials. Systematic Reviews. 2018; 7(1).

17. JLA: James Lind Alliance. The James Lind Alliance Guidebook. Version 6. 2016. Available at www.jla.nihr.ac.uk. Accessed 10 Apr 2017.

18. Buck D, Gamble C, Dudley L, et al. From plans to actions in patient and public involvement: qualitative study of documented plans and the accounts of researchers and patients sampled from a cohort of clinical trials. BMJ Open. 2014; 4:e006400.

19. Evans B, Bedson E, Bell P, et al. Involving service users in trials: developing a standard operating procedure. Trials. 2013; 14:1.

20. Pollard K, Donskoy AL, Moule P, Donald C, Lima M, Rice C. Developing and evaluating guidelines for patient and public involvement (PPI) in research. Int J Health Care Qual Assur. 2015; 28(2):141–55.

21. Williamson P, Altman D, Bagley H, Barnes K, Blazeby J, Brookes S, et al. The Comet Handbook: version 1.0. Trials. 2017; 18(Suppl 3): p. 280.

Download PDF

Journal Publication

published 15 Oct, 2019

Read the published version in Trials →

Editorial decision: Major revision
23 Jul, 2019
Review #2 received at journal
09 Jun, 2019
Reviewer #2 agreed at journal
22 May, 2019
Reviewers invited by journal
21 May, 2019
Reviewer #1 agreed at journal
21 May, 2019
Review #1 received at journal
21 May, 2019
Editor assigned by journal
16 May, 2019
Submission checks completed at journal
08 Mar, 2019
First submitted to journal
20 Feb, 2019

You are reading this latest preprint version

What are the most important unanswered research questions in trial retention? A James Lind Alliance Priority Setting Partnership – The PRioRiTy II (Prioritising Retention in Randomised Trials) Study

Status:

Journal Publication

Version 1

Abstract

Figures

Background

Methods

Results

Discussion

Conclusions

Appendix

Declarations

List of Abbreviations

References

Status:

Journal Publication

Version 1