Improving Prioritisation Processes for Clinical Practice Guidelines: New Methods and an Evaluation From the National Heart Foundation of Australia.

doi:10.21203/rs.3.rs-838801/v1

Background

Releasing timely and relevant clinical guidelines is challenging for organisations globally. Priority setting is crucial as guideline development is resource intensive. Our aim, as a national organisation responsible for developing cardiovascular clinical guidelines, was to develop a method of generating and prioritising topics for future clinical guideline development in areas where guidance was most needed.

Methods

Several novel processes were developed, adopted and evaluated including: 1) initial public consultation for health professionals and the general public to generate topics; 2) thematic and qualitative analysis, according to the International Classification of Diseases (ICD-11), to aggregate topics; 3) adapting a criteria-based matrix tool to prioritise topics; 4) achieving consensus through modified-nominal group technique and voting on priorities; and 5) process evaluation via survey of end-users. The latter was comprised of the organisation’s Expert Committee of 12 members with expertise across cardiology and public health, including two citizen representatives.

Results

Topics (n=405; reduced to n=278 when duplicates removed) were identified from public consultation responses (n=107 respondents). Thematic analysis synthesised 127 topics that were then categorised into 37 themes using ICD-11 codes. Exclusion criteria were applied (n=32 themes omitted), resulting in 5 short-listed topics: 1) congenital heart disease; 2) valvular heart disease; 3) hypercholesterolaemia, 4) hypertension and 5) ischaemic heart diseases and diseases of the coronary artery. The Expert Committee applied the prioritisation matrix to all five short-listed topics during a consensus meeting and voted to prioritise topics. Unanimous consensus was reached for the topic voted the highest priority: ischaemic heart disease and diseases of the coronary arteries, resulting in the decision to update the organisation’s 2016 clinical guidelines for acute coronary syndromes. Evaluation indicated that initial public consultation was highly valued by the Expert Committee, the matrix tool was easy to use and improved transparency in priority setting.

Conclusion

Developing a multistage, systematic process, incorporating public consultation, and an international classification system led to improved transparency in our clinical guideline priority setting processes and that topics chosen would have the greatest impact on health outcomes These methods are potentially applicable to other national and international organisations responsible for developing clinical guidelines.

Health Economics & Outcomes Research

priority setting

clinical practice guidelines

Clinical practice guidelines take considerable effort, time, and resources to develop. Numerous checklists and methods for selecting which topics to prioritise for clinical guideline development and updates are available, however how decisions are made, and by whom, are poorly understood.
We developed a new process for prioritising topics for clinical guideline development that involved both the public and health professionals to generate topics.
We used the World Health Organization’s International Statistical Classification of Diseases and Related Health Problems (ICD) codes to categorise topics. We developed a new tool to prioritise these topics. We explicitly documented how decisions were made, and by whom, to increase transparency.

By documenting our methods, our process and tools can be adopted and adapted by international guideline developers not only for cardiovascular diseases, but other conditions.

Ensuring clinical guidelines are contemporary with evolving evidence and practice standards is a global challenge [1]. Formulating guidelines may take between 18 – 30 months to complete [2] and therefore generating and prioritising topics is a critical first step in the development process. The importance of priority setting is recognised by many guideline developers and international organisations in the US [3], UK [4, 5], and Europe [6], but is frequently the least transparent step in publishing clinical guidelines.

Globally, many organisations involved in guideline development including the World Health Organization [7], and National Institute of Health and Care for Excellence [5] report using specific criteria or questions to prioritise which topics are selected for clinical guideline development and updates. While checklists and criteria exist for priority setting, the processes for applying these criteria and making decisions are often not explicitly documented [1, 6, 8, 9], which limits transparency and the ability to generalise their processes. Failure to document key information about how guidelines are developed, including how and why topics are chosen, may also impact implementation efforts [2].

There are a variety of approaches for priority setting available, and the James Lind Alliance, an initiative established in 2004 in the UK, provides step-by-step guidance on how to set up priority setting partnerships by, bringing together patients, carers, and clinicians to identify the top 10 unanswered questions or evidence uncertainties, particularly for setting research priorities [4]. However, some of the evidence available around priority setting methods for research are challenging to adapt and apply to clinical guideline development [4, 10, 11].

The National Health and Medical Research Council (NHMRC) are the organisation largely responsible for clinical practice guidelines in Australia. The NHMRC have published the 2016 NHMRC Standards for Guidelines [12] and a suite of tools for guidelines developers including self-contained peer-reviewed modules on the NHMRC website called the ‘Guidelines for Guidelines Handbook’ [13]. However, processes about priority setting including topic generation and selection is lacking from the guidance, which largely assumes topics have already been selected for guideline development.

The National Heart Foundation of Australia is an independent, non-for-profit organisation that funds cardiovascular research and works to improve heart disease prevention, detection, and support for all Australians [14]. Alongside funding research, the organisation, in partnership with others, develops clinical practice guidelines and position statements for health care professionals. Our primary aim was to determine areas within cardiovascular disease where clinical guidelines would have the biggest impact on health outcomes and where guidance was most needed. Our secondary aim was to improve transparency and certainty in our methods for generating and prioritising topics for clinical guideline development and updates.

Study Design

We undertook a quality improvement initiative to update our methods for priority setting for clinical guideline development. This involved conducting an online public consultation for health professionals and the public to generate topics for guidelines, an iterative process to refine topics through thematic analysis and categorisation using the current World Health Organization’s International Statistical Classification of Diseases and Related Health Problems (ICD) [15], developing a criteria-based matrix for prioritising topics and adopting established consensus techniques to decide on priorities. Each of these is steps is described below. The SQUIRE checklist was used for reporting [16] and is presented in Additional File 1.

Expert Committee

The organisation’s internal governance structures include an established Expert Committee who provide guidance on clinical issues including the development and implementation of clinical guidelines and position statements. The Expert Committee comprises 12 members with expertise across cardiology, public health, epidemiology and includes two people with lived experience of heart conditions. The Expert Committee approved, developed, and implemented this quality improvement initiative with support from the organisation’s clinical and evaluation teams. Three members of the Expert Committee (the Committee Chair, Deputy Chair, and the organisation’s Chief Medical Advisor) were closely involved in study design and processes, and hereafter are referred to as the Expert Subcommittee.

Public consultation to generate topics

Public consultation was the first step to generate topics for selection. A 5-item online public consultation survey (Table 1) was developed with approval from the Expert Subcommittee. The public consultation was held for a period of seven weeks, between 22^nd of November 2019 to 10^th of January 2020, aligning with national recommendations that a public consultation is open for a minimum 30-day period [17]. The online survey was developed using the software platform TypeForm® [18]. The survey included a privacy statement and data collection was anonymous. The distribution methods for the public consultation survey are presented in Table 2. People were provided with a description as to the purpose of the survey and a direct link to share with others, thus a snowball recruitment technique was also part of for the survey distribution approach. Members of the organisation’s committees comprised of people both internal (employees, volunteers, and committee members) and external to the organisation.

Data analysis and short-listing topics

An Excel spreadsheet containing responses was downloaded from the survey software platform at the close of the public consultation. Data was extracted from responses using descriptive coding to identify topics (e.g., valve disease, stroke, exercise). Topics were then categorised (e.g., cardiac arrythmia, hypertensive disease) using 118 codes from the International Classification of Diseases 11^th Revision (ICD-11) [15]. Any remaining topics that were unable to be classified using the ICD-11 were grouped into broader themes (e.g., obesity, cardiovascular disease prevention, supportive care). All themes were then grouped into four broad categories: i) diseases of the circulatory system; ii) other diseases related to cardiovascular disease; iii) risk factors and prevention strategies; and iv) general themes for guideline development such as patient support and secondary prevention. Two reviewers (BA and CC) were involved in all phases of data analysis. This iterative process of drafting and revising topics, themes, and categories resulted in consensus of the two reviewers. It was planned that disagreements between the two reviewers would be resolved by an Expert Subcommittee member (third reviewer), but this was not required. The final categories and results [Additional File 2], were reported to the Expert Subcommittee for validation. All agreed with the categorisation of topics without further refinement.

A 4-item exclusion criteria was developed (Table 3) and applied to short-listed topics. A 1-hour teleconference was held with the Expert Subcommittee on the 4^th of March 2020 to apply the exclusion criteria to create a finalised list of short-listed topics through consensus.

Development of criteria-based matrix tool to prioritise topics

We drew upon criteria used by other guideline developers [5-7, 19] and criteria identified in published literature [1, 5, 8, 9, 11, 20] to form common categories to prioritise topics (e.g., prevalence of a disease, health care expenditure). We followed an iterative process of consolidating and revising criteria for the proposed matrix and compared this to the organisation’s existing matrix tool. This process involved drafting of the matrix tool by one author (BA) and another author (CC) revising the draft matrix tool to further consolidate criteria and improve clarity including merging and modifying the criteria. The matrix tool was piloted with a subset of short-listed topics. Further refinement of the tool was achieved by the Expert Subcommittee until consensus was reached and the tool was finalised. The newly developed matrix tool [see Additional File 3] had five domains and considerations listed within each. This was presented in a descriptive version of the tool [see Additional File 4]. Criteria within the matrix was not ranked or weighted as evidence supports there is large variance in individuals’ ratings of the importance of criteria and no difference in topic selection when weighing the criteria [21].

Consensus meeting to decide on priorities

A face-to-face consensus meeting with the Expert Committee was planned to apply the matrix tool to prioritise topics. However, a videoconference was held due to COVID-19 travel restrictions (total attendees n=16: Expert Committee members n=11; plus, n=5 staff members from the organisation).

Prior to meeting, all attendees were sent the following documents: the organisation’s strategic plan, a description of the literature used to develop the matrix tool, the results of the public consultation responses in a report format, the final summary report of the categorised topics, the exclusion and inclusion list developed as part of the process, the matrix tool and a description of how to use the tool, and an economic report for short-listed topics prepared by a health economist within the organisation. The economic report included data and statistics on incidence, prevalence, mortality, hospitalisation, burden of disease and health care expenditure for each topic to support matrix criteria. The two people with lived experience of heart disease received additional information via telephone prior to the meeting including a description of the planned process to prioritise topics using the matrix tool and voting procedures. They were both provided with a glossary of clinical terms in plain language as a supplementary document prior to the meeting.

During the meeting the prioritisation process was described and where available, additional data was presented on emerging evidence, gaps in knowledge on these topics, variance in care and equity considerations. Discussions and all decisions made by the Expert Committee were documented on matrix tools. The Chair of the meeting invited members with expertise on a topic to lead discussions for that topic, e.g., a member with specialist expertise in Lipidology led the discussion on hypercholesterolaemia. A structured process of checking that each criterion had been addressed by the committee before moving to the next topic was adopted during the meeting.

A consensus-based approach using the modified-nominal group technique [22] was used to generate and refine discussion within the Expert Committee and prioritise short-listed topics. An online poll was created for attendees to anonymously vote on topics using the software Poll Everywhere® [23]. During the videoconference the Expert Committee agreed to verbally vote, and each Expert Committee member (n=11) listed topics in order of their priority.

Evaluation Survey

A survey was used to evaluate the effectiveness and acceptance of the new prioritisation process. The organisation’s evaluation unit – a group independent to those involved in the design and implementation of the new prioritisation process – designed, distributed, and analysed the survey results. The 23-item online anonymised evaluation survey was developed using the software Typeform® [18] [see Additional File 5]. The survey was sent to all attendees of the videoconference (n = 16) on the 2^nd of April 2020, followed by two reminders. The survey results were analysed and presented as a report and distributed to all Expert Committee members to review.

Public consultation

The initial online public consultation resulted in 405 potential topic suggestions from 107 people. Most people responding to the survey were health care professionals or researchers (66%; 71/107) or members of the public (34%; 36/107) (Table 4). Responses were received from across the country, including all six States and two Territories. Health care professionals and researchers were given the option to provide their workplace and 40 different organisations including hospitals, state health services, universities and research institutions were represented.

Data analysis and short-listing of topics

There were 405 topics identified from public consultation that were categorised to 127 themes, then mapped against 118 ICD-11 codes from 11 chapters [15], resulting in 37 themed topics across four broad areas: diseases of the circulatory system, other diseases related to cardiovascular disease, risk factors and general health themes. Ischaemic heart diseases and diseases of the coronary artery have separate ICD-11 codes, however it was decided by the Executive Subcommittee to group these as one topic, as guidance in this area would cover both topics. Application of the exclusion criteria resulted in five short-listed themes: Ii) congenital heart disease; ii) valvular heart disease; iii) hypercholesterolaemia; iv) hypertension and v) ischaemic heart diseases and disease of the coronary artery.

Application of the criteria-based matrix tool to prioritise topics

For each short-listed topic, a separate matrix tool was used to document discussions and decisions and served as minutes of the meeting. The completed matrix tool for ischaemic heart diseases and diseases of the coronary artery is provided as an example in Table 5. Evaluation indicated presenting topics with supporting data for criteria with the matrix helped structure information to facilitate decision-making. Following the meeting, completed matrix tools were reviewed by all members of the Expert Committee. Minor changes were made following review and all matrix tools were accepted as accurate and final by the Expert Committee at a subsequent meeting on the 16th April 2020.

Consensus meeting to decide on priorities

Expert Committee members (n=11) voted on topics verbally during the videoconference resulting in short-listed topics being prioritised with one being the highest priority and five being the lowest priority (Table 6). All members were in agreement with the topic that was voted the highest priority: ischaemic heart diseases and diseases of the coronary artery. All respondents agreed that the purpose of the meeting was clear, and that the meeting duration (5.5 hours) was acceptable. Most respondents felt they were able to share their opinions. Feedback from the evaluation to improve future processes included providing explicit information on dates and timelines when a clinical guideline was last reviewed.

Evaluation of process as a whole

The evaluation survey showed that respondents highly valued the public consultation as a useful first step in the process. There were varied responses regarding the quality of topics generated from the public consultation, however most respondents agreed that public consultation should continue as the first step in the prioritisation process. Common themes from individual respondents were that public consultation was best practice and vital to ensure the organisation remains relevant to all stakeholders. One Expert Committee member reported public consultation “is vital to ensure the organisation remains relevant to all stakeholders in the community”, and another that public consultation is an “important reality check for clinicians with specialist knowledge”. Overall, respondents agreed that the new processes were effective, systematic, and easy to follow. Five respondents reported they would have preferred a face-to-face meeting rather than a videoconference, however acknowledged it was not feasible given the restrictions on non-essential travel in Australia at the time.

The first step in our new multistage process was to generate topics for guideline development through public consultation involving the public and health professionals. Only topics that were generated from public consultation were considered to ensure topics were in areas where guidance was most needed from the wider community. The ICD codes were used to categorise topics. The Expert Committee used the new matrix tool to decide upon priorities. We explicitly documented how decisions were made, and by whom. Evaluation demonstrated this increased the transparency and certainty of our methods.

Public consultation to generate topics is increasingly recognised as an important component of clinical guideline development [9, 17, 24, 25]. Additionally, many clinical guideline developers recommend health consumer involvement as part of the broader guideline development process [5, 7, 24]. However, many do not explicitly document methods for health consumer and patient involvement [1, 6, 8, 9, 26]. Evaluation of our process demonstrated that public consultation was perceived as an important ‘reality check’ for clinicians by Expert Committee members, and most agreed public consultation should continue to be used to generate topics for clinical guideline development. Comparatively, the American Heart Association describe their processes involve a task force comprising of senior, well-respected individuals with a variety of expertise, to choose individual topics for guideline development and do not mention patient or health consumer involvement [3]. A study involving two parallel guideline development groups, one with and one without patient representatives, found patient involvement led to the inclusion of patient-relevant topics [27]. Patient representatives helped identify issues that may be overlooked by medical professionals and helped select patient-relevant outcomes [27].

A barrier to adapting existing criteria and checklists proposed by others is that they were often not specific to guideline development [1, 4, 10, 11]. Additionally, many processes, while comprehensive, propose numerous items to consider as part of their criteria and often there is duplication of criteria. The McMaster Group in Canada, for their project Guidelines 2.0, developed a checklist that includes 18 topics and 146 items [1]. The checklist is broad and therefore potentially has wider applicability, including public health and policy guidelines, however, may not be suitable or feasible in all settings given its length [1]. We developed a matrix tool through refining, merging, and synthesising criteria proposed by other organisations and by guideline developers to produce a practical tool that can help prioritisation. In Australia, the Australian Commission for Safety and Quality in Health Care (the Commission) leads and coordinates national improvements in the safety and quality of healthcare [28]. The Commission and the NHMRC have proposed a national framework to promote the efficient development of clinical practice guidelines [8]. All criteria in this framework [8] were incorporated into the priority setting matrix tool. Evaluation found most of the Expert Committee felt the new matrix tool contained all the important criteria needed for decision-making. As our organisation develops clinical guidelines in cardiology, we included additional criteria to assess a topics alignment with the Heart Foundation’s strategy [29]. Many organisations that focus on specific health conditions, such as cardiovascular, stroke or other neurological conditions need to align guideline priorities with their organisation’s missions and values and the needs of the communities they serve.

In 2014, the NHMRC analysed 515 clinical practice guidelines on their database and reported there were serious and systemic problems with the way guidelines are developed in Australia and called for new methods to promote more transparent documentation of guideline development processes [2]. The report from the NHRMC found only 15 of 102 Australian cardiovascular guidelines between the period 2005 to 2013 were found to be ‘evidence documented’, which requires guidelines to be published with evidence of a full systematic review [2]. Consensus-based methods have been established as an effective and accepted way to decide upon priorities [22]. Verbal ranking of topics was performed to decide upon priorities by each Expert Committee member. The matrix criteria was used as guidance for discussion to ensure all important aspects were considered. This process enabled each member to have an equal vote, which was important given the Expert Committee comprised of members with knowledge and expertise in different areas and included two members with lived experience of heart disease. The evaluation survey found Expert Committee members found this process transparent, systematic, and effective.

There are several contextual factors unique to the development and application of this process for prioritising topics for clinical guideline development. The skill set of the Expert Committee, which although common among clinical guideline developers [3, 5-7] were unique to the organisation and included experts in cardiology, public health, and epidemiology and those with lived experience of cardiac conditions, likely influenced results. The Chair of the consensus meeting, who was also a member of the Expert Subcommittee, was experienced in the role of chairing meetings and group consensus methods. This experience is likely to have positively influenced application of the matrix tool and overall results of the improvement initiative. Additionally, the organisation’s health economist assisted in preparing economic data and guidance on how this data could be used in combination with the prioritisation process and the matrix. Furthermore, we promoted the public consultation through our large networks of health professionals and community. We appreciate that not all organisations may have access to this expertise and breadth of resources.

Strengths and limitations

There are several strengths of our study. Firstly, it responds to the NHMRC and the Australian Commission for Safety and Quality in Health Care’scall for greater transparent criteria and processes for priority setting for clinical guideline development [2, 8]. We undertook thematic analysis of data extracted from public consultation results and used ICD-11 codes to map topics to prioritise clinical guideline topics for development. To our knowledge, this is the first time ICD codes, especially version 11, have been used to map and categorise domains for guideline development. Using standardised internationally recognised classification potentially increases the applicability of the prioritisation process for other guideline developers. Limitations to our study include a lower-than-expected response rate to the public consultation. The public consultation process occurred over the main national holiday period between December 2020 to January 2021. The seasonal timing of the survey was likely to have influenced the response rate, despite the consultation period being extended. However, over 400 topics were identified from the 107 responses, highlighting that those who did respond were engaged with the topic of cardiovascular health and that survey questions were effective in generating topics. Another limitation of this study is that not all members of the Expert Committee completed the evaluation survey, and positive response bias may have occurred.

Implications and next steps

The processes outlined in this quality improvement initiative, including application of the matrix tool, could easily be adopted, and adapted for other clinical guideline developers. Furthermore, the process by which we supplied economic data alongside the matrix criteria was valued by decision-makers and could be reproduced for other diseases and priority setting activities. Public consultation was the first step in our new process and was highly valued by the Expert Committee. Both end-users and the public should be involved in priority setting for clinical guideline development, including public consultation to generate topics. This was an important first step in our process, as it highlighted areas where guidance was most needed. Prioritising clinical guidelines for other health topics unrelated to cardiovascular disease may also be suitable using ICD codes, for example for neurological or endocrine disorders. By documenting how decisions were made, and by whom, we have increased the transparency of our approach and enabled other organisations to apply our methods. We recommend other guideline developers document decision-making and make this publicly available to improve transparency. Further methodological research is needed to refine this process and assess the effectiveness and acceptability for other guideline developers, including in areas other than cardiovascular disease.

We have described our approach for generating and prioritising topics for clinical guideline development, including initial public consultation, the development of a new matrix tool, adopting established consensus-based methods for voting on final priorities and evaluation of the process. These methods resulted in the prioritisation of topics and selection of a topic for guideline development in an area that demonstrated the most need with the potential to improve health outcomes. By explicitly documenting our methods, including decision-making, we have improved the transparency of our guideline development process. These methods can be adopted and adapted by other clinical guideline developers.

ICD-11: International Classification of Diseases 11^th Revision

Ethics approval and consent to participate

Not applicable, this manuscript does not include any human subject research.

Consent for publication

Not applicable, this manuscript does not contain data from any individual person.

Availability of data and material

Not applicable to this manuscript as no datasets were generated or analysed during the current study.

Competing interests

The authors declare that they have no competing interests.

Funding

No funding was received for this project. This project was supported by the National Heart Foundation of Australia. The views expressed are those of the author(s) and not necessarily those of the National Heart Foundation of Australia.

Authors' contributions

BA, TB, CC and GLRJ contributed to the study concept and design. BA and CC analysed the results from the public consultation. BA and AKB led the writing of the manuscript. All authors were involved in interpretation of the data from the public consultation, analysis and evaluation and provided intellectual and editorial input, and revision of the manuscript. All authors read and approved the final manuscript.

Acknowledgements

The authors would like to thank members of the public and health professionals for responding to the public survey and all members of the Expert Committee members who participated in the priority setting process and evaluation survey. We thank Anna Kean from the National Heart Foundation of Australia who helped develop the evaluation survey.

Availability of data and materials

No applicable

1. Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. Canadian Medical Association Journal. 2014;186:3:E123-E42.

2. National Health and Medical Research Council. 2014 Annual Report on Australian clinical practice guidelines. National Health and Medical Research Council. 2014. https://www.nhmrc.gov.au/about-us/publications/report-australian-clinical-practice-guidelines-2014. Accessed 31 Oct 2019.

3. American College of Cardiology and American Heart Association. Methodology Manual and Policies From the ACCF/AHA Task Force on Practice Guidelines. American College of Cardiology and American Heart Association. 2010. https://www.acc.org/-/media/Non-Clinical/Files-PDFs-Excel-MS-Word-etc/Guidelines/About-Guidelines-and-Clinical-Documents/Methodology/2014/Methodology-Practice-Guidelines.pdf?la=en&hash=CBE36C37EF806E7C7B193DD4450C5D190DEBB5E3. Accessed 31 Oct 2019.

4. James Lind Alliance. The James Lind Alliance Guidebook. Version 10. James Lind Alliance. https://www.jla.nihr.ac.uk/jla-guidebook/downloads/JLA-Guidebook-Version-10-March-2021.pdf. 2021. Accessed 11 Nov 2020.

5. National Institute for Health and Care Excellence. Developing NICE guidelines: the manual. National Institute for Health and Care Excellence. 2014. https://www.nice.org.uk/media/default/about/what-we-do/our-programmes/developing-nice-guidelines-the-manual.pdf. Accessed 31 Oct 2019.

6. Oxman AD, Schunemann HJ, Fretheim A. Improving the use of research evidence in guideline development: 2. Priority setting. Health Res Policy Syst. 2006;4:14.

7. World Health Organization. WHO handbook for guideline development. 2nd ed. World Health Organization. 2014. https://apps.who.int/iris/handle/10665/145714. Accessed 31 Oct 2019.

8. Australian Health Ministers' Advisory Council. Establishing national priorities for clinical practice guidelines. Australian Health Ministers' Advisory Council. 2015. https://www.safetyandquality.gov.au/sites/default/files/migrated/Establishing-national-priorities-for-clinical-practice-guidelines-2015-Discussion-paper.doc. Accessed 14 Oct 2019.

9. Eccles MP, Grimshaw JM, Shekelle P, Schünemann HJ, Woolf S. Developing clinical practice guidelines: target audiences, identifying topics for guidelines, guideline group composition and functioning and conflicts of interest. Implementation Science. 2012;7:1:60.

10. National Health and Medical Research Council. NHMRC: Framework for Identification and Prioritisation of Targeted Calls for Research. National Health and Medical Research Council. https://www.nhmrc.gov.au/file/15068/download?token=iMbglAlv. Accessed 31 Oct 2019.

11. Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Health Research Policy and Systems. 2010;8:1.

12. National Health and Medical Research Council. 2016 NHMRC Standards for Guidelines. National Health and Medical Research Council. https://www.nhmrc.gov.au/guidelinesforguidelines/standards. 2016. Accessed 11 Oct 2019.

13. National Health and Medical Research Council. Guidelines for Guidelines Handbook. https://www.nhmrc.gov.au/guidelinesforguidelines (2016). Accessed 14 Oct 2019.

14. National Heart Foundation of Australia. What we do. https://www.heartfoundation.org.au/about-us/what-we-do (2021). Accessed 13 May 2021.

15. World Health Organization. International classification of diseases for mortality and morbidity statistics (11th Revision). https://icd.who.int/browse11/l-m/en (2021). Accessed 23 Jan 2020.

16. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process: Table 1. BMJ Quality & Safety. 2016;25:12:986-92.

17. National Health and Medical Research Council. Guidelines for guidelines: Public consultation (2018). https://www.nhmrc.gov.au/guidelinesforguidelines/review/public-consultation. Accessed 19 Nov 2019.

18. Typeform. https://www.typeform.com. Accessed 3 Oct 2019.

19. Atkins D, Perez-Padilla R, Macnee W, Buist AS, Cruz AA. Priority setting in guideline development: article 2 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report. Proc Am Thorac Soc. 2012;9:5:225-8.

20. El-Harakeh A, Morsi RZ, Fadlallah R, Bou-Karroum L, Lotfi T, Akl EA. Prioritization approaches in the development of health practice guidelines: a systematic review. BMC Health Services Research. 2019;19:1.

21. Reveiz L, Tellez DR, Castillo JS, Mosquera PA, Torres M, Cuervo LG, et al. Prioritization strategies in clinical practice guidelines development: a pilot study. Health Research Policy and Systems. 2010;8:1:7.

22. Rankin N, McGregor D, Butow P, White K, Phillips J, Pearson S, et al. Adapting the nominal group technique for priority setting of evidence-practice gaps in implementation science. BMC Medical Research Methodology. 2016;16:110.

23. Poll Everywhere. https://www.polleverywhere.com. Accessed 3 Mar 2020.

24. Guidelines International Network. G-I-N public toolkit: Patient and public involvement in guidelines. Guidelines International Network. 2012. https://g-i-n.net/wp-content/uploads/2021/04/GIN-TOOLKIT-COMBINED-FINAL-2015.pdf Accessed 11 Oct 2019.

25. Schünemann HJ, Fretheim A, Oxman AD. Improving the use of research evidence in guideline development: 1. Guidelines for guidelines. Health Research Policy and Systems. 2006;4:1.

26. Armstrong MJ, Mullins CD, Gronseth GS, Gagliardi AR. Impact of patient involvement on clinical practice guideline development: a parallel group study. Implementation Science. 2018;13:1.

27. Australian Commission for Safety and Quality in Healthcare. About us. https://www.safetyandquality.gov.au/about-us#the-commission's-purpose. Accessed 11 Mar 2021.

28. National Heart Foundation of Australia. One Heart. Our 2018-2020 Strategy. National Heart Foundation of Australia. 2018. https://www.heartfoundation.org.au/getmedia/a28a3cf0-143f-4f95-bef2-bb2df103a6d1/One_Heart_2018_-_2020_Strategy.pdf. Accessed 30 Sep 2019.

Table 1. Survey questions included in public consultation

I am providing information as a:

A) Member of the public
B) Health care professional (13 categories of responses available)
What cardiovascular disease theme(s) do you think are a priority for the Heart Foundation?
Why is this clinical theme of significance to the Australian community?
(e.g., is there a significant burden of disease, prevalence, or economic impact? Is there opportunity to reduce inequity?)
Why is information and advice needed on this theme currently?
(e.g., is there new, emerging or rapidly changing evidence or new care options? Is there complexity, controversy or uncertainty about themes and treatment?)
What area(s) of care require information and advice about this theme?
(e.g., prevention, screening, diagnosis, treatment, management)

Table 2. Distribution of public consultation survey

Method	Date promoted	Number of recipients	Type of stakeholder
All staff announcement via internal newsletter	27/11/2019	272 staff members	Internal
Direct electronic mail to all staff	22/11/2019	272 staff members	Internal
Organisation’s advisory committees	22/11/2019 07/01/2020	40 members	Internal and external
Organisation’s Twitter account	25/11/2019 07/01/2020	Approximately 24,800 subscribed	External
Newsletter to health care professionals	22/11/2019	Approximately 20,000 heath care professionals subscribed	External
Direct electronic mail to health and consumer organisations	22/11/2019	69 unique IP addresses	External
Organisation’s webpage	22^nd of November 2019 – 10^th of January 2020	There were 43,825 webpage views during period survey was promoted on organisation’s website	External

Table 3. Exclusion criteria applied to categorised topics

Topic is non-clinical/public health/prevention
Topic is a general theme – not a topic for clinical guideline development
Topic is not limited to heart disease

3a. Topic is cardiovascular disease, but not specific to heart
3b. Topic is related to cardiovascular disease, but not specific to cardiovascular disease

Guidelines published, or published guidelines funded by the National Heart Foundation of Australia within the last two years

Table 4. Public consultation respondents by occupational categories and frequencies of responses

Type of respondent (mutually exclusive categories)	Frequencies of responses (n=107)
Member of the public	36
Pharmacist	15
Nurse	14
Allied health	11
Health care professional – other	11
Health care researcher	10
Cardiologist	5
General Practitioner	2
Cardiac surgeon	1
Junior Doctor	1
Nurse practitioner	1

Table 5. Completed matrix tool for ischaemic heart diseases and diseases of the coronary artery

1) Impact of disease

- Burden of disease, in terms of mortality, incidence or prevalence of disease
- Economic impact and costs

Does guideline development in this area have the potential to impact many people affected by this disease?

2) Potential to impact health outcomes

- Significantly improve health outcomes/promote health/reduce inequalities
- Significant or unexplained variation in clinical practice

- Feasibility of implementing a guideline

- Reduce avoidable mortality or morbidity

Would a guideline on this topic feasibly address or impact clinical practice and health outcomes and reduce variance in care?

3) Organisation’s strategy

- Relevance to our 2018-2020 Strategy

Would a guideline on this topic be on strategy for our organisation?

4) Need from our community

Topic representation during clinical themes public consultation by health care professionals and consumers

Misconception about topic within the general community

Is there demonstrated feedback from our community that a guideline on this topic is needed?

5) Relevance to broad range of health care professionals

- Relevance of topic to health care workers with a diverse level of expertise where most of the care is delivered by non-experts

Is care in this clinical area significantly delivered by non-experts?

6) Evidence base

- New, emerging or rapidly changing evidence or new care options
- Complexity, controversy or uncertainty about topics and treatment
- Level and quality of current evidence on topic

- Published guidelines or guidelines funded by our organisation within the past 2 years already exist

Would a guideline on this have a strong evidence base or address any controversy in the interpretation of the current evidence base?

Some deaths are the first presentation – sudden cardiac death due to coronary heart disease. This disease is daily work for general practice. Enormous benefit in doing guidelines in this area with a huge scope that may span a number of guidelines. Huge impact if applied well. Scope primary and secondary care. Suggest update guidelines for primary prevention – absolute risk and hypertension in one document.

If prevalence data includes risk factors, this will include a lot more people.

Noted ischaemic heart disease and coronary artery disease are used interchangeably, however latter does not cover stroke or peripheral vascular disease and is more specific to heart disease.

Decide if primary, subclinical, or manifest disease

Mental health is an area of inequality and there is high prevalence in this group. Psychiatric disease and coronary artery disease need specialised care

Early onset cognitive decline noted as another special population

Noted this is identified in our current strategy

Guidance for targeted use of aspirin needed for individuals who will truly benefit

Medication adherence, deprescribing, and shared decision making identified as areas requiring guidance/support
Misconception around statins and medication in general in patients
GLs on when medication can safely be withdrawn needed
Discussed intellectual commentary for GPs about deprescribing, duration of therapies and the evidence base around this or guidelines to include this and discuss the evidence base/impacts to align a common approach

Some components of specialist input, largely general practice

Current guidelines noted – hypertension, acute coronary syndromes, and absolute risk.
Noted Absolute Risk guidelines are being updated –will be discussing in the next 4-6 weeks, this will impact scope depending on what is covered by these guidelines
Update guidelines needed for primary prevention (like New Zealand did)

Discussion on living guidelines and Magic app – how will we update guidelines we already have?

Table 6. Ranked voting result of short-listed topics during consensus meeting

Short-listed topics	Priorities (scores from each Expert Committee member n=11)											Sum of raw scores*	Ranked priority
	1	2	3	4	5	6	7	8	9	10	11
Ischaemic heart diseases and diseases of the coronary artery	1	1	1	1	1	1	1	1	1	1	1	55	#1
Hypercholesterolaemia	3	2	3	2	3	3	2	5	3	4	2	35	#2
Hypertension	2	3	2	5	2	2	3	4	5	2	5	31	#3
Valvular heart disease	4	4	4	3	4	4	4	2	2	3	4	27	#4
Congenital heart disease	5	5	5	4	5	5	5	3	4	5	3	17	#5

*Sum of raw scores: Points allocated to each short-listed topic using raw scores (top priority = five points, fifth priority = one point).

Improving Prioritisation Processes for Clinical Practice Guidelines: New Methods and an Evaluation From the National Heart Foundation of Australia.

Status:

Journal Publication

Version 1

Abstract

Contributions To The Literature

Background

Methods

Results

Discussion

Conclusions

Abbreviations

Declarations

References

Tables

Supplementary Files

Status:

Journal Publication

Version 1