Literature search
The combined search strategies yielded a total of 320 records (266 after removing duplicates). 179 potentially relevant full-text papers were screened and 30 were included in the review(1-5, 8, 15-38). The flow chart of study selection is presented in Figure 1.
Study characteristics
A majority of included papers (26/30) were journal articles, followed by conference proceedings (3/30) and a book chapter (1/30). The studies represented a wide range of subject areas; 20 different subject areas were identified, the most common being ecology (6/30), followed by education (4/30), then manufacturing and regional (3/30), and healthcare, architecture and health economics (2/30). The papers originated from 14 countries; the most common was UK (8/30), followed by Greece, Germany, US and the Netherlands (3/30) and finally Italy (2/30). Basic study characteristics are presented in Additional File 3.
We found a variety of terms used to describe the methodological frameworks. This use of different terms was seen in both the title and the body of the study. Six studies did not include ‘methodological framework’ in the title (20.0%). Of these one included the words ‘methodological’ and ‘framework’ separately (2), four included only ‘framework’ in title and one used the term ‘conceptual framework’. Of these six studies two were identified from references (4, 5), two from citations (37, 38) and one from Google images (34).
Alternative terms for methodological frameworks were used interchangeably within the studies (Figure 2).
Most studies included a combination of ‘methodological framework’ and ‘framework’ to describe the methodological framework (63.3%). One used a combination of methodological framework and conceptual framework. Three used ‘framework’ only and one used ‘methodological framework’ only. One study used three terms and a further two studies used a combination of four terms.
Keywords used in the studies that related to methodological frameworks are summarised in Table 1. Half of the studies (15/30) did not have any keywords related to methodological frameworks. Of those that used keywords related to methodological frameworks most used ‘methodology’ (4/30), followed by ‘methodological framework’ (3/30), ‘design methodology’ (2/30), ‘simulation methodology’ (1/30), ‘methods’ (1/30) and ‘guidance’ (1/30). One study contained two relevant keywords(5). 4/30 studies had no keywords at all.
Table 1: Keywords relevant to methodological frameworks extracted from studies
Keyword
|
Number (n=30)
|
Percentage %
|
None relevant to methodological frameworks
|
15
|
50.0%
|
Methodology
|
4
|
13.3%
|
N/A: no keywords in study
|
4
|
13.3%
|
Methodological framework
|
3
|
10.0%
|
Design methodology
|
2
|
6.6%
|
Simulation methodology
|
1
|
3.3%
|
Methods
|
1
|
3.3%
|
Guidance
|
1
|
3.3%
|
Approaches used for the development of methodological frameworks
We identified eight different approaches used for developing methodological frameworks (Table 2), these are also summarised by study in Additional File 4
Table 2: Approaches used for the development of methodological frameworks
Reported approaches
|
Number
|
Percentage (%)
|
Based on existing methods and guidelines
|
20
|
66.7
|
Refined and validated
|
10
|
33.3
|
Experience and expertise
|
9
|
30.0
|
Literature review
|
8
|
26.7
|
Data synthesis and amalgamation
|
7
|
23.3
|
Data extraction
|
3
|
10.0
|
Iteratively developed
|
2
|
6.7
|
Lab work results
|
1
|
3.3
|
The most frequently reported approach was ‘Based on existing methods and guidelines’, which comprise previous methodological frameworks or guidance and published methodology. Whilst some studies did not explain how the existing methods formed the foundations of the framework being developed, most did expand this further: adapting the methods (19, 24), integrating methods, building on the existing methods (4, 37), based on the framework (20-22, 27, 30, 33), combined well established guidelines which comprised the same stages (16), and the framework was basic inspiration (28). Only one study specified how the frameworks or guidance was identified; Squires and colleagues used a literature review (5).
Ten studies reported ‘Refined and validated’ as a method. Approaches taken to refining and validating comprised; piloting the framework (35), trialling identified stages and using the results of the trial to further develop the framework (25), using a case study or Delphi panel to evaluate and refine the framework (5, 8, 33), using a case study to validate the framework (17, 29) and testing the framework (20). Two studies did not report details of the case study (18, 24).
Nine studies reported using ‘Experience and expertise’ to develop the methodological framework, and reported using experience from different levels: personal (15), school/university (25) and country level (28). One study restricted ‘experience’ to the authors’ experience(15), the rest included the experience of experts in the field of the methodological framework. In all but one study the experts were recruited specifically to develop the methodological framework, the remaining study used experience already reported(28). Methods used to extract experience and expertise comprise: during meetings (18), consultations (39) and collaboration (33). Two frameworks did not specifically mention experience but used surveys and interviews (34) and focus groups for extracting expertise (5). Whilst these studies did not explicitly mention experience the methods reported would have extracted experience or views on experience.
Eight studies reported conducting a ‘Literature Review’. Specifically; purposeful sampling (2, 26), sources for searches included databases, dissertation (23), library catalogue, key author, databases websites and citations (8). Other studies reported conducting a literature review but did not report specific methods used (5, 8, 23, 29, 33, 35).
Seven studies reported using ‘Data synthesis and amalgamation’. Specific methods included: identifying phases (2), themes (2, 34) and dimensions (23), analysing and grouping or categorising themes, or thematic analysis (2, 3, 8, 23, 26).
‘Data extraction’ was reported in three studies and includes extracting data from interviews and focus groups using transcribing methods (5, 34), and extracting key information from published literature (2).
‘Iteratively developed’ was a method reported in two studies, one framework had no details on this (20), the other explained that the framework evolved and developed as items were extracted, synthesised and revised (8).
The least frequently mentioned method was ‘Lab work results’, the study that reported using this method was from the field of explosives, where the results of lab tests were used to inform the framework (1).
A pattern emerged whilst reviewing the methods and in applying meaning to these results, they were split into three categories. The first category relates to identifying evidence or data to inform and shape the framework. This evidence comes from: existing methods, literature reviews, lab results and experience/expertise. The second category relates to developing the framework using the identified data, comprising: extracting data, and synthesising and amalgamating this data iteratively. The third and final category is refining and validating the framework: trialling the framework with pilot or case studies and or Delphi panels.
The scoping review results were used as a basis for the following outline of suggestions that may be considered for developing a methodological framework on. The three phases underpinned the structure and specific approaches were included within those phases. These are summarised in Figure 3 and explained in greater detail below.
Phase 1 – Identifying evidence to inform the methodological framework
This phase is split into two; the first is identifying previous frameworks or guidance which are used for the foundations of the new methodological framework, the second is identifying new data to help develop the methodological framework. This new data can be identified in numerous ways: purposeful literature searches, qualitative research (focus groups, interviews, surveys), collaboration between interested parties and the experience and expertise of the developers. If qualitative research is included, if possible it should be conducted with experts in the field of the methodological framework and not restricted to author experiences if possible.
Phase 2 – Developing the methodological framework
In this phase the frameworks or guidance identified in Phase 1 are adapted, combined with other guidance and built upon to create the foundations of the new methodological framework. Key information in the new data identified in Phase 1 should be extracted using appropriate methods. Appropriate methods include; transcribing qualitative data, entering themes into predesigned tables, and entering quantitative information into piloted data extraction forms. Once the information is extracted it should be analysed, synthesised, and grouped or amalgamated into categories to inform the new framework. This should be an iterative process; after grouping or amalgamation of the new data, it should be brought back to key experts and the study team for refinement. This iterative approach should be followed until consensus is reached on the proposed methodological framework.
Phase 3 – Evaluate and refine
In this final stage the proposed methodological framework should be evaluated and refined. Evaluation techniques include using case studies to pilot the methodological framework and Delphi panels. The results from this evaluation should be used to refine the methodological framework if appropriate. Refining will include updating the methodological framework with any changes identified from the evaluation stage and presenting these changes to key experts and the study team for verification.
These suggestions are not intended to be prescriptive, and the developer should adapt them to their specific situation. Finally, the developer should include the term ‘methodological framework’ at least in the title of the study, preferably in the body of the text too and as a keyword if possible.