15 of the 100 publications were only study protocols and 85 were reports of completed studies. 14 were published in 2017 (updated versions), 64 in 2016 and 22 in 2015. The details of the reported results are listed in the appendices B and C.
Characteristics for description of the study functions:
Description of the objective of the study:
No clear study question was observed in about 40% of the 100 papers. We consider this a serious problem that requires further discussion. However, a detailed analysis is beyond the scope of this paper and will be reported in a separate publication. We also observed considerable variation in the terminology used for describing the objective of the study and will include a discussion of this aspect in our next publication.
A detailed list of the identified study questions can be found in appendix D.
Differentiation of HRQoL as primary or secondary endpoint of the study:
The HRQoL was assigned as primary endpoint in 29 studies (29%) (including eight study protocols) and as secondary endpoint in 38 studies (38%) (including seven study protocols). In 33 studies (33%) no information was provided for the description of the primary endpoint or it was not clear from the description whether the HRQoL was chosen as the primary or secondary endpoint.
Classification as ‘Efficacy’ or ‘Effectiveness’ trial:
In 14 (14%) of the examined studies (including one study protocol) it was described that ‘Efficacy’ of a therapy was measured. 25 studies (25%) (including seven study protocols) reported that ‘Effectiveness’ of a therapy was measured. In 60 studies (60%) (including six study protocols) no specification of ‘Efficacy’ or ‘Effectiveness’ was described. In one study (1%) (study protocol) the terms ‘Efficacy’ and ‘Effectiveness’ were used synonymously.
Numbers of described inclusion and exclusion criteria:
Both inclusion and exclusion criteria were reported in 69 (69%) of 100 studies (including 15 study protocols). Only inclusion but no exclusion criteria were reported in six studies (6%). There was no publication that reported only exclusion but no inclusion criteria, but there were 25 (25%) publications that reported neither inclusion nor exclusion criteria. Inclusion criteria were reported in detail in 75 (75%) of 100 publications (including 15 study protocols). The median number of explicitly reported inclusion criteria in these 75 documents was four with an IQR (interquartile range) of three to five. Exclusion criteria were described in 69 (69%) of 100 publications, the median number of explicitly reported exclusion criteria being four with an IQR of two to six. A more detailed version of the data on inclusion and exclusion criteria in the 100 HRQoL studies has already been published previously [13].
In summary, the four aspects that describe the functions of a study are frequently not reported. The objective of the study was not clearly defined in 40%, the primary or secondary endpoint was not differentiated in 33%, the dimension (Efficacy or Effectiveness) was not defined in 60% and the selection of patients (inclusion or exclusion criteria) was not defined in 25% of all examined studies. We can conclude that in 60% of these papers at least one of the important functional criteria was not reported.
Characteristics for description of the study forms:
Specification of ‘Randomised Controlled Trial’ (‘RCT’) or other than ‘Randomised Controlled Trial’ (Non-‘RCT’):
In this category we differentiated two categories: studies which were described as ‘Randomized Controlled Trial’ and any other type of study except ‘Randomized Controlled Trial’.
91 (91%) of the studies (including 15 study protocols) were designated as ‘RCTs’. Nine trials (9%) were not designated as ‘RCTs’.
In Table 1a we demonstrate the high variability of different coincidences of the terms ‘Efficacy’ or ‘Effectiveness’ or ‘used synonymously’ or ‘not defined’ and of the terms ‘RCT’/other than ‘RCT’.
Terms used to describe the assessed outcome dimension
|
‘RCT’
|
Non-‘RCT’
|
Total
|
‘Efficacy’
|
14
|
0
|
14
|
‘Effectiveness’
|
23
|
2
|
25
|
Both terms used synonymously
|
1
|
0
|
1
|
Not defined
|
53
|
7
|
60
|
Total
|
91
|
9
|
100
|
Table 1a: Numbers of studies assessing Health-Related Quality of Life (n=100) which used the terms ‘Efficacy’ or ‘Effectiveness’ or used both terms synonymously or did not define the described outcome dimension.
RCT: Randomised Controlled Trial.
Specification as ‘pragmatic’ or other than ‘pragmatic’ trial:
All of the 100 studies were classified as ‘pragmatic’ or other than ‘pragmatic’ studies. Six (6%) of the studies (including two study protocols) were described as 'pragmatic' mostly in the method section. These six trials were also named ‘randomized’ in the title. More precisely five studies were called ‘Randomized Controlled Trial’ [14 - 18] and one study was called 'Randomized Clinical Trial' [19]. To report the details of the used terminology we found that one study [14] was named a 'Pragmatic Randomized Trial', three studies [15 - 17] (including two study protocols) a 'Pragmatic (...) Randomized Controlled Trial', one study [19] a 'Pragmatic (...) Randomized Clinical Trial' and one study [18] a 'Pragmatic (...) Controlled Trial'. In none of these six studies an explanation of the meaning of ‘pragmatic’ was described.
The remaining 94 studies are summarized as other than ‘pragmatic’ trials. These 94 studies include 85 ‘RCTs’ and nine Non-‘RCTs’.
Terms used to describe the assessed outcome dimension
|
‘Pragmatic’
|
Other than ‘pragmatic’
|
Total
|
‘Efficacy’
|
1
|
13
|
14
|
‘Effectiveness’
|
4
|
21
|
25
|
Both terms used synonymously
|
0
|
1
|
1
|
Not defined
|
1
|
59
|
60
|
Total
|
6
|
94
|
100
|
Table 1b: Numbers of studies assessing Health-Related Quality of Life (n=100) which used the terms ‘Efficacy’ or ‘Effectiveness’ or both synonymously or did not define the described outcome dimension for presentation of data in an ‘pragmatic’ (observational) or any other type of study.
The following 2x2 table describes the ‘RCT’/Non-‘RCT’ studies versus the studies designated as ‘pragmatic’/other than ‘pragmatic’:
|
‘RCT’
|
Non-‘RCT’
|
Total
|
Other than ‘pragmatic’
|
85
|
9
|
94
|
‘Pragmatic’
|
6
|
0
|
6
|
Total
|
91
|
9
|
100
|
Table 2: Number of studies assessing Health-Related Quality of Life (n=100) which were classified as ‘pragmatic’/other than ‘pragmatic’ and which were designated as ‘RCT’ or Non-‘RCT’. The bold figures represent the indicated numbers of studies. The remaining numbers were calculated. We identified six of 100 studies that were named ‘pragmatic’ und simultaneously ‘RCT’. Additional nine studies that were Non-‘RCTs’ were classified other than ‘pragmatic’. These numbers are shown as red figures in our table. This table seems to identify an oxymoron in terminology.
RCT: Randomised Controlled Trial.
Description of the HRQoL measurement instruments used:
In all 100 studies the used instruments for assessment of HRQoL were described. In 28 studies (28%) (including two study protocols), only generic measurement instruments were used. In 55 studies (55%) (including eight study protocols) only disease-specific measurement instruments were used. In 17 studies (17%) (including five study protocols) both types of measurement instruments were used.