Evaluation of the Self-reported Questionnaires used to Assess Mental Health After the January 2015 Terrorist Attacks in the Paris Region: IMPACTS Survey

doi:10.21203/rs.3.rs-315390/v1

Download PDF

Research Article

Evaluation of the Self-reported Questionnaires used to Assess Mental Health After the January 2015 Terrorist Attacks in the Paris Region: IMPACTS Survey

https://doi.org/10.21203/rs.3.rs-315390/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background: Gold standard measures in mental health are composed of structured clinical interviews. The design of a study could impede using gold standard measures. Several self-report questionnaires are then used to screen for a number of psychiatric symptoms. Therefore, the need to understand which among those would be most appropriate to be applied after terrorism. The present study examined the performance of self-report questionnaires used to assess post-traumatic stress disorder (PCL-S) and symptoms of depression and anxiety (HADS) compared to the MINI among civilians and first responders involved in terror attacks.

Methods: This study was based on the data from the IMPACTS Survey conducted from 6 to10 months among civilians (N=190) and first responders (N=232) after the January 2015 terrorist attacks in the Paris Region, France. Sensitivity and specificity of the PCL-S and HADS were estimated by the ROC curve. The optimal threshold for each of the questionnaires was defined using the Youden index. The area under the curve was determined to assess the ability of the instruments to diagnose or score the proposed disorders.

Results: Data of 190 civilians and 232 first responders were used to assess the performance of PCL-S and HADS. Concerning the PCL-S: for civilians, the overall AUC was 0.947, and the optimal threshold was 38.5; for first responders, the overall AUC was 0.899, and the optimal threshold was 39.5. Regarding the HADS-D: for civilians, the overall AUC was 0.908 and optimal threshold was 7.5; for first responders, the overall AUC was 0.617 and the “optimal” threshold was 1.5. About the HADS-A for civilians, the overall AUC was 0.823 and the optimal threshold was 9.5; for first responders, the overall AUC was 0.717 and the optimal threshold was 6.5.

Conclusions: Our findings demonstrated a satisfactory performance of the PCL-S and the HAD-D to screen for PTSD or depression (respectively) compared to the MINI, and an unsatisfactory performance of the HAD-A to assess anxiety disorders compared to the MINI. It would be interesting if the study were replicated in other countries and in different types of disaster.

Psychiatry

mental health disorders

self-report questionnaires

terrorism

HAD-S

PCL-S

MINI International Neuropsychiatric Interview

Individuals confronted with extreme events such as disasters are at an increased risk of experiencing mental disorders [1]. Transitory distress symptoms are prevalent after traumatic events. Within the first days or weeks, physical, behavioural, cognitive and emotional reactions are expected [2, 3]. Nevertheless, most exhibit psychological distress with pathways combining adaptation and mitigation, and do not develop psychiatric disorders [4, 5].

The most common mental health outcomes after terror attacks associate with symptoms of depression and anxiety [4], substance abuse [5, 6], and, in more severe cases, post-traumatic stress disorder (PTSD) [7–11]. Therefore, it is important to bear in mind the existence of comorbidities. Depression is frequently diagnosed in patients with anxiety. Studies show that 70% of patients who have a depressive disorder may also present anxiety symptoms [12, 13]. In disasters and emergencies, anxiety and depression are often comorbid with PTSD and vice-versa [14–16]. Consequently, the chances of a misdiagnosis are probable due to the presence of overlapping symptoms between different disorders.

Gold standard measures to provide diagnosis in mental health are composed of structured clinical interviews that should be conducted by mental health professionals and trained researchers/professionals [17, 18]. Spoon et al [19] suggest a well described list of gold standard structured interviews to screen for PTSD, including the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM–IV), Clinician Administered PTSD Scale, the Mini-International Neuropsychiatric Interview (MINI), the Composite International Diagnostic Interview (CIDI), and the PTSD Symptom Scale – Interview. For depression and anxiety, gold standard screening measures are still to be recommended [20–23].

Even though highly criticized because of scarce validation studies [24] and various types of bias [25, 26], self-report questionnaires can be greatly used to monitor the progress of a treatment or the evolution of a disorder. In research projects, depending on the design of a study, self-report questionnaires could be applied considering a number of different reasons: the simplicity to collect data, the possibility to include a great number of participants in a short period of time, as an alternative to face-to-face interviews with specialized or trained interviewers, the opportunity to reach participants across an extensive geographical area, the low costs, among others.

Currently, several different self-report questionnaires are used to assess PTSD (the PTSD Checklist for DSM (PCL), Impact of Event Scale – Revised (IES-R), Post-traumatic Diagnostic Scale for DSM-5 (PCL-5), Trauma Symptom Checklist – 40 (TSC-40) and Trauma Symptom Inventory (TSI)), symptoms of depression (the Hospital Anxiety and Depression Scale (HADS), Patient Health Questionnaire - 9 (PHQ-9) and Patient Health Questionnaire – 8 (PHQ-8), Beck Depression Inventory (BDI)) and symptoms of anxiety (the Depression, Anxiety and Stress Scale-21 (DASS-21), Generalized Anxiety Disorder – 7 (GAD-7), Beck Anxiety Inventory (BAI), Hamilton Anxiety Scale (HAM-A)). Despite this wide range of questionnaires, in case of the impossibility of applying gold standard measures in mental health (face-to-face interviews with trained researchers or specialized professionals), it is important to decide for a self-report questionnaire that would provide the most similar results compared to standard structured clinical interviews.

In January 7, 2015, a series of terrorist attacks targeted the greater Paris area. It started with the massacre at Charlie Hebdo magazine causing the death of 12 people. Several related attacks followed the shooting at the satirical weekly magazine from the 7^th until the 9^th of January. In total, 17 people were killed and 20 were injured. Bearing in mind the psychological components terror attacks may entail, the French National Public Health Agency (Santé Publique France), with the support of the Greater Paris Regional Health Agency (ARS-Île-de-France), launched the longitudinal survey IMPACTS (the French acronym for Investigation of trauma consequences among people exposed to the January 2015 terrorist attacks : support and mental care) to 1) evaluate changes over time in the familial and occupational dimensions and their interactions with mental health, and 2) to assess the associated care and support provided to civilians and first responders [27].

Due to the scarce research on the best instruments to assess mental health in the context of terrorist attacks, the IMPACTS survey investigated mental health outcomes for both, civilians and first responders, with the use of self-report questionnaires and structured clinical interviews performed by specialized professionals. The IMPACTS survey provided then the opportunity to assess, in the context of terrorist attacks, the performance of the following two self-report questionnaires which investigated PTSD and symptoms of depression and anxiety, respectively the PCL-S and the HADS, compared to a structured clinical interview proposed by the MINI.

Thus, the present study aimed to compare the performance of the PCL-S and the HADS compared to the MINI, civilians and first responders involved in the January 2015 terror attacks six to ten months after the events.

Survey design

The first wave of the IMPACTS survey was conducted among civilians and first responders who were exposed to the January terrorist attacks between June and October 2015 (six months after the terror attacks). The second wave of the study was conducted one year after the first wave (18 months after the terror attacks) between June and October 2016. The survey design has been described elsewhere [27, 28]. To meet the purposes of this study, we assessed the data from the first wave.

The IMPACTS survey received approval from the Committee of Ethics and Deontology (CED) of Santé Publique France in 2015, and from CNIL (the French National Commission on Informatics and Liberties, notice No. 915262), CPP (the French ethical research committee, notice No. 3283) and CCTIRS (the French Advisory Committee on Information Processing in Material Research in the Field of Health, notice No. 150522B-31). Written informed consent was obtained from all participants.

Civilian population

The survey was conducted among four different categories of civilians: 1) injured, hostages or witnesses, 2) members of the editorial staff of the Charlie Hebdo magazine, 3) residents or workers sited 100 meters from the events, 4) those identified by other victims. The inclusion criteria were the following: being classified in one of the four previous categories, aging 16 or more, meeting the exposure criteria A for PTSD of the DSM-5 [29]. There were 190 participants in the first wave of the study.

First responder population

The survey was conducted among i) fire fighters, ii) police officers, iii) medical and medico-psychological emergency teams, and iv) rescue workers (trained volunteers and professionals) from two different organizations: the French Red Cross, and the Civil Protection of Paris. To be included, first responders had to a) have worked in one of the attacks within the first 12 hours, b) meet at least one of the following conditions: 1) being in contact with someone directly threatened or injured; 2) being held responsible for the support/treatment of a victim; 3) being held responsible for medical or psychological support to a victim; 4) being a relative of a victim; 5) being in need to go back to scene of the attacks; 6) being in direct contact with the terrorists; 7) being in need to watch video or images of the event (e.g., action-cam such as GoPro); 8) having fatally lost a partner in the attacks. There were 232 participants in the first wave of the study.

Study Variables

For description of the study population, we used the following variables: sex, age at time of the Wave 1, educational level (higher or lower than high-school diploma), occupational status (employed/unemployed) and living with someone (yes/no) at the time of the Wave 1, and exposure category:

- directly threatened: suffering from physical injuries, taken hostage, or present at the scene of the event and exposed to at least one of the following situations: eye contact with/heard the voice of/talked with the terrorists; seen a weapon pointed directly to her/himself.

- indirectly threatened: present at the scene during the attacks - but not in the category "directly threatened" - and having at least one of the following exposures: seen/heard someone else being threatened/being injured/dying; seen blood or inert/dead bodies; touched injured/inert/dead bodies, smelled gunpowder.

- witnesses: living or working within a 100 meters radius of the events and not in the categories “directly/indirectly threatened”

- close relatives from those who were murdered, injured and/or taken hostage.

Description of the instruments (MINI, PCL-S, and HADS)

All characteristics of instruments used in this study were described in Table 1.

Mini-International Psychiatric Interview (MINI)

The version 6 of MINI was used in the study [30]: MINI is a short semi-structured diagnostic interview compatible with DSM-5 and International Classification of Disease, 11^th version (ICD-11). It explores the presence of diagnostic criteria for the 17 most common disorders in mental health. There are several versions of MINI and the standard version meets most of clinical and research needs. Questions are phrased to allow only “yes” or “no” answers With an administration time of approximately 15 minutes, it stands for a concise but precise structured psychiatric interview for multicentre clinical trials and epidemiology studies [30]. For this study, MINI was used as the gold standard to assess PTSD, depression and anxiety.

Post-Traumatic Stress Disorder Checklist

The PTSD Checklist (PCL) is one of the most applied self-report questionnaires to assess PTSD [31]. PCL for DSM-IV has three versions, PCL-M (military), PCL-C (civilian), and PCL-S (specific), which vary slightly in the instructions and wording of the phrase referring to the index event. PCL-S was the one used in this study. PCL-S it a short and simple self-administered questionnaire that measures the three main symptoms of PTSD [32] and it can be scored to present a provisional PTSD diagnosis [19, 33]. PCL-S focuses on symptoms related to a single event. It consists of 17 items assessing the intensity of the 17 symptoms of PTSD presented in the DSM-IV. These items are rated by the subject on a scale from 1 to 5. These 17 items can be grouped into three sub-scales corresponding to the 3 sub-syndromes of PTSD: repetition (items 1 to 5) corresponding to DSM-IV Criterion B; avoidance (items 6 to 12) corresponding to Criterion C; and neuro-vegetative hyperactivity (items 13 to 17) corresponding to Criterion D [34].

Hospital Anxiety Depression Scale (HADS)

HADS measures the perception’s intensity of seven indicative symptoms of depression (HADS-D) (7 questions scoring from 0 to 3) and seven indicative symptoms of anxiety (HADS-A) (7 questions scoring from 0 to 3). The questionnaire takes into consideration how the respondent felt in the previous week. HADS detects the states of anxiety (focusing mainly on symptoms of generalized anxiety) or depression (aiming attention to anhedonia) [35]. If the score is above 8, an assessment from a specialist should be required [36].

Table 1

Description of the instruments used in this study (IMPACTS survey, 2015)
Measured disorders	Scales used in the survey	Measurement period (past)	Number of items	Results
Depression	HADS-D	7 days	7 items (0-3)	Symptoms Score (0-21)
Depression	MINI	15 days	15 questions	Provisional diagnosis (yes/no)
Anxiety symptoms	HADS-A	7 days	7 items (0-3)	Symptoms Score (0-21)
	MINI Panic disorder without agoraphobia	1 month	18 questions	Provisional diagnosis (yes/no)
	MINI Agoraphobia without a history of current panic disorder	1 month	3 questions	Provisional diagnosis (yes/no)
	MINI Current social phobia	1 month	4 questions	Provisional diagnosis (yes/no)
	MINI Generalized anxiety	6 months	11 questions	Provisional diagnosis (yes/no)
PTSD	PCL-S	1 month	17 items (1-5)	Symptoms Score (17-85)
PTSD	MINI	1 month	14 questions	Provisional diagnosis (yes/no)

Statistical analysis

The reliability of the instruments under investigation was measured using Cronbach’s alpha internal consistency coefficient; results > 0.7 were considered adequate.

Using the MINI as criterion, the implemented analyses considered were as follows: MINI vs PCL-S for evaluating of PTSD, HADS-D vs MINI for evaluating symptoms of depression, and HADS-A vs. MINI for evaluating symptoms of anxiety. MINI investigates four different types of anxiety disorders (generalized anxiety, panic disorder, agoraphobia and social phobia). We compared MINI and HADS-A for these four types of anxiety disorders but only the generalized anxiety is presented in this paper. Indeed, the outcomes from the generalized anxiety were the closest to those measured by the HADS-A (data from the other three types are presented in the supplementary material).

The receiver operator characteristic (ROC) curve [37] was used to indicate the sensitivity (the true positive: the proportion of those who were diagnosed by the MINI and also met the score for depression or anxiety by HADS or PTSD by PCL-S) and specificity (the false positive: the proportion of those who were not diagnosed by the MINI and met the score for depression or anxiety by the HADS or PTSD by the PCL-S). The optimal threshold for each of the scales was defined using the Youden index [38]. The area under the curve (AUC) was determined to assess the ability of the instruments to diagnose or to score the proposed disorders. AUC values ≥ 0.70 are considered acceptable, while values ≥ 0.80 are considered good. The analyses were only performed on the case of at least 5 positive cases of PTSD, current major depressive episode or generalized anxiety according to the MINI.

Statistical analysis was performed using R version 4.0.0 for windows and plots were made using the ggplot2 package version 3.3.1 and the plot ROC package version 2.2.1. Confidence intervals for the AUC were obtained with bootstrap using the pROC package version 1.16.2.

Description of the participants

Civilians (n=190) and first responders (n=232) who participated to the IMAPCTS survey aged, in average, 42 years old and 36 years old, respectively (Table 2). There were 60.5% of women among civilians, and 31.4% among first responders. Approximately, 70% of participants were high school graduated for both populations. In the time of the survey (6 months after the attacks), 17.9% of civilians and 11.2% of first responders were unemployed; 24.2% of civilians and 38.4% of first responders lived alone.

Considering the exposition to the terrorist attacks, the majority of the civilians were less than 10 meters from the site of the attacks, very close or in a next room (44.2%), or in the neighbouring building (46.8%). First responders were mostly elsewhere (62.3%). Consequently, 30.5% of the civilians were directly exposed to the events and 43.2% indirectly threatened. Whereas, first responders were mostly indirectly threatened by the events (58.0%) or were witnesses (36.3%).

Among first responders, 19.4% constituted the emergency outreach psychosocial team, 25.9% were firefighters, 23.7% were police officers, and 31.0% were rescue workers.

Table 2

Descriptive statistics for both, civilians and first responders, 6 months after the January 2015 attacks in the Paris Region (IMPACTS survey, 1st wave, 2015).
	Civilians (N=190)		First responders (N=232)
*Sociodemographic*	mean	Min-max	mean	Min-max
Age	42	19-84	36	19-70
	N	%	N	%
Female Gender	115	60.5	73	31.4
Educational level <high-school diploma	57	30.2	69	29.7
Unemployed	34	17.9	26	11.2
Living alone	46	24.2	89	38.4
*Exposition*
Objective exposure
Directly threatened	58	30.5	14	6.1
Indirectly threatened	82	43.2	134	58.0
Witness	36	18.9	84	36.3
Close relative of victims	14	7.4	-	-
Type of rescue
Medical			45	19.4
Firefighters			60	25.9
Rescue Workers			72	31.0
Policemen			54	23.7

Mental health disorders

Regarding PTSD measured by the MINI, 17.9% of civilians (n=190) and 2.9% of first responders (n=232) met the criteria for PTSD in the month of the survey. As regards with depression measured by the MINI, 10.5% of civilians and 0.8% of first responders met the criteria for a major depressive episode two weeks previously the survey. Considering anxiety also measured by the MINI, 5.3% of civilians and 5.0% of first responders met criteria for generalized anxiety (Table 3). Concerning the others anxiety disorders, 2.6% of civilians and 1.3% of first responders have had symptoms of panic disorder in the previous year, 25.8% of civilians and 8.8% of first responders suffered from current agoraphobia, and 4.7% of civilians and 0.8% of first responders had suffered from social phobia in the previous month (supplementary material).

Considering PTSD measured by the PCL-S, the median score for symptoms of PTSD was 31 among civilians (ranging from 17 to 71) and 20 (ranging from 16 to 65) among first responders (Table 3). Regarding the use of the HADS, the median score for symptoms of depression was 3 (ranging from 0 to 17) among civilians and 1 among first responders (ranging from 0 to 10). The median score for symptoms of anxiety was 7 (ranging from 0 to 20) among civilians and 5 among first responders (ranging from 0 to 17).

Table 3

Descriptive statistics of the mental health disorders based on the PCL-S, HADS or MINI for both, civilians and first responders, 6 months after the January 2015 attacks in the Paris Region (IMPACTS survey, 1st wave, 2015).
Disorders		Civilians (N=190)		First responders (N=232)
		PCL-S or HADS
		Median score	Min-max	Median score	Min-max
PTSD (past month)		31	17-78	20	17-65
Depression (past week)		3	0-17	1	0-10
Anxiety (past week)		7	0-20	5	0-17
		MINI
		n	%	n	%
PTSD (past month)	No	156	82.1	225	96.9
	Yes	34	17.9	7	2.9
	Missing	0	0	0	0
Major depressive episode (past 15 days)	No	116	61.1	195	84.1
	Yes	74	38.9	37	15.9
	Missing	0	0	0	0
Current depression (past 15 days)	No	170	89.5	230	99.1
	Yes	20	10.5	2	0.8
	Missing	0	0.0	0	2.5
Generalized anxiety (past 6 months)	No	178	93.7	220	92.4
	Yes	10	5.3	12	5.0
	Missing	2	1.1	0	2.5
Panic disorder (past month)	No	185	97.4	229	98.7
	Yes	5	2.6	3	1.3
	Missing	0	0	0	0
Agoraphobia (past month)	No	140	73.7	210	90.5
	Yes	49	25.8	21	9.1
	Missing	1	0.5	1	0.4
Social phobia (past month)	No	181	95.3	230	99.1
	Yes	9	4.7	2	0.8
	Missing	0	0	0	0
At least one disorder (past month)	No	130	68.4	199	85.7
	Yes	59	31.1	32	13.8
	Missing	1	0.5	1	0.4

Diagnostic performances of PCL-S vs. MINI for PTSD screening

For civilians, the analysis was done on n=187 with 3 observations removed due to missing values in the PCL-S scale. The internal consistency of the PCL-S scale was very good with a Cronbach’s alpha of 0.93. The overall AUC was 0.947 [95% CI: 0.913 - 0.981] (Figure 1). The optimal threshold was 38.5, at a Youden index of 0.768 giving a sensitivity of 97.1% and a specificity of 79.7%.

For first responders, 229 individuals were included in the analysis with 3 observations removed due to missing values in the PCL-S scale. The internal consistency of the PCL-S scale was also very good for first responders with a Cronbach’s alpha of 0.91. The overall AUC was 0.899 [95% CI: 0.725 - 1]. The optimal threshold was 39.5, at a Youden index of 0.821 giving a sensitivity of 85.7% and a specificity of 96.4%.

Diagnostic performances of HADS-D vs. MINI for depression

For civilians, the analysis was done on 189 participants with 1 observation removed due to missing values in the HADS-D. The internal consistency of the HADS-D was found to be good with a Cronbach’s alpha of 0.83 when considering only the items relating to depression. The overall AUC was 0.908 [95% CI: 0.829 - 0.986] (Figure 2). The optimal threshold was 7.5, at a Youden index of 0.743 giving a sensitivity of 85% and a specificity of 89.3%.

The performances of HADS-D for screening Major depressive episode in the last year measured by MINI were not good in civilians neither in responders (Supplementary 1). In civilians, analysis done on n=189 with one observation removed due to missing values in the MINI and/or the scale. The overall AUC was 0.716 [95% CI: 0.638 - 0.794]. The optimal threshold was 3.5, at a Youden index of 0.356 giving a sensitivity of 60.8% and a specificity of 74.8%. For first responders, the analysis was done on n=231 with one observation removed due to missing values in the MINI and/or the scale. The overall AUC was 0.617 [95% CI: 0.518 - 0.716]. The “optimal” threshold was 1.5, at a Youden index of 0.204 giving a sensitivity of 62.2% and a specificity of 58.2%.

Diagnostic performances of HADS-A vs. MINI for generalized anxiety

For civilians, the analysis was done on n=188 with 2 observations removed due to missing values in the MINI. The internal consistency of the HADS-A when considering only the items relating to anxiety was lower than that of the whole scale, with a Cronbach’s alpha of 0.76. The overall AUC was 0.823 [95% CI: 0.681 - 0.964] (Figure 3). The optimal threshold was 9.5, at a Youden index of 0.592 (sensitivity: 80% and specificity: 79.2%.

For first responders, the analysis was done on n=231 with one observation removed due to missing values in the HADS-A. The internal consistency of the HADS-A scale when considering only the items relating to anxiety was found to be quite low with a Cronbach’s alpha of 0.68. The overall AUC was 0.717 [95% CI: 0.543 - 0.89]. The optimal threshold was 6.5, at a Youden index of 0.37 corresponding to a sensitivity of 66.7% and a specificity of 70.3%.

Diagnostic performances of HADS-A vs. MINI for others anxiety symptoms

The performances of HADS-A for screening agoraphobia or social phobia were not satisfactory. Concerning agoraphobia, in civilians, the analysis was done on n=189 with one observation removed due to missing values in the MINI and/or the scale (Supplementary 2). The overall AUC was 0.737 [95% CI: 0.656 - 0.819]. The “optimal” threshold was 6.5, at a Youden index of 0.366 giving a sensitivity of 81.6% and a specificity of 55%. In first responders, the analysis was done on n=230 with two observations removed due to missing values in the MINI and/or the scale. The overall AUC was 0.606 [95% CI: 0.47 - 0.743]. The “optimal” threshold was 10.5 at a Youden index of 0.195 giving a sensitivity of 23.8% and a specificity of 95.7%.

Concerning social phobia in civilians, the analysis was done on n=190 with 0 observation removed due to missing values in the MINI and/or the scale. The overall AUC was 0.836 [95% CI: 0.73 - 0.942]. The “optimal” threshold was 9.5, at a Youden index of 0.557 giving a sensitivity of 77.8% and a specificity of 77.9%. Analysis could not be done in first responders because of the insufficient cases of social phobia in this population.

Early evaluations with the most appropriate instruments are fundamental to implement adequate strategies for the promotion of mental health. Thus, the need to employ solid and validated self-report questionnaires.

Our findings reported high psychometric values, in particular specificity and sensitivity, related to the PCL-S as also found in previous studies [31, 49, 50]. Besides demonstrating a solid capacity of the PCL-S for assessing PTSD cases, our study demonstrated the correct identification of the presence of PTSD, or the non-presence of PTSD in 94.7% of the cases. It was also identified that the optimal threshold for civilians and first responders was 38.5 and 39.5, respectively. The PCL-S could be used as a very good alternative to a standard structured interview for assessing PTSD in critical and complex scenarios such as terrorist attacks.

In our study, HADS–A demonstrated a good performance for civilians. Our findings indicated the correct identification of the presence of generalised anxiety, or the non-presence of generalised anxiety, in 82% of the cases. However, the outcomes were not as promising considering the first responders’ population. The correct identification of the presence of generalised anxiety cases, or the non-presence of generalised anxiety, was of 71%. Our study indicated that the performance of HADS-A was not good for screening other anxiety symptoms such as social phobia or agoraphobia.

The presence of specific anxiety disorders screened by the HADS-A shows contradictory opinions among researchers. Julian [51] highlights that HADS-A detects symptoms of generalized anxiety but (as well as other questionnaires) does not adequately detects specific anxiety disorders. Nonetheless, Bjelland et al [52] demonstrated that HADS-A had proper discriminating properties to screen for anxiety different dimensions of anxiety.

As regards to HADS-D, our findings were satisfactory among civilians. HADS-D indicated the correct identification of the presence of current depression, or the non-presence of current depression, in 90% of the cases. It is important to highlight that we did not run the density plots or ROC Curves for the first responders’ population because of the small of the sample (0.8%) suggestion actual depression by the MINI. Bjelland et al [52] and Kjaergaard et al [53] demonstrated that HADS-D is acceptable for accessing symptoms of depression. Nevertheless, Pettersson et al [54] demonstrated that HADS-D did not meet minimum criteria for sensitivity and specificity.

Some strengths and limitations of the IMPACTS survey were described elsewhere [55]. Concerning the limitations that could impact the results, the sample size was small to carry out more in-depth analyses such as gender-specific analyses or the exposition among the different rescuers categories. For instance, in a meta-analysis conducted by Di Maggio and Galea [56], for studies with predominantly male populations (>80%), the global average prevalence of PTSD was 11%; while for predominantly female populations, the overall prevalence was 16%. Besides this, the heterogeneous professional background could lead to a bias of non-response or social desirability among the first responder. Another limitation refers to the timing of data collection. Data were collected at six months from the terror attacks, therefore, the stability of the results comparing the instruments over time could not be tested.

The strength of the IMPACTS survey is the variety of interviewed populations. Indeed, the recruitment of civilians was the combination of several lists from different authorities. A comprehensive search was carried out among residents and local workers who were potentially present when the events took place, as well as a pro-active search for people which were not listed. Consequently, it was possible to have the representation of several different occupations in the samples. Second, the IMPACTS survey used face-to-face structured clinical interviews performed by specialized professionals to assess psychiatric disorders. The professionals were trained to avoid re-activating PSTD symptoms, and referred participants to mental health services whenever needed. Third, to our knowledge, there is no study comparing the performance of the HADS, the PCL-S and the MINI in the context of terrorism. Besides, we used validated instruments, therefore the study could be easily replicable.

Our findings demonstrated the potential applicability of the PCL-S in the aftermath of terror attacks. The study showed a satisfactory performance of the PCL-S and the HAD-D to screen for PTSD or depression compared to the MINI. However, an unsatisfactory performance of the HAD-A to assess anxiety disorders compared to the MINI. We encourage the replicability of this study in different countries, and in other types of disaster (natural disasters, technological disasters, other mass violence disasters) in order to validate our results.

DSM-4: Diagnostic and Statistical Manual of Mental Disorders, fourth edition

IMPACTS: French acronym for Investigation of Trauma Consequences in People Exposed to the January 2015 Terrorist Attacks and their Support and Mental care (Investigation des manifestations traumatiques post-attentats et de la prise en charge thérapeutique et de soutien des personnes impliquées dans les attentats de janvier 2015 en Île-de-France in French)

MINI: Mini-International Neuro-psychiatric Interview

PTSD: Posttraumatic Stress Disorder

IES-R: Impact of Event Scale – Revised

PCL-S: Post-Traumatic Stress Disorder Checklist (specific)

PCL-5: Post-traumatic Diagnostic Scale for DSM-5

TSC-40: Trauma Symptom Checklist – 40

TSI: Trauma Symptom Inventory

HADS: Hospital Anxiety and Depression Scale

HADS-D: Hospital Anxiety and Depression Scale (depression)

HADS-A: Hospital Anxiety and Depression Scale (anxiety)

PHQ-9: Patient Health Questionnaire -9

PHQ-8: Patient Health Questionnaire -8

BDI: Beck Depression Inventory

DASS-21: Depression, Anxiety and Stress Scale-21

GAD-7: Generalized Anxiety Disorder – 7

BAI: Beck Anxiety Inventory

HAM-A: Hamilton Anxiety Scale

ARS: Greater Paris Regional Health Agency,

ROC: receiver operator characteristic

AUC: area under the curve

CI: Confidence Interval

Ethics approval and consent to participate

All methods were performed in accordance with the relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

The data that support the findings of this study are available from The French Public Health Agency (Santé Publique France), but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of The French Public Health Agency

Funding

The IMPACTS survey was financially funded by the “Fondation d’Aide aux Victimes” and was coordinated by Santé Publique France and the Greater Paris regional health agency (ARS-IdF). The funding bodies had no role in the design of the study, data collection, analysis, interpretation of data, and writing the manuscript.

Contributions

LB assisted in the interpretation of the data and drafted the manuscript. TEA, MH, performed statistical analysis, and revised the manuscript. LA supervised the data collection, designed the first wave of the IMPACTS survey, and contributed to the revision of the manuscript. YM, PP contributed to the questionnaire conceptualization of the first wave of the IMPACTS survey, data analysis and the revision of the manuscript. MEP contributed to the interpretation of the data analysis and revised the manuscript. FDC revised the manuscript. CV designed this study, performed statistical analysis, and revised the manuscript. SV designed, supervised the questionnaire conceptualization and the data collection of the IMPACTS survey, and contributed to the revision of the manuscript. All authors reviewed the draft versions of this manuscript, and read and approved the final version.

Acknowledgements

The authors are most grateful to all the study participants for their involvement especially given the very difficult context for them, and to all the interviewers. The authors would like to acknowledge the support from the national French agency for public health as well as the regional agency for health of Paris area: Anne Gallay, Thierry Cardoso, Anne Laporte, Jean Claude Desenclos, Clothilde Hachin, Naine Isserlis, Michel Gentile, Laurent Kosorotoff, Martial Mettendorf, Claude Evin, and François Bourdillon for generously contribution their time and energy to the conduct of IMPACTS study. A special thanks to Laurent Bernard-Brunel, Alexandra Botero, Jean-Michel Coq, Nathalie Cholin, Nicolas Dantchev, Elise Neff, Marc Grohens, Aurelia Rochedreux, Toufik Selma, Laure Zeltner and Julien Sonnesi for their field investigation. We wish to thank the members of the scientific committee of the study as Bruno Falissard, Nathalie Prieto, Louis Jehel, François Ducrocq, Humbert Boisseaux, Jean Jacques Chavagnat.

Morina N, Wicherts JM, Lobbrecht J, Priebe S. Remission from post-traumatic stress disorder in adults: a systematic review and meta-analysis of long term outcome studies. Clin Psychol Rev. 2014;34:249–55.
Inter-Agency Standing Committee (IASC) Reference Group for Mental Health and Psychosocial Support in Emergency Settings. IASC Reference Group Mental Health and Psychosocial Support Assessment Guide. 2013. www.humanitarianinfo.org/iasc>.
Inter-Agency Standing Committee (IASC) Reference Group for Mental Health and Psychosocial Support in Emergency Settings. A Common Monitoring and Evaluation Framework for Mental Health and Psychosocial Support in Emergency Settings. 2017. https://interagencystandingcommittee.org/mental-health-and-psychosocial-support-emer gency-settings>.
Salguero JM, Fernández-Berrocal P, Iruarrizaga I, Cano-Vindel A, Galea S. Major depressive disorder following terrorist attacks: A systematic review of prevalence, course and correlates. BMC Psychiatry. 2011;11:96.
Galea S, Nandi A, Vlahov D. The social epidemiology of substance use. Epidemiol Rev. 2004;26:36–52.
Schiff M, Zweig HH, Benbenishty R, Hasin DS. Exposure to Terrorism and Israeli Youths’ Cigarette, Alcohol, and Cannabis Use. Am J Public Health. 2007;97:1852–8.
Paz García-Vera M, Sanz J, Gutiérrez S. A Systematic Review of the Literature on Posttraumatic Stress Disorder in Victims of Terrorist Attacks. Psychol Rep. 2016;119:328–59.
Bossini L, Ilaria C, Koukouna D, Caterini C, Olivola M, Fagiolini A. PTSD in victims of terroristic attacks – a comparison with the impact of other traumatic events on patients’ lives. Psychiatr Pol. 2016;50:907–21.
Hansen MB, Birkeland MS, Nissen A, Blix I, Solberg Ø, Heir T. Prevalence and Course of Symptom-Defined PTSD in Individuals Directly or Indirectly Exposed to Terror: A Longitudinal Study. Psychiatry. 2017;80:171–83.
Motreff Y, Baubet T, Pirard P, Rabet G, Petitclerc M, Stene LE, et al. Factors associated with PTSD and partial PTSD among first responders following the Paris terror attacks in November 2015. Journal of Psychiatric Research. 2020;121:143–50.
Shalev AY, Freedman S. PTSD Following Terrorist Attacks: A Prospective Evaluation. AJP. 2005;162:1188–91.
Kessler RC, Gruber M, Hettema JM, Hwang I, Sampson N, Yonkers KA. Co-morbid major depression and generalized anxiety disorders in the National Comorbidity Survey follow-up. Psychol Med. 2008;38:365–74.
Lamers F, van Oppen P, Comijs HC, Smit JH, Spinhoven P, van Balkom AJLM, et al. Comorbidity Patterns of Anxiety and Depressive Disorders in a Large Cohort Study: the Netherlands Study of Depression and Anxiety (NESDA). J Clin Psychiatry. 2011;72:341–8.
Armenta RF, Walter KH, Geronimo-Hara TR, Porter B, Stander VA, LeardMann CA, et al. Longitudinal trajectories of comorbid PTSD and depression symptoms among U.S. service members and veterans. BMC Psychiatry. 2019;19:396.
Koenen KC, Moffitt TE, Caspi A, Gregory A, Harrington H, Poulton R. The developmental mental-disorder histories of adults with posttraumatic stress disorder: A prospective longitudinal birth cohort study. Journal of Abnormal Psychology. 2008;117:460–6.
Walter KH, Levine JA, Highfill‐McRoy RM, Navarro M, Thomsen CJ. Prevalence of Posttraumatic Stress Disorder and Psychological Comorbidities Among U.S. Active Duty Service Members, 2006–2013. JOURNAL OF TRAUMATIC STRESS. 2018;31:837–44.
First MB, Williams JBW, Karg RS, Spitzer RL. Structured clinical interview for DSM-5 disorders: scid-5-cv: clinician version. Arlington,VA: American Psychiatric Association Publishing; 2016.
Weathers FW, Bovin MJ, Lee DJ, Sloan DM, Schnurr PP, Kaloupek DG, et al. The Clinician-Administered PTSD Scale for DSM–5 (CAPS-5): Development and initial psychometric evaluation in military veterans. Psychological Assessment. 2018;30:383–95.
Michele Spoont , first, Paul Arbisi, Steven Fu, Nancy Greer, Shannon Kehle-Forbes, Laura Meis, et al. Screening for Post-Traumatic Stress Disorder (PTSD) in Primary care: a systematic review. 2013.
Dunstan DA, Scott N, Todd AK. Screening for anxiety and depression: reassessing the utility of the Zung scales. BMC Psychiatry. 2017;17:329.
Maurer DM, Raymond TJ, Davis BN. Depression: Screening and Diagnosis. Am Fam Physician. 2018;98:508–15.
Plummer F, Manea L, Trepel D, McMillan D. Screening for anxiety disorders with the GAD-7 and GAD-2: a systematic review and diagnostic metaanalysis. General Hospital Psychiatry. 2016;39:24–31.
Smithson S, Pignone MP. Screening Adults for Depression in Primary Care. Medical Clinics of North America. 2017;101:807–21.
Terhakopian A, Sinaii N, Engel CC, Schnurr PP, Hoge CW. Estimating population prevalence of posttraumatic stress disorder: An example using the PTSD checklist. J Traum Stress. 2008;21:290–300.
Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. Journal of Multidisciplinary Healthcare. 2016;9:211–7.
Demetriou C, Ozer BU, Essau CA. Self-Report Questionnaires. In: Cautin RL, Lilienfeld SO, editors. The Encyclopedia of Clinical Psychology. Hoboken, NJ, USA: John Wiley & Sons, Inc.; 2015. p. 1–6. doi:10.1002/9781118625392.wbecp507.
Vandentorren S, Pirard P, Sanna A, Aubert L, Motreff Y, Dantchev N, et al. Healthcare provision and the psychological, somatic and social impact on people involved in the terror attacks in January 2015 in Paris: cohort study. Br J Psychiatry. 2018;212:207–14.
Vuillermoz C, Stene LE, Aubert L, Motreff Y, Pirard P, Baubet T, et al. Non-participation and attrition in a longitudinal study of civilians exposed to the January 2015 terrorist attacks in Paris, France. BMC Med Res Methodol. 2020;20:63.
American Psychiatric Association, American Psychiatric Association, editors. Diagnostic and statistical manual of mental disorders: DSM-5. 5th ed. Washington, D.C: American Psychiatric Association; 2013.
Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59 Suppl 20:22-33;quiz 34-57.
Ibrahim H, Ertl V, Catani C, Ismail AA, Neuner F. The validity of Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5) as screening instrument with Kurdish and Arab displaced populations living in the Kurdistan region of Iraq. BMC Psychiatry. 2018;18:259.
Cottraux J, et al. Evaluation of Post-traumatic Stress Disorder: validation of a measure, the PCLS. L’Encephale. 2002;29 3 Pt 1:232–8.
Olff M. Choosing the right instruments for psychotrauma related research. European Journal of Psychotraumatology. 2015;6:30585.
Kilpatrick DG, Resnick HS, Milanak ME, Miller MW, Keyes KM, Friedman MJ. National Estimates of Exposure to Traumatic Events and PTSD Prevalence Using DSM-IV and DSM-5 Criteria: DSM-5 PTSD Prevalence. JOURNAL OF TRAUMATIC STRESS. 2013;26:537–47.
Hamilton M. The assessment of anxiety states by rating. Br J Med Psychol. 1959;32:50–5.
Bentz L, Pirard P, Motreff Y, Vandentorren S, Baubet T, Fabre R, et al. Health outcomes of the July 14, 2016 Nice terror attack among hospital-based professionals and students: the « ECHOS de Nice » health survey protocol. BMC Public Health. 2019;19:1163.
Twisk JWR. Applied Longitudinal Data Analysis for Epidemiology. 2nd edition. Cambridge: Cambridge University Press; 2013. doi:10.1017/CBO9781139342834.
Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3:32–5.
Norris FH, Friedman MJ, Watson PJ. 60,000 disaster victims speak: Part II. Summary and implications of the disaster mental health research. Psychiatry. 2002;65:240–60.
North CS, McCutcheon V, Spitznagel EL, Smith EM. Three-year follow-up of survivors of a mass shooting episode. J Urban Health. 2002;79:383–91.
Skogstad L, Heir T, Hauff E, Ekeberg Ø. Post-traumatic stress among rescue workers after terror attacks in Norway. OCCMED. 2016;66:528–35.
Charlson FJ, Steel Z, Degenhardt L, Chey T, Silove D, Marnane C, et al. Predicting the impact of the 2011 conflict in Libya on population mental health: PTSD and depression prevalence and mental health service requirements. PLoS One. 2012;7:e40593.
Mollica RF. Disability Associated With Psychiatric Comorbidity and Health Status in Bosnian Refugees Living in Croatia. JAMA. 1999;282:433.
Silver RC, Holman EA, McIntosh DN, Poulin M, Gil-Rivas V. Nationwide longitudinal study of psychological responses to September 11. JAMA. 2002;288:1235–44.
DiMaggio C, Galea S. The behavioral consequences of terrorism: a meta-analysis. Acad Emerg Med. 2006;13:559–66.
Whalley MG, Brewin CR. Mental health following terrorist attacks. Br J Psychiatry. 2007;190:94–6.
Pintor L. Relapse of major depression after complete and partial remission during a 2-year follow-up. Journal of Affective Disorders. 2003;73:237–44.
McKay D, Heisler M, Mishori R, Catton H, Kloiber O. Attacks against health-care personnel must stop, especially as the world fights COVID-19. The Lancet. 2020;395:1743–5.
Math SB, Nirmala MC, Moirangthem S, Kumar NC. Disaster Management: Mental Health Perspective. Indian J Psychol Med. 2015;37:261–71.
Wilkins KC, Lang AJ, Norman SB. Synthesis of the Psychometric Properties of the PTSD Checklist (PCL) Military, Civilian, and Specific Versions. Depress Anxiety. 2011;28:596–606.
Julian LJ. Measures of anxiety: State-Trait Anxiety Inventory (STAI), Beck Anxiety Inventory (BAI), and Hospital Anxiety and Depression Scale-Anxiety (HADS-A). Arthritis Care Res (Hoboken). 2011;63 Suppl 11:S467-472.
Bjelland I, Dahl AA, Haug TT, Neckelmann D. The validity of the Hospital Anxiety and Depression Scale. Journal of Psychosomatic Research. 2002;52:69–77.
Kjærgaard M, Wang CEA, Waterloo K, Jorde R. A study of the psychometric properties of the Beck Depression Inventory-II, the Montgomery and Åsberg Depression Rating Scale, and the Hospital Anxiety and Depression Scale in a sample from a healthy population. Scandinavian Journal of Psychology. 2014;55:83–9.
Pettersson A, Boström KB, Gustavsson P, Ekselius L. Which instruments to support diagnosis of depression have sufficient accuracy? A systematic review. Nordic Journal of Psychiatry. 2015;69:497–508.
Vandentorren S, Pirard P, Sanna A, Aubert L, Motreff Y, Dantchev N, et al. Healthcare provision and the psychological, somatic and social impact on people involved in the terror attacks in January 2015 in Paris: cohort study. Br J Psychiatry. 2018;212:207–14.
DiMaggio C, Galea S, Li G. Substance use and misuse in the aftermath of terrorism. A Bayesian meta-analysis. Addiction. 2009;104:894–904.

No competing interests reported.

AdditionalfileBMC.pdf

Download PDF

Version 1

posted

You are reading this latest preprint version

Evaluation of the Self-reported Questionnaires used to Assess Mental Health After the January 2015 Terrorist Attacks in the Paris Region: IMPACTS Survey

Status:

Version 1

Abstract

Figures

Background

Methods

Survey design

Civilian population

First responder population

Study Variables

Description of the instruments (MINI, PCL-S, and HADS)

Mini-International Psychiatric Interview (MINI)

Post-Traumatic Stress Disorder Checklist

Hospital Anxiety Depression Scale (HADS)

Statistical analysis

Results

Description of the participants

Mental health disorders

Diagnostic performances of PCL-S vs. MINI for PTSD screening

Diagnostic performances of HADS-D vs. MINI for depression

Diagnostic performances of HADS-A vs. MINI for generalized anxiety

Diagnostic performances of HADS-A vs. MINI for others anxiety symptoms

Discussion

Conclusions

Abbreviations

Declarations

Ethics approval and consent to participate

Consent for publication

Competing interests

Availability of data and materials

Funding

Contributions

Acknowledgements

References

Additional Declarations

Supplementary Files

Status:

Version 1