Participants
The present study encompassed two samples, both recruited from the department of child and adolescent psychiatry at Kamibayashi Memorial Hospital in Japan. The first sample consisted of 41 consecutive outpatients seen by the first author (TN) from March 3 to April 21, 2018, and was employed for validating the ASD module of the K-SADS-PL for DSM-5. The second sample consisted of 56 outpatients from January 4 to March 20, 2018, and was employed for validating the modules of ADHD, tic disorders, selective mutism, enuresis and encopresis. Since some of the five disorders were not prevalent in the outpatients, patients with at least one of the five disorders (either current or past episodes) were recruited to increase the number of disorders in the sample. The only additional inclusion criterion was age (between five and 18 years in both samples).
Of the 41 and 56 patients we approached, all (100%) and 54 (96%) in the first and second samples, respectively, consented to participate in the present study. The distribution of age and sex between those who did not participate and those who participated did not differ significantly (t-test of age: p-value = 0.557 and Fisher’s exact test of sex ratio: p-value = 0.386).
Measures
Diagnostic Interviews
Each patient’s attending child psychiatrist established a diagnosis for that patient. Consequently, during treatment, the psychiatrist had regular clinical contact with the patient, the multidisciplinary clinical stuff, the patient’s caregiver, and his/her teacher when possible. One of five experienced child psychiatrists made the diagnoses of each patient based on DSM-5 criteria using all available data sources (including previous records, a significant other, psychological assessments, laboratory test results and information provided from a caregiver and a teacher of the patient). Each patient was followed longitudinally for at least three months to assess the stability of the original diagnoses assigned. In the present study, we used a “best estimate” diagnosis22 thus obtained as a gold standard for demonstrating the criterion validity of the K-SADS-PL for DSM-5. Furthermore, this procedure included the Diagnostic Interview for Social and Communication Disorders (DISCO) for the DSM-5 diagnosis of ASD in the first sample.23 The DISCO interview was conducted by the first author (TN), who had been certified to use the instrument for research purposes.
The K-SADS-PL for DSM-5 is a semi-structured diagnostic interview that ascertains both lifetime and current diagnostic episodes in children and adolescents. A current episode refers to the period of maximum severity within the most recent episode. The episode is considered current if the subject has not been symptom-free for at least two months. While the instrument is capable of generating 35 child and adolescent psychiatric diagnoses, we used six modules of ASD, ADHD, tic disorders, selective mutism, enuresis and encopresis in this study. The instrument is divided into two sections: screen interview and diagnostic supplement. The diagnostic supplement for a disorder is skipped if the skip-out criterion in the screen interview for the disorder is satisfied. As noted in the introduction section of the instrument, screen interview is completed before any diagnostic supplements are administered to provide an overview of lifetime psychopathology. As an adaptation in the present study, however, we integrated screen interviews and diagnostic supplements in each disorder unit to use each module separately.
The gold-standard diagnoses were made based on DSM-5 before interviews using the K-SADS-PL for DSM-5 were conducted by nine experienced clinicians blind to the gold-standard diagnoses and questionnaire responses. The K-SADS-PL interviewers consisted of four child psychiatrists, one doctoral-level and four master’s-level psychologists. All the interviewers underwent extensive training regarding the instrument, the diagnostic classification, and the use of the instrument.
For the inter-rater reliability analysis, each patient was examined by an interviewer-observer pair according to the previous study procedure.24 The interviewer conducted the interview while the observer independently completed a diagnostic coding sheet. In addition, the observer
noted all the interviewer's mistakes or any items on which there was doubt about the participant’s answer. At the end of the interview the observer could re-administer the items in question. During this procedure, the interviewer was not allowed to change any of his/her previous codings. Interviewer-observer pairs were not constant, and all administrators alternated in interviewer and observer roles. Where there were discrepancies between interviewer-observer ratings, they resolved the discrepancies and derived consensus ratings that were used for criterion validity analysis.
Questionnaires
To investigate the construct validity of the K-SADS-PL for DSM-5, we used two questionnaires: the Strengths and Difficulties Questionnaire (SDQ) and the Social Responsiveness Scale-2nd Edition (SRS-2). The SDQ is a measure of 25 items that covers five broad areas: emotional symptoms, hyperactivity/inattention, prosocial behavior, peer problems and conduct problems. Each item of the SDQ is rated on a 3-point Likert scale: not true (0), somewhat true (1), or certainly true (2). The scores for these five areas range between 0 and 10 and represent difficulty for each area. The SRS-2 is a 65-item rating scale that measures deficits in social behavior associated with ASD. Each item of the SRS-2 is rated on a 4-point Likert scale (0–3) that ranges from ‘‘not true’’ to ‘‘almost always true’’. The total score ranges between 0 and 195 and serves as an index of quantitative autistic traits.
Although both questionnaires are available in self-reported and caregiver-rated (or teacher-rated) forms, we used caregiver-rated forms in the present study. Prior to the administration of the interview with the K-SADS-PL for DSM-5, caregivers were asked to complete the Japanese version of the SDQ and SRS-2.25,26
Translation and Cultural Adaptation
To preserve conceptual equivalence with the original version, we carefully followed the principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures.27 After permission from the original developer, the K-SADS-PL for DSM-5 was translated into Japanese independently by two Japanese investigators fluent in English for each module. Together with the first author (TN), the translators compared the two translations of each module and produced a single, reconciled version for them. Then, it was back-translated into English by a professional native English translator fluent in Japanese who was blind to the original sources of the measures before or during back-translation. The resultant back-translations were sent to another professional native English translator (YK) who had expertise in psychiatry to address any translation discrepancies that arise between English and Japanese versions. We repeated these procedures several times until the Japanese versions were approved by the latter native English translator.
Statistical Analysis
To describe the clinical and demographic variables in the samples, numbers and percentages were reported for categorical variables, and means and standard deviations were reported for continuous variables. To examine the inter-rater reliability of the K-SADS-PL for DSM-5, unweighted Fleiss's kappa (κ) was assessed as a measure of chance-corrected agreement.28 To examine the criterion validity of the instrument, sensitivity and specificity as well as unweighted Fleiss’s kappa were used. Both inter-rater reliability and criterion validity were examined without distinguishing subtypes of disorders because the relatively low sample size prohibited the subtype-level analysis.
To assess the construct validity of the instrument, receiver operating characteristics (ROC) analysis was conducted to evaluate the association between the diagnosis according to the instrument and each scale from the questionnaire. The area under the ROC curve (AUC) was calculated by the nonparametric method.29 All these measures were computed with a 95% confidence interval (95% CI).
All analyses were performed using R version 3.4.0 for Windows. The package rel was used for computing Fleiss’s kappa; the package epiR was used for sensitivity and specificity; and the package pROC was used for computing AUC.30