Clinical Performance
To assess the validity of mNGS as a diagnostic for a range of infectious pathogens, we compared the results of the mNGS faecal assay to pathogens identified by standard testing of 338 faecal samples from a cohort of patients in the Mater Hospital System, Brisbane, Australia. The clinical presentation of this cohort was gastrointestinal symptomology indicative of infectious disease.
Standard testing of faecal samples uses a combination of MCS, PCR and antigen testing. This testing was not uniform across the cohort due to technical limitations and individual clinician choice of test. From 338 faecal samples, initial test results indicated that 18 were positive for more than one pathogen, 170 were positive for one pathogen and 150 were negative for all tested pathogens. Four pathogens of clinical interest typically identified at the genus level by standard testing were used for validation testing of the mNGS assay (38), Campylobacter (87/338), Salmonella (50/338), Giardia (28/338) and Aeromonas (15/338)(38). Initially mNGS had modest concordance with standard testing, ranging from 46.7% (Aeromonas) to 75.9% (Campylobacter) sensitivity, requiring discordancy testing. Discordant test results were resolved according to the criteria outlined in the Methods (Supplementary Table 1). The notable difference in Aeromonas spp. testing outcomes was due to this genus not being included in the PCR assay used by the original testing laboratory. Therefore, identification of this pathogen was solely reliant on MCS testing, explaining the low concordance as MCS is known to be less reliable than molecular-based techniques (39, 40). Overall, MCS failed to detect a pathogen where PCR did in 21 of 176 cases. However, in 3 cases, PCR failed to detect a pathogen where MCS did, demonstrating the limitations of current testing (Fig. 3).
After resolution of discordant test results with independent PCR testing, a composite result was made from the standard validated tests to establish pathogen presence in each sample, resulting in 152 positive samples. Using the composite results as ground truth, the sensitivity and specificity for detection of the 4 pathogens with mNGS was determined (Fig. 4). mNGS had a clinically acceptable sensitivity for all 4 pathogens; 89.2% for Salmonella spp., 88.5% for Giardia spp., 89.5% for Campylobacter spp. and 100% for Aeromonas spp. Analogous to PCR testing outcomes (130 of 152), the identification of pathogens was higher with mNGS (137 of 152) than with MCS testing (83 of 152). Notably, mNGS testing resolves targets to the species level unlike MCS or PCR testing. Resolution of the positive Aeromonas samples at the species level revealed that each sample was dominated by a single species: A. caviae (4/15), A. dhakensis (5/15), A. hydrophila (5/15), and A. veronii (1/15). Our results are consistent with previous studies in which these four species are the most pervasive of the 19 known Aeromonas species considered potential pathogens (41).
Campylobacter results were mostly concordant (89.5%), with the majority of discordant results being due to disagreements as to which species should be included as pathogenic. Based on the composite validated testing, five samples were positive for Campylobacter (Supplementary Table 2). However, we scored these samples negative because at the species level they were identified as either C. hominis or C. concisus, which we and others considered to be non-pathogenic until further studies are available, due to their prevalence in healthy individuals (42, 43).
A number of samples were close to the PCR detection threshold (Ct > 35) for Giardia, Salmonella and Campylobacter spp., which presented as discordant results with mNGS. For example, in two cases duplicate PCR tests recorded alternating negative and positive results, indicating the inherent variability in assessing samples at the edge of clinical reporting ranges. Figure 5 demonstrates this trend, with the concordance of PCR test results when reported as units from a maximum of 40 cycles (assumed to be negative) to the number of informative sequencing reads identified in the mNGS assay for each species. Previous studies have concurred with this approach, with discrepant results with Ct values of 35 or higher considered to be of questionable clinical relevance (6) and potentially inflating the discrepancies reported (Supplementary Table 1). If these borderline results are removed, discrepancies between mNGS and standard testing are resolved in 92% of samples.
In total, mNGS testing identified 14 additional potential microbial pathogens in the sample set belonging to the genera, Aeromonas, Campylobacter, Giardia and Salmonella, equating to a diagnostic result for an additional 4.14% of the originally tested sample set.
The evaluation of the selected pathogen panel indicates that mNGS can be applied as a standard testing approach with equal sensitivity and increased taxonomic resolution in comparison to routine pathology testing. While not having enough samples to perform the appropriate statistical tests, preliminary results suggest that the assay could equally be applied to a wider range of targets, including Adenovirus, Cryptoporidium, Pleisomonas, Yersinia and others. For example, Adenovirus could be resolved to subtypes A-H, using mNGS providing clinically useful information as different subtypes have different clinical presentations and treatments e.g. A and F (44) which are associated with either diarrhea or cryptic enteric infection (44). Other subtypes that are not reported by standard testing, such as C and D, are detected by mNGS providing a more complete picture (Fig. 6).
In the rare instances of a confirmed co-infection in the complete data set (12 of 2713 samples), mNGS identified common sets of co-occurring pathogens (Table 2). The most prevalent pathogens in co-infections were Salmonella (6/12), Adenovirus (5/12) and Campylobacter (5/12), which have been reported previously (45, 46).
Table 2
Co-infections within the clinical cohort identified with mNGS testing
Sample Number | Coinfection Type | Organisms Identified |
BBF0525 | Bacteria-Virus | Salmonella bongori, Adenovirus F |
BBF8901 | Eukaryotic-Virus | Giardia intestinalis, Adenovirus F |
BBF8888 | Eukaryotic-Bacteria | Aeromonas dhakensis, Giardia intestinalis |
BBF8889 | Bacteria-Bacteria | Aeromonas caviae, Salmonella enterica |
BBF8902 | Bacteria-Virus | Salmonella enterica, Adenovirus F |
BBF8872 | Bacteria-Bacteria | Aeromonas veronii, Campylobacter jejuni |
BBG0794 | Bacteria-Bacteria | Aeromonas veronii, Campylobacter jejuni |
BBG0805 | Eukaryotic-Bacteria | Salmonella enterica, Giardia intestinalis |
BBG5065 | Eukaryotic-Bacteria | Salmonella enterica, Giardia intestinalis |
BBG5055 | Bacteria-Bacteria | Campylobacter jejuni, Salmonella enterica |
BBF0660 | Bacteria-Virus | Campylobacter jejuni, Adenovirus D |
BBF1911 | Bacteria-Virus | Campylobacter jejuni, Adenovirus D |
Beyond pathogen detection, mNGS can detect many other features of potential clinical interest due to the untargeted nature of the technique in comparison to PCR and the data can be reanalysed for features of interest once the data is in hand. For example, antimicrobial resistance gene presence was recorded in 212 of 388 samples and toxin genes were detected in 110 of 388 samples (Supplementary Table 1) indicating that further clinical value can be developed for mNGS testing with refinement of the assay.
A limitation of mNGS noted in our study was host contamination that can mask microbial detection (47). Human DNA content ranged from 0–5% in the healthy control samples and between 1–95% in the clinical sample set but was sufficiently low in the majority of samples within both populations to be generally appropriate for metagenomic based analysis. Results were unable to be generated from two samples due to high levels of human DNA (up to 99% of total DNA). These samples were from patients with active C. difficile infection which generally had higher human DNA concentrations. Methods to reduce host contamination prior to sequencing are one approach to mitigate this issue (48).
mNGS performance was also assessed using a negative asymptomatic donor population. Of the 200 asymptomatic samples, 98% had no detectable pathogens, using both PCR and mNGS testing. In 3 positive samples, a single pathogenic species was detected in a clinically relevant range, with both mNGS and PCR, namely Campylobacter coli, Aeromonas caviae, and Giardia intestinalis. mNGS also identified a single sample that was positive for Salmonella enterica that was not detected by PCR testing. This species was identified as Salmonella enterica subsp. arizonae, which is determined to be a less common pathogenic species. Analysis of the genetic region used for the PCR test indicated that this serovar had only 93.9% identity to the genomic region targeted by the PCR test, which accounts for the lack of detection in this assay format. These results in a presumptive healthy population were similar to preliminary studies in smaller cohorts, in which Salmonella and Camplyobacter spp. were detected (8) and can be attributed to asymptomatic carriers.
Test turnaround time (TAT) is an important variable for clinicians. The median TAT was 1 week, which is comparable to standard testing. The fastest time to report generation was 48 hours indicating that mNGS could be adopted for critical care.