Mobile phone-based computer-aided chest X-ray detects TB as well as trained radiologists

doi:10.21203/rs.3.rs-5310862/v1

Background: Chest radiography (CXR) is the most common imaging modality used to diagnose pulmonary tuberculosis (PTB). Accurate computer-aided detection (CAD) CXR software may aid physicians in low-resource, high tuberculosis (TB) endemic settings where radiologists are scarce.

Methods: A retrospective pilot study comparing the assessment of humans and CAD was conducted on analogue CXRs from Guinea-Bissau and Ethiopia with the objective of comparing the interpretation of CXRs regarding PTB by CAD (qXR, Qure.ai) with that of two experienced Ethiopian radiologists (A and B). To improve the applicability of this method in low-resource settings, an analysis was performed on images of CXRs taken by mobile phones. Two reference standards were applied: final TB diagnosis by clinical or laboratory findings (i.e., GeneXpert (GX)-confirmed TB).

Results: We included 498 CXRs. Radiologist A identified 50, radiologist B identified 99, and the software identified 81 as indicative of TB. The overall area under the curve (AUC) for the receiver operating characteristic (ROC) curve of the software was 0.84 for GX-confirmed cases. At the prechosen cut-off value of 0.5, the sensitivity of CAD CXR was 76.5%, and the specificity was 85.9%. Radiologist A’s assessments were 64.7% sensitive and 91.9% specific, while radiologist B´s assessments were 76.5% sensitive and 82.3% specific for GX-confirmed cases. The agreement regarding TB-related findings between the radiologists combined (k=0.45) and each radiologist and the software (k=0.56) was moderate.

Conclusion: Our study revealed that CAD CXR performs comparably to experienced radiologists in diagnosing TB even when it is applied to analogue CXRs photographed by mobile phones.

Trial registration: PACTR201611001838365

Tuberculosis

case detection

artificial intelligence (AI)

Chest X-ray (CXR)

computer-aided detection (CAD)

Tuberculosis (TB) remains a major threat to global health with an estimated 10.6 million people experiencing TB in 2022 and 1.3 million deaths due to the disease. Although the World Health Organization (WHO) aims at a 90% reduction in TB incidence by 2035(1), case detection rates are still insufficient to achieve this goal. (2)

With an estimated three million undiagnosed cases in 2021, the need to develop novel strategies and technologies aimed at improving case detection in low-resource, high-incidence settings is urgent.

In most low-resource settings, case-finding is predominantly passive and patients with presumed TB (preTB) must be identified among adults presenting at primary healthcare. Moreover, many TB cases are missed at first contact because TB is not suspected (3–5).

Sputum samples and smear microscopy have been the cornerstone of TB diagnosis in highly endemic areas for decades albeit with substantially varying sensitivities, ranging from 20–50% (6). The advent of PCR-based point-of-care solutions such as Xpert MTB/RIF (Xpert) has increased the sensitivity to 60–70%, however, this approach leaves ample room for improvement. (7, 8)

A recent meta-analysis suggested that chest X-ray (CXR) may substantially improve TB detection rates and called for the evaluation of computer-aided detection (CAD) platforms for TB screening.(9) CXR remains a diagnostic tool in smear-negative patients with presumed TB (preTB) in low-resource areas; however, there is a lack of high-quality studies assessing its diagnostic accuracy as highlighted recently by the WHO (10). Despite uncertain evidence, the WHO conditionally recommends the use of CAD CXRs in the population where TB screening is needed. (11) CAD software based on artificial intelligence (AI) algorithms has been shown to improve the sensitivity of detecting microbiologically confirmed TB from 50–60% (with a specificity of 80%) by experienced radiologists to 66–76%. (12)

Thus, CAD CXR may represent a useful add-on to improve the detection and diagnosis of TB in low-resource areas where experienced radiologists are scarce and microbiological methods fail or are unavailable. However, previous studies have focused on the analysis of digital CXRs, limiting applicability in low-resource settings where CXRs are frequently available only in analogue form.

In 2019, there were 6 conformité européenne (CE)-marked CAD products for TB: CAD4TB, qXR, Inferread dr chest, JViewer-X, Lunit insight CXR and AXIR (13). Previous studies have evaluated the diagnostic accuracy of qXR against human readers using Xpert MTB/RIF (12) as a reference standard. In a study conducted in India, qXR showed moderate specificity and sensitivity for detecting pulmonary TB (12). Another study conducted on CXRs from Nepal and Cameroon showed higher specificity than that of radiologists while maintaining similar sensitivity (14). An evaluation of multiple AI algorithms intended for detecting radiological signs of TB in CXRs using data from patients in a high tuberculosis-burden setting in Bangladesh reported that qXR has an AUC of 0.9081 (15). Similarly, an independent evaluation of 12 different AI algorithms for TB detection conducted in Vietnam revealed an ROC AUC for qXR of approximately 0.82(16)

The present pilot study aimed to assess the diagnostic value of qXR for TB using analogue CXRs photographed by mobile phones or digital cameras. The analysis used CXRs from public health centres in low-resource, high-endemic countries of Ethiopia and Guinea-Bissau and compared the results with assessments by two experienced Ethiopian radiologists. To our knowledge, there is little evidence on the performance of CAD CXRs on analogue images. Moreover, successful mobile phone-based analogue radiographic image analysis could represent an easily implementable tool for screening and reduce diagnostic barriers for people living in low-resource settings. This study therefore provides vital evidence for the implementation of CAD CXRs in low-resource settings with limited access to radiologists and digital CXRs.

Design, study population and TB diagnosis

This retrospective diagnostic pilot study included a set of 498 posterior-anterior (PA) CXR images of patients included in a cluster randomized clinical trial conducted in Ethiopia and Guinea-Bissau from January 2017 until the end of March 2018.(17, 18) (Figure 1 Flow diagram) The patients included were adults seeking help for symptoms indicative of TB at primary health care centers. They were part of a follow-up group seen two weeks after inclusion for TB-presumptive findings where initial sputum analysis was not performed or was negative. The CXR and Xpert analyses were performed within the same week and prior to the clinical assessment and were only carried out for patients who had persistent symptoms at the two-week follow-up.

Clinical TB was diagnosed by experienced physicians based on clinical signs and symptoms, CXR findings and a trial of antibiotics (17). This diagnostic standard was included because it is a common method for the diagnosis of TB in low-resource settings. (19) The clinicians assessing the patients were blinded to the radiologists’ assessments, as they were performed after the study conclusion. All PTB patients were assessed for treatment outcome, and all included participants were visited one year after inclusion to assess survival.

The CAD CXR

The software qXR was developed by qure.ai (Mumbai, India) as an artificial intelligence (AI)-based CAD for analysing posterior anterior (PA) CXRs for abnormalities.(16) The AI algorithm was trained using a multihospital training dataset of 2.3 million chest X-rays and their corresponding radiology reports to identify abnormal X-rays and specific abnormalities.

qXR assigns a score between 0 and 1 to indicate the presence of a certain finding. It also outputs a pixel map indicating the location of the target abnormality (see Appendix). This study used a threshold of 0.5, which is considered the default recommended for most settings. (20)

In the present study qXR version 3.2 was used to analyse the CXRs.

The radiologists

Both radiologists have longstanding experience in analysing CXRs for TB in a high incidence setting.

Radiologist A is an assistant professor of radiology at the College of Medical and Health Sciences, University of Gondar (CMHS UOG), and obtained his medical degree at CMHS UOG and his radiology specialty certificate at Addis Ababa University (AAU). In addition, he has served in an academic position and as a consultant radiologist at Gondar University Hospital (GUH). He has served as the head of the Department of Radiology at GUH for two years. He has 10 years of experience in the field of radiology.

Radiologist B is an associate professor of radiology at the Department of Radiology at CMHS UOG. He has worked as a consultant radiologist at the GUH since 2010. He obtained his medical degree and specialty certificate in radiology at AAU. He currently serves as head of department at the Department of Radiology at CMHS UOG. He has 14 years of experience in the field of radiology.

In the analysis we also analysed the pooled radiologists´ assessments against CAD CXRs.

The radiologists and AI crew were blinded to all clinical information and microbiological test results.

Assessment of chest radiographs

The PA CXRs were photographed using commonly available digital or mobile phone cameras (Sony Cyber-shot (20.1 MP), Samsung Galaxy A3 (13.0 MP), Canon EOS 70D (20.2 MP); resolution varying from 72 - 350 dpi) that placed the CXRs on a light box in daylight using a guide provided by qXR (Appendix), anonymized, numbered in a random order and sent digitally to qure.ai and the radiologists.

Classification of findings

The findings assessed by CAD CXR and the radiologists are presented in Table 1.

CXRs were classified as abnormal if one of the following findings was identified: consolidation, cavitation, fibrosis, nodule, blunted cardiopulmonary angle, pleural effusion, hilar lymphadenopathy, or tracheal shift.

Treatment outcome

According to the WHO, failure was defined as death during treatment, loss to follow-up and/or treatment failure (i.e., a patient whose sputum smear or culture was positive at month 5 or later during treatment), while treatment completion and cure were defined as successful.(21)

Analysis

All data analyses were performed using Stata version 11 (Stata Corporation, College Station, Texas, USA). Kappa coefficients, and associated 95% confidence intervals (CI), were used to investigate the interrater reliability of the radiologists and AI. The following scale was used for the interpretation of kappa coefficients: <0, poor; 0–0.20, slight; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.81–1.00, almost perfect (22). To assess the overall accuracy of the AI, ROC analysis was used to determine the AUC using the “roctab” command in Stata 11, which provides a nonparametric estimation of the ROC curve and Bamber and Hanley confidence intervals for the AUC of the ROC curve. The sensitivity and specificity were calculated.

The same accuracy measures were also used to compare the radiologist’s assessment to that of the CAD CXR and were plotted together with the ROC of CAD CXR .

The sensitivity, specificity, positive predictive value and negative predictive value were calculated, and microbiological confirmation (by GeneXpert Mtb/Rif (GX)) and/or clinical diagnosis were used as the reference standards.

The patients contributing the 498 CXRs to the analysis had a mean age of 38 years (IQR 23 – 50), 259 (52%) were female, and 28 (6%) were HIV-infected. Fifty-four (11%) reported having had TB previously. Compared to the proportion of smear-positive patients in the original study, the percentage of female patients was greater (56% vs 31%), and the cough length was shorter (20 days vs 33 days). (17) A clinical diagnosis of tuberculosis was made in 57 (11%) patients (Table 2),16 (28%) of whom were confirmed to have a positive GX. (Table 3)

The percentage of females was lower among patients diagnosed with TB (40%) than among patients not diagnosed with TB (54%) (p=0.045). TBscore was greater among patients diagnosed with TB (5.9, IQR 4 – 7) than among patients not diagnosed with TB (4.2, IQR 3 – 6) (p<0.001). (Table 2)

Treatment outcome and loss to follow-up

Of all patients diagnosed with TB, 32 (55%) had successful treatment outcomes, with no significant difference between clinically diagnosed (58%) and GX-confirmed patients (47%) (p=0.377) (Table 3)

Survival data were available one year after inclusion for 294 (59%) of the 498 included patients. Survival rates did not differ between patients diagnosed with TB (50%) and those not diagnosed with TB (59%) (p=0.411) (Table 2). Furthermore, the one-year survival rates did not differ significantly among those who were diagnosed clinically (17/41, 42%) and GX-confirmed patients (12/16, 71%) (p=0.112) (Table 3).

qXR interpretation – GX or clinically confirmed cases

The ROC curves computed from the scores assigned by the CAD CXR readings are presented in Figures 1-3. The AUC of CAD CXR was 0.78 (95%CI 0.78 – 0.85) for overall TB diagnosis, 0.84 (95%CI 0.73 – 0.96) for GX confirmed cases and 0.74 (95%CI 0.65 – 0.82) for clinically diagnosed patients.

At the prespecified cut-off value of 0.5, CAD CXR sensitivity ranged from 46.3% for clinically diagnosed patients to 76.5% for GX-confirmed patients, while the specificity varied between 85.9% for GX-confirmed and 88.9% for GX or clinically diagnosed PTB patients (Table 4).

Comparison of qXR and human readers

Figures 2 -4 display the operating points for radiologists A and B together with the CAD CXR ROC for GXconfirmed, clinically diagnosed, and overall TB, respectively. Overall, the pooled radiologists’ assessments exhibited greater sensitivity than did the CAD CXR assessment and ranged from 56.1% (81.2% specificity) among clinically diagnosed to 82.4% (80.2% specificity) among GX-confirmed patients. Assessing each radiologist against the CAD CXR showed that radiologist A performed with less sensitivity but higher specificity, while radiologist B was as sensitive as the CAD CXR but performed with lower specificity. (Table 4)

The radiologists’ agreement varied from slight (0.15 (95% CI 0.04 – 0.26)) for cardiomegaly to substantial (0.64 (95% CI 0.45 – 0.82)) for tracheal shift and was moderate (0.45 (95% CI 0.35 – 0.55) for the overall identification of TB. (Table 1)

Overall, the agreement between the radiologists´ pooled assessment and the CAD CXR regarding TB diagnosis based on CXR findings was moderate (0.56 (95% CI 0.46 – 0.65).

The different prespecified findings (Table 1) between the radiologists and CAD CXR ranged from fair (0.24 (95% CI 0.06 – 0.42)) for hilar lymphadenopathy to substantial (0.61 (95% CI 0.48 – 0.74) and 0.61 (95% CI 0.45 – 0.77)) for nodule and tracheal shift, respectively.

This pilot study indicates that CAD CXR software applied to analogue CXRs photographed by mobile phones or digital cameras may be as accurate in identifying TB as trained radiologists working in a high-TB-endemic setting.

Previous assessments of CAD software have been based on digital CXRs and to the authors’ knowledge the present study is the first to assess CAD applied to analogue CXRs photographed by mobile phones or digital cameras. These findings thus have substantial implications for settings where TB is highly endemic and CXRs are still predominant analogue. (13)

The use of CXR as a diagnostic tool for PTB has limitations. The sensitivity in previous studies ranged from 72–99% for any finding and 73–94% for findings suggestive of TB, while the specificity varied from 73–94% and 87–96%, respectively. (23, 24) In the present study, both the radiologists and the CAD software were slightly less accurate at the prespecified cut-off value. This may most likely be because our study mainly included CXRs from smear-negative patients (i.e., their CXRs may show patterns that are less specific to TB). Previous studies assessing the agreement between radiologists and CAD software found it to be fair (25), while we found it to be moderate between the radiologists and the radiologists and CAD software.

The operating points (i.e., sensitivity and 1-specificity) for both radiologists assessed here were very close to the ROC curve of the CAD software. This finding has been shown in previous studies assessing the qXR (12, 14, 26) and has resulted in the WHO recommending the use of CAD software in screening among individuals older than 15 years (11).

The present study included only 498 individual CXRs, 57 (11%) of which were diagnosed with TB and 16 (28%) of which were microbiologically confirmed by Xpert. This limited data is due to our use of CXRs from a previous case-detection study (17) and our focus on analogue CXRs. The CXRs were taken during the diagnostic cascade of the study (18) and thus belonged to individuals who were either smear negative or where sputum smear analysis was not considered necessary upon initial evaluation. All individuals were symptomatic two weeks after the first contact with the health care provider where they had received antibiotic treatment. Thus, these patients represent an important group of patients, who are often overlooked in the cascade of care making up part of the considerable group of missed TB cases in highly endemic settings.

The clinical diagnosis was based on signs and symptoms after a trial of antibiotics. The assessment also included the CXR that was read by the physician assessing the patient. Thus, CXR is both part of the reference and evaluation. The idea for the present pilot study came after the conclusion of the cluster randomized trial, which included these patients. Since TB in many parts of the world is still a clinical diagnosis that may or may not be partly based on CXRs read by the attending physician, we included it in our study to reflect current realities. The radiologist and CAD-team did not receive information on signs and symptoms and thus based their assessments solely on the CXRs.

This study did not confirm the diagnosis of TB by mycobacterial culture. This is due to the lack of access to culture in our settings. All diagnosed patients were followed up after treatment, and treatment success rates did not differ between those who were diagnosed with TB and those who were not, or between those who were clinically diagnosed and those confirmed by GX. One-year follow-up revealed no excess mortality among the diagnostic groups or between TB patients and non-TB patients; however, the loss to follow-up rate was high, especially one year after inclusion. Thus, our data may not be sufficient to conclude that both groups had equally low mortality.

As apparent from the CXRs shown in the appendix, the films were marked with text and numbers. qXR includes a preprocessing module that zooms in on the relevant portion of the CXR. This removes postprocessing artefacts outside of the lung parenchyma, meaning that writing on the CXRs did not affect the analysis.

Although more recent recommendations suggest using more costly diagnostic methods such as Xpert up front (11), this strategy may still miss an important group of patients for whom CXR may be a good diagnostic option such as those unable to produce adequate sputum. AI-guided CXR interpretation via mobile phones may guide health care workers with no or limited training in assessing CXRs for possible TB.

This pilot study showed that CAD CXR has diagnostic value for TB comparable to that of CXR interpreted by trained radiologists, even when applied to analogue CXRs photographed by mobile phones or digital cameras. Thus, the use of CAD of PTB has the potential to lessen the currently high case detection gap, particularly in low-resource settings where radiologists are scarce and digital CXRs are often unavailable. This should have implications for the implementation of CAD CXR as well as for the design of future diagnostic algorithms.

AAU: Addis Ababa University

AUC: area under the curve

CAD: computer-aided detection

CE: conformité européenne

CMHS UOG: College of Medical and Health Sciences, University of Gondar

CXR: chest radiography

GUH: Gondar University Hospital

GX: GeneXpert

HIV: human immunodeficiency virus

PA: posterior-anterior

preTB: presumed TB

PTB: pulmonary tuberculosis

ROC: receivers operating curve

TB: tuberculosis

WHO: World Health Organization

All patients provided written informed consent to participate in the study and were offered HIV-testing with pre- and post-test counseling. HIV-infected individuals were accompanied to nearby Anti Retroviral Treatment centers.

Approval was obtained from the Guinean Ministry of Health and the Ethics review board of the University of Gondar. Consultative approval was obtained from the Central Ethical Committee in Denmark.

The authors state no competing interests.

Data are available upon reasonable request to the corresponding author.

Conflict of interest

Qure.AI has not taken part in planning, design or analyses in connection with the present study. The analyses of the CXRs by the AI (qXR) was done free of charge. DR, SS and MT are employees of Qure.ai. The authors declare that they have no further conflicts of interest.

Funding

was received from: the Novo Nordisk Foundation, Scandinavian Society for Antimicrobial Chemotherapy Foundation, Gondar University and the Swedish research council.

Author Contribution

All authors contributed to the research and the manuscript. FR, TS and CW conceptualized the study. TT and ZD analyzed xrays, HF, SB and AS evaluated the patients clinically and ASB supervised the study locally. SS, DR and MT collaborated at qure.ai.

Acknowledgement

We thank the patients participating in the study and the staff at the health centers and local assistants for their consistent work.

Data Availability

Data is available upon reasonable request to the corresponding author.

WHO. GLOBAL TUBERCULOSIS REPORT 2023. 2023.
Harries AD, Kumar AMV. Challenges and Progress with Diagnosing Pulmonary Tuberculosis in Low- and Middle-Income Countries. Diagnostics (Basel). 2018;8(4).
Bates M, O'Grady J, Mwaba P, Chilukutu L, Mzyece J, Cheelo B, et al. Evaluation of the burden of unsuspected pulmonary tuberculosis and co-morbidity with non-communicable diseases in sputum producing adult inpatients. PLoS One. 2012;7(7):e40774.
Porskrog A, Bjerregaard-Andersen M, Oliveira I, Joaquim LC, Camara C, Andersen PL, et al. Enhanced tuberculosis identification through 1-month follow-up of smear-negative tuberculosis suspects. Int J Tuberc Lung Dis. 2011;15(4):459-64.
Yimer S, Holm-Hansen C, Yimaldu T, Bjune G. Health care seeking among pulmonary tuberculosis suspects and patients in rural Ethiopia: a community-based study. BMC Public Health. 2009;9:454.
Theron G, Zijenah L, Chanda D, Clowes P, Rachow A, Lesosky M, et al. Feasibility, accuracy, and clinical effect of point-of-care Xpert MTB/RIF testing for tuberculosis in primary-care settings in Africa: a multicentre, randomised, controlled trial. Lancet. 2014;383(9915):424-35.
Chakravorty S, Simmons AM, Rowneki M, Parmar H, Cao Y, Ryan J, et al. The New Xpert MTB/RIF Ultra: Improving Detection of Mycobacterium tuberculosis and Resistance to Rifampin in an Assay Suitable for Point-of-Care Testing. mBio. 2017;8(4).
Esmail A, Tomasicchio M, Meldau R, Makambwa E, Dheda K. Comparison of Xpert MTB/RIF (G4) and Xpert Ultra, including trace readouts, for the diagnosis of pulmonary tuberculosis in a TB and HIV endemic setting. Int J Infect Dis. 2020;95:246-52.
Bohlbro AS, Hvingelby VS, Rudolf F, Wejse C, Patsche CB. Active case-finding of tuberculosis in general populations and at-risk groups: a systematic review and meta-analysis. Eur Respir J. 2021;58(4).
WHO. Chest radiography in tuberculosis detection: summary of current WHO recommendations and guidance on programmatic approaches. 2016. p. WHO/HTM/TB/2016.20.
WHO. WHO consolidated guidelines on tuberculosis. Module 2: screening – systematic screening for tuberculosis disease. 2021.
Nash M, Kadavigere R, Andrade J, Sukumar CA, Chawla K, Shenoy VP, et al. Deep learning, computer-aided radiography reading for tuberculosis: a diagnostic accuracy study from a tertiary hospital in India. Sci Rep. 2020;10(1):210.
F. Digital Chest Radiography and Computer-Aided Detection (CAD) solutions for Tuberculosis diagnosis - Technology Landscape Analysis. 2021.
Qin ZZ, Sander MS, Rai B, Titahong CN, Sudrungrot S, Laah SN, et al. Using artificial intelligence to read chest radiographs for tuberculosis detection: A multi-site evaluation of the diagnostic accuracy of three deep learning systems. Sci Rep. 2019;9(1):15000.
Qin ZZ, Ahmed S, Sarker MS, Paul K, Adel ASS, Naheyan T, et al. Tuberculosis detection from chest x-rays for triaging in a high tuberculosis-burden setting: an evaluation of five artificial intelligence algorithms. Lancet Digit Health. 2021;3(9):e543-e54.
Putha P, Tadepalli M, Reddy B, Raj T, Chiramal JA, Govil S, et al. Can artificial intelligence reliably report chest x-rays?: Radiologist validation of an algorithm trained on 2.3 million x-rays. arXiv preprint arXiv:180707455. 2018.
Rudolf F, Abate E, Moges B, Mendes AM, Mengistu MY, Sifna A, et al. Increasing smear positive tuberculosis detection using a clinical score - A stepped wedge multicenter trial from Africa. Int J Infect Dis. 2021;113 Suppl 1:S55-S62.
Rudolf FA, E.; Moges, B.; Gomes, V.F.; Mendes, A.M.; Sifna, A.; Fekadu, H.; Bizuneh, S.; Wejse, C.; Schön, T. A structured two week follow up visit in the cascade of care for tuberculosis increases case detection. Int J Tuberc Lung Dis. 2023(In press).
Divala TH, Corbett EL, Kandulu C, Moyo B, MacPherson P, Nliwasa M, et al. Trial-of-antibiotics to assist tuberculosis diagnosis in symptomatic adults in Malawi (ACT-TB study): a randomised controlled trial. Lancet Glob Health. 2023;11(4):e556-e65.
Khan FA, Majidulla A, Tavaziva G, Nazish A, Abidi SK, Benedetti A, et al. Chest x-ray analysis with deep learning-based software as a triage test for pulmonary tuberculosis: a prospective study of diagnostic accuracy for culture-confirmed disease. The Lancet Digital Health. 2020;2(11):e573-e81.
WHO. Definitions and reporting framework for tuberculosis – 2013 revision (updated December 2014 and January 2020). 2020.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-74.
Assefa Y, Woldeyohannes S, Gelaw YA, Hamada Y, Getahun H. Screening tools to exclude active pulmonary TB in high TB burden countries: systematic review and meta-analysis. Int J Tuberc Lung Dis. 2019;23(6):728-34.
Dhana A, Hamada Y, Kengne AP, Kerkhoff AD, Rangaka MX, Kredo T, et al. Tuberculosis screening among ambulatory people living with HIV: a systematic review and individual participant data meta-analysis. Lancet Infect Dis. 2022;22(4):507-18.
Koesoemadinata RC, Kranzer K, Livia R, Susilawati N, Annisa J, Soetedjo NNM, et al. Computer-assisted chest radiography reading for tuberculosis screening in people living with diabetes mellitus. Int J Tuberc Lung Dis. 2018;22(9):1088-94.
Codlin AJ, Dao TP, Vo LNQ, Forse RJ, Van Truong V, Dang HM, et al. Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis. Sci Rep. 2021;11(1):23895.

Table 1 Findings on PA CXR, by interpreter

Finding, n (%)	Radiologist A	Radiologist B	Radiologists combined	qXR	p-value (Radiologists combined vs qXR)	Kappa statistic Radiologists (95%CI)	Level of agreement Radiologists	Kappa statistic radiologists combined vs qXR (95% CI)	Level of agreement Radiologists combined vs qXR
Consolidation	28 (6)	32 (6)	43 (9)	68 (14)	<0∙001	0.54 (0.38 – 0.70)	Moderate	0.57 (0.45 – 0.68)	Moderate
Cavitation	5 (1)	21 (4)	22 (4)	7 (1)	<0∙001	0.30 (0.07 – 0.53)	Fair	0.33 (0.11 – 0.55)	Fair
Fibrosis	45 (9)	66 (13)	74 (15)	39 (8)	<0∙001	0.63 (0.52 – 0.74)	Substantial	0.60 (0.49 – 0.71)	Moderate
Nodule	23 (5)	21 (4)	35 (7)	49 (10)	<0∙001	0.38 (0.19 – 0.57)	Fair	0.61 (0.48 – 0.74)	Substantial
Blunted Costophrenic angle	19 (4)	11 (2)	21 (4)	10 (2)	<0∙001	0.59 (0.38 – 0.80)	Moderate	0.44 (0.22 – 0.66)	Moderate
Pleural effusion	14 (3)	8 (2)	16 (3)	12 (2)	<0∙001	0.54 (0.28 – 0.79)	Moderate	0.49 (0.26 – 0.72)	Moderate
Pneumo- thorax	0	0	0	3 (0.6)	n/a	-	-	-	-
Hilar lymphadeno- pathy	15 (3)	24 (5)	32 (6)	7 (1)	<0∙001	0.33 (0.14 – 0.53)	Fair	0.24 (0.06 – 0.42)	Fair
Tracheal shift	15 (3)	22 (4)	25 (5)	29 (6)	<0∙001	0.64 (0.45 – 0.82)	Substantial	0.61 (0.45 – 0.77)	Substantial
Cardiomegaly	6 (1)	64 (13)	64 (13)	15 (3)	<0∙001	0.15 (0.04 – 0.26)	Slight	0.27 (0.14 – 0.40)	Fair
Scoliosis	1 (0.2)	2 (0.4)	2 (0.4)	7 (1)	<0∙001	0.67 (0.05 – 1.00)	Substantial	0.44 (0.04 – 0.85)	Moderate
Abnormal classified*	89 (18)	73 (15)	159 (32)	161 (32)	<0∙001	- 0.15 (-0.02 - -0.09)	Poor	0.54 (0.46 – 0.62)	Moderate
Abnormal reclassified^†	107 (21)	166 (33)	185 (37)	132 (27)	<0∙001	0.52 (0.44 – 0.60)	Moderate	0.56 (0.48 – 0.63)	Moderate
TB^‡	50 (10)	99 (20)	123 (25)	110 (22)	<0∙001	0.45 (0.35 – 0.55)	Moderate	0.56 (0.46 – 0.65)	Moderate
	*as judged by interpreter ^†classified at analysis if one of the above found ^‡as judged by interpreter

Table 2 Baseline characteristics, by TB diagnosis

	All n=498	TB-diagnosis n=58 (12%)	No TB-diagnosis n=440 (88%)	p-value (TB vs Not TB)
Female, n (%)	259 (52)	23 (40)	236 (54)	0.045
Mean age (IQR)	38 (23 – 50)	43 (25 – 62)	37 (22 – 50)	0.005
Mean TBscore (IQR)	4∙4 (3 – 6)	5.9 (4 – 7)	4.2 (3 – 6)	<0.001
Previous TB, n (%)*	54 (11%)	12 (21)	42 (10)	0.064
HIV-infected, n (%)	28 (6)	3 (5)	25 (6)	0.874
Alive 1 year after inclusion^†	289 (58)	29 (50)	260 (59)	0.411
Missing: *18 (4%), ^†204 (41%)

Tabel 3 Characteristics for TB diagnosed, by diagnosis

	All n=58	Clinical n=41 (72%)	GenXpert n=17 (29%)	p-value (GenXpert vs clinical)
Female, n (%)	23 (40)	18 (44)	5 (29)	0.081
Mean age (IQR)	43 (25 – 62)	45 (35 – 63)	34 (25 – 40)	0.996
Mean TBscore (IQR)	5.8 (4 – 7)	6.1 (5 – 7)	5.4 (3 – 8)	0.855
Previous TB, n (%)*	12 (21)	8 (20)	4 (24)	0.265
HIV- infected, n (%)	3 (5)	2 (5)	1 (6)	0.875
Successful outcome of TB treatment^†	32 (55)	24 (58)	8 (47)	0.377
Alive 1 year after inclusion, TB patients^‡	29 (50)	17 (42)	12 (71)	0.112
Missing: *1, ^†21, ^‡27

Table 4 Interpretation of Radiologist A, Radiologist B, Radiologists combined and qXR (at prespecified cut off). (GX confirmed)

	Cut-off score	TP	FP	FN	TN	Sensitivity	Specificity
All cases (GX or clinical)
Radiologist A	N/A	24	26	34	414	41∙4 (28∙6 - 55∙1)	94∙1 (91∙5 - 96∙1)
Radiologist B	N/A	32	66	26	374	55∙2 (41∙5 - 68∙3)	85∙0 (81∙3 - 88∙2)
Radiologists combined	N/A	37	72	21	368	63∙8 (50∙1 - 76∙0)	83∙6 (79∙8 - 87∙0)
qXR	≥ 0.5	32	49	26	391	55∙2 (41∙5 - 68∙3)	88∙9 (85∙5 - 91∙6)
GX confirmed
Radiologist A	N/A	11	39	6	442	64∙7 (38∙3 - 85∙8)	91∙9 (89∙1 - 94∙2)
Radiologist B	N/A	13	85	4	396	76∙5 (50∙1 – 93∙2)	82∙3 (78∙6 – 85∙6)
Radiologists combined	N/A	14	95	3	386	82∙4 (56∙6 - 96∙2)	80∙2 (76∙4 - 83∙7)
qXR	≥ 0.5	13	68	4	413	76∙5 (50∙1 – 93∙2)	85∙9 (82∙4 – 88∙9)
Clinical
Radiologist A	N/A	13	37	28	420	31∙7 (18∙1 - 48∙1)	91∙9 (89∙0 - 94∙2)
Radiologist B	N/A	19	79	22	378	46∙3 (30∙7 - 62∙6)	82∙5 (78∙9 - 86∙1)
Radiologists combined	N/A	23	86	18	371	56∙1 (39∙7 - 71∙5)	81∙2 (77∙3 - 84∙7)
qXR	≥ 0.5	19	62	22	395	46∙3 (30∙7 – 62∙6)	86∙4 (82∙9 - 89∙4)

No competing interests reported.

Mobile phone-based computer-aided chest X-ray detects TB as well as trained radiologists

Status:

Version 1

Abstract

Figures

Background

Methods

Results

Discussion

Conclusion

Abbreviations

Declarations