AI-driven convolutional neural networks for accurate identification of yellow fever vectors

doi:10.21203/rs.3.rs-4473317/v1

Download PDF

Research Article

AI-driven convolutional neural networks for accurate identification of yellow fever vectors

https://doi.org/10.21203/rs.3.rs-4473317/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 02 Aug, 2024

Read the published version in Parasites & Vectors →

You are reading this latest preprint version

Background

Identifying mosquito vectors is crucial for controlling diseases. Automated identification studies using Convolutional Neural Network (CNN) have been conducted for some urban mosquito vectors but not yet for sylvatic mosquito vectors that transmit the yellow fever and other arboviruses. Here, we evaluated the ability of the AlexNet CNN to identify four mosquito species: Aedes serratus, Aedes scapularis, Haemagogus leucocelaenus and Sabethes albiprivus and whether there is variation in AlexNet's ability to classify mosquitoes based on pictures of four different body regions.

Methods

The specimens were photographed using a cell phone connected to a stereoscope. Photographs were taken of the full-body, pronotum and lateral view of the thorax, which were pre-processed to train the AlexNet algorithm. The evaluation was based on the confusion matrix, the accuracy (10 pseudoreplicates) and the confidence interval for each experiment.

Results

Our study found that the AlexNet can accurately identify mosquito pictures of the genus Aedes, Sabethes and Haemagogus with over 90% accuracy. Furthermore, the algorithm performance did not change according to the body regions submitted. It is worth noting that the state of preservation of the mosquitoes, which were often damaged, may have affected the network's ability to differentiate between these species and thus accuracy rates could have been even higher.

Conclusions

Our results support the idea of applying CNNs for AI-driven identification of mosquito vectors of tropical diseases. This approach can potentially improve the surveillance of yellow fever vectors by health services and the population as well.

Deep Learning

Machine learning

Culicidae

Mosquito-borne diseases are a major public health concern. More than half of the world's population is exposed to arboviruses such as yellow fever, dengue, and Zika. Currently, 47 countries, including 34 in Africa and 13 in Central and South America, are endemic or have endemic regions for yellow fever [1, 2]. These pathogens are mainly transmitted by mosquito bites, with Aedes aegypti and Aedes albopictus species being the most important urban vectors of these arboviruses [3, 4]. Sylvatic mosquito species from the Sabethes and Haemagogus genera have been found to be naturally infected with the yellow fever virus and other arboviruses, for which no specific therapeutic agents exist [5–11].

One approach to prevent the spread of these diseases is by controlling the spread of the virus’ vector. Traditional mosquito surveillance relies on catches and species identification, which require regular manual inspection and dedicated personnel, making large-scale monitoring difficult and expensive. Additionally, identifying mosquitoes is a difficult task that demands specialized knowledge due to the vast range of morphological characteristics found in the Culicidae family, which includes all mosquitoes[12]. New approaches that rely on smartphones and the internet can enable new community and digital observatories with the task of species identification. These observatories allow individuals to submit photos of mosquitoes they come across. However, manually inspecting each picture is not a feasible long-term solution due to the large volume of pictures and professionals needed for the task [13–15].

Deep learning methods based on Convolutional Neural Networks (CNNs) have shown promise for mosquito identification [13, 14, 16–19]. CNNs simulate the human learning process for classifying pictures and extract important features from data automated without the need of human supervision[20]. AlexNet is a CNN that was pre-trained on 1.2 million images of objects, animals, and plants available in the ImageNet database. AlexNet has been successfully used for the identification of insect vectors [21]. Automated identification studies have been conducted for some urban mosquito vectors [13, 14, 16–19], but not yet for sylvatic mosquito vectors that transmit the yellow fever and other arboviruses [6, 11]. Therefore, this study aimed to test the AlexNet’s ability in identifying Aedes serratus, Aedes scapularis, Haemagogus leucocelaenus, and Sabethes albiprivus. These four mosquito species were chosen because they are either confirmed vectors of the yellow fever virus (Hg. leucocelaenus and Sa. albiprivus) or suspected to be vectors of the virus (Ae. serratus and Ae. scapularis). We also asked whether there is variation in AlexNet's ability to classify mosquitoes based on pictures of four different body regions. Our findings indicate that the AlexNet network can accurately identify yellow fever vectors with over 90% accuracy for the four body regions analyzed.

To build the picture database, four mosquito species were selected. Among them, Sa. albiprivus (n = 100) and Ha. leucocelaenus (n = 98) were vectors of the wild yellow fever virus, while Aedes serratus (n = 100) and Aedes scapularis (n = 100) were vectors of other arboviruses. A total of 565 full-body pictures were captured, including 294 of the lateral thorax and 484 of the pronotum, resulting in 1,343 pictures available at the Laboratory of Medical Parasitology and Vector Biology at the University of Brasilia. The Culicidae specimens were mounted on a cardboard triangle using a pin. The mosquitoes were attached to the triangle by the thoracic pleura, leaving the legs facing the pin and the upper pleura free for observation. Pictures were captured using a cell phone camera (Samsung Galaxy S9) attached to a stereoscope (Zeiss Stemi 508). Photographs of the whole body, pronotum, and lateral view of the thorax were taken (Fig. 1), cropped into a square format, selecting only the mosquito, and resized to 227 x 227 pixels. They were then organized into folders based on the species and experiment to be conducted.

The analyses were conducted using the MATLAB computer software and AlexNet machine learning algorithm which has been successfully used for automated identification of insect vectors [16, 22]. The picture bank was divided into three parts: 80% for training, 10% for internal validation, and 10% for testing (Additional file 2: Database S1). The algorithm randomly selected the pictures for each stage. Four experiments were conducted with four classes, and ten pseudoreplicates were made for each. The first experiment used all the pictures, including total body, pronotum, and lateral thorax. The experiments used distinct pictures of each body part. The evaluation was based on the confusion matrix/heatmap and the mean and confidence interval of the accuracy for each experiment using the ‘Hmisc’ package in the computational software R 4.7.1, together with the RStudio 2023.03.1.446 interface [23–25].

Algorithm Performance

The performance of the algorithm was evaluated by processing test pictures after training the AlexNet algorithm. Four experiments were conducted, in which each group (full-body, lateral thorax, and pronotum) was submitted to the AlexNet algorithm. Additionally, all 1,343 photos corresponding to the three groups were submitted together. The average confidence interval for accuracy was then calculated. The AlexNet algorithm demonstrated the highest accuracy in identifying mosquitoes when all pictures were analyzed together, providing the algorithm with the most comprehensive information about the mosquito's body. Each run was repeated 10 times, achieving an average accuracy of 94% (95% CI, 90–97). The run with the total body group achieved an average accuracy of 92.3% (95% CI, 83–97), while with the lateral view group the value was 93.7% (95% CI, 79–98). Finally, with the pronotum group, the average accuracy was 93.8% (95% CI, 83–98). The smallest confidence intervals were observed with pictures of all parts of the body (Fig. 2; see Additional file 1: Table S1).

The algorithm's best performance was observed when photos from all three groups, were available. We then evaluated the algorithm's performance for each species based on these three groups of photos. The performance of the AlexNet algorithm in identifying mosquitoes of the genus Aedes was poor, with the highest confidence intervals observed for the two species in this genus, compared to the others (95% CI, 72–98) (Fig. 3). The algorithm misidentified at least one picture of Ae. scapularis as Ae. serratus or vice versa when presented with pictures of all three groups. In the run where all 1,343 pictures were available, Ae. serratus was misidentified as Hg. leucocelaenus. In the run with pictures of the lateral view, only mosquitoes of the genus Aedes were misidentified, with the two Aedes species being swapped. In the run with the pronotum, several misidentifications occurred. Ae. serratus was incorrectly identified as either Sa. albiprivus or Hg. leucocelaenus, while Ae. scapularis was identified as Sa. albiprivus (Fig. 4). It is worth noting that many of the photographed specimens were not well-preserved, which may have contributed to the high rate of misidentification.

In this study, we aimed to identify four mosquito species that transmit yellow fever or other arboviruses by using a CNN (AlexNet). We also wanted to investigate whether the algorithm performance in classifying the mosquitoes changes according to the body regions shown on the pictures submitted. Our study found that the AlexNet can accurately identify mosquito pictures of the genus Aedes, Sabethes and Haemagogus with over 90% accuracy. Furthermore, the algorithm performance did not change according to the body regions shown.

Lorenz et al.[26] classified mosquitoes based on morphometric characteristics of their wings using neural networks, achieving accuracies ranging from 86–100%. However, an identification system based only on a body structure such as the wing is more fragile because if the structure is not present in the analyzed photo, the identification is compromised. Therefore, a good identification system should work with any part of the insect's body. Sauer et al.[27] showed that best-performing CNN configuration achieved a precision of 99% to discriminate between Aedes and non-Aedes mosquito species; the mean precision to predict the Aedes species was 91% for RGB pictures. Motta et al.[16] used three pre-trained networks to identify urban mosquitoes (Aedes and Culex) achieving an accuracy of 76.2% for the GoogleNet, 52.4% for LeNet, and 51.2% for AlexNet. Okayasu et al.[28] showed better accuracy results (92.3%) with the identification of Ae. albopictus, Anopheles stefensi, and Cx. pipiens pallens using AlexNet based on data augmentation and 12,000 training pictures. More recently, Motta et al[17] optimized the CNN hyperparameters and obtained 97.3% accuracy in distinguishing between the mosquitoes of the genus Aedes and the Culex mosquitoes. Similarly, Goodwin et al.[29] and Park et al.[13] achieved 97% accuracy rates for mosquito identification (Anopheles, Culex, Psorophora, and Aedes species) using deep learning neural networks. These networks rely on morphological features like those used by taxonomists [13]. Kittichai et al[15] using two YOLO v3 model identified Ae. aegypti, Ae. albopictus, and Cx. quinquefasciatus at a mean average accuracy of 98–100%. A recent study has shown that the accuracy and robustness of the CNN may reach 99% accuracy by incorporating spatial dropout layers to regularize the network and by modifying its structure to incorporate multi-view inputs [30]. Concatenating two YOLO v3 model exhibited the optimal performance in identifying mosquitoes, with mean average accuracy of 99%. The Swin MSI successfully identified 100% subspecieslevel in Culex pipiens Complex. Based on pictures of all body regions, AlexNet identified Ae. scapularis, Ae. serratus, Ha. leucocelaenus, and Sa.albiprivus at 94% accuracy on average (Fig. 2). Compared to previous studies that have used neural networks for mosquito identification, our accuracy rate of 94% is aligned with most results obtained by others.

In our study, we did not find a significant difference in AlexNet performance in identifying mosquitoes based on different body regions. In fact, other studies have shown that CNNs are able to detect morphological differences in various body regions of Aedes mosquitoes[13], some of which are consistent with the most used dichotomous keys [31]. Such results reveal that deep learning models learn the distinctive morphological features of mosquitoes body areas; these are the same ones used by taxonomists. For instance, Aedes scapularis can be identified by its serrated abdomen, a proboscis that is larger than the anterior femur, and the mesonotum with white scales forming a circle. Aedes serratus is identified by its serrated abdomen, a proboscis that is similar to or smaller than the anterior femur, and a mesonotum that may or may not have a longitudinal stripe of white scales. These two species bear a striking resemblance to each other. Sabethes albiprivus has medium-sized legs with bluish scales, a golden-scaled abdomen that forms quadrilaterals, and a proboscis that is much smaller than the anterior femur. Sabethes albiprivus and Ha. leucocelaenus are two species with different morphological characteristics. Sa. albiprivus can be distinguished from Ha. leucocelaenus by its predominantly dull, dark color and pleura with two vertical lines of silvery scales[7]. Other studies show that the accuracy of CNNs in identifying other insects is not significantly affected by the body region shown on the picture [22]. Our findings show that the morphological characteristics used for the identification of the mosquitoes included in this study are present in multiple regions of the body and therefore any of the body regions here studied allowed the AlexNet to accurately identify the mosquito species.

Deep learning neural networks consist of multiple convolutional layers, and databases with more pictures are more conducive to learning [21]. Additionally, many studies indicate that a larger picture bank improves the algorithm's performance [13, 14, 16, 17, 22, 32, 33]. Even though a database with thousands of pictures is always desired, using a database with only 1,343 pictures, we reached accuracy rates similar to those using databases 10x bigger than ours [17, 28] AlexNet accuracy to identify Sabethes and Haemogogus mosquitoes was similar to the accuracy obtained with other CNNs used to identify other genera [17, 28]. However, the accuracy of the AlexNet in identifying Ae. serratus and Ae. scapularis was below 90% and thus suboptimal when compared with the performance of other CNNs (VGG-16, ResNet-50, SqueezeNet) that apply data augmentation and fine-tuning techniques to identify Ae. aegypti [16, 17], Ae. albopictus and Ae. vexans [13].

These accuracy values may be due to differences in algorithm training [13]. For instance, optimization of CNN hyperparameters increase the accuracy of mosquito identification [17]. The poor performance of the algorithm in some cases may have been influenced by the state of preservation of the specimens. Analysis of the misidentified pictures in all experiments showed that the photographed specimens were not well preserved, especially in the pronotum area, where bristles and scales were missing, as well as the legs. Due to their size and the presence of scales and bristles, mosquitoes are easily damaged during capture, freezing, and drying, resulting in the loss of critical morphological features necessary for proper identification. The state of preservation of the mosquitoes was a limiting factor in this work, and good preservation of specimens is important for optimal algorithm performance.

In this study we successfully identified four species of mosquitoes that transmit yellow fever or other arboviruses using AlexNet. Our results support the idea of applying CNNs to the AI-driven identification of mosquito vectors of tropical diseases. This approach can potentially improve the surveillance of yellow fever vectors by health services and the population as well. Additional studies applying algorithms that identify mosquitoes may clarify which visual information is most relevant for the AI-driven identification of mosquito species.

We aimed to identify four mosquito species that transmit yellow fever and other arboviruses using AlexNet. We found that the AlexNet CNN can identify mosquito pictures of the genus Aedes, Sabethes and Haemagogus with over 90% accuracy, regardless of the body region being shown. Our results support the idea of applying CNNs for AI-driven identification of mosquito vectors of tropical diseases. This approach can potentially improve the surveillance of yellow fever vectors by health services and the population as well.

CI: confidence interval; CNN: Convolutional Neural Network.

Acknowledgments

We thank Marcia Triunfol at Publicase for reviewing this manuscript.

Authors’ contributions

TOA, RG-G, and VLM conceived the study. RG-G raised funds and administered the project. RG-G, and VLM contributed to the design of trial methods. TOA, RG-G, and VLM performed research. RG-G supervised students involved in laboratory research. TOA obtained the databases. RG-G, and VLM curated the dataset and analyzed the data. RG-G and TOA drafted the first version of the manuscript. All authors contributed to the interpretation of results, read, and commented on manuscript drafts, and approved the final version.

Funding

TOA received specific funding of Coordenação de Aperfeiçoamento de Pessoal de Nível Superior https://www.capes.gov.br/ Award Number: finance code 001. RGG received specific funding of Conselho Nacional de Desenvolvimento Científico e Tecnológico http://www.cnpq.br/. The funding sources of this study had no role in the study design, data collection, data analysis, data interpretation, writing of the report, or in the decision to submit the paper for publication.

Availability of data and materials

Data supporting the conclusions of this article are included in the article and its additional files.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

World Health Organization. World Health Day 2014: Preventing vector-borne diseases. Geneva: World Health Organization; 2014. Available from: https://www.who.int/news/item/02-04-2014-world-health-day-2014-preventing-vector-borne-diseases. Accessed May 16, 2024.
World Health Organization. Yellow fever. Geneva: World Health Organization; 2023. Available from: https://www.who.int/news-room/fact-sheets/detail/yellow-fever. Accessed Mar 03,2024.
Donalisio MR, Freitas ARR, Zuben APB Von. Arboviruses emerging in Brazil: challenges for clinic and implications for public health. Rev Saude Publica. 2017;51.
Ahebwa A, Hii J, Neoh KB, Chareonviriyaphap T. Aedes aegypti and Ae. albopictus (Diptera: Culicidae) ecology, biology, behaviour, and implications on arbovirus transmission in Thailand: Review. One Health. 2023.
Cunha MS, Faria NR, Caleiro GS, Candido DS, Hill SC, Claro IM, et al. Genomic evidence of yellow fever virus in Aedes scapularis, southeastern Brazil, 2016. Acta Trop. 2020;205.
Cardoso J da C, de Almeida MAB, dos Santos E, da Fonseca DF, Sallum MAM, Noll CA, et al. Yellow fever virus in Haemagogus leucocelaenus and Aedes serratus mosquitoes, Southern Brazil, 2008. Emerg Infect Dis. 2010;16:1918–24.
Segura MNO, Castro FC. Atlas de culicídeos na Amazônia brasileira: características específicas de insetos hematófagos da família Culicidae. 2007.
Vasconcelos PFC, Rosa APAT, Rodrigues SG, Rosa EST, Monteiro HAO, Cruz ACR, et al. Yellow fever in Pará State, Amazon region of Brazil, 1998-1999: Entomologic and epidemiologic findings. Emerg Infect Dis. 2001;7.
Vasconcelos PFC, Rodrigues SG, Degallier N, Moraes MAP, Travassos Da Rosa JFS, Travassos Da Rosa ES, et al. An epidemic of sylvatic yellow fever in the southeast region of Maranhao State, Brazil, 1993-1994: Epidemiologic and entomologic findings. Am J Trop Med Hyg. 1997;57.
Vasconcelos PFC, Travassos da Rosa APA, Pinheiro FP, Shope RE, Travassos da Rosa JFS, Rodrigues SG, et al. Arboviruses pathogenic for man in Brazil. An overview of Arbovirology in Brazil and neighbouring countries. 1998.
Cano ME, Marti GA, Alencar J, Silva SOF, Micieli MV. Categorization by Score of Mosquito Species (Diptera: Culicidae) Related to Yellow Fever Epizootics in Argentina. J Med Entomol. 2022;59(5):1384-1388.
Wilkerson RC, Linton YM, Fonseca DM, Schultz TR, Price DC, Strickman DA. Making mosquito taxonomy useful: A stable classification of tribe Aedini that balances utility with current knowledge of evolutionary relationships. PLoS One. 2015;10(3):e0133602.
Park J, Kim DI, Choi B, Kang W, Kwon HW. Classification and Morphological Analysis of Vector Mosquitoes using Deep Convolutional Neural Networks. Sci Rep. 2020;10(1):3144.
Pataki BA, Garriga J, Eritja R, Palmer JRB, Bartumeus F, Csabai I. Deep learning identification for citizen science surveillance of tiger mosquitoes. Sci Rep. 2021;11(1):206.
Kittichai V, Pengsakul T, Chumchuen K, Samung Y, Sriwichai P, Phatthamolrat N, et al. Deep learning approaches for challenging species and gender identification of mosquito vectors. Sci Rep. 2021;11(1):19375.
Motta D, Santos AÁB, Winkler I, Machado BAS, Pereira DADI, Cavalcanti AM, et al. Application of convolutional neural networks for classification of adult mosquitoes in the field. PLoS One. 2019;14(5):e0216427.
Motta D, Bandeira Santos AÁ, Souza Machado BA, Vicente Ribeiro-Filho OG, Arriaga Camargo LO, Valdenegro-Toro MA, et al. Optimization of convolutional neural network hyperparameters for automatic classification of adult mosquitoes. PLoS One. 2020;15(8):e0237151.
Kittichai V, Kaewthamasorn M, Samung Y, Jomtarak R, Naing KM, Tongloy T, et al. Automatic identification of medically important mosquitoes using embedded learning approach-based image-retrieval system. Sci Rep. 2023;13(1):13072.
Zhao OS, Kolluri N, Anand A, Chu N, Bhavaraju R, Ojha A, et al. Convolutional neural networks to automate the screening of malaria in low-resource countries. PeerJ. 2020;8:e8965.
Valan M, Makonyi K, Maki A, Vondráček D, Ronquist F. Automated Taxonomic Identification of Insects with Expert-Level Accuracy Using Effective Feature Transfer from Convolutional Networks. Syst Biol. 2019;68(5):876-895.
Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.
de Miranda VL, de Souza EP, Bambil D, Khalighifar A, Peterson AT, de Oliveira Nascimento FA, et al. Cellphone picture-based, genus-level automated identification of Chagas disease vectors: Effects of picture orientation on the performance of five machine-learning algorithms. Ecol Inform. 2024;79:101660.
Harrell FE. Hmisc: a package of miscellaneous R functions. 2014. Available from: http://biostat.mc.vanderbilt.edu/Hmisc.
RStudio: Integrated Development for R. RStudio, Version 4.4.0. Vienna: R Foundation for Statistical Computing. 2024. Available from: https://posit.co/products/open-source/rstudio/.
R Development Core Team. R: a language and environment for statistical computing. Version 4.4.0. Vienna: R Foundation for Statistical Computing. 2024. Available from: https://www.R-project.org.
Lorenz C, Ferraudo AS, Suesdek L. Artificial Neural Network applied as a methodology of mosquito species identification. Acta Trop. 2015;152:165–9.
Sauer FG, Werny M, Nolte K, Villacañas de Castro C, Becker N, Kiel E, et al. A convolutional neural network to identify mosquito species (Diptera: Culicidae) of the genus Aedes by wing images. Sci Rep. 2024;14:14147.
Okayasu K, Yoshida K, Fuchida M, Nakamura A. Vision-based classification of mosquito species: Comparison of conventional and deep learning methods. Appl Sci. 2019;9(18):3818.
Goodwin A, Padmanabhan S, Hira S, Glancey M, Slinowsky M, Immidisetti R, et al. Mosquito species identification using convolutional neural networks with a multitiered ensemble model for novel species detection. Sci Rep. 2021;11:14394.
Pora W, Kasamsumran N, Tharawatcharasart K, Ampol R, Siriyasatien P, Jariyapan N. Enhancement of VGG16 model with multi-view and spatial dropout for classification of mosquito vectors. PLoS One. 2023;18(7):e0253793.
Consoli RAGB, Oliveira RL de. Principais mosquitos de importância sanitária no Brasil. 1st ed. Rio de Janeiro: Editora Fiocruz; 1994.
Gurgel-Gonçalves R, Komp E, Campbell LP, Khalighifar A, Mellenbruch J, Mendonça VJ, et al. Automated identification of insect vectors of Chagas disease in Brazil and Mexico: The Virtual Vector Lab. PeerJ. 2017;5:e3040.
Khalighifar A, Komp E, Ramsey JM, Gurgel-Gonçalves R, Peterson AT. Deep Learning Algorithms Improve Automated Identification of Chagas Disease Vectors. J Med Entomol. 2019;56(2):461-468.

No competing interests reported.

Additionalfile1TableS1observedaccuracy.docx
Additional file 1: Table S1. Accuracy observed for each experiment
Additionalfile1Database.xlsx
Additional file 2: Database S1

Download PDF

Journal Publication

published 02 Aug, 2024

Read the published version in Parasites & Vectors →

Editorial decision: Revision requested
21 Jun, 2024
Reviews received at journal
19 Jun, 2024
Reviews received at journal
01 Jun, 2024
Reviewers agreed at journal
29 May, 2024
Reviewers agreed at journal
28 May, 2024
Reviewers agreed at journal
27 May, 2024
Reviewers invited by journal
26 May, 2024
Editor assigned by journal
25 May, 2024
Submission checks completed at journal
25 May, 2024
First submitted to journal
24 May, 2024

You are reading this latest preprint version

AI-driven convolutional neural networks for accurate identification of yellow fever vectors

Status:

Journal Publication

Version 1

Abstract

Background

Methods

Results

Conclusions

Figures

Background

Methods

Results

Algorithm Performance

Discussion

Conclusions

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1