FlightTrackAI: a convolutional neural network-based software for tracking the flight behaviour of Aedes aegypti mosquitoes

doi:10.21203/rs.3.rs-3991934/v1

Download PDF

Research Article

FlightTrackAI: a convolutional neural network-based software for tracking the flight behaviour of Aedes aegypti mosquitoes

https://doi.org/10.21203/rs.3.rs-3991934/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Monitoring the flight behaviour of mosquitoes is crucial for assessing their fitness levels and understanding their potential role in disease transmission. Existing methods for tracking mosquito flight behaviour are challenging to implement in laboratory environments, and they also struggle with identity tracking, particularly during occlusions. Here, we introduce FlightTrackAI, a novel convolutional neural network (CNN)-based software for automatic mosquito flight tracking. FlightTrackAI employs CNN, a multi-object tracking algorithm, and cubic spline interpolation to track flight behaviour. It automatically processes each video in the input folder without supervision and generates tracked videos with mosquito positions across the frames and trajectory graphs before and after interpolation. FlightTrackAI does not require a sophisticated setup to capture videos; it can perform excellently with videos recorded using standard laboratory cages. FlightTrackAI also offers filtering capabilities to eliminate short-lived objects such as reflections. Validation of FlightTrackAI demonstrated its excellent performance with an average accuracy of 99.9% and an average mean absolute error of 0.23 pixels. The percentage of correctly assigned identities after occlusions exceeded 91%. The data produced by FlightTrackAI can facilitate analysis of various flight-related behaviours, including diurnal and nocturnal locomotor activity, host-seeking behaviour, flight distance, volume coverage during flights, and speed measurement. This advancement can help to enhance our understanding of mosquito ecology and behaviour, thereby informing targeted strategies for vector control.

Artificial Intelligence and Machine Learning

Entomology

Behavioral Ecology

Infectious Diseases

Disease transmission

Convolutional neural network

Multi-object tracking

Flight behaviour

FlightTrackAI

Vector control

Diseases transmitted by mosquitoes are medically significant due to their global distribution and further implications for public health [1]. Annually, malaria alone leads to approximately 230 million cases [2], notably affecting children under the age of 5 [3]. Dengue virus is responsible for an estimated 390 million infections globally [4]. Beyond their impact on public health, mosquito-borne pathogens can also modulate various mosquito behaviours, such as locomotion, egg-laying, fertility, and feeding [5]. Furthermore, recent studies indicate that viruses transmitted by vectors can infect and substantially affect the nervous system of mosquitoes [6–8].

Understanding the behavioural characteristics of mosquitoes holds significance due to their role as crucial determinants in the transmission of pathogens and the epidemiology of mosquito-borne diseases. Attributes like fitness, host-seeking, and host-feeding play pivotal roles in the severity of the diseases they spread. Examining the flight behaviour of mosquitoes becomes essential for evaluating their fitness, offering insights into how diverse mosquito populations adapt to different environments [9, 10] and shedding light on the intricate relations between mosquitoes and the pathogen they carry [11, 12].

Traditionally, researchers have relied on manual observations to study the behaviours of mosquitoes. However, this method is resource-intensive and restricted the simultaneous monitoring of the number of mosquitoes. Additionally, some behavioural studies require continuous observations, leading to a laborious and time-consuming monitoring process. Various methodologies have recently been introduced to monitor mosquito flight behaviours. Khan et al. introduced a technique that involves the optimal placement of multiple pan-tilt-zoom (PTZ) infrared cameras within a 3D virtual environment for the observation and tracking of mosquito trajectories [13]. Wilkinson et al. employed in silico simulations to monitor the flight activity of Aedes albopictus mosquitoes [14]. McMeniman et al. utilised Track3D software and 210 Frames Per Second (FPS) cameras to monitor active mosquitoes within a swarm [15]. Straw et al. proposed simulating the 3D tracking of flying animals using a network of cameras with high spatial resolution and frame rate [16]. While advancements have been made in mosquito monitoring methods, their practical implementation in laboratory settings pose further challenges. Moreover, these approaches often suffer from limitations such as focusing only on active mosquitoes, the absence of individual mosquito identity while tracking, and significant performance degradation in identification in the cases of occlusions.

Recently, artificial intelligence (AI) has played a significant role in revolutionising visualisation [17–20]. AI imitates human intelligence processes via various processes incorporated into a dynamic computing environment [21]. In the past, models based on AI have been used to examine different aspects of mosquitoes, including identifying breeding sites [22], counting mosquito eggs [23], and determining gender [24]. AI has also demonstrated its utility in assessing mosquito control [25].

To address the limitations of existing methods, we previously employed a convolutional neural network (CNN)-based approach for monitoring mosquito flight behaviour [26]. This method effectively tracked both resting and flying mosquitoes. However, its performance was less effective in occlusion situations due to its reliance on a basic L2-norm distance-based approach for assigning identities to tracked mosquitoes. Furthermore, manual post-processing steps, such as interpolation, were required. In this study we introduce the FlightTrackAI software, which utilises CNN, a multi-object tracking algorithm, and cubic spline interpolation to track the flight behaviour of mosquitoes. FlightTrackAI does not require videos to be captured with a sophisticated setup; it can perform excellently with videos recorded using standard laboratory cages. The software takes a folder of videos as input, automatically processes each video without requiring supervision, and generates tracked videos with detailed mosquito locations and trajectory graphs before and after interpolation.

Mosquito rearing:

All experiments were conducted in accordance with biosafety level 3 (BSL-3) guidelines at the insectary facility located at the CSIRO Australian Centre of Disease Preparedness (ACDP). Aedes aegypti mosquitoes were raised under controlled conditions at a temperature of 27.0°C, maintaining a relative humidity of 70% and adhering to a 12-hour light/dark cycle. Adult mosquitoes were continuously supplied with a 10% sucrose solution for sustenance.

Experimental setup:

A plexiglass enclosure (25 cm x 25 cm x 40 cm) was used to isolate Aedes aegypti mosquitoes. This enclosure featured a movable white plexiglass wall to eliminate background interference from standard netting sleeves. By positioning the white wall between the net and the recording area of the cage, we ensured that video footage was free from obstruction for accurate inspection. Video data collection was facilitated using infrared capture technology (Flea3, Point Grey Research, Canada) (Fig. 1).

Data collection:

The recording was started two weeks post-emergence. Video data of 120 minutes duration was recorded at 60 FPS, with five mosquitoes inside the cage. Despite the Flea3 camera's native resolution being 1280 pixels in width and 1024 pixels in height, adjustments were made to set the frame size at 1040 pixels in width and 1024 pixels in height. This configuration was tailored to match the dimensions of the enclosure and prevent the capture of extraneous background.

Data processing:

The majority of locomotion-related behavioural analyses only focuses on individual mosquito's behaviour [27, 28]. In this study, the recorded video was subdivided into smaller segments, with each segment corresponding to a single mosquito flight. In instances where multiple mosquitoes were in flight concurrently, individual trimming was applied to ensure that each segment captured the flight of a single mosquito. With a total of 52 flights, the recorded video data was segmented into 52 individual video segments using [29].

Training, validation and test data:

Out of the 52 flights, 11 were short in duration lasting less than 10 seconds and were consequently excluded from the dataset. For the customised training of the Yolov8 model from the remaining video segments, we opted for 6 randomly selected video segments (around 15% of the total number of remaining videos), each spanning durations between 31 to 99 seconds, to serve for training and validation datasets. The remaining 35 segments, with durations ranging from 12 to 286 seconds, were reserved for performance testing. The average duration of the mosquito's flight was 52.13 seconds. Therefore, 10 consecutive videos from average duration onward, ranging from 54 seconds to 131 seconds were selected for trajectory testing, while all 35 videos were used for occlusion analysis. We converted training and validation videos into images at 60 FPS using the Python OpenCV library to prepare the data for training and validation. Out of these images, 900 were randomly chosen for training and 100 for validation. The CNN model underwent training for 500 epochs, but due to the early stopping patient set at 50, the training halted at 267 epochs, giving no observed improvement in performance and indicating that the best performance observed was at epoch number 217. An epoch represents a complete iteration over the entire training dataset during the training process. To annotate the training and validation data, we utilised the Makesense web tool, generating .text files [30].

Ground truth calculation:

Mosquito ground truth positions were also validated through the Makesense web tool [30]. The tool allows the import of images, permitting the confirmation of object positions by navigating the cursor across the screen (Fig. 2a).

Trajectory estimation evaluation:

Evaluating how well a tool performs its function is crucial in understanding its effectiveness. The FlightTrackAI tool's trajectory evaluation performance is assessed by considering both Mean Absolute Error (MAE) and accuracy. The accuracy metric evaluates whether the detected centroids meet the specified criteria, which considers both the position of the ground truth centroids and the tolerance allowed, while the MAE provides a detailed analysis of the variations between the detected centroids and the ground truth centroids, offering insight into the precision of the FlightTrackAI tool's performance. The MAE takes into account deviations along both the x and y axes in pixels, calculated using Eq. 1. The Python library Numpy is employed to compute MAE.

$$MAE=\frac{1}{n}\sum _{f=1}^{n}|{(x}_{f }-{\widehat{x}}_{f})|+|{(y}_{f}-{\widehat{y}}_{f}\left)\right|$$

Where ${x}_{f }$is x-axis estimated position in pixels, ${\widehat{x}}_{f}$ is x-axis ground truth position in pixels, ${y}_{f}$ is y-axis estimated position in pixels, ${\widehat{y}}_{f}$ is y-axis ground truth position in pixels. The value of f represents the frame number, $MAE$ is the mean absolute error across all the frames, and n indicates the total number of frames.

In this context, accuracy refers to how closely the FlightTrackAI tool captures trajectory points compared to the trajectory points of the ground truth. A tolerance of 6 pixels is set for accuracy, indicating that estimated points within this range (absolute value on both horizontal (x-axis) and vertical axis (y-axis)) of the ground truth centroids are considered accurate (Eq. 2). This choice is grounded in the understanding that mosquitoes are not single-pixel organisms, and given their varied positions, exact centroid estimation is challenging. Moreover, there are also chances of errors while calculating exact ground truth centroids manually, further emphasising the need for a reasonable tolerance in accuracy assessment. The 6-pixel threshold is determined by randomly selecting 20 frames, taking the average of mosquito lengths in selected frames, and dividing the average by two. This value corresponds to 0.57% of the x-axis total pixels and 0.58% of the y-axis total pixels.

$$if \left\{\begin{array}{c}|{(x}_{f }-{\widehat{x}}_{f})|<6 \cap |{(y}_{f}-{\widehat{y}}_{f}\left)\right|<6 Accurate\\ else Inaccurate\end{array}\right.$$

Occlusion calculation:

In our analysis, occlusions were determined using Python, specifically by computing the Intersection over Union (IoU) between the bounding boxes detected by the Convolutional Neural Network (CNN) for mosquitoes in the frames of videos. A threshold of 0.25 (25%) was chosen for IoU; if the IoU value between the detected bounding boxes of mosquitoes exceeded this threshold, the mosquitoes were considered occluded. This threshold, demonstrating partial overlap, was chosen to address the possibility that selecting a high threshold could result in overlapping mosquitoes being perceived as single mosquitoes before their calculation as occluded. Furthermore, occlusions were also manually verified by watching the videos. IoU was calculated using Eq. 3, given below.

$$IoU=\frac{Area of intersection}{Area of union}$$

Computation and programming system:

The FlightTrackAI was developed utilising an AMD Ryzen 9 5900HX processor integrated with Radeon Graphics (3.30 GHz), and 2x16GB SO-DIMM DDR4-3200 RAM, operating within a Windows 11 Pro 64-bit environment. FlightTrackAI performs detection and tracking with the assistance of Python, Numpy, Scipy, Tensorflow, and OpenCV.

Model architecture:

FlightTrackAI uses YOLOv8 for object detection [31], Deep SORT for multi-object tracking [32], and cubic spline interpolation to fill in the missing data points. The YOLOv8 architecture employs distinct backbone and head sections to execute object detection tasks. The CSPDarknet53 serves as the backbone, utilising convolutional layers to extract key features from the input image. The SPPF layer and the following convolution layers deal with features of different sizes, and the Upsample layers make the feature maps clearer. The C2F module combines important features and contextual data to improve detection accuracy. Finally, the Head section utilises a mix of convolutional and linear layers to convert complex features into output bounding boxes and classes.

Deep SORT performs multi-object tracking using the Kalman filter and the Hungarian algorithm equipped with deep association metrics. The Kalman filter is employed for predicting the state of objects, encompassing parameters like position, velocity, and acceleration. This prediction is based on the last known state of the object, incorporating considerations for the dynamic motion of the object. Simultaneously, the Hungarian algorithm performs matching by evaluating a cost matrix, which considers both the Mahalanobis distance for motion consistency and the cosine distance for appearance similarity. Incorporating a deep association metric enables Deep SORT to maintain tracking continuity even during short periods of occlusion. Then, FlightTrackAI employs cubic spline interpolation to smoothly fill in the missing data points, if there are any. This technique involves creating a smooth curve that passes through all the points in the dataset. Cubic spline interpolation seeks to create a smooth and uninterrupted curve at every data point using a cubic polynomial sequence (Fig. 2b). FlightTrackAI uses the Python library Scipy to perform interpolation. In the end, mosquito positions' data is saved in .xlsx format and before and after interpolation trajectories are generated using the Matplotlib Python library (Fig. 2c).

Mosquito flight tracking using FlightTrackAI:

The FlightTrackAI software's user interface provides five adjustable input parameters for effective mosquito flight tracking. The model also requires a trained model, which can be selected by clicking on the "Select Model" button. A pre-trained model is included with the software. However, if custom training is desired, users can follow the straightforward instructions provided in [31] to train their own model. The adjustable input parameters include Image Size, Confidence, Intersection over Union (IoU), Max Age, and Filtration. The Image Size parameter lets users set the maximum dimension of the input image, allowing adaptability across various resolutions. Considering the largest dimension of our experimental videos, its default value is set to 1040; it can be adjusted to any required value, and then FlightTrackAI will automatically re-adjust it to the nearest acceptable value, which should be a multiple of maximum stride 32, for compatibility with the convolutional neural network's architecture. For example, in our case, it was automatically adjusted to 1056. The Confidence parameter filters out detections with confidence scores below the specified threshold, ensuring that only high-confidence detections contribute to the final results. The default value is set to a low value (0.05) to accommodate small sizes of mosquitoes. The next parameter is the IoU, which is critical for Non-Maximum Suppression (NMS) during the detection phase. It helps to eliminate redundant bounding box predictions. The default IoU threshold is set at 0.05, promoting a lenient evaluation of spatial overlap during the NMS step, which is particularly useful for detecting small and closely spaced objects.

Additionally, FlightTrackAI includes the Max Age parameter, determining how many frames a detection remains in the tracking system if undetected for a certain number of frames. This parameter is essential for accurate tracking over time, preventing the system from retaining outdated or incorrect information. The last parameter is the Filtration parameter, which is crucial as it helps eliminate identities with short durations. This parameter is essential for excluding identities resulting from reflections on plexiglass surfaces, ensuring more precise and clean final results. By default, its value is set to 5, meaning that if an identity appears for less than 5% of total frames, it will be removed from the final results.

After setting these parameters, the input folder containing the videos can be selected by clicking on the "Select Folder" button. Selecting the input folder will automatically start the processing. After processing each video, the FlightTrackAI software generates a subfolder within the 'Results' directory located inside the input folder, named after the respective video in the input folder. This subfolder stores processed video and information about the positions of mosquito axes and the trajectories before and after interpolation. Besides saving the output results in the 'Results' folder, the FlightTrackAI will also show the unprocessed input and processed output frames in the user interface during processing. The FlightTrackAI allows users to halt the operation at any point by simply pressing the stop button at the bottom. Furthermore, it ensures that any partially processed video is automatically saved along with the information about the positions of mosquito axes and the trajectories before and after interpolation. The FlightTrackAI graphical user interface can be visualised in Fig. 3.

The object detection model has demonstrated outstanding performance in detecting mosquitoes. The validation results exhibited high precision (0.986), recall (0.988), and mAP50 of 0.989. Where mAP50 is the mean average precision calculated at an intersection over union (IoU) threshold of 0.50. The precision, recall, and mAP50 values across the training epochs are shown in Fig. 4a-c.

Analysis of the FlightTrackAI's performance on 10 test videos revealed that the object detector consistently identified mosquitoes in the majority of frames, resulting in a minimal percentage of interpolated data ranging from 0–0.67% across all videos, with an average of 0.20%. Additionally, the multi-object tracker exhibited exceptional capabilities, correctly assigning identities in 92 out of 101 occlusion instances, achieving a success rate exceeding 91% (Table 1). Visual representations in Fig. 5a,b showcase trajectories of two distinct flights before and after interpolation, highlighting their nearly identical appearance due to the minimal amount of interpolated data. Processed videos can be visualised in the supplementary data.

Table 1

Test videos' CNN detection and interpolation results
Flight Number	Total Frames	CNN Detected	Interpolated	Interpolation Percentage
1	3241	3221	20	0.61%
2	3361	3361	0	0%
3	3661	3661	0	0%
4	3721	3696	25	0.67%
5	3841	3818	23	0.59%
6	4381	4374	7	0.15%
7	4741	4738	3	0.06%
8	4741	4734	7	0.14%
9	5821	5821	0	0%
10	7861	7852	9	0.01%
Total	45370	45276	94	0.20%

Trajectory estimation evaluation:

FlightTrackAI's trajectory estimation performance was validated by assessing its performance on 10 test videos using accuracy and mean absolute error (MAE) encompassing error across both the x and y axes. The tool demonstrated a commendable accuracy, ranging from 99.89–100%, with an overall average of 99.98%. The mean absolute error, encompassing both axes, varied between 0.20 and 0.27 pixels, yielding an average of 0.236 pixels. Figure 6 is displaying the estimated and ground truth trajectories of two different flights. Estimated and ground truth trajectories are looking almost identical in both charts, highlighting the tool's outstanding performance.

Table 2

Trajectory estimation evaluation results.
Flight Number	Total Frames	Incorrect Detection Frames	Mean Absolute Error (Pixels)	Accuracy%
1	3241	3	0.25	99.91%
2	3361	0	0.23	100%
3	3661	0	0.24	100%
4	3721	3	0.26	99.91%
5	3841	4	0.27	99.89%
6	4381	0	0.24	100%
7	4741	0	0.22	100%
8	4741	0	0.24	100%
9	5821	0	0.21	100%
10	7861	1	0.20	99.99%
Total	45370	11	0.236	99.98%

Figure 7 illustrates the average of the first 3241 frames across all 10 videos, selected based on the minimum frame count of the smallest video. To facilitate a clear interpretation of the data, a moving average with a 50-frame period was applied, enhancing comprehension of the chart.

The FlightTrackAI software can facilitate comprehensive analysis of mosquito flight behaviour before and after infection. Data generated from FlightTrackAI can be used to study various aspects of flight behaviour, such as diurnal and nocturnal locomotor activity, host-seeking behaviour, flight distance, and velocity [33–36]. Additionally, synchronising two cameras facilitates the capture of three-dimensional data [37], enabling assessment of the volume covered during flights by processing the data from each camera individually. Moreover, considering the tool's commendable performance in analysing the flight behaviour of mosquitoes, which are small in size and exhibit complex flight patterns, it demonstrates the potential for broader use in studying other organisms like flies, ticks, and zebrafish.

Table 4 compares the tool's performance with other available mosquito flight tracking methods, offering valuable insights into its effectiveness. While most methods did not offer accuracy metrics, those that did indicate that our proposed method achieves a superior accuracy rate of 99.98%. Apart from its superior performance, the proposed approach features a user-friendly graphical interface. This interface simplifies the video processing workflow by enabling batch processing of videos within designated folders without constant supervision. Furthermore, the tool incorporates filtering capabilities to eliminate short-lived objects, such as reflections from the final results, thereby enhancing the reliability of the outcomes.

Table 3

Comparing mosquito flight tracking methods
Year	Tracking Method	Accuracy (%)	Reference
2011	Network of cameras with high spatial resolution and frame rate	-	[16]
2012	Stereo vision techniques	-	[38]
2014	In silico simulations	> 90%	[14]
2014	Track 3D software and high-speed cameras	-	[15]
2015	Pan-Tilt-Zoom (PTZ) infrared cameras' placement within a 3D virtual environment	-	[13]
2023	Convolutional neural network and interpolation	96%	[26]
2024	Proposed Method	99.98%	This Work

FlightTrackAI has demonstrated excellent performance in tracking mosquitoes from videos captured at 60 frames per second, even with occlusions. However, its performance in assigning the correct identities may drop at lower video frame rates. Previous animal tracking studies have recommended frame rates above 25 per second for animals such as mice, zebrafish, and ants [39, 40]. Considering mosquitoes' small sizes, fast speed, and intricate flight patterns, it may be necessary to use videos with higher frame rates compared to frame rates required for mice, zebrafish, and ants. Furthermore, significant gaps in data points resulting from background distortions and light reflections might impact interpolation performance. Therefore, this study suggests utilising a clean background, as demonstrated by introducing a simple white wall in this work. Additionally, increasing the training data and utilising anti-glare plexiglass can also help to enhance performance by minimising environmental distractions and optimising the input quality for data processing.

In conclusion, the development of FlightTrackAI utilising CNN, multi-object tracking algorithm, and cubic spline interpolation represents a significant advancement in understanding the flight behaviour of mosquitoes. This tool enables accurate analysis of mosquito flight behaviour through automatic video data processing and robust tracking capabilities. The validation results of FlightTrackAI demonstrate its high accuracy (99.98%) and lower average mean absolute error (0.236 pixels), making it a valuable asset for researchers looking into exploring the intricate pathogen-mosquito relation.

The FlightTrackAI sets the stage for creating automated mosquito behaviour monitoring systems which are crucial for assessing the fitness of insects, particularly in the context of infection or genetic modification studies. Furthermore, such advancements hold immense potential for advancing vector-borne disease modelling and the development of innovative trapping methods.

Future work could expand the artificial intelligence's capabilities to automatically track feeding-related behaviours, such as feeding instances and feeding durations, thereby providing a more comprehensive understanding of the pathogen-mosquito relationship. Overall, this research paves the way for further exploration and application of automated tracking technologies in entomology and beyond.

Acknowledgement:

The authors acknowledge the capabilities of the CSIRO Australian Centre for Disease Preparedness (grid.413322.5) in undertaking this research, including infrastructure by the National Collaborative Research Infrastructure Strategy (NCRIS).

Animal Ethics Statement:

The experiments for chicken blood collection were conducted under animal ethics guidelines with approval from ACDP Animal Ethics Committee (#ACDP22010).

Authors’ contributions:

Writing—original draft preparation, N.J.; writing—review and editing, A.B., A.J.L., and P.N.P.; visualisation, N.J.; Software, N.J.; Data curation, N.J., A.J.L.; supervision, A.B., A.J.L., and P.N.P. All authors have read and agreed to the published version of the manuscript.

Competing interests:

The authors declare no competing interests.

Funding declaration:

The study was partly funded by CSIRO strategic funding provided to P.N.P. and N.J. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Data Availability:

Supplementary videos, videos for testing, and FlightTrackAI graphical user interface .exe file is available at: https://drive.google.com/drive/folders/1fI0PbEfFs53tcndoB7yPG9Pp4paDLnJN?usp=sharing

World Health Organization, Global technical strategy for malaria 2016-2030,. 2015.
World Health Organization. The "World malaria report 2019" at a glance. 2019; Available from: https://www.who.int/news-room/feature-stories/detail/world-malaria-report-2019.
World Health Organization. Malaria in children under five. 2018; Available from: https://www.who.int/malaria/areas/high_risk_groups/children/en/.
Brady, O.J., et al., Refining the global spatial limits of dengue virus transmission by evidence-based consensus. 2012.
Javed, N., A. Bhatti, and P.N. Paradkar, Advances in Understanding Vector Behavioural Traits after Infection. Pathogens, 2021. 10(11): p. 1376.
Bhatti, A., et al., Emerging Trends in Neuro Engineering and Neural Computation. 2017: Springer.
Gaburro, J., et al., Zika virus-induced hyper excitation precedes death of mouse primary neuron. Virology journal, 2018. 15(1): p. 1-13.
Gaburro, J., S. Nahavandi, and A. Bhatti, Insects Neural Model: Potential Alternate to Mammals for Electrophysiological Studies, in Emerging Trends in Neuro Engineering and Neural Computation. 2017, Springer. p. 119-130.
Sharma, A., et al., Species B of Anopheles culicifacies (Diptera: Culicidae) is reproductively less fit than species A and C of the complex. Acta Tropica, 2009. 112(3): p. 316-319.
Hogg, J., M. Thomson, and H. Hurd, Comparative fecundity and associated factors for two sibling species of the Anopheles gambiaecomplex occurring sympatrically in The Gambia. Medical and Veterinary Entomology, 1996. 10(4): p. 385-391.
Lyimo, I., et al., The impact of uniform and mixed species blood meals on the fitness of the mosquito vector Anopheles gambiae ss: does a specialist pay for diversifying its host species diet? Journal of Evolutionary Biology, 2012. 25(3): p. 452-460.
Hall, R.A., et al., Commensal viruses of mosquitoes: host restriction, transmission, and interaction with arboviral pathogens. Evolutionary Bioinformatics, 2016. 12: p. EBO. S40740.
Khan, B., et al. Activity and Flight Trajectory Monitoring of Mosquito Colonies for Automated Behaviour Analysis. in Neural Information Processing. 2015. Cham: Springer International Publishing.
Wilkinson, D.A., et al., Straightforward multi-object video tracking for quantification of mosquito flight activity. Journal of Insect Physiology, 2014. 71: p. 114-121.
McMeniman, C.J., et al., Multimodal integration of carbon dioxide and other sensory cues drives mosquito attraction to humans. Cell, 2014. 156(5): p. 1060-1071.
Straw, A.D., et al., Multi-camera real-time three-dimensional tracking of multiple flying animals. Journal of The Royal Society Interface, 2011. 8(56): p. 395-409.
AlShamsi, M., et al., Artificial intelligence and blockchain for transparency in governance, in Artificial Intelligence for Sustainable Development: Theory, Practice and Future Applications. 2021, Springer. p. 219-230.
Matthews, G., et al., Evolution and revolution: Personality research for the coming world of robots, artificial intelligence, and autonomous systems. Personality and individual differences, 2021. 169: p. 109969.
Nawaz, M.S., et al., Using artificial intelligence techniques for COVID-19 genome analysis. Applied Intelligence, 2021: p. 1-18.
Yang, S.J., et al., Human-centered artificial intelligence in education: Seeing the invisible through the visible. Computers and Education: Artificial Intelligence, 2021. 2: p. 100008.
Shabbir, J. and T. Anwer, Artificial intelligence and its role in near future. arXiv preprint arXiv:1804.01396, 2018.
Bravo, D.T., et al., Automatic detection of potential mosquito breeding sites from aerial images acquired by unmanned aerial vehicles. Computers, Environment and Urban Systems, 2021. 90: p. 101692.
Javed, N., et al., EggCountAI: a convolutional neural network-based software for counting of Aedes aegypti mosquito eggs. Parasites & Vectors, 2023. 16(1): p. 341.
Alar, H.S. and P.L. Fernandez, Accurate and efficient mosquito genus classification algorithm using candidate-elimination and nearest centroid on extracted features of wingbeat acoustic properties. Computers in Biology and Medicine, 2021. 139: p. 104973.
Joshi, A. and C. Miller, Review of machine learning techniques for mosquito control in urban environments. Ecological Informatics, 2021. 61: p. 101241.
Javed, N., P.N. Paradkar, and A. Bhatti, Flight behaviour monitoring and quantification of aedes aegypti using convolution neural network. PloS one, 2023. 18(7): p. e0284819.
Lima-Camara, T.N., et al., Dengue infection increases the locomotor activity of Aedes aegypti females. PloS one, 2011. 6(3): p. e17690-e17690.
Tallon, A.K., et al., Dengue infection modulates locomotion and host seeking in Aedes aegypti. PLoS neglected tropical diseases, 2020. 14(9): p. e0008531-e0008531.
Digiarty. WinX HD Video Converter Deluxe software. Available from: https://www.winxdvd.com/hd-video-converter-deluxe/.
Piotr Skalski. Make Sense. 2019; Available from: https://www.makesense.ai.
Jocher, G., A. Chaurasia, and J. Qiu. Ultralytics YOLO. 2023; Available from: https://github.com/ultralytics/ultralytics.
Wojke, N., A. Bewley, and D. Paulus. Simple online and realtime tracking with a deep association metric. in 2017 IEEE international conference on image processing (ICIP). 2017. IEEE.
Hockmeyer, W.T., et al., Brugia pahangi: effects upon the flight capability of Aedes aegypti. Experimental Parasitology, 1975. 38(1): p. 1-5.
Schiefer, B.A., R.A. Ward, and B.F. Eldridge, Plasmodium cynomolgi: effects of malaria infection on laboratory flight performance of Anopheles stephensi mosquitoes. Experimental parasitology, 1977. 41(2): p. 397-404.
Somerville, A.G., et al., The consequences of Brugia malayi infection on the flight and energy resources of Aedes aegypti mosquitoes. Scientific Reports, 2019. 9(1): p. 18449.
Townson, H., The effect of infection with Brugia pahangi on the flight of Aedes aegypti. Annals of Tropical Medicine and Parasitology, 1970. 64(4): p. 411-420.
Brackenbury, J., Regulation of swimming in the Culex pipiens (Diptera, Culicidae) pupa: kinematics and locomotory trajectories. Journal of Experimental Biology, 1999. 202(18): p. 2521-2529.
Butail, S., et al., Reconstructing the flight kinematics of swarming and mating in wild mosquitoes. Journal of The Royal Society Interface, 2012. 9(75): p. 2624-2638.
Pérez-Escudero, A., et al., idTracker: tracking individuals in a group by automatic identification of unmarked animals. Nature Methods, 2014. 11(7): p. 743-748.
Romero-Ferrero, F., et al., idtracker.ai: tracking all individuals in small or large collectives of unmarked animals. Nature Methods, 2019. 16(2): p. 179-182.

The authors declare no competing interests.

ProcessedVideos.zip
Processed Videos

Download PDF

Version 1

posted

You are reading this latest preprint version

FlightTrackAI: a convolutional neural network-based software for tracking the flight behaviour of Aedes aegypti mosquitoes

Status:

Version 1

Abstract

Figures

Introduction

Materials and methods

Mosquito rearing:

Experimental setup:

Data collection:

Data processing:

Training, validation and test data:

Ground truth calculation:

Trajectory estimation evaluation:

Occlusion calculation:

Computation and programming system:

Model architecture:

Mosquito flight tracking using FlightTrackAI:

Results

Trajectory estimation evaluation:

Discussions

Conclusions

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1