Drone Detection and Alert System Using Deep Learning

doi:10.21203/rs.3.rs-2794450/v1

Download PDF

Research Article

Drone Detection and Alert System Using Deep Learning

https://doi.org/10.21203/rs.3.rs-2794450/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Drone Detection is the application of detecting Unmanned Aerial Vehicles (UAVs) or drones. In recent years, the drone industry has seen a tremendous increase in manufacturing and sales because of its versatility in applications such as delivery and surveillance. This industry grew to $24.72 Billion in 2020 alone, and almost all of these drones are in the hands of common public, which has caused numerous security and prevaricated concerns in recent times. This paper represents a drone detection and alert system that has extensive use of optical vision, and uses third party softwares for alert notifications. Based on the camera images, the system deduces whether there is a drone in the sky or not, and sends out an alert notification.

Drone Detection

YOLO

CNN

In recent years, the drone industry has seen a tremendous increase in manufacturing and sales because of its versatility in applications such as delivery and surveillance. This industry grew to $24.72 Billion in 2020 alone, and almost all of these drones are in the hands of common public, which has caused numerous security and prevaricated concerns in recent times. The model works efficiently and accurately, with an fps greater than 15, and is able to detect any drone passing through the camera while simultaneously sending out an alert notification to the concerned members. The purpose of this work is to develop a system which is capable of identifying objects, and can tell the difference between 2D and 3D objects in a photograph.

Drone detection is to identify drones and distinguish them from the rest of the similar looking objects in the sky. This can be achieved by deep learning models to detect drones from a given picture using fast Convolutional Neural Networks (CNN) architecture. This technology helps to monitor and manage these machines so that the skies stay friendly. The effort here is to make the aerial vehicles more safe and easily accessible to public without any casualties, now that drones have found use in every industry. For example, a company called Wing Aviation LLC recently received federal aviation administration (FAA) clearance to begin using drones for routines. On the other hand, Amazon has said it could begin delivering online orders using drones "in the coming months"[1]-[4].

Drones are rapidly being used to inspect elements of super tall structures such as turbine on windmills and as cheaper alternative to helicopters. Even media houses are turning to them for receiving reports from natural disaster to traffic updates. The law enforcement agencies themselves are exploring drone usage for mapping crime scenes, serving warrants and high risk traffic signals. The possibilities are almost endless, which is undoubtedly a part of the reason why millions of drones will be registered for use. And it is difficult to differentiate between two drones that look similar, but have opposite intentions[5]-[8].

This is the most exciting time for many who envision a world using drones helps solve a variety of challenges that us humans often struggle with in industries such as agriculture, healthcare, insurance, finance, manufacturing, and transportation[9]-[12].

The major idea is to detect any drone that passes the camera and to send an alert to the concerned people. The model is also capable of differentiating between harmful and safe drones, based on the classification algorithm and the parameter of separation.

YOLO (You Only Look Once) v4 has been used for the detection of drones/UAVs. The entire detection process is done only using a single architecture network. CNN is used to predict various class probabilities and bounding boxes simultaneously, which improves the speed of detection. The model is predicting objects in real time, and reduces background errors.

The paper presents a holistic approach to the entire problem associated with drones, and provides the best outcome possible at the lowest economic factors.

CNN-based object detection methods are directly applicable to static images. Moreover, many annotated video datasets of the target object are required to train both the object detectors and visual trackers. The proposed algorithms are tested using a drone detection task and the experimental results demonstrated their effectiveness. YOLO v4 algorithm is built on the darknet framework. The model is majorly based on the following three techniques:

A. Residual Blocks (ResNet Architecture)

A residual block is a stack of layers set in such a way that the output of a layer is taken and added to another layer deeper in the block.In this, each image/frame of a video is divided into multiple grids, and each grid detects the objects within them. If an object center appears within a certain grid cell, then that cell will be responsible for its detection.

B. Bounding Box Regression

A bounding box is an outline that highlights an object in an image. Every bounding box in the image consists of the following attributes: height, Width, Bounding Box Centre and Class. YOLO uses a single bounding box regression to predict all of these attributes. In the image above, it represents the probability of that object appearing in the bounding box. Bounding-box regression is a popular technique to refine or predict localization boxes in recent object detection approaches. Typically, bounding-box regressors are trained to regress from either region proposals or fixed anchor boxes to nearby bounding boxes of a pre-defined target object classes.

C. Intersection over Union (IoU)

It’s a term used to describe the extent of overlap of two boxes. From the image shown in Fig. 3, the green box is the correct box and the red box is the predicted box. The aim of all algorithms is to reduce the overlap as much as possible, to produce higher accuracy. The IOU of two boxes can have any values between 0 and 1. In case there are 2 boxes that do not intersect, the area of their intersection would be 0, and therefore the IOU would also be 0. In case there are 2 boxes that completely overlap, the area of the intersection would be equal to the area of their union, and therefore the IOU would be 1.

YOLO v4 has 75 CNN layers, and 32 other layers such as route layers and shortcut layers. YOLOv4 is designed based on recent research findings, using CSPDarknet53 as a Backbone, SPP (Spatial pyramid pooling) and PAN (Path Aggregation Network). The CSPDarknet53 has 53 CNN layers. CSPNet is an optimization method aiming at partitioning feature map of the base layer into two parts and then merging them through a cross-stage hierarchy presented below. SPP is a method of acquiring both fine and coarse information by simultaneously pooling on multiple kernel sizes. PAN is a technique that leverages information in layers close to the input by conveying features from different backbone levels to the Detector.

The following are the building blocks of YOLO v4:

Input: Image, Patches, Pyramid
Backbone: ResNet50, CSPResNext50, VGG16, SpineNet, EfficientNet-B0-B7
Neck: Feature Pyramid Network (FPN), SSP
Head: YOLO, MatrixNet, Faster R-CNN

All of these are the options or possibilities of choosing a YOLO v4 architecture depending upon the requirement. It explores different backbone networks and data augmentation methods. The final architecture includes CSPDarknet53, because of its higher Frames per Second (fps).

This architecture takes an image as input and resizes it to 448*448 by keeping the ratio same and performing padding. This image is then passed in the CNN. This particular model has 24 convolution layers, 4 max-pooling layers followed by 2 fully connected layers.

The process starts with creating a dataset. Multiple high-resolution images of different kinds of drones in different colors have been used to create a database to train the model.
Now, the database is augmented through the “Augmentor” module in python. Common augmentation techniques are cropping, adding noise, changing exposure, and rotation.
Data Annotation is carried out thereafter, which is the process of labelling data to show the outcome required for the model to predict. The dataset is labeled with features that are expected of the model to predict.
The final dataset is now set according to the format specified under YOLO and the darknet framework is now set.
The custom model is trained using the augmented and annotated dataset. mAP (Mean Average Precision) values are calculated from this, and adjustments in the database are made for better precision.
Once a drone has been detected, the model also checks for whether it is an authorized drone or not. This is verified by taking distinguishing parameters as inputs, and then making a dataset of the same to train the model and labeling it as unauthorized.
An alert system is also in place for whenever a drone is passing through the camera. The alert system sends a mail whenever the accuracy of the model is beyond a certain threshold.

Since our eyes cannot see the sky for all of day and night, it is important to make use of an alert system that sends a mail to the concerned tenants of the house, whenever there is a drone hovering above it.

The alert system is in place to make sure that the problem at hand is dealt with, and the threat is neutralised at the earliest without causing any delay. Third party websites such as Twilio and Sendgrid are being used to set up the alert system. SendGrid is an online platform that helps in sending any and all kinds of mails to any number of users with different parameters, such as marketing mails and promotional mails.

Once the deep learning algorithm is run and the drone is detected, a mail notifying the user or the concerned people will be sent in less than a second.

This is possible because of the Sendgrid and Twilio modules that are available in python. To make use of the alert system, an account is to be created in Sendgrid, and then an API (Application Programming Interface) Key is to be made to be able to link the website to the deep learning model.

Once an API Key is created, the module is imported in python and its in-built mail function is called. In this function, the user can specify the ‘to’ and ‘from’ email-id’s, the subject line and the body. This is useful and efficient since the module is also capable of sending multiple custom mails to different e-mail id’s in a short span of time.

The model works efficiently and accurately, with an fps greater thhan 15, and is able to detect any drone passing through the camera while simultaneuously sending out an alert notifiation to the concerned members.

Object recognition based on deep learning has become a prominent topic in recentyears. This project begins with generic drone detection pipelines, which serve as the foundation for other activities. Noise in the camera sensor and poor illumination might skew the results, making it harder to recognize things. In general, GPUs are required for this technique rather than CPUs. However, we've done it with CPUs, which means it now runs in a quarter of the time and is more efficient. Object detection algorithms integrate object detection and image categorization into a single algorithm. The image is taken to be the input, and provides an output of bounding boxes that are appropriate for the number of things in the image, with the highest level category name attached to each bounding box. The bounding box's scenario is projected in the form of location, height, and width. It is now possible for the model to detect drones with greater precision and identify them individually with the location of the said object anywhere in the two-dimensional coordinate plane, and then send an alert;utilizing this thesis and basing this on experimental results.

The drone detection system can be utilized for a variety of purposes, includingfacial recognition, surveillance, fault detection, and character recognition. The purpose of this work is to develop a system which is capable of identifying objects, and can tell the difference between 2D and 3D objects in a photograph.

The performance of this system is determined by the characteristics employed and the classifier used for recognition. The goal of this research is to offer a new feature extraction method to extract local and global features from the study area.

Furthermore, the project tries to mix traditional classifiers in order to recognizeobjects.

As a potential area for future development the following can be considered:

Increase in the global or local features which are employed for recognition, that can improve effectiveness of the system
Inclusion of geometric properties of the image onto the feature vector can be done for faster recognition
Instead of using a supervised classifier, an unsupervised classifier can be used for object recognition
Size of the template remains set for tracking in Single Visual tracking.When the size of the object being monitored shrinks over time, the background takes precedence over the object being tracked. The objectmay not be tradable in this instance.

DISCLOSURE STATEMENT

No potential conflict of interest was reported by the author(s).

Authors Bibliography.

C.T.Manimegalai is currently working in the Department of ECE, SRM Institute of Science and Technology, Kattankulathur. She has done her PH.D in Wireless Communication in the year 2014. Her area of specialization include Wireless communication, Signal processing, Optical communication and Image processing. She has received a funded project from “Studies on Disaster - resilient management and Bidirectional Multiband signal transmission Using Double Clad Fibers in Radio Over Fiber Systems” From DST, India. She has filed a patent , “ An Optical Wireless charging and data Transfer Apparatus”. She is a member of several professional societies such as IEEE, IEI, IETE, OSI and ISTE. She has published several papers in refereed Journals

K.Kalimuthu is currently working in the Department of ECE, SRM Institute of Science and Technology, Kattankulathur. He has done his PH.D in Wireless Communication in the year 2014. Her area of specialization include Wireless communication, Signal processing, Optical communication and Image processing. he has received a funded project from “Studies on Disaster - resilient management and Bidirectional Multiband signal transmission Using Double Clad Fibers in Radio Over Fiber Systems” From DST, India. He has filed a patent , “ An Optical Wireless charging and data Transfer Apparatus”. He is a member of several professional societies such as IEEE, IEI, IETE, OSI and ISTE. He has published several papers in refereed Journals.

C. Aker and S. Kalkan, "Using deep networks for drone detection," in 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2017, pp. 1-6.
M. Saqib, S. D. Khan, N. Sharma, and M. Blumenstein, "A study on detecting drones using deep convolutional neural networks," in 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2017, pp. 1-5.
A. Richards J., Remote sensing digital image analysis, fifth ed. springer, 2013 p. 494
S. Al-Emadi and F. Al-Senaid, "Drone Detection Approach Based on Radio-Frequency Using Convolutional Neural Network," 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT), Doha, Qatar, 2020, pp. 29-34, doi: 10.1109/ICIoT48696.2020.9089489.
L. Mejia’s, S. McNamara, J. Lai, and J. Ford, "Vision-based detection and tracking of aerial targets for UAV collision avoidance," in 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010, pp. 87-92
B. Taha and A. Shoufan, "Machine Learning-Based Drone Detection and Classification: State-of-the-Art in Research," in IEEE Access, vol. 7, pp. 138669-138682, 2019, doi: 10.1109/ACCESS.2019.2942944.
G. U. o. F. Gokc ¨ ,e, E. S, ahin, and S. Kalkan., “Vision-Based Detection and Distance Estimation of Micro Unmanned Aerial Vehicles," 2015.
M. Jian, Z. Lu, and V. C. Chen, "Drone detection and tracking based on phase-interferometric Doppler radar,” in 2018 IEEE Radar Conference (RadarConf18), 2018, pp. 1146-114
Y. Seo, B. Jang, and S. Im, "Drone Detection Using Convolutional Neural Networks with Acoustic STFT Features," in 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2018, pp. 1-6.
Park and S. Park, "Configuration of an X-band FMCW radar targeted for drone detection,” in 2017 International Symposium on Antennas and Propagation (ISAP), 2017, pp. 1-2.
Shengxiang Qi, W. Z. (2018). Detecting Consumer Drones from Static Infrared Images. Proceedings of the 4th International Conference on Communication and Information Processing, (pp. 62-66). Qingdao
P. Nguyen, M. Ravindranatha, A. Nguyen, R. Han, and T. Vu, “Investigating cost-effective rf-based detection of drones,” in Proceedings of the 2nd workshop on micro aerial vehicle networks, systems, and applications for civilian use, 2016, pp. 17–22.

Download PDF

Reviewers agreed at journal
24 Jul, 2023
Reviewers invited by journal
24 Jul, 2023
Editor assigned by journal
10 Apr, 2023
First submitted to journal
08 Apr, 2023

You are reading this latest preprint version

Drone Detection and Alert System Using Deep Learning

Status:

Version 1

Abstract

Figures

Impact Statement

I. INTRODUCTION

II. OBJECTIVE

III. TECHNIQUES USED

IV. ARCHITECTURE

V. METHODOLOGY

VI. ALERT SYSTEM

VII. RESULTS

VIII. CONCLUSION

IX. FUTURE ENHANCEMENT

Declarations

References

Status:

Version 1