2.1 OBJECT DETECTION
Object detection is a computer vision technique used to identify and locate objects within an image or video. It goes beyond simple image recognition by not only assigning labels to objects but also drawing bounding boxes around them, providing more detailed information about the objects' locations in a given scene. While image recognition focuses on assigning a single label to an entire image, object detection goes a step further by accurately detecting and localizing multiple objects within the image. Instead of just labelling the entire image as "dog," for example, object detection would draw separate bounding boxes around each dog in the image and label each box as "dog." This enables precise identification of the location and number of objects in the scene. In summary, object detection combines both identification (labelling) and localization (bounding box detection) to provide a more comprehensive understanding of the objects present in an image or video.
In order to harness the capabilities of object detection techniques such as YOLO and address its limitations, we propose the integration of a novel workflow (ak.2) [Fig. 2] that combines YOLO's efficiency with complementary methodologies. This workflow introduces a multi-stage approach, where YOLO's initial detections are further refined using advanced post-processing algorithms. By incorporating feature-based refinement and context-aware analysis, our proposed workflow aims to enhance the accuracy of object detection, particularly in scenarios involving small objects or fine-grained classification. In the following sections, we detail the components of this workflow and present experimental results that demonstrate its effectiveness in pushing the boundaries of object detection performance.
The YOLO algorithm has had a significant impact on the field of computer vision, particularly in object detection. It is known for its speed and efficiency, making it a popular choice for real-time object detection tasks. YOLO has achieved state-of-the-art performance on various benchmarks and has been widely adopted in numerous real-world applications. One of the key advantages of YOLO is its ability to process images quickly, allowing it to perform object detection in real-time. This makes it well-suited for applications such as video surveillance, self-driving cars, and augmented reality. Additionally, YOLO has a simple architecture and requires relatively small amounts of training data, making it easy to implement and adapt to new tasks. While YOLO has many strengths, it also has some limitations. It may struggle with detecting small objects accurately due to the nature of its single-shot approach. Additionally, it may not perform as well in tasks requiring fine-grained object classification. Despite these limitations, YOLO has significantly advanced object detection and has opened up new possibilities for researchers and practitioners in computer vision. As the field continues to progress, it will be interesting to observe how YOLO and other object detection algorithms evolve and improve to overcome existing challenges and address new ones.
A comprehensive methodology and techniques for detecting eggs in poultry conveyor systems using the YOLOv5 object detection algorithm are presented in this article. The proposed approach involves training a deep learning model on a labelled dataset of images to recognize eggs on the conveyor belt. The process includes detailing the materials, methods, and techniques utilized, including data collection, annotation, model training, evaluation, and deployment. Additionally, strategies for fine-tuning and optimization are discussed to enhance the performance of the mode [5].
Let us now first review the materials that would be required for our task -
- The poultry conveyor system includes the conveyor belt or any other mechanism used for transporting eggs in the poultry production line.
- A camera or multiple cameras are installed to capture images or video frames of the conveyor system. It is ensured that the camera(s) are positioned appropriately to capture the eggs from different angles and perspectives.
- A computer/server with sufficient processing power and memory is needed to train and deploy the YOLOv5 model. Once the model is trained for detection, a simple tracking code is used for getting the count of the eggs. The specifications of the computer/server will depend on the size of the dataset and the complexity of the model.
- A dataset of images or video frames of the poultry conveyor system is collected, containing examples of eggs in different positions, orientations, and lighting conditions. The dataset should be diverse and representative of the real-world scenarios encountered in the poultry production line.
- Annotation tools, such as Roboflow (ak.3) [Fig. 3], are used to label the eggs in the dataset by drawing bounding boxes around them. Each annotated bounding box is assigned a class label (e.g., "egg").
The various steps [Fig. 4] involved are as follows -
Data Collection:
For the data collection process, we followed the step guide of the CRISP-ML(Q) methodology. The Images or video frames of the poultry conveyor system are captured to ensure that the collected data represents a variety of egg
instances under different conditions. This is essential for training a robust model that can accurately detect and count eggs in real-world scenarios.
To broaden the dataset and include a diverse range of egg instances, it is common to collect images from various sources, including the internet. Collecting images from the internet can provide access to a wide array of egg images captured in different settings, lighting conditions, and environments. This helps to introduce more variability into the training data, making the model more robust and capable of handling diverse situations.
When collecting images from the internet, it is important to ensure that the images are relevant to the poultry conveyor system and reflect real-world conditions. Curating a dataset that accurately represents the range of scenarios encountered in the actual environment will contribute to the model's ability to generalize well and perform effectively in practical applications.
However, it is crucial to note that when collecting images from the internet, there may be copyright restrictions and usage rights associated with those images. It is essential to comply with copyright laws and obtain proper permissions or use images that are available under open licenses or for educational purposes.
By incorporating a variety of egg instances, including those collected from the internet, into the training dataset, the resulting model can effectively handle different conditions and improve the overall performance of the egg detection and counting system.
Dataset Preparation:
The captured data from the poultry conveyor system, which includes a collection of images or video frames, is carefully organized into a structured dataset suitable for training the YOLOv5 model. This dataset comprises visual information capturing various instances of eggs under diverse conditions, including factors such as different lighting, orientations, and environmental variations. The goal is to ensure that the dataset encompasses a comprehensive representation of real-world scenarios encountered in the poultry conveyor system. By incorporating this diverse and well-organized captured data, the YOLOv5 model can be trained to accurately detect and label eggs with high precision and generalization capabilities.
The dataset is then divided into training and validation sets, typically following an 80:20 or 70:30 split, respectively. To facilitate the training process, annotation tools are employed to label the eggs within the dataset. This involves drawing precise bounding boxes around each egg instance. The assigned class label for all annotated bounding boxes is "egg". Following the annotation process, various pre-processing techniques and data augmentation methods are applied to augment and diversify the annotated dataset, increasing its size and variability. Finally, the annotations are converted into the YOLOv5 format, where a corresponding text file is created for each image or frame. Each text file contains one row per object, specifying the object's class and the bounding box coordinates relative to the image size. This structured dataset, enriched with annotations and properly formatted for YOLOv5, serves as the foundation for training the model to accurately detect and localize eggs in the poultry conveyor system [6].
Model Training: The YOLOv5 notebook is opened in Google Colab. The drive is connected to store the files. The required dependencies are installed. The API key generated from Roboflow is used to connect to the dataset. The training parameters, such as learning rate, batch size, and number of epochs, are configured. The training process is initiated using the annotated dataset. During training, the model learns to detect eggs based on the labelled examples [7].
Model Evaluation: The performance of the trained model is evaluated after training for over 100 epochs using a sample image from the validation set. Metrics such as precision, recall, and mean average precision (mAP) are calculated to assess the accuracy of the model in detecting eggs. If necessary, the model or training parameters are adjusted to improve performance. The trained model is then converted to the ONNX (Open Neural Network Exchange) [8] format for use in OpenCV [9]. Models trained in one framework can be exported to the ONNX format and seamlessly imported into another framework without the need for extensive code rewriting or model retraining. This interoperability simplifies the process of integrating models into different software environments.
Object Tracking: A unique ID is generated for each detected object, and the objects are tracked as they move around in a video while maintaining the ID assignment.
3.2 CENTRE-BASED OBJECT TRACKING
The object tracking algorithm used in this approach is based on centroid tracking [Fig. 5]. It relies on the distance between centroids of each frame as the objects move. The position of the current frame is compared with the position of the previous frame to assign a unique ID that represents the object. At the beginning of the loop, an empty array is initialized to store the centroid points of the current frame. The loop continuously compares the current frame with the previous frames. Before the loop ends, a copy of the points from the current frame is made for later comparison [10].
Following are the steps involved in the tracking:
- We used Bounding Boxes to detect objects.
- From these Bounding Boxes, we calculated the centroids for each object.
- Then we assigned each of these objects with unique IDs.
- We calculated the centroid for each frame of the video.
- We also calculated the distance between the centroids of each consecutive frame. This is done to prevent the generation of multiple IDs for each object. A simple a mathematical formula is utilized to calculate the distance. The distance between the center of the current frame and the center of the previous frame is calculated using the Pythagorean theorem. The formula used for calculating the distance is as follows:
Distance = math.hypot (x2-x1,y2-y1)
- If the distance between the centers of the current frame and the previous frame was less than 100 pixels, it was considered that both points represented the same object in motion. Consequently, the existing ID of the last frame was carried over to the current frame. This helped maintain the same unique ID for a particular object.
- If the distance between the centers of the current frame and the previous frame exceeded the threshold, a new object id was added.
- When objects detected in the previous frame cannot be matched to any existing
objects, the object id was removed from tracking.
- With this looping process, the eggs in a conveyor were counted accurately.
3.3 Model Deployment:
The process employed for deploying the egg detection and tracking system using Streamlit (an open-source app framework in Python language) [11] involved several steps:
- Preparing the Model: Firstly, a trained model for egg detection and tracking was required to be developed. This typically involved training a deep learning model, such as YOLO or Faster R-CNN, on a labelled dataset of egg images or videos. We had already achieved this.
- Integration with Streamlit: Once the model was trained, it needed to be integrated with Streamlit [Fig. 6]. This was achieved by writing a Streamlit application script that incorporates the model's functionality. The script defines the user interface, handles file uploads, and calls the model for inference.
- User Interface Design: The Streamlit application script included the design of the user interface (UI). This may include elements such as upload buttons, video playback controls, and result visualization components. The UI should be intuitive and user-friendly, allowing users to easily upload images or videos and view the detection and tracking results.
- Real-Time Processing: To enable live stream processing, the Streamlit application was required to support real-time video input. This was achieved by accessing the live video stream from a webcam or connecting to a video feed source. Streamlit provides functionality to handle real-time video processing, allowing the model to continuously analyse frames as they are captured.
- Model Inference: Within the Streamlit application script, the trained model was called for inference on the uploaded images or live video frames. The model processed each frame, identified the presence and location of eggs, and tracked their movement across frames. The results were then displayed to the user in real-time.
- Deployment: Once the Streamlit application script was developed and tested locally, it can be deployed to a server or hosting platform. This ensures that the application is accessible to users over the internet. Deployment options may include platforms like Heroku, AWS, or deploying on a dedicated server.
- User Access: After deployment, users can access the egg detection and tracking system through a web browser. They can upload images or videos, or provide a live video stream for real-time analysis. The system processes the input and provides immediate results, allowing users to monitor egg detection and tracking activities effortlessly.