Instance segmentation of objects is a significant stage in real-time video detection. Object detection is the task of detecting the information of all types of objects in an image by marking the detected object's position in the image with a rectangular box. Using its great feature learning capabilities, deep learning attains a development in object detection research. Many researchers have performed object detection with different machine learning algorithms and focused on enhancing the accuracy of feature extraction. But the accuracy of the lower and higher-level features are not featured properly due to this error that occurred in the detection of the object at a lower level. To overcome this issue, FPN is used for feature extraction, which extracts both lower and higher-level features for accurate detection of an object, and the classification process is done by the proposed model (Mask R-CNN). The proposed model is implemented on python for finding the performance metrics such as accuracy, precision, execution time, error, etc. The accuracy for real-time object detection using FPN and Mask R-CNN is 97.8%, which is greater compared to the existing techniques such as TLD (Tracking-Learning detection), R-CNN, Fast R-CNN, Mask R-CNN and FPN. Thus, the designed model instantly segments the real-time video for object detection in an effective manner.