2.1 Simulated storage of rice after inoculation
Aspergillus niger, Penicillium citrinum, and Aspergillus glaucus are dominant strains that infect rice grains during storage. Among them, Aspergillus niger and Penicillium citrinum produce ochratoxin and penicillin, which are very toxicity, while Aspergillus aeruginosa can grow in a relatively dry environment. Aspergillus aeruginosa is the precursor strain in the process of the dominant strains during storage replacing the dominant strains in the field (Tang et al., 2009; Atungulu et al., 2015). In this study, we chose the above three mold strains to perform the experiments. The three mold strains were purchased from Beina Biotechnology (Henan, China).
The procedures of mold inoculation and simulated storage are as follows:
(1) Select a certain number of clean and mildew-free rice grains (indica rice, purchased from Huainan, Anhui Province, China). Put the sample grains into an oven and bake them at 80℃ for 4 hours to kill the original field molds attached to the rice grains. Put the dried sample grains into a set of 90mm round petri dishes (15 g grains for each petri dish).
(2) Inoculate the three mold strains into the potato glucose agar (PDA) medium separately and activate them at 28 ℃ for 3 days. Elute the activated colonies with sterile distilled water to prepare spore suspension samples, measure the spore concentrations in the spore suspension samples using the plate counting method, and dilute the spore suspension samples of three molds to 1.5×104 CFU/mL.
(3) Take 30 petri dishes containing 15 g rice grains. Inoculate 1.5 ml of spore suspension to the sample grains in each petri dish (each kind of spore suspension will inoculate 10 Petri dishes). Add 1 ml of sterile water into each petri dish and shake the Petri dish to allow the grains to fully absorb the water. After inoculation, the moisture content of the rice grains will be greater than 20%, thus creating a suitable condition for simulating accelerated mold growth in rice grains in a highly humid environment. Then, place the inoculated sample grains in a constant temperature and humidity incubator, and simulate rice storage under the conditions of 28 ℃ and 90% relative humidity. Take out five grain samples randomly from each petri dish every day to test the degree of mildew. The simulated storage will last 13 days until the grains reach a high mildew degree. During the course of the simulated storage, 1,950 sample rice grains with different contamination levels of Aspergillus niger, Penicillium citrinum, and Aspergillus cinerea are obtained (650 samples for each mold strain).
2.2 Acquisition of rice micro image
In order to acquire micro images of rice grains, we set up an MCV image acquisition system as shown in Fig. 1.
The system is composed of HD industrial camera, micro lens, LED ring light source, sample table, bracket, base, object distance adjustment screw rod, and sample table translation screw rod. The HD industrial camera is a Dahua A7A20MU30 color area array industrial camera (Huaray Technology Inc.) with a 1.2-inch target surface and a resolution of 4,096,×3,000 pixels. The microscope lens is a Lapson high-power lens, which supports 9X optical amplification. The LED ring light source has a power of 10 W and a color temperature of 6,500 K. In addition, its light divergence is uniform, and it will not produce shadow when irradiating the sample grains. The sample table is square, on which the petri dishes of different sizes can be placed and fixed. The light source is fixed at the position of 20 mm on the sample table. The height of the microscope lens and industrial camera can be adjusted based on the object distance by using the screw rod controlled by the computer, thus enabling automatic focusing during shooting. Two-dimensional translational movement of the sample table are realized through the action of the translation screw rod controlled by the computer.
The procedure of acquiring micro images of rice grain is as follows. First, turn on the ring light source. Then, place a single rice grain on the sample table, and adjust the position of the sample table until the rice grain is at the center of the camera's field of vision. Start automatic adjustment of the object distance to get a clear image and acquire the micro image of the rice grain. Turn the rice grain over along its central axis, and adjust the position of the sample table and the object distance to get a clear picture before acquiring the image of the second side of the rice grain. Thus, two micro images are acquired for each rice grain (as shown in Fig. 2). The image acquisition parameters include exposure time of 20 ms, gain of 1 dB, and automatic white balance. The size of the captured images is 4,096×3,000 pixels, and the spatial resolution of a single pixel is about 0.01 mm. The above parameter settings ensure that a whole rice grain is contained in a single image and all the details can be captured clearly.
2.3 Pre-process of micro images of rice grains
Each rice grain image contains a large portion of background, and the orientation of the grain placed on the sample table is not fixed (see Fig. 2). These factors create some difficulties for the subsequent analysis. Therefore, it is necessary to perform preprocessing on the acquired micro images of rice grains. The preprocessing is performed using MATLAB 2014b (Mathworks Inc. USA). The steps are as follows: (1) The color space of the original image (Fig. 3A) is converted to the Lab mode. There is a large difference between the gray scale values of rice grain and the background in the B-component. Therefore, the B-component gray image (Fig. 3B) is extracted for segmenting the rice region. Then, the B-component gray image is segmented using the automatic bimodal threshold segmentation method to obtain the binary image of the grain area (Fig. 3C). (2) The background of the original image is removed by using the binary image of the grain area as the template. Then, the minimum circumscribed rectangle of the grain in the binary image is calculated, and the angle between the long side of the rectangle and the horizontal direction is calculated. (3) The area within the minimum circumscribed rectangle of the grain is cropped from the original image, and is then rotated in the horizontal direction (Fig. 3E). Finally, the cropped grain image is placed at the center of a black image of size 3,000×1,000 pixels to obtain the preprocessed image (Fig. 3F). After preprocessing, the size of the micro image is reduced from 4,096×3,000 pixels to 3000×1000 pixels. Finally, all the micro images of rice grains are of the same size and the grains in the images have the same orientation.
2.4 Image marking
YOLO model is a deep learning method that learns in a supervised manner. It requires manual marking of the mildew sites in the rice images. The mildewed regions in all rice micro images are marked using the LabelImg 1.8.6 image marking tool developed using Python. In this study, single category marking is used to mark the mildewed regions without grading the degree of mildew at each mildew site. In order to obtain more data and more accurate mildewed regions, it is necessary to draw the marking box as close to the outline of the mildewed region as possible, and the size of the bounding boxes does not exceed 200×200 pixels.
2.5 Model establishment
YOLO-v5 is used to establish the model for identifying the mildewed regions on the surface of rice grain. The model architecture is shown in Fig. 5. The function of the backbone is to extract features from the image, the function of the neck is to aggregate the image features, and the function of the head is to calculate the model output according to the aggregated image features produced by the neck. The software platform used to establish model includes Pycharm 2021.3.1 (JetBrains, CZ). The toolbox used in experiments include YOLOv5-master, the YOLO-v5 official toolbox based on PyTorch deep learning library. The graphics card includes Nvidia RTX 2060 with 6 GB of video memory.
In this study, three YOLO-v5 model for identifying mildewed regions in the micro images of rice grains contaminated by Aspergillus niger, Penicillium citrinum and Aspergillus cinerea is established in the following way. 1,050 rice grain samples are taken from the prepared rice grains contaminated with Aspergillus niger, Penicillium citrinum, and Aspergillus cinerea (350 samples for each mold strain). The micro images of the samples are acquired (700 images for each mold strain), and the mildewed regions are marked. In the end, 70497, 62355, and 50689 mildewed regions are marked in the microscopic images of rice grains contaminated by Aspergillus niger, Penicillium citrinum, and Aspergillus cinerea, respectively. The micro images of rice grains contaminated by the three mold strains are randomly divided into training set and verification set according to the ratio of 6:4, for establishing the models for recognizing mildewed regions of rice grains contaminated by the three mold strains. In order to improve the training speed and reduce memory occupation, we chose YOLOv5-s model characterized by relatively small number of layers and small number of nodes for establishing the mildew recognition model. The learning rate is set to 0.01. Mosaic enhancement (mosaic), image right-left flipping (Fliplr), and image up-down flipping (Flipud) strategies are used for image data enhancement. The classification model of COCO dataset is used as the pre-trained model. As many marked mildewed regions are tiny mildew spots (about 20×20 pixels), it is difficult for the model to detect such tiny mildew spots if the input resolution of the image is too low. Therefore, the input image size is set to 1,280×448 pixels. The batch size is set to 4 so as to save the video memory, and the number of training epochs is set to 100 during the training process. In order to reduce the chance of the model misjudging the background and improve the degree of fit between the model output boundary box and the mildewed region, we set both the confidence threshold and intersection-over-union threshold of the model to 0.3. After training is completed, the best model is selected according to the variation of box loss during the training. The method for calculating box loss is shown in Eq. 1:
$$Box loss=1-\frac{A\cap B}{A\cup B}$$
1
where, A∩B is the number of pixels in the intersection area of the model-predicted boundary box and the manually marked box, A∪B is the number of pixels in the union area of model-predicted boundary box and the manually marked box.
Then the confusion matrix of the recognition results yielded by the model in detecting mildewed regions in the image of rice grains contaminated by molds was calculated to evaluate the accuracy of model detection. Finally, the feature images output by the three C3 modules (marked with red box in Fig. 5) in the second, fourth and sixth stages of the Backbone part of the model were collected to analyze the effectiveness of the model in extracting the features of mildewed regions of rice grains.
2.6 Analysis of relationship between MAI and TVC of rice grain
900 rice grain samples are taken from the rice grains contaminated by Aspergillus niger, Penicillium citrinum, and Aspergillus cinerea (300 samples for each mold strain), and the micro images of these samples are acquired. After the micro images are preprocessed, the proposed model is used to identify the mildewed regions in the images. After the mildewed regions are identified, the MAI value of each rice grain is calculated. The calculation method is shown in Eq. 2:
$$MAI= \frac{1}{2}\left(\frac{{\text{M}}_{A}}{{\text{S}}_{A}}+\frac{{\text{M}}_{B}}{{\text{S}}_{B}}\right)$$
2
where, MAI is the proportion of mildewed regions. Let the two sides of each rice grain be called side A and side B, then, MA is the total area of mildewed regions recognized in the micro image of side A of the rice grain, MB is the total area of mildewed regions recognized in the micro image of side B of the rice grain, SA is the total area of side A of the rice grain in the micro image, and SB is the total area of side B of the rice grain in the micro image.
After the micro images of the sample rice grains are acquired, the TVC of each rice is obtained using the colony plate counting method in the following way. First, weigh the mass of the rice grain, and then put it in the test tube. Add 10 ml of sterile distilled water into the test tube. Then, put the test tube into a shaking table and let the shaking table work for 1 h at the speed of 150 R/min so as to fully elute the mold spores on the grain surface. After the operation of the shaking table is over, dilute the resulting bacterial suspension 10 times, 100 times, and 1,000 times to obtain three suspensions samples of different dilution ratios. Inoculate the original bacterial suspension and the bacterial suspensions of different dilution ratios separately into the PDA medium added with chloramphenicol. For each dilution ratio, two petri dishes are provided for inoculation, and 0.5 ml bacteria suspension is inoculated in each petri dish. After inoculation in each Petri dish is completed, shake the petri dish to ensure full mixing, and put the petri dish into the constant temperature and humidity incubator for cultivation at 28 ℃ for 48 hours. After that, take out the petri dish and calculate the TVC value of the rice grain, and perform regression analysis on the TVC and MAI values.