應用深度學習方法發展高通量溫室番茄果實表型分析系統

Development of a High-Throughput Phenotyping System for Greenhouse Tomato Fruits Based on Deep Learning

黃怡瑄(Yi-Hsuan Huang)

指導教授 ： 林達德

國立臺灣大學/生物資源暨農學院/生物產業機電工程學研究所/碩士(2019年)

https://doi.org/10.6342/NTU201903006

摘要

近年來，全球糧食危機和糧食不安全的問題越來越嚴重，利用高效率的育種技術培育出較高產量的作物是其中一種解決方式，而育種技術仰賴準確的作物表型分析系統。本研究之目的即為建構一高通量之果實偵測、定位及量測系統，利用RGB-D相機拍攝目標果實之影片，並採用電腦視覺及深度類神經網路結構，達到自動化之目的。相較於其他作品利用單一影像進行單一作物之量測，本研究將利用影片提高果實量測之精準度，且方便進行影像之取得，選擇番茄為目標果實。深度學習架構採用YOLOv2，為一即時之物件偵測演算法，作為果實之辨識及定位的物件偵測器，其於靜態影像之最佳偵測命中率為88.86%。接著，開發個別果實追蹤演算法以追蹤影片中數個果實，包括線上追蹤及線下追蹤，線上追蹤利用特徵點偵測和配對、光流法及剛體轉換，搭配有限向量機進行各顆果實的追蹤，亦針對遮蔽問題，採用閾值設定及去噪。線下追蹤則利用投票法降低由物件偵測器及深度影像造成的假警報。完成追蹤演算法後，將分析得果實形態學參數如個別果實之成熟度、大小、總果實之計數結果，以及二維空間分布圖。本研究共進行三次實驗，前兩次為系統架設及測試組，第三次實驗為驗證組，果實實際計數之最佳平均絕對相對計數誤差為9.91%，前兩次實驗中大部分的果實成熟度為二級和三級，且果實平均截面積大小分別為45.68及42.01平方公分，而驗證實驗之平均絕對相對計數誤差為15.15%，果實成熟度則大部分為三至五級，且果實平均截面積大小為40.11平方公分，由結果之合理一致性證明此實驗及提出之系統具備重複性及再現性。利用此自動化之高通量果實表型分析系統，可即時獲得溫室中果實之分布及各顆果實的生長資訊，提供農民果實的生長過程量化指標，亦可進行自動化之紀錄與分析，提供產量評估及栽培作業改善資訊，以持續優化栽種技術及提高產量。

關鍵字

物件偵測 ； 物件追蹤 ； 電腦視覺 ； 表型分析 ； 自動化

並列摘要

Food crises and security issues are getting worse. One of the sought solutions is by using efficient breeding systems which requires accurate and detailed phenotyping of plants. In this work, a high-throughput technique for fruits detection, localization and measurement from video streams using computer vision, RGB-D camera and deep neural networks is proposed. Contrary to other works that different methods are developed for each type of fruits with image information, our work utilize the video information to do the precise detection and analysis fruits’ morphological parameters. A real-time object detection algorithm using YOLOv2, a deep neural network-based detector, is used for fruit detection and localization on video frames with a highest hit rate of 88.86%. An individual fruit tracking algorithm is performed throughout the video stream to perform tracking of multiple fruits. The online tracking algorithm based on finite state machine includes feature matching, optical flow and rigid transformation which is optimized by occlusion handling techniques such as by applying threshold indices and denoising. On the other hand, the offline tracking algorithm uses voting method to reduce the false alarms caused by the object detector. Finally, the morphological parameters such as individual fruit ripening stage, fruit size, and 2D spatial distribution maps are obtained. There are three experiments in the study. First two experiments are for system construction and testing. The third experiment is for system validation. The best absolute relative fruit count error is 9.91%. In the first two experiments, most of the ripening stage of the fruits are 2 and 3, and the average cross-sectional area of tracked fruits are 45.68 cm2, 42.01 cm2, respectively. The absolute relative fruit count error is 15.15% in the validated experiment, and most of the fruit ripening stage is 3 to 5. The average cross-sectional area of tracked fruits in the third experiment is 40.11 cm2. By the consistent results from the experiments, the validated experiment shows that the system proposed has the reproducibility. With the proposed automatic high-throughput fruit phenotyping system, it is able to obtain the fruits distribution and growing information. By recording and analyzing the numerical growing statistics, farmers are able to do the field estimation and improved the growing skill to increase the yield.

並列關鍵字

Automation ； Computer vision ； Object detection ； Object tracking ； Phenotyping

參考文獻

Aich, S., & Stavness, I. 2017. Leaf counting with deep convolutional and deconvolutional networks. In “Proc. of the 2017 IEEE International Conference on Computer Vision Workshops (ICCVW)”, 22-29. Venice, Italy.

Andriyenko, A., Roth, S., & Schindler, K. 2011. An analytical formulation of global occlusion reasoning for multi-target tracking. In “2011 IEEE International Conference on Computer Vision Workshops”, pp. 1839-1846. IEEE.

Bargoti, S., Underwood, J. 2016. Deep Fruit Detection in Orchards. arXiv preprint arXiv: 1610.03677.

Bordes, A., Chopra, S., & Weston, J. 2014. Question answering with subgraph embeddings. arXiv preprint arXiv:1406.3676.

Breitenstein, M. D., Reichlin, F., Leibe, B., Koller-Meier, E., and Van Gool, L. 2009. Robust tracking-by-detection using a detector confidence particle filter. In “Proc. IEEE Int. Conf. Comput. Vis.”, pp. 1515–1522.