運用卷積類神經網絡技術於海上物件偵測與方向估計之研究

Study of the Object Detection and Direction Estimation Using Convolutional Neural Network in the Sea

王新誠(Sin-Cheng Wang 王新誠)

指導教授 ： 林春宏

共同指導教授 ： 劉冠顯(Kuan-Hsien Liu 劉冠顯)

國立臺中科技大學/資訊與流通學院/資訊工程系碩士班/碩士(2019年)

摘要

在現今海上的交通發展中，無人船隻的海上監控系統已經成為非常重要的一環。無論是在軍事用途或在搜救的運用上，對於在海面上的無人船隻，都能夠於加強敏感區域進行監督與管理，以防止產生海事糾紛或查處違法之行為，同時也能在海上的民生用途帶來更多的便利性。無人船隻在海面上航行時，由於海上的天候多變難以掌控，造成船隻航行不易，加上航行中可能遇到大型漂流物的阻擋、海底動物的衝撞或海面下的暗礁等異物，導致發生船隻的事故。因此，本研究主要目的係解決無人船隻於海面上航行及巡邏時，能夠精準且快速的偵測海面上的其他船隻，並測量無人船與其他船隻之間的距離與方向，進而啟動必要的避障處理。
本研究主要分為三個部分的處理，第一部分的處理是利用海平面線面作為基準，並對無人船隻拍攝到的影像，進行海平面線偵測及影像穩定的處理；第二部分的處理係利用卷積類神經網路(convolutional neural network, CNN)對海面上的其他船隻進行自動化偵測，第三部分的處理則利用船隻上裝設的雙鏡頭攝影機，進行海上物件距離與方向的偵測。其中第三部分的處理，首先對相機做校正(camera calibration)，再利用雙鏡頭攝影機所拍攝到的左右影像，進行匹配後即可得到立體深度影像(stereo deep image)，並稱為立體視覺(stereo vision)。此視覺原理係透過左右攝影機所拍攝到的兩張影像之間的視差，進行距離資訊的計算，以此判斷出船隻在現實環境中的位置與方向。
最後，在實驗的部分，本研究使用兩類不同的影片，分別對海平面線的偵測與影像的穩定、物件的偵測以及立體匹配測距分別進行實驗。兩類的實驗影片，分別從網路上取得的影片，以及本研究在淡水灣所自行拍攝的影片。本文在海平面線偵測的部分偵測率佳，正確率皆能夠達到91%以上，並且在影像穩定的部分表現良好，讓影片整體看起來平穩許多。在物件偵測的部分，無論船隻遠近皆能夠有良好的偵測，並且正確偵測出該物件所在位置。最後，在立體匹配測距的部分，結合物件偵測所框出的位置，以及立體匹配計算出的視差圖，本研究預測的物件距離誤差值皆能夠控制在10%以下。經過實驗證明，本研究在海平面線的偵測與影像的穩定、物件的偵測以及立體匹配測距的實驗結果上，其處理的精確度與時間都有不錯的效果。期望本研究提出的方法，在未來能夠真正應用在海上無人船隻的自動避障上，進而達到無人船隻於海上進行搜救或軍事的監督與管理之用途。

關鍵字

無人船 ； 海平面線 ； 物件偵測 ； 卷積類神經網路 ； 深度影像 ； 雙鏡頭攝影機

並列摘要

In today's development of maritime traffic, the unmanned marine monitoring system has become a very important factor. Whether in military use or in the use of search and rescue, unmanned vessels on the sea can strengthen the monitoring and management of sensitive areas to prevent maritime disputes, investigate illegal activities, and also for aiding people's livelihood activities at sea, bringing more convenience. When an unmanned vessel sails on the sea, it is difficult to control due to the changing weather on the sea, rendering it difficult for the vessel to sail. In addition, it may encounter the blocking of large drifting objects during navigation, collision with underwater animals or the reef under the sea, causing ship accidents to occur. Therefore, the main purpose of this study is to solve the above problems, so that the unmanned vessels can accurately and quickly detect other ships on the sea when navigating and patrolling on the sea, and measure the distance and direction between the unmanned ship and other ships, thus avoiding barrier encounters.
This study is mainly divided into three parts. The first part uses the sea level line as the reference to process the images captured by the unmanned ship, sea level line detection and image stabilization. The second part of the processing uses a convolutional neural network (CNN) to automatically detect other nearby ships. The third part uses a two-lens camera mounted on the ship to detect the distance and direction of objects on the sea for measurement purposes. In the third part of the process, the camera calibration is first corrected camera, and then the left and right images captured by the dual-lens camera are matched to obtain a stereo deep image, called stereo vision. This visual principle is based on the parallax between the two images captured by the left and right cameras, and; the distance information is calculated is employed to determine the position and direction of the vessel in the real environment.
Finally, in the experimental part, this study used two different types of films, independently, to detect the sea level line, achieve image stability, object detection and stereo matching ranging. The two types of experimental film are videos taken online, and films taken by the study itself in Tamsui Bay. In this paper, the detection rate of sea level line detection is good, as the correct rate can reach more than 91%, and the performance in the stable part of the image is also good, making the overall picture look much smoother. In the object detection part, the vessel and the position of the object can be well detected regardless of the distance. Finally, in regard to the stereo matching ranging, combined with the position of the object detection and the disparity map calculated by the stereo matching, the error value of the object distance predicted in this study can be controlled below 10%. Experiments have proven that the accuracy and time required for the processing have good effects on the detection of sea level lines and the stability of images, object detection and stereo matching ranging. It is expected that the method proposed in this study can be applied to the automatic obstacle avoidance of unmanned vessels at sea in the future, thereby achieving the purpose of unmanned vessels for search and rescue or military supervision and management at sea.

並列關鍵字

unmanned surface vehicle ； sea horizon ； object detection ； convolutional neural network ； depth image ； 3D camera.

參考文獻

參考文獻

[1] 地理大發現 Wiki 百科 網址：https://zh.wikipedia.org/wiki/%E5%9C%B0%E7%90%86%E5%A4%A7%E5%8F%91%E7%8E%B0, 搜尋日期：2018年11月。

[2] Unmanned Marine Systems. 網址：https://www.unmannedsystemstechnology.com/company/autonomous-surface-vehicles-ltd/ , 搜尋日期：2018年11月。

[3] Piccardi, M. (2004, October). Background subtraction techniques: a review. In 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No. 04CH37583) (Vol. 4, pp. 3099-3104).

[4] Cheng, F. C., Huang, S. C., & Ruan, S. J. (2010, July). Advanced background subtraction approach using Laplacian distribution model. In 2010 IEEE International Conference on Multimedia and Expo (pp. 754-759).