應用於傾斜車牌辨識系統之研究

License plate detection and recognition system based on convolutional neural network

許庭瑄(Ting-Hsuan Hsu)

指導教授 ： 王周珍

共同指導教授 ： 周培廉(Pei-Lien Chou)

義守大學/電機資訊學院/電子工程學系/碩士(2019年)

摘要

近年來，深度學習(deep learning)廣泛被應用在自動車牌辨識(automatic license plate recognition :ALPR )系統，然而傳統 ALPR 大多截取正面的車牌，才能達到高準確的辨識率，這是因為傾斜車牌將使得傳統 ALPR 無法截取正確的車牌區域，導致後續車牌字元辨識發生錯誤或字元遺漏，大幅降低車牌辨識的準確率。因此，Silva 等學者最近提出一扭曲平面物件檢測(warped planar object detection: WPOD)網路來克服傾斜車牌的問題[12]，為了實現高準確率的 ALPR 系統，他們提出的 ALPR 架構主要分成三個部分，首先透過 YOLO(you only look once)來定位車輛[6]，接著利用 WPOD 網路來進行傾斜車牌定位和轉正，最後將轉正車牌輸入字元辨識網路來辨識出車牌上的字元。
雖然 WPOD 網路可以完成車牌定位和轉正，來大幅提高傾斜車牌辨識的準確率，但是 Silva 等學者在設計 WPOD 網路的損失函數(loss function)時，因為置信度(confidence)的計算相當複雜，所以在損失函數中並沒有採用置信度參數，這也導致網路在預測時，可能選擇到非最佳的車牌框，使後續的字元辨識網路容易發生辨識錯誤或出現字元遺漏的情形。為了提高 WPOD 網路車牌辨識的準確率，本論文提出一改良型 WPOD 網路，我們首先推導出簡易 IOU(intersection over union)的計算來快速獲得置信度，並將置信度加入損失函數，來完成更準確的車牌辨識網路。
本論文利用傾斜車牌預測框的四頂點座標來產生矩形框，再透過簡易 IOU運算，可以快速的獲得置信度參數，完成更精準的損失函數計算。因此，我們所提改良型 WPOD 網路，可以很容易將傾斜的 IOU 帶入損失函數中，完成更高的車牌辨識率。將所提方法與 WPOD 在相同公開數據集進行比較，從測試結果可以發現，我們在 IOU 的評分比高於 WPOD 平均約 3%，而在後續字元辨識的準確率，論文所提方法除了可以達到 95%的準確率外，也比 WPOD 系統的準確率平均高約 1%。

關鍵字

none

並列摘要

In recent years, automatic license plate recognition (ALPR) system is applied in some traffic-related applications based on deep learning. However, most ALPR systems capture a mostly frontal view of the vehicle and license plate (LP) to obtain high LP recognition rates. This is because the traditional ALPR cannot capture the correct area of oblique LP which results in an error in character recognition or missing characters. As a result, the traditional ALPR will largely reduce the accuracy of recognition for oblique LP. Recently, Silva et al. [12] proposed a warped planar object detection (WPOD) based on convolutional neural network (CNN) to overcome the oblique views of LP. In order to achieve an ALPR system of high accuracy, they divided ALPR into three stages. The first stage is to locate vehicles through YOLOv2 [9]. And then, the second stage locates the oblique LPs and allows a rectification of the LPs area to a rectangle which resembles a frontal view through the WPOD network. Finally, the rectified LPs are fed to an optical character recognition (OCR) in the third stage.
Although the WPOD network can achieve the location and rectification of LPs, the loss function of WPOD render the confidence parameter due to high computational complexity. This also leads to WPOD network cannot locate the optimal LP bounding box. In order to further improve the accuracy of ALPR system, we proposed a modified WPOD network using a complete loss function. The proposed method first develop a simple intersection over union (IOU) algorithm to speed up the calculating process of confidence. Therefore, the modified WPOD network can obtain higher LP recognition rate since it considers the confidence parameter in loss function.
In this thesis, the four-vertex coordinates of the label bounding box and prediction bounding box of oblique LP are used to generate two rectangular boxes, and then a simple IOU algorithm is used to fast calculate the approximate value of IOU. As a result, a more exact loss function can be finished. In order to compare the performance of LP recognition rate, we train and test the WPOD and the proposed modified WPOD in databases including OpenALPR EU, BR [19], and AOLP RP [17]. Simulation results show that the proposed ALPR system can obtain higher score ratios than those of Silva’s method. And the proposed system can arrive a high accuracy of LP recognition about 95% on an average. In addition, the proposed system also can achieve higher recognition rate about 1% when compared to the Silva’s ALPR system.

並列關鍵字

none

參考文獻

[1] Kaggle is an online community of data scientists and machine learners, owned by Google LLC, https://www.kaggle.com/

[2] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and F. F. Lin, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision (IJCV), pp. 211-252, 2015

[3] A. Krizhevsky, I. Sutskever, and G. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, pp. 1097-1105, 2012

[4] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 580-587, 2014

[5] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in Advances in neural information processing systems, pp. 91-99, 2015