智慧傾斜車牌辨識系統之研究

A study of intelligent oblique license plate detection and recognition system

曾貫惟(Kuan-Wei Tseng)

指導教授 ： 王周珍

共同指導教授 ： 周培廉(Pei-Lien Chou)

義守大學/電機資訊學院/電子工程學系/碩士(2020年)

摘要

深度學習(deep learning)已廣泛被應用在自動車牌辨識(automatic license plate recognition: ALPR)系統，然而現今常用的ALPR系統，大多數僅適用於固定環境且正面車牌的情況，但在實際的行車監視系統中，常會遇到傾斜車牌的問題，如路口監視器或手持拍攝，一般的ALPR系統對於傾斜車牌，通常無法擷取正確車牌區域，導致後續字元辨識準確度下降。
為了解決此一問題，最近有很多學者提出改善傾斜車牌辨識的方法，在2018年，Silva等人提出一扭曲平面物件檢測(warped planar object detection: WPOD)網路[11]，主要是利用仿射轉換(affine transform)來克服傾斜車牌辨識的問題，但是他們在設計WPOD網路的損失函數(loss function)時，因為置信度(confidence)在傾斜框的計算相當複雜，所以沒有採用置信度參數，導致在網路預測時，可能無法選取到最佳車牌區域，以至於不能準確完成傾斜車牌辨識。為了進一步提高傾斜車牌辨識的準確度，最近Hsu提出改良式WPOD 網路[12,13]，利用傾斜車牌預測框的四頂點座標產生矩形外接框，以方便估算IOU(intersection over union)，再利用所得IOU來計算置信度，並將置信度參數加入損失函數，來完成更精準的損失函數計算，使後續傾斜車牌區域的預測更加精準，達到更高的車牌辨識率。

雖然改良式WPOD網路利用簡易IOU估算，雖可獲得更精準的損失函數值，但所估算IOU值與實際IOU值仍有一定的誤差。因此，本論文經由不同頂點數(三點或四點)所組成的外接矩形框來進行IOU估算，發現在不同的車牌傾斜框情況下，其他頂點數的外接框會比四點外接框IOU的誤差值更低。所以，我們分析各種傾斜框所對應之最佳頂點數的外接框，並建立所相對應的仿射係數表，之後在訓練過程中利用查表(look-up table)來獲得更精準的傾斜框IOU值。由實驗結果得知，我們所提出方法的IOU平均量測值達到0.794，皆比Silva和Hsu的IOU值高，而在後續字元辨識的平均準確率達到91.85%，分別比Silva提高2.27%和比Hsu提高1.24%。

關鍵字

none

並列摘要

Deep learning has been widely used in automatic license plate recognition (ALPR) systems. However, a commercial ALPR systems only suitable for a fixed environment with front license plate (LP), but it usually fails in recognition when an oblique license plate (OLP) occurs. This is because a traditional ALPR system is difficult to capture the correct area of oblique license plate. Therefore, it also leads to reduce the accuracy of following optical character recognition (OCR).
In order to solve this problem, some studies had been proposed to improve the accuracy of OCR for oblique license plate. In 2018, Silva et al. proposed a warped planar object detection (WPOD) network based on affine transform to overcome the problem of OLP detection. However, the WPOD didn’t consider the confidence parameter in loss function due to high computational complexity. This also leads to the accuracy of WPOD network cannot further increase since it is unable to locate the optimal LP bounding box. Recently, Hsu proposed a modified WPOD (MWPOD) network [12-13] to further improve the accuracy of OLP detection. He firstly makes use of four-vertex coordinates of the OLP prediction bounding box to generate a rectangular bounding box and estimate intersection over union (IOU). And then, the estimated value of IOU is easily used to calculate the confidence and get a more precise loss function. From the simulation results, the MWPOD indeed can achieve a higher accuracy rate of OCR recognition when OLP occurs.
Although the MWPOD network uses a simple IOU estimation to obtain more practical loss function, there is still some errors exist between the estimated IOU and the actual IOU. Therefore, to further increase accuracy rate of OLP detection, we firstly analyzed the outer bounding boxes composed of different vertices, and found the error of estimated IOU by three-vertex outer bounding box is lower than the four-vertex outer bounding box in some OLPs. Secondly, we analyze all constructed rectangular bounding boxes by every vertex outer bounding box of OLP and find the best estimated IOU. And then, we build the corresponding affine parameter table according to the best estimated IOU. Finally, we take a look-up table to obtain a more accurate oblique bounding box IOU value during the data training process. From the experimental results, we can find that an average IOU value of our method is 0.794 which is higher than both of Silva’s and Hsu’s methods. In addition, the proposed ALPR system can obtain a high accuracy of OCR about 91.85% on an average, which it can achieve higher recognition rate about 2.27% and 1.24% as compared to Silva’s and Hsu’s system, respectively.

並列關鍵字

none

參考文獻

[1] M. Nielsen "Neural Networks and Deep Learning" http://neuralnetworksanddee plearning.com/

[2] Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document recognition," in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998, doi: 10.1109/5.726791.

[3] A. Krizhevsky, I. Sutskever, and G. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, pp. 1097-1105, 2012

[4] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and F. F. Lin, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision (IJCV), pp. 211-252, 2015

[5] R. Girshick, J. Donahue, T. Darrell and J. Malik, "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation," 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, 2014, pp. 580-587, doi: 10.1109/CVPR.2014.81.