基於YOLO之自動傾斜車牌辨識系統

An Effect Method for Automatic Oblique License Plate Recognition System Based on YOLO

李威震(Wei-Chen Li)

指導教授 ： 王周珍

共同指導教授 ： 黃克穠(Ke-Nung Huang)

義守大學/智慧科技學院/電子工程學系/博士(2022年)

摘要

近年來，採用YOLO系列[5]的自動車牌辨識(automatic license plate recognition: ALPR)系統，已被廣泛應用在現今的智慧交通，一般ALPR系統對矩形車牌辨識有很高的準確度，然而，當車牌因轉彎或攝影角度變成傾斜時，則常因無法擷取正確車牌區域，導致後續字元辨識準確度下降。為了克服傾斜車牌的辨識問題，最近Silva等人利用仿射轉換(affine transform)提出一扭曲平面物件檢測(warped planar object detection: WPOD)網路[9]，來克服傾斜車牌辨識的問題，但因置信度(confidence)在傾斜框的計算相當複雜，所以WPOD的損失函數(loss function)忽略置信度，導致預測時常無法選取最佳車牌區域，進而影響後續車牌字元的辨識。首先，我們提出改良型WPOD 網路，利用傾斜車牌預測框的四頂點座標連接成矩形外接框來估算IOU(intersection over union)，再利用IOU來計算置信度，並將置信度加入損失函數中，來選取更準確的車牌區域，進一步提高車牌辨識率。雖然所提改良型WPOD網路利用簡易IOU計算，可獲得更精準的損失函數值，但所估算的IOU值與實際IOU仍有一定的誤差。因此，本論文進一步實驗不同頂點數所組成的矩形外接框，發現在不同傾斜車牌框的情況下，其他頂點數的外接框會比四點外接框的IOU誤差更低。因此，我們分析各種傾斜框所對應之最佳頂點數的矩形外接框，並建立所相對應的仿射係數表，最後利用查表找到最佳外接矩形框，來獲得更精準傾斜框IOU值。由實驗結果得知，我們所提改良型和查表型WPOD的平均IOU值分別達到0.810及0.837，後續字元辨識的平均準確率達到91.245%和92.586%，和WPOD網路比較，則字元辨識分別提高0.757%和2.098%。

關鍵字

無

並列摘要

In recent years, automatic license plate recognitions (ALPR) based on YOLO [5] detector have been widely applied in intelligent transportation system. However, the new ALPR is very difficult to obtain high detection and recognition rates for oblique license plate (LP). Recently, Silva et al. proposed a warped planar object detection (WPOD) network based on YOLO detector to overcome the oblique views of LP [9]. Although the WPOD network can achieve the location and rectification of oblique LPs, the loss function of WPOD neglects the confidence parameter due to high computational complexity. This also leads to WPOD network cannot locate the optimal LP bounding box. Therefore, we firstly proposed a modified WPOD (MWPOD) network to further improve the accuracy of oblique LP detection. The MWPOD makes use of four-vertex coordinates of prediction bounding box of the oblique LP to generate a rectangular bounding box, so that it easily estimates intersection over union (IOU). And then, the value of IOU is used to calculate the confidence and get a more precise loss function. From the simulation results, the MWPOD indeed can achieve a higher accuracy rate of optical character recognition (OCR) when oblique LP occurs as compared with WPOD.
Although our MWPOD network uses a simple IOU estimation to obtain more practical loss function, there is still some errors exist between the estimated IOU and the true IOU. In order to further increase accuracy rate of oblique LP detection, we firstly calculated the IOUs of rectangular outer bounding boxes which composed of different vertex coordinates, and found the error of estimated IOU by outer bounding box of three-vertex may be lower than the four-vertex in some oblique LPs. And then, we analyzed all constructed rectangular bounding boxes by different vertices which from prediction bounding box of oblique LP and found the best estimated IOU. Therefore, we can build the corresponding affine transform parameter table according to the best estimated IOU and propose a lookup table for WPOD (LUT-WPOD). The proposed LUT-WPOD takes lookup table to obtain a more accurate IOU value of oblique bounding box during the data training process. From the experimental results, we can find those average IOU values of MWPOD and LUT-WPOD networks are 0.810 and 0.837, respectively. In addition, the proposed both WPOD-based ALPR systems can obtain a high accuracy of OCR about 91.245% and 92.586%, respectively, on an average. On the other hand, the proposed MWPOD and LUT-WPOD can achieve higher recognition rate of OCR about 0.757% and 2.098% as compared to the original WPOD-based ALPR system, respectively.

並列關鍵字

none

參考文獻

[1] G. Hinton, L. Deng, D. Yu, et al., “Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups,” IEEE Signal Processing Magazine, vol.29, no.6, pp. 82-97, Nov. 2012.

[2] Y. Lecun, et al., “Backpropagation Applied to Handwritten Zip Code Recognition,” Neural Computation, vol. 1, no. 4, pp. 541-551, 1989.

[3] R. Girshick, J. Donahue, T. Darrell and J. Malik, “Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation,” 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 580-587, 2014.

[4] S. Ren, K. He, R. Girshick and J. Sun “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, June 2017

[5] J. Redmon, S. Divvala, R. Girshick and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, pp. 779-788, 2016.