城市街景影像定位與多描述子之研究

City Street View Image Localization and Study on Multiple Descriptors Matching

陳治安(Zhi-An Chen)

指導教授 ： 劉震昌

國立暨南國際大學/科技學院/資訊工程學系/碩士(2017年)

摘要

環景影像相較於傳統拍攝影像，它可以記錄更多豐富的空間資訊，因此越來越多環景攝影的相關應用出現在生活中，例如：Google 街景（Google street view） 、無人機全景空拍、VR 搭配環景導覽。
自從 Google 啟動街景服務之後，使用者可以使用 Google 街景查詢景點，藉由該景點之記錄的環景影像，使用者可以清楚得知景點豐富的空間資訊，例如：商家資訊、道路資訊、車流量…等等，Google 街景的推出獲得非常廣大的迴響，因此拍攝紀錄街景具有非常大的價值。
本文延續論文 [1] 資料收集，持續以車載環景影像擷取設備 Ladybug3 [2]，以一座城市區域範圍，定期拍攝埔里街道環景影像與紀錄 GPS 座標，作為使用者查詢時的資料來源，從前述論文 [1] 開始資料收集至今，一共進行兩年約 22 次拍攝，目前原始視訊串流檔案已達 2.63 TB，轉換成 80004000 的全景影像 (equirectangular image) 約694,301 張，儲存至前端 NAS 伺服器，提供給使用者觀看。研究部分分為兩個階段，第一階段我們從記錄的環景影像中，由人工挑選取出具有標誌性的物件，並使用單一描述子 SIFT 擷取影像特徵，進行影像定位搜尋的應用實驗。第二階段我們參考多描述子融合 [3] 方法，結合多種描述子的特性後，進行多描述子與單一描述子進行效能比較和多描述子影像搜尋實驗。

關鍵字

環景影像 ； 街景 ； 影像定位 ； 多描述子

並列摘要

Panoramic image is difference from traditional image because it can record rich space information. More applications of panorama appear in our daily life, such as Google street view, panoramic recording using Unmanned Aerial Vehicle, virtual reality with panoramic guide.
Since Google started the service of street view, users can search attractions from Google street view by the Panoramic images which recorded from this attraction, and they can know rich space information from this attraction, such as information of stores, roads, traffic flow, etc. Google street view is praised from people after it was released, so recording street view has great value.
We continue the work [1] for collecting street view data in a city by the car mounted with Ladybug3 [2], and regularly recording the panorama of Puli street view and GPS locations. It will be the data source for image-based localization. Started from the work [1] until now, we recorded 22 times in 2 years, and the original video stream data is 2.63 TB, which transforming to 694,301 pieces of the equirectangular images in 8000 x 4000 resolution and stored in NAS server for users to query. Two applications were studied in this thesis. The first application is about image-based localization using the iconic images, such as the flags. We manually cut a set of iconic images from the recorded panoramic images, and used the SIFT descriptor to extract features for image search. The second study is about the method of Multiple Descriptors Fusion [3], which fuses many descriptors of different properties. An experiment was conducted for image-based localization using multiple descriptors, and the other experiment was to compare the performance using multiple descriptors and a single descriptor.

並列關鍵字

Panoramic Images ； Street View ； Image-Based Localization ； Multiple Descriptors

參考文獻

[1] 許耕瑄，劉震昌 (2016)。城市尺度之環景街道影像資料收集與影像定位。碩士論文，國立暨南國際大學。

[2] Ladybug3，https://www.ptgrey.com/ladybug3-360-degree-firewire-spherical camera-systems

[3] Y. -T. Hu and Y . -Y. Lin. “Robust feature matching via multiple descriptor fusion,” in Proc. Asian Conf. Pattern Recognit. , Nov. 2015.

[4] D. G. Lowe. “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis., vol. 60, no. 2, pp. 91–110, Nov. 2004.

[5] A. C. Berg and J. Malik, “Geometric blur for template matching,” in Proc. Conf. Comput. Vis. Pattern Recognit., 2001, pp. I-607–I-614.