多級別特徵驅動之監視影像儲存管理系統

Multi-level Feature-driven Storage Management of Surveillance Videos

蔡旻翰(Min-Han Tsai)

指導教授 ： 徐正炘

國立清華大學/電機資訊學院/資訊系統與應用研究所/碩士(2021年)

摘要

在智慧場域中，隨著串流監視影像技術的興起，許多創新的分析應用也應運而生，其中包含了各式各樣能將純影像轉化為對使用者來說具有意義的結果。除了即時的串流服務，這些監視影像也被保存在存儲伺服器中，以提供未來使用者隨選式、自定義的分析請求。不同於現有專注於最大化影片品質的隨選式影音串流服務，監視影像存儲伺服器需要面對的是：在有限的空間和運算資源下決定並最大化儲存影片的資訊含量，同時還要替未來新進的影片預留空間。在本論文中，我們設計、實作、優化、並評估了一個多級特徵驅動存儲系統，該系統能提供各種規模的智慧場域使用，如智慧校園、建築、社區或城市。我們專注於此系統的設計和實作，並且解決了兩個核心的問題：（一）有效率地捕捉新進影片的資訊含量（二）聰明地決定影片保存的品質。我們首先採取近似分析的方式推算出影片的資訊含量，並使得伺服器免於過載的問題。此近似分析的算法基於多級特徵（語意及視覺特徵）正式定義所謂的「資訊量」。接著，我們根據此量化的數值決定最佳降採樣的方法和影片的目標保存品質，目的在於最大化存儲系統中的的總資訊量。我們嚴格地制定上述兩個研究問題並以數學方法將其轉化為最佳化問題。對此兩個問題，我們分別給出了最佳、近似、和高效共六個算法。除了一系列優化的算法，我們也利用清華大學的物聯網測試平台中的影片評估我們提出的系統。此平台包含了八支裝有各式感測器的智慧路燈，其中四支裝了監視錄影機，並且所有捕捉的影片都會送回至機房做儲存和分析。在實驗階段，我們以實際的影片評估我們提出的演算法的效能，而我們提出的解決方案在多方面都勝過了目前業界的做法，例如：（一）在捕捉資訊量上，達成了和最佳解僅有7\%的差距（二）在一週的實驗後存下了近三倍的影片數量（三）減少平均58\%的請求誤差（四）能在100毫秒內做出決定所有影片保存的品質（五）不超出系統可負荷的儲存空間（六）能適應各種規模的儲存空間。

關鍵字

物聯網 ； 邊端計算 ； 機器學習 ； 電腦視覺 ； 監視影像 ； 儲存系統

並列摘要

Surveillance videos in smart environments have become commodities nowadays, which enable many novel applications, including various video analytics that turn videos into semantic results. In addition to live feeds, the surveillance videos may be saved in a storage server for on-demand user-defined queries in the future. Different from on-demand video streaming servers whose design objective is to maximize the user-perceived video quality, a surveillance video storage server has limited space and must retain as much information as possible, while reserving sufficient space for incoming videos.
In this thesis, we design, implement, optimize, and evaluate a multi-level feature driven storage server for diverse-scale smart environments, for example buildings, campuses, communities, and cities.
We focus on the design and implementation of the storage server and solve two key research problems in it, namely: (i) efficiently determining the information amount of incoming videos and (ii) intelligently deciding the qualities of videos to be kept. In particular, we first analyze the videos to derive approximate information amount without overloading our storage server. This is done by formally defining the information amount based on multi-level (semantic and visual) features of videos. We then leverage the information amounts to determine the optimal downsampling approach and target quality level of videos to save storage space, while preserving as much information amount as possible. We rigorously formulate the above two research problems into mathematical optimization problems, and propose optimal, approximate, and efficient algorithms to solve them. Besides the suite of optimization algorithms, we also implement our proposed system on a smart campus testbed at NTHU, Taiwan, which consists of eight smart street lamps. The street lamps are equipped with a wide spectrum of sensors, network devices, analytics servers, and a storage server. We compare the performance of our proposed algorithms against the current practices using real surveillance videos from our smart campus testbed. Our efficient algorithms outperform the current practices in multiple dimensions, meaning we: (i) achieve a mere $7\%$ approximation gap on captured information amount compared to the optimal solutions, (ii) save almost 3 times more clips after a week, (iii) achieve $58\%$ less per-query error on average, (iv) always terminate in less than 100 ms, (v) do not consume excessive storage space, and (vi) scale well with larger storage spaces.

並列關鍵字

Internet-of-Things ； EdgeComputing ； MachingLearning ； ComputerVision ； SurveillanceVideo ； StorageSystem

參考文獻

[1] G. Ananthanarayanan, P. Bahl, P. Bod ́ık, K. Chintalapudi, M. Philipose, L. Ravindranath, and S. Sinha. Real-time video analytics: The killer app for edge computing. IEEE Computer, 50(10):58–67, October 2017.

[2] A. Anjomshoaa, F. Duarte, D. Rennings, T. Matarazzo, P. deSouza, and C. Ratti. City scanner: Building and scheduling a mobile sensing platform for smart city services. IEEE Internet of things Journal, 5(6):4567–4579, May 2018.

[3] K. Ashton et al. That Internet of Things thing. LLC RFID, 22(7):97–114, June 2009.

[4] A. Astaras, H. Lewy, C. James, A. Katasonov, D. Ruschin, and P. Bamidis. Unobtrusive smart environments for independent living and the role of mixed methods in elderly healthcare delivery: the USEFIL approach. In Handbook of Research on Innovations in the Diagnosis and Treatment of Dementia, chapter 15, pages 290–305. IGI Global, May 2015.

[5] Azure. Azure status history, 2021.