以圖像分析方法辨識潛在異常施工活動

Identification of Potential Abnormal Construction Activities Using Image Analytics

林子皓(Zi-Hao Lin)

指導教授 ： 謝尚賢

共同指導教授 ： 陳柏華(Albert Y. Chen)

國立臺灣大學/工學院/土木工程學研究所/碩士(2020年)

https://doi.org/10.6342/NTU202001611

摘要

對營造業而言，增加利潤、降低成本是首要目標，意味著維持施工順暢極其重要。異常施工活動不僅容易拖延進度，且與人員安全息息相關。因此，本研究提出時序性圖像之分析方法，藉此辨識潛在異常活動，並視覺化呈現施工過程。分析流程共可分為四個模組：物件偵測、物件追蹤、行為辨識、施工活動分析。物件偵測模組採用遷移式學習訓練辨識模型，於影像中辨識工地人員及重機具，再透過物件追蹤模組匹配辨識到的物件。接著，行為辨識模組利用物件追蹤結果辨識出人員及重機具所執行的動作，再根據時間軸繪製成折線圖。施工活動分析模組採用統計理論從所有動作週期中過濾出潛在異常活動，並於折線圖中標註異常區塊。

本研究以實際於工地拍攝之開挖作業影片進行驗證，物件辨識模組達到70.30%之精度，物件追蹤模組達到82.12%之準確度。施工活動分析模組不僅辨識出影片中的潛在異常活動，亦過濾出卡車交班。為達到實際應用效益，該模組統計出施工過程各式資訊，同時標註潛在異常事件之起終時間，方便使用者回顧監視影像。透過折線圖紀錄並分析施工活動，工地主任將能快速回顧整個施工過程，探討異常事件發生原因，進而維持順暢施工，確保人員安全。

關鍵字

施工活動追蹤 ； 生產力分析 ； 深度學習 ； 物件偵測 ； 物件追蹤

並列摘要

To increase profit, ensuring project schedule is the primary objective for the construction industry. Abnormal activities may reduce productivity or even cause accidents. Therefore, this study adopts image analytics approach to automatically identify potential abnormal events and visualize construction process. The pipeline consists of four modules: object detection, object tracking, action recognition, and operational analysis. At first, Faster Region-proposal Convolutional Neural Network (Faster R-CNN) is adopted to detect workers and heavy equipment on the construction jobsites, and Simple Online and Realtime Tracking (SORT) algorithm is improved to associate detected objects between images. Afterwards, a hybrid model integrating CNN and Long Short Term Memory (LSTM) is employed for the purpose of action recognition. The results are documented into a line chart form of Crew-balance Chart (CBC) to visualize the construction process, in which irregular operations are identified through statistical theory. The approaches were validated with the videos of earthmoving operation. The Average Precision (AP) of the trained detector is about 70.31%, and the Multiple Object Tracking Accuracy (MOTA) of the modified SORT is around 82.12%. In the testing videos, not only potential abnormal activities were pre-screened, but truck exchanges were filtered. Meanwhile, an activity log with operational information and starting and ending times of irregular events was created. Through the line chart form of CBC and activity log provided by the image analytics approach, potential abnormal events can be deeply investigated for further enhancement.

並列關鍵字

Activity Tracking ； Productivity ； Deep Learning ； Object Detection ； Object Tracking

參考文獻

[1] Michael C. Gouett, Carl T. Haas, Paul M. Goodrum, and Carlos H. Caldas. Activity analysis for direct-work rate improvement in construction. Journal of Construction Engineering and Management, 137(12):1117–1124, 2011. doi: 10.1061/(ASCE)CO. 1943-7862.0000375.

[2] Jacob J. Lin and Mani Golparvar-Fard. Visual data and predictive analytics for proactive project controls on construction sites. In Workshop of the European Group for Intelligent Computing in Engineering, pages 412–430. Springer, 2018. doi: 10.1007/978-3-319-91635-4_21.

[3] CII (Construction Industry Institute). Guide to Activity Analysis. The University of Texas, Austin, Texas, 2010.

[4] Hyunsoo Kim, Changbum R. Ahn, David Engelhaupt, and SangHyun Lee. Application of dynamic time warping to the recognition of mixed equipment activities in cycle time measurement. Automation in Construction, 87:225–234, 2018. doi: 10.1016/j.autcon.2017.12.014.

[5] Ardalan Khosrowpour, Igor Fedorov, Aleksander Holynski, Juan Carlos Niebles, and Mani Golparvar-Fard. Automated worker activity analysis in indoor environments for direct-work rate improvement from long sequences of RGB-D images. In Construction Research Congress 2014: Construction in a Global Network, pages 729–738, 2014. doi: 10.1061/9780784413517.075.