- 學位論文
以手部結構引導三維沙漏式網路預測手部姿勢
Hand Structure Guided 3D Hourglass Network for Hand Pose Estimation
摘要
近年來，因為AR與VR的蓬勃發展，為了讓使用者能便利的操作人機介面，手部姿勢的辨識成為了熱門的研究主題。過去主要使用影像處理搭配機器學習的方法來解決手部辨識的問題，但由於手指過於相似和手部自身遮蔽等問題，導致機器學習的方法一直得不到太好的結果，近年來因為類神經網路的崛起，手部辨識的準確率得到了大幅的提升，也研究出了許多不同的深度學習模組，但大部分的學習方法都是著重於學習預測出最後手部關節點的位置，但未考慮到輸入的資料未包含完整的手部結構，導致有時會出現手部節點嚴重錯誤的情形。 因此在本論文中，我們想探討在原本類神經網路中加入整體手部結構的學習，透過預測出完整的近似手部模型進而使網路提升準確率。在原始的手部預測中，我們加入了一個平行的分支模組去預測整體手部的模型，並將原本的關節點預測結果、近似手部模型和原始輸入特徵合併，並將其當成輸入再預測一次手部節點的位置，當成最後的輸出結果。
並列摘要
In recent years, since augment reality (AR) and virtual reality (VR) become more and more popular, to produce a more conveniently Human Machine Interface, hand pose estimation become a popular study topic. In the past, it was usually using image processing and machine learning to estimate hand pose, but self-occlusions and similar fingers’ shape will cause the inaccurate estimation. Recently, there are many different researches that created different network models based on convolution neural networks (CNNs) produce better performance than before in hand pose estimation, but most of them trained the network model without considered that input dose not showed the complete hand structure, so that makes structure error in some cases. In this study, we want to estimate a complete hand model to complement hand structure, and we hope that it can make learning more accurately and improve some structure error cases. In our work, we add a new branch to learn the complete hand structure and output a hand model, then we concatenate the original input, hand joints estimation and hand model as a new input to re-estimate final hand joints result.