一個以自動編碼為基礎及無監督式的手寫數字分群方法

An Autoencoder-Based and Unsupervised Method for Handwritten Digits Clustering

葉翰挺(Han-Ting Yeh)

指導教授 ： 劉長遠

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2017年)

https://doi.org/10.6342/NTU201700622

摘要

這篇論文提出了一個以2個模組組合成的手寫數字分群分法，第1個模組是堆疊稀疏自動編碼器，第2個模組是以空間拓撲距離測量為基礎的手寫字辨認器，這2個模組分別屬於無監督式及監督式的方法，重點是本文作者把它們變成了1個全無監督式的分群方法。
與現今流行的深度架構不同，本方法採橫向思維，以較淺但擴充神經元的數目來做，避開深度架構前層2個群沒分開，後層就分不開的問題。
本方法使用在60000個手寫數字的MNIST DataSet上分群結果為77.4%，超過了相關論文的76%。而且有現成的方法能再提升效能。
本方法的優點是一個模組化的設計，輸入的手寫數字通過自動編碼器模組抽取出數字樣本特徵後，再交給手寫字辨認模組做分群，不但組成了全無監督式的分群方法，稍加訓練可以再變成分類器使用。模組化的設計使得功能更彈性靈活，任何新的技術的出現可以用替換模組來改變功能或提升效能。
豐富的應用是另一個優點，本方法不只能用在手寫數字的分群，隨著分群過程中亦衍生出3種應用: (1) 在一堆資料中找出標準樣本，例如標準字形、圖形等。(2) 在圖像中搜尋像標準樣本的東西，例如掃描空拍圖搜尋像標準數字的地形。(3) 使用者定義搜尋，例如以圖搜圖。
此外只要變更訓練的資料庫，它也能應用在別的領域，例如在音頻上的分群及搜尋。

關鍵字

無監督式分群分法 ； 使用者定義搜尋 ； 稀疏自動編碼器 ； 淺度學習 ； 手寫數字辨識 ； 空間拓樸距離

並列摘要

This thesis presents a handwritten digits clustering method consisting of two modules. The first module is a Stacked Sparse Autoencoder, and the second module is a Handprinted Character Recognizer Based on Spatial Topology Distance Measurement. These two modules are unsupervised and supervised method, respectively. The point is that the author of this thesis converted them into a fully unsupervised clustering method.
Different from the presently popular deep structure, this method adopts transverse thinking and chooses to do with shallower structure but expanded number of neurons. The purpose is to avoid the problem in the deep structure that if two clusters are not separated in front layer they will not be separated in rear layers.
This method is applied to the MNIST dataset with 60000 handwritten digits and the clustering accuracy is 77.4%, more than 76% of the related paper. Furthermore, there is a ready-made way to improve performance.
The advantage of this method is a modular design. The digit template features are first extracted by the autoencoder module from input handwritten digits, and then handed over to the handwritten character recognition module to do the clustering. They not only form the fully unsupervised cluster method, but with a little training it can be transformed into a classifier for use. Modular design makes the function more elastic and flexible. With the emergence of any new technology it can replace the module to change the function or improve performance.
Rich application is another advantage. This method not only can be used in handwritten digital clustering, but also derives three kinds of applications from clustering process: (1) Find standard templates in a pile of data, such as standard characters, graphics etc. (2) Search for things like standard template in images, such as scanning satellite images to search for terrains like standard digits. (3) user-defined search, such as search by image.
In addition, by only change the training database, it can also be applied in other areas, such as clustering and search in audio.

並列關鍵字

Unsupervised Clustering method ； User defined search ； Sparse Autoencoder ； Shallow Learning ； Handwritten Digits Recognition ； Spatial Topology Distance

參考文獻

[1] Jain, A.K., Murty, M.N., Flynn, P.J. “Data clustering: a review”. ACM Computing

[3] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. “Gradient-based learning applied to document recognition”, Proceedings of the IEEE, november 1998.

[5] G.E. Hinton and R.R. Salakhutdinov. “Reducing the Dimensionality of Data with Neural Networks”. Science (New York, N.Y.), 313:504–507, 2006.

[7] David E. Rumelhart, James L. McClelland, CORPORATE PDP Research Group. “Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations”. MIT Press Cambridge, 1986

[8] Geoffrey E. Hinton.“Training products of experts by minimizing contrastive divergence”. Neural Computation, 14(8):1771–1800, 2002