大規模分解機器之最佳化方法比較

A Comparison of Optimization Methods for Large Scale Factorization Machine

張智堯(Chih-Yao Chang)

指導教授 ： 林智仁

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2017年)

https://doi.org/10.6342/NTU201704285

摘要

近年來，非凸最佳化問題變得相當熱門。非凸最佳化問題是個充滿許多未知數的領域，也非常值得花費力氣去研究最佳化方法在這類問題上的行為模式。此外，分解機器也漸漸廣泛地被使用在各類型的應用上面，特別是推薦系統。為了深入了解，我們分析了交替牛頓法以及常見方向在分解機器上的行為。本作品的主要貢獻是：詳細的比較了模型之間的相對目標函式值、訓練時間、偽數據遍歷。實驗結果顯示交替常見方向在收斂速度上較交替牛頓法來的快。

關鍵字

交替架構 ； 牛頓法 ； 常見方向方法 ； 分解機器

並列摘要

Recently, non-convex optimization has been a popular domain. Non-convex optimization
is a domain full of unknowns and it is worth investigating behaviors of
optimzation techniques on such kind of problems. Also, Factorization Machine has
also been a popular model in many applications, especially for recommendation systems.
To know the details, we analyze the behaviors of alternating Newton method
(ANT) and alternating common-directions method on the model. In this work, we
compare their relative objective function value, training time, and pseudo data passe.

並列關鍵字

alternating framework ； Newton method ； common-directions method ； Factorization Machine

參考文獻

S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press,2004.

R. H. Byrd, G. M. Chin, W. Neveitt, and J. Nocedal. On the use of stochastic Hessian information in optimization methods for machine learning. SIAM J. Optim., 21(3):977-995, 2011.

S. Rendle. Factorization machines. In ICDM, 2010.

C.-C. Wang, C.-H. Huang, and C.-J. Lin. Subsampled Hessian Newton methods for supervised learning. Neural Comput., 27:1766-1795, 2015.

M. Blondel, M. Ishihata, A. Fujino, and N. Ueda. Polynomial networks and factorization machines: new insights and e cient training algorithms. In Proceedings of the 33rd International Conference on Machine Learning (ICML), 2016.