透過Lasso於高維度線性模型之下 訊號的重要性判讀

Lasso and Its Oracle Properties in Importance Assessment of Regressors

戴齊賢(Chi-Shian Dai)

指導教授 ： 陳宏

國立臺灣大學/理學院/應用數學科學研究所/碩士(2016年)

https://doi.org/10.6342/NTU201600735

摘要

在高維度線性稀疏模型下，Lasso的軌跡解是相當複雜的。採用Lasso的同時，為了選擇微調參數$lambda$，研究Lasso的軌跡解是極為重要的。本文中，當微調參數$lambda$從無限大遞減時，我們給了一個充分條件使得Lasso的軌跡解的支集遞增直到此支集覆蓋住稀疏集(sparsity patten)。根據這個性質，我們可以用變異數分解法來決定微調參數$lambda$要選多少。
Lockhart(2014) 為了決定微調參數$lambda$而提出了 extit{共變異檢定統計} (covariance test statistic)。然而，他們並沒有強調:是否此序貫假設檢定能一路拒絕虛無假設，直到Lasso的解的支集覆蓋住稀疏集(sparsity patten)?為了回答這個問題，我們證明了Lasso的軌跡解有以下的特性，當微調參數$lambda$從無限大遞減時，變數會根據訊號強弱依序進入Lasso的支集。
而我們可以根據進入Lasso的支集的先後來判斷訊號的重要性。此外根據以上的性質，我們可以證明 extit{共變異檢定統計}不會乏適 (underfitting).

關鍵字

高維度線性模型 ； 壓所感知 ； 模型選擇 ； L1 正規化 ； 最小角回歸 ； 重要性判別 ； 共變異檢定統計

並列摘要

In the high dimensional sparse linear regression setting, the Lasso solution path would be rather complicated. And while exploiting Lasso, to choose the tuning parameter $lambda$, analyzing the Lasso solution path is crucial. In this thesis, as the tuning parameter $lambda$ decreases from infinity, we provide a sufficient condition such that the support of the Lasso solution increases until its support recover the sparsity pattern. With this property, exploiting variance decomposition could determine which parameter $lambda$ should we choose.
Lockhart2014 proposed the covariance test statistic of Lasso to choose the parameter $lambda$. However, they pay little attention to whether the sequential hypothesis testing could reject the null hypothesis until it recovers the sparsity pattern. In order to deal with this issue, we show that the Lasso solution path would have the ordering property by which we could assess the importance of regressors and the assessment is identical to the oracle importance. And with this property, covariance test statistic would not be underfitting.

並列關鍵字

High dimensional linear regression ； Model selection ； L1-constraints ； Compressed sensing ； Lasso path ； Least angle regression ； important assessment.

參考文獻

Davidson, K. R. and Szarek, S. J. (2001). Local Operator Theory, Random Matrices and Banach Spaces. Handbook of the Geometry of Banach Spaces, 1:317–366.

Laurent, B. and Massart, P. (2000). Adaptive estimation of a quadratic functional by model selection. Annals of Statistics, 28(5):1302–1338.

Lockhart, R., Taylor, J., Tibshirani, R. J., and Tibshirani, R. (2014). A significance test for the lasso. Annals of Statistics, 42(2):413–468.

Tibshirani, R. (1994). Regression Selection and Shrinkage via the Lasso. Journal of the Royal Statistical Society B, 58(1):267–288.

Tibshirani, R. J. (2013). The lasso problem and uniqueness. Electronic Journal of Statistics, 7(1):1456–1490.