Hierarchical Overlapping Group Lasso for GMANOVA Model

This paper deals with the GMANOVA model with a matrix of polynomial basis functions as a within-individual design matrix. The model involves two model selection problems: the selection of explanatory variables and the selection of the degrees of the polynomials. The two problems can be uniformly addressed by hierarchically incorporating zeros into the vectors of regression coefficients. Based on this idea, we propose hierarchical overlapping group Lasso (HOGL) to perform the variable and degree selections simultaneously. Importantly, when using a polynomial basis, fitting a highdegree polynomial often causes problems in model selection. In the approach proposed here, these problems are handled by using a matrix of orthonormal basis functions obtained by transforming the matrix of polynomial basis functions. Algorithms are developed with optimality and convergence to optimize the method. The performance of the proposed method is evaluated using numerical simulation.