PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting

Tung Duy Luu (1) , Jalal Fadili (1) , Christophe Chesneau (2)

1 Equipe Image - Laboratoire GREYC - UMR6072
2 LMNO - Laboratoire de Mathématiques Nicolas Oresme

In this paper, we consider a high-dimensional non-parametric regression model with fixed design and i.i.d. random errors. We propose a powerful estimator by exponential weighted aggregation (EWA) with a group-analysis sparsity promoting prior on the weights. We prove that our estimator satisfies a sharp group-analysis sparse oracle inequality with a small remainder term ensuring its good theoretical performances. We also propose a forward-backward proximal Langevin Monte-Carlo algorithm to sample from the target distribution (which is not smooth nor log-concave) and derive its guarantees. In turn, this allows us to implement our estimator and validate it on some numerical experiments.

Mots clés: High-dimensional regression; sparse learning; exponential weighted aggregation; group-analysis sparsity; group-analysis sparse oracle inequality; frame; forward-backward Langevin Monte-Carlo algorithm

Domaines: Statistiques [math.ST]

Tung Duy Luu, Jalal Fadili, Christophe Chesneau. PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting. 2016. ⟨hal-01367742v1⟩