Regularized Robust Optimization with Application to Robust Learning

William Piat (1, 2) , Jalal Fadili (1) , Frédéric Jurie (2) , Sébastien da Veiga (2)

1 GREYC - Groupe de Recherche en Informatique, Image et Instrumentation de Caen
2 Safran Tech

In this paper, we propose a computationally tractable and provably convergent algorithm for robust optimization, with application to robust learning. First, the distributional robust optimization is approached with a point-wise counterpart at controlled accuracy. Second, to avoid solving the generally intractable inner maximization problem, we use entropic regularization and Monte Carlo integration. The approximation errors induced by these steps are quantified and thus can be controlled by making the regularization parameters and the number of integration samples decay at an appropriate rate. This paves the way to minimizing our objective with stochastic (sub)gradient descent whose convergence guarantees to critical points are established without any need of convexity/concavity assumptions. To support these theoretical findings, compelling numerical experiments on simulated and benchmark datasets are carried out and confirm the practical benefits of our approach.

Mots clés: Robust optimization; Smoothing; Deep learning; Statistical learning

Domaines: Optimisation et contrôle [math.OC]; Intelligence artificielle [cs.AI]

William Piat, Jalal Fadili, Frédéric Jurie, Sébastien da Veiga. Regularized Robust Optimization with Application to Robust Learning. 2022. ⟨hal-03689825v2⟩