Deep Multi-Task Learning with evolving weights

Soufiane Belharbi (1, 2, 3, 4) , Romain Hérault (2, 3, 4, 1) , Chatelain Clement (2, 3, 4, 1) , Sébastien Adam (3, 2, 4, 5)

1 INSA Rouen Normandie - Institut national des sciences appliquées Rouen Normandie
2 DocApp - LITIS - Equipe Apprentissage
3 LITIS - Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes
4 NU - Normandie Université
5 UNIROUEN - Université de Rouen Normandie

Pre-training of deep neural networks has been abandoned in the last few years. The main reason is the difficulty to control the overfitting and tune the consequential raised number of hyper-parameters. In this paper we use a multi-task learning framework that gathers weighted supervised and unsupervised tasks. We propose to evolve the weights along the learning epochs in order to avoid the break in the sequential transfer learning used in the pre-training scheme. This framework allows the use of unlabeled data. Extensive experiments on MNIST showed interesting results.

Domaines: Machine Learning [stat.ML]; Réseau de neurones [cs.NE]

Soufiane Belharbi, Romain Hérault, Chatelain Clement, Sébastien Adam. Deep Multi-Task Learning with evolving weights. European Symposium on Artificial Neural Networks (ESANN), Apr 2016, Brugge, Belgium. ⟨hal-02345855⟩