Instance segmentation in fisheye images

Rémi Dufour (1) , Cyril Meurie (2) , Clément Strauss (1) , Olivier Lezoray (3)

1 IRT Railenium - L'Institut de Recherche Technologique (IRT) de la filière Ferroviaire
2 COSYS-LEOST - Laboratoire Électronique Ondes et Signaux pour les Transports
3 Equipe Image - Laboratoire GREYC - UMR6072

In this paper we propose a data augmentation method for instance segmentation in fisheye images. A lot of progress has been made on the task of instance segmentation in the last few years, particularly for rectilinear images. In fisheye images, detection tasks have mostly been explored as a semantic segmentation task. Instance segmentation in fisheye images is challenging and has not yet been fully explored. There is also much interest in the development of deep neural networks that can handle both rectilinear and fisheye images. Indeed, this can be interesting to have control on computing resource requirements, of paramount importance for real-time systems e.g., in transportation systems. This paper aims to explore these two challenges using Mask R-CNN trained with a data augmentation method designed to provide good performance on both rectilinear and fisheye images. We show that performance on fisheye augmented images can be increased by 9% while only decreasing performance on rectilinear images by 2%, and that performance on wide angle fisheye cameras can be increased by 18.4% compared to the reference, which provides more benefits than a simple vertical flip augmentation.

In this paper we propose a data augmentation method for instance segmentation in fisheye images. A lot of progress has been made on the task of instance segmentation in the last few years, particularly for rectilinear images. In fisheye images, detection tasks have mostly been explored as a semantic segmentation task. Instance segmentation in fisheye images is challenging and has not yet been fully explored. There is also much interest in the development of deep neural networks that can handle both rectilinear and fisheye images. Indeed, this can be interesting to have control on computing resource requirements, of paramount importance for real-time systems e.g., in transportation systems. This paper aims to explore these two challenges using Mask R-CNN trained with a data augmentation method designed to provide good performance on both rectilinear and fisheye images. We show that performance on fisheye augmented images can be increased by 9% while only decreasing performance on rectilinear images by 2%, and that performance on wide angle fisheye cameras can be increased by 18.4% compared to the reference, which provides more benefits than a simple vertical flip augmentation.

Mots clés: INSTANCE SEGMENTATION; DEEP LEARNING; DATA AUGMENTATION; Index Terms-fisheye images; instance segmentation; deep learning; data augmentation; FISHEYE IMAGES

Mots clés: TRAITEMENT DES IMAGES; SEGMENTATION; RESEAU NEURONAL; TRAITEMENT EN TEMPS REEL

Domaines: Traitement du signal et de l'image [eess.SP]

Rémi Dufour, Cyril Meurie, Clément Strauss, Olivier Lezoray. Instance segmentation in fisheye images. IPTA 2020, International Conference on Image Processing Theory, Tools and Applications, Nov 2020, Paris, France. ⟨10.1109/IPTA50016.2020.9286623⟩. ⟨hal-02963004⟩