Gesture sequence recognition with one shot learned CRF/HMM hybrid model

Selma Belgacem (1) , Clément Chatelain (2) , Thierry Paquet (2)

1 LITIS - Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes
2 DocApp - LITIS - Equipe Apprentissage

In this paper, we propose a novel markovian hybrid system CRF/HMM for gesture recognition, and a novel motion description method called gesture signature for gesture characterisation. The gesture signature is computed using the optical flows in order to describe the location, velocity and orientation of the gesture global motion. We elaborated the proposed hybrid CRF/HMM model by combining the modeling ability of Hidden Markov Models and the discriminative ability of Conditional Random Fields. In the context of one-shot-learning, this model is applied to the recognition of gestures in videos. In this extreme case, the proposed framework achieves very interesting performance and remains independent from the moving object type, which suggest possible application to other motion-based recognition tasks.

Keywords: Gesture recognition; One-shot-learning; Hybrid system; Hidden Markov model; Conditional random field; Gesture characterisation

Domains: Computer Science [cs]; Computer Vision and Pattern Recognition [cs.CV]

Selma Belgacem, Clément Chatelain, Thierry Paquet. Gesture sequence recognition with one shot learned CRF/HMM hybrid model. Image and Vision Computing, 2017, 61, pp.12-21. ⟨10.1016/j.imavis.2017.02.003⟩. ⟨hal-02075733⟩