Using Multimodal Information to Enhance Addressee Detection in Multiparty Interaction

Usman Malik (1) , Mukesh Barange (1) , Julien Saunier (2) , Alexandre Pauchet (2, 1)

1 LITIS - Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes
2 MIND - LITIS - Equipe Multi-agent, Interaction, Décision

Addressee detection is an important challenge to tackle in order to improve dialogical interactions between humans and agents. This detection, essential for turn-taking models, is a hard task in multiparty conditions. Rule based as well as statistical approaches have been explored. Statistical approaches, particularly deep learning approaches, require a huge amount of data to train. However, smart feature selection can help improve addressee detection on small datasets, particularly if multimodal information is available. In this article, we propose a statistical approach based on smart feature selection that exploits contextual and multimodal information for addressee detection. The results show that our model outperforms an existing baseline.

Keywords: Human-Computer Interaction; Intelligent Agents; Machine Learning

Domains: Computer Science [cs]; Artificial Intelligence [cs.AI]

Usman Malik, Mukesh Barange, Julien Saunier, Alexandre Pauchet. Using Multimodal Information to Enhance Addressee Detection in Multiparty Interaction. International Conference on Agents and Artificial Intelligence, Feb 2019, Prague, Czech Republic. pp.267-274. ⟨hal-02117658⟩