Towards an adaptive Virtual Reality Serious Game System for Motor Rehabilitation based on Facial Emotion Recognition

Yousra Izountar (1, 2) , Samir Benbelkacem (2) , Samir Otmane (3) , Abdellah Khababa (1) , Nadia Zenati (2) , Mostefa Masmoudi (2)

1 University of Ferhat Abbes Setif 1 Department of Computer Science
2 CDTA - Centre de Développement des Technologies Avancées
3 IBISC - Informatique, BioInformatique, Systèmes Complexes

In this paper, an adaptive serious game-based functional rehabilitation system was proposed. This system provides each patient with the appropriate game during his rehabilitation sessions. The game's complexity, speed, and level vary according to patients’ facial emotions. Moreover, two modules were presented namely: Convolutional Neural Networks-based motion recognition module which analyzes the patient's facial emotions and the Virtual Reality-based serious game module that retrieves the updated emotion's data and provides the patient's suitable game so that he/she can perform rehabilitation gestures in good conditions. When a patient expresses emotions of stress or tiredness, the first module analyzes this situation and sends a notification to use the same game but with less complexity, or another game. Preliminary tests of the system were carried out on three patients and the feedback is very encouraging.

Keywords: 3D Serious game; Deep learning; Adaptation; Rehabilitation; Facial emotion recognition; CNN; Virtual reality

Domains: Machine Learning [cs.LG]; Human-Computer Interaction [cs.HC]

Yousra Izountar, Samir Benbelkacem, Samir Otmane, Abdellah Khababa, Nadia Zenati, et al.. Towards an adaptive Virtual Reality Serious Game System for Motor Rehabilitation based on Facial Emotion Recognition. International Conference on Artificial Intelligence for Cyber Security Systems and Privacy (AI-CSP 2021), Nov 2021, El Oued, Algeria. pp.1-5, ⟨10.1109/AI-CSP52968.2021.9671149⟩. ⟨hal-03528216⟩