Towards an adaptive Virtual Reality Serious Game System for Motor Rehabilitation based on Facial Emotion Recognition

Abstract : In this paper, an adaptive serious game-based functional rehabilitation system was proposed. This system provides each patient with the appropriate game during his rehabilitation sessions. The game's complexity, speed, and level vary according to patients’ facial emotions. Moreover, two modules were presented namely: Convolutional Neural Networks-based motion recognition module which analyzes the patient's facial emotions and the Virtual Reality-based serious game module that retrieves the updated emotion's data and provides the patient's suitable game so that he/she can perform rehabilitation gestures in good conditions. When a patient expresses emotions of stress or tiredness, the first module analyzes this situation and sends a notification to use the same game but with less complexity, or another game. Preliminary tests of the system were carried out on three patients and the feedback is very encouraging.