Affective Computing Models: from Facial Expression to Mind-Reading

ongoing

Mind reading plays an indispensable role in human social interaction and communication. One of the key factors in mind reading is facial expression analysis. It is thus crucial to understand and model facial behaviors reflecting mental states to enhance communication experience with robots and other intelligent agents in either real social interaction or augmented virtual environments. As well, consistent cross-cultural disagreement about the emotion and intensity conveyed by gold-standard universal facial expressions questions the universality hypothesis. In recent years, even though automatic facial expression analysis has achieved excellent progress, the research of emotion understanding in realistic environment across cultures is still lacking. Substantial psychological works support the use of the appraisal theory for internal emotion detection through facial behaviors, while research in computer science mainly focuses on facial modelling but ignoring the underlying biologically-driven mechanism. Therefore, we plan to reconstruct facial macro- and micro-expressions and digitise emotion appraisal, collect 4D (dynamic 3D) facial expression dataset with culture diversity and develop autonomous 4D facial model based on the collected 4D data and cultural emotion appraisal theory, and push forward commercialisation. ACMod project aims to advance the state of the art in understanding and modelling naturalistic facial behaviours in social interaction cross various cultures, which is essential to the next generation of human-computer interaction, robotics and augmented virtual reality techniques. This project will open up a new avenue for modelling facial behaviors by taking a fundamentally different and interdisciplinary approach, combining theories and techniques in e.g. psychology and computer science with an emphasis on social interaction applications. The project will promote knowledge exchange between EU and East Asian partners to foster the researchers involved.