ICE-GAN: Identity-aware and Capsule-Enhanced GAN for Micro-Expression Recognition and Synthesis

Micro-expressions can reflect peoples true feelings and motives, which attracts an increasing number of researchers into the studies of automatic facial micro-expression recognition (MER). The detection window of micro-expressions is too short in duration to be perceived by human eye, while their subtle face muscle movements also make MER a challenging task for pattern recognition. To this end, we propose a novel Identity-aware and Capsule-Enhanced Generative Adversarial Network (ICE-GAN), which is adversarially completed with the micro-expression synthesis (MES) task, where synthetic faces with controllable micro-expressions can be produced by the generator with distinguishable identity information to improve the MER performance. Meanwhile, the capsule-enhanced discriminator is optimized to simultaneously detect the authenticity and micro-expression class labels. Our ICE-GAN was evaluated on the 2nd Micro-Expression Grand Challenge (MEGC2019) and outperformed the winner by a significant margin (7%). To the best of our knowledge, we are the first work generating identity-preserving faces with different micro-expressions based on micro-expression datasets only.