Micro-Expression Recognition Using Few-Shot Learning

Lim, Ming Jun (2023) Micro-Expression Recognition Using Few-Shot Learning. Final Year Project (Bachelor), Tunku Abdul Rahman University College.

Text Full Text -Lim Ming Jun.pdf Restricted to Registered users only Download (8MB)

Abstract

Micro-expression is a momentary, small intensity facial expression. Due to this nature, it is difficult to spot much less recognize the emotion exhibited even for trained human professionals. That is why the task of micro-expression recognition is left to machines instead. In this topic, there are only few micro-expression datasets accessible, datasets have small sample sizes (247 samples for CASME II, 57 micro-expression samples for CAS(ME)2), datasets are labeled in a variety of methods, and datasets have an unbalanced distribution of classes. Because of these issues, identifying micro-expression is more difficult than it needs to be. This study aims to investigate and overcome limited dataset by incorporating techniques involved in Few-Shot Learning (FSL) as well as evaluate the effectiveness of FSL methods against conventional methods. In the datasets CASME II and CAS(ME)2, the proposed framework only uses the apex and offset frame of microexpression in a collection of image sequences for three of the image pre-processing methods investigated (Difference of Image, LoG, Optical Flow Estimation), and the entire video frame for sliced image. These images are paired positively as well as negatively and into the Siamese network as paired inputs. The Siamese CNN network takes two pre-processed from distinct microexpression sources and attempts to calculate and categorize whether they belong to the same class or not, as well as how similar or dissimilar they are, for the goal of fine tuning through contrastive loss function. Optical flow estimation and mapping the magnitude and angle output information into HSV images before converting it into RGB image pre-processing method proves to be the best out of the image pre-processing methods investigated. The model achieves 71.3954% accuracy and 0.66 F-measure, which is superior performance compared to the baseline methods of LBP-TOP, LBP-SIP and others. Siamese network proves to be an effective FSL method which achieves a reasonably high accuracy. The potential of Siamese network should be further explored upon in the field of micro-expression recognition