Non-Frontal-Face Deepfake Detection Using Neural Network



Wong, Jun Yen (2021) Non-Frontal-Face Deepfake Detection Using Neural Network. Final Year Project (Bachelor), Tunku Abdul Rahman University College.

[img] Text
Wong Jun Yen.pdf
Restricted to Registered users only

Download (2MB)


Nowadays, Deepfake is easily generated by either computers and smartphones. As it grows more prominent day by day in the filming industry and social media, deepfake are becoming more and more realistic where it reaches a point the human eye is having difficulty identifying the highly advance deepfake manipulated video. Recently, deepfake are starting to tamper on videos that are non-frontal-face oriented. However, existing deepfake detectors do not perform well for non-frontal deepfake manipulated facial expression vides. This is due to most of the existing deepfake detectors are trained on the dataset of journalists talking directly to the camera. Clips with the target faces that are not facing directly to the camera, such as the side of the face facing the camera, have higher difficulty for neural network to detect than faces that are facing the camera. In this project, main objective is to create a deepfake detector that can detect deepfake manipulated facial expressions on non-frontal-face oriented videos using deep learning and identify the accuracy of the proposed model. MTCNN Face Detector and EfficientNet are selected as the backbone for non-frontal-face deepfake detection. Facial landmarks and SSIM difference masks are extracted from the Deepfake Detection Challenge dataset. The features are then pass to EfficientNet for training. After training, EfficientNet is used to predict the confidence score that the video has gone through deepfake manipulation, with 1 being FAKE, 0 being REAL. In the end, the highest mean confidence score generated through the frame-by by-frame method, 0.98978 for 1 video and achieve an accuracy of 0.842 with 2 maximum frame per video.

Item Type: Final Year Project
Subjects: Technology > Electrical engineering. Electronics engineering
Faculties: Faculty of Engineering and Technology > Bachelor of Engineering (Honours) Electrical and Electronics
Depositing User: Library Staff
Date Deposited: 09 Jul 2021 10:06
Last Modified: 12 Jul 2021 06:25