Facial Expression Recognition for Motion Control of Robotic Arm

Thum, Jun Ming (2025) Facial Expression Recognition for Motion Control of Robotic Arm. Final Year Project (Bachelor), Tunku Abdul Rahman University of Management and Technology.

Text THUM JUN MING_FULL TEXT.pdf Restricted to Registered users only Download (20MB)

Abstract

This project presents the development and evaluation of a Facial Expression Recognition (FER) system for controlling the motion of an industrial robotic arm. Deep learning models based on YOLOv11n and YOLOv11m were trained using datasets such as FER2013, AffectNet, and CK+ to recognize seven basic emotions: happiness, sadness, anger, fear, surprise, disgust, and neutral. The models were evaluated using classification performance metrics including precision, recall, F1-score, confusion matrix analysis, and learning curve assessment. Learning curves were used to analyze model training behavior and fitting characteristics, revealing that YOLOv11m exhibited better generalization and stability, with a lower risk of overfitting on complex datasets. In contrast, YOLOv11n demonstrated limited learning capacity and a higher tendency toward underfitting. YOLOv11m, trained on the AffectNet dataset, achieved the highest classification accuracy of 58.80% at epoch 143. Pre-recorded videos were used for facial expression detection, and the recognized emotions were mapped to control the speed of a KUKA KR6 R600 industrial robotic arm. After system integration, the FER-based speed control achieved an overall average accuracy of 95.67% across all emotion segments. The results demonstrate that facial expression recognition can be effectively applied to enhance human-robot collaboration and improve industrial safety.