Visual Perception Approach for Motion Control of Robotic Arm

Tang, Kung Yen (2026) Visual Perception Approach for Motion Control of Robotic Arm. Final Year Project (Bachelor), Tunku Abdul Rahman University of Management and Technology.

Text Tang Kung Yen_Full Text.pdf Restricted to Registered users only Download (15MB)

Abstract

This project addresses the critical need for adaptive safety in human-robot collaborative environments by developing a vision-based fatigue detection and control system for an industrial robotic arm. Recognizing that traditional safety measures such as emergency stops and proximity sensors do not account for operator fatigue, this project proposes a real-time monitoring solution using a fixed-position camera and deep-learning models (YOLOv8, YOLOv11, YOLOv12). The system detects visual indicators of fatigue (close eyes, open eyes, cloesemouth, open mouth, tilted head, upright head) and dynamically adjusts the speed of a KUKA robotic arm: slowing movement upon initial signs of fatigue and halting operation under extreme conditions. To build and validate the approach, a labeled dataset was created and used to train and compare multiple YOLO architectures on detection accuracy and Frames Per Second (FPS). The integrated prototype was evaluated in a controlled laboratory setting, measuring detection precision, robot response time, and overall impact on operational safety. Results demonstrate that incorporating fatigue-aware motion control significantly reduces risk of collision and improves reaction windows without compromising productivity. By proactively adapting robot behavior to human condition, this system advances human-centered automation in line with Industry5.0 principles and Sustainable Development Goal9, laying the groundwork for safer, more resilient, and sustainable industrial operations