Real-Time Operator Safety Monitoring System for Industrial Human-Robot Collaboration

How, Man Kien (2026) Real-Time Operator Safety Monitoring System for Industrial Human-Robot Collaboration. Final Year Project (Bachelor), Tunku Abdul Rahman University of Management and Technology.

Text HOW MAN KIEN_Full Text.pdf Restricted to Registered users only Download (19MB)

Abstract

Current industrial Human-Robot Collaboration (HRC) safety systems cannot distinguish a distracted operator from an attentive one, creating a “false safe” condition that proximitybased systems cannot address. This project develops an integrated real-time safety monitoring system that eliminates this gap by fusing two computer vision streams: YOLOv8-based helmet detection for physical compliance and MediaPipe FaceMesh head pose estimation for cognitive state monitoring. A dual-process producer-consumer architecture decouples vision inference from the KUKA robot communication loop, enabling concurrent processing without jitter in the real-time control cycle. The fused output drives a five-state Finite State Machine (SAFE, ATTN_WARN, PPE_WARN, EMERGENCY_STOP, IDLE) mapping combined PPE and attention states to graduated robot speed overrides of 100% (SAFE), 50% (ATTN_WARN), 30% (PPE_WARN), and 0% (EMERGENCY_STOP or IDLE) via the KUKA Robot Sensor Interface UDP protocol. Experimental validation across 13 trials (10 for Operator 1, 3 for Operator 2) and two independent operators yields a combined helmet detection accuracy of 99.41% with 99.90% precision, and a combined attention classification accuracy of 98.92%, both exceeding the defined target thresholds. The frame-to-frame inter-update latency (used as a bound on stale-state duration) has a mean of 61.50 ms and a worst-case maximum of 123.25 ms across 13,232 measured frames, well within the 300 ms limit mandated by ISO/TS 15066. The system is implemented on accessible commercial hardware, establishing a proof-of-concept for context-aware HRC safety monitoring without proprietary sensor infrastructure