Development of Learning-Based Algorithm for Vision-Guided Robotic Arm Performing Basic Tasks

Wee, Max Ming Liang (2026) Development of Learning-Based Algorithm for Vision-Guided Robotic Arm Performing Basic Tasks. Final Year Project (Bachelor), Tunku Abdul Rahman University of Management and Technology.

Text MAX WEE MING LIANG_Full Text.pdf Restricted to Registered users only Download (28MB)

Abstract

Programming industrial robots, such as the KUKA KR 4 R600, for high-mix, sequential pick-and-place tasks remains a significant bottleneck, often requiring specialized expertise and extensive reconfiguration time. This thesis addresses this challenge by presenting the development and validation of an intelligent, vision-guided system that autonomously learns, optimizes, and replicates complex cube arrangement tasks from a single, non-expert human demonstration. Leveraging an observational Learning from Demonstration (LfD) paradigm, the system integrates a dual-camera Intel RealSense D435 setup for 3D perception with a state-of-the-artYOLOv11 object detection model, custom-trained to identify and localize 2.5 cm cubes. The methodology involves monitoring a workspace to record the sequence of final cube poses demonstrated by an operator. The core contribution of this work, however, moves beyond simple mimicry by introducing a threestage intelligent pipeline to process this raw demonstration. First, a novel grid-snapping algorithm idealizes the imprecise human input into a geometrically perfect, machine-executable goal plan. Second, a heuristic-based task planner autonomously re-orders this goal plan, generating a stable and efficient "inside-out" assembly sequence that enhances structural integrity and minimizes collision risk. Third, during execution, the system employs dynamic, perception-driven strategies, including adaptive grasp orientation for picking randomly oriented source cubes and real-time collision avoidance for placing cubes in dense configurations. This optimized sequence is then translated into executableKUKAmotion commands communicated via the EthernetKRLInterface (EKI). The system’s end-to-end performancewas rigorously validated through 400 trials across four test patterns of increasing complexity, achieving an overall task success rate of 92.5% and a mean placement error of 2.26 mm. This research provides a complete framework for a more intuitive and robust form of robotic teaching, demonstrating that by interpreting and optimizing human intent, a vision-guided system can achieve a level of precision and reliability that surpasses the original demonstration, thereby enhancing automation flexibility in alignment with the principles of Industry 4.0