Smart Virtual Mouse Using Computer Vision

Yap, Boon How (2025) Smart Virtual Mouse Using Computer Vision. Final Year Project (Bachelor), Tunku Abdul Rahman University of Management and Technology.

Text YAP BOON HOW_FULL TEXT.pdf Restricted to Registered users only Download (6MB)

Abstract

The advancements in artificial intelligence technology nowadays have led to the increasing human desire to communicate with technology in ever-more-intuitive and accessible ways has driven progress in the field of Human-Computer Interaction (HCI). This has resulted in the development of an AI virtual mouse. This technology can utilize computer vision with hand gesture recognition, relying on high-resolution cameras and deep learning models. However, deploying in devices with limited processing capabilities and low-resolution cameras poses essential challenges, such as excessive latency, poor feature extraction and reduced recognition accuracy of a hand gesture. To address these limitations and letting users have a more user-friendly approach using hand gesture, this study offers a lightweight hand gesture recognition system on resource-constrained hardware with low-resolution camera inputs and enhancing user experience with using dynamic gesture classification that uses MediaPipe Hand Landmarks for efficient spatial feature extraction and using Long Short-Term Memory (LSTM) network for dynamic gesture classification to bring a more user-friendly approach using hand gestures. The model is being trained using a custom dataset of dynamic gestures (rock, stop ,swipe 2 finger to left, swipe 2 finger down, drag, release, and no gesture) taken with low-resolution cameras.