Development of Microcontroller-based Image Classification Using CMSIS-NN for Gesture Recognition

Kang, Jo En (2021) Development of Microcontroller-based Image Classification Using CMSIS-NN for Gesture Recognition. Final Year Project (Bachelor), Tunku Abdul Rahman University College.

Text Kang Jo En.pdf Restricted to Registered users only Download (13MB)

Abstract

The processing capabilities of microcontrollers have improved such that it is now possible to perform machine learning. As the amount of smart devices around us increases, effective interaction with these devices is becoming increasingly difficult. A gesture recognition system using microcontroller-based computer vision is proposed. Investigation of image classification performance (accuracy & response time) using CMSIS-NN library on suitable resource constrained Cortex-M microcontroller platforms are carried out. The basic design of the project is to recognize a few different hand gestures from image captured from different angles or size or orientation. The propose system will be coded by python3, software Oracle VM VirtualBox Manager, Ubuntu 20.04, wine, Keil MDK v5.33 and the hardware is STM32F746G-DISCO discovery board (Cortex-M7) and STM32F4DIS-CAM Camera module to capture the image. CNN model of the hand images will be trained and converted into Tensorflow Lite model to put it in microcontroller.