Study of Sign Language Recognition System on Mobile Phone

Yeoh, Tian Woei (2018) Study of Sign Language Recognition System on Mobile Phone. Final Year Project (Bachelor), Tunku Abdul Rahman University College.

Text YEOH TIAN WOEI - FULL THESIS.pdf Restricted to Registered users only Download (2MB)

Abstract

Sign language is essential for people who have hearing impaired condition. Sign language are gesture and movement of hand sign to express or communicate their thought. Two approaches are normally study in the sign language recognition system which is hardware based that utilize glove and assistive tools. The second approaches is the visual based approaches where camera and artificial intelligence are needed. The objective of this project is to study the visual based approach by investigating the behavior and possibility of sign language recognition system in smartphone. The system consist of a neural network generated in matlab which are used to interpret the images from the phone. A deep neural network architecture is constructed that achieve accuracy of 91.3% testing with confusion matrix analysis. The deep neural network architecture consist of 3 autoencoder and 1 layer of softmax layer. The system could recognize the images for 24 alphabet excluded J and Z. An application created through android studio could store the image taken from a person sign gesture which later can be read in matlab on the computer through file transfer protocol connection with the help from ES file explorer. The result from classifying the images are later display in the phone screen.