Engineering Girl Builds AI Model To Translate Sign Language Into English Instantly

Communicating with someone who cannot speak or hear can be challenging and even though ASL (American Sign Language) exists, only a few learn to communicate using it.

Unsplash

Also Read: 5 Amazing Gadgets That Are Helping People With Hearing Impairment

However, browsing through LinkedIn, I stumbled upon a post by one Priyanjali Gupta that had developed an AI model that translated some ASL signs to English, bridging the gap between the divide.

Gupta, who is currently a student at the prestigious Vellore Institute of Technology, has been able to achieve this by harnessing Tensorflow object detection api. At its heart, it¡¯s making use of transfer-learning via a pre-trained model dubbed ssd_mobilenet.

In her Github post, she has revealed that the dataset was made manually by running a file (dubbed Image Collection python file from her GitHub repository) that collects images of ASL actions via a webcam.

Priyanjali Gupta

As of now, it is capable of recognising six gestures -- Hello, I Love You, Thank You, Please, Yes, No.

In a comment to one of the LinkedIn users, she has acknowledged to the fact that creating a deep learning model from scratch just for sign detection is quite complex, ¡°To build a deep learning model solely for sign detection is a really hard problem but not impossible and currently I'm just an amateur student but I am learning and I believe sooner or later our open source community which is much more experienced and learned than me will find a solution and maybe we can have deep learning models solely for sign languages.¡±

Also Read: Seven-Year-Old Mumbai Girl Built An App To Help Deaf Kids Easily Learn Sign Language

She added, ¡°Therefore for a small dataset and a small scale individual project I think object detection did just fine. It's just the idea of inclusion in our diverse society which can be implemented on a small scale.¡±

While this is still at a nascent stage, it¡¯s commendable that developers are looking towards creating more inclusive applications to help those in need.

Keep reading Indiatimes.com for the latest science and technology news.