This Smartphone App Uses Artificial Intelligence To Help Blind People 'See' Like Never Before

It's amazing to think how technology is coming to the aid of people with special abilities, helping them get over their disabilities and empowering them to lead a better life.

Take Seeing AI, for instance, a smartphone-based camera app that opens a whole new window of possibilities for a visually-impaired user.

Seeing AI is an app developed by Microsoft to showcase the impact of AI's potential to elevate our lives. The app uses state-of-the-art computer vision through your smartphone's camera to describe the world around you. Using artificial intelligence that runs locally on the phone, Seeing AI users can use the app and their phone's camera to point and recognize people, objects, and scenes -- all of which is read out by the app's integrated robot voice.

The app requires your smartphone camera to be on, for obvious reasons -- it tries to substitute lack of human vision with computer vision. You can save people's mug shots with their name tags and Seeing AI can recognize them in a heartbeat, announcing their presence in the camera's field of view, even trying to estimate how far away they are (in terms of distance). The app can scan barcodes and identify everyday objects, it can also read out loud and scan documents -- faster than many OCR apps we've used before.

The app's Scene Mode (which is still in beta) tries to guess what it's seeing in front. While there are a few hits and misses, Seeing AI managed to guess most of our test scenarios quite handsomely. When scanning a piece of document, the app constantly directed me to manoeuvre the camera until I had the entire document surface in the frame -- this is again very useful from a visually impaired user's perspective.

While the app is a lot of fun to use to wow an unsuspecting audience (as I did, while trying it out over the past week or so), it's clearly not an amusement item. Behind the scenes, Seeing AI is powered by state-of-the-art neural network working and learning all the time to better identify the world around it -- not unlike a self-driving car.

Whatever Seeing AI tries to do, with varying degree of success, it does so extremely fast. Seeing AI's speed of detecting anything it sees within its frame is surprisingly quick. Point it to a random piece of text and off it goes, trying to read out the words in a toneless drawl. It constantly scans faces appearing in its field of view, confirming a familiar one in the blink of an eye. This is because Seeing AI's most used features -- like identifying a person, object or reading out text -- are carried out on the device itself. More experimental features like describing a scene or recognizing handwriting requires an Internet connection for advanced AI magic.

A Microsoft Research Project, Seeing AI's lead developer is a Microsoft engineer suffering from visual impairment. No wonder it functions so beautifully, taking into care exactly how it would benefit users who can't see. The app is available for free on iOS, so you can try it on your iPhone or iPad. No news yet about an Android version.