Instagram Will Use AI To Scan Captions And Photos To Prevent Bullying On The Platform

Where there is social media, there will be trolls. And while smart asses and bad jokes can be ignored as banter, trolls stoop to new lows with race-baiting, sexism, verbal abuse, and just downright horrible vitriol. Instagram wants to change that.

Last year, the photo sharing app introduced a new artificial intelligence (AI) it uses to scan comments and filter out those with offensive words and phrases. It wasn¡¯t your typical filter system that just banned specific predetermined words either. Instead, the machine learning system looks at a whole sentence, and also takes into account a reply¡¯s context. That way it could catch more bad comments, while avoiding the sort of accidental false positives word ban systems throw up.

With this system, Instagram played a smart card too. While the banned comment was made invisible to everyone else, the commenter wasn¡¯t informed in any way. After all, telling a troll their response is banned only serves to make them think of better ways to cheat the system. Instead, making them think everyone was simply ignoring their hateful comments might be the best way to get them to stop.

Now, Instagram, is also rolling out the AI filter to photos and captions. The idea is to ¡°proactively detect bullying¡± before having human moderators review the cases. The feature will roll out in the coming weeks.

The interesting thing here is that the AI won¡¯t just read the text in images, but will analyse the photos themselves for bullying. As an example given to The Verge, a spokesperson explained how the AI would check for split-screen images as potential bullying, like if one person was being negatively compared to another. It¡¯s also possible Instagram has trained the AI to ban offensive memes already in circulation, though it¡¯s unclear what else the AI might be scanning for.

The move certainly puts Instagram in a positive light, which is something it desperately needs at the moment. Thanks to Mark Zuckerberg chasing off the founders of WhatsApp and Instagram with his aggressive monetisation policies, the impression is that they¡¯re all doomed. Add to that the recent Facebook breach, and the incumbent ads in WhatsApp, and Instagram looks like the saving grace of the entire establishment. The fate of the organisation and Facebook¡¯s stranglehold on social media may rest on its very shoulders.