Instagram to use artificial intelligence to detect bullying in photos

Instagram said on Tuesday that it has started using new technology capable of detecting online bullying in photos, a move that highlights efforts from tech companies to use artificial intelligence to moderate their platforms.

“While the majority of photos shared on Instagram are positive and bring people joy, occasionally a photo is shared that is unkind or unwelcome,” Adam Mosseri, the new head of Instagram, said in a press release. “We are now using machine learning technology to proactively detect bullying in photos and their captions and send them to our Community Operations team to review.”

Instagram’s parent company, Facebook, also introduced its own anti-bullying tools this month. Those features allow users to remove “troll comments” from their feeds, with the options to delete or hide comments in “bulk” and report comments on behalf of the victim.

Social media companies have been under pressure to better manage harassment and hate speech, a difficult challenge since many of them have millions of users. Facebook has hired thousands of people to look over content that may run afoul of its rules.

Tech companies have also been investing in specialized technology to moderate comments and posts, which is particularly difficult when dealing with photos. Facebook also uses specialized technology to detect hate speech in images — leading to something of an arms race between the company and people who post offensive material.

Fiona Brown, well-being lead for communications at Instagram, said the new anti-bullying feature uses artificial intelligence to contextualize the image itself and flag it as harassment.

“It takes several signals into account from the photo itself and, if there’s a caption, from the caption, too.” Brown explained. “An example of a bullying tactic that the photo technology detects is comparing, ranking and rating images and captions, such as split-screen images where a person is compared to another person in a negative way.”

The update to the platform’s technology aims to recognize and block bullying, targeting posts on the app that “many people who experience or observe bullying don’t report.”

Mosseri also announced an expansion of the company’s “bullying comment filter” to now scan through comments made during live videos, and a new feature called the Kindness Camera Effect intended to counteract negativity spread on the platform.

“While stopping bullies is important, we must also do more to celebrate and inspire kindness on Instagram.” Mosseri states.

In collaboration with Maddie Ziegler, a dancer and well-known anti-bullying advocate, the Kindness Camera Effect encourages users to support their friends through kind comments and filter overlays. While users who follow Ziegler will have the feature installed automatically, other users will have the option to tap “try it” to add it to their in-app camera features.

“In selfie mode, hearts will fill the screen — and you’ll be encouraged to tag a friend you want to support.” Mosseri explains. “Your friend will receive a notification that you mentioned them in your story. They can share it to their own story or use the camera effect to spread kindness to someone else.”

Mosseri, who previously ran Facebook’s News Feed, was announced as the head of Instagram at the start of October. Instagram’s co-founders, Kevin Systrom and Mike Krieger, announced their departure from the company in September.

The companies launched their new features at the beginning of October, which is National Bullying Prevention Month.