From Bad to Worse: Auto-generating & Autocompleting Hate

By The Anti-Defamation League, Center for Technology and Society

Executive Summary Do social media and search companies exacerbate antisemitism and hate through their own design and system functions? In this joint study by the ADL Center for Technology and Society (CTS) and Tech Transparency Project (TTP), we investigated search functions on both social media platforms and Google. Our results show how these companies’ own tools–such as autocomplete and auto-generation of content–made finding and engaging with antisemitism easier and faster.1 In some cases, the companies even helped create the content themselves. KEY FINDINGS: • Facebook, Instagram, and YouTube are each hosting dozens of hate groups and movements on their platforms, many of which violate the companies’ own policies but were easy to find via search. Facebook and Instagram, in fact, continue hosting some hate groups that parent company Meta has previously banned as “dangerous organizations.” • All of the platforms made it easier to find hate groups by predicting searches for the groups as researchers began typing them in the search bar. • Facebook automatically generated business Pages for some hate groups and movements, including neo-Nazis. Facebook does this when a user lists an employer, school, or location in their profile that does not have an existing Page–regardless of whether it promotes hate. Our researchers compiled a list of 130 hate groups and movements from ADL’s Glossary of Extremism, picking terms that were tagged in the glossary with all three of the following categories: “groups/ movements,” “white supremacist,” and “antisemitism.”2 The researchers then typed each term into the respective search bars of Facebook, Instagram and YouTube, and recorded the results. The study also found that YouTube auto-generated channels and videos for neo-Nazi and white supremacist bands, including one with a song called “Zyklon Army,” referring to the poisonous gas used by Nazis for mass murder in concentration camps. • In a final test, researchers examined the “knowledge panels” that Google displays on search results for hate groups–and found that Google in some cases provides a direct link to official hate group websites and social media accounts, increasing their visibility and ability to recruit new members.

New York: Anti-Defamation League, Center for Technology and Society, 2023. 18p.