RISIUS, MARTEN; NAMVAR, MORTEZA; AKHLAGHPOUR, SAEED; XIE, HETIAO (SLIM)

The following excerpt from the document contains multiple links embedded in the original text: "'Content Warning: This insight contains antisemitic, racist, and hateful imagery.' [...] Extremists exploit social media platforms to spread hate against minority groups based on protected attributes such as gender, religion, and ethnicity. Platforms and researchers have been actively developing AI tools to detect and remove such hate speech. However, extremists employ various forms of implicit hate speech (IHS) to evade AI detection systems. IHS spreads hateful messages using subtle expressions and complex contextual semantic relationships instead of explicit abusive words, bringing challenges to automatic detection algorithms. Common forms of IHS include dog whistles, coded language, humorous hate speech, and implicit dehumanisation. Moreover, the forms and expressions of IHS evolve rapidly with societal controversies (e.g., regional wars). Identifying and tracking such changes in IHS is crucial for platforms trying to counter them. In this Insight, we report and analyse 'Substitution' as a new form of IHS. Recently, we observed extremists using 'Substitution' by propagating hateful rhetoric against a target group (e.g., Jews) while explicitly referencing another label group (e.g., Chinese). We show that Substitution not only effectively spreads hate but also exacerbates engagement and obscures detection."

GLOBAL NETWORK ON EXTREMISM AND TECHNOLOGY (GNET). 24 JUN, 2024. 8p.

By Kleemann, Aldo

From the document: "Deception and media manipulation have always been an integral part of wartime propaganda. But never before has it been so easy to create high-quality fabrications of images as well as sound and video recordings. The tendency to react emotionally to these media opens up a whole new possibility for abuse by their creators. A call to surrender by President Volodymyr Zelensky, which was immediately exposed as a deepfake, is the first attempt to use the new technology in an armed conflict. The quality of such fabrications is improving, detecting them is becoming increasingly complex and there is no end in sight to these developments. Banning deepfakes would be futile. It is therefore time to look at current and potential applications and possible counter-strategies."

German Institute For International And Security Affairs. 2023. 6p.

By Moonshot

Over a one year period from April 2021 - March 2022, Moonshot partnered with three violence prevention organizations to deliver an online interventions pilot in two Canadian provinces. The pilot advertised psychosocial support services to individuals engaging with extremist content online. Access to these services was voluntary, confidential, and anonymous by design. Our goal was to offer a secure pathway for at-risk individuals to contact a trained therapist or social worker. We built this approach around offering integrated care. Together with our intervention partners, we crafted our advertising messages and service websites to emphasize the confidential, non-judgemental support that callers would receive. Individuals who reached out were connected to an interdisciplinary team, which included a therapist, youth engagement workers, a psychiatrist, and other intervention staff who could offer services like counseling, employment support, addiction support, or simply a space to talk. Our partners were the Estimated Time of Arrival (ETA) team in Ontario, and Recherche et Action sur les Polarisations Sociales (RAPS) in Quebec. The Canadian Practitioners Network for the Prevention of Radicalization and Extremist Violence (CPNPREV) acted as a convening and best practice provider, and supported our pilot evaluation. A description of each organization is at the end of this report. Moonshot’s intervention campaigns ran for a total of six months, and reached individuals consuming incel and violent far-right extremist content on Google Search and YouTube. Our online interventions focused on meeting individuals’ psychosocial needs, and appealed to vulnerabilities and grievances, such as anger, frustration, exhaustion, and isolation. Executive summary Key outcomes Moonshot redirected 786 at-risk individuals to our intervention partners’ websites. 22 initiated a conversation with a counselor. Four individuals formally registered and engaged with a service provider for several months, in addition to those who accessed virtual counselling without going through the registration process. At least one person who initially shared violent impulses has been able to find positive, hopeful alternatives for the future. Moonshot’s ads reached users engaging with harmful content on Google and YouTube 44,508 times. Among the hundreds of users redirected to ETA and RAPS’ websites, 26 were watching influential incel YouTube channels and 39 had searched Google for high-risk keywords related to incel and violent far-right ideology (“looksmax org”; “1488 tattoos”). Moonshot, ETA, RAPS, and CPN-PREV established an effective multi-sectoral partnership. During our pilot program, we co-designed support pathways and risk escalation procedures for each service area, built teams’ capacity to deliver online interventions safely and effectively, and engaged at-risk audiences online. This pilot provides a blueprint for future interventions to reach and engage at-risk internet users. New iterations of this work can reach larger audiences by expanding advertising beyond the pilot platforms, strengthening and expanding cross-sectoral partnerships, and testing new ways to reach often-isolated internet users.

Washington, DC: Moonshot, 2023. 13p.

By The McCain Institute , Moonshot and Ketchum

In September 2022, the McCain Institute, in collaboration with Moonshot and Ketchum, launched SCREEN Hate, an initiative that provides caregivers and concerned adults with the knowledge, tools, and resources needed to keep youth safe from online messages that could incite acts of hate-based violence. SCREEN Hate is the first nationwide campaign aimed at equipping bystanders to prevent acts of hate-based violence perpetrated by youth. SCREEN Hate was created as part of a two-year project for the Center for Prevention Programs and Partnerships (CP3) at the Department of Homeland Security (DHS), funded through the 2021 Targeted Violence and Terrorism Prevention Grant Program.1 The SCREEN Hate resource hub and accompanying online campaigns launched on 15 September 2022, having been announced at the “United We Stand Summit” at the White House. The campaigns—designed and implemented by Moonshot, the McCain Institute, and Ketchum—included behavior-based bystander engagement campaigns on Google Search and YouTube, as well as wider community outreach campaigns on Reddit, Facebook, and Instagram. This report details the national findings from the SCREEN Hate online campaigns, conducted between September 2022 and July 2023. It includes geographic and behavioral insights into user engagement, as well as the results of comparative testing on user engagement with our YouTube campaign ads. The final section includes recommendations for future programming for practitioners working to engage with concerned bystanders through online campaigns.

Washington, DC: Moonshot, 2023, 28p.

By Institute for Strategic Dialogue (ISD)

This report aims to provide a first look into Irish far-right activity on the messaging app, Telegram, where the movement is operating both as identifiable groups and influencers, and anonymously-run channels and groups. The report looks at the activity across 34 such Telegram channels through the lens of a series of case studies where content posted on these channels resulted in real life consequences. Using both quantitative and qualitative methods, the report examines the tactics, language and trends within these channels, providing much-needed detail on the activity of the Irish far-right online. This report was produced in conjunction with TheJournal.ie and its investigative platform Noteworthy.ie as part of their Eyes Right series, examining the growth of far-right ideology on Irish online networks, and its influence on wider public opinion.

Beirut; London; etc.: Institute for Strategic Dialogue (2021). ,30p.

By Moustafa Ayad, Nadeem Khan and Aymenn al-Tamimi

his report highlights the networks, supporters, and the platforms of Islamic State disinformation disseminators, focusing on popular social media platforms as well as encrypted messaging applications. These disinformation networks are creating self-branded media outlets with followers in the tens of thousands, and often with innocuous names like “Global Happenings,” “DRIL” and “Media Center,” to evade moderation and takedowns. These same networks use coded language and a codebook of emojis to spread Islamic State “news” to other networks of supporters, who similarly evade moderation. These ‘alternative news outlets’ are trying to outcompete narratives publicized by government officials as well as independent mainstream media and individual journalists – groups that were also heavily targeted by Islamic State.

London: IDS - Institute for Strategic Dialogue, 2022. 22p.

By Anna Kruglova

This policy brief analyses the phenomenon of conspiracy theories, and how they fit in the realm of research on violent extremism. Using the case study of QAnon, this Policy Brief looks at how the movement mobilises people to violence on social media and attempts to determine how different this process is from any other process of radicalisation to violence. By combining discourse analysis of the movement’s Gab posts, interviewing former supporters, and analysing three illustrative case studies, this Policy Brief identifies five discourses - such as revenge, “the Other”, chosenness/ specialness, apocalypse, and urgency for action and altruism - that are used to bring people into the violent mindset. It emphasises the similarity of these discourses to the ones used by other extremist organisations and argues against exceptionalising the threat of QAnon and other conspiracy theories. It concludes by making recommendations about how to tackle QAnon propaganda.

The Hague: International Centre for Counter-Terrorism (ICCT) , 2023. 22p.

Armstrong-Scott, Gabrielle L.; Waldo, Jim

From the document: "Ten years ago, U.S. national security agencies grew concerned about a relatively new and powerful weapon used by terrorists: the World Wide Web. What had begun as an effort to connect end users from across the world to share information and to serve as a force of human liberation, instead began to be used as a tool for destruction of life. Terrorists were exploiting technology companies' lax content moderation policies to recruit new members, spread violent extremist ideology, and plan terrorist attacks. [...] Technology companies and governments have spent the past decade trying to better address the evolving threat of terrorist and violent extremist content online (TVEC). However, there are few studies examining just how effective these efforts have been, where we are today in managing the problem, and wherein lie gaps for improvement. This paper argues that companies' efforts to deal with TVEC have been hampered at the outset by a tendency to define TVEC extremely narrowly. [...] This paper also explores the idea of ethical obligations and norms as an alternative to a legally required definition. On the technical side, this paper finds that even if there was consensus on the legal and ethical questions surrounding TVEC, the technical tools currently available are no panacea. Trade-offs across efficiency, scalability, accuracy, and resilience are persistent. Current technical tools tend to disadvantage minority groups and non-English languages. They are also less robustly implemented across small and non-U.S./European firms, generally either because they are left out of inter-firm initiatives or because they lack resources and capability. This paper does not claim to cover every issue relevant to TVEC; however, it highlights several important gaps that could be addressed by policymakers and tech companies and identifies avenues for future research."

Belfer Center For Science And International Affairs. 2023.

By Michael Wolfowicz, Badi Hasisi, David Weisburd

Most national counter-radicalization strategies identify the media, and particularly the Internet as key sources of risk for radicalization. However, the magnitude of the relationships between different types of media usage and radicalization remains unknown. Additionally, whether Internet-related risk factors do indeed have greater impacts than other forms of media remain another unknown. Overall, despite extensive research of media effects in criminology, the relationship between media and radicalization has not been systematically investigated. This systematic review and meta-analysis sought to (1) identify and synthesize the effects of different media-related risk factors at the individual level, (2) identify the relative magnitudes of the effect sizes for the different risk factors, and (3) compare the effects between outcomes of cognitive and behavioral radicalization. The review also sought to examine sources of heterogeneity between different radicalizing ideologies.

. ampbell Systematic Reviews, Volume18, Issue2. June 2022. e1244

By Europol

  Terrorism remains a significant threat to the internal security of the European Union. Terrorists operate across borders, leveraging new technologies and modi operandi to target innocent people. EU law enforcement authorities, in particular the counter-terrorism community, remain vigilant and united in countering the threat. The EU Terrorism Situation and Trend Report (TE-SAT) 2023 contributes to this effort, by presenting prevalent terrorism trends in the EU. Terrorism constitutes an attack on our society intending to create fear and chaos among EU citizens. In 2022, 16 terrorist attacks took place in the EU and an additional 12 attacks failed or were foiled. While the attacks directly affected seven Member States, more Member States arrested suspects in terrorism-related cases. Tragically, four people were killed in terrorist attacks in the EU, two resulting from jihadist attacks and two from a right-wing terrorist attack. We honour the memory of all those lost to terrorism in 2022 and in previous years.

• More....

  Member States continue to view jihadist terrorism as the most prominent terrorist threat to the EU despite the fact that the number of jihadist attacks has decreased compared to 2021 and 2020. The threat from right-wing terrorist lone actors, radicalised online, remains significant. Left-wing and anarchist terrorists and violent extremists continue to pose a threat to public safety and security in the EU. Additionally, the spread of propaganda online and its potential for radicalisation remains a key concern. Social isolation and the lack of a solid support system remain key vulnerabilities which terrorists take advantage of in order to propagate their messages and to recruit new followers. This is particularly worrying with the increasing number of young people, including minors, exposed to online terrorist propaganda. Furthermore, the terrorist and violent extremist environment is becoming more decentralised and volatile. Diffuse actors connect and inspire one another, uniting behind grievances beyond ideology or group affiliation. This context is ripe for self-radicalisation and for lone actors to begin engaging in attack planning, and poses significant challenges for preventing and combating terrorism and violent extremism in the EU.   

  Publications Office of the European Union, Luxembourg, 2023. 94p. 

By Anna Kruglova

This policy brief analyses the phenomenon of conspiracy theories, and how they fit in the realm of research on violent extremism. Using the case study of QAnon, this Policy Brief looks at how the movement mobilises people to violence on social media and attempts to determine how different this  process is from any other process of radicalisation to violence. By combining discourse analysis of the movement’s Gab posts, interviewing former supporters, and analysing three illustrative case studies, this Policy Brief identifies five discourses - such as revenge, “the Other”, chosenness/ specialness, apocalypse, and urgency for action and altruism - that are used to bring people into the violent mindset. It emphasises the similarity of these discourses to the ones used by other extremist organisations and argues against exceptionalising the threat of QAnon and other conspiracy theories. It concludes by making recommendations about how to tackle QAnon propaganda.

The Hague: International Centre for Counter-Terrorism (ICCT),    2023. 22p.

By Rudiger Lohiker

Jihadism has been an important issue of public discussions since 9/11. Internet media have been used by Jihadis as means of communication, propaganda, recruitment, and even training purposes. In this volume, the processes of interaction on Jihadi internet sites are analysed. Particular attention lays on the mechanisms of spread of propaganda via the internet by diverse technical means. The process of transformation of Islamic knowledge into Jihadi knowledge, the rhetorics  of videos, the development of South Asian Jihadi organisations and some conceptual issues are discussed.

Gottingen: V&R unipress, 2012. 224p.

By Sara Zeiger and Joseph Gyte

In the age of selfies, snaps, likes and shares, the internet and social media have transformed the way in which people communicate. In early 2019, global internet penetration reached 57%, or 4.4 billion users, and the overall number of mobile social media users reached 42%, or 3.2 billion people.1 This means that people are able to share ideas, communicate and interact more rapidly than ever before, including with audiences on the other side of the world. Terrorist groups have certainly leveraged these new mechanisms and platforms for communicating amongst themselves and to potential recruits. For example, the Islamic State of Iraq and alSham (ISIS) has been known for producing sleek videos circulated on YouTube and Twitter, and has mastered new and emerging technologies and social media platforms, such as Telegram; all to promote its messages and recruit new members in cyberspace. This chapter focuses on the prevention of radicalization on social media and the internet in this digital age. It first reviews the relevant methods and approaches that terrorists employ to spread their propaganda and recruit online. Subsequently, it looks at some of the more common and emerging prevention and preparedness strategies which address the online space. Besides reviewing the theoretical foundations to prevent radicalization on social media and the internet, this chapter will also draw upon specific examples, predominantly from three regions: Europe, Southeast Asia and East Africa, to illustrate what some countries are doing to tackle the problem of online radicalization.

From: International Handbook of Terrorism Prevention and Preparedness, 2023. 38p.

By Editor James J.F. Forest and Bart Schuurman

The first article by Deven Parekh, Amarnath Amarasingam, Lorne Dawson and Derek Ruths offers a detailed critique of Twitter data collection methods and propose suggestions for improving the collection of data in future research on terrorists use of social media. Then Ariel Koch examines the impact of Islamic State beheading videos among jihadists and other extremists. Next, Adesoji Adelaja, Abdullahi Labo and Eva Penar examine how the Nigerian public’s views about the root causes and objectives of Boko Haram differ from those of the government. And in our final article of this issue a team of researchers at the International Center for Study of Violent Extremism (ICSVE’s) report on their recent Facebook ad-campaign aimed at raising awareness about the realities of living under ISIS and protecting vulnerable potential recruits from considering joining.

Terrorism Research Initiative (TRI), headquartered in Vienna, Austria, and the Institute of Security and Global Affairs (ISGA) of Leiden University, Campus The Hague. Vol.XII, Issue 3, June 2018. 189p.

By Jytte Klausen

This report analyses the networks and organizations that mobilize and direct Americans for jihadist action, or that raise money in the US for Hamas and Hezbollah. The study employs a quasi-experimental method using a control-case design, comparing the network structures of American terrorism offenders inspired by Hezbollah with those of Sunni extremist groups aligned with Al Qaeda, and in recent years ISIL.

Washington, DC: U.S. Department of Justice, 2016. 71p.