By James Lewis, Sarah Marsden, James Hewitt, Chloe Squires, Anna Stefaniak

Reliability and Validity of Risk Assessment Tools for Violent Extremism: A Systematic Review

By Sébastien Brouillette-Alarie, Ghayda Hassan, Wynnpaul Varela, Emmanuel Danis, Sarah Ousman, Pablo Madriaza, Inga Lisa Pauls, Deniz Kilinc, David Pickup, Robert Pelzer, Eugene Borokhovski, the CPN-PREV team

Assessment of the risk of engaging in a violent radicalization/extremism trajectory has evolved quickly in the last 10 years. Guided by what has been achieved in psychology and criminology, scholars from the field of preventing violent extremism (PVE) have tried to import key lessons from violence risk assessment and management, while bearing in mind the idiosyncrasies of their particular field. However, risk tools that have been developed in the PVE space are relatively recent, and questions remain as to their level of psychometric validation. Namely, do these tools consistently and accurately assess risk of violent extremist acting out? To answer this question, we systematically reviewed evidence on the reliability and validity of violent extremism risk tools. The main objective of this review was to gather, critically appraise, and synthesize evidence regarding the appropriateness and utility of such tools, as validated with specific populations and contexts. Searches covered studies published up to December 31, 2021. They were performed in English and German across 17 databases, 45 repositories, Google, other literature reviews on violent extremism risk assessment, and references of included studies. Studies in all languages were eligible for inclusion in the review. We included studies with primary data resulting from the quantitative examination of the reliability and validity of tools used to assess the risk of violent extremism. Only tools usable by practitioners and intended to assess an individual's risk were eligible. We did not impose any restrictions on study design, type, method, or population. We followed standard methodological procedures outlined by the Campbell Collaboration for data extraction and analysis. Risk of bias was assessed using a modified version of the COSMIN checklist, and data were synthesized through meta-analysis when possible. Otherwise, narrative synthesis was used to aggregate the results. Among the 10,859 records found, 19 manuscripts comprising 20 eligible studies were included in the review. These studies focused on the Terrorist Radicalization Assessment Protocol (TRAP-18), the Extremism Risk Guidance Factors (ERG22+), the Multi-Level Guidelines (MLG-V2), the Identifying Vulnerable People guidance (IVP guidance), and the Violent Extremism Risk Assessment (VERA)—all structured professional judgment tools—as well as Der Screener—Islamismus, an actuarial scale. Studies mostly involved adult male participants susceptible to violent extremism (N = 1106; M = 58.21; SD = 55.14). The types of extremist ideologies endorsed by participants varied, and the same was true for ethnicity and country/continent of provenance. Encouraging results were found concerning the inter-rater agreement of scales in research contexts (kappas between 0.76 and 0.93), but one of the two studies that examined it in a field setting obtained disappointing results (kappas ranging between of 0.47 and 0.80). Content validity studies indicated that PVE risk tools adequately cover the risk factors and offending processes of individuals who go on to commit extremist violence. Construct validity analyses were few and far between, with results indicating that empirical divisions of scales did not match their conceptual divisions. The internal consistency of subscales was lackluster (Cronbach's alphas between 0.19 and 0.85), whereas full scales demonstrated acceptable internal consistency when assessed (0.80 for the ERG22+ and 0.64 for the IVP guidance). Only one study examined convergent validity, and it revealed a lack of convergence, primarily due to particularities of the scale under study (the MLG-V2). Discriminant validity analyses were exploratory in nature, but suggested that PVE risk tools might not be ideology-specific and may apply to both group and lone actors. Finally, although the TRAP-18 showed a relatively strong postdictive effect size (pooled r = 0.62 [0.35–0.77], p = 0.000), the results were highly heterogeneous (I2 = 86%), and all studies used retrospective designs, meaning the outcome was already known at the time of assessment. As such, no included study evaluated true predictive validity (i.e., the ability to forecast future violent extremist outcomes based on prospective risk assessment). This represents a significant evidence gap. Threats to validity were substantial: (a) Many studies were case studies or had very small samples, (b) nearly all samples were constituted through the triangulation of publicly available data, and (c) convenience outcome measures were often used. Although having imperfect data is better than having no data, the current state of empirical validation precludes the recommendation of one tool over another for specific populations and contexts, and calls for higher-quality validation studies for PVE risk assessment tools. Nevertheless, these tools constitute useful checklists of relevant risk and protective factors that could be taken into account by evaluators who wish to assess the risk of violent extremism and identify intervention targets.

Campbell Systematic Reviews, Volume21, Issue4

December 2025, 2025.

By Lucy Cooper and Kevin D. Reyes

Once considered a mere novelty app, TikTok is now a certified force in the information ecosystem.  

The short form video giant is now being used by 14% of Americans as a news platform, according to a Pew Research Centre from 2023, an amount four times more than in 2020.  The impact of the platform, once best known for dance crazes and being a tastemaker for online trends, cannot be ignored. To better understand the impact that TikTok has, in 2023 ISD analysts gathered and analyzed data on trends in hate speech and extremist content on TikTok, and how effectively they were being moderated by the platform. The results, which center on a particular moment in time, have come to inform a series of studies – the first two of which focus on white supremacist content, and anti-migrant and -refugee content. While TikTok appears to have taken measures to improve content moderation practices since ISD’s 2021 study on extremism and hate speech on the platform, this new series demonstrates that TikTok is still ineffective in removing violative content. For example, data for the white supremacy content study was collected during one week in mid-August 2023 and indicates that such content was alive and well on the platform: 70 of the 108 video samples studied were uploaded to TikTok within the most recent three months at the time of collection. Of those 108 videos, the median number of views at the time of analysis was 6,097, a significant increase from ISD’s 2021 report where the median across 1,030 videos was 503 views. The last nine months have been tumultuous for TikTok as a company. In April 2024, President Joe Biden signed a bill that could result in a nationwide ban of the app should TikTok’s parent company, the Beijing-based ByteDance, not sell the platform within 12 months. As part of an ongoing legal fight over the possible ban, the Justice Department, according to the Associated Press, this summer alleged that TikTok was gathering bulk information on users’ “views on divisive social issues like gun control, abortion, and religion,” and harvesting data in violation of children’s online privacy law. As TikTok’s future remains undecided, content moderation issues on the platform persist. In July 2024, ISD published a report detailing the millions of views garnered by a network of neo-Nazi accounts on the platform. Just a month earlier, however, TikTok had published an updated transparency report in which they claimed that in the first four months of this year, moderators proactively removed 97.7% of violative content. Of that same sample, 89.8% were removed within 24 hours, down .1% from that same period in 2023. Despite TikTok’s statements, ISD and similar organizations consistently find content in clear violation of the platform’s policies

London: Institute for Strategic Dialogue 2024. 

By Richard Rosenfeld, James Austin

Employing a small number of predictive variables, the authors of this report created statistical models to forecast violent and property crime rates in New York City. The models estimated yearly changes in New York City’s crime rates from the early 1960s through 2021, estimates that corresponded very closely to the actual rates. The authors then used these models to forecast annual changes in crime rates through 2026. The forecast for violent crime is a slight decrease each year through 2026, while the forecast for property crime shows slight yearly increases. Finally, the projected impact on New York City’s violent crime rate of reducing the state imprisonment rate by 25% would be minimal. No association was found between imprisonment rates and property crime.

New York: Harry Frank Guggenheim Foundation. 2023, 18pg

By Richard Matzopoulos

Not much is known about the perpetrators of male homicide in South Africa, which has rates seven times the global average. For the country’s first ever male homicide study we describe the epidemiology of perpetrators, their relationship with victims and victim profiles of men killed by male versus female perpetrators. We conducted a retrospective descriptive study of routine data collected through forensic and police investigations, calculating victim and perpetrator homicide rates by age, sex, race, external cause, employment status and setting, stratified by victim-perpetrator relationships. For perpetrators, we reported suspected drug and alcohol use, prior convictions, gang-involvement and homicide by multiple perpetrators. Perpetrators were acquaintances in 63% of 5594 cases in which a main perpetrator was identified. Sharp objects followed by guns were the main external causes of death. The highest rates were recorded in urban informal areas among unemployed men across all victim-perpetrator relationship types. Recreational settings including bars featured prominently. Homicides clustered around festive periods and weekends, both of which are associated with heavy episodic drinking. Perpetrator alcohol use was reported in 41% of homicides by family members and 50% by acquaintances. Other drug use was less common (9% overall). Of 379 men killed by female perpetrators, 60% were killed by intimate partners. Perpetrator alcohol use was reported in approximately half of female-on-male murders. Female firearm use was exclusively against intimate partners. No men were killed by male intimate partners. Violence prevention, which in South Africa has mainly focused on women and children, needs to be integrated into an inclusive approach. Profiling victims and perpetrators of male homicide is an important and necessary first step to challenge prevailing masculine social constructs that men are neither vulnerable to, nor the victims of, trauma and to identify groups at risk of victimisation that could benefit from specific interventions and policies.

BMJ Global Health; Volume 9, Issue 4. 2024, 12pg