Mapping the scientific knowledge and approaches to defining and measuring hate crime, hate speech, and hate incidents: A systematic review

By Matteo Vergani, Barbara Perry, Joshua Freilich, Steven Chermak, Ryan Scrivens, Rouven Link, Daniel Kleinsman, John Betts, Muhammad Iqbal

Background

The difficulties in defining hate crime, hate incidents and hate speech, and in finding a common conceptual basis constitute a key barrier toward operationalisation in research, policy and programming. Definitions disagree about issues such as the identities that should be protected, the types of behaviours that should be referred to as hateful, and how the ‘hate element’ should be assessed. The lack of solid conceptual foundations is reflected in the absence of sound data. These issues have been raised since the early 1990s (Berk, 1990; Byers & Venturelli, 1994) but they proved to be an intractable problem that continues to affect this research and policy domain.

Objectives

Our systematic review has two objectives that are fundamentally connected: mapping (1) original definitions and (2) original measurement tools of hate crime, hate speech, hate incidents and surrogate terms, that is, alternative terms used for these concepts (e.g., prejudice-motivated crime, bias crime, among many others).

Search Methods

We systematically searched over 19 databases to retrieve academic and grey literature, as well as legislation. In addition, we contacted 26 country experts and searched 211 websites, as well as bibliographies of published reviews of related literature, and scrutiny of annotated bibliographies of related literature.

Inclusion Criteria

This review included documents published after 1990 found in academic literature, grey literature and legislation. We included academic empirical articles with any study design, as well as theoretical articles that focused specifically on defining hate crime, hate speech, hate incidents or surrogate terms. We also reviewed current criminal or civil legislation that is intended to regulate forms of hate speech, hate incidents and hate crimes. Eligible countries included Canada, USA, UK, Ireland, Germany, France, Italy, Spain, Australia and New Zealand. For documents to be included in relation to research objective (1), they had to contain at least one original definition of hate speech, hate incidents or hate crimes, or any surrogate term. For documents to be included in relation to research objective (2), they had to contain at least one original measurement tool of hate speech, hate incidents or hate crimes, or any surrogate term. Documents could be included in relation to both research objectives.

Data Collection and Analysis

The systematic search covered 1 January 1990 to 31 December 2021, with searches of academic databases conducted between 8th March and 12th April 2022 yielding 35,191 references. We carried out country-specific searches for grey literature published in the same time period between 27th August and 2nd December 2021. These searches yielded a total of 2748 results. We coded characteristics of the definitions and measurement tools, including the protected characteristics, the approaches to categorise the ‘hate element’ and other variables. We used univariate and bivariate statistical methods for data analysis. We also carried out a social network analysis.

Main Results

We provide as annex complete lists of the original definitions and measurement tools that met our inclusion criteria, for the use of researchers and policy makers worldwide. We included 423 definitions and 168 measurement tools in academic and grey literature, and 83 definitions found in legislation. To support future research and policy work in this area, we included a synthetic assessment of the (1) the operationalisability of each definition and (2) the theoretical robustness and transparency of each measurement tool. Our mapping of the definitions and measurement tools revealed numerous significant trends, clusters and differences between and within definitions and measurement tools focusing on hate crime, hate speech and hate incidents. For example, definitions and measurement tools tend to focus more on ethnic and religious identities (e.g., racism, antisemitism, Islamophobia) compared to sexual, gender and disability-related identities. This gap is greater in the definitions and measurement tools of hate speech than hate crime. Our analysis showed geographical patterns: hate crime definitions and measurement tools are more likely to originate from Anglophonic countries, especially the USA, but hate speech definitions and measurement tools are more likely to originate from continental Europe. In terms of disciplinary fragmentation, our social network analysis revealed that the collaboration and exchange of conceptual frameworks and methodological tools between social sciences and computer science is limited, with most definitions and measurement tools clustering along disciplinary lines. More detailed findings are presented in the results section of the report.

Authors' Conclusions

There is an urgent need to close the research and policy gap between the protections of ‘ethnic and religious identities’ and other (less) protected characteristics such as gender and sexual identities, age and disability. There is also an urgent need to improve the quality of methodological and reporting standards in research examining hate behaviours, including transparency in methodology and data reporting, and discussion of limitations (e.g., bias in data). Many of the measurement tools found in the academic literature were excluded because they did not report transparently how they collected and analysed the data. Further, 41% of documents presenting research on hate behaviours did not provide a definition of what they were looking at. Given the importance of this policy domain, it is vital to raise the quality and trustworthiness of research in this area. This review found that researchers in different disciplinary areas (e.g., social sciences and computer science) rarely collaborate. Future research should attempt to build on existing definitions and measurement tools (instead of duplicating efforts), and engage in more interdisciplinary collaborations. It is our hope that that this review can provide a solid foundation for researchers, government, and other bodies to build cumulative knowledge and collaboration in this important field.

Campbell Systematic Reviews, 

28 April 2024