New Data Suggests Risk Assessment Tools Have Little Impact on Pretrial Incarceration

A first-of-its-kind database and report documenting the use of risk assessment tools shows little evidence that the tools are leading to reductions in pretrial incarceration rates or eliminating racial disparities in pretrial release decisions. The organizations behind the work, Media Mobilizing Project and MediaJustice, collected data on risk assessment tools from jurisdictions in 46 states, plus the District of Columbia, encompassing more than 1,000 counties across the United States.

The release of the database, called Mapping Pretrial Injustice, was accompanied on Thursday by a report documenting in-depth interviews with officials at 38 pretrial services agencies and email interviews with officials at 18 others. The interviews revealed that although use of risk assessment tools is widespread, few jurisdictions are monitoring whether the tools serve their intended goals of increasing pretrial release and decreasing missed court appearances and pretrial recidivism. 

While some jurisdictions, like Manhattan, have used risk assessment tools for more than 50 years, many, including Arizona, Utah, and Kentucky, have adopted or expanded their use of the tools in the last decade, fueled in part by the work of Arnold Ventures, which created the Public Safety Assessment in 2012. Proponents of the tools argue that they are less biased than human judges and say that they help courts identify people who can be safely released pretrial, allowing jurisdictions to reduce their use of money bail and pretrial incarceration. 

However, advocates behind the database say their findings bolster a growing body of evidence that risk assessment tools are not effective at reducing the use of money bail and pretrial incarceration and have little impact on racial disparities in pretrial release rates. 

“We’re starting to see more and more information that shows that there is not a one-to-one correlation between risk assessment and a reduction in racial disparities, or risk assessment and decarceration,” said Hannah Sassaman, policy director for the Media Mobilizing Project. “We have all of these decision-makers saying you have to have risk assessment to end pretrial incarceration, [but] that’s not what the facts are showing on the ground.”

Only nine jurisdictions interviewed for Mapping Pretrial Injustice reported that their pretrial jail populations had decreased after their adoption of pretrial risk assessment tools. Roughly two-thirds of the jurisdictions said they didn’t track the impact of risk assessment on their jail populations at all, making it difficult to determine whether the tools were increasing pretrial release rates.

However, pretrial incarceration rates have increased in several jurisdictions that have adopted risk assessment tools in recent years. For instance, Spokane County, Washington, expanded its use of risk assessment tools in January 2017 using a $1.75 million grant from the MacArthur Foundation, but nearly three years later, its jail population remained above 2016 levels. Likewise, in Fresno County, California, which adopted the Virginia Pretrial Risk Assessment Instrument in 2012, the pretrial jail population jumped from around 1,600 in January 2012 to more than 2,000 two years later.

Kentucky, which made the use of pretrial risk assessment tools mandatory in 2011, saw an initial increase in pretrial release rates followed by a gradual return to pre-2011 levels over the next three years. Research published last year by Megan Stevenson, a law professor at George Mason University, found that judges often used their discretion to impose harsher release conditions than those recommended by the risk assessment tools: Although the tools recommended immediate release without bail for defendants in 90 percent of cases, judges followed this recommendation in just 29 percent of the cases Stevenson examined. 

According to Mapping Pretrial Injustice’s report, none of the jurisdictions they interviewed reported any link between their use of pretrial risk assessment tools and reductions in racial disparities. Only a small number told the interviewers they were attempting to measure racial disparities at all. Just one jurisdiction, Buncombe County, North Carolina, told interviewers that it trained judges and other pretrial decision-makers on racial disparities and implicit bias. 

Critics of risk assessment tools have long argued that the algorithms used to measure risk themselves are racially biased. They note that studies have found Black and Latinx defendants are more likely to be flagged as “high risk” than white defendants because the tools rely heavily on factors that are shaped by racial disparities in policing, like criminal history. 

According to data scientist Cathy O’Neill, author of “Weapons of Math Destruction,” any risk assessment algorithm based on arrest data will necessarily reflect the racial biases that already exist in the criminal legal system.

“We don’t have crime data. We have arrest data. Most crimes do not lead to arrest, but, in particular, the distribution of the missingness is not equally distributed. We have way more missing white crime than Black crime. And what that means is that when you feed this so-called crime data, which is actually arrest data; into these algorithms, of course they predict that the police are going to go rearrest Black people,” O’Neill said during a launch webinar organized by Mapping Pretrial Injustice.

In 2018, a coalition of more than 100 civil rights organizations released a statement calling for the abolition of pretrial risk assessment tools, writing that they “are not a panacea to reforming our unjust and broken bail systems, and that, in fact, these tools can worsen racial disparities and allow further incarceration.”

The advocates behind Mapping Pretrial Injustice hope that the data they gathered will help demystify the often confusing debate around pretrial risk assessment. In December, Community Justice Exchange released a toolkit for organizers who are seeking reforms to pretrial risk assessment in their own communities.

“Directly impacted communities have a right to take up space in these conversations because these are issues that are actually impacting their lives. … The data from these tools is data from our actual communities, people that are justice-involved,” said Myaisha Hayes, director of campaign strategies for MediaJustice. “I hope that organizers and activists see the fact that a community-based, grassroots organization put together this research as an indication that we have the agency to create our own analysis and critiques about the way that these tools are being used.”

The bottom line, according to advocates, is that pretrial release isn’t all that “risky” in the first place, and the risks that do exist rarely justify detention.

In Cook County, Illinois, data released by the county court system showed that fewer than 1 percent of people released pretrial were rearrested for a violent crime, even if the assessment flagged them as being a high risk for violence. Likewise, in Washington, D.C., where 94 percent of people accused of a crime are released without cash bail, fewer than two percent are rearrested for violent crime during their release.

“Caging people in pretrial detention does not make us safe,” Lola Rainey, director of the Tucson Second Chance Community Bail Fund, said during the database launch webinar. “We need to ask ourselves why, as a community, why, as a society, we think some people are entitled to freedoms and liberties and some people are not. And most of the people we’re denying those liberties to are people of color or people who are otherwise marginalized. That’s the real conversation.”