Get Informed

Subscribe to our newsletters for regular updates, analysis and context straight to your email.

Close Newsletter Signup

How Zombie Crime Stats, Phantom Stats and Frankenstats Paint a Misleading Picture on Crime

Jorg Greuel for Photodisc via Getty

How Zombie Crime Stats, Phantom Stats and Frankenstats Paint a Misleading Picture on Crime

In September 2017, newspapers across the country ran headlines of a similar theme: According to data from the FBI’s Uniform Crime Reports, the agency’s official report on criminal behavior nationwide, crime — or at least violent crime — had risen for the second year in a row.

That’s not entirely true. “Violent crime” hadn’t risen. The violent crimes that we count — the so-called “index crimes” of murder/manslaughter, rape, robbery, and aggravated assault — had risen. Simple assaults? Sexual assaults that don’t rise to the level of rape? We don’t measure those crimes. The crimes we do measure were all chosen during the development of the UCR in the late 1920s, on the grounds that they were common, serious, and generally reported — which is true, but we haven’t updated the list since.

And even saying that “index violent crimes” rose isn’t quite right. Index violent crimes reported to the police had gone up. But a large fraction of crimes are never reported, perhaps fewer than half of all violent crimes and barely 50 percent of all serious violent crimes. And the widely reported UCR data are based only on crimes recorded by the police.

Well, some of the police. Participation in the UCR is voluntary, so it provides data on index crimes reported to the police by departments that then report to the FBI, with some efforts to fill in the gaps from those that don’t report at all or provide incomplete data. About 5,000 of the nation’s 18,000 or so police agencies — so something on the order of 20 to 25 percent — don’t appear to report sufficient data.

Oh, and the data is nearly an entire year out of date by the time it is reported to the public. The headlines in September 2017 about the rise in violent crimes were about the just-released UCR data… from 2016. Which, to be clear, is as close to just-in-time statistics as criminal justice stats get, but still potentially misleading. The number of homicides in Chicago rose by almost 60 percent from 2015 to 2016, but by the time the 2016 crime stats were released, Chicago was on course to see a 14 percent drop by the end of 2017.

So, “Violent crime is up!” is what the headlines say, but “According to agencies providing data to the FBI, the number of incidents of four serious types of violent crimes reported to or seen by the police rose nearly a year ago” is what they ought to say.

Welcome to the world of criminal justice statistics. At the heart of the push from being tough on crime to smart on crime is a desire to create a criminal justice system based on what works, and that should mean a criminal justice system that has accurate, up-to-date data that can shape and influence policy.

What we have instead is something akin to a horror movie bestiary. We have zombie statistics — numbers that haven’t been updated in years, like a detailed inmate survey that is supposed to be conducted every seven years but was last run in 2004. We have phantom statistics, those numbers that we ought to have but are invisible since we never gather them at all, such as anything on plea bargaining (despite the fact that about 95 percent of all guilty verdicts come from pleas… we think). And, perhaps worst of all, we have Frankenstatistics, those numbers that at first blush seem to measure one thing, but when looked at closely are tracking something altogether different.

Recidivism stats, for instance, are completely blind to an entire way of thinking about trends in reoffending. They can’t measure if someone is committing fewer crimes than before, only if he or she manages to completely avoid re-arrest. And they don’t really measure the trend they purport to measure in the first place (since they don’t track if the person fails to reoffend, only if he or she fails to be rearrested, which depends a lot on what the police are doing).

Far too often, journalists and policymakers alike invoke zombie statistics without acknowledging that they may no longer reflect current conditions, they cite Frankenstats at face value without considering what they are reallymeasuring, and they rely on anecdotes to fill the gaps left by the phantoms. None of these practices is acceptable, even if some of them often feel unavoidable. (I myself have been forced to rely on anecdotes more than I’d like.)

But it is also understandable. The defects in our criminal justice statistics are buried deep in the fine print, invisible to all but those who spend their days mired in them. The UCR stats are noisy and complex and imperfect, yet the FBI reports them with such specificity — there were “exactly” 803,007 aggravated assaults 2016, not 803,006 or 803,008 — that most people would likely think they are precisely measured. Our prison population statistics provide detailed national numbers, but unless you have access to the underlying data, which requires an application and a special encrypted hard drive and (it appears) an academic affiliation, you’d never know that Southern states systematically under-report data, which may introduce a bias, though of what sort we can’t really say.

There’s no reason we can’t have better criminal justice data. After all, other agencies produce detailed data far more rapidly. The Bureau of Labor Statistics releases employment data monthly, not with a nearly year-long lag. Of course, the BLS has a budget of almost $650 million, compared to under $40 million for the Bureau of Justice Statistics. Reliable up-to-date statistics cost money, money we’ve been so far unwilling to spend.

But there is some good news on the horizon. The FBI is hoping to complete a decades-long revamp of the UCR by 2021. The BJS is in the process of expanding its important national survey of criminal victimization to help explain what is happening at the local level. And while hamstrung by inadequate budgets, researchers at the BJS continue to work to improve and modernize other datasets as well.

In the meantime, however, it is essential that we understand exactly what our criminal justice statistics can and cannot say, and how they can both inform and mislead, which is what I intend to do here in the months ahead. The increased focus on data-driven criminal justice policy is an essential step forward, but it has to be done with a unflinching appreciation of just what that data looks like.

Dallas County Democratic DA Candidates Promise Change, But Their Plans Don’t Go Far Enough

Candidate Elizabeth Frizell speaking at the ACLU of Texas candidate forum

Dallas County Democratic DA Candidates Promise Change, But Their Plans Don’t Go Far Enough

On February 10, the ACLU of Texas held a forum for the Dallas County district attorney candidates in anticipation of the March 6 primary. For almost two hours, Democratic candidates Elizabeth Frizell and John Creuzot fielded questions as Anthony Graves, a man who was wrongfully convicted of murder and spent 18 years on death row, led the conversation. Questions on issues like immigration and addiction came from members of the audience who, like Graves, had been directly affected by the criminal justice system. Notably absent was the Republican incumbent, DA Faith Johnson, who declined to participate.

Frizell and Creuzot are both Black Democrats and former judges who left the bench to practice criminal defense. Both are part of a new generation of district attorney candidates who willingly admit that overzealous prosecution is largely to blame for the nation’s incarceration crisis. “There’s one reason and one reason only that I want to be DA,” Frizell said, by way of introduction. “I want to reduce mass incarceration.” Similarly, Creuzot proclaimed that prosecutors have a “legal, moral, and ethical responsibility to reduce incarceration in our criminal justice system.”

Frizell and Creuzot seem to grasp the urgency and importance of criminal justice reform, in stark contrast to Johnson, who has embraced a tough-on-crime ethos and overwhelmingly resisted efforts at reform. If elected, either Frizell or Creuzot will have the chance to reorient the office toward more progressive prosecution, giving Dallas the potential to follow in the footsteps of other major cities, like Philadelphia, Houston, and Chicago, where voters have demanded a different approach to crime and punishment.

Yet, the candidates have shared few details regarding how they would reform the office. Neither has adequately articulated exactly how they would reduce mass incarceration with the existing tools at the district attorney’s disposal. And neither has set clear targets that voters could later use to hold them accountable.

The Dallas district attorney’s office is in dire need of reform. It is the ninth largest prosecutor’s office in the nation, and its prosecutors handle over 100,000 cases a year. And Johnson, who was appointed in 2016, has failed to address even the most obvious flaws.

Take the county’s bail program, for example. Currently, county magistrates set bail in secret hearings, with “little to no information” about a defendant. They depend on a fixed bond schedule, and the bail amount is determined solely by the alleged offense, without any consideration of a defendant’s ability to pay. As a result, 70 percent of the approximately 5,000 people in Dallas jails each night are stuck there simply because they can’t afford bail. And hundreds of those people are charged only with misdemeanors. Dallas has two million fewer people than Harris County, Texas, but almost twice as many people in jail pre-trial on misdemeanor charges.

In fact, Dallas’s bail system is so unfairly punitive that just last month, Civil Rights Corps, a nonprofit organization dedicated to challenging systemic injustice in the American legal system, filed a lawsuit against the county, alleging that the county’s money bail system is a “rampant and flagrant” violation of poor defendants’ equal protection and due process rights.

Johnson has said that “her office does not support any policy that results in the defendant being held in jail before trial or disposition only because they do not have the financial means to get out jail.” She and other county officials claim they are working to develop a bail system that relies on risk assessments, which are used to predict which defendants are likely to get rearrested or fail to return to court, rather than a fixed bond schedule. But that approach is moving too slowly — at best, county officials hope to implement a new procedure before the end of the year.

Johnson hasn’t just dragged her feet on bail reform. She was also reluctant to introduce a common sense cite-and-release program. These policies vary between jurisdictions, but they generally allow law enforcement to issue a citation rather than arrest someone for a particular low-level offense, helping to stem the tide of arrests and incarceration — while potentially saving the county millions of dollars.

In other big Texas counties, prosecutors have coordinated with local law enforcement and used their broad discretion to implement these reforms. Last year in Harris County, for example, District Attorney Kim Ogg announced plans to expand the county’s cite-and-release program dramatically, stating that the office would no longer prosecute most misdemeanor marijuana cases. “We have spent … over a quarter-billion dollars prosecuting a crime that has produced no tangible evidence of improved public safety,” she said at the time. Now, a person caught with less than four ounces of marijuana must hand it over to law enforcement, sign a contract promising to attend a four-hour drug education class, and pay a $150 fee, which can be waived for those who can’t afford it. No charges, no arrest, no record.

There are a number of other cite-and-release policies in the state, including a program enacted just last month in Bexar County, the fourth largest county in the state, and another in Nueces County.

But what about Dallas? Johnson could have easily followed in the footsteps of these other prosecutors. She had the discretionary authority and the political power to implement her own cite-and-release program, and she had support from the city and county legislative bodies. Like other district attorneys, she could have partnered with law enforcement and saved her county a significant amount of money while reducing unnecessary prosecution. And yet, Johnson was slow to implement a cite-and-release program in Dallas.

Eventually, Dallas officials passed a cite-and-release policy for low-level marijuana arrests, which went into effect December 1. But the program still doesn’t go as far as it should. Under this policy, a citation still results in a misdemeanor charge. For many, the penalties for possession are the same as they were before the policy — up to 180 days in jail and a $2,000 fine. And, unlike many other cite-and-release policies, only first-time offenders are eligible for Johnson’s diversion program, which is also particularly onerous — participants have to take a 15-hour class, do a community project, and pay a $620 fee.

And the program isn’t affecting a significant number of people, either. At the end of January, the Dallas Observer reported that police had written just 23 citations for marijuana possession, and that every single recipient was Black or Latino. Meanwhile, in 2018 alone, the district attorney’s office has filedover 500 low-level marijuana possession cases.

As district attorney, Johnson has squandered opportunities to be a visionary, and her refusal to push for changes in Dallas’s criminal justice system means the city has a long way to go before it could be considered a bastion of reform. Still, both Democratic candidates seem determined to lead the way.

For instance, both Frizell and Creuzot have made it clear that they believe Dallas’s bail system is unconstitutional. Frizell supports ending money bail entirely and wants to move toward a risk assessment system. In the meantime, however, she promised to train prosecutors to recommend release on personal recognizance and not to argue against lowering bonds for first time non-violent offenders. Creuzot has not said he wants to end money bail entirely, promising instead to “train [his] assistants to recognize and appreciate the role of risk in determining the appropriate bond amount and to request only an amount necessary to ensure appearance in court.”

Meanwhile, they have also both pledged to make changes to the cite-and-release process, although their promises remain untenably vague. Frizell has pledged to further decriminalize marijuana, while Creuzot has listed the expansion of cite-and-release as one of his top priorities.

There are some areas where the candidates display unfortunate similarities to Johnson. On the death penalty, for example, neither candidate has categorically ruled out capital punishment for the most extreme cases. Last year, Johnson twice sought the death penalty — the office’s first and second time doing so since 2014. Ultimately, both attempts were unsuccessful. Erbie Bowser, who was charged in a deadly shooting spree, was convicted of capital murder but received an automatic life without parole sentence last May after the jury deadlocked during the penalty phase. And in November, Johnson’s office reversed its decision to seek the death penalty for Antonio Cochran, who killed a teenage girl, after a psychiatric evaluation proved he was intellectually disabled.

Despite their similarities on the death penalty, there’s no denying that both the Democratic candidates support more humane policies than Johnson does. Yet, it’s unclear if they would truly reform the office.

Ultimately, they both support most of the broad strokes of criminal justice reform. But the broad strokes are not enough to reduce mass incarceration. Voters need to know how the DA would wield his or her discretion under current law, without the benefit of external funding, outside approval, or a shift in legislation.

Discretion is a prosecutor’s greatest tool and most powerful weapon. To reduce mass incarceration, prosecutors must use that discretion to imprison significantly fewer people for significantly less time. In this office alone, poorly wielded discretion has the potential to destroy the lives of thousands and thousands of people. These candidates must be clear about what offenses they’ll categorically decline to prosecute, and set achievable goals to reduce the prison population by a designated amount. It’s not enough to hint at these objectives. For voters to make informed decisions at the ballot box, these plans must be plainly stated.

Both Creuzot and Frizell have accurately identified the criminal justice system’s problems in general — from mass incarceration and racial disparities to overcharging and police brutality. Both have said they will focus on treatment over prosecution for drug offenses and promise to send more mentally ill defendants to diversion programs. Both pledge to consider immigration consequences in every case. They’ve both promised to engage more with the community, increase trainings for prosecutors and improve transparency.

These may be steps in the right direction, but they are far too vague to inspire confidence. Without additional action, these commitments will not reduce mass incarceration. And anything that falls short of that goal is simply not enough.

Good rhetoric and good intentions often do not translate to effective policy. The candidates for district attorney must also articulate the end game — fewer people spending less time in prison, and far fewer people involved in the criminal justice system. Otherwise, they risk falling into the same traps as Johnson and her more draconian predecessors, and Dallas residents will end up grappling with the same injustices to which they currently fall prey.

More in Explainers

Court Watch NYC Is Here To Hold New York City’s ‘Reformer’ District Attorneys Accountable

@CourtWatchNYC on Twitter

Court Watch NYC Is Here To Hold New York City’s ‘Reformer’ District Attorneys Accountable

As the conversation about criminal justice reform increasingly focuses on the nation’s broken bail system, prosecutors across the country have announced new policies that purportedly aim to keep low-income people from being denied their freedom simply because they can’t afford to pay bail.

In New York City, Brooklyn District Attorney Eric Gonzalez and Manhattan District Attorney Cyrus Vance recently announced reforms that would limit requests for bail when people are arrested for some nonviolent offenses. The media has rapturously greeted these announcements with headlines that allow the prosecutors to portray themselves as part of the solution to America’s incarceration crisis.

But following the announcements by the district attorneys, public defenders saw little to no change in the actual practice of prosecutors, who were still requesting bail on the exact crimes they had explicitly pledged not to. When pressed, the prosecutors’ offices replied that they’re indeed asking for bail less often, but have so far released no data to back up their claim. In addition, many of the new policies include carve-outs for people on parole, people who have been arrested in the past, and those who have had other interactions with the criminal justice system. So, the very people most likely to be arrested in New York City are also the ones who would be the most likely to be held on bail.

Enter Court Watch NYC, a new project that’s officially launching today, which aims to hold “reformer” prosecutors accountable, and to get as many people as possible to sit in on arraignments, where bail determinations are made. These hearings have an outsized impact on whether people who get arrested can keep their jobs, reunite with their family, or stay in school. If bail is set, they can sit in jail for weeks, months, or years awaiting trial simply because they can’t afford it.

“There are so many caveats to these policies,” said Alyssa Aguilera, co-executive director of VOCAL-NY, one of the organizations behind Court Watch NYC, “that they’re not having as much of an impact as they should, and it’s our job to make sure that they’re actually impactful and achieving their intended goals.” The other two organizations behind Court Watch NYC are the Brooklyn Community Bail Fund, which raises money to pay bail for low-income defendants while advocating for the abolition of cash bail, and 5 Boro Defenders, a coalition of public defenders in New York City organizing around injustices in the legal system.

Before its official launch today, Court Watch NYC in January began training “courtwatchers,” who have already sat in on arraignments in Manhattan and Brooklyn, taking note of when prosecutors request bail, the charge the person was arrested for, and other information such as the defendants’ language skills and who was representing them.

So far, the project has trained over 150 “courtwatchers,” who can sign up for shifts to sit in on arraignments in the two boroughs. Court Watch NYC organizers hope that knowing observers are present in the courtroom might encourage judges and prosecutors to change how they set bail. “We know that by putting these people who are electing them into the actual courtroom … it can shift power and court practices in that very specific case,” said Rachel Foran, managing director of the Brooklyn Community Bail Fund. “That power shift is going to change the way that justice is meted out on a day-to-day basis.”

The project takes inspiration from court-watching initiatives like Court Watch NOLA, Participatory Defense in California, and the Chicago Community Bond Fund. Another court-watching project in New York City, the Police Reform Organizing Project, focuses on racial disparities in arrests made by the NYPD.

Court Watch NYC plans to share its findings in weekly reports on its website, and will release the data it collects from its watchers. While the reports and data will be helpful in keeping prosecutors accountable, the organizers behind the project say the most important part of Court Watch NYC will be getting more New Yorkers to see how the court system functions, and whether true reform is actually occurring in the courtroom.

“The data is important, but it’s not our end goal,” said Foran. “The project is about community engagement and continual participation in the system and opening an avenue for people to have that engagement.”

More in Podcasts