For the 2024 election season, we've updated this reporting to cover the latest changes — and debates — around crime stats.
Nearly 40% of law enforcement agencies around the country did not submit any data in 2021 to a newly revised FBI crime statistics collection program, leaving a massive gap in information sure to be exploited by politicians in midterm election campaigns already dominated by public fear over a rise in violent crime.
The gap includes the nation’s two largest cities by population, New York City and Los Angeles, as well as most agencies in five of the six most populous states: California, New York, Illinois, Pennsylvania, and Florida. (See if your local police reported.)
In 2021, the FBI retired its nearly century-old national crime data collection program, the Summary Reporting System used by the Uniform Crime Reporting (UCR) program. The agency switched to a new system, the National Incident-Based Reporting System (NIBRS), which gathers more specific information on each incident. Even though the FBI announced the transition years ago and the federal government spent hundreds of millions of dollars to help local police make the switch, about 7,000 of the nation’s 18,000 law enforcement agencies did not successfully send crime data to the voluntary program last year.
By contrast, in 2020, around 2,700 agencies did not report crime data to the FBI. (In some cases, an agency will submit data, but the FBI rejects it.)
Since 1930, the nation has relied on the FBI’s data collection to understand how crime is changing, such as how many murders or rapes took place last year, which city had the highest murder rate, or how many people were arrested.
The data gap will make it harder to analyze crime trends and fact-check claims politicians make about crime, and we’ll likely have to live with greater uncertainty for at least a couple of years, criminologists say. Jacob Kaplan, criminologist at Princeton University, said because many big cities and populous states stopped reporting, it’s especially difficult to draw conclusions from the 2021 data.
“I don't think you could get national numbers, at least not useful national numbers, from this data,” Kaplan said. “It's going to be really hard for policymakers to look at what crime looks like in their own community and compare it to similar communities.”
After the FBI changed its crime data collection program in 2021, nearly 40% of local law enforcement agencies did not successfully report data to the federal government, while more than 7,700 agencies reported full year's data to the FBI, and nearly 4,000 agencies reported partial data.
Richard Rosenfeld, a criminology professor at the University of Missouri-St. Louis, called the 2021 data uncertainty “a mess.”
“It's not going to do the national debate over crime levels or crime solutions any good at all,” he said.
Last week, the FBI released crime statistics for the first quarter of 2022, compiled from 56% of law enforcement agencies across the country. Data from more than 8,000 agencies was missing, signaling that the national crime statistics may continue to miss a significant number of agencies in the near future.
Under the old collection system, agencies reported the number of crimes by types of offenses, and the FBI put out annual reports on nationwide trends. But that system didn’t include details like the crime’s time of day or the race and age of victims. It missed modern crimes like cyberstalking, and it only counted the most serious offense for each incident: If someone was shot and robbed, agencies only counted the shooting.
Aware of these shortcomings, in 1988, the FBI launched a new data collection system, designed to track more crimes and in more detail, that ran in parallel with the old system. For decades, police departments that reported crime data to the FBI can choose if they want to transition to the new reporting system.
Then in 2015, the FBI announced that it would retired the old data collection system in 2021. The announcement came after the 2014 police killing of Michael Brown in Ferguson, Missouri, sparked national mass protests against police brutality.
Crime observers suspect large police departments may be wary of the new system precisely because it usually records slightly more crimes than the old one — not because there’s been a crime wave, but because the legacy system undercounted less serious crimes.
To be fair, when the FBI announced years ago that it would retire the old data collection system in 2021, no one could have foreseen the COVID-19 pandemic, the murder of George Floyd and nationwide calls for police reform, or the recent rise in homicides and shootings. Switching their data-reporting system simply wasn’t the priority of most police departments, said Rosenfeld, the criminology professor.
However, the FBI might have prevented the crisis on national crime data by keeping its previous systems running until enough agencies transitioned to the new one, Rosenfeld said.
“The FBI refused to do that, clearly on the assumption that if they gave agencies the option of continuing to report under the old system, they wouldn't transition rapidly,” Rosenfeld said. “But we see they didn't anyway.”
In a statement, a spokesperson said that the FBI surveyed local law enforcement agencies in the spring of 2020 to determine their ability to meet the transition deadline. “Based on the results of the survey, the FBI continued with the January 1, 2021, deadline,” the spokesperson said.
The FBI said it will develop a methodology to account for the gaps and create national crime estimates with the Bureau of Justice Statistics. In a statement, an FBI spokesperson said it has “identified a way forward to account for missing large agencies,” and will focus on producing reliable estimates for states with decent participation rates.
With approximately 65% of agencies participating in the 2021 national crime collection, estimates using the new system “will provide a nationally representative depiction of crime offenses, along with an analysis of crime trends over the past two years,” the spokesperson said.
Many criminologists, however, wonder if national crime estimates are possible with so much missing data.
People are used to working with “5% or 10% of missing data,” said James Lynch, criminology professor at the University of Maryland and former director of the BJS. But making an estimate when 35% of the data is missing, “that's not malpractice necessarily, but you don't want to do that.”
Perhaps the deeper issue is that the collection of national crime statistics has always been under-resourced. “The statistical agencies are generally forgotten and underfunded,” said William Sabol, a criminology professor at Georgia State University and another former BJS director. “There is a sense that somehow these numbers magically appear, and there’s not an understanding on what’s the cost on the local and national level on how these numbers come up.”
For police departments and sheriff’s offices, which handle most law enforcement in the country, participating in the FBI’s new data collection poses considerable challenges.
It means having to train almost everyone on the new data collection — from 911 operators to the patrol officers to the detectives, said Brent Jordan, crime and intelligence analysis manager at the San Diego Sheriff’s Office.
The costs are high. The federal government gave San Diego $2.6 million in 2016 to help the city’s police department and the county’s sheriff’s office make the switch. But the city spent even more to make the transition and to catch mistakes in the agencies’ data.
Those two agencies were among a few in California that sent crime data directly to the FBI in 2021, according to The Marshall Project’s analysis. Most police departments across the country report their numbers to a state agency, who then submit it to the FBI. But California, along with Florida, did not meet the FBI’s criteria to submit data by the deadline, which means most agencies in these states could not contribute to the crime data in 2021.
More than 100 California agencies are expected to transition to the new system later this year, according to a spokesperson from the California Department of Justice. But that won’t include some of the state’s largest police departments. San Francisco police, for example, don’t plan to submit crime data to the FBI until 2025, said William Sanson-Mosier, the department’s acting chief information officer. He said the department asked city leaders for nearly $14 million for a new record system and staff. Running the new system will cost at least $4 million each year for maintenance and support, he said.
That kind of estimate doesn’t surprise Tim Boyle, chief transformation officer at CentralSquare, a software company that sells record management systems to government agencies. He said many small law enforcement agencies are still gathering data with pen and paper, and for larger agencies transitioning to the FBI’s new system “is a Herculean effort”.
Even if a police department can find the money, buying a new system takes 18 to 24 months, Boyle said. “That already puts you two years out, and then implementation is another two to three years.”
There’s also a political challenge for police chiefs, sheriffs and other community leaders: how to explain to the public why the new incident-based data system records seemingly higher crime numbers than the old summary system.
Eric Larson, a major who oversaw the data change at the St. Louis Police Department, emphasized that if three crimes took place within one incident, the old system would count only one, whereas the new system counts all three.
“There have been some difficulties, both internally and externally, to explain why the number looks higher,” Larson said. “It’s not as if new crimes are happening now. We just haven’t been counting them in the summary system.”
Another challenge is how to analyze data stored in the new system, Larson said. For example, As St. Louis was experiencing a spike in homicides, the police department did not update its crime statistics page for six months after it made the transition to the new system last year in December 2020. And it would take longer for the department to produce customized neighborhood-level crime data.
So what will happen with national crime data?
Analysts working on the new data collection have pointed to the federal government’s effort to present state-level data and estimations — detailed reports that would not have been possible without the granular information recorded in the FBI’s new system. One of those reports, Sexual Assaults Recorded by Law Enforcement, includes analysis for 20 states with a good level of participation in the new system. In most states, the report found, children younger than 13 are much more likely to be assaulted by family members, an insight that can only be extracted with the new data collection.
This, many criminologists predict, may be what the FBI's crime collection can produce in the next couple of years: detailed state-level analysis, instead of estimated national statistics on crime trends.
Kaplan, the criminologist at Princeton University, estimated the nationwide agency participation rate in the FBI’s new data system won’t reach 80% until at least 2025.
But the pandemic, the calls for policing reform, and the recent rise in murders and shootings all mean the public needs better crime data than ever, Kaplan said. He said 2021 may have been the worst time to switch the crime data collection system that the nation has been relying on for nearly a century.
“You can’t predict them,” Kaplan said. “But it's a very unfortunate timing.”
