In late 2016,  the threat of the impending Trump Administration to access to reliable and accurate government information caused a number of academic libraries, data scientists and others to initiate Data Refuge focused on climate and environmental data. The Environmental Data & Governance Initiative (EDGI), another collaboration, is focused on potential threats to federal environmental and energy policy, and to the scientific research infrastructure built to investigate, inform, and enforce them. Its Capacity and Governance working group monitors changes to federal agency governance, budgets, enforcement, scientific research, and rulemaking capacity.ⁱ

At the time, openness advocates warned that the utility of government data could be easily undermined by not just being removed (or made difficult to find) but also by changes to the data fields. That warning has now proved true — but not in the areas that are being closely monitored by the above efforts. We have learned that it is not only the new leaders of environmental and energy agencies that are a threat to the accurate information needed to hold government accountable. FiveThirtyEight has documented that AG Sessions is yet another.

In late October, the site reported that “The First FBI Crime Report Issued Under Trump Is Missing A Ton Of Info“:

Every year, the FBI releases a report that is considered the gold standard for tracking crime statistics in the United States: the Crime in the United States report, a collection of crime statistics gathered from over 18,000 law-enforcement agencies in cities around the country. But according to an analysis by FiveThirtyEight, the 2016 Crime in the United States report — the first released under President Trump’s administration — contains close to 70 percent fewer data tablesI than the 2015 version did, a removal that could affect analysts’ understanding of crime trends in the country. The removal comes after consecutive years in which violent crime rose nationally, and it limits access to high-quality crime data that could help inform solutions.  …

Among the data missing from the 2016 report is information on arrests, the circumstances of homicides (such as the relationships between victims and perpetrators), and the only national estimate of annual gang murders.

While changes to the report typically go through a body called the Advisory Policy Board (APB), responsible for managing and reviewing operational issues for a number of FBI programs, these changes did not. Rather, the FBI Office of Public Affairs — rather than the Advisory Policy Board — determined which data tables to remove  based on a “review the number of times a user actually viewed the tables on the internet.”

So, can a concerned person or organization obtain the removed fields?  According to FiveThirtyEight,

While the UCR says that the data no longer included in the report was available upon request, the FBI only provided a raw data file, which is more difficult to analyze — especially compared to easily accessible data tables — and does not always match the figures posted online in the UCR reports.³

The FBI noted that in addition to its decision to streamline the report, UCR had launched a Crime Data Explorer, which aims to make crime data more user-interactive. But data contained in the explorer does not replicate what is missing from the 2016 UCR report, and it doesn’t allow users to view data for particular years, but rather aggregates trends over a minimum period of 10 years. The National Incident-Based Reporting System is another tool the FBI uses to provide more detailed information on crimes, but it too does not replicate what is missing from the 2016 UCR report and has a substantially lower participation rate⁴ from police departments across the country.

 

ⁱ The End of Term Harvest and subsequent EOT Web Archive also received intense new interest.The Web Archive contains federal government websites (.gov, .mil, etc) in the Legislative, Executive, or Judicial branches of the government — with a focus on websites that were at risk of changing (i.e., whitehouse.gov) or disappearing altogether during government transitions.