You are viewing this article in the AnnArbor.com archives. For the latest breaking news and updates in Ann Arbor and the surrounding area, see MLive.com/ann-arbor
Posted on Fri, Jun 4, 2010 : 6 a.m.

FOIA Friday: publishing FOIA logs in Chicago and more with Socrata

By Edward Vielmetti

The Freedom of Information Act requests that you send to a public body are themselves public records, which you can request by virtue of the FOIA.

I've done that before, in order to audit what other news organizations are up to and to see just what the range of requests are that make their way to an agency. The copy of all requests from agencies that regularly get a lot of requests, like police departments, can be very thick once you print it out on paper at a cost of a nickel or more per page.

There are two ways for an agency to publish its FOIA logs. One is to give requesters a copy of each and every individual request, photocopied or scanned from a standard piece of paper. A better way is to compile a spreadsheet with the relevant details of each request and release that.

Once you get to the point of thinking about things in a spreadsheet format, it's worthwhile taking a look at how that spreadsheet gets published. That brought me to Socrata, a "Government 2.0" styled system used by Chicago to publish FOIA logs. It's also used by Seattle and Medicare to publish other spreadsheet-styled data.


Chicago: Office of the Mayor

FOIA Request Log - Office of the Mayor

Powered by Socrata

The Office of the Mayor's log is an interesting first start.

The City of Chicago is subject to the new State of Illinois FOIA law, which places a heavier burden on agencies throughout the state to respond promptly to FOIA requests.

The first thing that you will notice is that it's impossible to directly cut and paste the text on the screen, and impossible to download it as a spreadsheet. To interact with the mayor's FOIA log, you need to look at it in such a way that the mayor's office can track your every move. In order to get a data dump, you first need to get to the menu, select "download this dataset," and you'll get a menu of options.

The tool being used to publish this, Socrata, offers several formats for the download. You can get the Mayor's FOIA log as CSV (spreadsheet), PDF (printable), or even JSON. The latter would allow you to embed this data into a web application very easily.

Chicago-area investigative reporters were incensed when Mayor Richard Daley announced the new FOIA log policy. Their carefully crafted requests would no longer be secret, and the investigative strategy they were pursuing would be made public well before their actual stories were filed. "Tomorrow's headlines today" said Timeout Chicago, noting that "Apparently, Mayor Daley’s pretty pleased about this new move toward transparency. But how about newsrooms that can now readily keep tabs on - and potentially duplicate - requests filed by competitors? Could get interesting."

As to the actual requests, Mick Dumke of the Chicago Reader asked on May 4 for a "copy of the mayor's appointment calendar daily schedule from Nov. and Dec. of 2009 and Feb. through April of 2010." He was promised a response by May 18.

The actual responses are not included in this log. You would have to make a separate request for a copy of what Dumke got, and it might take you the full FOIA cycle time to get it back.

Dumke has an interesting relationship with Mayor Daley. A few days after his request was scheduled to be returned to him, Daley pointed a rifle with a long bayonet directly at Dumke during a press conference on gun control in the city.

I doubt that this happened because of the FOIA request, but then again, you never know.


Socrata as a publishing tool: Seattle

Socrata is used by several government agencies as an outsourced publishing tool for structured electronic records and public databases. The Socrata weblog is as good a place as any to start exploring what is available.

Seattle Real Time Fire 911 Calls


The Seattle data catalog has a bunch of data sets online using Socrata. A relevant point of comparison is the City of Ann Arbor data catalog. I won't do a point by point comparison now, but it's information of comparable interest and worth a future study. The excerpt published is a real time 911 log, with geocoded information about police calls for service; the comparable City of Ann Arbor police calls for service log has much more detail about each call, but it lags real time by about three weeks.


Socrata as a publishing tool: Medicare public data

Dialysis Facility Compare - Listing by Facility


The Medicare data catalog has a lot of interesting data, which I haven't sifted through; one of its popular datasets is a comparison of dialysis facilities.


FOIA requests in progress

Last week I wrote that I was looking for information about what I called "day lily tickets," but which are more precisely called enforcement notices for violations of city code related to the lawn extension ordinance.

I'm holding off on filing a formal FOIA request for some of the data that I want, because I don't know yet how it is organized and I really, really don't want to force someone to do a lot of work. On the other hand, I really do want to know a lot more than I do about routine enforcement of city codes.

Edward Vielmetti writes the FOIA Friday column for AnnArbor.com. Click here to read past FOIA Friday columns. Contact him at edwardvielmetti@annarbor.com.

Comments

Safouen

Sat, Jun 5, 2010 : 1:27 a.m.

Aside from the intrinsic value of the data being published in these three examples, each use case that you describe points to some interesting story lines and invites further discussion. This is why it's important to publish these data sets in a format that makes it easy for non-technical constituents to access, visualize and share on blogs and elsewhere on the web. The other consideration is to enable API (or programmatic) access to the data so that software developers can create interesting applications that make use of the data, much like we have with GPS/GIS data in the last 10 years. Or, in the case of Seattle, be able to update the 911 data set in realtime as opposed to every few weeks. Unfortunately, all too often, public data is still being published in a CSV or another inert format that really limits its usefulness. Thank you for using the Socrata data player on your blog, we'd be very interested to hear your feedback on the experience.