Where do you start to standardize legislative information for all 50 United States? Blazing an open data trail for one state government isn't easy, so shifting 50 must be nearly impossible. Or is it? The Open State Project is making progress towards the impossible—and closing in on the goal.
When I first heard about this project I thought, that's cool—I wonder how they do it? Then I thought, this must be a nightmare. Can you imagine trying to scrape, sanitize, and standardize data from hundreds of different sources?
James Turk is a developer and open source coordinator for Sunlight Labs. Using an open source approach, Turk is over halfway to completing the impossible—scraping and standardizing legislative information from all 50 states. And the best part is that states are seeing the value in open data and making changes where they have an impact—directly at the source. Turk has a great story about how Open States is a key influencer for a more open government.
I was inspired. Empowering citizens with open data and making it available in a useful format is changing the way our governments operate. Elected officials are held accountable and the transparency of their decisions helps create more informed citizens.
We were fortunate enough to get to talk with Turk, and find out more about his work.
Tell our readers about Open States, why it exists, and how it makes our government more transparent.
Open States is an effort to collect, and to some extent, standardize legislative information from all 50 states. In practical terms, what this means is that we have a huge set of web scrapers that extract details on bills, votes, legislators, and committees.
At the federal level there are a lot of wonderful sites and applications that help people keep tabs on what their representatives in Washington are doing. Many of these were made possible by Josh Tauberer's govtrack.us, a site that takes data from house.gov, senate.gov, and thomas.gov and makes it available in a machine-readable format. This has made it possible for sites like OpenCongress.org, and Sunlight's Congress for Android app to exist.
By working to make this data available for every state in a common format, we hope to build a platform to allow people to do similarly wonderful things at the state level.
What is the biggest impact Open States has had to date?
I suppose it depends on what kind of impact we're talking about. Governments are slowly coming to terms with this and we've seen states like Minnesota and Kansas start to move towards machine-readable access of their data—and I think we can take some of the credit for that. A big part of it is that they just have smart people working there that get the importance of making this data available in as many ways as possible.
I'm particularly partial to the impact that the project has had on individuals. Sunlight open sources everything we do, and as a result, we have over 130 projects on GitHub. Most of these projects aren't things that the average developer uses, so they don't see a ton of attention from outside developers. Open States has been a real success in a unique way—it has gotten developers that were otherwise unaware of open government involved. We've had contributions from approximately 50 developers, ranging from minor tweaks to a parser to fix an error that a user noticed to entire states contributed. I think it has made a real impact in providing a gentle introduction for developers looking for a way to contribute.
How is Open States related to opengovernment.org?
OpenGovernment.org is a project that Sunlight funds but that is developed by our friends at the Participatory Politics Foundation, which also created OpenCongress (with our support). It is being created with the goal of providing a user-facing view of all of the data that we collect in Open States. Right now it supports six states (California, Wisconsin, Louisiana, Minnesota, Texas, and Maryland) so it lags a bit behind where we're at with OpenStates (35 states and counting). But we're working on securing funding to bring it to all 50 states in the next year.
How are these initiatives made possible? Didn't Congress just slash the budget for government transparency programs (from the Electronic Government Fund)?
Data.gov and other programs did face substantial cuts, and every state is facing a budget crunch of its own. Hopefully, they'll see that it is counterproductive to cut spending on open data. Our hope is that by creating access to this data ourselves, states will look at what citizens do with the data once it is out there and realize just how much value they get out of making sure their data is open. We've seen similar things with the various app contests in the open government space. For example, instead of the NYC MTA building their own sub-par transit app, they opened their data and had the people who had really strong opinions about what a good transit app should be, make their own.
What's the next milestone for Open States?
Our biggest push right now is getting to all 50 states. We're in the mid 30s right now and have been going live in around three states a month, which will put us at 50 (actually 52, since we have scrapers for Puerto Rico and Washington, D.C.) in early 2012. Aside from that, we have some really interesting things in the pipeline--a mobile app built on top of our data and some interesting text analysis work that we hope to unveil later this year.
How does someone get involved and help out?
The first step is to engage with us and let us know what you're interested in doing. There's a ton of work available at a wide range of skill levels, so we can find tasks for just about anyone that wants to volunteer their time. We have a mailing list, a contributor's guide and all of our source is on GitHub.
Comments are closed.