New CAL-ACCESS code release offers more command options, flexibility for users

Today we’re releasing version 0.2 of django-calaccess-raw-data, our open-source software for handling California’s campaign finance and lobbying activity data.

The goal of the app is simple: Make access to the bulk data behind CAL-ACCESS — the jumbled, dated and difficult site tracking money in state politics — as simple as a few keystrokes.

With the improvements included in this release, our users now have greater control over the flow of the data pipeline.

What’s in the release

• A new command, updatecalaccessrawdata, downloads, cleans and loads the raw data into your database in one straightforward step
• If your download is interrupted, restart were you left off by invoking the --resume-download command line option
• Load files into any database configured in your Django project by invoking the --database option
• The loadcalaccessrawfile command takes a --csv option in case you need to load from another source into our data models
• Since downloading and processing up to 12 gigabytes of data has been an issue for some of our users, each of the app’s commands will now clear out the original and intermediate files once they are loaded into the database. If you’d prefer to keep these files, you can invoke the --keep-files option.
• Expanded documentation of all the management commands
• Support for Django versions 1.8 and 1.9
• Support for Python version 3.5
• Hundreds of table and field definitions patched by dozens of contributors at our California Code Rush events
• Automatically generated table documentation based on those contributions

Who to thank

Dozens of contributors have volunteered their time and energy to make this release possible.

In particular, Ben Cipilloni and Aaron Borden authored or instigated many of the improvements to the management commands that form the backbone of the application. Ben and Aaron are both doing great work with CA Civic Lab, a collection of regional groups using our code as a foundation to analyze money in local elections.

What’s next

There’s still much to do. This release is only an early milestone toward the goals set out last year when our project was named a winner of the Knight News Challenge.

Stay tuned for future releases, which will include a public website where simplified versions of the state’s bulk data will be available for download and reuse.

How you can help

Dozens of issues remain unresolved on our GitHub repositories. Many more will be added soon. Jump in and see if anything interests you.

If you’d like to contribute to the raw data application, a good place to start is installing the source code for yourself, as outlined here in our getting started guide.

If you have any questions, contact us at [email protected]. Or find us anytime in News Nerdery’s #california-civic-data Slack chat or in #californiacivicdata on Freenode IRC.