5/10/2017

Enigma Public

 

The world's broadest collection of public data.

By India Kerle, Associate Product Manager, Enigma

Data released by federal departments, state agencies and the like can reveal trends or anomalies that are in the public’s interest for data journalists to cover. From housing and pricing to hiring and the environment, public data can be critical to piecing together a story. Yet, as valuable as it is, public data can be equally as difficult to track down. While some cities and states have moved forward with open data portals, many critical datasets are still only accessible by way of a Freedom of Information Act (FOIA) request — a practice in patience, often involving check writing and weeks of waiting for a CD to arrive via snail mail. As most journalists working on a time sensitive piece would agree, this process is, well, less than ideal.

Enigma, an operational data management and intelligence company, believes that accessing this information shouldn’t be such a challenge. This summer, they launched Enigma Public, a community platform to help connect you to public data. Best of all? The tool is free to use for journalists and for all non-commercial purposes.

Enigma Public, the world’s broadest collection of public data, contains datasets from tens of thousands of sources for you to search, browse, download, access via our API and integrate into data driven storytelling. We bring together topics ranging from building permits and corporate registrations to the oil production of a specific well in North Dakota and the contents of all container ships arriving in US ports.

Now that you know Enigma Public exists, how should a journalist put it to use? We’ve compiled some tips below to help get you rolling.

Image:The Enigma Public homepage featuring guides on specific datasets.

Search Enigma Public

Investigating a particular person, business, or address? Search an entity or individual to investigate how it is represented across datasets from a variety of different sources. You can  search terms both within the contents of a dataset and across dataset names and collections to find unexpected connections in the data. Take, for example, a subject particularly well-covered: the pop superstar, Beyoncé. Searching for the singer in Enigma Public reveals her appearance in over 60 datasets, ranging from her stage equipment shipments around the world in the 2016 Bill of Lading Summary to records of the dates and purposes of her White House visits under the Obama administration.

Image: Searching Enigma Public, which scans the contents of datasets as well as the titles.

Filter and view statistics quickly

Once you find information relevant to you, make use of our summary statistics to get a better sense of what is represented within a specific dataset. We’ve identified the most frequent values, making it easier to spot outliers quickly. For instance, if viewing the summary stats for our San Francisco 311 data — a collection of details about non-emergency calls made to the city’s 311 department — you’d learn that most calls are made in the Mission police district and the most frequent complaint relates to ‘other loose garbage’.    

Image: Quickly viewing fill-rate, frequency rank and other summary statistics.

Export and access via API

Found something useful for a story? Pull the relevant data via our free-to-use API to keep your visualizations up-to-date as we update the data.

Find data in context

Beyond acquiring and updating a wide range of datasets, we also seek to place data in its appropriate context. We do this in a variety of ways by providing rich metadata, all the information about where the data comes from and what it means, derived both from source data dictionaries (when available) and through extensive outside research. We also offer a curated collection of datasets where we’ve already done the needed cleaning and standardization work across states.

Image: Understand what a dataset means quickly through detailed descriptions shown at left.

Discover a new idea

Just browsing? Tens of thousands of datasets is quite a lot to sift through. Now on our homepage you’ll find a rotation of featured datasets highlighting interesting, unusual, or newsworthy data. The guides offer descriptions of the dataset as well as some helpful tips and observations about how to work with and ask questions of the data. One example is a reminder that our H-1B Visa Applications data is just that: applications. It’s not a list of all companies that have won the H-1B visa lottery as it is often misunderstood and misused.

To stay in the know (or find inspiration) subscribe to our monthly newsletter, Between Two Rows, a monthly round up of new, noteworthy, and interesting public data and events.  

Reach out

We are always looking for new ideas for datasets to add and love to hear what our users have done with the data. If you have a question, dataset suggestion, or project you’d like to share, don’t hesitate to .(JavaScript must be enabled to view this email address).

Explore Enigma Public here.

Comments