Although we’re known for more advanced data science, we also like to have fun and co-discover potentially real, reliable patterns within datasets with government leadership and the general public. We do this via interactive exploratory analyses (i.e. visualizations), either online or in-person. We then hope that these will spark some hypotheses we can statistically test, so that our civic/non-profit partner(s) can know for sure what patterns are real and, thus, which policies are truly best to pursue.
The following interactive visualizations are a good example of this…
Per the request of San Francisco’s 311 Deputy Director, Andy Maimoni, and as a part of our larger study of 311 case data, we’ve used Microsoft’s Power BI to come up with a way for 311 leadership -and you the public- to explore a subset of 311’s case data: those cases that were ultimately transferred from some originating department to another.
We invite you to interact with this data, by clicking on these charts’ shapes, and let us know (in the comments) if you have any hypotheses you think 311 should have us test. We look forward to reading your ideas!
Data Source: SF OpenData portal (data pull from 7-1-08 through 3-16-16; please note, dataset extremely large).
Dashboard 1: Overview of 311 Transferred Cases
Please note: this overview dashboard is based on a representative sample of the data, albeit large, rather than the full dataset. Unfortunately, the 3rd party date selection widget/visualization below is limited to some maximum number of records. Still, as this sample is representative (i.e. random and large enough for 99% statistical power), these visuals should well represent the patterns and proportions of the full dataset.
Dashboard 1b: Responsible Agencies, Origin Mix vs. Destination Mix (where N transfers > 20)
This is only possible because of the great work by Matthew Mollison, Ph.D., who employed some NLP (Natural Language Processing) to extract destination case #s from noisy “status notes” with a high degree of accuracy. For more details on his approach, including Python code, click here >>.