The views expressed here are mine, and mine alone, and not my employer’s (DCLG) official policy or opinion
Those of you with longer memories will recall my post back in May, promoting the then new triple store for DCLG’s Indices of Deprivation.
I’d love to report that this has generated a flurry of new apps. I think I’m not alone when I say: we need some more real-world examples which show us the true power and potential of blending together different data sources….over the web…..using open standards.
The reality is: no apps, and no more than a passing curiosity in all the nice linked data, triple store wizardry.
Now, please don’t get me wrong. I’m not complaining. I think there are many very good reasons why we are where we are. I’ll spare you my thoughts on the possible causes. I’m guessing you’ll be much more interested in what I’ve actually done about it…..(which I’m hoping will spark a bit of discussion about going further, to tackle the underlying causes).
What I’ve done is: create this small, but hopefully important demonstrator application, along with some slides.
Before I outline how the app works, and what I’m trying to test and prove, I wanted to say a huge thank you to the various data aggregators, API providers, and generous human beings without whom none of this would have been possible. A round of applause please for, in no particular order:
- The good folk at MySociety, for theyWorkforYou – which my app calls directly to show information about MPs. Also, for the fab MapIT API, which I’m using to get and display info about geographic areas (Wards, Districts, and Constituences)
- Chris Taggart and his OpenlyLocal crew – which I use to show data about local councillors, and get the links to a Local Authority’s twitter feed.
- Friends at data.gov.uk, for their fantastic work on LinkedData and Sparqly endpoints. You must learn to love this stuff, as I have done.
- The Guardian Datastore, and Alasdair Rae, for opening my eyes to the possibilities if only the data were available in more open, re-usable forms.
- And last, but by no means least, to Bill and Ric Roberts at Swirrl – for their generosity in hosting the IMD dataset, and their patience in answering my various dim, newbie questions.
About the Indices of Multiple Deprivation (IMD) explorer application
Where has this come from?
I’ll begin with an excuse/confession. I am not a professional software developer. My code is a bit like my waistline: functional, but a little bloated. All I have is my trusty old PC, a copy of Aptana studio that I’m not afraid to use, and copious volumes of coffee and paracetamol.
I’m doing this stuff in my spare time, to help me understand what open data really means from a data users and publishers perspective. My passion for this stuff comes from a long, sometimes bitter experience of managing data in its various guises. Over the years, I’ve spent many, many hours with my head inside a SQL database, wrestling with the madness of pulling in content from a mix of magnetic tapes, floppy disks, pen drives, CDs (and yes, even good old handwritten input). I even remember punch cards and ticker tape!
So, my main motivation for developing the app was to test and prove how and why open data really is different. For instance, how can it help us to quickly bring together disparate sources, and tell us something we didn’t know? Can it really cut out some of the cost and complexity of publishing and integrating ever increasing volumes of information?
I went in to this knowing a little bit about the LinkedData theory – with a head full of acronyms and phrases like APIs; ontology; RDF; triples; and Sparql. I had zero experience of actually using this stuff in the wild.
Four weeks, and many late nights later, out pops the IMD explorer.
For those of you who aren’t familiar with the IMD, it is – as the Guardian Datastore puts it – “possibly the most significant research into poverty in England ever put together“.
The IMD is particularly significant because it measures deprivation at a fine-grained, geographic level which relates directly to people, and where they live. For the map geeks, the IMD data is published at the Lower Layer Super Output Area (LSOA) level, which the ONS defines as an area with
“a minimum population of 1,000, with an overall mean of 1,500. LSOAs are built from groups of Output Areas. There are around 34,000 LSOAs in England and Wales”
IMD is also significant because is measures deprivation from different perspectives: for example, to show how deprived an area is according to crime, education, health, and so on.
So what, I hear you cry. This matters because we often see the IMD being used as a sort of lens for viewing other, related datasets. For a really topical example of this, check out the Guardian’s article exploring whether deprivation was a factor in the recent riots.
Enough of all this theory – show me how the app works!
On one level, the application is simply about helping ordinary people to understand deprivation in all its guises where they live, work and play.
On the opening screen, I’ve provided a national map showing the top 50 most and least deprived areas (LSOAs). You can switch the view to different measures (or domains) of deprivation via the drop down list, just below the page title.
Alongside that, is a box to type in a postcode, street, or place name, which then zooms to a map showing deprivation in the immediate local area. From there, you can then move around the map – e.g. to zoom out to see deprivation across the whole Local Authority, or explore Wards within that locality.
Now here’s the twist, and the reason why linked/linkable data matters.
When exploring data at Ward-level, the application hops over to OpenlyLocal to pull back data about the councillors who you’ve elected to represent your interests in that ward. So, if you’ve found issues with deprivation in your local area and are motivated to do something about it, you now have information about some of the people to talk to!
It’s a similar deal when exploring the Parliamentary Constituency – where the app goes and gets data about your MP from TheyWorkForYou, with links back to that site if you want to know how your MP is working in your local interests.
And there’s another twist. Take a look at the “Education, Skills and Training” Deprivation domain. Provided you’re not using Internet Explorer, you’ll see a map overlaid with the location and other information about local schools and other educational establishments. I’ve particularly enjoyed developing this bit. It’s interesting, but not suprising to see the proportion of kids attending schools in deprived areas, who receive free school meals. The data also suggests that schools in deprived areas are, relatively speaking, running below capacity – which is just an observation on what we can see and learn by joining up disparate datasets, and not a party-political point.
Anyway, that’s probably quite enough by way of an introduction to what I’ve done and why I’ve done it. I will be following this up with further posts, exploring some of the more technical, “under the bonnet” aspects.
In the mean time, enjoy the app. And my previous challenge still stands. Start creating, and show us what you can do!