Indices of Deprivation: Linked Data Prototype

The views expressed here are mine, and mine alone, and not my employer’s (DCLG) official policy or opinion


Those of you with longer memories will recall my post back in May, promoting the then new triple store for DCLG’s Indices of Deprivation.

I’d love to report that this has generated a flurry of new apps.  I think I’m not alone when I say: we need some more real-world examples which show us the true power and potential of blending together different data sources….over the web…..using open standards.

The reality is: no apps, and no more than a passing curiosity in all the nice linked data, triple store wizardry.

Now, please don’t get me wrong.  I’m not complaining.  I think there are many very good reasons why we are where we are.   I’ll spare you my thoughts on the possible causes.  I’m guessing you’ll be much more interested in what I’ve actually done about it…..(which I’m hoping will spark a bit of discussion about going further, to tackle the underlying causes).

What I’ve done is: create this small, but hopefully important demonstrator application, along with some slides.


Before I outline how the app works, and what I’m trying to test and prove,  I wanted to say a huge thank you to the various data aggregators, API providers,  and generous human beings without whom none of this would have been possible.  A round of applause please for, in no particular order:

  • The good folk at MySociety, for theyWorkforYou – which my app calls directly to show information about MPs.  Also, for the fab MapIT API, which I’m using to get and display info about geographic areas (Wards, Districts, and Constituences)
  • Chris Taggart and his OpenlyLocal crew – which I use to show data about local councillors, and get the links to a Local Authority’s twitter feed.
  • Friends at, for their fantastic work on LinkedData and Sparqly endpoints.   You must learn to love this stuff, as I have done.
  • The Guardian Datastore, and Alasdair Rae, for opening my eyes to the possibilities if only the data were available in more open, re-usable forms.
  • And last, but by no means least, to Bill and Ric Roberts at Swirrl – for their generosity in hosting the IMD dataset, and their patience in answering my various dim, newbie questions.

 About the Indices of Multiple Deprivation (IMD) explorer application

Where has this come from?

I’ll begin with an excuse/confession.  I am not a professional software developer.  My code is a bit like my waistline: functional, but a little bloated.  All I have is my trusty old PC,  a copy of Aptana studio that I’m not afraid to use,  and copious volumes of coffee and paracetamol.

I’m doing this stuff in my spare time, to help me understand what open data really means from a data users and publishers perspective.   My passion for this stuff comes from a long, sometimes bitter experience of managing data in its various guises. Over the years, I’ve spent many, many hours with my head inside a SQL database, wrestling with the madness of pulling in content from a mix of magnetic tapes,  floppy disks, pen drives, CDs (and yes, even good old handwritten input).   I even remember punch cards and ticker tape!

So, my main motivation for developing the app was to test and prove how and why open data really is different.  For instance, how can it help us to quickly bring together disparate sources, and tell us something we didn’t know? Can it really cut out some of the cost and complexity of publishing and integrating ever increasing volumes of information?

I went in to this knowing a little bit about the LinkedData theory –  with a head full of acronyms and phrases like APIs; ontology; RDF; triples; and Sparql.  I had zero experience of actually using this stuff in the wild.

The result….

Four weeks, and many late nights later, out pops the IMD explorer.

For those of you who aren’t familiar with the IMD, it is – as the Guardian Datastore puts it – “possibly the most significant research into poverty in England ever put together“.

The IMD is particularly significant because it measures deprivation at a fine-grained, geographic level which relates directly to people, and where they live.   For the map geeks, the IMD data is published at the Lower Layer Super Output Area  (LSOA) level, which the ONS defines as an area with

“a minimum population of 1,000, with an overall mean of 1,500. LSOAs are built from groups of Output Areas. There are around 34,000 LSOAs in England and Wales”

IMD is also significant because is measures deprivation from different perspectives: for example, to show how deprived an area is according to crime, education, health, and so on.

So what, I hear you cry.  This matters because we often see the IMD being used as a sort of lens for viewing other, related datasets.  For a really topical example of this, check out the Guardian’s article exploring whether deprivation was a factor  in the recent riots.

Enough of all this theory – show me how the app works!

On one level, the application is simply about helping ordinary people to understand deprivation in all its guises where they live, work and play.

On the opening screen, I’ve provided a national map showing the top 50 most and least deprived areas (LSOAs).  You can switch the view to different measures (or domains) of deprivation via the drop down list, just below the page title.

Alongside that, is a box to type in a postcode, street, or place name, which then zooms to a map showing deprivation in the immediate local area.  From there, you can then move around the map – e.g. to zoom out to see deprivation across the whole Local Authority, or explore Wards within that locality.

Now here’s the twist, and the reason why linked/linkable data matters.

When exploring data at Ward-level, the application hops over to OpenlyLocal to pull back data about the councillors who you’ve elected to represent your interests in that ward.   So, if you’ve found issues with deprivation in your local area and are motivated to do something about it, you now have information about some of the people to talk to!

It’s a similar deal when exploring the Parliamentary Constituency – where the app goes and gets data about your MP from TheyWorkForYou, with links back to that site if you want to know how your MP is working in your local interests.

And there’s another twist.  Take a look at the “Education, Skills and Training” Deprivation domain.  Provided you’re not using Internet Explorer, you’ll see a map overlaid with the location and other information about local schools and other educational establishments.    I’ve particularly enjoyed developing this bit. It’s interesting, but not suprising to see the proportion of kids attending schools in deprived areas, who receive free school meals.    The data also suggests that schools in deprived areas are, relatively speaking, running below capacity – which is just an observation on what we can see and learn by joining up disparate datasets, and not a party-political point.

Anyway, that’s probably quite enough by way of an introduction to what I’ve done and why I’ve done it.  I will be following this up with further posts, exploring some of the more technical, “under the bonnet” aspects.

In the mean time, enjoy the app.  And my previous challenge still stands.  Start creating, and show us what you can do!


22 responses to “Indices of Deprivation: Linked Data Prototype

  1. Pingback: GoogleBig » Archive » England’s Deprivation on Google Maps·


  3. Pingback: Public data’s Desert Island challenge: which dataset would you pick? – latest news and updates | B.log·

  4. Pingback: Public data’s Desert Island challenge |·

  5. Pingback: England’s Deprivation Mapped « GIS Use in Public Health and Health Care·

  6. Pingback: Boot up: BlackBerry PlayBook sales forecast slashed, Dolby drops RIM lawsuit, and more | Richard Hartley·

  7. Pingback: Public data’s Desert Island challenge: which dataset would you pick? | Richard Hartley·

  8. Pingback: Public data’s Desert Island challenge « Science Technology Informer·

  9. Does your website have a contact page? I’m having problems locating it but, I’d like to shoot you an email. I’ve got some creative ideas for your blog you might be interested in hearing. Either way, great blog and I look forward to seeing it grow over time.

  10. I will right away grab your rss as I can’t find your email subscription link or e-newsletter service. Do you have any? Kindly allow me understand so that I may just subscribe. Thanks.

  11. Hello There. I found your blog using msn. This is a really well
    written article. I’ll make sure to bookmark it and return to read more of your useful info. Thanks for the post. I will definitely return.

  12. An outstanding share! I’ve just forwarded this onto a friend who has been conducting a little homework on this. And he in fact ordered me dinner due to the fact that I stumbled upon it for him… lol. So let me reword this…. Thank YOU for the meal!! But yeah, thanks for spending time to discuss this issue here on your internet site.

  13. Howdy! I realize this is kind of off-topic however I needed to ask.
    Does operating a well-established blog like yours require a
    large amount of work? I am completely new to running a blog but I
    do write in my diary daily. I’d like to start a blog so I will be able to share my experience and thoughts online. Please let me know if you have any kind of ideas or tips for brand new aspiring bloggers. Appreciate it!

  14. Heya outstanding website! Does running a blog like this require a
    great deal of work? I have absolutely no knowledge of programming
    but I had been hoping to start my own blog in the near future.
    Anyways, if you have any suggestions or tips for
    new blog owners please share. I understand this is off subject nevertheless I just had
    to ask. Appreciate it!

  15. I believe what you said was very logical. But, what about this? what if you composed a catchier title? I ain’t saying your content is not solid., but suppose you added something to maybe grab people’s attention? I mean is a little plain. You ought to peek at Yahoo’s home page and watch how they create post titles to grab viewers interested. You might add a video or a related picture or two to grab readers excited about everything’ve written. Just my opinion, it might make your website a little bit more interesting.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s