An article by Steve Peters (stephen.peters1@sky.com)
Disclaimer: the views expressed in this article are personal, and do not represent official DCLG policy or opinion.
Introduction – why you should read this
Its been a while since I blogged about my work at Communities and Local Government to start opening up the Department’s public data.
I wanted to bring you up to speed with some things I’ve done in recent months. I think they are interesting and important first steps in the Department’s journey towards fully open, re-usable data. I hope you agree.
This article majors on work to test publishing of statistical data in a highly accessible, re-usable way. Thanks to friends at Swirrl Ltd, we have now built a full LinkedData version of the 2007 and 2010 English Indices of Deprivation (IMD). You’ll find it here.
The dataset sits on Swirrl’s “PublishMyData” platform, which means – if you’re a software developer – it can be directly queried and results obtained in a variety of formats.
Before I say more about that, I first wanted to recap on why open data matters – i.e. explain the context and rationale for our work with the IMD and other DCLG datasets. Feel free to skip over this, and go straight to find out more on the IMD open data experiment.
Open data – why it matters
Much has been written and said about why open data is important. For me, there are two main reasons.
First, it should allow data owners – like me – to publish their data in a consistent, standardised form. This has the potential to save money (e.g. by rationalising or consolidating what we publish, and how we publish it), and reduce the burden of shunting data between different systems (e.g. by standardising on formats and definitions which let the data flow more easily and efficiently).
Second, open data should remove some of the pain for our data users (you!) in finding and understanding what we publish, discovering links to related external sources, and combining or blending it in lots of new and innovative ways.
We want to see data from various sources, including our own, brought together and actively used to tell us something we (or you) didn’t know, to help people understand and engage with public services, and to hold service providers to account if/where things aren’t going so well.
What we’re doing about it
I like to think that DCLG is already doing quite well with making its data more open and accessible. For instance, friends in our Finance and Procurement Teams are working really hard to give you better data on departmental spending over £500. We’ve recently incorporated columns which tell you whether a supplier is a Voluntary or Charity Sector organisation, and whether the payment was a grant to the organisation concerned, or money for DCLG’s own day-to-day activities.
As part of its Business Plan, the Department has also recently published information about the data it will use to track and measure its performance against key, so-called, impact and input indicators. DCLG has a good presence on data.gov.uk too.
Testing open data standards and approaches using the Indices of Deprivation
Whilst this is all very positive, the focus is primarily on helping you understand what data the Department has. On its own, it doesn’t really help you to find the data more quickly, or re-use it – alongside other information – in new, innovative ways.
I am trying to address that through our work to publish the recently released 2010 Indices of English Deprivation in a searchable, “LinkedData” form, along with data for 2007.
We’ve chosen the IMD because it provides rich, comparable figures about deprivation over time in different localities. The data measure a broad concept of ‘multiple deprivation’ across seven domains: Income, Employment, Health and Disability, Education Skills and Training, Barriers to Housing and Other Services, Crime and Living Environment.
We know that the IMD is widely used across the public sector, both to plan Government programmes, and to coordinate or target services at those most in need. Examples of previous uses include: determining optimal locations for the initial wave of Sure Start and Children’s Centres; and reducing Stamp duty for transactions on property and land based in deprived localities.
If you want to know more, please check out OCSI’s article on Why the Indices of Deprivation are still important in the open data era, or the Guardian data blog’s Indices of multiple deprivation: find the poorest places in England.
Given its broad appeal and importance, I believe that the IMD is ideal as a candidate for sharing and re-use in open formats. Perhaps more importantly, I also believe that IMD is ripe for blending and combining with other external sources. For example, it would be great to see IMD used as a sort of lens for comparing and contrasting other related information – such unemployment housing benefit, or crime statistics.
Get involved!
My hope is that publishing this searchable, open and re-usable version of the IMD will spark a range of creative, innovative uses and applications.
Perhaps you work in a Local Authority or other public body, and have a bright idea around combining IMD statistics with your own local intelligence? Perhaps you are a software developer or innovator, and want to find a neat way of packaging and presenting the data in a powerful, citizen-friendly manner?
Whoever you are, I would love to see you making use of this data. Please let me know what you create, and give me some feedback on what works well and what could be better. What are you waiting for! Head on over to http://opendatacommunities.org/ and help us make a difference!
Technical stuff you need to know
We have utilised several standards from the excellent work by the data.gov.uk team. At the heart of the data representation is their Data Cube ontology.
Although we created new geographic identifiers for the IMD Lower Super Output Areas, these are cross-linked to the local authority identifiers from http://statistics.data.gov.uk and Ordnance Survey’s Linked Data for postcodes.
To help get you started, Swirrl friends have kindly posted this blog, outlining how to retrieve IMD data by starting with a postcode.
If you get stuck, or need some advice, feel free to drop me or Swirrl a line – we are contactable on hello@opendatacommunities.org , or stephen.peters1@sky.com
Other things in the pipe-line (separate blog posts to follow in the coming weeks)….
1. Improving access to and usability of DCLG data on payments to suppliers over £500.
2. Building a “dashboard” for DCLG’s impact and input indicators
3. A nice, new, interactive Departmental organogram
4. Experimenting with on-line policy engagement
5. Open data on business insolvency – a personal venture.
Open Data Blog 
Steve,
Thanks very much indeed for writing about the work that has been going on in this area. I live in Scotland and know Bill.
Our government’s organogram is here http://www.scotland.gov.uk/About/strategic-board/whoswho
As some of the brightest people build an Alphagov for web-site, is it also conceivable that such a concept might apply to LOD ?
As an outsider, and not a technical one, if it took government 15 years to build 1,367 web sites and then to decide one is more sensible, what about LOD and the architecture there ? Is there a similar risk, or should we let 1,000 flowers bloom ?
How seriously are these things taken by the Data Transparency Board, Baroness Hanham of Transparency or other institutions ?
Is LOD on the CIO or CTO agenda ; do they still exist ?
Thanks
Alex
Thanks, Alex for your comments and I’m glad you like the blog post.
On a general level, I’m very much a fan of the rapid, agile approach to developing systems. I would certainly agree that public service people (me included) have in the past been a bit too risk averse – sometimes getting bogged down in the process of specifying what we want for example, and somehow forgetting what we set out to create.
On data specifically, I’m squarely in the “1,000 flowers” camp, and bought in to the concept of our data being part of the rich ecosystem that Rufus Pollock describes beautifully here
My sense is that this new thinking – where data is the fuel to power greater openness and accountability – is very much at the heart of the coalition’s objectives for transparency. We see it, for example, in the work of the Public Sector Transparency Board, the Local Public Data Panel, and in flagship projects such as data.gov.uk. So, we’re finding that Ministers are taking this very seriously indeed, and pushing hard to turn words into sustained action.
I’ve also had some involvement in what I see as key elements of the LOD infrastructure – such as ontologies and standards for describing geographic entities, statistical datasets, or lists of payments over £500. I’ve been mightily impressed by how, when we bring together very talented open data technologists and passionate, motivated data owners, great things can and do happen in pretty short order. Hats of here to friends at Swirrl, National Archives, and the Local e-Govt Standards Body who are making huge inroads on developing practical, workable standards with virtually no money or resource.
Clearly, we’re a long way from LOD being embedded as business-as-usual. But I believe that some of the key building blocks – such as political will, acceptance that this is evolution through rapid, agile growth – are now in place.
Pingback: DCLG Spending data – making it more useful | Open Data Blog·
Pingback: Is this the summer of social? June SocMed CoP roundup | Public Path·
Pingback: Indices of Deprivation: Linked Data Prototype | Open Data Blog·