Neighbourhood Planning – mapping the change

Have you heard about Neighbourhood Planning?

It is a new way for communities to decide the future of the places where they live and work.  They will be able to:

• choose where they want new homes, shops and offices to be built
• have their say on what those new buildings should look like and what infrastructure should be provided
• grant planning permission for the new buildings they want to see go ahead.

A Neighbourhood Planning Area (NPA) can cover any neighbourhood, as long as it is coherent, consistent and appropriate; they will differ for different areas across the country, and should be what is most appropriate for that area.   At the time of writing, around 500 local areas have either applied for or been officially designated by their Local Authority as a NPA.  Importantly, local residents in three areas (Upper Eden, St James in Exeter and Thame, South Oxfordshire) have now voted “yes” to formally adopting their Neighbourhood Plans

Opportunities for Open Data

As an open data enthusiast, I believe that Neighbourhood Planning  communities could benefit significantly from on-line tools and data sources that help them capture and share information, and work together on their plans and issues in their neighbourhoods.

Working with friends at DCLG, I’ve been exploring how to bring together various related data sources in fully open, accessible and re-usable ways.

The result is this new demonstration application

 The demonstration application

Currently, the application joins up three relevant, and (hopefully) useful and relevant sources.

The first is a new informal dataset that we’ve curated by scouring Local Authority and Neighbourhood Planning websites.   This captures basic information about each NPA; things like, where it is, key dates, and links to documents such as the original application and local authority designation papers.    Note that this data is not complete, not 100% accurate,  and I’d encourage you to get involved and help improve it – see Improving the map section below.

The opening map shows clusters of NPAs.  To view individual areas, simply zoom in (using the controls on the left of the map), or click on the numbers in the circles (which will be squares, if you’re using Internet Explorer).

Individual NPAs are then shown on the map as a blue balloon.  To view information about each area, hover your mouse over the balloon, and the information will display in the top right hand corner of the map.

Alongside the map, we have the second main data source: the Twitter conversation under the #NeighbourhoodPlanning hash tag.  You can join in the conversation by clicking on the box at the bottom of the list – note this may not work correctly if you’re using the Firefox browser.

The third data source is local photographs published via Panoramio.com.  To view these, click on the balloon icon for the area of interest.  Photos are then retrieved directly from Panoramio in real-time, and displayed in an interactive gallery beneath the map.   Of course, you could always add more photos by uploading them to Panoramio!

Improving the map: get involved!

The underlying data is held in OpenStreetMap.  This is a global initiative to “create and provide free geographic data, such as street maps, to anyone”.  The project is managed by the OpenStreetMap Foundation, “an international not-for-profit organization supporting, but not controlling, the OpenStreetMap Project. It is dedicated to encouraging the growth, development and distribution of free geospatial data and to providing geospatial data for anyone to use and share”.

This means that the data on the map is open and available to NPAs to edit and improve.   I’d value your help to do that, in two main areas.

Firstly, I’d like to extend the map beyond points, to show the boundaries of each NPA.   I have published a proposal for achieving that on this OpenStreetMap wiki.  Please get in touch if you’re interested in helping out.

Secondly, it would be great to see Local Communities mapping more detailed information within their NPA: e.g. to show areas targeted for housing development.    There are various free tools available for editing OpenStreetMap. For further information, please visit the Beginners Guide, and the list of editing tools.  I use JOSM.  Potlatch is great too for editing in the browser.

One small step…the DCLG business plan indicator dashboard

If you’ve read my guest blog at the ODI, you’ll have spotted that I’m trying to push forward DCLG’s open data contribution on three main fronts.

One of those is: testing how to grow and extend our open data store to incorporate more DCLG data.

Late last week, we released the first new, and (I think) exciting output: our Business Plan Indicators dashboard., and supporting page about DCLG’s priority actions.

If you’re unfamiliar with Departmental Business Plans, the indicators are designed to help you, the public, assess how we’re doing on our various policies and reforms. Our Business Plan includes further information about each indicator, and how they align with Departmental and the coalition government’s priorities. For ease of ref, I’ve repeated the list below.

Introducing the Business Plan Indicators Dashboard

You’ll see from the opening page that we’ve tried to present the latest headline data in a neat, engaging way. For each indicator, you’ll find a panel showing the latest figure with links to more detailed information.

Clicking on “more detail” provides additional information including: what the latest figures show; how the figure is calculated; and why the indicator is in our business plan. You’ll also find a time series line chart, and a link to embed the summary in your own web page.

Note too that each “more detail” page includes a link to the underlying source datasets. Click on the “latest source” or “more data” links, and you’ll jump across to our open data store, from where you can query, retrieve and re-use data in a range of open formats.

Where next?

I think of the dashboard as the top of DCLG’s data pyramid, beneath which are lots more detailed datasets.

Let’s take the Housing Starts impact indicator as an example. This is calculated from our official statistics on House building. We also routinely publish live-tables containing further detailed statistics; e.g. breaking down the figures by local authority and tenure. The same is true for most other indicators in the Business Plan.

So, we have a great opportunity here to join-up this lovely content. I want to make it possible for you to quickly and easily move up and down the data pyramid, discovering and exploring links within and between the myriad of related datasets at each layer. I also want you (and DCLG itself!) to be able to retrieve it, re-use it, and join it up over the web to related sources that we don’t hold: for example, to link to more detailed data in, say, local authorities on land for housing, completions and so on.

Clearly, this will take time to achieve. As outlined in my ODI blog post, a key first step is to establish our open data store as a permanent, sustainable resource for routinely releasing DCLG statistical outputs in 5-star, accessible formats. We’re on track to appoint a technology partner in February 2013.

As we move forward, I’m keen to work with our data users and third-party software innovators. I want to know that we’re releasing the right data, in the right formats, at the right time. I’d also like to explore and exploit opportunities for joining-up our datasets to external sources, over the web, using open re-usable standards.

Which leaves me with some questions for you:

1. Which DCLG datasets do you want us to release first in 5-star formats? If you’re not sure what we hold, please take a look around data.gov.uk, or search our statistics and our transparency data publications on gov.uk.

2. How can we help you to find and understand our datasets more quickly and easily? For instance, do you search for data using particular terms? Do you like (and use) visualization tools like the dashboard? Do you have any ideas for new tools and visualisations?

3. Do you use DCLG data alongside other third-party sources? If yes, which sources are you using? How can we make it easier for you to retrieve and combine multiple dataset? And, do you have ideas on how we could do that over the web, using open standards?

DCLG Business Plan priorities and Business Plan indicators

Decentralise power as far as possible

1. Percentage of local authority revenue expenditure funded by general government grant, broken down by class of authority (Input indicator)

2. Percentage of local authorities committed to identifying and beginning work with troubled families (Input indicator)

Reinvigorate local accountability, democracy and participation

3. Number of groups supported to submit an expression of interest as part of the Community Right to Challenge (Impact indicator)

4. Fire-related casualties (per 100,000 population) (Impact indicator)

Support and incentivise local growth

5. Business rates yield within Enterprise Zones (Impact indicator)

Meet people’s housing aspirations

6. Affordable Rent payment per dwelling by the Homes and Communities Agency (Input indicator)

7. Average New Homes Bonus grant payable per dwelling per year to different classes of authority (£) (Input indicator)

8. Total number of housing starts and completions (seasonally adjusted) (Impact indicator)

9. Number of affordable housing starts and completions delivered through the Homes and Communities Agency (Impact indicator)

10. Energy efficiency of new build housing (average Standard Assessment Procedure energy rating score) (Impact indicator)

11. Households in temporary accommodation (seasonally adjusted) (Impact indicator)

Put communities in charge of planning

12. Percentage of local planning authorities having an adopted local plan prepared under the 2004 Planning and Compulsory Purchase Act (Impact indicator)

13. The number of planning permissions granted as a percentage of all applications for major and minor schemes (Impact indicator)

Are you satisfied with where you live? New open data on neighbourhood-level wellbeing

Today marks the launch, by DCLG, of a new mapping tool and open dataset for you to explore and understand the nation’s wellbeing at the neighbourhood-level.   We’ve also incorporated wellbeing data in our Local Authority Dashboard, for you to view alongside a selection of other DCLG statistics on housing, deprivation, and local government finance.

This builds on the first set of new, experimental wellbeing statistics released in July by the Office for National Statistics  

As the ONS describes in its statistical release, the data provide new insight in to “how people think and feel about their own lives”.   When used alongside the more traditional economic measures, this data can help us to better understand the nation’s problems, issues and progress with resolving them.

 Since July, my analyst colleagues at DCLG have been busy modelling ONS’s data at neighbourhood-level.  I’ve provided further background below on how they did this.  We think the local modelled estimates are a useful resource for local authorities and councillors, and their local communities and residents. 

We think of the data as a sort of lens, through which anyone can view wellbeing in their local area, and compare/contrast our estimates against their own local data and knowledge.  We’d love to hear your views on if/how/where the data is useful to you, in your local area: feel free to e-mail wellbeing@communities.gsi.gov.uk or leave a comment on my blog.

We’re also very interested in any software tools that use our data alongside other sources, in new and innovative ways.   As an open data geek, I think the most exciting aspect is that the underlying estimates are also available as full, 5-star LinkedData.   You’ll find the datasets on our opendatacommunities site   

 

 If you’re a software developer or innovator, you can develop queries to retrieve the data via our API, and display it in your own application.  This extends to any dataset held in opendatacommunities.  That means you could write queries which blend and combine wellbeing with other data: for instance, to retrieve wellbeing and deprivation data for a particular postcode, or local authority.  

 To get started, please take a look at our developer documentation.   We’ve also provided some example queries, such as this one (http://opendatacommunities.org/datasets/wellbeing-lsoa-life-satisfaction-mean ), in the “API” tab on the dataset about ‘Life Satisfaction’.

Modelling the results – technical background

The Department for Communities and Local Government (DCLG) has estimated the expected wellbeing of residents at Lower-layer Super Output Area (LSOA) level. The purpose is to illustrate the likely degree of variation in wellbeing between neighbourhoods.

These are modelled estimates for local areas based on national findings from the ONS Annual Population Survey 2011-2012. DCLG analysts used CACI’s ACORN geo-demographic segmentation to estimate the likely wellbeing characteristics of each neighbourhood.

These estimates are not the actual survey responses of people living in those areas. This is because sample sizes from the APS do not permit reliable estimates of subjective wellbeing below the 90 unitary authorities and counties reported in the First ONS Annual Experimental Subjective Well-being Results. As such, DCLG encourage local areas to use these estimates alongside their own local knowledge and data.

The maps reflect the differences in wellbeing observed among the 56 ACORN Types and the unique ACORN profile of each neighbourhood. The method assumes the national profile of wellbeing for the ACORN types is broadly the same in each local authority. For all of the subjective wellbeing measures, DCLG tested this assumption broadly held across the nine regions.  As a result, we made a minimal number of adjustments to the profiles for life satisfaction, worthwhile, and happy yesterday, and determined that the method was not robust for modelling anxiety.

Requests for further details of the methodology can sent to wellbeing@communities.gsi.gov.uk.

Data excavation : news from the DCLG quarry

The views expressed here are mine, and mine alone, and not my employer’s (DCLG) official policy or opinion.

Following today’s exciting announcements in the Open Data White Paper, I wanted to update you my recent work with open data at DCLG.

Despite the lack of posts to my blog since January,  I’ve been pushing forward with open data on two main fronts.

Firstly, I’ve been testing out ways of publishing more data in the full (5 star) LinkedData form.  The result is our proof-of-concept OpenDataCommunities triplestore, with a nice SPARQL endpoint and API for you to enjoy.

I’m really pleased, and proud that this gets a mention in the Open Data White Paper.

You’ll see that the triple store comprises a selection of DCLG’s housing, local government finance and deprivation statistics.  We think we’ve chosen a collection of data that is widely used – particularly by Local Authorities – and ripe for linking together in new and interesting ways; including linking up with related sources over the web.   We are currently trying to test and prove that by working with a small group of local authorities and voluntary organisations.  For me, this is an essential part of gathering evidence to help me scope, plan and cost how to move to routinely releasing all DCLG data in this way.  So, if you are a regular user of DCLG data, and want to find smarter/faster ways to acquire and blend it with other sources, I’d love to hear from you.

We’ve done some neat things with geographic data too.  One example is the new set of identifiers for local authorities of various types in England, with links to other schemes, such as those defining the geographic extent of each authority.  We think this is important because it now enables local authorities to be referenced in terms of the organisation itself, and the geographic areas they serve.  Let’s take Amber Valley as an example.  The new URI - http://opendatacommunities.org/id/district-council/amber-valley - essentially provides information about the council as a public body, or legal entity.  If you follow the link, you’ll see basic information such as the link to the council’s website, area codes defined by the Office for National Statistics, and DCLG’s code for local government financing purposes.  This link also provides connections through to data, from ONS and Ordnance Survey about the geographic area governed by Amber Valley.

The ambition is for local authorities to start adopting these URIs, include them in their own web sites and APIs, and start attaching additional information at source about their organisation – such as “contact us” type information; and details about local councillors and their senior management team.  Again, if you’re based in a local authority and agree this is worth pursuing, then please do get in touch.

The second strand of my work has been on building relatively simple demonstration applications which “show and tell” how open data, served from our triple store, can enable innovative new tools and insights.   The three recent developments are:

1.   The fantastic Local Authority Dashboard

2. My new application showcasing DCLG’s Household Projections Statistics

3. An updated version of my Index of Deprivation Explorer

I’ve also built a landing page for the various apps I’m working on, and will be extending this to APIs and data source’s I’ve used.

OpenData needle work: stitching together the local debate

The views expressed here are mine, and mine alone, and not my employer’s (DCLG) official policy or opinion

Hi.

I’m sure you’ve heard about the Government’s localism agenda.  It is at the core of shifting power into the hands of individuals, communities and local authorities.

For this to work, I think we’ll need new and smarter ways for citizens, communities and public service providers to work together and solve real-world problems.   I’ve been wondering about the potential role for on-line technologies here  - in particular social media.   This led me to one obvious key question, which is: to what extent is the debate and pro-active collaboration already happening in different localities, via tools like Twitter or  Facebook?

To help answer that, I’ve used some of my spare time over Christmas to build a new prototype application.   At its core, is a register of news feeds and twitter streams provided by all English local authorities, and several hundred hyperlocal sites.

At the time of writing, I’ve catalogued around 350 news and twitter feeds from local councils, and over 400 equivalent feeds from the hyper-local world.   To find out more about the sources I’ve used, please see the section below.

The application then uses this register to “read” individual feeds, highlight trends  and show you the latest tweets and news articles.  I’ve called the app “drumalism”.  You’ll find it here: http://dclgexamples.mywebcommunity.org/localities/drumalism.htm.

Where next for the app and data feeds register?

As with my previous prototypes, I’ve developed drumalism in my spare time, using my own equipment.  This really is a “one man and his dog” operation: where the dog is Lillie, my Bassett Hound, who is usually wrapped around my computer chair with a “when are we going out” expression on her face.

I’ve licenced the app and data registry under Creative Commons’ NonCommercial-ShareAlike (v3.0), so you’re free to re-use it on those terms.  I hope that what I’ve done serves as a good example of what can be done to join-up established sources of news and opinion in different localities.  It would be great to see similar sources and tools incorporated in Local Authority and hyperlocal sites.

Currently, I have no plans to develop drumalism as a commercial venture, but please do get in touch if you’re interested in exploring that further.

Drumalism – data sources

The feeds registry behind drumalism comprises four main as Google Fusion tables.

1.  A list of all English Local Authorities – fusion table ID 2456608 .  I’ve sourced this from Local DirectGov, adding in grid references by geo-coding the main office address.

2.  A list of twitter, RSS,  facebook and open data feeds provided by Local Authorities – fusion table ID 2455274.   I’ve developed this myself, by manually trawling through local authority websites.

3.  A list of hyperlocal sites – fusion table ID 2452252.  I’ve sourced this from the list at OpenlyLocal.com, and then extended it to incorporate additional sites I’ve uncovered as I developed the application.

4. A list of twitter and RSS feeds provided by hyper-local sites – fusion table ID 2455251.  Big thanks again here to OpenlyLocal for providing the initial list, which I then manually checked and extended.

Indices of Deprivation: Linked Data Prototype

The views expressed here are mine, and mine alone, and not my employer’s (DCLG) official policy or opinion

Hi.

Those of you with longer memories will recall my post back in May, promoting the then new triple store for DCLG’s Indices of Deprivation.

I’d love to report that this has generated a flurry of new apps.  I think I’m not alone when I say: we need some more real-world examples which show us the true power and potential of blending together different data sources….over the web…..using open standards.

The reality is: no apps, and no more than a passing curiosity in all the nice linked data, triple store wizardry.

Now, please don’t get me wrong.  I’m not complaining.  I think there are many very good reasons why we are where we are.   I’ll spare you my thoughts on the possible causes.  I’m guessing you’ll be much more interested in what I’ve actually done about it…..(which I’m hoping will spark a bit of discussion about going further, to tackle the underlying causes).

What I’ve done is: create this small, but hopefully important demonstrator application, along with some slides.

Acknowledgements

Before I outline how the app works, and what I’m trying to test and prove,  I wanted to say a huge thank you to the various data aggregators, API providers,  and generous human beings without whom none of this would have been possible.  A round of applause please for, in no particular order:

• The good folk at MySociety, for theyWorkforYou - which my app calls directly to show information about MPs.  Also, for the fab MapIT API, which I’m using to get and display info about geographic areas (Wards, Districts, and Constituences)
• Chris Taggart and his OpenlyLocal crew – which I use to show data about local councillors, and get the links to a Local Authority’s twitter feed.
• Friends at data.gov.uk, for their fantastic work on LinkedData and Sparqly endpoints.   You must learn to love this stuff, as I have done.
• The Guardian Datastore, and Alasdair Rae, for opening my eyes to the possibilities if only the data were available in more open, re-usable forms.
• And last, but by no means least, to Bill and Ric Roberts at Swirrl – for their generosity in hosting the IMD dataset, and their patience in answering my various dim, newbie questions.

 About the Indices of Multiple Deprivation (IMD) explorer application

Where has this come from?

I’ll begin with an excuse/confession.  I am not a professional software developer.  My code is a bit like my waistline: functional, but a little bloated.  All I have is my trusty old PC,  a copy of Aptana studio that I’m not afraid to use,  and copious volumes of coffee and paracetamol.

I’m doing this stuff in my spare time, to help me understand what open data really means from a data users and publishers perspective.   My passion for this stuff comes from a long, sometimes bitter experience of managing data in its various guises. Over the years, I’ve spent many, many hours with my head inside a SQL database, wrestling with the madness of pulling in content from a mix of magnetic tapes,  floppy disks, pen drives, CDs (and yes, even good old handwritten input).   I even remember punch cards and ticker tape!

So, my main motivation for developing the app was to test and prove how and why open data really is different.  For instance, how can it help us to quickly bring together disparate sources, and tell us something we didn’t know? Can it really cut out some of the cost and complexity of publishing and integrating ever increasing volumes of information?

I went in to this knowing a little bit about the LinkedData theory –  with a head full of acronyms and phrases like APIs; ontology; RDF; triples; and Sparql.  I had zero experience of actually using this stuff in the wild.

The result….

Four weeks, and many late nights later, out pops the IMD explorer.

For those of you who aren’t familiar with the IMD, it is – as the Guardian Datastore puts it – “possibly the most significant research into poverty in England ever put together“.

The IMD is particularly significant because it measures deprivation at a fine-grained, geographic level which relates directly to people, and where they live.   For the map geeks, the IMD data is published at the Lower Layer Super Output Area  (LSOA) level, which the ONS defines as an area with

“a minimum population of 1,000, with an overall mean of 1,500. LSOAs are built from groups of Output Areas. There are around 34,000 LSOAs in England and Wales”

IMD is also significant because is measures deprivation from different perspectives: for example, to show how deprived an area is according to crime, education, health, and so on.

So what, I hear you cry.  This matters because we often see the IMD being used as a sort of lens for viewing other, related datasets.  For a really topical example of this, check out the Guardian’s article exploring whether deprivation was a factor  in the recent riots.

Enough of all this theory – show me how the app works!

On one level, the application is simply about helping ordinary people to understand deprivation in all its guises where they live, work and play.

On the opening screen, I’ve provided a national map showing the top 50 most and least deprived areas (LSOAs).  You can switch the view to different measures (or domains) of deprivation via the drop down list, just below the page title.

Alongside that, is a box to type in a postcode, street, or place name, which then zooms to a map showing deprivation in the immediate local area.  From there, you can then move around the map – e.g. to zoom out to see deprivation across the whole Local Authority, or explore Wards within that locality.

Now here’s the twist, and the reason why linked/linkable data matters.

When exploring data at Ward-level, the application hops over to OpenlyLocal to pull back data about the councillors who you’ve elected to represent your interests in that ward.   So, if you’ve found issues with deprivation in your local area and are motivated to do something about it, you now have information about some of the people to talk to!

It’s a similar deal when exploring the Parliamentary Constituency – where the app goes and gets data about your MP from TheyWorkForYou, with links back to that site if you want to know how your MP is working in your local interests.

And there’s another twist.  Take a look at the “Education, Skills and Training” Deprivation domain.  Provided you’re not using Internet Explorer, you’ll see a map overlaid with the location and other information about local schools and other educational establishments.    I’ve particularly enjoyed developing this bit. It’s interesting, but not suprising to see the proportion of kids attending schools in deprived areas, who receive free school meals.    The data also suggests that schools in deprived areas are, relatively speaking, running below capacity – which is just an observation on what we can see and learn by joining up disparate datasets, and not a party-political point.

Anyway, that’s probably quite enough by way of an introduction to what I’ve done and why I’ve done it.  I will be following this up with further posts, exploring some of the more technical, “under the bonnet” aspects.

In the mean time, enjoy the app.  And my previous challenge still stands.  Start creating, and show us what you can do!

Steve

DCLG Spending data – making it more useful

An article by Steve Peters – stephen.peters1@sky.com, or tweeting at @SemanticTourist

Following my recent post, about trialling publication of statistical data as LinkedData, I wanted to update you on other things I’m doing to (hopefully) make it easier to find, understand and re-use DCLG’s data outputs.

I’ve been playing around with DCLG’s data on spending over £500, trying to find ways to:

• Make it more accessible – by amalgamating separate spreadsheets in to one long list of transactions, with the ability to search or filter that list in various ways; and
• Make it more useful – by including additional information about each transaction, to help the non-accountants amongst us begin to understand how  or why the money was spent.

The result is this Google Fusion table and – to illustrate what can be done – this prototype Spending explorer application.

Making the data more accessible

The Fusion table brings together all data for the 2010-11 financial year, as published in 10 or spreadsheets on the DCLG site.   Thanks to Google’s Fusion Table’s API, you can slice and dice this single list of transactions in a number of ways.

Here’s an example, which retrieves a Comma Separated list of all payments we’ve made to MITIE CATERING SERVICES LIMITED

http://tables.googlelabs.com/api/query?sql=SELECT%20*%20%20FROM%20334877%20where%20Supplier=’MITIE%20CATERING%20SERVICES%20LTD’

And another example, this time to get a CSV list of payments for Agency Staff – as recorded under the “Agency Staff” Expense Type.

http://tables.googlelabs.com/api/query?sql=SELECT%20*%20%20FROM%20334877%20where%20′Expense%20Type’='Agency%20Staff’

So much easier that wading through separate spreadsheets, and thousands of lines of data – and a nice illustration too of how and why APIs are an important alternative to publishing flat, largely disconnected files on a website.

Making the data more useful

The fusion table incorporates two additional columns – i.e. information that isn’t available in the published spreadsheets.

First, is data on the organisational “Units” in DCLG which made each payment.  The values here correspond to units in the organogram we published  in June 2010 - with some additional entries, reflecting more recent changes to the Department’s structure.

Second,  and with thanks to my colleagues in DCLG’s Finance Team, I’ve included a flag for each payment indicating whether it relates to: (a) money the Department spends on its own, day-to-day operations; or (b) money it has paid to external  organisations, typically as government grants.   You’ll see the Fusion table includes a new “FundingType” column, containing, with values of OVERHEADS or GRANT to indicate the type of payment.

So how does this make a difference?

In part as a personal learning exercise, I wanted to test whether and how offering data through an API with additional contextual information can make a difference.

I’m thinking about this from the perspectives of:

• Data owners and publishers – e.g. to begin to understand how APIs help us to remove some of the publishing burden, or  ”scale-up” to release more data, without getting sucked in to a file-based, content management quagmire.
• Data users and consumers – to determine how APIs can help you retrieve or data much more quickly and easily, understand it once its been retrieved,  and blend/combine it with other sources.

The prototype Spending Explorer application is my first stab at demonstrating why APIs are good.

For example,  its very easy to calculate headline trends in the data, then – using open free visualisation tools – compare, contrast and explore spending in various ways.  The application provides treemaps  for you to rummage through the data, starting at either total spend over the year under individual expense types, or – as in the example below – spending by DCLG Units.

Here, we can see that DCLG spent just over £311m running the organisation in 2010/11.  The treemap shows us how this breaks down across individual DCLG Units, which are “clickable” for you to drill in for more detail.

Because we’re calling data through the API, we can also quickly examine and explore it from different perspectives.  The application demonstrates this through its “By supplier” tab,  where the start point is now a list of payments to individual suppliers, ordered by total cash value, with the number of payments made, and the average value per-payment.

Here, we can see that DCLG paid a total of £15.5m to Land Securities, in 35 separate payments.  The smaller tables indicate how these payments break down across organisational units, and individual expense types.

Remember that, in these and all other cases, the application is simply asking the API to provide some data; which returns results in an open format; for the application to manipulate and display in any number of ways.

So what’s next then?

I’m hoping that I’m on to something here.  I think I have proved, in a modest way, that APIs can and do offer real benefits over publishing flat, disconnected files, especially when we incorporate additional contextual information.

I think too that we can see through these examples clear advantages for data publishers and data consumers – e.g. efficient, scaleable publishing in searchable, open formats, can and does enable re-use and innovation.

As we move through the next year, I can see a number of opportunities to improve and extend the volume and quality of data available via APIs.   Specific things I’m looking to do include:

1.  Incorporating links to the excellent Open Corporates site’s information about registered companies – so you can begin to join the dots between payments made by DCLG and other public bodies to the same individual suppliers.

2.   Aligning spending information with our soon-to-be-released, updated organogram – so you can see more clearly who in DCLG is accountable for different bits of departmental expenditure

3.   Providing additional context – such as postcodes for supplier addresses (to show where in the UK our suppliers are based), or to indicate whether the supplier is a private sector or voluntary/charity sector organisation.

4.   Joining up payment and budget information – so you can determine whether we’re on track to spend to agreed budget allocations.

I’m trying to move this forward in bite-sized chunks, as time and resource allows.   This work is running alongside a significant restructuring of the Department’s responsibilities, and downsizing of its resources.    In the mean time, please bear with me if things to be moving a little too slowly for your liking.

I would of course welcome any offers of help or support!  You know where I live.