Wednesday, November 18, 2009

Ordnance Survey free data: right decision, various wrong justifications cited

So yesterday the UK government announced that some data sets (not all) from the Ordnance Survey (the UK national mapping agency) will be made available for free - 1:10,000 scale data and above is included (so this includes popular OS maps like the 1:25,000 and 1:50,000, in digital form). The more detailed maps (1:1250 and 1:2500) are not included - but I believe that issues related to derived data will also be resolved, which will be useful in regard to those datasets too. Overall I think this is the right decision, in what is a much more complex issue than most people realize, as I discussed briefly in my recent georant on the topic. This approach makes available most of the data that is useful to the broader world, while minimizing the impact to Ordnance Survey's revenue, most of which comes from the large scale 1:2500 and 1:1250 map data (known as MasterMap). I was recently asked for my opinions by some consultants working for the UK government on the Ordnance Survey strategy, and this is the option I favored.

As I pointed out in my georant, making all Ordnance Survey data free would cost the UK taxpayer an extra 50 million pounds a year (a total of 100 million pounds a year). This approach should cost the taxpayer substantially less - though there will still be a cost, in the order of a few tens of millions a year. Nobody has said where spending will be cut to pay for this - but I personally think it will be money well spent in this scenario (Update: I have heard other estimates that the lost revenue due to sales may be more in the £5-10m range, but nobody seems to have a firm estimate yet - well nobody knows which products are involved so that makes it harder. My number was just an order of magnitude guess).

The Guardian, though my favorite newspaper, continues to make several incorrect statements in support of this move. They say "the move will bring the UK into line with the free publication of maps that exists in the US". What maps are they talking about? Again as I talked about in my georant, there are two main sources of central government maps in the US, the USGS "national map" and the US Census Bureau TIGER data. Both of these have very limited funding (as they are paid for by the taxpayer and deemed low priority compared to other things), and their map products are not good enough quality to be used as base maps by local government or infrastructure companies like gas, electric and telecoms. As a result, all of these companies and agencies do their own base mapping, leading to huge inefficiencies with all cities being mapped multiple times in inconsistent fashion, so data from the electric utility often doesn't line up at all with data from the gas utility, for example. More detailed map data created by local government agencies (like parcel maps) has a huge array of different policies - some give it away free, some free for non-commercial use only, and some charge significant amounts for it. So please don't hold up the US as an example of what you want in a national mapping infrastructure, it's a real mess here I'm afraid! I really hope that the UK government will step up to the increased taxpayer funding that Ordnance Survey now needs to continue its work as one of the premier National Mapping Agencies in the world, and that funding for mapping won't be cut drastically as it has been in the US (where for example, USGS has gone from having ~2500 people working on the national map in the 1980s to two hundred and something today).

The other thing that annoys me is that the Guardian cites the so-called "Cambridge Report" (PDF), which in my opinion is a very questionable document anyway, in a totally incorrect way. They say that the report says "making all OS data free would cost the government £12m and bring a net gain of £156m". Firstly, that quote alone is very misleading, which is just one of the problems I have with the Cambridge Report, but I won't get into that here (but may return to it in a future post). However, the scenario studied in the Cambridge report was not "making all OS data free", it studied the option of giving away all the large scale data for free, and not giving away the small scale data - in other words the EXACT OPPOSITE of what is being proposed. So the specifics of the Cambridge Report in regard to the Ordnance Survey have ZERO RELEVANCE to the decision which has been announced (except to reinforce that there is some benefit to society in making free map data available, which is stating the obvious anyway). I am in favor of the decision as I said, but like to think of myself as a scientist and a scholar :), and it really annoys me when people blatantly misrepresent evidence to make their case.

So anyway, don't get me wrong - I think that this is a very good thing for the UK geospatial industry, and for the general principle of open data, which I am a strong supporter of, despite that fact that I will also point out the challenges with it when appropriate! I think that the right broad option has been chosen out of a complex array of possible choices. But there are risks with the decision too, including the potential for reduced funding and deterioration in quality of Ordnance Survey Maps. And there are likely be some big losers too - including NAVTEQ and Tele Atlas (again), and in many ways OpenStreetMap, which is the topic of my next post.
Reblog this post [with Zemanta]

6 comments:

by James said...

Peter

Great piece. Have pointed up other flaws in the Guardian's reporting myself! Also mentioned similar concerns regarding the 'Cambridge Study' earlier in the year to one of those at the table yesterday. With scant other props not surprising it is the fallback position, however flawed. More independent work is certainly required on the costs and benefits of data economics and in particular on the full spectrum of both across the range of outcomes still possible from the excitement surrounding this new direction of travel.

Peter Batty said...

Thanks James, I appreciate the feedback! I have been thinking of writing a more critical review of the Cambridge report but haven't got round to it yet. It says what pretty much everyone in the geo industry wants to hear, that they should get data for free, so I don't think anyone has been inclined to point out various glaring issues which are there if you read it with even a slightly critical eye. So I'm glad to hear I'm not the only one with doubts about it (and concerns on the Guardian's misrepresentation of the issues).

Charles said...

"making all Ordnance Survey data free would cost the UK taxpayer an extra 50 million pounds a year (a total of 100 million pounds a year)."

Don't follow this. OS's income from the public sector is £50m pa (roughly). End those charges and you "save" £50m of public spending. You then have to fund OS's collection and collation of data, but not its licensing, legal, mapmaking or other functions. We don't know how much it spends on that, because it won't reveal it.

Your estimate on foregone revenue is lower than I've heard, but in the same region.

The point about US government maps is that they *are* free, for reuse in whatever way. That's what's brought into line. The piece doesn't say that the US has a Mastermap-like product for free, but you're suggesting it does. I think that's unfair.

Read the DCLG press release for clues on who's going to bear the burden of the lost revenues.

re the Cambridge report, your point that it talks about large scale mapping is a very good one. But it hasn't been self-evident to government over the past 10 years that making map data free would benefit the economy more than it would cost, so again, it's useful to have something empirical - even if OS's internal pricing mechanisms are so cloudy it can only stab at the numbers.

As to OSM - it will be interesting to see if it can incorporate the newly copyright-free, no-derived-data OS info into its database. That would be an amazing outcome.

Peter Batty said...

@charles I simply mean (and talking ballpark numbers here) that you have to make up for 50 million pounds in lost revenue from non-government sources (if you make all Ordnance Survey data free). You argue here that some costs could be saved in terms of OS legal, licensing, etc, and that may well be true, but I have seen no evidence that these savings would be anywhere close to 50 million, which seems very unlikely to me.

On the US, I think it's entirely fair to bring that up - the Guardian has on multiple occasions held up the US as a better model, yet there is no effective national mapping infrastructure here like there is in the UK, because of lack of funding. And that's where I am concerned that what the "free our data" campaign has been advocating (that the Ordnance Survey should be completely funded by the taxpayer) is not necessarily the right solution. I briefly summarized some of the issues in my georant. Should taxpayers be subsidizing very profitable private companies like utilities and telephone companies who would gain significantly from this data? You can make valid arguments either way, but a lot of the arguments that have been made have distorted the facts so that important questions like this haven't received the attention they deserve. There are significant but very different issues in both the UK and the US, but I would argue that the UK is much better off (and when this proposal is implemented that will be even more the case).

This new proposal seems to me as though it is probably a pretty good balance between realizing the benefits to society of a more open data policy (which I support), while helping ensure that OS continues to have reasonable funding and (at least some) private companies who gain substantial benefit from OS data contribute appropriately.

I remain skeptical that the Cambridge report proves anything, but even if you do believe it, it's not at all relevant to the current proposal. I'll probably write more about that in a future post.

Anonymous said...

I'm trying to understand the Guardian article to work out what map scales are actually being considered. I am confused where it says: "The prime minister said that by April he hoped a consultation would be completed on the free provision of Ordnance Survey maps down to a scale of 1:10,000, (not the scale of a typical Landranger map set at 1:25,000)."

Why "not" 1:25k? Clearly "down to 1:10k" would seem to include 1:25k, so is 1:10k actually a typo for 1:100k, or do they just mean that the cutoff scale is 1:10k rather than 1:25k? (And anyway Landranger is 1:50k.)

Peter Batty said...

@Anonymous, yes I think this was an error in the Guardian article - the general guidance was scales 1:10,000 and smaller, which would include 1:25,000 and 1:50,000, but not the most detailed maps at 1:1250 and 1:2500. The exact details of which products will be released is still to be determined.