Triple geo-conference goodness coming to Denver!!

Denver has always been known as a center for geospatial activity, and we have a great triple bill of events lined up, one in the near future and two back to back in September 2011.

The one coming up is WhereCamp5280 on November 19th. Eric Wolf, Ben Tuttle and I ran the inaugural one last year which was a great success, see James Fee's review. I hear a rumor that James will be back this year, so I guess he must have liked it! Eric and I have both been a bit swamped on other things recently, so Steve Coast has kindly taken up the organizing reins this year, thanks to Steve for that! Last year we were kindly hosted for free by Denver University (DU), this year we will be at University of Colorado Denver on their Auraria Campus, which has the advantage of being within easy walking distance of downtown. And this year we've decided to do one day rather than two. But two things that haven't changed since last year is that the event is FREE, and we'll be holding the social event on Friday evening at my loft, I expect there will be plenty of geo-beer from the Wynkoop Brewing Company downstairs and that may fuel some geo-karaoke later on. All this is thanks to our kind sponsors, who at the time of writing include Enspiria Solutions, ESRI, Google, MapQuest and Waze.

I'm expecting a great group of interesting attendees and presentations again this year, so highly encourage you to come along. And remember it's an unconference, so we are looking for as many people as possible to participate - prepare a short presentation or come prepared to lead a discussion on a topic that interests you!

Sign up for WhereCamp5280 here, and if you feel like sponsoring at anywhere from $16 to $1024 (can you tell that a techie geek set the sponsorship amounts?!) that would be great, but otherwise just sign up and enjoy the great free education, networking, and beer :).

So WhereCamp5280 is a great local event, but in September 2011 the global geo community will be converging on Denver for a fantastic double bill of FOSS4G and SotM.

For those who don't know, FOSS4G stands for Free and Open Source Software for Geospatial and is an annual international gathering organized by OSGeo. The last North American event was in 2007 in Victoria, BC, and since then it's been in Cape Town, Sydney and Barcelona, so we're delighted to have Denver join that list, and expecting a great turnout from around the world.

Eric Wolf and I led the bid to bring FOSS4G to Denver (which is one of the things we were busy on that was competing for time with WhereCamp5280). Eric was originally slated to be the conference chair, but unfortunately due to circumstances beyond his control he has had to stand down from that, and I have just taken over that role in the last week (well unless the OSGeo board fails to approve the change at their next meeting, but I'm assured that's not very likely!). I'd like to publicly thank Eric for all the work he did to bring the conference here - it was his idea initially, and definitely wouldn't have happened without all his efforts. We have the core of a great local organizing group set up already, but are still interested in recruiting a couple more folks, so if you'd like to help out please let me know. It's going to be a great event, and I'll be blogging plenty more about it over the coming months.

And on top of that it was announced today that Denver has also been selected to host State of the Map (SotM), the global OpenStreetMap conference, also in September 2011. I attended SotM in Amsterdam in 2009 and thought it was a fantastic event. Unfortunately I wasn't able to make it this year, but I will definitely be there next year :) ! The two events are distinct, but several people were involved in both bids, and we recognized that a lot of people would be interested in attending both, so the intent is for them to run back to back. The SotM date isn't fixed yet, but FOSS4G is locked in for September 12-16.

So if you're in the Denver area already, plan to be at WhereCamp5280 on Nov 19, and if you're not, make plans to be here in September 2011!

OpenStreetMap in Haiti - video

I am sure most readers of this blog have heard about how there has been a huge effort to map Haiti using OpenStreetMap - Harry Wood gave a good summary of efforts a few weeks ago. Currently Schuyler Erle and Tom Buckley are down in Haiti helping out with mapping for the relief efforts on ground - Schuyler's blog has really interesting accounts of what they've been doing. Schuyler tweeted today that he and Tom were "starting to post raw, iffy quality, unedited audio and video - please feel free to edit/remix/repost". So here's my quick edit of what they've posted so far - I'll try to do something similar if they post more in the coming days. The video features people from UNOSAT and UN OCHA explaining how useful OpenStreetMap data was to them - and also some clips from a prayer march. Click through on the video to see a larger version (and choose 480p).

Update: some quotes from the video ...

Catherine from OCHA (the UN Office for the Coordination of Humanitarian Affairs): "Basically it's the best source of transportation information that we have for Haiti ... it's the most comprehensive one, and the most up to date, of course. This is information that everyone is using here. There is a big GIS group here for this humanitarian response, and everybody is using OpenStreetMap."

Schuyler: "OpenstreetMap was used for the UNOSAT building damage assessment ..."

Olivier from UNOSAT (UN Operational Satellite Applications Programme): "Of course we used OpenStreetMap on a daily basis ... every day it was possible to improve our maps ... with new information. Usually it is impossible to get this information ... so thanks to this volunteer platform, it has been fantastic for us, that's for sure."

OpenStreetMap meetup in Denver next Wednesday

On Wednesday next week, there will be an informal meetup at the Wynkoop Brewing Company in downtown Denver, to drink beer and discuss OpenStreetMap. Steve Coast, the founder of OpenStreetMap, will be there, and it should be an interesting group of local geo folks. We'll be there from 5:30pm until 9ish or later!

If you’re already involved with OpenStreetMap, this is a chance to get together with other contributors and discuss ideas – I would love to see us having more coordinated initiatives in the Denver area than we’ve had so far. And if you haven’t been involved but are curious about OpenStreetMap, this is a great opportunity to learn more. Hope to see a good turnout – please sign up here! That's Steve and me in the photo above by the way, to help you track us down if you don't know us!

The great OpenStreetMap license debate

If you are involved with OpenStreetMap, you may or may not be aware that a lot of work has been been going on to develop a new license. If you are signed up for any of the OpenStreetMap mailing lists, then you certainly know that this work has been going on, as there has been a torrent of emails containing very heated debate on the topic over the past week or so (although work in this area has been going on for a couple of years).

From the long and passionate emails of a small number of people, you might get the impression that the new license is some sort of subversive scheme to somehow take over or undermine OpenStreetMap.

Everyone is entitled to their opinion of course, but a few key points I would make are as follows:

• I support the license change and encourage you to do so too.
• A lot of very bright people who know way more about data licensing than I do have spent a huge amount of time working on the new license and I respect their knowledge and opinions very highly.
• The License Working Group followed a very open and consultative process that allowed for lots of input from the community (you can see the minutes of over 50 meetings they have held this year alone).
  
• The current license from Creative Commons has some significant shortcomings for databases - it is intended to apply to "creative works" and even Creative Commons says that it should not be used for databases.
• The new license embodies very much the same spirit as the old license, but is much more enforceable, and better protects both the data and users of the data. So anyone who was comfortable contributing data to OpenStreetMap under the old license should be comfortable with the principles of the new license.
• Switching from a "ShareAlike" license (which basically is more aggressive about ensuring that OpenStreetMap data remains open) to a "Public Domain" license (which has no restrictions), which is something that has been discussed, but would be a much bigger change than what is being proposed. This is worthy of further debate in the future but even if you support the Public Domain approach (which I lean towards but am undecided), I believe that the right short term approach is to get the project on a more solid legal footing than it is on currently (but without a major change to the original license philosophy).
• There have been concerns expressed about the possibility of losing some data if existing contributors refuse to make their data available under the new license terms. There is a little risk here but I think it is overstated. The two licenses are so similar in their philosophy that it is hard for me to see a sensible reason why someone who was happy to contribute data under the old one would not do so under the new one. With data imports from larger organizations it may take a little time to work through some bureaucracy but I really think that should be doable, and various members of the community have volunteered to help out with this as needed.
  

This "human readable summary" does a good job of simply conveying the key principles of the new license.

If you would like to hear a longer discussion on these topics, you can listen to this 45 minute podcast which is a discussion on this topic including myself and various other more knowledgeable people, including several members of the License Working Group.

As I said in an email to one of the OpenStreetMap mailing lists, I think that the License Working Group has put in a huge amount of work on this effort, and I would like to sincerely thank them for that. As I said above, I think that this proposal would move OpenStreetMap onto a much stronger legal footing.

Finally, as I said at the end of the podcast and in an email to one of the lists, while the license is important, it is not the main aim of OpenStreetMap - that is to produce a great free and open map of the world. A lot of work has gone into the new license and big improvements have been made, but now is the time to vote and move on, and to let the community focus more of its energy on more important items like getting more people mapping, further improving the quality of the map data, and so on.

Ordnance Survey free data: right decision, various wrong justifications cited

So yesterday the UK government announced that some data sets (not all) from the Ordnance Survey (the UK national mapping agency) will be made available for free - 1:10,000 scale data and above is included (so this includes popular OS maps like the 1:25,000 and 1:50,000, in digital form). The more detailed maps (1:1250 and 1:2500) are not included - but I believe that issues related to derived data will also be resolved, which will be useful in regard to those datasets too. Overall I think this is the right decision, in what is a much more complex issue than most people realize, as I discussed briefly in my recent georant on the topic. This approach makes available most of the data that is useful to the broader world, while minimizing the impact to Ordnance Survey's revenue, most of which comes from the large scale 1:2500 and 1:1250 map data (known as MasterMap). I was recently asked for my opinions by some consultants working for the UK government on the Ordnance Survey strategy, and this is the option I favored.

As I pointed out in my georant, making all Ordnance Survey data free would cost the UK taxpayer an extra 50 million pounds a year (a total of 100 million pounds a year). This approach should cost the taxpayer substantially less - though there will still be a cost, in the order of a few tens of millions a year. Nobody has said where spending will be cut to pay for this - but I personally think it will be money well spent in this scenario (Update: I have heard other estimates that the lost revenue due to sales may be more in the £5-10m range, but nobody seems to have a firm estimate yet - well nobody knows which products are involved so that makes it harder. My number was just an order of magnitude guess).

The Guardian, though my favorite newspaper, continues to make several incorrect statements in support of this move. They say "the move will bring the UK into line with the free publication of maps that exists in the US". What maps are they talking about? Again as I talked about in my georant, there are two main sources of central government maps in the US, the USGS "national map" and the US Census Bureau TIGER data. Both of these have very limited funding (as they are paid for by the taxpayer and deemed low priority compared to other things), and their map products are not good enough quality to be used as base maps by local government or infrastructure companies like gas, electric and telecoms. As a result, all of these companies and agencies do their own base mapping, leading to huge inefficiencies with all cities being mapped multiple times in inconsistent fashion, so data from the electric utility often doesn't line up at all with data from the gas utility, for example. More detailed map data created by local government agencies (like parcel maps) has a huge array of different policies - some give it away free, some free for non-commercial use only, and some charge significant amounts for it. So please don't hold up the US as an example of what you want in a national mapping infrastructure, it's a real mess here I'm afraid! I really hope that the UK government will step up to the increased taxpayer funding that Ordnance Survey now needs to continue its work as one of the premier National Mapping Agencies in the world, and that funding for mapping won't be cut drastically as it has been in the US (where for example, USGS has gone from having ~2500 people working on the national map in the 1980s to two hundred and something today).

The other thing that annoys me is that the Guardian cites the so-called "Cambridge Report" (PDF), which in my opinion is a very questionable document anyway, in a totally incorrect way. They say that the report says "making all OS data free would cost the government £12m and bring a net gain of £156m". Firstly, that quote alone is very misleading, which is just one of the problems I have with the Cambridge Report, but I won't get into that here (but may return to it in a future post). However, the scenario studied in the Cambridge report was not "making all OS data free", it studied the option of giving away all the large scale data for free, and not giving away the small scale data - in other words the EXACT OPPOSITE of what is being proposed. So the specifics of the Cambridge Report in regard to the Ordnance Survey have ZERO RELEVANCE to the decision which has been announced (except to reinforce that there is some benefit to society in making free map data available, which is stating the obvious anyway). I am in favor of the decision as I said, but like to think of myself as a scientist and a scholar :), and it really annoys me when people blatantly misrepresent evidence to make their case.

So anyway, don't get me wrong - I think that this is a very good thing for the UK geospatial industry, and for the general principle of open data, which I am a strong supporter of, despite that fact that I will also point out the challenges with it when appropriate! I think that the right broad option has been chosen out of a complex array of possible choices. But there are risks with the decision too, including the potential for reduced funding and deterioration in quality of Ordnance Survey Maps. And there are likely be some big losers too - including NAVTEQ and Tele Atlas (again), and in many ways OpenStreetMap, which is the topic of my next post.

PublicEarth goes live

Boulder-based startup PublicEarth, a free wiki for places, has gone live today. I've been on their Advisory Board since they started last year, so am excited to see them launch - congratulations to founder Duncan McCall (pictured below talking at WhereCamp5280), Chairman David Hose, and the rest of the team.

Duncan talks about his original vision for PublicEarth here:

So the idea for a wiki of ‘interesting’ or ‘long tail’ places was born, with ability for users to create, edit and share information in an structured, categorized and intelligent format – with the simple ability to get this information onto mobile and GPS devices.

I see PublicEarth being complementary to OpenStreetMap, which is of course another well known crowdsourced effort around geospatial data. OpenStreetMap is primarily focused on creating a free and open basemap for the world, with some basic points of interest. PublicEarth is not about the basemap, but about creating richer information about interesting places. For example, there's a lot of good information about Public Art in there - I just found out that the big Blue Bear sculpture in Denver was created by a fellow Brit, which I hadn't known before! That data came from Public Art Review - one of the things that PublicEarth has done, and is looking to do more, is to help a variety of organizations who have interesting datasets (like this one about contemporary public art) and make them more accessible, in a way in which they can be easily queried by location. There are all kinds of other interesting datasets in there, with more to come. And of course the whole idea of a wiki or crowdsourced approach is that anyone can create or update information, so I encourage you to give it a try and create some interesting places of your own!

Was the Google Maps data change a big mistake?

So the discussions about the great Google map data change in the US rage on, and we are seeing more and more reports of significant data quality issues. I wrote about how Central City Parkway was completely missing, and I reported this to Google to see how the change process would work. I posted later about how it had been partially fixed, with a new geometry visible but not routable, and with the wrong road name and classification. The latest state (on November 5, after reporting the error on October 9), is that is now routable, but it still has the wrong road classification, being shown as a minor road rather than a major highway. This means that if you calculate the best route from Denver to Central City, Google gets it wrong and doesn't go on Central City Parkway, choosing some much smaller mountain roads instead, which take a lot longer. Microsoft Bing, Yahoo, MapQuest and CloudMade (using OpenStreetMap) all calculate the correct route using Central City Parkway. Another substantial error I found recently is that if you search for one of the companies I work for, Enspiria Solutions (by name or address), the location returned was about a mile off. This has now been partially but not entirely fixed (after I reported it).

Steve Citron-Pousty recently wrote about whether Google made the data change too soon. He talked about how his wife has always used Google Maps, but it has got her lost four times in the past week and she has now switched to using MapQuest or Bing. And Google got Steve lost in the Bay Area last week too. Maitri says she is "splitting up with Google Maps" over issues in Ohio, as "there is no excuse for such shoddy mapping when MapQuest and Yahoo do exceptional work in this area the first time around". She links to an article in a Canton, Ohio, newspaper about how the town was mis-named after the recent data change (we call it Canton, Google calls it Colesville). James Fee pointed out an error with Google showing a lake that hadn't been there for 25 years or so. Matt Ball did a round-up discussion on the importance of trusted data. The well known tech journalist Walt Mossberg reviews the new Motorola Droid phone (which uses the new Google data for navigation), and in passing says when reviewing the navigation application "but it also gave me a couple of bad directions, such as sending me the wrong way at a fork in the road". And then in news which is presumably unrelated technically (being the in the UK), there was a lot of coverage of a story about how Google Maps contained a completely fictitious town called Argleton - which even though a separate issue does not help the public perception of the reliability of Google Maps data.

Update: see quite a few more stories about data issues in the comments below.

So anyway, this is a long and maybe somewhat boring list, but I think that it is worth getting a feel for the number of stories that are appearing about map data errors. As anyone in the geo world knows, all maps have errors, and it's hard to do a really rigorous analysis on Google's current dataset versus others. But I think there is strong evidence that the new Google dataset in the US is a significant step down in quality from what they had before, and from what Microsoft, Yahoo and MapQuest have (via Tele Atlas or NAVTEQ).

Google clearly hopes to clean up the data fairly quickly by having users notify them of errors. But looking at the situation, I think that they may have a few challenges with this. One is just that the number of errors seems to be pretty large. But more importantly, I think the question for Google is whether consumers will be motivated to help them fix up the data, when there are plenty of good free alternatives available. If Google gives you the wrong answer once maybe you let it slide, and perhaps you notice the link to inform them of the problem and maybe fill it out. But if it happens a couple of times, is the average consumer likely to keep informing Google of errors, or just say "*&#% this, I'm switching to MapQuest/Bing/Yahoo"?

Google has made some reasonable progress with Google MapMaker (its crowdsourced system for letting people create their own map data) in certain parts of the world, but these are generally places where there are not good alternative maps already, or people may be unaware of alternatives like OpenStreetMap. So in those cases, people have a clearer motivation to contribute their time to making updates. People who contribute time to OpenStreetMap have a range of motivations, but in general for most of them it is important that the data is open and freely available, which is not the case with Google (at least not so much, I won't get into the details of that discussion here). Most if not all the people I know who contribute effort to OpenStreetMap (myself included) would not be inclined to contribute significant updates to Google (except for some experiments to see how good or bad the update process is).

Consumer confidence is a fickle thing, and you probably don't need too many stories in the newspapers of mishaps due to bad data, or more than a couple of direct experiences of getting lost yourself due to bad data, to switch to a different provider (especially when you are choosing between different free systems - you have a bit more incentive to stick with a navigation system and try to make it work if you've spent a few hundred dollars on it).

The risks are even higher with real time turn by turn directions - no matter how many caveats you put on it, you are likely to get some drivers who follow the directions from the system and don't notice relevant road signs. You only need a couple of accidents because people drove the wrong way up one way streets because of bad data to damage consumer confidence even further.

So I think it will be very interesting over the next few months to see whether the data quality issues are bad enough to result in significant numbers of users moving away from Google Maps in the US or not - and whether Google will get significant uptake in terms of number of people contributing error reports in the US (beyond the initial wave of curiosity-driven updates just to test if the process works). Obviously the answer to the second question is likely to have a big influence on the first. Stay tuned ...

Google shakes up the geospatial data industry

Well, the big news of the day is that Google has dumped Tele Atlas as the main data provider for Google Maps in the US, and is providing its own map data from a variety of sources (presumably also including its own Streetview teams). They've also added the ability to point out errors in the map, another addition to the crowdsourcing techniques they've been using. The announcement has caused a flurry of discussion of course. James raises questions about various aspects of the data (especially parcels). Steve speculates that the same thing will happen in Europe and that the beneficiary there will probably be AND.

The new Google data certainly adds details in some places, from a quick random sampling - for example check out Commons Park in downtown Denver using the nice GeoFabrik Map Compare tool. None of those paths were there previously in Google. Still not quite as good as OpenStreetMap in this case though :).

This does dramatically reshape the geospatial data industry though. Previously there were two commercial providers with a detailed routable database of roads in the US, NAVTEQ (owned by Nokia) and Tele Atlas (owned by TomTom), now at a stroke there is a third in Google. OpenStreetMap is a fourth provider of course, not quite up with the other three in terms of coverage and routing quality in the US yet, but getting there very quickly.

This raises lots of interesting questions:

• Will Google sell its data to providers of third party navigation systems and compete with Tele Atlas and NAVTEQ? Or indeed will they sell/license it to others who could use it (users of GIS, etc)?
• Will Google Maps on the iPhone (and other mobile devices) get real time turn by turn directions? This was previously prohibited by licensing terms from Tele Atlas and NAVTEQ. Existing real time navigation systems using data from these two providers generally cost in the region of $100. Will Google add this to the free maps offering? Or sell a version that does real time turn by turn directions?
• Will Google contribute any of this data to open data initiatives like OpenStreetMap? Or make it available to USGS for the US National Map? In the past they have cited licensing constraints from their data providers as a reason for not being more open with their geospatial data, that reason largely goes away now (though we don't know all the new data providers and their terms). I'm not holding my breath on this one, but we can hope!
• Will this negatively or positively impact OpenStreetMap? Previously in areas with active communities, OpenStreetMap had significantly more detail, and more current data, than Google - this appears to move Google forward in that regard. But will Google taking another step towards total world domination encourage more people to want an open alternative?

So anyway, definitely a very interesting development for the geospatial data industry (albeit one that has been on the cards for a little while). It will take a little while to understand the full implications. I'm sure Tele Atlas is glad they are no longer an independent company, I wouldn't like to have seen how their stock price would have dropped today otherwise :O !!

Update: some more discussion and a link to a podcast on this topic featuring Steve Coast, James Fee and me in this follow up post.

My GeoWeb 2009 presentation

Here's a video of my GeoWeb presentation on "Building a new location-aware infrastructure for calendaring and scheduling". Recorded and edited using ScreenFlow.

GeoWeb 2009 presentation by Peter Batty from Peter Batty on Vimeo.

I'll post more thoughts on GeoWeb soon - in the mean time you can check out my tweets and the geoweb stream on twitter. And there are some pics of the social event here.

Twitter discussion on "neogeography" versus "GIS"

My post yesterday triggered some interesting discussion on Twitter (I guess the cool kids don't comment on blogs any more, that's so 2008!). Here's a quick cut and paste with most of the relevant stuff (the relevance fades towards the end but I figured I would leave some of those bits in!). I will post more thoughts shortly on what I think "neogeography" actually is. If you want to figure out who someone is on twitter, just go to twitter.com/username - for example I'm at twitter.com/pmbatty.

cageyjames: @pmbatty isn't all that stuff still very niche? I'm just not seeing the quick pickup you are I guess.

cageyjames: @pmbatty That said I'm coming off #esriuc and you #sotm so we skew on our last experiences. Reality is probably between us.

cageyjames: Of course Neo and GIS are just labels people put on others to limit their growth.

pmbatty: @cageyjames early days still but my point is more rapid growth of neo in areas like cartography, analysis, etc than many people think

pmbatty: @cageyjames I agree I'm not a big fan of labels like neo, paleo, GIS, it's all the same problem space

cageyjames: @pmbatty but can you call that neo? What Stamen is doing isn't neo in my book. You seem to say innovative is neo. That isn't true at all.

cageyjames: @pmbatty When I was taking carto classes back in college, was I neo because I used Freehand over pen and paper? Not in my book.

pmbatty: @cageyjames Stamen is using OSM, Mapnik, Virtual Earth, etc - would think that was "neo" though we agree the label is a bit pointless

geobabbler: @pmbatty Neo is GIS. Just like "desktop mapping" was.

pmbatty: @geobabbler I agree that Neo is GIS - I think that is what I was trying to say but you were more concise :)

geokaren: @pmbatty @geobabbler I think of neo as less rooted in technology and more the concept of moving it beyond traditional GIS

cageyjames: @pmbatty ESRI uses OSM, Virtual Earth and other "neo" packages. That doesn't make them neo. Tools make not neo.

geobabbler: Neogeography represents the intersection of GIS with the mainstream web. It's the web telling GIS "this is how you do it."

geobabbler: @pmbatty and I was trying to agree with you but not very well. :-)

pmbatty: @cageyjames Not clear on your definition of "neo"?

pmbatty: @geobabbler Good, I think we're agreeing we agree - 140 chars is a limitation sometimes :)

SeanGorman: @cageyjames @pmbatty 1) getting GIS stuff to work in a browser 2) making GIS stuff accessible to non-GIS people = current innovation

pbissett: @geobabbler @pmbatty I sometimes wonder if we shouldn't maintain an Geo-IRC channel for such discussions. Is there interest?

SeanGorman: Tough bit will be doing new things GIS or others have not thought of - handling/analyzing/visualizing big data and mobile data = my bet

geobabbler: @SeanGorman The "new things" will arise from the questions asked by new audiences as GIS continues to diffuse.

cageyjames: @pmbatty neo is a label, nothing more

SeanGorman: Serving tiles for viewing imagery/maps seems the biggest innovation that GIS was not doing at all followed by user generated geo-content

cageyjames: @SeanGorman it is all a moving target isn't it. What was once hacker is now helping me locate coffee shops near crime.

geobabbler: @cageyjames spot on.

geobabbler: @pbissett it has been bandied about before.

geobabbler: it's somewhat ironic that I am engaged in a "neogeography" discussion while waiting for @ajturner.

pbissett: @geobabbler If I remember correctly from "Innovators..", it was the search for new solutions to old problems that caused the disruption.

SeanGorman: @geobabbler tell him has to wear a wig every time he is referred to as the father of neogeography

SeanGorman: @geobabbler possibly some knickers and a waist coat as well

SeanGorman: @cageyjames yes - because I want to be perky when I get mugged. If life was stationary it would be quite dull.

pmbatty: @geobabbler @SeanGorman I was just going to say that @ajturner will explain all this to you, he is our father :)

cageyjames: @SeanGorman classic, now give me the GeoRSS feed!

SeanGorman: @pmbatty hahahaha I think that quote now makes him Darth Vader or possibly Darth Helmet

SeanGorman: Ok I'm going to quit heckling before @ajturner brings up the fact I can't write code and that I'm of dubious use in general

jharpster: @SeanGorman I bet if we brought out some VBA code you'd be all over it. Don't take @ajturner too seriously.

jharpster: Blog by @pmbatty on the influence of neogeography. "people in the traditional GIS space remain largely unaware" http://bit.ly/21QiKC

ajturner: consider this corollary for those asking what 'neogeography' is: is using an iPhone to update Facebook the same as 'Information Technology'?

cageyjames: @ajturner yup, my point exactly. We are all neo and all paleo. Thus labels fail us when applying at a high level.

fantomplanet: @cageyjames I once heard @ajturner used walk to school uphill both ways!

fantomplanet: @ajturner Answer to your corollary: It's an element and activity that is part of secondary orality (http://bit.ly/10SJeQ). Same with carto.

fantomplanet: @SeanGorman Yeah, but I can still code... A little. #fullspectrumgeographers

SeanGorman: @fantomplanet bragger - I programed my VCR last week to tape "the View" while I was at work.

cageyjames: @fantomplanet coding is for sissies, you get hair on your back by managing projects.

pmbatty: Finished beer and interesting conversation with @perryevans, solved world's neogeography problems, now time for dinner :)

fantomplanet: @pmbatty "Solved world's neogeography problems?" #neogeodeathakawherecamps?

pmbatty: @fantomplanet I just hope I can still remember all the solutions tomorrow :)

geomantic: @pmbatty @SeanGorman @geobabbler @cageyjames I'm late to the neo paleo geo GIS discussion, but see you found agreement 140 chars at a time?

pmbatty: Wish I was clever enough to understand @ajturner, the Yoda of neogeography :) (@SeanGorman: gift suggestion http://tinyurl.com/l6z62g)

fantomplanet: @pmbatty All @SeanGorman needs to buy @ajturner is a burlap sack and turn it in-side-out. #neogeoyoda

geomantic: @fantomplanet @pmbatty @SeanGorman ...or buy @ajturner a monk's habit. Oh, and get him an appt for a Friar Tuck haircut, too. #picturethis

fantomplanet: @geomantic I think @ajturner already has the haircut.

geobabbler: @ajturner it looks like @gezjames found your wig.

DruidSmith: With all this #neogeography kum-ba-ya, I'm surprised we haven't yet seen any choice words from @fakeajturner entering the fray...

fakeajturner: What? Have I been summoned? I was too busy showing the neogeo "who's your daddy"

OffMaps for iPhone review

On my recent trip to Amsterdam for the OpenStreetMap State of the Map conference, I made extensive use of the OffMaps mapping application for the iPhone - which incidentally uses OpenStreetMap data. The big advantage of OffMaps compared to the standard Google Maps application is that it can run offline, which is really important if you are going abroad, as data usage charges make it prohibitively expensive to use an online mapping application (you quickly get into hundreds of dollars worth of data charges!). The functionality is fairly simple - currently it's basically map display, without routing or search for points of interest (though these things are planned for the future, I heard from Felix Lamouroux, the main developer, who I met at the conference). But it makes good use of the GPS and the compass in the iPhone 3GS, which make it very intuitive and easy to use when walking around. Performance is very good since all the data is local. There is a simple interface for choosing the data to download and store locally (which you want to do ahead of time when you're at home, or connected via WiFi). You can run it in an online mode too, where it will dynamically fetch data over the network if it isn't already stored locally.

The OpenStreetMap maps for Amsterdam were very good quality and nice looking. This application illustrates one of the key advantages of OpenStreetMap over Google Maps and other commercial solutions, which is that Google licensing prohibits you from using the data offline. This fact, plus the availability of the Cloudmade iPhone Maps Library, is making OpenStreetMap data a popular option for mapping applications on the iPhone - just today, Dopplr announced a new iPhone application which also uses OpenStreetMap data (though strangely it uses different rendering from the other applications I've tried, and none of the freeways in Denver show up, though they do in the other applications - but I'm sure that will be fixed shortly).

So I would strongly recommend OffMaps, especially if you're going on an overseas trip - I am currently downloading data for Vancouver in preparation for my upcoming trip to GeoWeb, so if you're there and want to check it out please ask me and I'll show you!

WhereCamp5280 - cool event coming up in Denver!

We’ll be holding a fun, cool, educational and FREE geospatial event in Denver on August 14-15 – WhereCamp5280 (for those not from round here, 5280 is the elevation of Denver above sea level in feet – exactly one mile high!). If you’re already familiar with WhereCamp, you probably don’t need any more persuading to come along. The event is being organized by Eric Wolf, me, Ben Tuttle, and anyone else who would like to help out!

For those not familiar with WhereCamp, it’s an “unconference” on the subject of location, mapping and things "geo". An unconference is basically an informal conference which is organized on the fly by the participants. I’ve attended two unconferences now, both of which were excellent. The first was held by FRUGOS (Front Range Users of Geospatial Open Source), which you could also consider a predecessor to this event – though we intend for this to cover not just open source solutions (though that’s up to the participants, of course!). Here are a couple of reports on the FRUGOS unconference by me and Charlie Savage. The second was WhereCamp in San Jose this year, which I found really interesting and fun. I have to say that based on these two unconferences, the quality of presentations has been at least as good if not better than the “formal” conferences I have been to.

We have already had expressions of interest in attending from a wide variety of interesting geo-people, both locally and from out of town, so I’m sure there will be lots of great discussions. For an up to date list of people likely to attend check here.

If you are potentially interested in participating, please sign up to our Google Group and add your name to the likely attendee page. We encourage you to list topics that you would be interested in talking about or leading a discussion on, but this is definitely optional! While the agenda will certainly be flexible in unconference style, we plan to get some ideas together in advance to help us make the most of our time during the two days.

If you’re interested in helping out, please let me or Eric know. While DU has kindly donated the rooms for the event, we are aiming to get some modest sponsorship to be able to cover other costs including:

• A fun social event on Friday evening (probably either at the Wynkoop Brewing Company, or my loft which is right above the Wynkoop)
• Cost of projector rental from DU (which is not free) – if you have access to a projector(s) that we could use on Friday and/or Saturday, please let me know
• Lunch on Friday and Saturday

Please see the Google Group for more info – your questions, suggestions and assistance are appreciated! We really hope to make this a good venue for exchanging ideas between the “neogeographers”, open source users, and users of traditional GIS (and some of us, like me, fit into all of those camps). If you work in the geo area in Denver, tell your manager that this is the best free education you can get on the latest developments in the space!

My presentation on "Geodata creation: past, present and future" at State of the Map 2009 #sotm09

A video of my presentation at the recent OpenStreetMap State of the Map conference is now online at Vimeo. There's also a copy of the slides at SlideShare. I talk about the four major business models that have been tried in regard to creating geodata, and how they are all handicapped by the very high costs involved when you use the traditional approach which involves paying the people who create the data. And I discuss how crowdsourcing completely changes things by reducing this labor cost to zero. From everything I saw at State of the Map, I am convinced that use of crowdsourcing in general, and OpenStreetMap in particular, is going to massively grow over the next couple of years.

Geodata creation: past , present and future - Petter Batty (Spatial Networking) from State of the Map 2009 on Vimeo.

Other videos are being posted online at the sotm09 channel on Vimeo - I recommend you take a look. There are lots of great presentations, but one that I guarantee will make you smile is this 5 minute one on Mapping of Historical Sites in Japan - check it out :).

Quick report on OpenStreetMap State of the Map 2009 conference

This is just a quick initial report on the OpenStreetMap State of the Map conference, which has just finished in Amsterdam. First I'd say read Steven Feldman's summary, which I completely agree with. As I said in my keynote talk here (see the slides, video should be online at some point soon, and I'll summarize it in a future post), the industry has always been hampered by the very large cost of creating and maintaining geodata. Despite all the technological advances over the years, this remains a very labor-intensive process and therefore is just fundamentally expensive by traditional methods (where you pay the people doing the data capture).

Crowdsourcing changes the paradigm by having volunteers contributing their time, and having access to free data (without complex licensing constraints) completely changes the economics of developing geospatial applications. Obviously the first reaction of most traditional geospatial people is to ask whether you can get good enough data quality using this approach. Dr Muki Haklay presented a very thorough and rigorous analysis of the quality of OpenStreetMap data against Ordnance Survey data in the UK, and his conclusion was "OpenStreetMap quality is beyond good enough, it is a product that can be used for a wide range of activities" (not in general - today - for very large scale mapping, but for small or medium scale mapping - the type of applications that today might use today data from Google or Microsoft, NAVTEQ or Tele Atlas).

The conference itself had a tremendous buzz about it, with 250 people from thirty-something countries (I think), and great enthusiasm and excitement from everyone participating. The presentations were a great mixture of people talking about really cool and innovative technical things, and heart-warming and moving stories about people creating maps from all parts of the world - in many parts of the developing world, OpenStreetMap is way ahead of any other data source.

Anyway, I'm off to be a tourist in Amsterdam for the day, I will write more in due course. If you are attending the ESRI user conference this week I really recommend you seek out the OpenStreetMap booth to find out more about what they are up to. And thanks to everyone who attended State of the Map for making it such a fun and inspiring event!

Statements that come back to haunt you

Just read this article in Forbes, which says:

"I laugh when I hear that you can make a map by community input alone," says Tele Atlas founder De Taeye. He says that if tens of thousands of users travel a road without complaining, then Tele Atlas can be fairly certain that its map of the road is correct.

The first statement is demonstrably false already (see my earlier post about Oxford University and OpenStreetMap for just one example). But then it's a little bizarre that he follows that up by saying that the reason they know that their data is correct is through (lack of) feedback from the community. I've done my share of interviews and it's quite possible that these two statements were taken out of context, but it's an odd juxtaposition. I'm sure the owners of Encyclopedia Britannica laughed at the notion that you could produce a comparable online encyclopedia by community input alone, but they aren't laughing any more and are even moving to accommodate community input, as are most of the main mapping data providers of course (including Tele Atlas).

Community input isn't the answer to all data creation and maintenance problems, but it provides an excellent solution in a number of areas already, and the extent of its applicability will increase rapidly.

Google Local Search better than Google Geocoding?

Google offers two (apparently) unrelated solutions for doing geocoding (converting a text string into a latitude-longitude and a structured address): the Google Geocoding API (which is part of the Google Maps API) and the Google Local Search API (which has JavaScript and non-JavaScript variants). This post discusses our experiences with both and the results of recent testing we have been doing, which has led us to a decision to switch from our current hybrid approach to using only local search.

Initially we were attracted to using Google Local Search for whereyougonnabe, as it lets you search for places of interest like "Wynkoop Brewing Company" or "Intergraph Huntsville", rather than having to know an address in all cases - whereas the geocoding API only works with addresses.

However, in our initial testing with local search we found a number of cases where it returned a location and address string successfully, but did not properly parse the address into its constituents correctly (for example, returning a blank city). For our application it was important to be able to separate out the city from the rest of the address. In our initial testing, the geocoding API seemed to do a better job of correctly parsing out the address. In addition, it returned a number indicating how accurate the geocoding was (for example street level or city level). So we ended up implementing a rather ugly hybrid solution, where we first used local search to allow us to search for places of interest in addition to addresses, and then passed the address string which was returned to the geocoding API to try to structure it more consistently. In most cases this was transparent to the user, but in a number of cases we hit problems where the second call would not be able to find the address returned by the first call, or where the address would get mysteriously "moved". With so much to do and so little time :) we elected to go with this rather unsatisfactory solution to start with, but I have recently been revisiting this area, and doing some more detailed testing of the two options.

Before I get into more details of the testing, I should just comment on a couple of other things. One is that a key reason we are revisiting our geocoding in general is that we recently introduced the ability to import activities from external calendar systems, which means we need to call geocoding functionality from our server from a periodic background process, whereas before that we just called it from the browser on each user's client machine. This is sigifnicant because the Google geocoding API restricts you to 15,000 calls per IP address per day, which is not an issue at all if you do all your geocoding on the client but quickly becomes an issue if you need to do server based geocoding. Interestingly, Google Local Search has a different approach, with no specified transaction limits (which is not to say that they could not introduce some of course, but it's a much better starting point than the hard limit on the geocoding API).

Secondly, a natural question to ask is whether we have looked at other solutions beyond these two, and the answer is yes, though not in huge detail. Most of the other solutions out there do not have the ability to handle points of interest in addition to addresses, which is a big issue for us. Microsoft Virtual Earth looks like the strongest competitor in this regard, but it seems that we would need to pay in order to use that, and we need to talk to Microsoft to figure out how much, which we haven't done yet - and obviously if we can get a free solution which works well we would prefer that. Several solutions suffer from lack of global coverage and/or even lower transaction limits than Google. We are using the open source geonames database for an increasing number of things, which I'll talk about more in a future post, but that won't do address level geocoding and points of interest are more limited than in Google (currently at least).

Anyway, on to the main point of the post :) ! I tested quite a variety of addresses and points of interest on both services. Some of these were fairly random, a number were addresses which specific users had reported to us as causing problems in their use of whereyougonnabe. In almost all the specific problem cases, we found that the issue was with the geocoding API rather than local search. The following output shows a sample of our test cases (mainly those where one or other had some sort of problem):

I=Input, GC=geocoding API result, LS=local search API result, C=comments

I: 1792 Wynkoop St, Denver
GC: 1792 Wynkoop St, Denver, CO 80202, USA
LS: 1792 Wynkoop St, Denver, CO 80202
C: LS did not include country in the address string returned, but it was included separately in the country field. The postcode field was not set in LS, even though it appeared in the address string. Other fields including the city and state (called "region") were broken out correctly. This was typical with other US addresses - LS did not set the zip code / postal code, but otherwise generally broke out the address components correctly.

I: 42 Latimer Road, Cropston
GC: 42 Latimer Rd, Cropston, Leicestershire, LE7 7, UK
LS: null
C: The main case where GC fared better was with relatively incomplete addresses, like this one with the country and nearest large town omitted (Cropston is a very small village)

I: 72 Latimer Road, Cropston, England
GC: 72 Latimer Rd, Cropston, Leicestershire, LE7 7, UK
LS: 72 Latimer Rd, Cropston, Leicester, UK
C: Both worked in this case, with slight variations. GC included the postal code (a low accuracy version) and LS did not. LS included the local larger town Leicester which is technically part of the mailing address.

I: 8225 Sage Hill Rd, St Francisville, LA 70775
GC: null
LS: 8225 Sage Hill Rd, St Francisville, LA 70775
C: One of several examples from our users which didn't work in GC but did in LS.

I: Wynkoop Brewing Company Denver
GC: null
LS: 1634 18th St, Denver, CO
C: As expected, points of interest like this do not get found by GC

I: Kismet, NY
GC: Kismet Ct, Ridge, NY 11961, USA
LS: Kismet, Islip, NY
C: Another real example from a user, where GC returned an incorrect location about 45 miles away from the correct location, which was found by LS.

I: 1111 West Georgia Street, Vancouver, BC, Canada
GC: 1111 E Georgia St, Vancouver, BC, Canada
LS: 1111 W Georgia St, Vancouver, BC, Canada
C: Another real example from a user which GC gets wrong - strangely it switches the street from W Georgia St to E Georgia St, which moves the location about 2 miles from where it should be.

I: London E18
GC: London, Alser Strafle 63a, 1080 Josefstadt, Wien, Austria
LS: London E18, UK
C: Another real user example. London E18 is a common way of denoting an area of London (the E18 is the high level portion of the postcode). GC gets it completely wrong, relocating the user to Austria, but LS gets it right.

So in summary, in our test cases we found a lot more addresses that could not be found or were incorrectly located by the Google Geocoding API, but were correctly located by Google Local Search, than vice versa. It is hard to draw firm conclusions without doing much larger scale tests, and it is possible that there is a bias in our problem cases as our existing application may have tended to make more problems visible from the Geocoding API than from Local Search (it is hard to tell whether this is the case or not). But nevertheless, based on these tests we feel much more comfortable going with Local Search rather than the Geocoding API, especially given its compelling benefits in locating points of interest by name rather than address. These point of interest searches can also take a search point, which is also a very useful feature for us, which I will save discussion of for a future post. Advantages of the geocoding API are that it returns an accuracy indicator and generally returns the zip/postal code also, neither of which are true for Local Search. Neither of these were critical issues for us, though we would really like to have the accuracy indicator in local search also. For our application we are not too concerned about precisely how the addresses align with the base maps - one reason I have heard given in passing for Google having these two separate datasets is that they want a geocoding dataset which uses the same base data as Google Maps uses, to try to ensure that locations returned by geocoding align as well as possible with the maps. If this is a high priority for your application, you might want to test this aspect in more detail. It also appears that the transaction limits are more flexible for server based geocoding with Local Search.

So we'll be switching to an approach which uses only Google Local Search in an upcoming release of whereyougonnabe. I'll report back if anything happens to change our approach, and I'll also talk more about what we're doing with geonames relating to geocoding in a future post.

Nice site on "walkability"

Apologies for the lack of posting recently ... a combination of being heads down on whereyougonnabe together with a couple of other projects, plus fitting in some vacation time over in Europe (I highly recommend visiting Porto). We will have a significant upgrade to whereyougonnabe coming very shortly, but I'll save that for another post.

But in the mean time, I noticed this post from Paul Ramsey on how glad he was to live in a walkable neighborhood, which reminded me of a nice site called Walk Score which I've been meaning to post about. It ranks where you live based on the amenities you can walk to (based on the Google Local Search API), and like Paul I'm glad to report that I am in a good spot - my loft in downtown Denver scores 95 points out of 100.

The site seems to have reasonable international coverage too - I tried it for a few locations I know in Canada and the UK, and it gave good results for both. They even kindly provide results for various "celebrity locations" - Bill Gates' home only scores 11 out of 100, for example, while President Bush's ranch in Texas scores 0! Anyway, I thought it was a nicely done and interesting site.

New release of whereyougonnabe

Well, we've been busy in the three weeks since we released our first beta of whereyougonnabe, and today we put out some significant improvements, including support for the Safari web browser, email notification when we find close interactions with your friends, a cool Facebook profile widget, the ability to easily re-use previous locations, and a number of other usability improvements and bug fixes.

This is what the new profile widget looks like:


A major focus for us is making it very easy to create activities, and we have added the ability to dynamically select previously created locations. This is a screen shot showing how this looks on the create activity screen:

We also took some of Stefan's comments to heart, and have made the activity title optional, so you can just say "Peter will be in Boulder" rather than "Peter will be working in Boulder". As well as giving you more flexibility this makes it faster to create activities, as the minimal things you need to specify now are a location, plus start date and end date (which both default to today) - but you have options to enter various other details if you want to. We also fixed an issue with handling accented characters, and made some improvements to our icons, fixing some aspect ratio issues and adding a default icon for people who don't have a picture on Facebook.

We will now email you when we find that you are going to be close to some of your friends, which is just the first step in providing a range of notification capabilities.

We have a lot more things in the pipeline, including further Facebook integration (support for the newsfeed and minifeed coming very soon, plus the ability to set your Facebook status when an activity starts), our first steps in calendar integration, and a simplified interface suitable for mobile browsers.

If you haven't signed up yet or you want to check out the new features, you can do so here. And we welcome your feedback on our new forums (or feel free to email me).

Coming out of stealth mode soon!

We have been making good progress on the application we are building at Spatial Networking, which in its initial release will run on the Facebook Platform. We plan to get an alpha release out to a small number of testers by the end of this week, and have a closed beta out within a few weeks after that. So I've decided it's time to start telling the world more about what we're up to, and I'll be doing this, and giving our first public demo, at next week's Boulder Denver New Technology Meetup Group, which is on Tuesday February 5 at 6pm in Boulder. I have found this to be a pretty interesting local group for anyone interested in technology startups. They had about 200 people at the last one. The general format is that 5 companies present for 5 minutes each (with 5 minutes for questions). So I'll have to be pretty concise, but I've been getting used to that with doing pitches to potential investors, who have pretty short attention spans :). If you're in the area please feel free to come to the Meetup, and if not I'll be posting much more information here in the coming weeks.

iPhone uses WiFi as well as cell signals to determine location

I just went out for a walk around my neighborhood in downtown Denver to test out the new "locate me" capability on my iPhone, and discovered something rather interesting: apparently the system is using not just cell phone signals but also WiFi to determine location, which explains the unexpectedly good first test result I mentioned in my previous post. I discovered this somewhat accidentally, but turning WiFi on and off on the phone made a big difference to accuracy. You do not have to be connected to a WiFi network in order to use WiFi for location determination.

Since I am in a WiFi-rich downtown location, using WiFi gives much more accurate results than triangulating cell phone signals. From a quick walk around the neighborhood here, I am consistently getting within half a block, or perhaps about 50 meters accuracy most of the time, and as good as 10-20m in some instances, when I have WiFi turned on. Of course it won't be nearly that good if you don't have a dense WiFi network, but this is much better than I expected. If I turned off the WiFi on the phone, the results were less accurate. It is a little hard to gauge the accuracy as the map display is zoomed out depending on the level of accuracy that the system thinks it can obtain (and there is a variable size circle displayed). But generally speaking, even without WiFi, the location generally seems to be within a block or two (say 100-200 meters) of the right location - though again this is in a downtown location with a high density of cell towers, so in a rural location it will be significantly less accurate.

If you look at Google's explanation of how "My Location" works, they only mention using cell tower location. I suppose that most of the other phones which have previously run Google Maps Mobile do not support WiFi, so this makes sense. Other vendors such as Navizon have used WiFi location for a while, and indeed they announced a version for the iPhone a little while ago (if you had hacked your phone, since of course it is still officially closed to third party applications at this point). But in general I haven't seen anything saying that Google was using WiFi for location determination.

I did find that the accuracy varied a fair amount from one attempt to another - I could stand on the same street corner and locate myself multiple times in quick succession, and get three different map scales (representing different accuracy levels) back. The true geo-geeks among you can check out this flickr album, which shows photos of the location given by my iPhone as I walked around the neighborhood, and each one is geotagged using my Garmin handheld, which gives a pretty accurate location. I have added various notes as I went through. On an individual photo you can click the map link to get an accurate view of where the photo was taken (you'll need to zoom in a little), and you can also see a map of the whole set.

Anyway, in summary, I am very pleasantly surprised with the performance of the location capabilities so far - the use of WiFi gives a much higher accuracy than I had expected. Not good enough for in-car navigation applications, but definitely good for local search applications.

Update on my original post: Slashgeo reports that the WiFi positioning technology used in the iPhone is from Skyhook Wireless. One other comment I forgot to make before is that the location determination is pretty quick - typically 3-5 seconds for the map to refresh with the location, most of which is probably just the time to redisplay the map using the EDGE network.