Ubisense looking for Senior Project Manager and Technical Manager

Just a quick post to mention that Ubisense is looking for a couple of people to join our geospatial team in the US. We've been growing rapidly and have a lot of exciting things going on. First is a Senior Project Manager.  Ideally we'd like someone with experience managing complex geospatial projects in utilities and/or telecommunications companies, and preferably but not necessarily experience with GE Smallworld. This would ideally be in Denver but location is somewhat flexible - a fair amount of travel to customer sites would be involved with this role.

We're also looking for a Technical Manager based in Camden, Maine, to manage development of some innovative mobile products for the gas industry that we're working on there (this group was formerly part of InMaps, who we acquired recently).

For more details check our web site (second item down, and last item, currently).

Crowdsourcing my State of the Map keynote talk

A couple of months ago I gave a talk at the monthly OpenStreetMap meetup in Denver and I decided to try crowdsourcing the content (in the best OpenStreetMap tradition), and it worked out very well. I received a good amount of interesting content from several folks (thanks everyone!). I rashly agreed to do a keynote at the upcoming State of the Map conference in Denver (rashly since I'm chairing the FOSS4G conference right after that, and am more than a little busy with that, not to mention my day job :O !!). So I thought I would do the same thing again and appeal for ideas on things I should include in a tour of what's new and cool in OpenStreetMap.

You can send me links to stories, slides if you have them, or anything else that you think would be useful. If you did a presentation on something cool and interesting at SotM EU but can't make it to Denver, I'd be happy to mention it over here. And even if you are presenting something over here, I'm happy to include a slide or two as a "trailer" for your talk. Of course I can't promise to include everything, depending on how much material I end up with (and I do have some of my own of course!), but will do my best.

In general I'm interested in hearing about things that you think are (reasonably) new and interesting in the OpenStreetMap world including:

• Cool applications using OpenStreetMap data 
• New (or improved) tools for creating / editing OpenStreetMap data 
• Examples of businesses or government organizations using OpenStreetMap 
• Anything else you think is interesting! 

Please just drop me an email, or comment below. I will give credit to all contributors, of course!

How FOSS4G had a profound impact on my geospatial technology path

As some of you will know, the FOSS4G 2011 conference is coming up in Denver and I am the conference chair. I have only been to one previous FOSS4G, which was in Victoria in Canada in 2007. That event had a profound impact on my perspective on the geospatial industry, and on the software platforms I've chosen to implement geospatial applications on since then. And it has saved my companies a lot of money! So I wanted to share some perspectives about my experience at FOSS4G and what I've learned about open source software, to explain why you should come to FOSS4G, especially if you've never been before.

Back in 2007 I had just left my job as CTO of Intergraph and was weighing up what to do next, and looking at ideas for a geospatial startup company. I'd spent 20 years in the geospatial industry working with closed source products, and knew very little about open source geospatial products. Towards the end of my time at Intergraph, I'd been getting quite a few questions from sales guys, in Canada in particular, about the fact that their customers were showing interest in open source software, which was free, and how should they sell against that?  So I'd done a bit of research and had become interested in particular in PostGIS, the open source spatial database, as a possible platform for applications I was looking at in my new (yet to be created) startup, Spatial Networking. The fact that it was free was obviously attractive to a new startup owner, especially as I was looking at a system that (I hoped) would need to be deployed on many servers to cope with large numbers of users.

Open source software tends not to have such flashy marketing material as closed source software, so after a bit of digging around online and not finding all the information I was after, I got in touch with Paul Ramsey, who is one of the main people behind PostGIS. He suggested I should come up to FOSS4G to find out more. I did and was really impressed by the whole experience, both the event in general and what I found out about PostGIS - you can read my writeup at the time here. As I said in that writeup, there was much more energy and buzz than I had seen at other geospatial conferences I was used to attending.

I went ahead and used PostGIS at Spatial Networking and on other projects, including my current project Ubisense myWorld. I continue to be very impressed with PostGIS - it does all the core things you expect a spatial database to do (I had a lot of previous experience with Oracle Spatial and other systems), and it's FREE! I have never hit a bug during my time using it.

So what have I learned about open source geospatial software in those four years? First of all let me say that I have no strong predisposition to open or closed source development approaches, per se. I am happy to chose either open or closed source products depending on what I need in a given situation. What I do have s strong predisposition to is FREE. Obviously a product needs to meet your requirements, but assuming it does then free is rather attractive compared to having to pay for something. This is especially true in a cloud environment, where you may scale up to running many servers, and traditional per server licensing costs can really hurt you financially.

People in the closed source world often raise concerns about support in the open source world. My experience hasn't borne out this concern. With PostGIS, I've never needed support, it just works. With MapFish, another open source product we're using for myWorld, we needed a few enhancements. Some were addressed by the community within a month or two, others we were able to do ourselves as we had access to the product source code. I very much doubt that we could have got enhancements made in a mature closed source product in that timeframe. There are also more and more options in the open source geospatial world to pay people to do enhancements or fixes for you. Again it's a bit dangerous to make generalizations, you can get good support or poor support on different closed source products, and you can get good support or poor support on different open source products. But my experience with the open source products we've chosen has been very good.

I think that after price, perhaps the aspect of open source that I value most is longevity and predictability. Many times during my career, I have seen projects suffer because a vendor has decided to stop development of a product (or feature). I have also seen dramatic changes in terms of service or costs of online services. Google App Engine is one example of the latter - many people put significant effort into developing applications that were running for free, then Google changed the pricing model and people found themselves facing large costs they hadn't planned on. With an open source product that has a strong community behind it, there is much more long term stability. You know it's not going to go away tomorrow. Even if some developers leave, others are there to cover for them. And in the worst case you have access to the source code so could continue to maintain it yourself (though that's a very unlikely scenario as long as the community of developers has a certain critical mass). I have been moving more components of myWorld towards open source because of this predictability.

So anyway, if you are still paying for geospatial software you owe it to yourself and your company to come to FOSS4G and find out what all this open source software is about. There is a parallel universe out there with software products that have great capabilities and are FREE! It's also worth saying that in general there is good interoperability between open and closed source systems, so it's not an all or nothing proposition. In general open source web and database products are very strong, and they may well be able to complement your investment in existing applications.

This is the first time that FOSS4G has been in North America for 4 years, and it is not likely to be here again for another 3, so this is a rare opportunity if you are based here to meet a wide range of people developing and using these products. So I hope to see you in Denver in September at FOSS4G!

To finish up, check out this video featuring members of the organizing team talking about why you should be there:

FOSS4G 2011 Denver from Tyler Mitchell @ OSGeo on Vimeo.

Speaking at OpenStreetMap Meetup in Denver tomorrow

As mentioned previously, I'll be speaking at tomorrow's OpenStreetMap meetup in Denver tomorrow, at the cool new MapQuest offices downtown. My attempt to crowdsource the presentation got a great response, and I got lots of suggestions for interesting content. I plan to have something for everyone, along the following lines:

• A quick intro to OpenStreetMap for any newcomers
• Some tips on using the Potlatch 2 map editor, which provides some cool new features - this should be good for both newcomers and experienced mappers
• Examples of how government agencies from various parts of the world are using OpenStreetMap
• Cool new tools for developers using OpenStreetMap, like Leaflet and Kothic
• Some interesting applications using OpenStreetMap
• Last but not least, how OpenStreetMap was used to show that you can in fact walk across Dublin without passing a pub (pubs have always been a mainstay of OpenStreetMap!)

We'll also have some discussion about the state of OpenStreetMap in Denver and Colorado, and brainstorm on things that we might want to focus on before State of the Map and FOSS4G are in Denver in September! You can get full details on the meetup and sign up here.

Looking for content for "State of OpenStreetMap" presentation

I'm doing a couple of upcoming presentations on OpenStreetMap, the first one next week at the very cool MapQuest office in downtown Denver, so I encourage you to come along to that if you're in the neighborhood ... and this may well evolve into a presentation for State of the Map in Denver too!

So in the best OpenStreetMap tradition I thought I'd try a little crowdsourcing to help me pull this together. I'd be interested if you could send me links (or other info) about things that you think are (reasonably) new and interesting in the OpenStreetMap world including:

• Cool applications using OpenStreetMap data
• New (or improved) tools for creating / editing OpenStreetMap data
• Examples of businesses or government organizations using OpenStreetMap
• Anything else you think is interesting!

Please just drop me an email, or comment below. In fact if anyone has presentation slides covering interesting stuff that would be relevant, those would be great too. Happy to give credit to all contributors, of course!

A couple of discussions on location privacy and the iPhone

So my recent posts analyzing the iPhone location data log have gained a lot of traffic and attention over the past few days, from places including the Toronto Star, the Wall Street Journal, TUAW, PC World, MacDailyNews, Apfeltalk (German), Cisco, Pete Warden at O'Reilly, Business Insider, and more.

This led to me being invited to participate in a discussion on the Brian Lehrer Show yesterday on radio WNYC, the NPR affiliate in New York, together with Jennifer Valentino-Devries of the Wall Street Journal. We had a good sensible discussion, in contrast with a lot of the hysterical reporting that has been going on.

Today I will be taking part in a longer discussion on the KQED Forum discussion show in San Francisco. Also participating will be Congresswoman Jackie Speier, Jim Dempsey from the Center for Technology and Democracy, and Kara Swisher of All Things Digital. This will be at 10am mountain time, there should be a recording here sometime after that.

Apple issues Q&A on "Locationgate", and addresses key issues

Apple rather belatedly issued a Q&A on the whole "LocationGate" saga. This confirms what I said about the data being a cache of cell tower and wifi locations. The fact that this was kept for up to a year was a bug. Within the next few weeks they will reduce this to 7 days, they will not back up the cache any longer, and they will turn off the cache when you turn location services off, which addresses the issue reported by the Wall Street Journal and widely re-reported.. These are all good actions to take, and address the key issues in my opinion. It does reinforce the importance of developers being careful about location security, and Apple was slack in this case, even though the potential risks were much less dire than widely reported.

Note that in the short term if you are concerned, you can encrypt your iPhone database backup just by checking a box on the front page in iTunes (after plugging in your iPhone). If you do this, the current location log cannot be accessed by someone who hacks into your computer.

The scoop: Apple's iPhone is NOT storing your accurate location, and NOT storing history

The Summary
So in my previous two posts I discussed how the data I was seeing in my iPhone location logs was actually not very accurate, and certainly didn't reveal where I lived or worked or had stayed on my travels - beyond showing the cities I had been to, including general areas I had visited, as well as some I hadn't. There had been some discussion that the data appeared to be, in a number of cases, the location of cell towers you had been in communication with, although in some cases locations were a long way from where you had been.

The quick summary: I believe I have confirmed that Apple is not storing your location, but the (actual or estimated) location of cell towers (and WiFi access points) that are close to you, to help locate you as you move (these are not necessarily towers that you have been in communication with). In the data I have examined there is nothing that is based on the accurate location of the iPhone. For a good example, see my previous post showing the location of cell equipment in Coors Field baseball stadium, and not revealing the location of my home which is very close to there. In my opinion, if Apple was storing this data in order to know where you had been, they would be storing different, more accurate location data that they have access to.

And, importantly, they are not storing history - the only thing that can be found from the files is when you last visited a general area, not if you made repeat visits. This is especially important as it means that many of the concerns expressed about this data are simply not valid: it cannot be used to determine where you live, or work, or go to school, or who your doctor is.

Here is a report of what Al Franken said:

Sen. Al Franken, a Minnesota Democrat, said it raises “serious privacy concerns,” especially for children using the devices, because “anyone who gains access to this single file could likely determine the location of a user’s home, the businesses he frequents, the doctors he visits, the schools his children attend and the trips he has taken — over the past months or even a year.”

The only part of this that is correct is that the data will show what cities you've visited, with some indication of which parts of a city you may have visited, though nothing definite - there will be records in areas you didn't visit. And it doesn't show repeated visits to the same location, only the last one.

Update: see below for a very interesting comment from "Anonymous", who includes a link to a document submitted by Apple to Congress in July 2010. This includes the following:

"When a customer requests current location information ... Apple will retrieve known locations for nearby cell towers and Wi-Fi access points from its proprietary database and transmit the data back to the device" ... "The device uses the information, along with GPS coordinates (if available), to determine its actual location. Information about the device's location is not transmitted to Apple, Skyhook or Google. Nor is it transmitted to any third-party application provider, unless the customer expressly consents". 

The data under discussion in this whole debate is clearly (in my opinion) a cache of the data mentioned here of nearby cell towers and Wi-Fi access points. I guess the remaining valid concern is that this cache is not stored as securely as it could be, and a fairly large amount of data is stored in the cache. But still this data provides only relatively coarse information as discussed here, and is stored only on the user's own computer, so the risks are relatively minor compared to many of the more dramatic scenarios that have been raised.

Update April 27: Apple has issued a Q&A document about all this, which confirms the conclusions I had drawn, and talks about changes they will make. See my thoughts here.

Read on to find out how I reached these conclusions.

The details
Last night someone called Jude commented on my last post, saying:

My Guess?

It's not a list of cell phone locations that you've been to, but the opposite, a list of cell phone locations near you downloaded to the iPhone from Apple in case you move into range of one of them. i.e. At a guess what is happening is location services identifies a cell tower and asks for its location, and is replied to with the list of locations that contains that cell tower, that list is then cached so that it does not need to be requested again.

Of course, this is only a guess based on the wide range of addresses people are seeing and how its near to, but not exactly where, the people have traveled.

Good thinking Jude! I thought this could explain a lot, so I investigated further. First I looked at some data from my fairly recent New York trip. I looked at the timestamps on some locations and did a query to display all the locations with the same timestamp. I found out that in general, quite a number of records shared the same timestamp, and they would be clustered in the same area. For example, this screen shot shows a set of records that were all loaded at exactly the same time:

This cluster of points is some way above where I drove, I was driving along the Long Island Expressway going east from LaGuardia Airport. The timestamp appears to be in seconds and has 7 decimal places, so it is apparent that this set of data must have been downloaded in a single transaction, it was not obtained by communicating with cell towers at each of these locations independently. It seems reasonable to assume that this data was downloaded to help locate me in the event that I drove into this area (which I didn't). You can observe similar clusters by clicking a dot at random, copying the timestamp, and running a filter in Google Fusion Tables to display all dots with the same timestamp.

What I really wanted to do now was to animate my data, to more easily visualize what was happening. I couldn't figure out an easy way to do this in Google Fusion tables - although it has some capability for this, it wasn't recognizing the timestamp field as a date-time. So I went to look at the data that Sean Gorman had posted of his logs at GeoCommons (my original file had been too large to visualize there without me doing a little more work). GeoCommons has a cool animation capability, which you can try out on Sean's map by dragging the sliders at the bottom left.

I found something really interesting when I zoomed in around the geoIQ office in Arlington, where Sean works. This screen shot shows that between November 11, 2010 and April 20, 2011, there is no record of Sean being at his office.

Now I know that Sean likes to escape for a spot of skiing in Colorado now and then, but that's a pretty long absence for a company President :) ! And I know I have met with him in the office during that time period.

If you drag the time slider a little further, then at the same instant, about 20 more locations appear on the map, covering a general area around the office, roughly half a mile square:

So from this data I can tell that Sean was somewhere in the general area of this half mile square (not necessarily inside it) on April 20. I know nothing about whether he was there before that, and I don't know anything about exactly where he went.

So, this data stored in the iPhone logs is much less revealing than it may initially seem. At a quick glance it does look like it is recording your location history, and I think that Pete Warden and Alasdair Allan were quite right to raise the concerns that they did. It takes some digging in the data to realize that the concerns are not nearly as bad as they appeared at first sight. By publicizing it as they did, and providing their tools and documentation on how to examine the data, they made it easy for others like myself, Sean Gorman and Will Clarke to analyze the data and figure out more about what is going on.

It's still not clear exactly what the data is for, but my guess, as Jude suggested, is that it is to aid in fast location determination - once the iPhone figures out that you're in an area, it downloads data for surrounding cell towers (and Wifi hotspots, a detail I haven't gone into here but the data is available for those too, as discussed in my previous post), so it can quickly locate you as you move around that area (update: see the first comment below, and my addition to the initial summary, which reference a document from Apple that confirms that this is the case).

So to summarize again, there are still some concerns with this data - it does give an approximate indication of places you've been, but not good enough to identify specific buildings or businesses. It doesn't record history - there is no way to tell if you've visited a location multiple times, you can just tell the last time you visited a general area (though there might be clues about multiple visits - for example data showing you visited a neighboring area on a different date, but nothing definitive or detailed about repeat visits). But it definitely doesn't reveal the sort of detailed information that many people have been concerned about.

More on Apple recording your iPhone location history

In my previous post I discussed how the location data being recorded from my iPhone actually wasn't very accurate, and certainly not accurate enough to tell where I live or work (based on the data I've examined so far, which is in a table called CellLocation in the iPhone backup, and is the data discussed by Pete Warden and displayed by his iPhoneTracker app, which is what I used for the visualizations in my previous post). Pete's app aggregated data to a regular grid, partly to provide additional security.

However, I was sufficiently intrigued to follow Pete's instructions to get at the raw data. My investigations with this reinforced the conclusion of the previous post, that the data does not accurately represent your location. But it did show up some interesting new patterns. I loaded the data into Google Fusion Tables and have made it public, you can view it here (and feel free to play around with it).

Here is an interesting map of downtown Denver, where I live.

This shows all the raw point data, with no aggregation or changes. There are actually no dots at all in the block where I live. However, there is a noticeable cluster in Coors Field, the Colorado Rockies baseball stadium which is 3 blocks away from where I live. I haven't been to the Rockies stadium over the time period that this data was recorded. There's also a strong cluster in Mile High Stadium, home of the Denver Broncos.

I would assume that there is additional cell phone infrastructure in these stadiums, to help cope with the heavy concentration of people. A quick Google search found this article about AT&T infrastructure at Coors Field

This reinforces the notion that at least some of these locations are the locations of cell equipment that your phone is communicating with. But I'm not sure that's the whole story.

Here's a map of Cropston, where I spent most of my time on my last two visits to England. It's a small village in a fairly rural area.

Here there are no locations shown in the village itself where I spent most of my time. A lot of the locations are clustered in towns or along streets, but some seem to be more in the middle of nowhere. Hard to draw any definite conclusions.

I just received a suggestion from Jonathan Barnes, via Pete Warden, that the HorizontalAccuracy field may be significant, with lower values indicated accurate locations via GPS. However, I did a quick test, for example this map filters to only show records with this field set to 500.0, the minimum value I found from a quick skim (Fusion tables seems to treat this as a string rather than a number), and while this reduces the number of records it doesn't offer any noticeable change in accuracy - it still includes all the readings from Coors Field and Mile High Stadium where I haven't actually been.

Pete also pointed me at another table in the backup called WifiLocation, which in my case was about 5 times larger than CellLocation. I have loaded this to Fusion Tables here. One interesting thing about this table is that data only shows up in North America (with one random exception in Munich, where I haven't been recently). It seems a little more focused on areas I've been to, but no more revealing in terms of showing specific locations where I've spent time.

As I said, feel free to play with the tables I uploaded, and let me know if you find anything interesting! But my conclusion remains that this data doesn't reveal where you've been with any degree of accuracy.

Update: see my latest post where I conclude that this is data being downloaded from Apple, rather than uploaded, and that detailed history is not stored - thanks to Jude in the comment below for his suggestion about this.

So actually, Apple isn't recording your (accurate) iPhone location

So over the past couple of days there has been mass hysteria, questions in Congress, etc, over the fact that Apple is apparently recording all the locations you've been to with your iPhone without telling you, and storing it without encryption. The news was broken by my friend Pete Warden at Where 2.0 last week and has escalated rapidly since then. As someone who publishes their location anyway (you can see where I am right now by checking the right hand panel on my blog) I was less concerned about this than many, though I agree that Apple should make it clear that they are recording this information and give you the option to turn it off, plus it should be stored more securely.

However, yesterday Sean Gorman posted that he had analyzed his data, and the interesting thing is that it wasn't accurate - it showed the general areas he'd been to, but didn't reveal where he lived or where he worked. And then I also found this post by Will Clarke, followed by this one, which also conclude that whatever the data is, it isn't your accurate location (though I think Will prematurely concludes that it is cell tower locations - Sean's analysis suggests that isn't the case, though it seems it may well be related to this).

I just had a good chat on the phone with Pete about these posts, and about my findings which I'll get onto in a moment, which similarly conclude that whatever is being tracked, it isn't your accurate location. Pete said that their conclusions were similar, but also that he didn't think it was simply cell towers. I know that my iPhone knows my location much more accurately than the locations that I see in the data I've looked at. For me, as for Sean, there was no cluster of points either at my home or my office. Pete asked me if I'm on WiFi rather than 3G at home and at work, and the answer is yes, so there may be some clue there.

But the main point of these posts, and mine, is that this data does NOT indicate where you live, where you work or any exact locations you've been to. This is not reflected in most of the reporting you see about the topic.

I thought I'd share some screen shots of maps that I got, which I actually thought were cool :). Since I travel quite a bit, I have a few interesting examples which might give some clues as to what this location data actually does represent. The detailed (larger scale) maps here show a grid of dots, which is something introduced by Pete's map display tool rather than how the underlying data is. I will try to play around a bit more to get at the raw data, but thought I would share these initial findings first.

So to start with, here's an overview of my world travels of the past few months, which seems pretty accurate, and goes back to at least September:


Here's a view zoomed in on the US. The interesting thing here is that New York has the largest bubble over it, but I only spent two days there on a recent trip (1 day in Manhattan, 1 day on Long Island). Denver where I live has a much smaller blob.


Here's a map of Colorado - there seem to be quite a few outliers here on the south side of the map - I think that the closest I've been to these in recent months is Keystone, where you see a cluster of dots. Some of these dots are probably 50 miles away from where I was.


Zooming in on Denver, you see a lot of activity. I'm sure I haven't covered Denver quite as comprehensively as the dots here suggest.


In this map of downtown Denver you see the gridding which somewhat obscures the underlying data. However, the largest dot is some way away from my home (which as I think everyone knows is above the famous Wynkoop Brewing Company), and the dots are fairly evenly spread - these certainly do not indicate where I spend most of my time downtown.


Similarly, my office (where I usually work a couple of days a week, the other days I work at home) does not jump out on this map of the Denver Tech Center (as you can find out from our web site, the Ubisense office is at 5445 DTC Parkway).


On to my UK travels - I think the data includes two trips there. There seem to be quite a few outliers here also, and some fairly large clusters in places I just passed through on the train. I spent most of my time on these trips at my mother's house in Cropston, just north of Leicester, which isn't reflected in the data. I spent some time in London, but it has a disproportionate representation on the map (as New York did in the US map).


Zooming in to the Leicester area, you can see Cropston just to the north of the city, which is where I spent nearly all my time, and this has no readings. I didn't travel around Leicester nearly as much as the dots would suggest. So this map is very misleading in terms of where I spent my time in this area.


This map of Zurich is interesting: I connected through Zurich airport in November, en route to Denmark. I spent maybe 3 hours in the airport and didn't leave it, but you can see lots of outliers, which are up to about 20 miles away.


Here's a map of my trip to Denmark, where I spent time in Copenhagen, Aarhus and Naestved. The interesting thing on this one is that I just drove straight across the island of Funen (Fyn) in the middle of the map, but you can see quite a scatter of readings on either side of the road, especially to the south on the east side of the island.


Almost at the end ... I included this map of Paris as I thought it was interesting that we traveled from London to Paris and back on the Eurostar train, but no points show up along the route. There's an odd horizontal line of locations to the north of Paris, but nothing apart from that between Paris and London.


And finally an example from Sydney. This shows a disproportionate number of readings at the airport in the south, where I just arrived and left but didn't spend any time. It doesn't show that I spent a good amount of time downtown, and I gave a talk in Paramatta where there is just one isolated dot. I stayed with friends north of Sydney but again you can't tell where.


While I don't want to be an apologist for Apple, and what they are doing here is careless at best, my general conclusion is that this is likely something unintentional, similar to the Google Street View WiFi data fiasco. If Apple wanted to track your location history, why wouldn't they use your accurate location, which I know my phone knows much more accurately than is shown in the data in these files.

The interesting question for us geo-geeks is exactly what the location data is - something related to cell towers seems plausible. I will try to poke around in the raw data a little more. Since I have a few interesting example cases, am happy to share my data if anyone wants to look at it. Pete just tweeted that there is another table with WiFi locations, that would be an interesting thing to explore.

Update: I've done a new post which includes maps with the raw data, using Google Fusion tables. Doesn't change the conclusion that the data doesn't accurately represent your actual location, but does show some interesting new patterns.

So long to the GITA "annual conference" and thanks for the memories

I just wrapped up the closing panel at the 34th and last GITA "annual conference" (officially known as the geospatial solutions conference these days), which was quite a sad moment for me. I attended my first GITA (then AM/FM, Automated Mapping and Facilities Management) conference in 1992, and have only missed one since then. Especially back in the 1990s, and into the early 2000s, it was always the highlight of the year in the part of the geospatial industry that I worked in, focused on organizations managing infrastructure like gas and electric utilities, telcos, water and waste water and local government. I should say before going too far that this does not mean that GITA the organization is going away. They already do a lot more than just the annual conference, and are actively looking at various new opportunities - I'll come back to this below.

Above is a portion of the Smallworld team that attended the 1993 conference, which is where we launched Smallworld in the US, and I moved over here from the UK later that year. From left to right you see Sean Newell, Jay Cadman, Ali Newell and me. Somehow we retrieved various signs at the end of the conference, in order to illustrate our predictions for the GIS industry :). In the late 90s attendance would be over 3000 people I think. It was the place to go for organizations selecting new GIS products. I remember sometime in the late 90s we upped the ante at Smallworld by giving away a car at the show (a new Volkswagen beetle) ... will have to try to dig out some of the (pre-digital) pics of that! And I gave many presentations at GITA over the years.

But since 2005 the GITA "annual conference" (under a couple of different names) has had declining attendance, and so GITA Executive Director Bob Samborski and the GITA Board, which I rejoined this year after a few years off it, reluctantly concluded that the right thing to do was to make this the last annual conference in its current form (a decision I agree with). Here's a picture of me and Bob with then Denver Mayor, now Colorado Governor, John Hickenlooper at the GIS in the Rockies conference in 2004...



GITA has always been associated with the annual conference which was historically its biggest event, but for quite some time now this has just been a part of its activities. It also runs the GIS for Oil and Gas conference, now in its 20th year, which remains strong and will continue. And international affiliates such as GITA Australia and New Zealand continue to be strong. This year it is also playing a leading role in organizing the FOSS4G conference, for which I am conference chair. It organizes a successful program called GECCo, Geospatially Enabling Community Collaboration, to encourage collaboration between infrastructure organizations, with a particular focus on emergency response. GITA recently received a grant from the Department of Homeland Security to expand the GECCo program and organize quarterly workshops over the next 3 years. GITA has an active network of regional chapters that organize local events, and those will likely receive more focus now. Also I think there will be a stronger focus on delivery of educational material via various online channels. And there are a number of other ideas under consideration for new events and alliances with other industry organizations.

So GITA will continue, but with a somewhat modified focus. Things are changing rapidly in the geospatial world and I agree it's the right decision, but it's still a bit sad for those of us who have had such a long association with the annual conference. Everyone involved can be proud of what the GITA conference has contributed to the growth of the industry over the last 34 years! (Yes, probably longer than a lot of you "neo" readers have been alive!).

Pete Warden's Data Science Toolkit offers cool geo capabilities

I just had an interesting chat with Pete Warden, a fellow Brit who was living in Boulder for a while and is now out in San Francisco, and who has worked on various interesting development projects including quite a bit of geo stuff. He is most famous for his cool map of Facebook users, which led to Facebook threatening to sue him :( !!


He has just launched a new project called the Data Science Toolkit at the GigaOM Structure Big Data Conference.

Watch live streaming video from gigaombigdata at livestream.com

It's an open source project and contains a variety of tools for analyzing data, including several geospatial ones. Everything is nicely packaged up as an Amazon AMI, including some large databases, so you can just fire up one or more Amazon machines and use the functionality (and you can do some basic testing here).

• Geocoding (currently US only) - this uses the open source geocoder developed by geoIQ using TIGER data. The nice thing about this is that there are no transaction limits or restrictive terms associated with it. And you can run it offline if you like. Lack of good geocoding is currently a weakness of OpenStreetMap, so this is a nice complement to that. It found my home address very accurately - not an extensive test but a good start :). While this is all open source, it takes a good bit of effort to download the full TIGER database and get this all set up, so having a packaged version is a good thing.
  
• Reverse geocoding - takes a point and gives you information about where it is - country, city, district, etc. Again this is not unique, but has the same advantages as the geocoding functionality in terms of setup and lack of restrictions.
  
• GeoDict is a tool that emulates Yahoo's Placemaker and pulls location data out of unstructured English text - its API is identical to Placemaker.
• IP address to location

It's a nicely packaged collection of free and open source tools to add to your kitbag!

New open source server options for Ubisense myWorld

We have been busy working away on various aspects of Ubisense myWorld. One of the biggest enhancements is behind the scenes, with support for new server options, so that we can run in the cloud or in house.

Up to this point we’ve been working with Arc2Earth, which runs on top of Google App Engine, and both these platforms have worked very well for us, and were a great way of getting an initial system up and running quickly. We see a lot of benefits to running in the cloud, as I’ve talked about on several occasions.

However, a number of our customers, including large utilities and telecom companies, have said that they really like what we’re doing with myWorld, but they would be more comfortable with a solution where the server can run in house. So to support this we have added a new server architecture based on the open source products MapFish and PostGIS. As many of you will know, PostGIS is a very robust spatial database, built on top of PostgreSQL. I have used this on a few projects including whereyougonnabe and have always been very impressed with its functionality and performance. MapFish provides services using data from PostGIS (or from other spatial data sources, including Oracle Spatial, MySQL and Spatialite), using a very similar REST API to that used by Arc2Earth, so that made the migration straightforward and means we can support both server options with a largely common set of code. We’re just using server side components of MapFish, not its client side components (though we might consider using those in the future).

This new server code can run on various operating systems, including Linux and Windows (and Mac!). Customers can run the server in house, while we can now offer services using many different cloud infrastructure providers. We’re currently using Amazon, which has been working well, but it’s good to have alternatives available. We've continued to be pleased with PostGIS, and MapFish too based on our experience so far.

Stay tuned for more news on other cool new functionality on the front end of myWorld coming soon!

Who knew glass was so cool?

Very slick future technology vision video from glass maker Corning. Quite a few maps make an appearance. Worth watching! (Courtesy of fellow FortiusOne advisory board member Jeff Harris).

GITA Geospatial Solutions Conference 2011

If you haven't already signed up for the GITA Geospatial Solutions Conference, which is coming up in Dallas (Grapevine) from April 10-13, you should check it out. There's a lot of great content in the program, including an open source track, which is a first for GITA. I'll be speaking on that track with my colleague Jason Sanford, about some of the latest developments we've been working on with our Ubisense myWorld product. Jason will also be talking in a workshop on geo in the cloud, and I'll be speaking in the opening session together with Geoff Zeiss. I'll also be chairing the closing panel. I'm just finalizing the participants, watch this space for details, but am aiming for something which will be as interesting as the panel I chaired at the GITA conference last year on the topic of "Not your father's approach to geospatial data creation and sharing" featuring Steve Coast, Andrew Turner, Ron Lake and James Fee. This year we'll be covering a broader range of topics across the geo-universe.

GITA Panel: Not your father's approach to geodata creation and sharing from Peter Batty on Vimeo.

I'm looking forward to the conference and hope to see you there!

Cool Egypt tweet map on GeoCommons

My friend Chris Helm, who is a co-conspirator both at FortiusOne and on the organizing committee of FOSS4G, has been doing a lot of interesting work with analyzing and mapping tweets recently. He produced a cool animated map showing the location of tweets tagged with #Egypt #Jan25 or #Tahrir over time.

View full map


When you play the animation (click through to the full map and press play at the bottom left) you can clearly see the moment when Mubarak finally stood down, everything goes crazy. Diane Sawyer from ABC news featured the map on her blog too, which was very cool!

Geospatial in the cloud

As mentioned previously, earlier in the week myself, Brian Timoney and Chris Helm did a set of presentations and demos on geospatial technology in the cloud, to the Boulder Denver Geospatial Technologists group. We were aiming to give a quick taste of a variety of interesting geo-things currently happening in the cloud, and we did it as six slots of about ten minutes each, and apart from my introductory opening slot these were all demos:

• Peter: Why the cloud?
• Brian: Google Fusion Tables
• Chris: the OpenGeo stack on Amazon (PostGIS, GeoServer, OpenLayers, etc)
• Peter: Ubisense myWorld and Arc2Earth
• Chris: GeoCommons
• Peter: OpenStreetMap

We got a lot of good feedback on the session. Here's the video (for best quality click through to vimeo and watch in HD):

Geo in the cloud from Peter Batty on Vimeo.

Here are links to the demos we used, or related sites:

• Utah Oil Wells in Fusion Tables (Brian)
• Denver Elections map in Fusion Tables (Brian)
• Centennial address lookup (Brian)
• FOSS4G 2010 attendance map in Fusion Tables (Peter)
• FOSS4G 2010 attendance map in GeoCommons (Peter)
• How to run an OpenGeo Amazon instance (Chris) - blog post from OpenGeo, our demo instance is not running permanently
• Video demo of Ubisense myWorld (Peter) - public demo not currently available
• GeoCommons (Chris)
• OpenStreetMap (Peter)

And finally, here are my slides on slideshare:

Geo in the cloud

View more presentations from Peter Batty.

Google Chrome notebook review

Today I was very happy to receive out of the blue a Google Chrome notebook. It's very black and stealth-bomber-like! I've written a short review of it over on my new posterous blog, where I'll be writing about (mainly) non-geo-things, including general tech, photography, videography, travel and anything else that comes to mind!

Speaking at Boulder-Denver Geospatial Technologists meeting tomorrow

Just a quick post to say that I'll be speaking at the Boulder-Denver geospatial technologists meeting tomorrow, together with Chris Helm and Brian Timoney (and maybe Andrei Taraschuk, though he's doubtful because of other commitments). We'll be doing a pretty informal set of demos and discussions on geospatial in the cloud. Examples we'll cover will include Google Fusion Tables, GeoCommons, the OpenGeo stack (PostGIS, GeoServer, OpenLayers) running on Amazon, and Arc2Earth Cloud and Ubisense myWorld running on Google App Engine. It's at the Boulder Open Space and Mountain Parks Building at 6pm tomorrow, Jan 25.