My GeoWeb 2009 presentation

Here's a video of my GeoWeb presentation on "Building a new location-aware infrastructure for calendaring and scheduling". Recorded and edited using ScreenFlow.

GeoWeb 2009 presentation by Peter Batty from Peter Batty on Vimeo.

I'll post more thoughts on GeoWeb soon - in the mean time you can check out my tweets and the geoweb stream on twitter. And there are some pics of the social event here.

Freezing whereyougonnabe and moving on to new things

I have been trying to raise investment funds for Spatial Networking for some time now but despite some interest still haven’t closed anything, unfortunately – it’s a tough time to be raising money. After weighing up all my options, I have decided to stop new work on whereyougonnabe (for now at least) and move on to some new things. I continue to believe that we have a good idea and have developed some valuable technology, and that there will be a strong market for future location and location aware calendars at some point – and it is quite possible that the work we have done in this area will be revived somehow.

However, one of the factors in my decision is that there are also a lot of other really interesting things going on in the geospatial industry right now that I would like to get more involved in, and it’s hard to do everything :) !! I have been working on several things in parallel with whereyougonnabe over the past year or two, including my role as Chief Technical Advisor at Enspiria Solutions, being on the Advisory Board of PublicEarth (both of which are ongoing activities), and working with Netezza on the launch of their spatial capabilities last year. I also did some interesting work with the Bill and Melinda Gates Foundation last year, to help start what is now the AgCommons project.

Some of the main areas that interest me right now include:

• The disruptive transition from the previous generation of GIS to the “neogeography” systems, which is well and truly under way – I will blog more about this shortly
• The growth in free and open source geospatial software
• The huge potential of crowdsourced data, most notably OpenStreetMap
• In a somewhat different direction, the Smart Grid, which is something I have been working on at Enspiria and which will be bringing major changes to the electricity industry, an industry I have worked in a lot during my career

So I’m on the lookout for interesting opportunities in these areas (or others). I am quite interested in additional advisory or part time roles that I could pursue alongside the existing part time work I’m doing, which would give me some flexibility to dabble with some other interesting technology ideas on the side. But I’m also open to considering the right full time role. If you'd like to discuss possibilities please get in touch!

I’d like to thank my fellow developers Glen Marchesani and Nate Irwin for all the effort they have put into whereyougonnabe – and as I said, even if it goes dormant for a while, it may re-emerge in some form in the future! We are still weighing up whether to keep the current whereyougonnabe system up and running for a while, or take it down, and will notify users what we decide within the next few weeks.

I’ll be talking at GeoWeb next week about what we’ve done with whereyougonnabe, why I continue to see it as an interesting space, and some lessons learned from the whole experience.

My Ignite Where presentation on Future Location and Social Networking

Here's the (5 minute) video of my presentation last week at the Ignite Where event, part of the Where 2.0 conference. It was a great week - will be posting lots more on my impressions over the next couple of days.

Future Location and Social Networking, by Peter Batty at Where 2.0 from Peter Batty on Vimeo.

This clip is an extract from the O'Reilly video of the Ignite session (under Creative Commons license) - this starts at around 35 minutes in.

P.S. No fish in this one :) !!

New whereyougonnabe release, including support for Fire Eagle and GeoRSS

This week we added quite a number of new capabilities to whereyougonnabe, including support for Fire Eagle and GeoRSS, substantially expanded help documentation, improved synchronization capabilities, support for SMS, and exclusive use of Google Local Search in the whereyougonnabe browser application, as discussed previously (having introduced this approach for external sync in the previous release). For more detail on the new functionality from a user perspective see the whereyougonnabe blog - I'll touch on aspects of this here, but also discuss a little more of the technical background.

As most readers of this blog probably know, Fire Eagle is a service from Yahoo that is essentially an independent location broker supported by a number of applications which use location. You can sign up for free, and authorize applications to update and/or read your location. In the case of whereyougonnabe, we update your location automatically based on the plans you have told us about in advance. So for example, if you are using our Tripit interface, just by forwarding a single email with your itinerary, we will update your location throughout your trip, indicating the airports and cities you are in.

I really like our new RSS feed capability, which lets you easily view data from whereyougonnabe from outside the application, for example in web home pages like My Yahoo or iGoogle. The following is an example:



The content of the feed uses a similar algorithm to the one we use for WYTV, with a somewhat random mix of activities from your friends which will close to you, and some which won't. While a major focus of the system is of course identifying opportunities to meet friends, another important aspect is just keeping in touch with your friends. This is similar to using Facebook status or Twitter, but we have more information about an activity, including its location and time, which lets us select "interesting" items more effectively (for example, activities further from home will probably be more interesting than those close to home, in general).

You can click through on any element in the RSS feed to see more details in whereyougonnabe:



This ability to link directly to a specific activity in whereyougonnabe is new with this new release. We now use these links in various places where you see data outside whereyougonnabe, including Facebook status and newsfeeds, Twitter messages, and iCal feeds. People will only be able to see the details if they are authorized to do so (i.e. typically if they are your friend on Facebook). We see this as an important development in terms of integration with other systems, and in encouraging people to use whereyougonnabe more often.

As I mentioned above, the RSS feed is actually a GeoRSS feed, so you can display elements on a map in software which supports GeoRSS, such as Google Maps (just copy the feed URL into the search box):


Lastly, we have made various enhancements to the synchronization capabilities, including a sync on demand capability which gives you feedback on what is happening with the sync, which is helpful in confirming that it is working as expected, and/or diagnosing any problems.

There are several other functional improvements in the release too - as I mentioned, you can read more about these at the whereyougonnabe blog.

Which city is this airport in? (An exercise in using geonames)

Today I spent some time working updating on updating the airport dataset we use with whereyougonnabe. I thought that it was an interesting process to iteratively come up with a reasonable algorithm for what I needed to do, so I thought I would explain it here. I'd be interested in any feedback both on improvements to the technical approach (or entirely different approaches), and also on what the "right answer" should be from a user perspective in some of the questionable cases I mention.

As I've mentioned previously, we use Google Local Search for geocoding in general, but in our experience this really doesn't work well for geocoding airports based on the three letter airport codes. When we import travel itineraries from Tripit, the three letter airport codes are the best way to unambiguously identify which airports you will be in, so we need a reliable way of geocoding those.

We have been using the geonames dataset to do this, a rich open source database, which includes cities, airports and other points of interest from around the world. There are about 6.6 million points in the whole dataset, of which about 23000 are airports, and of those about 3500 have three letter IATA codes. Coverage seems to be fairly complete, but we have found a few airports with IATA codes missing - Victoria, BC, is one (code YYJ), and today I found that Knoxville was missing its code (TYS). I updated Victoria in the source data online, and it shows up in the online map but for some reason still does not appear in the downloaded data, I have that on my list of things to look into.

In general, the geonames airport records usually have null data in the city field. This is something I really wanted for each airport, so I can show a short high level description like "departing from Denver" or "arriving in London". Often the city name is included in the airport name, but not always - for example, the main airport in Las Vegas is just named "McCarran International Airport", which would not be an immediately obvious location to many people. And even where the city is included, the name is often longer than I want for a short description, for example "Ronald Reagan Washington National Airport".

For our first quick pass at populating the city for each airport, we used the Yahoo geocoding service, as it has a reverse geocoding function (and we thought we would try it out). You just pass this a coordinate and it gives you a city name (and other data) back. This is the data we are using in the live version of whereyougonnabe today. However, I noticed that this doesn't always give me what I would like - for example, it tells me that London Heathrow airport is in Ashford, and Vancouver airport is in Richmond. This may be technically correct (or not, I don't know for sure), but I'd really find it more useful to see the main city nearby in my high level description - so it will say I am arriving in London or Vancouver, in these two examples.

One nice aspect of the geonames dataset is that it includes the population of (many) cities, so we can use this in determining which is the largest nearby city. So in my first pass at solving the problem, I ran a PostGIS query for each airport to look for "large" cities nearby. Somewhat arbitrarily I initially specified population thresholds of 250000, 100000 and 0, and distance thresholds of 5, 25 and 50 miles. I would first search for cities with population > 250000 within 5 miles, and if I didn't find any I would extend to 25 miles and then 50 miles. If I still hadn't found anything, I would decrease the population to the next threshold and try again. If I found more than one in a given category I chose the closest, in this initial approach (the largest is obviously another alternative, but neither will be the right answer in all cases).

This worked reasonably well, returning London and Vancouver for the examples already mentioned. But it gave some wrong answers too, including Denver Airport which was associated with Aurora (a large suburb of Denver), and for a lot of small town airports it returned a larger town nearby, which in some cases might have been reasonable but in many cases wasn't.

So I decided to extend my algorithm to include matching on names - if there was a city called 'Denver' near 'Denver International Airport' then that is the one I was after. This required a little bit of SQL trickery. If I knew the city name and wanted to search for airports this is easy, you just use a SQL clause like:

where name like '%Denver%'

But it was the other way around, the value I had was a superstring of the value in the field I was querying. I found that in PostgreSQL you can do a statement like the following:

where 'Denver International Airport' like '%' || name || '%'

(|| is the string concatenation in PostgreSQL, so if the value in the field "name" is "Denver", the expression evaluates to 'Denver International Airport' like '%Denver%', which is true). I wouldn't really want to be running that sort of clause alone against the very large geonames table, for performance reasons, but since I also had clauses based on location and population to use as primary filters, performance was fine. And in fact to eliminate issues of upper versus lower case, the clause actually looked like:

where 'denver international airport' like lower('%' || name || '%')

So this would match the city names I found to any substring of the airport name (whether the city name was at the beginning or not). If I found a matching city name within a specified distance (I am currently using 40 miles), I use that, if not then I fall back to using the first approach to look for a "large city" nearby. On my first pass through with this approach, I was able to match names for 2356 of the 3562 airports with an IATA code.

I noticed that some cases were not matching when they looked like they should, where the relevant city names had accented characters. I realized that in some cases, the airport name had been entered with non-accented characters, so it was not matching the version of the city name which did have accented characters. The geonames data includes both an accented and non-accented (ascii) version of each city name, so I extended my where clause to do the wild card matching above on either the (accented) name field or the (non-accented) asciiname field. This increased the number of matching names to 2674, which was a pretty good improvement.

I noticed that in a few places, there were two potential "city" names in the airport name - in particular London Gatwick was matching on a "city" called Gatwick, rather than London, and Amsterdam Schiphol was matching a "city" called Schiphol. Looking at maps of the local area, I'm actually not convinced there is a city (or even a village) called Gatwick, and on further examination the geoname record shows its population as zero, and the same turned out to be true of Schiphol. But regardless, I decided to handle this by checking whether there was more than one city name match with the airport name, within the search radius, and if so to choose the largest city found (rather than the closest). There are some airports which serve two real cities also, and have both in their title, so this is a reasonable approach in those cases (no offense intended to the smaller cities!). This change returned me London instead of Gatwick and Amsterdam instead of Schiphol, as I wanted.

I was now getting pretty close, but I stumbled on a couple of small local airports where I was choosing a really really small town which happened to be closest, rather than a not quite so small town nearby which was really the obvious choice. For example, for Garfield County Airport in the mountains of Colorado, I was picking the town of Antlers (population listed as 0, though I think it might be a few more than that) rather than Rifle, population 7897! This was fixed just by added one more value to my population thresholds (I used 5000).

Obviously tweaking the population and distance thresholds would change some results, but from a quick skim I think the current results are pretty reasonable. One example which I am not sure whether to regard as right is JeffCo airport, a small airport which is in Broomfield, just outside Denver. The current thresholds associate this with Denver, which is probably reasonable, but you could argue that Broomfield is large enough to be named (45000).

If you're interested in checking out the full list, to see if your favorite airport is assigned to the city you think it should be associated with, you can check out this text file (the encoding of accented characters is garbled here, though it is correct in our database - but I haven't taken the time to figure out how to handle this using basic line printing in Java). Let me know if you think anything is wrong! The format of a line in the report is as follows:

13: GRZ Graz-Thalerhof-Flughafen -- 8073 Feldkirchen to Graz (Graz), true

The IATA airport code is first (GRZ), then the airport name from geonames (Graz-Thalerof-Fluhafen), then the city name we got from Yahoo reverse geocoding (8073 Feldkirchen), then the associated city name generated by our algorithm (Graz), followed by the ascii version of the name in parentheses (the same in this case), and lastly the "true" indicates that a match was found on the city name, rather than just using proximity and population.

So anyway, I thought this was an interesting example of how you can use the geonames dataset. I would be happy to contribute these city names back to geonames if people think this would be valuable - it is on my list to look into how best to do this, but if anyone has particular suggestions on this front let me know.

New release of whereyougonnabe

We are pleased to announce another new release of whereyougonnabe.

Some key features include:

1. WYTV, a new animated map display inspired by the cool Twittervision, but using data from whereyougonnabe. This displays an interesting mix of activities from your friends and public activities from other users. You can click through on friends’ activities to see more details.
2. Calendar synchronization now uses Google Local Search, based on the context of which city you’ve told us you’re in. So if you’re in Denver, CO and you put a location of “1100 Broadway” or “Apple Store” on an activity in your calendar that week, it will find a suitable location in Denver.
3. We now support display of distances in kilometers as well as miles, a request we have had from several users. If your home is in the US or UK the default setting is miles, elsewhere it is km. You can change your preference on the settings page.
4. The experience for new users is improved – when you add the application you can immediately view data from your friends, or public activities from other users, without having to enter any data. Users just need to tell us their home location before creating their own activities. There is more to come on this theme.
   
5. Various small usability improvements and bug fixes.

As always, we welcome your feedback on the new features, and ideas for further improvements. We will have another new release coming very shortly with a number of things that didn’t quite make it in time for this one. You can try the new release here.

A tale of two presentations

I have two presentations scheduled next week, with a major contrast in styles - one is 5 minutes and the other is 90 minutes. The first is at Ignite Denver, which is at Fado Irish pub in downtown Denver on Wednesday evening next week (September 10th). Ignite is an O'Reilly sponsored event with an interesting format:

If you had five minutes on stage what would you say? What if you only got 20 slides and they rotated automatically after 15 seconds? Around the world geeks have been putting together Ignite nights to show their answers.

You can register here for free (and if you want to be on my team for the trivia quiz let me know!). I'll be talking about my thoughts on future location and social networking, among a fairly eclectic agenda which ranges from a variety of techie topics to the wonderful world of cigars and how to swear in French! I'm looking forward to the challenge of trying out this presentation format for the first time. If you're in the neighborhood I encourage you to stop by, I think it should be fun.

The following day, Thursday September 11, I switch from having to talk about future location for 5 minutes to talking for an hour and a half at the GIS in the Rockies conference in Loveland, CO. The standard presentations there are 30 minutes but the organizers asked if I would talk for longer, which was nice of them! It was a tough job to get my GeoWeb presentation into 30 minutes, so I'll have the luxury of being able to expand on the topics I talked about there, as well as talking more about the technology we're using, including PostGIS and Google Maps, more about some of the scalability testing and system design challenges we have addressed, and more about how I see the overall marketspace we're in and how we fit into that. I'm on from 3:30pm to 5pm - hope to see some of you there.

First experience with Google Chrome

Whereyougonnabe makes heavy use of JavaScript in the browser, using the Ext JS library, which provides a very rich and dynamic user experience but also tests the limits of browser compatibility. This is especially true when running inside Facebook, which provides another level of complexity and has a knack of showing up obscure low level differences between browsers - we have hit several tricky issues in handling sessions and security recently. FireFox has generally given us the fewest issues. Early on we had some problems with Safari, and today we have had a day of major frustration as we were about to launch a new release, everything was running fine on FireFox and Safari, and on Internet Explorer if you ran our application outside a Facebook frame, but inside Facebook Internet Explorer has suddenly started abruptly refusing to load pages with no helpful error messages - we're still bashing our heads against the wall trying to track this one down.

So I have to say my initial reaction to seeing that Google has announced a new browser called Chrome was "oh no, not another browser to support".  It's currently Windows only which I also wasn't too keen on. But nevertheless I fired up my Windows laptop for the second time in a week to try it out. And I was pleasantly surprised and impressed that it worked first time with whereyougonnabe with no apparent problems :) !!  And subjectively it seemed fast too. We'll be testing it more thoroughly of course, but first impressions are encouraging in terms of compatibility with complex Javascript applications, which Google claim as a major design aim for Chrome.

More on Google Local Search

As I recently discussed, we have decided to use Google Local Search rather than Google Geocoding (well, rather than a hybrid approach) for whereyougonnabe. That post got some interesting comments, including one from Pamela Fox of Google who said that Local Search will soon be using Tele Atlas data (rather than NAVTEQ according to one of the other posts, though she didn't say that explicitly), and the implication was that this was the same source data as the Google Geocoder. One would hope that this may mean that they are bringing the two services together, which would be a lot nicer for users. I also ran into Martin May from Brightkite yesterday at the Techstars investor day (which was excellent) and he mentioned that they are making the same move, which was interesting - it turned out we had been looking at a lot of common issues at the same time.

Anyway, I thought I would follow up with a little more discussion on some of the benefits we see of using Google Local Search, as well as some outstanding issues. One aspect that I really like is that you can give it a search point to provide context, and this is very useful for our application. We are especially leveraging this in our calendar synchronization. A typical (and recommended) way of using the system when you are traveling is to create a high level activity saying that, for example, you are in New York City for four days next week, and then subsequently enter more specific activities for those days. When you add an activity in your calendar, we look at the location string in the context of where we think you are on that date. So for example, if I just said I was having lunch at 1100 Broadway, it would locate me at 11 Broadway in New York during those four days, or at 1100 Broadway in Denver if I was at home. And as I discussed in the previous post, the feature we like most about Local Search is that you can search for business names, and again these take that location context, which makes it very easy to specify many locations. Some examples that I have used successfully at home in Denver include "Union Station", "Vesta Dipping Grill" (or just "Vesta"), "Enspiria" (a company where I have an advisory role"), and even "Dr Slota" (my dentist). It's very convenient to be able to use names like this rather than having to look up addresses.

So that's great, but there are still a few issues. One is that Local Search really doesn't work well for airports - if you enter a 3 letter airport code it is very hit or miss whether it finds it. I'm pretty sure this used to work a lot better, though I haven't tracked down specific tests to prove that. But we plan to do our own handling of this case (unless we see an improvement really soon), using geonames (which we already use for locating airports in our tripit interface, but we haven't hooked this into our general geocoding).

I reported over a year ago that I felt there were some issues with Google Local Search on the iPhone, so I thought it would be interesting to revisit some of the same problem areas I reported on then. These problems generally revolved around local search being too inclusive in the data it incorporates, which is good in terms of finding some sort of result, but the risk is that you get bad data back in some cases. Given the good results I had found in my initial testing this time round, I expected that I would probably see some improvement, so I was a little surprised to find that in general most of the same problems were still there. You can see these by trying the search terms in Google Maps (and in MapQuest for comparison).

My test cases were as follows, centered around my home at 1792 Wynkoop St, Denver:

• King Soopers (a local supermarket chain): last time in the top 10 on Google, there were three entries with an incomplete address (just Denver, CO), all of which showed as being closer than the closest real King Soopers, and there was one (wrong) entry which said it was an "unverified listing". This time, there was just one incomplete address like this, so that was a definite improvement, but unfortunately this appeared top of the list so the closest King Soopers was not returned as the "best guess" from the local search API. There were two other bogus looking entries in the top 4, for a total of 3 apparently bad results in the top 10. On MapQuest, 10 legitimate addresses are returned, though in some cases they appear to have multiple addresses for the same store (e.g. entry points on two different streets) - but this is not a serious error as you would still find a King Soopers. This was the same as last time. I tried typing "King Soopers Speer" (the name of the street where the store is located) and that returned me the correct location as the top result.
  
• Tattered Cover (a well known Denver book store with 3 locations). MapQuest returns just 3 results, all correct. Google Maps returns 9 entries in its initial list, of which three have incomplete addresses (duplicates of entries which do have complete addresses, but they show up at entirely different locations on the map), one is a store which closed two years ago, and one is a duplicate of a correct store. The old store does say removal requested, but it seems surprising that this closed two years ago and is still there.
  
• Searches for Office Depot and Home Depot were more successful, with no obvious errors - hooray :) !! This was the same as last time.
  
• Searching for grocery in Google Maps online last time returned Market 41, a nightclub which closed a year before, and four entries with incomplete addresses. This time there were only two which looked bad (including the King Soopers with an incomplete address we saw before). The same search on MapQuest had one incomplete address, the rest of the results looked reasonable. So a definite improvement in this case.
  

So in summary, Google showed a little improvement over last year but not a huge change - they still could use some more quality control to remove bad data entries. But as I've discussed, overall we've obtained very good results for our application needs, so we are looking forward to rolling out the new Local Search based capabilities shortly.

Off to San Diego next week - and more GeoWeb coverage

Well, after my first ever visit to the ESRI User Conference last year, I'm going to be back there again this year, with my Enspiria hat on this time. Hope to catch up with some of you there ... I'll be around at the Enspiria booth a good bit of the time (#1907). Also, Enspiria has organized a cruise on Wednesday evening which is filling up but we have a few spots left so if you'd be interested in joining us let me know and I'll see if I can get you a ticket (no promises though!).

In other news, my GeoWeb presentation has been continuing to get some good coverage - after Ogle Earth gave it a good review, it also got mentions at Google Earth blog, Google Maps Mania (twice), and by Paul Bissett at Fiducial Marks. I have a lot of notes from GeoWeb and will try to do a post writing up my thoughts on the conference soon. But I'm off to the Santa Fe Opera tomorrow morning for a long weekend, where I look forward to catching up with my old friend Warren Ferguson, so it may have to wait until next week :).

Nice review of my GeoWeb presentation

Stefan at Ogle Earth gave a nice review of my GeoWeb presentation:

I ended up watching Peter Batty's entire presentation at GeoWeb on "Future Location and Social Networking". I hadn't planned to, but I was surprised to find a lot more deep thinking than I imagined was possible about this market segment.

Thanks Stefan! As I try to explain in the presentation, I do think there's a lot more to this space than meets the eye, when you get more deeply into it. Stefan also mentions that there are some cool Google Earth visualizations at the end, which you can see separately here, though of course I enourage you to watch the whole thing!

My presentation on "Future Location and Social Networking" at GeoWeb

I gave my presentation at the GeoWeb conference yesterday and it generated a good bit of interest. I recorded it with the brilliant ScreenFlow software which I have been using extensively to do screen capture videos, but this is the first time I tried it with a conference presentation and it worked very well. I talked previously about what I was planning to talk about - in summary, it's about the ideas we have about modeling future location for whereyougonnabe, and I talk quite a bit about the bigger picture of where we plan to take it - combining space, time and the social graph to create a new infrastructure for calendaring and scheduling. 

GeoWeb presentation: The Power of Future Location for Social Networking from Peter Batty on Vimeo.

It's about 30 minutes long but I encourage you to take the time to check it out, I really think that the whole future location area is a very interesting one!

Presenting at GeoWeb next week

I'll be up in Vancouver next week for the GeoWeb conference, which was excellent last year and it looks like there will be a good program and group of attendees this year too. Whereyougonnabe tells me that 13 of my friends will be in Vancouver at the same time as me, so it's good to see a few people starting to use the system!

I'll be doing a presentation on topics relating to future location, which is of course our focus with whereyougonnabe. As we've got further into this, I've really come to realize that there are some hard problems to solve in the space of future location which have a lot of value, and many ways that the core technology we're developing for our initial social networking application can be applied. I thought I'd just post a brief preview of some of the topics I'll be talking about.

Broadly speaking, I think that the hard problems fall into three main areas:

Modeling and performance. How you model future space-time has a huge impact on what you can do in your application - for example Dopplr is a nice application in many ways, but its underlying model of space-time has some serious limitations which really constrain the functionality they can provide compared to the approach we are using.  I'll talk about coarse-grained versus fine-grained modeling of space-time, and using discrete "places" versus a continuous coordinate system. Implementing a fine-grained system with the ability to handle variable search radii in both space and time is pretty challenging from a performance and scalability perspective with large data volumes, and I'll discuss some aspects of how we've approached this. I'll also talk about various interesting modeling challenges including handling overlapping activities, uncertainty (possible / probable activities), alternative plans (may be either here or there), flexible plans (I'm intending to go to the shopping mall sometime tomorrow afternoon), linear activities (like road or train trips).

Determining what's "interesting". Since we are focused on fine-grained future activities, which may be close to home or far away, users may be "close" to their friends frequently, and what constitutes "close" may vary significantly depending on the context. If I am in England for two weeks I am interested in notifications happening between me and friends who are within a relatively large distance of me (a hundred miles or so, say), whereas if I'm going out for a drink in downtown Denver, where I live, my notification radius should be relatively small. If I have friends who live close to the bar I'm going to, should I notify them about my plans or not - they may well be interested but we don't want to overwhelm people with too much information. I'll talk about factors involved in determining what's interesting, including the distance from home of each person, normal distance between the people, how often you see them, length and nature of the activity, and more.

Gathering data on future activities. Of course one of the big challenges with future location compared to current location is creating the raw data about where people will be and when. Obviously making it easy for people to create activities is one aspect of this, but integration with other data sources is also key, including calendars, online booking systems for travel, restaurant reservations, concert or sporting tickets, etc etc. As we pull data from multiple sources, merging those together in the optimal way can be challenging. I'll discuss a variety of ideas and plans we have in this area.

So that's just a taster of some of the areas I'll be talking about - I'll also talk about more specifics of what we've implemented so far and our experience with the technologies we're used, including PostGIS on the back end (which I've been really impressed with), Google Maps, geocoding and local search, the Ext JS Javascript Library, and more. And I'll talk more about the broader range of applications I see for what we're doing.

I'm talking from 4:30 to 5pm on the Wednesday, July 23, so if you'll be at the conference please stop by!

Major new release of whereyougonnabe

Apologies again for the lack of posting recently ... a primary reason for this has been we have been heads down adding lots of new stuff to whereyougonnabe recently, plus I've been juggling a number of side projects. But anyway, we are excited about our latest release which went live over the weekend (you can access it here). 

The big theme of the release is integration - we can now automatically import activities from Google Calendar, Tripit, and Dopplr, we have added an iCal feed so you can view whereyougonnabe information in your regular calendar, and you can set your Facebook and/or Twitter status whenever an activity begins. In general, a big challenge with future location is how to create data as easily and automatically as possible, and this is a big step forward in that regard.

We also made significant changes to the way that we calculate interactions and decide what to notify you about, which I have come to realize is a very hard problem, as I have got more into this (which is good, as it is harder for other people to copy what we are doing).

You can read more specific information at our whereyougonnabe blog, or watch the following video for an overview of the new features:

whereyougonnabe July 2008 release overview from Peter Batty on Vimeo.

We also have several more detailed videos explaining how to set up and use the new features. Our big focus for the next release is on the user interface, an area where we have a lot of ideas and plans that we are excited about.

New release of whereyougonnabe

Well, we've been busy in the three weeks since we released our first beta of whereyougonnabe, and today we put out some significant improvements, including support for the Safari web browser, email notification when we find close interactions with your friends, a cool Facebook profile widget, the ability to easily re-use previous locations, and a number of other usability improvements and bug fixes.

This is what the new profile widget looks like:


A major focus for us is making it very easy to create activities, and we have added the ability to dynamically select previously created locations. This is a screen shot showing how this looks on the create activity screen:

We also took some of Stefan's comments to heart, and have made the activity title optional, so you can just say "Peter will be in Boulder" rather than "Peter will be working in Boulder". As well as giving you more flexibility this makes it faster to create activities, as the minimal things you need to specify now are a location, plus start date and end date (which both default to today) - but you have options to enter various other details if you want to. We also fixed an issue with handling accented characters, and made some improvements to our icons, fixing some aspect ratio issues and adding a default icon for people who don't have a picture on Facebook.

We will now email you when we find that you are going to be close to some of your friends, which is just the first step in providing a range of notification capabilities.

We have a lot more things in the pipeline, including further Facebook integration (support for the newsfeed and minifeed coming very soon, plus the ability to set your Facebook status when an activity starts), our first steps in calendar integration, and a simplified interface suitable for mobile browsers.

If you haven't signed up yet or you want to check out the new features, you can do so here. And we welcome your feedback on our new forums (or feel free to email me).

Nice review of whereyougonnabe at Ogle Earth

Stefan Geens of Ogle Earth has written a very thorough review of whereyougonnabe, and has some nice things to say, as well as some good suggestions - thanks Stefan! He compares us to Dopplr, who are really the closest site out there to what we do, and says "Dopplr is simpler to use, while whereyougonnabe is more full-featured". He thought that the "pièce de resistance" was the Google Earth interface and in his summary, he says "in sum, I very much like whereyougonnabe".

To be honest, I'm delighted that Stefan thinks we are already more full-featured than Dopplr when we're still at such an early stage of beta release, and they have been out for some time, and have - rightly - been getting a lot of interest and good reviews. And I would agree with him that Dopplr is easier to use today than we are. Part of this is that they are doing something much simpler than we are, though similar in some respects (I will discuss this in more detail in a future post), but nevertheless we know we want to make things simpler to use and we have a lot of things in our short term plan which will help on that front. But as I said in an earlier post, while there are lots of things on our to do list, it would have been easy to keep improving things for another several months before releasing, but I felt we were far enough along to get something out there and start to build up some users and get some feedback. And while we still have a few more rough edges than I would like, I think the response so far has been very good and I'm glad we did decide to get it out there rather than waiting longer.

And lastly, one small point of clarification / correction ... Stefan suggests it would be nice to have color coding to show "close encounters", and actually we do have that in there already ... I suspect he may not have any close encounters yet so the color coding has not shown up. But a green box around a friend's picture shows that you are "very close" and a yellow box shows that you are "somewhat close". What constitutes "very close" and "somewhat close" varies depending on a number of factors, including how far from home both you and your friend are (in general, if either you or your friend are far away from home, a larger distance is used than if you are close to home). The ability to have a variable distance for determining what constitutes a close encounter is something that Dopplr's approach does not handle incidentally, and I think that this will be a powerful enabler for us as we add more sophisticated features to the application (though not something that the user is necessarily conscious of).

So anyway, thanks again to Stefan for the detailed review. It's been an exciting week, with getting the system out to the world, and by and large everything went very well ... on to working through the big enhancement list now :) !!

Why not just talk to your friends?

Well, we have a few of the Luddites out in the comments on James' blog - there's Andy "whatever happened to talking to your friends" and Bubba "Facebook is inherently evil and a waste of time" (incidentally, Ned Ludd is from Anstey, the village in England where I went to High School, in case you were wondering - though according to Wikipedia there is no proof he actually existed, which I hadn't heard before). But I digress!While I fairly obviously disagree with most of their comments, there are elements of sentiment in there that I do agree with, and they raise some topics which are worth a discussion in terms of how we communicate with people, and how communication mechanisms have evolved. First I should say that obviously a primary aim of whereyougonnabe is to help you meet your friends more often in the real world - I certainly agree with the principle that you can spend far too much of your life sitting in front of a computer, and in front of Facebook in particular.

But electronic communications obviously have a strong role in being "social" and maintaining your relationships with friends - as someone with friends all round the world and most of my family 5000 miles away in the UK, email enables me to keep in touch with them much better than I could without it. That's not a substitute for meeting face to face of course, and that's part of the motivation for whereyougonnabe.

With the rise of social networking systems recently we have really seen an evolution from one to one communication to many to many communication among friends. It's interesting if you use Facebook (which I assume Andy and Bubba don't) to see how a sequence of short and often trivial status messages can give you a good feel for what distant (or local) friends are up to, and every so often prompt you to get in touch with them in response to something you see - much more than you do without that impetus, in my experience. Twitter is the same type of experience. As I've heard Mark Zuckerberg say (paraphrasing), the cost (or effort) of communicating with your friends is significantly lowered by systems like Facebook. Just by taking a few seconds to type in a short message, you can communicate with hundreds of friends wherever they are in the world. I certainly feel more connected with my distant friends who are on Facebook (in general) than those who are not. Now there's a lot of crap on Facebook too (I can do without vampire biting applications personally) and yes, a lot of people waste a lot of time on it, but you can say the same of the web in general, or television, or whatever. But that doesn't change the fact that it does represent a new paradigm in how you can communicate with your friends.

However, one of the challenges with this type of many-to-many communication is that if it becomes too successful, then ironically it can lose its effectiveness. Once you get hundreds of friends using Facebook or Twitter or something similar and they all start posting messages frequently, you reach a point where you just can't keep up with the volume of messages (unless you really don't have a life, in which case Bubba may have a point). So you need some way of filtering that stream of messages down to a manageable volume, trying to select the messages that are most interesting or relevant to you. Facebook claims to do some clever filtering in determining which stories to display in your Newsfeed, but they don't have too much information to go on, and it's not clear to me that they really pull out messages that are most interesting to me.

One of the things that we are doing with whereyougonnabe is to add extra information to certain messages you want to communicate to your friends - specifically location and time - which gives us a much greater ability to extract items of specific interest to you from this big message stream. We do this primarily by choosing messages where you will be close to your friend in space and time, to help you meet your friends in the real world more often than you would do otherwise (back to having a real life :). If that approach doesn't yield enough messages to display, we can display other messages using either random selection, other approaches like those Facebook uses, or by leveraging location again to pull out messages which seem like they may be more interesting - for example, if someone is far from home that is probably an unusual situation and may well be of more interest than one close to home (in general).

Changing direction a little, let's look at a couple of specific examples of how whereyougonnabe can help you meet your friends more often, in a way that really can't be achieved just by "talking to your friends" (or phoning them or emailing them). The first is meeting people in airports. When I worked at Intergraph and was traveling frequently (more or less every week), I would randomly run into someone I knew at Denver airport about once a month. When you consider the huge size of Denver airport, this leads me to conclude that there is a high probability that there are one or more people I know in Denver airport every time I'm there, I just don't know that they're there. Now hanging around in airports is pretty boring. Sure, I can read or work and that's fine, but if I had a chance to spend all those spare half hours between getting through security and boarding the plane by meeting up with friends who I wouldn't otherwise have seen, that would add a lot of value and enjoyment to my life. And whereyougonnabe can give me the information to let me do that. It's really not a practical proposition for me to call the hundreds of friends who are candidates for being in Denver airport before every trip to compare notes on schedules. And if I did an email blast to all of them before every trip I would rather quickly become unpopular, and if everyone started doing that then you're back to the information overload problem. With whereyougonnabe I can just take 30 seconds to enter details on my planned time at the airport and it won't bother anyone else with that information unless it's specifically relevant to them or they ask to see it. And once we have calendar integration working, which will be coming soon, then it's zero extra time on my side, since I have to enter the information into my calendar anyway.

I was also going to discuss the example of very local interactions - like the one I include in the intro video about going to the supermarket - but this post has got way too long already so I think I'll save that for another day. This example is an interesting one as several people who like the general concept of whereyougonnabe have said that they see themselves using it more for long distance trips than local activities. I can certainly understand that as an initial reaction but, having thought about it a lot, I feel there is a lot of potential in both scenarios. But I'll come back to that.

Anyway, I'd like to summarize a couple of things before signing off. I think there are (at least) two rather different ways to look at what whereyougonnabe does. The first is that it's an application specifically focused on helping you meet up with your friends. That's how I started out thinking about it, and still that's certainly a major focus and primary benefit of the system. However, you can also think of it as an evolution of many-to-many friend communication systems, like the Facebook messaging system or Twitter. You get many of the same benefits, of seeing what your friends are up to in a general stream of short messages, but with the added benefit that we can highlight (notify you about) items of specific interest, in particular in regard to opportunities to meet with friends. This becomes increasingly valuable as the system becomes more heavily used, and it becomes hard to keep up with the full unfiltered message stream. If you look at the system only from the first perspective, then as one person said to me, you only get a "reward" from the system when it identifies an opportunity for you to meet someone, which may not happen very often, especially when you are new to the system and don't have many friends on it. But if you look at it from the second perspective, you get "rewards" from the system just through finding out what your friends are up to, in the same way that you do from a Facebook or Twitter, and with the added bonus of being able to see things on a dynamic map which gives you a whole new dimension compared to a simple text stream.

And last but not least, while I disagree with my friends Andy, Bubba and Ned in one sense, and believe that there is significant value in social networking systems, I agree with them that you can spend too much time using such systems, and a key aim of whereyougonnabe is to enable us to spend more time with our friends in the real world.

whereyougonnabe one day in

Well, overall our first day with whereyougonnabe went well. A couple of hundred people from 20 countries have signed up so far, we've had a good amount of positive feedback, and the system seems to have been running well. Personally I have already discovered quite a few interesting interactions with friends I didn't know about before, which bodes well for the system being useful. We got some coverage in the blogosphere from James, Glenn (who will be close to me next week) and Jonathan (who had some constructive feedback and liked the Google Earth integration). As Jonathan commented, the effectiveness of it is very much dependent on the "network effect" - the more friends you have using it, the more useful it is. This is one of the key reasons we built it on top of Facebook (initially) rather than doing yet another independent networking system. I personally am pretty reluctant to invite friends to a new online networking system until I'm really convinced it's valuable - but of course it's not really valuable to you until you have a lot of friends on there, which is a big barrier to success for a new system.

We had a few issues with browser compatibility which we hadn't uncovered in our alpha testing, but which got quickly exposed with a larger group of users. We currently don't support Safari but we're working on that (we knew about this before but it turned out that our nice error trapping worked in the main application but not on the initial sign up page, so apologies to the Safari users who experienced ungraceful crashes trying to sign up - this will be fixed tonight). We've also discovered some issues with IE6 which we hadn't come across before, and we're debating whether to try to support that or whether to follow the "Save the Developers" campaign!

We now have a whereyougonnabe forum up, so please use that to let us know about any problems or to post suggestions for future enhancements. Thanks to everyone for all their interest and words of encouragement!

Our whereyougonnabe application is now open to the world!

Well, after a lot of effort over the past six months or so, and having been in alpha testing for the past couple of months, we have now declared our "where you gonna be" application to be in beta and opened it up to the world. Thanks to my co-developers Glen and Nate for all their efforts in getting us to this point! There are still a huge number of things on our to do list of course, and you will see a lot more very cool functionality appearing over the next couple of months, but we wanted to get the basic application out there to start building up some users and get feedback from more people. See the following video for a general introduction to the application (I strongly recommend clicking on the link underneath the video and selecting full screen HD if you have a fast connection, the quality is much better):


Introduction to whereyougonnabe from Peter Batty on Vimeo.

The key idea of the system is that you and your friends tell us where and when you will be doing things in the future, and we look at this information to tell you when you will be close by to your friends. Even if you end up not being close to your friends, we give you various ways to see what they are doing, which is a good way of keeping up with them in a similar way to Facebook status messages or Twitter, but in a richer fashion which adds both a spatial and temporal element - the ability to display what they are doing on a map, and see what they are doing over time, rather than just right now. This focus on the future is our biggest differentiator - there are lots of systems focused on where you are right now (and we see lots of opportunities for integrating with them, but that's a topic for a future post).

Future location has two significant advantages over current location. The first is that if you only find out right now that a friend is close by, the chances are that one or other of you already has commitments and so you can't meet up. But if you knew yesterday or last week that you would be close by, you could have worked out your schedule so that you could meet up. Secondly, "where are you now" applications typically require a location-aware phone, and these still don't have big enough market penetration to really give this type of application the critical mass that it needs to take off. All you need to use our application is a web browser. Of course, the challenge of future location is that you can't infer it automatically, so one of the critical success factors for us is making it very easy to create activities. Currently this is a manual process, which we have tried to make as easy as possible, but this is an area where you will see a lot of new functionality over the next month or two, with much more in the way of calendar integration and functionality to make it much easier to create a trip with multiple activities.

We have a precise spatial and temporal resolution, unlike other applications we have seen in this space, which tend to have spatial resolution of a city and temporal resolution of a day. This means we have many local applications as well as more global ones - like identifying which friends will be in an airport at the same time as you, or identifying opportunities to coordinate rides when transporting kids to after school activities, or working out which friends will be in bars close to you this Friday night.

We have developed what we think is a very cool interface to Google Earth, which leverages its time capabilities nicely. Check out the following video to get a flavor of this (again, I strongly recommend clicking on the link beneath the video and then displaying the full screen HD version):


whereyougonnabe integration with Google Earth from Peter Batty on Vimeo.

Our application currently runs on the Facebook Platform. The big advantage of this is that you don't need to re-enter your network of friends, you can just leverage the existing social graph that you have in Facebook. In future we will support other social and business networking systems - LinkedIn is an obvious target for the business traveler, and of course OpenSocial is on our list too.

So anyway, please give the application a try and let us know what you think. Now that we have the beta release out "in the wild" I'll be blogging a lot more about some of the technology behind it and some of the things we plan to do in the future.