I just thought I would mention that I am planning to be at the upcoming GeoWeb conference in Vancouver. They have an impressive set of big names lined up for keynotes, with Vint Cerf, Jack Dangermond, Michael Jones, Geoff Zeiss and Vincent Tao, as well as a pretty packed set of technical sessions, three concurrent tracks for two and a half days (and that's excluding pre-conference workshops which I probably won't make it to). And a trip to Vancouver is always something I'm happy to find an excuse for! I look forward to catching up with lots of friends there, and might even end up on a panel if they'll have me (I have volunteered but they are still finalizing plans for those).
I'm actually planning to do a family road trip up there from Denver and take in a bit of scenery along the way. I've been playing a bit with Rand McNally's trip planning site, though haven't used it enough to see if there is any advantage over the new and cool additions to Google Maps routing which the whole world has blogged about - will report back on that if I find anything of particular note. I think we will take in Jasper and Banff en route, if anyone has any other specific suggestions for good places to go through, please let me know!
Tuesday, July 3, 2007
Local search on Microsoft Local Live
Steve Lombardi from Microsoft emailed me about the local search issues I raised in regard to Google Maps on the iPhone (and other mobile platforms) in my earlier post. In that post I compared the results I got with Google to those I got from MapQuest - I would have tried Microsoft Local Live also, except I had been under the mistaken impression that it didn't work on my Mac ... turns out it doesn't work with Safari (you get a very scaled back site which is pretty useless), but it works fine with FireFox (for 2D stuff - Virtual Earth in 3D, and Photosynth, are actually the things I miss most in Mac land so far!). So I was pleased to discover that I can at least use the 2D stuff still.
Steve had tried the same tests I did on Local Live, and I tried them for myself and it fared well. With luck, this live link will give you more or less the same thing as the screen shot above. I ran several tests in my previous review, all centered on 1792 Wynkoop St, Denver, CO. For full details see the earlier post, but I'll summarize the earlier results here too. These were the test searches and the results I got with Local Live:
A couple of other quick observations on the Microsoft Local Live implementation. One is that it lets you place a pin at your original location of interest, and also display pins in different colors showing multiple different query results at the same time - this feature is nicely done and not available in Google either online or mobile. The results include some ads, but these are clearly separate from the result listing. The results come back sorted in order of "relevance", which I think is probably in most cases a euphemism for "more or less by distance, but with scope to move sponsored results further up the list" - which is what Google appears to be doing with its mobile maps as I discussed previously. But with one click I can change this to sort by distance, and on both these lists it shows the distance of each result from my starting point. As a user I am quite happy with this approach - it gives the service provider (Microsoft) the chance to monetize their service with ads and preferred placement, which ultimately is necessary otherwise service providers won't be able to continue to provide their service, but it doesn't hamper my ability to easily get the specific information I want (as Google Maps Mobile does).
Steve tells me that the same local search capabilities are available in Microsoft Live Search for mobile ... but unfortunately there's not much chance of me testing that any time soon, as I am already suffering enough abuse over having both an iPhone and a BlackBerry 8800, so I don't think I can justify adding a Windows-based smartphone to the collection :) !!
Steve had tried the same tests I did on Local Live, and I tried them for myself and it fared well. With luck, this live link will give you more or less the same thing as the screen shot above. I ran several tests in my previous review, all centered on 1792 Wynkoop St, Denver, CO. For full details see the earlier post, but I'll summarize the earlier results here too. These were the test searches and the results I got with Local Live:
- King Soopers (a local supermarket chain): Google returned 4 non-existent results in the top 10; Microsoft and MapQuest both had no errors but some duplication, in the sense of having multiple addresses close by for the same store. M&M both correctly located the closest store but Google didn't (unless you count manually discarding results with incomplete addresses)
- Tattered Cover (a well known Denver bookstore with 3 locations): Google returned 4 non-existent results in a list of 8, while M&M both returned only the 3 correct store locations.
- Office Depot and Home Depot seemed to work fine with everyone, with no obvious errors.
- A search for grocery yielded 4 out of 10 incorrect results on Google and 10 reasonable looking results on MapQuest (though I didn't verify them all). With Microsoft the top result was incorrect, an incomplete address which just said "Denver, CO", similar to those which caused a lot of the errors with Google. And the second address on the list was interesting - it was Cowboy Lounge, which is a nightclub which used to be Market 41, which appeared on the Google list also, incorrectly categorized as a grocery store (you can understand how the mistake occurred given the original name). Interesting that Microsoft picked up the name change, which Google didn't, but still has the incorrect categorization. However, one good thing with Microsoft is that I was given the option to provide feedback that the result was incorrect, so I did that and asked this it should be removed from the grocery categories (and I provided feedback on the previous incorrect result too). The rest of the list appeared to be legitimate establishments, though the Rocky Mountain Chocolate Factory and Cookies by Design are a broad interpretation of "grocery" :) !! So Microsoft did better than Google but not as well as MapQuest on this one. I will be interested to see if and when my feedback on the incorrect categorization gets through the system, I will check it out every so often to find out!
A couple of other quick observations on the Microsoft Local Live implementation. One is that it lets you place a pin at your original location of interest, and also display pins in different colors showing multiple different query results at the same time - this feature is nicely done and not available in Google either online or mobile. The results include some ads, but these are clearly separate from the result listing. The results come back sorted in order of "relevance", which I think is probably in most cases a euphemism for "more or less by distance, but with scope to move sponsored results further up the list" - which is what Google appears to be doing with its mobile maps as I discussed previously. But with one click I can change this to sort by distance, and on both these lists it shows the distance of each result from my starting point. As a user I am quite happy with this approach - it gives the service provider (Microsoft) the chance to monetize their service with ads and preferred placement, which ultimately is necessary otherwise service providers won't be able to continue to provide their service, but it doesn't hamper my ability to easily get the specific information I want (as Google Maps Mobile does).
Steve tells me that the same local search capabilities are available in Microsoft Live Search for mobile ... but unfortunately there's not much chance of me testing that any time soon, as I am already suffering enough abuse over having both an iPhone and a BlackBerry 8800, so I don't think I can justify adding a Windows-based smartphone to the collection :) !!
Follow up discussion on GE next generation system (about customization and open source)
My two part post last week on General Electric's next generation system based on Oracle (part 1 and part 2) generated some interesting follow up comments and questions (mainly after part 2). I got somewhat sidetracked with the whole iPhone extravaganza over the past few days, but wanted to circle back and follow up on a couple of threads.
The first thread related to customization versus configuration, and the fact that there are potential attractions in being able to provide an "off the shelf" system, which is just configured and not customized, for a specific application area - in this case design and asset management for mid-size electric utilities in North America. If you can achieve this, then potentially you can significantly reduce implementation costs and (in particular) ongoing support and upgrade costs. However, the challenge lies in whether you can meet customer needs well enough in this type of complex application area with just configuration. People have tried to do this multiple times in the utility industry, but so far nobody has been successful, and everyone has fallen back to an approach which requires customization. Both Roberto Falco and Jonathan Hartley were skeptical that a pure configuration approach could be successful, and Jonathan makes a good argument in his response that if you implement a very complex configuration system, then you may end up just recreating a customization environment which is less flexible and less standard than if you had just allowed appropriate customization in the first place. I don't disagree with either of them in general - though I would make the comment to Jon that I don't think we're talking about "GIS" in general, it's a specific application of that to electric utilities in a relatively narrow market segment, so that gives you a little more chance of defining the problem specifically enough that a configurable system could make sense. One other observation I would make is that I think that cost is an element of this equation. If an "off the shelf" system meets 90% of an organization's stated requirements and costs 80% of the amount of a fully customized system which meets 100% of their requirements, that is much less compelling than if the off the shelf system cost (say) 20% of the fully customized system, in which case they might be more willing to make a few workarounds or changes to business processes to accommodate such a system. This may be stating the obvious, but I wonder whether organizations trying to develop this "off the shelf " approach go into it thinking that they will have to offer a substantially lower price (or provide other compelling benefits) to persuade customers to adopt it. Finally on this thread, I think it is also worth observing that the "GIS" (if you call it that) in a utility is not a standalone system - it typically requires some sort of integration or interfaces with many other systems, such as customer information, work management, outage management, workforce management, etc etc. This makes it an even bigger challenge to develop something which does not require customization.
Paul Ramsey raised a different question, that of open source, which I think is interesting in this context. But first I will just answer a couple of other questions that Paul raised. He asked if this was essentially a replacement market, and the answer is yes - especially among the medium and large utilities, and in the more established markets (North America, Western Europe, Japan, Australia / New Zealand, etc), pretty well everyone has a system implemented, and GE/Smallworld, Intergraph and ESRI are the dominant vendors. Because of the customized nature of these systems, and the amount of integration with other business systems, switching to a new vendor is typically a multi-million dollar project, even if the new software is provided for free. So this is obviously a big reason why this is a "sticky" market and customers don't change systems very often. The other clarification is that Paul asks about defining the market of customers "who think that 'classic' Smallworld doesn't cut it anymore", and overall I wouldn't particularly categorize things that way. Overall I think that satisfaction levels are probably as high among the Smallworld customer base as with any of the other vendors, the system is functionally very competitive still, the main concern is probably in obtaining and retaining people with the appropriate specialized skills, especially for smaller organizations. But there has been very little movement of utilities from Smallworld to other companies up to this point. Ironically, GE bringing out a next generation system (which they had to do for new business) is something which may cause existing customers to re-evaluate their strategy. Again though this is just a standard challenge of moving to a new generation architecture for any software company - you're damned if you do and damned if you don't.
Anyway, back to open source - Paul raises the question of whether GE doing something with open source may be an option. I have heard a few people raise this before in regard to Smallworld, and it is an interesting question. There are three major elements to the Smallworld core software. The first is the Magik programming language, which was developed by Smallworld back in the late eighties, and it is very similar to Python and Ruby. Smallworld was way ahead of its time in recognizing the productivity benefits of this type of language, and this was a key reason for its success. The core Magik virtual machine has been stable for a long time, the original developers of it left Smallworld a number of years ago and I suspect that nothing much has changed at this level of the system recently. The second key element is VMDS, the Version Managed Datastore. There is some very complex and low level code in both of these components which I suspect would make it hard to open source them effectively, given the amount of time it would take people to get into the details of these components (currently known to only a very few people either inside GE, or who have left GE), and the relatively small size of the market for the products would probably be a disincentive for people to make the effort to learn this. However, both these components are very stable and probably don't really need much maintenance or change. The vast majority of the system is written in Magik, and the bulk of the Magik source code for Smallworld GIS has always been shipped with the product, which has allowed for a hugely flexible customization environment (much more flexible than Smallworld's main competitors). There is a pretty large base of technical resources in customers and Smallworld partners who know how to enhance the system using this environment. If GE could manage enhancements at this level of the system in an open source fashion, to leverage the strength of the existing Smallworld technical community, who are on the whole very loyal to Smallworld and have a strong interest in seeing it continue to be successful, this could be a very powerful thing. As I noted in my previous post, one of the challenges for GE is that to develop new applications on two platforms concurrently (Smallworld and the new Oracle/Java based platform) will significantly decrease the amount of new functionality they can develop with a finite set of resources.
Of course there are a lot of challenges for an existing commercial software company in making such a switch. A critical one for GE would be that they could maintain their existing maintenance revenue from the Smallworld user base (at least to a large degree), otherwise it would be a non-starter from a business perspective I think. But it is quite conceivable that this approach could harness the talents of a much larger group of developers than Smallworld currently has working on the product, and produce more enhancements and fixes to the products than customers currently see, so you can see scenarios in which customers would be happy to continue to pay maintenance for support. As Paul points out, Autodesk has made this work and most of their customers still pay maintenance. There are differences in the Autodesk scenario as they open sourced a new product rather than an old one, and I think that a big driver for them was that they saw that it was going to be increasingly hard to make money for basic web mapping products, as that area becomes increasingly commoditized, due to the efforts of both the Googles and Microsofts of the world as well as the open source community. I think that this factor probably helped them decide to make the big leap to an open source approach, and it seems to have been a successful one for them.
Could GE also consider open sourcing portions of the new product, which I think was really Paul's question? That could also be an interesting possibility, to help gain traction in the market. If they open sourced some of the more generic base level components of the product, they could still build their specific business applications on top of these components and charge money for those, but leverage the open source community to provide additional functionality.
So the motivations are somewhat different for GE than for Autodesk, and there are multiple scenarios where open source could play a role, but I could envision an open source version of Smallworld significantly extending the product's life, by harnessing the very good technical resources which exist in the broader Smallworld community. Internal GE resources could then be freed up to work on the new product line (with just a small number focused on managing the open source developments of the established Smallworld product line). There are certainly a number of challenges and a lot of complexity in making something like this fly from a business perspective, and I'm not sure if GE would have the imagination or the boldness to take a risk on it, but it's an interesting thing to speculate about!
The first thread related to customization versus configuration, and the fact that there are potential attractions in being able to provide an "off the shelf" system, which is just configured and not customized, for a specific application area - in this case design and asset management for mid-size electric utilities in North America. If you can achieve this, then potentially you can significantly reduce implementation costs and (in particular) ongoing support and upgrade costs. However, the challenge lies in whether you can meet customer needs well enough in this type of complex application area with just configuration. People have tried to do this multiple times in the utility industry, but so far nobody has been successful, and everyone has fallen back to an approach which requires customization. Both Roberto Falco and Jonathan Hartley were skeptical that a pure configuration approach could be successful, and Jonathan makes a good argument in his response that if you implement a very complex configuration system, then you may end up just recreating a customization environment which is less flexible and less standard than if you had just allowed appropriate customization in the first place. I don't disagree with either of them in general - though I would make the comment to Jon that I don't think we're talking about "GIS" in general, it's a specific application of that to electric utilities in a relatively narrow market segment, so that gives you a little more chance of defining the problem specifically enough that a configurable system could make sense. One other observation I would make is that I think that cost is an element of this equation. If an "off the shelf" system meets 90% of an organization's stated requirements and costs 80% of the amount of a fully customized system which meets 100% of their requirements, that is much less compelling than if the off the shelf system cost (say) 20% of the fully customized system, in which case they might be more willing to make a few workarounds or changes to business processes to accommodate such a system. This may be stating the obvious, but I wonder whether organizations trying to develop this "off the shelf " approach go into it thinking that they will have to offer a substantially lower price (or provide other compelling benefits) to persuade customers to adopt it. Finally on this thread, I think it is also worth observing that the "GIS" (if you call it that) in a utility is not a standalone system - it typically requires some sort of integration or interfaces with many other systems, such as customer information, work management, outage management, workforce management, etc etc. This makes it an even bigger challenge to develop something which does not require customization.
Paul Ramsey raised a different question, that of open source, which I think is interesting in this context. But first I will just answer a couple of other questions that Paul raised. He asked if this was essentially a replacement market, and the answer is yes - especially among the medium and large utilities, and in the more established markets (North America, Western Europe, Japan, Australia / New Zealand, etc), pretty well everyone has a system implemented, and GE/Smallworld, Intergraph and ESRI are the dominant vendors. Because of the customized nature of these systems, and the amount of integration with other business systems, switching to a new vendor is typically a multi-million dollar project, even if the new software is provided for free. So this is obviously a big reason why this is a "sticky" market and customers don't change systems very often. The other clarification is that Paul asks about defining the market of customers "who think that 'classic' Smallworld doesn't cut it anymore", and overall I wouldn't particularly categorize things that way. Overall I think that satisfaction levels are probably as high among the Smallworld customer base as with any of the other vendors, the system is functionally very competitive still, the main concern is probably in obtaining and retaining people with the appropriate specialized skills, especially for smaller organizations. But there has been very little movement of utilities from Smallworld to other companies up to this point. Ironically, GE bringing out a next generation system (which they had to do for new business) is something which may cause existing customers to re-evaluate their strategy. Again though this is just a standard challenge of moving to a new generation architecture for any software company - you're damned if you do and damned if you don't.
Anyway, back to open source - Paul raises the question of whether GE doing something with open source may be an option. I have heard a few people raise this before in regard to Smallworld, and it is an interesting question. There are three major elements to the Smallworld core software. The first is the Magik programming language, which was developed by Smallworld back in the late eighties, and it is very similar to Python and Ruby. Smallworld was way ahead of its time in recognizing the productivity benefits of this type of language, and this was a key reason for its success. The core Magik virtual machine has been stable for a long time, the original developers of it left Smallworld a number of years ago and I suspect that nothing much has changed at this level of the system recently. The second key element is VMDS, the Version Managed Datastore. There is some very complex and low level code in both of these components which I suspect would make it hard to open source them effectively, given the amount of time it would take people to get into the details of these components (currently known to only a very few people either inside GE, or who have left GE), and the relatively small size of the market for the products would probably be a disincentive for people to make the effort to learn this. However, both these components are very stable and probably don't really need much maintenance or change. The vast majority of the system is written in Magik, and the bulk of the Magik source code for Smallworld GIS has always been shipped with the product, which has allowed for a hugely flexible customization environment (much more flexible than Smallworld's main competitors). There is a pretty large base of technical resources in customers and Smallworld partners who know how to enhance the system using this environment. If GE could manage enhancements at this level of the system in an open source fashion, to leverage the strength of the existing Smallworld technical community, who are on the whole very loyal to Smallworld and have a strong interest in seeing it continue to be successful, this could be a very powerful thing. As I noted in my previous post, one of the challenges for GE is that to develop new applications on two platforms concurrently (Smallworld and the new Oracle/Java based platform) will significantly decrease the amount of new functionality they can develop with a finite set of resources.
Of course there are a lot of challenges for an existing commercial software company in making such a switch. A critical one for GE would be that they could maintain their existing maintenance revenue from the Smallworld user base (at least to a large degree), otherwise it would be a non-starter from a business perspective I think. But it is quite conceivable that this approach could harness the talents of a much larger group of developers than Smallworld currently has working on the product, and produce more enhancements and fixes to the products than customers currently see, so you can see scenarios in which customers would be happy to continue to pay maintenance for support. As Paul points out, Autodesk has made this work and most of their customers still pay maintenance. There are differences in the Autodesk scenario as they open sourced a new product rather than an old one, and I think that a big driver for them was that they saw that it was going to be increasingly hard to make money for basic web mapping products, as that area becomes increasingly commoditized, due to the efforts of both the Googles and Microsofts of the world as well as the open source community. I think that this factor probably helped them decide to make the big leap to an open source approach, and it seems to have been a successful one for them.
Could GE also consider open sourcing portions of the new product, which I think was really Paul's question? That could also be an interesting possibility, to help gain traction in the market. If they open sourced some of the more generic base level components of the product, they could still build their specific business applications on top of these components and charge money for those, but leverage the open source community to provide additional functionality.
So the motivations are somewhat different for GE than for Autodesk, and there are multiple scenarios where open source could play a role, but I could envision an open source version of Smallworld significantly extending the product's life, by harnessing the very good technical resources which exist in the broader Smallworld community. Internal GE resources could then be freed up to work on the new product line (with just a small number focused on managing the open source developments of the established Smallworld product line). There are certainly a number of challenges and a lot of complexity in making something like this fly from a business perspective, and I'm not sure if GE would have the imagination or the boldness to take a risk on it, but it's an interesting thing to speculate about!
Monday, July 2, 2007
The secret behind iPhone mania - iLaunch
Well, so after all the hype Apple is estimated to have sold around 500,000 iPhones over the weekend. I only discovered today that this success was largely due to Apple's iLaunch product, announced in the Onion in March - somehow I missed this previously! There is of course plenty of other spoof coverage out there to choose from, like this:
Overall I would say that the reviews were very positive, albeit with various caveats about missing functionality and features - there's a good roundup at Time. I would agree with Hiawatha Bray in the Boston Globe who says "For it's not just cool; this phone is important, in the same way that Apple's first Macintosh computer was important. The Mac showed us a better way to interact with computers, and forced the entire industry to follow its lead."
Issues with Google local search are on other platforms too
I played around with the local search capabilities on Google Maps Mobile on my BlackBerry 8800, and found that it also has the same issues that I discussed with local search on the iPhone. I guess I just hadn't used local search much with Google on my BlackBerry, as I'd been using Telenav which has richer functionality (in particular voice driving directions using the GPS).
Sunday, July 1, 2007
Google Maps local search on the iPhone really has some serious flaws
Yesterday I posted my main review of Google Maps on the iPhone, as well as my general impressions of the iPhone. To say some positive things first, I really think that while I've commented on a few areas for improvement, I really think the iPhone is great - it's definitely a jump forward in terms of user interfaces, and I think introduces a lot of ideas in this regard which will become much more widely used. It's just a lot of fun to use. On Google Maps specifically, I talked about a lot of things I liked yesterday, and one other nice feature I have discovered is that the Bookmark page also has a "Recents" tab which saves all your recent searches, routes, etc, and will restore those with all the context - this is especially nice for retrieving routes. Oh, and the built in camera, which I had low expectations of, actually isn't too bad - here are a few sample snapshots in case you're interested.
However, I expressed some reservations about the way that local search was implemented yesterday, and having played around with this some more I really feel that there are some serious issues that need addressing here. There are two main issues: one is the way that the functionality has been implemented, and the other is the quality of the data for points of interest.
Let's take the functionality first. Frankly, Google seems to have lost track of the basic user scenario for local search. I suspect that probably a (conscious or unconscious) contributor to this was the desire to include establishments which paid for placement higher in the list of results, as I commented on yesterday, without it being too obvious that they are doing this. But anyway, the basic scenario for local search is that you identify a location, which might either be where you are now (the most common scenario), or perhaps an address where you plan on being in the future, or where a friend is located. You then search for something close to that place - this might fairly specific, like "tattered cover" (a bookstore I used in my example yesterday) or might be more generic, like "coffee". In either case, knowing how far away each of the search results are from the specified location is a really crucial piece of information in deciding whether that search result is of interest to you. Sorting the results by distance, and knowing that the closest (or the closest 3, or 5, or whatever) results have been shown to you are fundamental requirements. And then if you decide that one of the search results is of interest to you, by far the most common follow up action is that you want to figure out how to navigate from the point you first specified to the search result you have selected.
Most other local search implementations I have used meet these requirements pretty well, but local search on the iPhone fails pretty badly. Using the TeleNav software on my BlackBerry 8800, I choose "Find business" and the first thing it asks me for is the search point, and I have six options for specifying this: current location (which it knows from the GPS, which is of course its big advantage over the iPhone), recent addresses, favorites, key in an address, near airport, or contacts. You then specify the search term, and are immediately given a list of results sorted in order of distance from the search point, and showing how far away each result is. You click on a result and it shows more information including the address, and gives you the option to drive to it, save it, map it, or call it. The one feature which TeleNav does not provide which the iPhone does is the ability to display multiple results on a map at the same time - but it meets the main user scenario much better.
Google Maps online also addresses this user scenario well. You type in the address of the point you are interested in, and hit "Search maps". Then you choose "Find businesses" and you now have two text boxes, one containing the address of the search point, and one for your search term. You hit search and get an ordered list back, by distance, with the distance from the search point displayed for each result. If you click on a search result and ask for directions, the start point is already filled in with the address of the original search point.
But all this goes by the board in Google Maps on the iPhone. You will typically precede a search by centering the screen on a location, most often by typing an address (or retrieving a bookmark). But that point is not explicitly used by the search on the iPhone. Instead, you just get back any ten results which satisfy your search term and which are somewhere within the bounds of the current screen display - except that it's not just any ten, it seems that there is a ranking order which is presumably determined by how much establishments have paid for placement. There is no guarantee that the closest (or closest 3, or 5, etc) are returned. There is no easy way to find out how far away a given search result is from the point you were interested in (the only way I have found is to select a result, display its details, choose "directions to here", and then you have to fill in the start address, which you can do by selecting a bookmark or recent search address, and then calculate a route - and you have to repeat this for every item whose distance you want to know).
I tested a couple of scenarios where there were fewer than ten results in the original map display window, and in this case the map window stays the same and displays pins for the results which are in that area, but you still get a list of ten items (not sure if these are guaranteed to be the ten closest in this situation or not). A bad thing in this scenario is that if you select an item from the list which was not in the original map window, it pans you to that location, but now you have no idea how that relates geographically to your original search point (beyond re-entering the original point and calculating a route).
If there are no search results in the original map window, the map window zooms out so that it fits in all the results (at least in the cases I tried). However, there is no symbol on the map showing where your original point was, so you risk losing your orientation in this situation if it's an area you don't know well, unless you calculate a route from your original point again.
So functionality wise I really think someone lost the plot here. Yes, if you are zoomed in close (i.e. at the default scale which is displayed which you search for an address), the system will find you some results that are "sort of close" to the address you typed, and you get a nice pretty display with animated flying pins. But there appears to be no guarantee that the closest result will definitely be returned (though it probably will in most cases, especially if you are zoomed in close), there's no easy way of determining how far away any given result is (you have to calculate routes individually), there is no way at all of sorting results in order of distance, and in order to calculate a route to a search result you have to re-enter the original search point again (which you can admittedly do via a bookmark or recent result, but this still adds at least two or three clicks which shouldn't be necessary). One other thing I noticed while thinking about distances in this context is that there is no scale bar on the map, so you can't even estimate graphically. I know it's not a big screen, but all in car navigation systems I have used have some nice compact scale bar, so I think this should be added in there too.
So anyway, that's the functionality rant, sorry about that :) !! Now for the data rant!
These issues are exacerbated by the fact that Google's point of interest data really seems to have a lot of issues. I mentioned some examples in yesterday's post, but categories of error that I have found in more than half of my test cases so far include incomplete addresses (either no street number, or no street), establishments which have closed, and incorrect categorization (examples below). I tested Google Maps online and it had the same errors (which is at least consistent - you would hope that they use the same data). For comparison, I went to try MapQuest (online) and it did MUCH better. Here are just a few example searches I tried, centered on 1792 Wynkoop St, Denver, CO:
So anyway, to finish on a not entirely negative note, let's reiterate a few good things. Overall, the iPhone rocks! Google Maps on the iPhone has a lot of great features - the quality of the map display is great, panning and zooming using the touch screen is really fast and intuitive, routing works well and the way that it saves the full state of complex operations including routing and searches in "Recents" is really nice. BUT, the local search really doesn't match up to the high standards that we all expect from Google. Yes, it will give you what you want in many cases, but is missing key functionality like displaying and sorting by distance, common workflows like navigating to the point of interest you want to go are much more complex than they should be, and in my tests I have encountered way too many data errors to be comfortable relying on it. The good news is that fixing the functionality really shouldn't be difficult technically, though I think it will need Google to accept that sponsored results should be displayed separately somehow (as they are with regular Google search). Fixing the data problems may be a bigger effort, but others have done it as the MapQuest comparison shows.
Fixing local search on Google Maps for the iPhone is now top of my wish list, but I also still think that auto-correction of typing and rotation of the map are high priorities, as I mentioned yesterday. That's enough iPhone reviewing for this weekend I think :) !!
However, I expressed some reservations about the way that local search was implemented yesterday, and having played around with this some more I really feel that there are some serious issues that need addressing here. There are two main issues: one is the way that the functionality has been implemented, and the other is the quality of the data for points of interest.
Let's take the functionality first. Frankly, Google seems to have lost track of the basic user scenario for local search. I suspect that probably a (conscious or unconscious) contributor to this was the desire to include establishments which paid for placement higher in the list of results, as I commented on yesterday, without it being too obvious that they are doing this. But anyway, the basic scenario for local search is that you identify a location, which might either be where you are now (the most common scenario), or perhaps an address where you plan on being in the future, or where a friend is located. You then search for something close to that place - this might fairly specific, like "tattered cover" (a bookstore I used in my example yesterday) or might be more generic, like "coffee". In either case, knowing how far away each of the search results are from the specified location is a really crucial piece of information in deciding whether that search result is of interest to you. Sorting the results by distance, and knowing that the closest (or the closest 3, or 5, or whatever) results have been shown to you are fundamental requirements. And then if you decide that one of the search results is of interest to you, by far the most common follow up action is that you want to figure out how to navigate from the point you first specified to the search result you have selected.
Most other local search implementations I have used meet these requirements pretty well, but local search on the iPhone fails pretty badly. Using the TeleNav software on my BlackBerry 8800, I choose "Find business" and the first thing it asks me for is the search point, and I have six options for specifying this: current location (which it knows from the GPS, which is of course its big advantage over the iPhone), recent addresses, favorites, key in an address, near airport, or contacts. You then specify the search term, and are immediately given a list of results sorted in order of distance from the search point, and showing how far away each result is. You click on a result and it shows more information including the address, and gives you the option to drive to it, save it, map it, or call it. The one feature which TeleNav does not provide which the iPhone does is the ability to display multiple results on a map at the same time - but it meets the main user scenario much better.
Google Maps online also addresses this user scenario well. You type in the address of the point you are interested in, and hit "Search maps". Then you choose "Find businesses" and you now have two text boxes, one containing the address of the search point, and one for your search term. You hit search and get an ordered list back, by distance, with the distance from the search point displayed for each result. If you click on a search result and ask for directions, the start point is already filled in with the address of the original search point.
But all this goes by the board in Google Maps on the iPhone. You will typically precede a search by centering the screen on a location, most often by typing an address (or retrieving a bookmark). But that point is not explicitly used by the search on the iPhone. Instead, you just get back any ten results which satisfy your search term and which are somewhere within the bounds of the current screen display - except that it's not just any ten, it seems that there is a ranking order which is presumably determined by how much establishments have paid for placement. There is no guarantee that the closest (or closest 3, or 5, etc) are returned. There is no easy way to find out how far away a given search result is from the point you were interested in (the only way I have found is to select a result, display its details, choose "directions to here", and then you have to fill in the start address, which you can do by selecting a bookmark or recent search address, and then calculate a route - and you have to repeat this for every item whose distance you want to know).
I tested a couple of scenarios where there were fewer than ten results in the original map display window, and in this case the map window stays the same and displays pins for the results which are in that area, but you still get a list of ten items (not sure if these are guaranteed to be the ten closest in this situation or not). A bad thing in this scenario is that if you select an item from the list which was not in the original map window, it pans you to that location, but now you have no idea how that relates geographically to your original search point (beyond re-entering the original point and calculating a route).
If there are no search results in the original map window, the map window zooms out so that it fits in all the results (at least in the cases I tried). However, there is no symbol on the map showing where your original point was, so you risk losing your orientation in this situation if it's an area you don't know well, unless you calculate a route from your original point again.
So functionality wise I really think someone lost the plot here. Yes, if you are zoomed in close (i.e. at the default scale which is displayed which you search for an address), the system will find you some results that are "sort of close" to the address you typed, and you get a nice pretty display with animated flying pins. But there appears to be no guarantee that the closest result will definitely be returned (though it probably will in most cases, especially if you are zoomed in close), there's no easy way of determining how far away any given result is (you have to calculate routes individually), there is no way at all of sorting results in order of distance, and in order to calculate a route to a search result you have to re-enter the original search point again (which you can admittedly do via a bookmark or recent result, but this still adds at least two or three clicks which shouldn't be necessary). One other thing I noticed while thinking about distances in this context is that there is no scale bar on the map, so you can't even estimate graphically. I know it's not a big screen, but all in car navigation systems I have used have some nice compact scale bar, so I think this should be added in there too.
So anyway, that's the functionality rant, sorry about that :) !! Now for the data rant!
These issues are exacerbated by the fact that Google's point of interest data really seems to have a lot of issues. I mentioned some examples in yesterday's post, but categories of error that I have found in more than half of my test cases so far include incomplete addresses (either no street number, or no street), establishments which have closed, and incorrect categorization (examples below). I tested Google Maps online and it had the same errors (which is at least consistent - you would hope that they use the same data). For comparison, I went to try MapQuest (online) and it did MUCH better. Here are just a few example searches I tried, centered on 1792 Wynkoop St, Denver, CO:
- King Soopers (a local supermarket chain): in the top 10 on Google, there were three entries with an incomplete address (just Denver, CO), all of which show as being closer than the closest real King Soopers, and one entry which says in Google Maps online that it is an "unverified listing" (at 1150 Stout St) - but on the iPhone this just appears in the list without any indication that it is unverified (this one does not appear in MapQuest and I am pretty sure that it is not a real location). So 4 out of 10 results are wrong (non-existent), therefore you would have a 40% chance of ending up somewhere with no King Soopers. On MapQuest, 10 legitimate addresses are returned, though in some cases they appear to have multiple addresses for the same store (e.g. entry points on two different streets) - but this is not a serious error as you would still find a King Soopers.
- Tattered Cover (a well known Denver book store with 3 locations, which I mentioned in my post yesterday). MapQuest returns just 3 results, all correct. Google Maps online returns 8 entries in its initial list, of which two have incomplete addresses (duplicates of entries which do have complete addresses, but they show up at entirely different locations on the map), one is a store which closed a year ago, one says it is unverified and is in Boulder (where there is no Tattered Cover), and one is a duplicate of a correct store. The same 8 entries appear on the iPhone, again with no indication that the Boulder one is unverified - so in this case you have a 50% chance of not showing up at a Tattered Cover.
- Searches for Office Depot and Home Depot were more successful, with no obvious errors - hooray :) !!
- Searching for grocery in Google Maps online returned Market 41, a nightclub which closed a year ago, and four entries with incomplete addresses (two King Soopers which we saw before, and two Safeways). The iPhone returned a different list in this case, but included Market 41, two entries with incomplete addresses (one the same, one different), and Lemon Sisters market, which was a small grocery store but closed several years ago. So a 40% chance of going to the wrong place in this case. The same search on MapQuest did not return Market 41 and had 10 complete addresses, all of which looked legitimate.
So anyway, to finish on a not entirely negative note, let's reiterate a few good things. Overall, the iPhone rocks! Google Maps on the iPhone has a lot of great features - the quality of the map display is great, panning and zooming using the touch screen is really fast and intuitive, routing works well and the way that it saves the full state of complex operations including routing and searches in "Recents" is really nice. BUT, the local search really doesn't match up to the high standards that we all expect from Google. Yes, it will give you what you want in many cases, but is missing key functionality like displaying and sorting by distance, common workflows like navigating to the point of interest you want to go are much more complex than they should be, and in my tests I have encountered way too many data errors to be comfortable relying on it. The good news is that fixing the functionality really shouldn't be difficult technically, though I think it will need Google to accept that sponsored results should be displayed separately somehow (as they are with regular Google search). Fixing the data problems may be a bigger effort, but others have done it as the MapQuest comparison shows.
Fixing local search on Google Maps for the iPhone is now top of my wish list, but I also still think that auto-correction of typing and rotation of the map are high priorities, as I mentioned yesterday. That's enough iPhone reviewing for this weekend I think :) !!
Saturday, June 30, 2007
Review of Google Maps on the iPhone
Okay, so here it is, the review of Google Maps on the iPhone. As I said in my previous post giving general impressions of the iPhone, the maps look great and panning and zooming is very interactive and intuitive, but there are a number of little niggly details that counterbalance the cool parts leaving me somewhat neutral - impressed by some things, but overall left feeling that it could have been better (and this is aside from the obvious drawback of not having a GPS). So maybe "good but not as great as it could have been" is the overall verdict.
I did most of my testing using my WiFi connection, and then turned this off and used EDGE for a little while. Performance was excellent using the WiFi - it typically took around a second to download data for an uncached area, for either a street map or imagery, but up to two seconds or so occasionally, more so with imagery than street maps (which you would expect as the compressed images for street maps are typically smaller in size). The iPhone keeps a pretty good sized cache, and when data is cached display is pretty much instant, just as it is online. So overall the experience of panning and zooming around is very smooth and fast. Using EDGE instead of WiFi, I found that the download of data for an uncached area was generally taking around 3-4 seconds for street maps, and 5-7 seconds for imagery. But when data was cached, it was just as fast as with WiFi, as you would expect. So it was slower, but still fast enough to give a pretty good user experience, in my opinion. When zooming in or out, it animates the zoom using the data layer that is already there, so you see some sort of data while waiting for the data at the new scale to download.
The user interface for panning and zooming is intuitive once you learn the basics. Double tap to zoom in on a point (it doesn't center the point, but does keep that point on the screen - so if you double click on something in the northeast corner, after you zoom in it will still be in the northeast corner). Tapping with two fingers (at any distance apart and any orientation) will zoom out. You can also zoom in or out by "pinching" with two fingers and moving them closer together or further apart, which is a really intuitive interface which leverages the multi-touch capabilities.
One thing I really didn't like about the basic map display is that you can't rotate it. With other applications like the browser and photos, you just change the rotation of the iPhone from portrait to landscape, and the application automatically rotates the display (with a nice bit of animation). This is such a natural thing to want to do with a map display, I'm pretty disappointed that they didn't implement this - hopefully it will be there in a future release. This is one of multiple places where Google really didn't take advantage of some of the good features in other iPhone applications, and which makes for a less seamless and intuitive user experience as it doesn't do some things that you hope, and expect, it to do.
For the next set of observations, I took a pretty detailed set of photos which you can see here on flickr. Follow this through as a slide show to look at some search scenarios.
By and large, search worked pretty much as expected - you can type in a specific business name like "Tattered Cover" (my excellent local bookstore), or a generic term like "coffee", and get appropriate results back. But I did find a few things I hadn't anticipated, as follows:
The real time traffic works well (I have the same thing on the BlackBerry also). Here are pictures showing central Denver, with and without traffic information, and you can see that the southbound carriageway of I-25 is currently congested.
The routing also seems to work well - in the picture gallery I show creation of a route from Denver to Vancouver, which was pretty much instantaneous, and you can list turn by turn directions either as text or on the map, and you can easily skip backwards and forwards in the turn list.
So general conclusion: I like the application, it's fast and intuitive in most regards, but does have a few things that need fixing. My order of priority on these would be:
I did most of my testing using my WiFi connection, and then turned this off and used EDGE for a little while. Performance was excellent using the WiFi - it typically took around a second to download data for an uncached area, for either a street map or imagery, but up to two seconds or so occasionally, more so with imagery than street maps (which you would expect as the compressed images for street maps are typically smaller in size). The iPhone keeps a pretty good sized cache, and when data is cached display is pretty much instant, just as it is online. So overall the experience of panning and zooming around is very smooth and fast. Using EDGE instead of WiFi, I found that the download of data for an uncached area was generally taking around 3-4 seconds for street maps, and 5-7 seconds for imagery. But when data was cached, it was just as fast as with WiFi, as you would expect. So it was slower, but still fast enough to give a pretty good user experience, in my opinion. When zooming in or out, it animates the zoom using the data layer that is already there, so you see some sort of data while waiting for the data at the new scale to download.
The user interface for panning and zooming is intuitive once you learn the basics. Double tap to zoom in on a point (it doesn't center the point, but does keep that point on the screen - so if you double click on something in the northeast corner, after you zoom in it will still be in the northeast corner). Tapping with two fingers (at any distance apart and any orientation) will zoom out. You can also zoom in or out by "pinching" with two fingers and moving them closer together or further apart, which is a really intuitive interface which leverages the multi-touch capabilities.
One thing I really didn't like about the basic map display is that you can't rotate it. With other applications like the browser and photos, you just change the rotation of the iPhone from portrait to landscape, and the application automatically rotates the display (with a nice bit of animation). This is such a natural thing to want to do with a map display, I'm pretty disappointed that they didn't implement this - hopefully it will be there in a future release. This is one of multiple places where Google really didn't take advantage of some of the good features in other iPhone applications, and which makes for a less seamless and intuitive user experience as it doesn't do some things that you hope, and expect, it to do.
For the next set of observations, I took a pretty detailed set of photos which you can see here on flickr. Follow this through as a slide show to look at some search scenarios.
By and large, search worked pretty much as expected - you can type in a specific business name like "Tattered Cover" (my excellent local bookstore), or a generic term like "coffee", and get appropriate results back. But I did find a few things I hadn't anticipated, as follows:
- My search for the Tattered Cover yielded 3 incorrect locations, in addition to the (only) three correct ones. One was a location of an old store which closed a year ago, and two were the result of incomplete addresses for current stores (in addition to complete ones), one which just included the street with no number, and one which included the town but no street. I may have just been especially unfortunate with my choice of example, but this illustrates the importance of good and up to date data in any LBS application. An out of town visitor in the south side of Denver looking to visit the famous Tattered Cover bookstore could easily have driven to three locations, none of which had a Tattered Cover, by which point they would feel about Google Maps like I do about AT&T right now (see previous post)!
- Google Maps does not include the autocorrection feature for typing which is in all (or at least most) of the other iPhone applications. This is really bad. Typing is somewhat fiddly with the on screen keyboard, but you can go pretty fast if you have the autocorrect feature, and you get used to it in all the other applications. You have to type much more slowly and deliberately in Google Maps than anywhere else. This really needs fixing!
- When you search for items, it doesn't order them by distance from your location, neither does it show the distance to a search result in the list, which is very common in these type of applications and I think this is a serious omission too. On further investigation, I have come to the conclusion that the order in which results are returned is almost certainly determined by payment for higher placement - I found that the "Market" coffee store was consistently returned at the top of the list, and highlighted on the map, for a wide variety of different spatial queries in downtown Denver. This is not necessarily surprising - at some point Google needs to make some money back for all its investment in Google Maps and Earth, or it won't keep on investing - but it is good to know about. And if they remove the very useful function of showing how far away different search results are to hide the fact that they are (apparently) returning establishments which pay higher up the list, I'm not too happy about that. I will do a separate post about this as I think it's sufficiently interesting to highlight. You can see the detailed examples relating to this in the photo gallery.
The real time traffic works well (I have the same thing on the BlackBerry also). Here are pictures showing central Denver, with and without traffic information, and you can see that the southbound carriageway of I-25 is currently congested.
The routing also seems to work well - in the picture gallery I show creation of a route from Denver to Vancouver, which was pretty much instantaneous, and you can list turn by turn directions either as text or on the map, and you can easily skip backwards and forwards in the turn list.
So general conclusion: I like the application, it's fast and intuitive in most regards, but does have a few things that need fixing. My order of priority on these would be:
- Needs to support autocorrection when typing
- Should be able to display search results in order of distance from the center of the screen (and show distances on the list) - even if this is an option, and the initial order is determined "at Google's discretion")
- The map display should rotate when you rotate the iPhone, as with other applications where this makes sense
Subscribe to:
Posts (Atom)