Traditional GIS vendor market share for 2008-2009

Daratech has published its annual analysis of the GIS industry. I thought there were a few interesting things worth commenting on there. First I should say though that I am generally somewhat skeptical of these type of reports - there is a lot of subjectivity in what gets included and what doesn't. One illustration of this was when Bentley suddenly jumped to the number 2 position ahead of Intergraph a couple of years ago - and apparently kept that for a second year. While I was clearly not in an impartial position at that time (being CTO of Intergraph), this went against the intuition of myself and others I talked to in the industry - the general feeling was that somehow Bentley had persuaded Daratech to count a lot of "CAD" revenue as "GIS". And to be fair, there is a lot of debate about how much of Intergraph's revenue is "GIS" and should be counted here. For example, Intergraph does a lot of business providing 911 call taking systems in the Public Safety space, which have a strong geospatial element to them - do you or don't you count those in this number? There is a good argument in favor, but it might not be what everyone calls "GIS". In this year's version, Intergraph's revenue is twice that of Bentley (which is hard to reconcile with Bentley being second last year).

Anyway, all those caveats notwithstanding, the top three vendors for 2009 are projected to be ESRI with 30%, Intergraph with 16%, and GE Energy (Smallworld) with somewhere around 8% (exact figure not stated in this summary, approximate figure deduced from chart). No surprise to see ESRI and Intergraph in the top two spots, which has been the generally accepted state of affairs for most industry observers for a long time. From a personal perspective it is nice to also see Smallworld (GE) still in the number 3 spot, where they have been on and off - there are several others vying for that (so my former companies get 2 of the top 3 slots!). The report also says that GE/Smallworld has top position in utilities with 24% of the market - almost as strong as ESRI's position in the overall market.

I think it's interesting though that ESRI's share (which is consistent with previous reports) is actually lower than many people perceive. ESRI clearly is the dominant player in the traditional GIS space, they enjoy an effective monopoly in many markets (for example see Andrew's recent post on a US Air Force sole source bid), and their "mindshare" is pervasive. Many people I talk to, especially those in the "neo" world, assume that "everyone" doing "traditional GIS" is using ESRI. But what this report says is that 70% of people are not using ESRI (well 70% of the revenue comes from people not using ESRI, which is not necessarily the same thing - but nevertheless, a lot of people are not using ESRI). But you have to give ESRI credit for the way they have achieved such "thought domination" with "only" a 30% market share. I often think that there are several interesting comparisons to draw between the dominant positions of ESRI and Microsoft in their respective markets (maybe there is more for a future post there). Microsoft has operating system market share of 90%-ish, and Apple somewhere around 8%, but Apple has arguably more thought leadership and buzz around its offerings than Microsoft does. But there is very little of that sort of alternative thought widely seen in the traditional GIS space, even though Intergraph and Smallworld still have certain areas where they have technical advantages and/or alternative approaches to offer compared to ESRI.

However, we are now seeing more diversity of thought reaching a broader audience from the "neogeography" side of the house, which is a good thing for the industry. And this leads into the other area that I wanted to comment on, which is how it is increasingly hard to do this kind of market share analysis on the geospatial market, as geospatial technology becomes more imbedded in mainstream IT. As geospatial data becomes just another data type and becomes an element of many different applications, how do you say that a given application is "geospatial" or not? There is no place on this list for Google, Microsoft or Oracle, for example, all of whom clearly play a major role in the geospatial market these days. And there is nothing that captures the strong growth that I perceive in open source geospatial software (by definition, software revenue is not going to be a good metric for measuring the market share of free software!).

So overall, while there are certainly some interesting tidbits in this Daratech report summary, it is not at all reflective of the overall state of the broader geospatial industry, where it is increasingly hard to even define the market, let alone to measure market share in a quantitative way.

For those that are interested in quantitative analysis of the "traditional geospatial" market, I would also suggest considering the annual Geospatial Technology Report from GITA, which uses an alternative approach based on survey responses from GITA member organizations. There are potential flaws in that methodology too, but in many ways it is less subjective than something based on vendor revenues, and it provides a lot of additional detail on various aspects of how people are using the technology, which I have found interesting in the past. It's also a lot cheaper! (I should add for full disclosure that I am a former member of the board of directors of GITA, but I receive no remuneration for sales of GITA reports!).

Update: Matt Ball has a good post discussing the relevance of these type of industry reports.

What is the Smart Grid?

Here is the second (of two) short video presentations on what the Smart Grid is, at a high level. This follows on from my previous one on why we need a Smart Grid - you should watch that one before watching this one.

In future installments I'll talk more about some of the IT challenges in making the Smart Grid happen, the impact of geospatial technology / GIS on the Smart Grid and vice versa.

The Smart Grid, Part 2 from Peter Batty on Vimeo.

WhereCamp5280 is almost upon us!

As I have posted about previously, I'm helping to organize a cool, and FREE, "geo-unconference" in Denver called WhereCamp5280 (5280 = the altitude of Denver in feet, for those not from round here!), which is now almost upon us - it's this Friday and Saturday, August 14-15. We've been really pleased with the response - check out the list of likely attendees and potential talks (being an unconference, the schedule isn't fixed in advance - and there will be a lot more talks and discussion topics added "on the fly" during the event).

We initially thought it would just be a local event, but we have quite a lot of distinguished visitors from around the country, including founder of OpenStreetMap Steve Coast, uber-geoblogger James Fee, Executive Director of OSGeo Tyler Mitchell, ESRI product gurus Bern Szukalski and Victoria Koujoumjian from Redlands, and more. We have a lot of founders of local geo startups coming along, including Martin May of Brightkite, Andrei Taraschuk of UMapper, Duncan McCall of PublicEarth, Charlie Savage of Zerista, Tom Churchill of Churchill Navigation, and Brian Timoney of the Timoney Group. Oh and me :). We have geogeektv stars Dave Bouwman and Brian Noyle. And loads of local geospatial software users and developers who are doing lots of cool things with all kinds of technologies - and sorry to all the attendees I haven't mentioned by name ... I guess I really shouldn't have started this list, it will get me into trouble with someone! (The beer's on me at the social event if I missed you off the list!).

Talking of which, we've also had a great response from sponsors, which is really gratifying in the current economy - thanks a lot to our gold sponsor ESRI, silver sponsors Bohannan Huston, DTSAgile and PublicEarth, bronze sponsors Enspiria Solutions, A Mountain Top and UMapper, media sponsor AnyGeo / GISuser, and facilities sponsor GTAC at DU.

I'm looking forward to a really fun and interesting event. The sessions will be at University of Denver in Sturm Hall, and we'll have a fun social event on Friday night at my loft above the Wynkoop Brewing Company. There's still time to sign up - hope to see you there!

Why do we need a Smart Grid?

In my role as Chief Technology Advisor at Enspiria Solutions, I've recently been researching the Smart Grid. I've found that even a lot of people working in the electric industry have somewhat fuzzy ideas about it, or understand some aspects but not others. I've put together some short presentations on the subject. The first two you might call "Smart Grid 101" - the first one, below, talks about why we need a Smart Grid, and the second one, which I'll publish shortly, talks about what the Smart Grid is, at a high level. I've tried to make these understandable without any particular knowledge of the electric industry. In subsequent talks I'll discuss some of the IT challenges in implementing the Smart Grid, and look in particular at how the Smart Grid will impact geospatial technology, and vice versa.

So here's part 1, it's about ten minutes long ...

The Smart Grid, Part 1 from Peter Batty on Vimeo.

Update - see Part 2 on What is the Smart Grid?.

Netezza announces new architecture with 10-15x price-performance improvement

I have previously discussed Netezza, who produce data warehousing appliances that provide outstanding performance and simplicity for complex analytics on very large data volumes. I did some consulting work with them last year as they added spatial capabilities to their system. Today they announced a major new architecture, which they say gives a 3-5x performance improvement for typical workloads (more for some operations, less for others), and reduces price per terabyte by a factor of 3. So overall price performance improves by a factor of 10-15. Database guru Curt Monash has a good discussion of the new architecture and pricing implications on his blog.

The new hardware architecture is more flexible than the old one, which makes it easier to vary the proportions of processor, memory and disk, which will allow them to provide additional product families in future:

• High storage (more disk, lower cost per terabyte, lower throughput)
• High throughput (higher cost per terabyte but faster)
• Entry level models

I think that the entry level and high throughput models will be especially interesting for geospatial applications, many of which could do interesting analytics with a Netezza appliance, but may not have the super large data volumes (multiple terabytes) that Netezza's business intelligence customers have. Another interesting change for the future is that Netezza's parallel processing units (now Snippet blades, or S-blades, formerly snippet processing units or SPUs) are now running Linux, whereas previously they were running a rather more obscure operating system called Nucleus. In future, this should make it easier to port existing analytic applications to take advantage of Netezza's highly parallel architecture (though this is not something that is available yet). The parallel processing units also do floating point operations in hardware rather than software, which should also have a significant performance benefit for their spatial capabilities.

I continue to think that Netezza offers some very interesting capabilities for users wanting to do high end geospatial analytic applications on very large data volumes, and that there will be a lot of scope for its use in analyzing historical location data generated by GPS and other location sensors. And I am just impressed by anyone who produces an overnight 10-15x price performance improvement in any product :) !

GeoWeb 2009 review

The GeoWeb 2009 conference was very good as usual. The presentations I attended were a bit of a mixed bag - some excellent, a few so so - but the networking and hallway conversations were great. James has done a good writeup. I'll just comment on a few things that I thought were interesting.

One trend was increasing acknowledgment that the traditional approach to SDI (Spatial Data Infrastructure) using OGC standards is really not working well in comparison to newer "geoweb" type mechanisms for sharing data that have come from the "neogeography" developments of recent years (I won't get back into the label debate just now!). This was most striking for me in the presentation by Clemens Portele, who is a highly respected member of the OGC community - he won the Gardels Award in 2007. Clemens gave an excellent and very frank talk on "The gap between SDIs and the GeoWeb". A couple of quotes that I noted were "So far the impact of SDIs on the integration of data as a ubiquitous component of the web seems low" and "There is not evidence that SDIs have increased the market volume of government data by a significant amount". He noted that OGC web services are based largely on an architecture and approach to web services developed around 10 years ago, well before many recent web innovations. Another interesting comment was on metadata - he observed that there is much more interest in user-centric and user-provided metadata – did others use the data? Who? What worked, what didn’t? - than in "traditional" metadata. This is not a new debate, but it was interesting to see someone who has invested so much in OGC and who is so well respected in OGC circles taking a very "neo" view of the world (for lack of a better short label!). My friend Carl Reed, CTO of OGC, was moderating the session and commented at the end that OGC was aware of the issues with not having a simpler API and were looking at them, but the solutions were complex.

Andrew Turner gave an excellent talk on geoweb standards which you can see at geogeektv (kudos to Dave Bouwman for broadcasting a number of presentations - we really should see a lot more live transmission and recording of presentations, all you need is a cheap webcam and and Internet connection). There were some interesting dynamics as the talk was introduced by Ron Lake, the father of GML, and Andrew discussed his views on shortcomings of GML pretty frankly! Andrew observed that we are lacking a "middle ground" in web standards between the simple (GeoRSS, KML) and the complex (GML, Shape).

Update: Ron responded via twitter (as I said recently, hardly anyone seems to comment on blogs any more!), and this is a summary of his points (in bullets of 140 characters or less :)

• Good review. I also thought that there was a growing realization of limits/roles of things like KML, GeoRSS outside mapping.
• I think GML already fills that middle ground although I agree that the spec is daunting!
• It also does not reflect the fact that one can profile GML (take subsets) - can be small like lo-pidf proposed in ipv6!
• Or a bit larger like geoRSS GML - which is only slightly more complex than geoRSS "simple"
• That is one of the things that struck me also - the range of views on what constitutes the "web" stood out very clearly (this was in response to my comment that one of the interesting things about GeoWeb is bringing together a range of perspectives)
• Many versions of the web to consider. The web of hyperlinked documents. The web of database access. The web of real time collaboration ..
  

Thanks for the comments Ron, definitely an interesting debate!

I think we have seen in this area and others that on the web especially (and probably in computing / life in general!), simple is good. We have repeatedly seen greater success in the technology space with an approach that says do something simple and useful that covers maybe 80% of what you want to do eventually, with 20% of the effort, get it out there and being used, and iterate from there. Versus an approach which tries to tackle all the complex cases up front, takes much longer to implement, and ends up being complex to use - which is not uncommon in a consensus based standards process. OpenStreetMap is another example of the "keep it simple" philosophy which has been very successful.

Another general trend was a lot of discussion on how the "geoweb" is not just about running geospatial applications on the web, but making applications and geospatial data fit in with key web concepts like search and linking. Jason Birch gave a good presentation on the work he is doing at Nanaimo, which is discussed on his blog and by James. To see an example, do a Google search for 2323 Rosstown Rd. A tweet I liked from Kevin Wiebe said "If a dataset available on the web is in a format that can't be indexed by Google, does it make a sound?". Clearly this approach to making geospatial data searchable is hugely useful, and there are places where making data linkable is very useful too. I think there is a question about how far the linking paradigm can go with geospatial data though - Mano Marks from Google has an interesting discussion on this (and other related topics). Another potential item for a future post!

In many ways the "geoweb" is not a good concept (as Sean Gillies and others have observed), except as a very broad label for geospatial activity on the web (let's not get back into terminology discussions again!). But there isn't a separate "geoweb", there's one web which handles many types of data, one of which is data that has a location / geo element to it. We don't talk about the "numeric web" or the "varchar web". Alex Miller from ESRI included in his presentation a picture of the web with lots of "GIS servers" and I always have an uncomfortable feeling about that. It is often said that 80-85% of data has a location component. Does that mean that 80-85% of servers on the Internet (or in your enterprise) should be "GIS servers"? No, of course not. But any general data server should be able to support location data types. Of course there may be some servers that are especially focused on certain kinds of geospatial processing, and maybe "GIS server" is a reasonable term for those. But I think it's important to understand that the great majority of location data we use will not be coming from those specialized "GIS servers".

Michael Jones of Google gave a very animated, entertaining and thought-provoking talk as always, which you can see (cut into 10 minute chunks) on youtube (hint to GeoWeb organizers: use Vimeo, where you aren't constrained to 10 minute videos!). His major themes were the pace of change in our world, something which was a common theme from other invited speakers too, and user created data. He said "the future is user created data" and talked a lot about Google Mapmaker. He also made a big deal about the fact that Google has an internal organization called the "Data Liberation Front", whose mission is to ensure that any data you put into Google, you can get out - you see this piece here. I thought it was curious that he made such a big deal of this, since one of the primary differences between Google Mapmaker and OpenStreetMap is that you can't get the raw data (vector data with attributes) that you have created out of Google Mapmaker, but you can get this back from OpenStreetMap. Google Mapmaker just lets you display a standard Google Map created from your data, but doesn't let you get the data itself. Ed Parsons argues that getting the underlying data is "an edge case". I would agree that most users don't want to access this data directly - but I think that most users would want the result of their efforts to be more broadly available. I haven't yet used any of the raw data I've contributed to OpenStreetMap directly, but I like the fact that a whole range of third party applications can use that data for innovative purposes - things that they couldn't do if they just had access to something like the Google Maps API. I asked Michael about this at the end of his presentation, and also about his thoughts on OpenStreetMap in general (see video). He said that he didn't think it was likely that the policy on making map data available would change (it is available to non-profits but nobody else) - though he did say "perhaps for non-commercial use", so maybe there is a glimmer of hope there :). In regard to OpenStreetMap he said that Google had had conversations with them for a year or so, but couldn't resolve some legal issues - with the key problem being that contributors of data retained the rights to that data, which opens up the possibility of Google being sued by contributors if they are using that data. He made the interesting comment that "if they [OpenStreetMap] had their legal act in order we would not ever have done anything on our own" (i.e. Google MapMaker). OpenStreetMap is in the process of changing its licensing in a way which will resolve this specific issue, so it will be interesting to see whether this re-opens the possibility of collaboration (which probably some people in the OSM community would like, and others wouldn't) - though there are other licensing complexities which may be obstacles too, and it may be that the two of them are too far down separate paths now.

One other interesting snippet mentioned by Michael was that Google Maps has had 470 versions in 4 years :O !!!! There is a new one every Tuesday, and additional ones if necessary to fix a problem. That certainly puts a whole new perspective on agile development. As he said, it's hard for companies who do a software release once every year or two to keep up with innovation these days.

Chris Helms (A fellow Denverite) from the National Renewable Energy Laboratory (NREL) gave an interesting talk on an application he has developed called IMBY (In My Back Yard), which lets you look at the potential effect of installing a solar photovoltaic (PV) array or wind turbine at your home or business.

Sean Gorman gave a good overview of his efforts at FortiusOne to demystify and democratize geospatial analysis, and empower end users who are not "GIS professionals". I very much agree with his philosophy - as he says, you don't need to do a training course in statistics to use a spreadsheet, or have a qualification in graphic design to use PowerPoint.

Dale Lutz of Safe Software talked about the new opportunities they see in 3D data and BIM (Building Information Modeling) - I think this is a great new opportunity for Safe, with much greater complexity than traditional (largely) 2D geospatial data. The growth in number of formats supported by FME is always an interesting chart in Dale and Don's presentations - it has now reached 238 and continues to show linear growth since they started - and Dale said if anything it is now trending slightly above linear growth (getting into 3D data may well contribute to that). He also said in passing that his mother can handle a 3D PDF, which I thought might explain something about Dale ;) !

Just as at Where 2.0, one of the coolest demos of the show was the automatically generated 3D models done by C3 - check out some of the videos on their site.

I think that had better do, I have other things to get to - apologies to those who did other interesting presentations that I didn't mention! And if you made it this far, I should mention that I haven't forgotten my "top 10" influential people list, that will be coming soon. Quite a few of them were at GeoWeb :). Thanks to the organizers including Ron Lake, Galdos, and GITA, for another excellent event.