Review of FOSS4G

I've been up at the FOSS4G conference in Victoria this week, which has just finished (apart from the related "code sprint" tomorrow). It was really an excellent conference - congratulations to conference chair Paul Ramsey and the rest of the organizing committee for putting on a great event. The quality of the sessions I went to was consistently high, and there was a real energy and buzz around the whole event (much more than at most of the more established geospatial conferences I have been to recently). Adena Schutzberg said in her closing comments that her overall impression of the conference and the open source geospatial community was one of maturity (and she will expand on that theme at Directions magazine next week). The event reaffirmed the belief I had before coming here that the role of open source software in the geospatial industry will continue to grow quickly. And personally I enjoyed learning a lot of new things and meeting a new crowd of people (as well as quite a few old friends).

One specific aim for me in coming here was to learn more about PostGIS, which I regarded beforehand as the front runner for the database technology to use for my new company Spatial Networking. Paul Ramsey, who was a busy man this week, giving a number of very good presentations, presented an interesting set of PostGIS case studies. These included IGN, the French national mapping agency, who maintain a database of 100 million geographic features, with frequent updates from 1800 users, and a fleet management company which stores GPS readings from 100 vehicles 10 times a minute for 8 hours a day, or 480,000 records a day. In a separate presentation, Alejandro Chumaceiro of SIGIS from Venezuela, talked about a similar fleet management application with very high update volumes. Interestingly, they use partitioned tables and create a new partition every hour! Incidentally, I talked with Alejandro afterwards and it turns out that he worked for IBM on their GFIS product from 1986 to 1991, and knew me from those days - it's a small world in the geospatial community :). Kevin Neufeld from Refractions Research also gave a lot of useful hints about partitioning and other performance related topics. Brian Timoney talked about the work he has done using both Google Earth and Virtual Earth on the front end, with PostGIS on the back end doing a variety of spatial queries and reports, including capabilities like buffering, in a way which is very easy to use for people with no specialized knowledge. And Tyler Erickson of Michigan Tech Research Institute talked about some interesting spatio-temporal analysis of environmental data he is doing using PostGIS, GeoServer and Google Earth. Overall I was very impressed with the capabilities and scalability of PostGIS, and was reassured that this is the right approach for us to use at Spatial Networking.

Another topic which featured in several sessions I attended was that of data. As Schuyler Erle said in introducing a session about the OSGeo Public Geodata Committee, a key difference between geospatial software and other open source initiatives is that the software is no use without data, so looking at ways to create and maintain, share, and enable users to find, freely available geodata is also an important element of OSGeo's work, in addition to software. Nick Black, a fellow Brit, gave a good talk about Open Street Map, which is getting a lot of interest. The scope of what they are doing is broader than I had realized, including not just streets but points of interest (pubs are apparently especially well represented!), address information which can be used for geocoding, and they are working on building up a US database based on TIGER data. The ubiquitous Geoff Zeiss, a man without whom no GIS conference would be complete :), gave an interesting review of the wide variety of government policies with regard to geospatial data around the world. One curious snippet from that was that in Malaysia and some other Asian countries, you need to have an application reviewed by the police and the army before being able to receive a government-issued map! In the opening session, I enjoyed the talk by Andrew Turner of Mapufacture on Neogeography Data Collection, which was a great overview of the wide range of approaches being used for "community generated" data, including things like cheap aerial photography using remote control drones from Pict'Earth - they have a nice example of data captured at Burning Man. This was one of a number of five minute lightning talks, which went over pretty well - several people told me that they enjoyed the format. I also gave one of those, on the topic of the past, present and future of the geospatial industry, and managed to fit into the allotted time - though next time I might choose a slightly more focused topic :) ! I will write up my talk in a separate post at some point (it will take a lot longer than 5 minutes to do that though!). Ed McNierny of Topozone had the most intriguing title, "No one wants 3/8 inch drill bits" - the punchline was that what they actually want is 3/8 inch holes, and we should focus on the results that our users need, not the specific tools they use to achieve those. Schuyler Erle gave one of the more entertaining presentations on 7-dimensional matrices that I have seen (and I say that as a mathematician).

Also in the opening session, Damian Conway gave a good talk entitled "geek eye for the suit guy", on how to sell "suits" on the benefits of open source software. Roughly half his arguments applied to geospatial software, and half were more specific to Linux - Adena has done a more detailed writeup.

Brady Forrest of O'Reilly Media gave an interesting presentation on "Trends of the Geo Web". His three main themes were "Maps, Maps Everywhere", "The Web as the GeoIndex", and
"Crowdsourced Data Collection". One interesting site he mentioned that I hadn't come across before was Walk Score, which ranks your home based on how "walkable" the neighborhood is (my loft in downtown Denver rated an excellent 94 out of 100). It seems as though every time I see a presentation like this I discover some interesting new sites, and now I listen slightly nervously hoping that I don't discover someone doing what we plan to do with Spatial Networking, but so far that hasn't happened!

I also was on the closing panel for the conference, which I thought went well - we had a pretty lively discussion. The closing session also included a preview of next year's conference which will be in Cape Town, South Africa. I had the pleasure of spending a few days in Cape Town in 2002, followed by a safari in Botswana which still ranks as the best of the many trips I've done to different parts of the world (check out my pictures). So I certainly hope to make it to the conference, and highly recommend that others try to make it down there and spend some additional time in that part of the world too.

Apologies to those I missed out of this somewhat rambling account, but the Sticky Wicket pub is calling, so I will wrap it up here, for now at least.

Oh the irony

I switched to using a Mac a few months back, as I've mentioned before. I have been getting quite into Keynote, Apple's presentation tool which competes with PowerPoint, and you can do some really cool things with it that blow PowerPoint away. So I'd been working on my five minute lightning presentation for FOSS4G and it was really a thing of beauty, the transitions and animations were amazing, it had really cool special effects, videos, everything ... George Lucas would have been proud of it.

And then I hear from Paul Ramsey on Friday and he says "we need your presentation in Open Office or PowerPoint format as we'll be presenting them all from the same PC". PowerPoint? PC? Aargghhh!!! Of all the places to force me to convert my beautiful Mac presentation back into a boring old PowerPoint to run on a PC, I didn't think it would be at an open source conference ;) !! Oh well, I'll have to save the launch of my Keynote pyrotechnics for another occasion :). To be fair, Paul has had a huge amount of work organizing all sorts of things for the conference so I can quite understand him wanting to simplify the presentation logistics ... but I figured I still needed to give him a bit of a hard time!

Anyway, it will force me to focus on content versus style. I am talking about the "past, present and future of the geospatial industry" in five minutes, so will be interesting to see if I get through it all before I'm thrown off :) !!

Ignacio Guerrero retires from Intergraph

I recently heard from my friend Ignacio Guerrero that he has retired from Intergraph after 26 years there. He is going to continue doing some work in the geospatial industry though, and has set up his own consulting company called Andes Geoconsulting. Ignacio has been very involved with OGC over the years and has contributed a great deal to the industry, so I am glad to see that he will still be around, and wish him good luck with his new venture - it seems to be the season for it! He mentioned that he will be at the FOSS4G conference next week, so I look forward to catching up with him there.

Me and Ignacio in a restaurant in Poland last year

Finally, the big decision on my next career move!

Well, after taking a couple of months off and spending the past couple of months exploring various options for my next career move, I have finally reached a decision. First, thanks for everyone who voted in my "going to the dark side" poll - I was honored that 474 votes were cast (though a few people admitted to voting multiple times!), with the final result (as of today) being:

• No, don't do it! - 231 votes (49%)
• Yes, it is your destiny! - 177 votes (37%)
• Don't know / don't care / who is Peter Batty? - 66 votes (14%)

You can check out the geographical distribution of the US votes at pollmappr. Who am I to go against the wishes of my loyal blog readers, so I have elected to rule out the "dark side" option (this time round at least :) !!). Also, thanks to everyone out there who I had discussions with about potential opportunities - I appreciate everyone taking the time and showing so much interest.

But after all that I have decided that I am going to go ahead and do my own startup, which I mentioned as a possibility in an earlier post looking for developers. I'm pleased to say that I got a number of responses to that post and have a couple of really good people signed up to start with, and a few others who might come on board as things move along. The new company is called Spatial Networking, and we will be focused on location related applications in the social networking space. I don't want to say too much more than that right now, but we will be looking to get our initial offering out in a few months, and I'll talk more about what we're up to as things move along. I'm excited about the opportunity in the space we're looking at, it should be a fun ride!

Incidentally, a couple of people asked me if I was looking for investors, and I will be at some point fairly soon - if you have any interest in finding out more when I get to that stage, feel free to email me.

GIS in the Rockies panel on Service Oriented Architecture (SOA)

As mentioned previously, last week I was on the "all star panel" at GIS in the Rockies, with a distinguished cast of characters. Joe Berry published his response, so I thought I would do the same. The following is based on the notes I made in advance, with a few edits to reflect changes that happened on the fly.

The six panelists were asked to spend five minutes each answering the following question: Do you agree that service oriented architecture (SOA) is the key to enterprise data integration and interoperability? If so, how do you see geospatial technology evolving to support the concept of location services and data interoperability? If not, what is the alternative?

My short answer to the question was that SOA is one of various elements which can play a role in implementing integrated enterprise systems, but it is not THE key, just one of various aspects of solving the problem. The preceding panelists had given quite varied responses about different aspects of interoperability and SOA, which was perhaps indicative of the fact that SOA is a bit of a fuzzy notion with no universally agreed upon definition. The one constant in the responses was that 5 minute time limit was not well adhered to :) !!

So in my response, I first tried to clarify what SOA was from my point of view (based on various inputs), and then talked about other aspects of enterprise integration and interoperability.

SOA is really nothing radically new, just an evolution of notions of distributed computing and encapsulating functionality that have been around for a long time. In fact, a reader survey of Network Computing magazine ranked SOA as the most-despised tech buzzword of 2007! SOA is an approach to creating distributed computing systems, combining functionality from multiple different systems into something which appears to be one coherent application to a user.

CORBA and DCOM were both earlier attempts at distributed computing. The so-called "EAI" (Enterprise Application Integration) vendors like Tibco (who existed since since 1985 as part of Teknekron, since 1997 independently) and Vitria (since 1994) pioneered many of the important ideas for a long time before SOA became the buzzword de jour. More recently the big IT vendors have been getting involved in the space, like Oracle with their Fusion middleware and Microsoft with Biztalk

In general, most commentators seem to regard SOA as having two distinguishing characteristics from other distributed computing architectures:

• Services are fairly large chunks of functionality - not very granular
• Typically there is a common “orchestration” layer which lets you specify how services connect to each other (often without requiring programming)
  

SOA is most often implemented using web services to talk to various applications, but this does not have to be the case. One thing that has helped adoption though is the widespread usage of web services, which owes much to the simplicity of XML. Back in early days of OGC there were three separate standards which didn’t talk to each other - COM, CORBA and SQL - and OGC gained a lot of momentum at the end of the 90s when it switched its focused to an XML-based approach.

Anyway, SOA certainly is a logical approach for various aspects of data integration with geospatial as with other kinds of data.

Now let's talk about some other issues apart from SOA which are important in enterprise integration and interoperability ...

Integration of front end user functionality
SOA is really focused on data integration (as the question said), but this is only a part of enterprise integration. Integration or embedding of functionality is important also, especially when you look at geospatial functionality - map display with common operations like panning, zooming, query, etc is a good example. There are various approaches to doing this, especially using browser-based applications, but still you need some consistency in client architecture approach across the organization, including integration with many legacy applications.

Standards
One key for really getting full benefits from SOA is standards. One of the key selling points is that you can replace components of your enterprise system without having to rewrite many custom interfaces - for example, you buy a new work management system and just rewrite one connector to the SOA, rather than 20 point to point interfaces. But this argument is even more compelling if there are standard interfaces which are implemented by multiple vendors in a given application space, so that custom integration work is minimized.

On the geospatial side, we have OGC standards and more recently geoRSS and KML, and there are efforts going on to harmonize these. The more “lightweight” standards have some strong momentum through the growth in non-traditional applications.

But in many ways the bigger question for enterprise integration is not about geospatial standards, it’s about standards for integrating major business systems, for example in a utility you have CIS (Customer Information System), work management, asset management, outage management, etc.

There have been various efforts to standardize these types of workflows in electric utilities - including multispeak for small utilities (co-ops), and the IEC TC57 Working Group 14 for larger utilities. Multispeak is probably further along in terms of adoption.

Database level integration
Database level integration is somewhat over-rated in certain respects, especially for doing updates - with any sort of application involving complex business logic, typically you need to do updates via some sort of API or service. But having data in a common database is useful in certain regards, especially for reporting. This could be reporting against operational data, or in a data warehouse / data mart.

Scalability
SOA approach largely (but not entirely) focused on 3 tier approach with thin client
Still for certain applications, especially heavy duty data creation and editing, a thick client approach is preferable for performance and scalability

Separation of geospatial data into a distinct “GIS” or “GIS database”
Enterprise systems involving geospatial data have typically had additional barriers to overcome because of technological limitations which required the geospatial data to be stored in its own separate "GIS database". Increasingly though these artificial barriers shouldn’t be necessary. All the major database management systems now support geospatial data types, there are increasingly easy and lightweight mechanisms for publishing that data (geoRSS and KML) ... so why shouldn’t location of customers be stored in the CIS, and location of assets be stored in the asset database, rather than implementing complex ways of linking objects in these systems back to locations stored in the “GIS” database? This is not just a technology question, we also need a change in mentality here - to spatial is not special, just another kind of data, a theme I have talked about before (and will talk about again!).

Summary
SOA is a good thing and helps with many aspects of enterprise integration, but it’s not a silver bullet - many other things are important in this area, including the following:

• Standards are key, covering whole business processes, not just geospatial aspects
• As geospatial data increasingly becomes just another data type, this will be a significant help in removing technological integration barriers which existed before - this really changes how we think about integration issues
• Integration of front end user functionality is important for geospatial, as well as data / back end service integration (the latter being where SOA is focused)
  
• There are still certain applications for which a “thick client” makes sense