Smallworld's 20th birthday

It was brought to my attention the other day that we have just passed the 20th anniversary of the founding of Smallworld (December 3rd 1988 was its first day). Many readers of my blog these days may not be familiar with Smallworld, which is now part of GE, who acquired us in 2000. I joined Smallworld in 1992 in the UK when it was still a fairly small startup (about 30 people or so I think), was the first person to move from the UK to the US when we started here in 1993 (working with our sales guy Steve Bonifas), and I left in 2002. Smallworld grew to become the global market leader in GIS for utilities and communications companies, and almost certainly still holds that title today, depending on which market survey you believe. It certainly remains very strong in the high end of the utility and telecom market.

Smallworld really revolutionized the GIS market back in the early nineties, introducing several radical new ideas, and several ESRI insiders have since told me that ArcInfo 8.0 (now ArcGIS) was developed in order to respond to Smallworld's impact on the market - it clearly copied a number of ideas from Smallworld, as well as missing a number of important aspects too.

Some of the key innovations introduced by Smallworld at the beginning of the nineties included:

• Handling of long transactions using a new technique called version management (which became an industry standard approach some ten years later, being adopted in some form by ESRI, Intergraph and Oracle, among others)
• Outstanding performance and scalability, with the ability to interactively pan and zoom around very large continuous databases, and immediately edit features without having to extract anything, which was revolutionary at the time (most comparable systems required you to check out data from a database into a local file, which typically took minutes to do).
• Both the preceding features were due to the Version Managed Data Store (VMDS), a specialized database management system developed by Smallworld (specifically by Mark Easterfield) which was highly optimized for long transactions and spatial data - interestingly the high performance characteristics came from an elegant caching approach that was specific to the way that long transactions were handled. You can read more about this in a white paper called GIS Databases are Different, which I co-wrote with Dick Newell. There's an interesting debate of course on the extent to which the title of this paper is true - it is in some ways but not in others, but that could take up several posts in its own right so I won't go there just now! In the late nineties as use of mainstream relational databases for spatial data became more common, VMDS was really a two-edged sword from a sales perspective. The market didn't like the idea of a "proprietary" database, even though in many ways it was and still is a superior technology for interactive graphical applications in a long transaction environment.
• An extraordinarily flexible and productive development environment, based on Smallworld's own object-oriented language called Magik (initially developed by Arthur Chance, with significant contributions from Nicola Terry), which was way ahead of its time and very similar to Python, which has of course become very popular in the last few years and recognized for being an extremely productive environment. Smallworld was a pioneer in the use of object-orientation for large scale systems. The system had a unique approach which made almost any aspect of the system incredibly customizable, yet in a way which minimized support and upgrade issues. Getting into detail on that is beyond the scope of this post but maybe that's another item I'll return to in future.
• A data model based on spatial attributes rather than spatial objects. This is one of the really important features that ESRI didn't get into ArcGIS, which is still based around the idea that a feature is a point, line or polygon (with a few minor extensions, but still the concept is that a feature has a specific spatial type). In Smallworld, an object (feature) could have any number of spatial attributes, each of which could be a point, line or area, which is a much more flexible and powerful approach in many applications. This is something that you really have to design into your system from the beginning, as it fundamentally changes many aspects of the system from data modeling through to many aspects of the user interface, so I suspect that it is unlikely that ESRI will ever adopt this model at this point. I talk about some examples of how this model is useful in the technical paper AM/FM Data Modeling for Utilities, as well as discussing some other modeling constructs that were unique to Smallworld at the time, like multiple worlds.
• Integration of raster and vector data in a common database, with features for automated line following to help with "heads up digitizing" for data capture from (scanned) paper maps. Smallworld never did support traditional digitizing tablets, which were the primary means of data conversion back then, and we fought quite a lot of battles in RFPs explaining why we didn't support this often "mandatory" requirement.
• An integrated CASE tool for graphical design of your data model, which was itself version managed (the CASE tool was actually just a specialized Smallworld GIS application). Another feature of Smallworld was that you could modify the data model of a production database just within an individual version, which was incredibly powerful for being able to test changes without impacting users of the system, and then just apply them to all versions once you had tested everything. There is also a technical paper on the CASE tool.

So if Smallworld was so great, you may be asking, why it isn't the leading GIS in the marketplace today? I think that's also a story for another post (or several)! But as I said, Smallworld did become the leader in the utility and telecoms market and still holds a very strong position there today, based on the same core technologies of Magik and VMDS that were initially developed 20 years ago, which is a testimony to their strength. I would say that the early to mid nineties when Smallworld burst onto the scene in the US were the most enjoyable time of my career - it's rare that you have the opportunity to work with a technology that is so far ahead of the competition, in ways which matter to the customer, combined having great sales and marketing and all the other aspects you need to be successful - and to be in this position in a market where lots of companies are buying new systems. These days the utility GIS market is a very tough one - the three major high end systems (Smallworld, Intergraph and ESRI) all have pros and cons but none of them has a clear technical lead, and there are very few new system sales as most utilities have invested a lot in one of these and there really isn't a compelling business case to switch.

So anyway, I'd like to finish by recognizing the 10 founders of Smallworld, and congratulate them for building such a ground-breaking product and great company, and thank them for all the good times. The founders were Dick Newell, Tim Cadman, Richard Green, David Theriault, Mike Williamson, Hugh Fingland, Arthur Chance, Mark Easterfield, Cees Van Der Leeden, and Klas Lindgren. (Slightly belated) Happy Birthday!!