Thursday, December 18, 2008

Smallworld's 20th birthday

It was brought to my attention the other day that we have just passed the 20th anniversary of the founding of Smallworld (December 3rd 1988 was its first day). Many readers of my blog these days may not be familiar with Smallworld, which is now part of GE, who acquired us in 2000. I joined Smallworld in 1992 in the UK when it was still a fairly small startup (about 30 people or so I think), was the first person to move from the UK to the US when we started here in 1993 (working with our sales guy Steve Bonifas), and I left in 2002. Smallworld grew to become the global market leader in GIS for utilities and communications companies, and almost certainly still holds that title today, depending on which market survey you believe. It certainly remains very strong in the high end of the utility and telecom market.

Smallworld really revolutionized the GIS market back in the early nineties, introducing several radical new ideas, and several ESRI insiders have since told me that ArcInfo 8.0 (now ArcGIS) was developed in order to respond to Smallworld's impact on the market - it clearly copied a number of ideas from Smallworld, as well as missing a number of important aspects too.

Some of the key innovations introduced by Smallworld at the beginning of the nineties included:
  • Handling of long transactions using a new technique called version management (which became an industry standard approach some ten years later, being adopted in some form by ESRI, Intergraph and Oracle, among others)
  • Outstanding performance and scalability, with the ability to interactively pan and zoom around very large continuous databases, and immediately edit features without having to extract anything, which was revolutionary at the time (most comparable systems required you to check out data from a database into a local file, which typically took minutes to do).
  • Both the preceding features were due to the Version Managed Data Store (VMDS), a specialized database management system developed by Smallworld (specifically by Mark Easterfield) which was highly optimized for long transactions and spatial data - interestingly the high performance characteristics came from an elegant caching approach that was specific to the way that long transactions were handled. You can read more about this in a white paper called GIS Databases are Different, which I co-wrote with Dick Newell. There's an interesting debate of course on the extent to which the title of this paper is true - it is in some ways but not in others, but that could take up several posts in its own right so I won't go there just now! In the late nineties as use of mainstream relational databases for spatial data became more common, VMDS was really a two-edged sword from a sales perspective. The market didn't like the idea of a "proprietary" database, even though in many ways it was and still is a superior technology for interactive graphical applications in a long transaction environment.
  • An extraordinarily flexible and productive development environment, based on Smallworld's own object-oriented language called Magik (initially developed by Arthur Chance, with significant contributions from Nicola Terry), which was way ahead of its time and very similar to Python, which has of course become very popular in the last few years and recognized for being an extremely productive environment. Smallworld was a pioneer in the use of object-orientation for large scale systems. The system had a unique approach which made almost any aspect of the system incredibly customizable, yet in a way which minimized support and upgrade issues. Getting into detail on that is beyond the scope of this post but maybe that's another item I'll return to in future.
  • A data model based on spatial attributes rather than spatial objects. This is one of the really important features that ESRI didn't get into ArcGIS, which is still based around the idea that a feature is a point, line or polygon (with a few minor extensions, but still the concept is that a feature has a specific spatial type). In Smallworld, an object (feature) could have any number of spatial attributes, each of which could be a point, line or area, which is a much more flexible and powerful approach in many applications. This is something that you really have to design into your system from the beginning, as it fundamentally changes many aspects of the system from data modeling through to many aspects of the user interface, so I suspect that it is unlikely that ESRI will ever adopt this model at this point. I talk about some examples of how this model is useful in the technical paper AM/FM Data Modeling for Utilities, as well as discussing some other modeling constructs that were unique to Smallworld at the time, like multiple worlds.
  • Integration of raster and vector data in a common database, with features for automated line following to help with "heads up digitizing" for data capture from (scanned) paper maps. Smallworld never did support traditional digitizing tablets, which were the primary means of data conversion back then, and we fought quite a lot of battles in RFPs explaining why we didn't support this often "mandatory" requirement.
  • An integrated CASE tool for graphical design of your data model, which was itself version managed (the CASE tool was actually just a specialized Smallworld GIS application). Another feature of Smallworld was that you could modify the data model of a production database just within an individual version, which was incredibly powerful for being able to test changes without impacting users of the system, and then just apply them to all versions once you had tested everything. There is also a technical paper on the CASE tool.
So if Smallworld was so great, you may be asking, why it isn't the leading GIS in the marketplace today? I think that's also a story for another post (or several)! But as I said, Smallworld did become the leader in the utility and telecoms market and still holds a very strong position there today, based on the same core technologies of Magik and VMDS that were initially developed 20 years ago, which is a testimony to their strength. I would say that the early to mid nineties when Smallworld burst onto the scene in the US were the most enjoyable time of my career - it's rare that you have the opportunity to work with a technology that is so far ahead of the competition, in ways which matter to the customer, combined having great sales and marketing and all the other aspects you need to be successful - and to be in this position in a market where lots of companies are buying new systems. These days the utility GIS market is a very tough one - the three major high end systems (Smallworld, Intergraph and ESRI) all have pros and cons but none of them has a clear technical lead, and there are very few new system sales as most utilities have invested a lot in one of these and there really isn't a compelling business case to switch.

So anyway, I'd like to finish by recognizing the 10 founders of Smallworld, and congratulate them for building such a ground-breaking product and great company, and thank them for all the good times. The founders were Dick Newell, Tim Cadman, Richard Green, David Theriault, Mike Williamson, Hugh Fingland, Arthur Chance, Mark Easterfield, Cees Van Der Leeden, and Klas Lindgren. (Slightly belated) Happy Birthday!!

10 comments:

Tony said...

Hear Hear! Happy Birthday, and I am proud to have worked with many of the illustrious founders as a partner and colleague.

Aaron said...

Wow - 20 years is kind of a long time ;-) From (distant) memory, the CASE tool came out in version 2.0(??), and the couple of 1.92 implementations I worked on back then meant writing the actual Magik to implement the model? Maybe it was the release before, I don't know.... Glad I didn't have to do that too often though.

I remember thinking I had won the lottery when I showed up for a week in Imass's office straight out of college, followed by about 4 or 5 weeks on expenses in Cambridge straight afterwards. I ate and drank better than any time during college, with pepper vodka at the Bird in the Hand (for lunch!! - those Smallwordl trainers were a bad influence!), followed by a drunken train ride back up to Newcastle on Christmas Eve after the last class. Happy days, and weird how some things work out in ways you never quite expect.....

OK, reminiscing over....

Peter Batty said...

@Aaron - yes, makes you feel old eh? You're right, the CASE tool appeared in version 2.0 in 1993. In regard to pubs, I can remember that my customer MANWEB would call the payphone in the Zebra if they needed support after hours, as they knew that was the place to find us :) (that was in pre mobile phone days, of course!).

Unknown said...

Thanks Peter for this retrospective -- we turned "15" this month but I never knew Smallworld was only 5 years older. Prior to starting Safe, the company I was with considered using Smallworld for a national mapping system but in the end went with System 9 (which went on to become Spatialware for MapInfo). One of the most common misconceptions about Smallworld from that era (which seems to keep coming up) was that it had something to do with the Smalltalk programming language, which I'm quite sure is completely false. Anyone want to confirm or deny that?

Dale

Peter Batty said...

Hi Dale, I also heard the occasional comment about the Smallworld name being related to Smalltalk, but you're right that it wasn't anything to do with that.

Anonymous said...

Hi Peter - from Down-Under,

great summary, maybe worth mentioning the work of Mike and Lesley Bundock in this part of the World. I first bumped into Mike in an on-line bulletin board (remember those?) on object-oriented programming in the early 90's. Mike would not stop chatting about the wonders of Smallworld until such time I organised a demo for him to the Intergraph shop (long since gone) I was working for at the time. I quickly "saw the light" but it took me another year or so of frustration before I left the joint and started we-do-IT.com (First Smallworld World Class Partner in Southern Hemisphere in 1996) now 40+ Smallworld Consultants and still loving it - even after 8 years of GE trying to starve the partner model (now re-introduced with great fanfare). Thank you Mike Bundock for introducing me to Smallworld, thank you Eric Proust for teaching an old FORTRAN hack a little bit of Magik and thank you Nigel Went for restoring my faith in GE partnering ...

Unknown said...

Happy Birhtday to big smallworld!
Good to hear the history and comments on smallworld from Walter and many of the old smallworld guys. I am young(4years)in smallworld, still I am very happy and proud working with top GIS product...Smallworld

Anonymous said...

Got started with Smallworld in 1991, I believe it was version 1.9.4. There was no CASE, ACE and various other tools, they came in version 2. And the symbol editor was horrible, I remember. I think Gillian worked on the CASE tool. Smallworld was still pretty small in '91, my first training in Cambridge was done by the accountant while Nicola came in if something crashed or they 'cooked' a new image :-) It'll be interesting to see what the next 20 years will bring. Happy Birthday Smallworld, thanks for the opportunity to work with so many great people in different countries.

GIS mapping services said...

Happy birthday!!!!

regards
GIS data processing

Anonymous said...

Where is the opensource-version of Smallworld?