Could there be a future without big grids?

So, it’s late on a Friday in the UK and I’m in the mood to write another blog post.  But I’ve just finished grinding out another day in my (soon to be ex) non-OpenSim related day job, so instead of writing something detailed, technical and real, I’m going to indulge in a bit of speculation as to what a future internet wide virtual worlds architecture might look like.  This is going to be heavily influenced by OpenSim since this is the project in which I have the most experience by far (though I have played around with Croquet).  And so by extension, what follows also reflects Second Life architecture, since Second Life is the scaffold upon which OpenSim has been built.

What really strikes me about the current situation regarding OpenSim deployments is the emphasis upon grids.  In the current Second Life architecture, I would define a grid as a set of regions that share the same central backend services (asset, inventory, user, etc.) and which have discrete positions on a map.  Some of these regions may be neighbours to each other such that an avatar can move between them without discontinuity in the 3D world (i.e. without teleporting).

The Linden Lab grid is obviously by far the biggest of these grids, containing as it does tens of thousands of regions in both contiguous ‘mainland’ areas and individual private islands.  But as OpenSim has become a little bit more mature, other grids using the software have sprung up, such as OSGrid, OpenLife and the New World Grid.  These grids differ a lot in their approach (e.g. OSGrid is a non-profit allowing anybody to connect, OpenLife is a closed grid with their own unreleased extensions to OpenSim), but I think that to a greater or lesser extent they all buy into the notion, inherent in OpenSim (and Linden Lab’s) current architecture, that a large grid serving many region simulators is the natural unit or organization.  People owning or running these region simulators might have wildly different interests (e.g. education, sex, corporate meetings), but they’re all bound into a bigger grid entity by virtue of the services provided by the grid operators.

But if you compare OpenSim (potentially a general 3D virtual worlds server) to Apache (a general 2D web server) as is often done, this approach seems a little strange.  Though it is true that the company you buy web hosting from will also provide hosting for many completely different websites, there’s little that binds these sites together beyond shared physical hardware.  The hosting company doesn’t need to allocate you a slot on their grid.  And when you bring up a new Apache instance, there’s no need for you to immediately hook it up to a load of support services provided by some other entity.

In a sense, you do still need a place in a map in order to host your website – in the case of the web that place is a domain name.  So there is a service, the Domain Name Service, that’s necessary to support human navigation via URLs between different websites.  But this is a service external to the web hosting company.  In the same way, I would speculate that many of the services that are supplied by grids today (asset, inventory, etc.) will in the future be supplied by companies that can be completely independent of grid operators.  For this to happen, virtual world viewers will need to interact with these services directly, rather than via a grid.  Asset services, for instance, are little more than basic storage (as described in one of Adam’s experiments) that a viewer could easily deal with directly.  Inventory services are a little more involved, but interacting directly with them could also make it far easier to take inventory between grids, and prevent items from being seen by a grid unless the user actually places them in a region.

Under this model, a grid goes from being a big central entity you need to hook your region simulators into, to instead being the 3D equivalent of an individual website.  It could consist of a single region, or a large number of regions bound together for a common purpose.  They could execute in a single region server or on many machines in a cluster (which is why I still think of these things as mini-grids rather than standalone instances).  Just as with websites, you could navigate between completely independent OpenSim sites by entering urls in an address bar in the viewer.  Much as with web hosting today, such OpenSim sites could be hosted by a very large number of independent hosting companies.

Linden Lab is making some steps in this direction with the Open Grid Protocol being produced by their Architecture Working Group.  They have an architecture with an Agent Domain, which so far handles user logins, and a Region Domain containing the region simulators themselves.  As we’ve seen with the OpenSim – Linden Lab interop work, one can log into an OpenSim region server via an Agent Domain. However, it remains to be seen as to how inventory will be supported.  Making such services completely independent of grids will require some boldness on the part of Linden Lab since it could require major changes to their current business model.

If the natural equivalent of a website is an independently hosted mini-grid where most services are supplied externally, where does this leave the idea of large grids?  I’m going to speculate that there could still be large grids, but that these would probably be dedicated to well-defined communities.  For instance, Facebook is a huge single website dedicated to social networking – perhaps there could be a large OpenSim site along the same lines (though I have a feeling that in the virtual worlds space the situation won’t be quite so monolithic – we already see quite a few grids dedicated to individual countries and languages).

I do think, though, that the idea that OpenSim sites will be hosted on large grids of regions bound together by little more than a common backend service infrastructure is one that may well go away.  Instead, URL navigated OpenSim sites could be hosted by independent companies just as website hosting is done today.

17 comments for “Could there be a future without big grids?

  1. Peter Quirk
    August 16, 2008 at 2:26 pm

    Everything you say is correct, except that Apache doesn’t deal with issues of proximity and direction. It’s quite natural that a 3D world would deal with geographic concepts that the 2D web world neglected because it was modeling the printed page and citations that linked these pages. The grid is a natural response to the navigational imperative involved in building out a “place”.

    It’s not the only model. VW platforms like VastPark choose to connect rooms together. It reminds me of the old Adventure/Zork games where the challenge was to draw a map based on your peregrinations through “the maze of twisty little passages, all alike”. Likewise Croquet connects these isolated places through portals. There are advantages to each model. Intuitively, I feel that the grid model supports persistence of space much better, enabling concepts like search and directories much better, but that’s just a hunch.

  2. August 16, 2008 at 7:04 pm

    Great blog!!! I think when you make analogies to some existing technologies and then highlight differences it really brings it into perspective for technical people who are new to virtual worlds.

    I’d like your vision of a more service oriented environment, and hopefully some good standards can be created between OpenSim and Linden Labs AWG. Not only would this allow competition for who can provide the best services (inventory storage, asset storage etc) but it allows for greater innovation because you’ve broke something that was extremely complex into simpler more maintainable parts.

    Right now we’re more like a Ford Model T. Hopefully in the next couple years we can churn out some standardized car parts and begin to create our own assembly lines!

  3. Virtual
    August 16, 2008 at 7:23 pm

    Interop receives all this attention today, interop is trendy today, interop is nothing more than a hyperlink. Compare a grid to a website, there are big sites and small ones, same goes for grids. Grids will want to link to grids they like and that match with their vision and their way of working.
    Religious grid won’t link to nasty porn grid and vice versa but they will link through 3th party grids:
    porn grid — roleplay BDSM grid — fashion grid — mainstream general grid — christian grid

    I think that at this point it’s way to early to spend so much attention to interop. Opensim is becoming a product yes, but there is still a lot that needs to be fixed, focus on the product, ask yourself how to improve the product, don’t let trends like google web integrated Lively distract you into coming up with Xenki, work on improving the core product, make it more robust. We still haven’t got meshes, teleports take 5 seconds and should be reduced to 2. Scripting isn’t done and is still unstable, region crossing isn’t smooth yet, more work needs to be done between RealXtend and Opensim, they core of the product needs to be improved, ask yourself questions on how to improve it, what is needed inworld. Look at platforms like Vastpark and their physics engine and the programming language they use for game scripting. How they integrate light in a scene, how they use shaders in world etc…. Some good ideas there which could enrich Opensim.

    The in world experience has to impress by the platform technology.

    All that talk about transfer of assets, permissions, interop, teleporting between worlds is placed way to high on the agenda I think. Focus on the core, impress with the software, after that when you can say ok we have now this great product then you can take a look for web integration, interop between the worlds, transfer of assets during teleports and so on.

    When you build a website the hyperlinks are the last thing that you put on the site. First you make your site.

    Large grids will come, SL exist for 8 years, they have 21000 regions which isn’t that huge when you think there are billions of webpages.
    I read somewhere that there are already several 1000’s of regions running on Opensim software which is impressive for something so young in alpha stage. Large grids will form themselves when people find a use for great technology. One thing brings on the other.
    People don’t want to put on their clothes every single time they login, so they have to be able to rely on the software, some sort of trust between the platform and the end user.

    Smart modules are the way to go compared to hyperlinking.

  4. August 17, 2008 at 10:45 pm

    @Peter – Interesting comments. I think that connecting OpenSim instances together using ‘portals’ where you can see the live remote OpenSim through a ‘window’ (a la Croquet) is one that makes sense. The analogy to me is one of embedding into a web page an RSS feed from another site.

    However, I believe that the notion of the 3D web as one seamless geographic space (a la Linden Lab’s grid or William GIbson’s cyberspace) is really a hangover from the gaming roots of virtual worlds and will only be relevant within individual virtual world sites.

    @Lee – Thanks Lee :) I do suspect that an architecture were everything is broken out into individual services accessed directly by clients visiting individual 3D sites is going to be more scalable than a large grid based model (with agent services lumped into monolithic agent domains).

    Of course, this is a very high level view with quite a few unanswered questions – there are people out there who have a much better grasp of the technical nitty-gritty than I do who might be able to give good objections to this approach. It’s going to be very interesting to see where the AWG work eventually evolves.

    @Virtual – Even if interop is little more than a hyperlink, that’s still tremendously important!

    I do agree with you that OpenSim needs a lot more work on core functionality and stability. Personally, I think that OpenSim is becoming an adolescent – we’ve finished with the first growth phase and now there’s going to be a lot of spot squeezing and moodiness as we try to implement the more difficult missing functionality and deal with scalability and concurrency problems that were previously hidden by more basic failures.

    But I think that interop shows us what the future could be and is invaluable in spurring on a lot of work on the base platform.

    @Adam – Thanks man – nice to know that another core developer has thoughts along the same lines :-)

  5. August 22, 2008 at 4:19 pm

    Great post. As my job consists of steering virtual worlds in this direction I can’t help but think my agreement is a bit biased :-). However, I think the term grid will still hold meaning, just not as a contiguous virtual land mass. When I go to facebook.com, I don’t choose which DNS server is authoritative for resolving their IP address, which login server to use (although the option may open up with OpenID), which database backend to use, or what HTML content they serve up. All of these services and assets together make up the “facebook.com grid”. Maybe the usage of grid in that context should go away, and instead we can talk about virtual world sites or hosting providers?

  6. August 25, 2008 at 1:24 am

    @John – thanks :-). It’s very interesting that you’re now engaged in moving things in thie kind of direction. I’d be very interested to know more.

  7. September 13, 2008 at 5:44 am

    For an indication of how educators are running their own nodes via self-contained grids take a look at the Education Grid which is part of the non-profit Immersive Education Initiative (http://ImmersiveEducation.org/). The Education Grid is presently comprised of Wonderland, OpenSim and realXtend nodes, with other sim node types scheduled to be available soon as well. Although the Education Grid is for educators and students, you can probably a guest account through the Web site (http://TheEducationGrid.org/) or by contacting the director as well. Barbara

  8. September 13, 2008 at 5:47 am

    Hmmm.. my links didn’t come through in my previous comment so here they are again. Hopefully you can jump directly from here

    http://ImmersiveEducation.org/

    http://TheEducationGrid.org/

  9. October 30, 2008 at 6:25 pm

    I agree with Justin here. To summarize my concept would be a “universe” map of maybe even smaller definable “galaxies” and ultimately the current grids filling them. I would like to see an option in the ini that allows us to map out interconnect of grids with perhaps a way to block certain ones grids not fitting your TOS. We would love to share traffic among grids and cross promote. A series of Stargate like portals where handshaking and various housekeeping to avoid UUID violations happens is something we would love to participate in.

Leave a Reply

Your email address will not be published. Required fields are marked *