Simulator GIS – Darb Dabney’s OpenSim

It’s not fully 3D immersive, but hey, 2-1/2D ain’t half bad. The “dsm40cm” model of Marin County has been published as the county’s default terrain on Google Earth. It’s a great pleasure to work with folks who are not troubled by a county representing its surface on a 40cm single-precision float grid that weighs in at 77 GB. In terms of data bulk, that is about the same as the entire 30-meter version of the US National Elevation Dataset.

What one gets when piling that much detail into a single county of around 520 square miles of land area is every building pad, driveway, and crown of road paving that were resolved. The dsm40cm model was derived from an ESRI Terrain Dataset that incorporates our best available topographic contours (1:4800 scale 10-foot; 1:2400 scale 2-foot,) photogrammetric break and water lines, FEMA LiDAR and NCALM (GeoEarthScope) LiDAR data sets. The Terrain Dataset currently comprises 40 GB of vector GIS data.

When the finely detailed surface grids were first developed, we broke the county up into 20 work areas to maintain ArcGIS 9.3.1 in a stable and productive state, and 30cm posting interval grids were generated that covered the entire county–at least during development. When necessary, these grid tiles were mosaicked with ERDAS Imagine into a single seamless grid. The 40cm version was produced directly as a single seamless grid using ArcGIS 9.4 beta 1, on a workstation imaged with Windows Server 2003. The WGS84 UTM, NAVD88-Geoid 2003 result was provided to the Google Earth team earlier this year.

As with all GIS data sets, it seems, the more detailed it is, the more rapidly it may need updating. In the works for the next year or so are several improvements to the dsm40cm model. First: the photogrammetric break lines will be segregated into steeper sets that tend to run along ridges, and shallower slopes that tend to delineate road cuts and building pads. The ridge set will be used as soft constraints to resolve some artifacts where they rise above some contours.
Second: incorporate new LiDAR data as it becomes available. Some data has already been provided for the lowest part of Lagunitas creek, and it appears that Prof. Ellen Hines of San Francisco State University’s Department of Geography and Human Environmental Studies has been funded by USGS to gather LiDAR county-wide this year.

So there will be revisions, but an exciting aspect is to see data flows being brought into existence that support different levels of mirror world development.
Publishing the dsm40cm model in Google Earth is an important (and beautiful) threshold to cross. Making use of the dsm40cm model in county operations such as creek and watershed delineation will be the practical benefit that drives the work in the first place. And before too many more weeks, there may be entirely new approaches to publishing the data in an immersive environment (neither Second Life nor Opensim) to share.

Kent Woodlands building pad and driveway, in the shadow of Mt. Tam

<<updated 20090424>>

As my patience with Second Life wanes, and I wait for more architectural input for my next SL build project, I have a dark OpenSim server with no fixed IP.  I’m having stability issues with the Linux SL client, but have upgraded the workstation to Ubuntu 9.04 Jaunty Jackalope.  Google Earth client there is more stable, the NVidia drivers install themselves (sans Envy), and everything Ubuntu-wise seems to be getting incrementally better by the quarter.

I’m grinding some large images that have taught me that one very special difference between Windows XP variants and Windows Server 2003 is the latter’s ability to open files on the high side of 80 GB.  I’d never quite realized it before but the moderately massive mosaics that I have created in years past (edging toward 250 GB single files) actually depended on Server 2003 to get created.  Once the destination file exists, then XP can take it from there, and in all cases Windows Explorer can copy the monster files.  But in that tenuous moment when a mosaic first grabs its space on disk for a huge output—one can’t seem to do that with XP.

So while I’m enjoying Google Earth on Ubuntu, there is something cool that I go back to Windows for, and that’s the new Google Earth browser plug-in.  Since I’m gaining a bit of facility with the keyboard shortcuts in the full-stop Earth client, these all carry over to the plugin.  My first test page has been stood up here and I’ve been deep into four continents with it so far.  I understand that the plugin is only available for Windows and Mac systems at this time.  If you can,  Enjoy!

http://earth.jedi.bz 

Also, as I get even faster with my keyboard navigation of G-Earth, I’ve actually seen some artifacts that are quite familiar from OpenSim.  While zipping about between the Gulf of Yakutat and Canada’s Mount Logan, at certain viewing elevations I can accelerate the point of view forward quite fast.  Doing so in this very mountainous terrain, I saw blocks of terrain standing up along what look like sim edges, resolving in a few seconds as more (sculpty?) bumpmap arrives.  This is the same sort of artifact I’ve seen with terrain sculpties and sometimes, with region crossings in OpenSim.  Also, I’ve found a couple of wild terrain grid errors in G-Earth.  In one, a quarry dug hundreds of feet below sea level, right next to the sea, is displayed as positive elevation (absolute-value terrain, anyone?).  In another, a boundary between US and Canadian terrain has a glacier flowing uphill onto a plateau.  Go figure.  Blame Canada! ;^)

The past four days have been a tremendous blur of internalizing NURBS into my mind, at least the SL sculptie variant of them. Now I’ve been aware for several months of how NASA used sculpted prims to represent detailed Mars craters (as published by Ireton), and I’ve certainly followed the beautiful work for David Rumsey done both by Telemorphic in 2003 (3D plots of historic Lake Tahoe area) and more recent historic Yosemite by Nathan Babcock). But there was something confusing and ultimately mysterious about using sculpties for terrain.

Not so much any more. Through several helpful blog and forum posts, and a score of hours spent in experimentation, I feel that I’ve brought the sculptie to heel for my terrain rendering purposes. Mostly, especially for OpenSim, it’s simply to display draped orthoimagery over an already precisely customized region terrain.

What I’ve learned is that for 1:1 mapping, where regional terrain is not available at more detail than the 10-meter postings from seamless.usgs.gov, then one can configure precise sculpted megaprims, only four to a region, and drape imagery quite effectively. The result is real-life imagery draped in the style of Google Earth, but coming out of a free OpenSim server into a free Second Life client, for a dozen or more regions on one server core. When using the technique that I’ve worked out, having only four scuplties to seamlessly cover the region means that the terrain sculpties will rez fully sixteen times (16X) faster than will either the David Rumsey or NASA educational islands.

There’s no special magic here: the region terrain is far superior as a way to represent real life terrain, as it can hold 64K of single-precision floating point values. A sculptie, by comparision, holds a mere 900 usable values that must be compressed to an 8-bit signed integer, for any one of the 900 points’ X, Y, or Z values that are practical to use to guide facets in a terrain “diamond”. This “diamond” is a way of describing what the terrain sculptie looks like after defining the outermost ring of UV values to wrap around to a single point safely below terrain surface, so as not to interfere with the 30×30 values useful to describe terrain in a way that cleanly tiles to cover multiple regions. The vertical scale of the terrain described this way is adjusted with the Z-dimension of the sculptie spheroid, which must be tuned using back-end OpenSim command “edit-scale” if one is manipulating a megaprim.

Anyway, when I get a chance to demo this for some Berkeley terrain, I’ll be sure to post a dramatic screen shot. As it is, the 12-region sim looks so realistic right now that almost any shot would be immediately recognizable by someone familiar with the site, so the good ones will wait for project time. Meanwhile, this one is intended to prove validity of sculpted terrain megaprims for draping orthoimagery.

The tools that I used were: ArcGIS to reproject the seamless terrain and orthoimagery into a rotated local grid variation of WGS84-UTM; ERDAS Imagine to perform mosaicking, dicing, image stretch/rescaling, and layer stacking to build precise UV maps from real life terrain; and OpenOffice.org spreadsheet to calculate precise gradient values for the X and Y components of the UV maps. On the back end was OpenSim 0.5 using region definitions in the 0.4 style, and the terrain build was performed using the standard Second Life 1.19.0.(5) client running on Windows XP with a Radeon X1300 Pro.

A couple of days later, I revisited the sim and made a couple of updates to sculpties with the new 1.19.1.(4) Second Life client, and the orthoimage colors look different depending on sun angle–thanks to Windlight.  It’s not a bad thing, and gives one a reason to look up and appreciate the beautiful sky!   Back on Agni (standard Second Life Grid) by comparison, all the prims seem far more intensely colored and somehow more detailed with the new client.