Darb Dabney’s OpenSim Blog

There has been a bit of head scratching as other distractions apparently clouded an obvious scale issue. The first terrain megaprim sculptie project done last month, had available imagery at 30cm.  For that, it only made sense to oversample to 25cm to make 512×512 textures.  That decision led to my adding a collar around the original 512’s until they clicked into the proper size without rescaling on a quarter-region megaprim. With Berkeley, the source imagery is almost 10cm (103mm pixels) and the challenge has been to size the resample so as to make best use of the 1024×1024 texture size limit per prim.

Where I took a wrong turn was trying to proportion the collar that was added to the 512’s, rather than going back to basic principles with sculpties. Bottom line: my efforts of the past week went astray as I allowed confusion to set in, casting about for the proper maximum texture dimensions working down from 1024. (and I’ve got the awkward attempts at 1008, 994, and 978 pixels to prove it).

In fact, the answer is very simple in reference to basic sculptie principles, as the maximum dimensions of the sculptie bumpmap are 32×32, and due to the need to wrap it around to an apex underneath, this can only represent a 30×30 terrain patch. Thus, the maximum imageable area is simply (30/32)*1024, or 960 pixels square, collared out to 1024 square to make each orthophoto tile. This means that an OpenSim 1.024:1 model can accomodate 130mm orthophoto imagery, and I now have 160 tiles ready to go with the bumpmaps.

So far I’ve configured twelve regions with their megaprims, and only one seems to have issues with the height of the sculptie to stay 30cm afloat the terrain surface. Nine of these reigons use the flattest setting, one uses the intermediate, and two use the steepest. Here’s some shots for update’s sake. The full set of orthophoto textures have been uploaded (450 MB of Targa files) and seem to show up reasonably well in inventory. I am using a local MySQL instance on the OpenSim machine for prim storage.

After spending plenty of time getting all the terrain megaprims stamped out, and starting some refinements of how to squeeze the imageable part of the ortho into a portion of the 1024 square texture, I found myself rather unhappy with how lumpy the megaprims were in the flat part of the model. Berkeley has this sort of dual terrain personality (no comment on the residents) that has certain types of details in the distal fan and floodplain parts of town to the westerly, and very different types of details in the hilly and steep areas.

When one loads the natural terrain into OpenSim, the values are 64K of single-precision floating points per region. When loading terrain into a megaprim, there are a mere 900 usable values that must be mashed into an 8-bit integer of Z values. So when the whole sim ranges from 45–267 meters (and it could go up to 581 meters if a sim ran all the way up to Vollmer Peak) , one gets dynamic range issues if all the terrain megaprims are scaled to fit over the entire sim. So to mitigate this, I divided the sim into distal (the “flats”), proximal (the “foots”) , and hills proper (easterly of the Hayward Fault).

The upside of this extra work is better fidelity in the different parts of the model when the floating point values are approximated by 8-bit integers. When I used the entire elevation range for the whole UC Berkeley sim, each integer Z value was 87 cm, or close to three feet. With the sim broken into three regions of megaprim Z-scaling, I have each integer worth 11 cm in the distal, 31 cm in the proximal, and 63 cm in the hills, so everything is a little better everywhere. I’ll fess up to not having the prim placement fully automated, otherwise it would be practical, and perhaps desirable, to use the full dynamic range over just those elevation values in the region (or even in the quadrant of the region that the terrain megaprim covers). But by that point, I’d consider even denser grids of terrain megaprims, and it would then be a different process for representing the terrain.

After all, the whole point of this exercise is to devise a work-around for not being able to load the ortho imagery directly into the region as a draped terrain texture!

Enough blabbing - please enjoy the graphics.

Here is how the terrain maps out versus the buildings

Not intended as digital Cubism, this odd-looking approach makes better terrain megaprims. Really, it does!

In case one knows particular buildings on the UC Berkeley campus or environs, this is how the zoning worked out. There is a certain logic to it, geomorphically.

Enough carefree hours in the main SL Agni grid, already! Back to matters of creation.

Next big thing should be a terrain prototype for civic application. No special business process in mind here, just a demo of the draped imagery on real-life terrain in a way that could scale up city-wide. For starters, there must be a better correspondence between the US National Grid and the dimensions of the simulated region. Sure Neal Stephenson may have suggested binary 2^n dimensionality, and there may be plenty of reasons in the simulator code to make use of the full range of 256 meters. But after more than a handful of regions, the starting corners get downright ugly.

So I won’t do it that way. By scaling up, larger even than real-life, the regions can be built sixteen-to-a square kilometer. In a worldwide sense, except for the matter of 62 or 64 matchlines, the US National Grid (a.k.a. Military Grid Reference System or MGRS) has the whole world in its hands, so harmonizing region design with that grid plan covers a whole lot of ground. To minimize my effort at constructing regions, while planning for worldwide sim grid extensibility, I have chosen to configure the overall sim to represent 250-meter square patches of real earth using each of its 256-meter square regions.

This scales the real-world up a shade in the sim, to (1.024 : 1) but allows every fourth region in X and in Y to start on an exact grid kilometer. That scale produces 16.0000 regions per square kilometer, rather than 15.2588 regions/square km. From the geography side of things, this harmony is attractive since every fourth region will snap to a grid kilometer instead of every 1000th region. Even at that, the grid kilometer that 1000 of those 256 meter regions snap to is 256 kilometers, which is much clumsier to locate by name.

Thus the “well-tempered” moniker for this scale is well deserved, as any real-world USNG/MGRS grid coordinate could then be used to search for the relevant simulator region from a moderately simple bit of string manipulation. For Berkeley, and the western part of California, the zone is “10S” and the 100-kilometer grid within that is “EG” for San Francisco and Berkeley area. Put together, the US National Grid designator for the 100-km square is sometimes called “10SEG”, depending on where folks do or don’t put spaces.

If we always have exactly sixteen (16) regions per square kilometer, then we can use the shorthand version of the USNG grid names that only detail down to 10-meter increments. In this way, a region with its southwesterly corner at WGS84 UTM zone 10 north, 564000 meters Easting, 4191250 meters Northing, can have the US National Grid 10-meter designation of “10SEG 6400 9125″, which could be mashed together without spaces, or used to name a simulator region such as “10SEG_6400_9125″ in a slightly more readable form. For those of us who no longer have youthful eyes, the tiny little display on the Second Life client for the region name motivates the use of spaces.

So here’s a graphic of the plan: a 40-region prototype (5 x 8 regions), which will be configured with only Basic Physics, but real-life LiDAR-based terrain, and four megaprim sculpties per region to drape imagery (10 x 16 terrain sculpties) such as 10cm natural color orthophotography. Here’s where I hope to take this:

Over the past four weeks I’ve participated in a burst of effort that has involved multiple resolutions of GIS data to describe a campus terrain in an eastern US state. It’s been over twelve years since I’d worked with a 3D CAD design for a slope design, and this one brought back memories and put detail into a much finer scale, working with 30cm contours around one new building, where my earlier efforts were more like 2m contours over a large industrial site. All the same, having once “written” a slope design, I found it possible to “read” the design work that existed in various layers, some 3D and some 2D, in a new site design.

One of the beautiful and satisfying aspects of GIS work can be the integration of disparate data sets into a spatially coherent whole. In this case, I sought to make an OpenSim terrain that would be home to a very complicated build where two CAD drawings held structure details, a third held the site and design terrain, and as is often the case, a merge with surrounding seamless USGS DEMs was desired.

The first step was to ensure that the site base map and structure plans matched scale and could be lined up. A combination of grounding mat and retaining walls in the site plan made that possible, and in ArcGIS, the AutoCAD DWGs for the structure were translated onto the site plan drawing. That was a lot of work, with over 2Meg of polylines in the structure, and I knew from my Berkurodam build that keeping a complex structure square with the region grid would be frequently appreciated by those working this build.

To earn the appreciation of the build workers while merging the region with the surrounding world, it was necessary to define a projection-referenced local coordinate system for the campus. Mercifully, the site plan contained many building footprints across the street from the construction site, and these were easily identified in publicly available regional 30cm orthoimagery. I chose a well-defined corner of one building as my pivot point, calculated both WGS84 UTM meters and local campus grid inches (converted to meters) for that point, and defined grid north as 014 degrees azimuth on the UTM grid.

The facility for defining a local grid this way is easy in ESRI ArcGIS, and with it I could create a Feature Dataset dimensioned in meters for the campus grid. I was able to export the CAD drawing polylines into an inches-denominated version of the campus grid, then select a fistful of layers and export those into the meters-denominated campus grid for editing and 3D terrain work. It was especially helpful to have ArcGIS’s ability to project the UTM-based orthos into the local grid, as I was not able to figure out how to include an arbitrary grid rotation in an ERDAS custom projection. Once I knew that I had a projection-based local grid definition and could move GIS features and imagery back and forth from UTM, I imported two 1:24,000 sheets’ worth of 10-meter DEM data, about 6km by 7km of 30cm natural color imagery, and I was comfortable diving into the grading plan model knowing that regional context would be available.

The greatest labor involved reading the site plan drawing in detail, clipping out regraded areas from the existing terrain which was a 3D drawing, then editing each segment of each contour in the grading plan (sadly a 2D drawing) to have the proper elevation attribute so that it could flow into the existing terrain contour. A lot of this took careful contour reading and switching views between the multilayer DWG view in one ArcMap session (where I could read all the annotation like contour labels), and the imported polyline contours being edited sans annotation in another ArcMap session (where I had not had success importing the CAD annotation). One of my first realizations when I started keeping two ArcMap sessions running (to keep an eye on the CAD annotation) was that I had misread the drainage swales and had grown them out of the terrain, placing drainage lines farther upslope that was actually in the design.

After a couple of weeks calendar time, I had completed what might be the first 3D CAD model of the site as it was designed. At least, if a 3D model already existed, I did not have access to it. When every segment of every contour had the proper elevation attribute, I regenerated the 3D polylines with Z values only from the attributes. Then using typical techniques, I created a TIN with hard breaks on the contours, and ground out both 1-meter and 0.5-meter postings of gridded terrain from the TIN. This provided highly detailed terrain for at least one OpenSim region.

Merging this 1-meter posting terrain with 10-meter regional data that had been rotated 14 degrees was the next challenge. First, I made an effort to harmonize the vertical datums. The regional DEMs had typically excellent metadata and with very little interpretation, made it possible to define them as WGS84 NGVD29 meters, which I recalculated to WGS84-Geoid2003 NAVD88 meters. The CAD site plan fit most closely with the resulting grid when I just defined it as WGS84-Geoid2003 NAVD88 meters (from design Z-inches), although it remains about 1.5 meters lower at its edges than regional DEM, when overlaid on the regional data.

The typical challenges of oversampling coarse grids were quite graphic in the regional data. After using ArcGIS to project a WGS84 UTM copy of the regional DEM into the campus grid, I used ERDAS Imagine to resample that 10-meter source into a 1-meter working model. Having 100 grid cells with identical elevation looked unnatural at best, so I used an ERDAS focal analysis with a 7×7 kernel to estimate a mean value for every cell in the 1-meter resampled version of the local-grid-projected version of the regional DEM. While normally this would be an extreme focal analysis, it seemed to produce a reasonable result for the 10×10 oversampled regional DEM.

When the regional grid was prepared, I used ERDAS mosaicking to overlay the detailed site plan, and then found a desirable origin point and clipped a subset 768m by 1024m area that, based on the co-registered 30cm ortho imagery, just covered the campus area with 1-meter posting terrain and could be stood up as 12 OpenSim regions at 1:1 scale.

For texturing purposes, I used ERDAS to resample the campus grid version of the orthos that had been reprojected by ArcGIS, from the original 30cm to 25cm pixels. This allowed clipping of a subset of the model area as a 3072 x 4096 image, providing a 1024×1024 texture for each OpenSim region after dicing. For demonstration purposes, and as a guide to build-out of less detailed structures around the campus, these ortho tiles are used to texture on a 256m by 256m by 0.4m horizontal megaprim plate centered on each region. These textured megaprim plates make it easy to calculate sim region coordinates for buildings and objects visible in the ortho imagery.  These calculations should be useful when synchronizing real-world activity with its simulator facsimile.

What’s real-life terrain without real-life imagery to top it off with?

This is a view of Kentfield, Greenbrae, and the Corps of Engineers-enhanced portion of Corte Madera Creek.  The College of Marin athletic fields are on the left, Marin Catholic High School fields in the upper right, and Marin General Hospital in the lower right.  The NAIP imagery is textured on a flat 256 meter square prim fixed at 15 meters scaled elevation, so the sim’s terrain shows above it in several hilly places.

Here, another sim has Phoenix Reservoir and Bill Williams Canyon, both above Ross, California.  Mt. Tamalpais East Peak is behind the lake but does not yet have specific texture on it.  The edge of another 256-meter square prim with the next orthoimage tile to the west is visible in the right.  Two sims farther west, all of Bon Tempe reservoir imagery has been nestled within its proper terrain.

I wish that I could simply use my orthoimagery as a drape over the terrain; alas, that does not seem to be an OpenSim (or Second Life) priority at this time.  None the less, I have already created the first seven sims’ worth of image tiles, and they can be used to provide horizontal control for a great amount of future construction.

Tonight the ramps that serve as placeholders for the main entrance and exit escalators are in place under a provisionally textured cupola. Pudgy Darb fits down the ramps with a ~20% boost in width relative to RL escalators.

I’ve started to get the notion of how Darb now maps into full scale RL. He’s somewhere around 183 cm (6 ft) tall and likely would scale up to a weight of 135 kilograms (298 lbs). Because of this, and the lack of flexibility in his arms, some of the 1/3.048 scaled dimensions may need minor expansions.

Also, it’s not clear to me precisely at what level does Darb hit his head and can’t walk under an object, because the blockage is occurring somewhere above his hair in what looks to be vacant space. When ducking through the mezzanine which is now under trusses and jumping down to the platform (because there aren’t yet stairs or escalators in place) it is a lot easier to use Mouselook. Pulling back to the typical view where I see the av’s back as the SL client normally does, the view point is about where Darb’s head would be if he weren’t a Tiny avatar. That is to say, POV is about 1.8 meters above his feet, although his Tiny makes him only about 60 cm tall.

In open space having a full-height POV on a Tiny av is not a problem. Inside a tightly scaled structure, however, it can be very disorienting–and sometimes just doesn’t work if the POV tracks an av walking underneath the floor the POV is floating over at nose-length.
All the same, it’s pretty cool to see Darb walk down the escalator turn around and walk back up the other escalator, walk around the gates and hop down to the platform. I’m just getting the first flashes of what a Mouselook machinima might look like.

Next up will be placing the four-foot-high divider properly to separate the paid and unpaid areas of the station, visit the RL station with a laser tape measure to get real ceiling heights and floating escalator lengths, and nail down structure dimensions.

After the structures are in place, I expect to rent a dual-head strobe unit at Adolph Gasser, buy a fistful of Ektapress for my venerable Canon A-1, and put my vintage FD-14L rectilinear lens to some serious texture acquisition. I’ve spoken about this lens’ great suitability for this purpose for many years—and finally I seem to be a couple of weeks away from actually putting textures usefully into place!

Some have suggested that translating a real-world model into SL can be done more economically by reducing the scale to say 1:2 and using 1/4 as many sims. I wonder if that would cause the current density of prims to be not enough to fill 64K square meters?