Simulator GIS – Darb Dabney’s OpenSim

Many thanks to Singularity U, director Matt Rutherford, and to Randall Hand who brought it to my attention After chatting at SLCC 2009 this past summer, I appreciate the immediacy of this lecture.

It’s hard for me to listen to the entire talk and retain the best explanations – but clear and current they are.

I don’t have much to say about these regions that hasn’t been written already, and my views have been less aesthetic than Shenlei’s.
But in the interest of boosting the bandwidth by which I can share OpenSim, I’ve invested in a much newer Adobe Premiere Elements than I’d been using for the past five or so years. It’s a gas to have it multi-thread while rendering, and I have direct-to-FLV write. Trying to share as much of the motion and fidelity via YouTube as I possibly can, I’ve crafted a video resolution that is a multiple of my Hippo / SL viewer screen. The FRAPS video direct to AVI (sorry, it’s Win XP) is 1600 x 1140 @ 10 fps. Yup, those are video frames. In the interest of surviving an upload, I’ve rendered them highly time-compressed, with output at 1515 x 1080 @ 15 fps. As of tonight there’s no sound, no intertitles, just the rushes.
oops, if I read the YouTube Instructions for best formats, I should have trimmed the width to 1440, which is a multiple of 16.
Also, I have more direct upload options now with Premiere 8 than I had with my (recently demised) copy Premiere 1.0. Go Figure ;^)
While the Windows box grinds out the video print, I’m over here on Ubuntu blogging in a tab of 64-bit Chrome 4.0.249.43 and it is fine & fast.

For these videos, I visited ScienceSim Geography22_44 region and set the view to wide angle, then sat up at about 500 meters and watched the regions rez their terrain. For some folks, it will rank right up there with watching lead-based paint dry. For geography folks I’m hoping that these few minutes of sped-up video will convey, by dogged repetition, the primacy of regions in the provision of virtual environment simulators.

By the way, I’ve got a task: I need to find a better buzz word for the GIS community. I’ve been advised by some serious and well-intentioned (not to mention well-informed) folk that terms like “virtual” and “immersive” are actually boring to GIS’ers. So I’ll need to think about how to convey the concepts of “Mirror World”, “Multiuser Virtual Environment”, “Immersive Connected Experience”, “Third-Person Virtual World”, and related concepts into a catchy moniker. Hopefully, one that is not presently trademarked, either!

I’m trying to remain serious about this, but some of the options are treacherous. Geography in Social Media has a possibly awkward acronym; maybe it can be saved in recognizable form as “GIS for Social Media” or “Geography for Social Media”: GFSM
The term “3D Map with Me” is terse, slightly ambiguous

Here is the video chopped as it was when uploaded with 1515 x 1080 resolution. Problem with that is that by not preserving dimensions at a multiple of 16, and saving my viewer’s aspect ratio rather than the (standard since 16mm film) 4:3 aspect, my upload is clobbered into something perhaps suitable for a smartphone. So please consider this the Smartphone Version of last night’s rushes:

Then, once again with feeling, or at least with a little more rest, there is what I hope to be an HD-friendly moving vision of OpenSim, as it appears on the ScienceSim Geography regions. Yes! After it ripened on the YouTube servers for a few hours, I now see all the higher-res versions available. At 1440 x 1080, this is pretty close to what I see on my screen with a live Hippo viewer.

And after a day’s cogitation: anyone care to comment on the term: “Social Immersive Media GIS” as a moniker? Oops — I used “immersive” 8^(

I’d read about this, but never before experienced the agony first-hand.  Extracting funds from SL, the wait for funds to arrive at PayPal was a bit slow.  In fact, in the time it took funds to go from Linden to PayPal, a bamboo shoot in my back yard could have grown taller than me (that’s my RL not SL height!), and would have been over 2 meters tall.  Anyway, Process Credits are quite lacking in symmetry with how quickly credit charges can flow into the Linden realm.

During this week of waiting my random prims have been cleared out from Amida and nary a trace of Berkurodam BART Station remains besides a video in Gualala.  The video screen was actually entombed by a neighbor, who may not like it but did not send any message.

Anyway–for me this week is all about generating maps and graphics while keeping up with work.  I’ve generated a 50cm terrain grid for parts of my county where perhaps 150,000 people live.  With computational process improvements I should be able to make production stable enough to generate a 25cm grid.  The point is to model terrain slope and aspect within urban parcels.  OpenSim can pack 64 terrain megaprim sculpties over each region to refine terrain more than the built-in 1-meter postings, and display 10cm orthoimagery at full resolution.

Last year, I used first-return LiDAR data of the UC Berkeley campus to generate a 25cm grid for 10cm imagery.  Now, I’m working with bare-earth LiDAR data from FEMA, topographic contours (densified to 1.5m vertex spacing), and most importantly, photogrammetric terrain and water break lines.

Throwing all those data into the mix, the data are built into an ESRI Terrain Dataset, from which I generate TIN and GRID models at various reolution and extent.  The ESRI ArcGIS 3D Analyst Terrain-to-TIN generator breaks down after about 10 mega-faces (so would I…)  And the ArcGIS Terrain-to-GRID generator seems to drift into Windows-unconsciousness after about 1.0 giga-cells.  So for the grid, I break it down and do the pieces, then merge the tiles using ERDAS Imagine, because the ESRI ArcGIS raster mosaic function does not produce output grids much over 10 GB.  As annoying as learning these ArcGIS limits can be, it is very satisfying (and instructive) to see huge swaths of seamless terrain with great detail once it all comes together.  Thanks to the break lines, many driveways and most home building site cuts and fills are resolved.  And it will be a lot of terrain by OpenSim standards–enough to calibrate terrain for over 20,000 contiguous regions–not that I ever expect to build it all at 1:1 scale!

<<updated 20090424>>

As my patience with Second Life wanes, and I wait for more architectural input for my next SL build project, I have a dark OpenSim server with no fixed IP.  I’m having stability issues with the Linux SL client, but have upgraded the workstation to Ubuntu 9.04 Jaunty Jackalope.  Google Earth client there is more stable, the NVidia drivers install themselves (sans Envy), and everything Ubuntu-wise seems to be getting incrementally better by the quarter.

I’m grinding some large images that have taught me that one very special difference between Windows XP variants and Windows Server 2003 is the latter’s ability to open files on the high side of 80 GB.  I’d never quite realized it before but the moderately massive mosaics that I have created in years past (edging toward 250 GB single files) actually depended on Server 2003 to get created.  Once the destination file exists, then XP can take it from there, and in all cases Windows Explorer can copy the monster files.  But in that tenuous moment when a mosaic first grabs its space on disk for a huge output—one can’t seem to do that with XP.

So while I’m enjoying Google Earth on Ubuntu, there is something cool that I go back to Windows for, and that’s the new Google Earth browser plug-in.  Since I’m gaining a bit of facility with the keyboard shortcuts in the full-stop Earth client, these all carry over to the plugin.  My first test page has been stood up here and I’ve been deep into four continents with it so far.  I understand that the plugin is only available for Windows and Mac systems at this time.  If you can,  Enjoy!

http://earth.jedi.bz 

Also, as I get even faster with my keyboard navigation of G-Earth, I’ve actually seen some artifacts that are quite familiar from OpenSim.  While zipping about between the Gulf of Yakutat and Canada’s Mount Logan, at certain viewing elevations I can accelerate the point of view forward quite fast.  Doing so in this very mountainous terrain, I saw blocks of terrain standing up along what look like sim edges, resolving in a few seconds as more (sculpty?) bumpmap arrives.  This is the same sort of artifact I’ve seen with terrain sculpties and sometimes, with region crossings in OpenSim.  Also, I’ve found a couple of wild terrain grid errors in G-Earth.  In one, a quarry dug hundreds of feet below sea level, right next to the sea, is displayed as positive elevation (absolute-value terrain, anyone?).  In another, a boundary between US and Canadian terrain has a glacier flowing uphill onto a plateau.  Go figure.  Blame Canada! ;^)

Only the four-story Administration building (wing), not the two-story Hall of Justice. I’m tired so I’ll let the shot speak for me.

photo from 2009 01 30

To me, it’s mildly amazing to realize that F.Ll.Wright’s design fits so snugly in 1/8 of a Second Life region at 1:1.00 scale.  The Civic Center Administration building is a Real-Life building that can be visited, providing an easy way to get a true RL immersive sense of its scale.  Building at 1:1 scale in Second Life for the first time, this has been my first experience of transferring that awareness into the multi-region contigous space of the very beautiful Second Life.  Sure, I’ve built large areas at 1:1 using draped LiDAR data, but to have a rather large single building (or at least its footprint for now) in context with existing builds that I’ve seen for months, well, at the moment SL seems larger than I’d thought.  That shift in my perception of SL scale may be the contrast between flying (quite fast as it turns out) around 40 to 100 OpenSim regions versus walking around the site and knowing how long it takes to traverse the RL building.

Anyway, check out the build’s progress at secondlife://Stanford/100/235/30

Today the (San Francisco) Bay Area Automated Mapping Association hosted a wonderful URISA Certified Workshop given by Tim Case, describing Best Practices and Project Implementation Methods for 3D Geospatial work. The all-day event provided a very broad and even-handed overview of many 3D technologies that hold promise for the near future.

With this presentation as an extra boost for my focus on a new build, I’m gearing up with even more enthusiasm for a new build in the Agni grid Mainland.  I’ve also tuned the Berkeley parcel for sale.  Its price amounts to about US$382.00, and that price is set to help cover purchase costs for the next build’s likely parcel.  The tuning involved reducing the parcel size by 64 square meters, so that the three Gualala parcels total 4608 square meters, or exactly the maximum amount allowed for Linden Lab’s US$25/month tier rate.  With that size, it would be possible for an interested party to purcahse the Berkeley BART station and maintain it for $300/year in tier (the Linden land property tax).

Also, based on today’s Geospatial tag, I’ve noted just this morning two mentions of the Berkeley BART build.  The New World Notes item by Wagner James Au 2009 01 19 was wonderful to find after our in-world messages last month.  For clarification, while true at the time of that conversation, no longer do I work for City of Berkeley.  The TidalBlog item by Peter Miller mentions interesting developments in the overlap between simulators and geospatial models, as well as some shots from his visit to the Berkeley BART model.  Thanks to both authors for their posts!

Silent though these pages be, much has been thought.  I’ve had some quality time with inquisitive Lindens and learned to expect some sort of standalone Linden server product along about 2009.  For me, that’s a game-changer as it’s hard enough to suggest (at work) creating content without also keeping up with an open source thread to stand that content up upon.

This past week I’ve made a real-life geographic shift for a family event, and learned that I’ve got a relation involved in the study of architecture.  That insight has reinvigorated my interest in Jon Brouchoud and some of his writing here.  The notion of architecture as it is currently an academic subject, versus architecture as a current professional practice, and the disruptive possibility of widespread virtual world deployment—this is a notion not so different from geographic science as an academic subject, GIS as a professional practice, and the possibility of immersive 3D disruption of the status quo.

Others in academic circles, including University College London, Centre for Advanced Spatial Analysis (CASA), published a Working Paper about the time this past summer when I was so focused on my 1:1 immersive build.  It was gratifying to see the CASA acknowledge Second Life technology’s place in the world of neogeography and geospatial informatics.

Sitting side by side yet somehow abstracted from mapping, gaming and digital earths
is Second Life and other similar virtual environments. Second Life and their like are
easy to dismiss as pure distraction and entertainment. Yet look under the lid of
Second Life and it contains one of the most powerful geographical data visualisation
kits available

And the fine writing and attention to detail of Jon / Keystone was spotlighted in NY Times’ Style magazine this past weekend.  It was a pleasure to share that link with architecture students!

It’s a big world, and immersive 3D systems must balance the tradeoff between quality and performant physics, and an economically practical level of large land areas served up to relatively sparse users, if we are to identify applications that consume vast tracts of GIS data.  Spanning that scale will require that standalone Linden servers have the ability to shortcut some of HAVOK’s demands to pile in many more than four regions per physical server.  After all, if I can get 100 regions stood up on a 1 GHz Celeron using OpenSim, then a four-threaded dual 64-bit Xeon server really ought to do the same for standalone Linden regions, right?  I surely hope so.

The San Francisco Bay Area Automated Mapping Association is our local URISA chapter, and publishes a twice-yearly journal that covers some interesting local geospatial projects.  The latest,  BAAMA Journal Volume 2, Issue 2 was released today for GIS Day.  It contains one article that provides an overview of the work blogged here: “IMMERSIVE 3D SIMULATOR-BASED GIS: SHARING THE 3D EXPERIENCE” The shot below details Mulford Hall on the UC Berkeley campus where our local GIS Day event was held again this year.  Thanks to the GIF, ASPRS, and BAAMA organizers!

Detail of 1:16 Level 2 model, in Second Life Agni grid, Amida region, on 2008 11 19 GIS Day

I’m finding little time for keeping the OpenSim instance current.  For me, there’s been a lot of problems with the more recent (last six weeks) versions simply working on first try.  Also, since I have so many hours invested in the content that was created at rev 5411 that I’m a bit skittish with the bleeding edge updates.

Most recently, I’ve had the experience at osim.bargc.org of having only a single region be accessible at a time with the latest 1.21.6 SL viewer.  So I vacillate between thinking “how convenient and attractive to use hosted Second Life Grid servers” and the hot-rodder thoughts “My 40-region sim is worth $7525 up front and $1610/month in tier for a nonprofit, so I can be tough.”  I do tire of keeping the OpenSim server up and running with its load of content, yet with this real-world economy even avatars need to be frugal, no?

Thanks to Gualala neighbor Misty Rhodes for background

One thing about these tiny builds is that they’re easy to see from one end to the other, so why not make a video of these miniature builds in Second Life?  I offer this for the amusement of Geospatial Information Service (or Geographic Information Systems if you prefer) folks who may be introducing themselves to immersive 3D.  International GIS Day will be here in a couple of weeks, so I’m posting this now.

 I’ve also challenged myself to improve my video production standards.  Who knows, maybe more than 1300 people will view it if I make it more fun to watch with a bit of editing and title-based metadata?  Nothing deep is intended with the score, it just caught my attention as matching the length of the machinima rushes tonight.

I’ve tried to improve the video with some titles to explain what’s being seen at the Level 1 (bare earth with draped ortho) 1:42-scale build, Level 2 (first-return reflective LiDAR gridded surface with draped ortho) 1:16-scale build, and Level 3 (full immersive 3D vector features in Second Life primitives with real world textures) 1:3-scale build.

If the embedded link does not work, the video is hereколи под наем which is at http://www.youtube.com/v/6joRvDH52jU