The core team of the two companies that I run (the Institute for the Visualization of History & Learning Sites, Inc.) is the oldest continually operating group active in the field known as virtual heritage--that is, using interactive computer graphics for the collection, study, teaching, publication, display, and broadcast of information about the past (see a sampling of our projects at the left; hover over to enlarge). We are archaeologists, art historians, architectural historians, and information scientists working with programmers and computer graphic artists.

Among other things, we have been providing museums with innovative visual content for over 10 years (see the sample listings at the left; hover over to enlarge).

And we've collaborated with curators and educators at such institutions as Old Sturbridge Village and Germantown, Pennsylvania, the British Museum, the Miho Museum, the Oriental Institute Museum, the University of Pennsylvania Museum, and the Petrie Museum. Our projects include digital reconstructions based on architectural fragments in museum collections, putting objects from galleries back into simulations of their original contexts, assisting in staff research projects, and developing outreach materials.

3D computer graphics offer many advantages for conveying alternate realities for museum visitors; of providing them with portholes to the past (see the aerial rendering of Nemrud Dagi, Turkey, at the left; from the Learning Sites virtual reality model of the site; hover over to enlarge). Not simply pretty pictures, but instead imparting engaging adventures, personalized experiences, and contextual interpretations of the items displayed before them. 3D computer graphics can tell stories, inform, educate, engage, excite, and provide memorable experiences in ways not possible with traditional visualization modes or media; in ways that supplement and enhance other themed events and exhibits; with the goal of offering visitors a glimpse into what happened at an ancient or historic site, who some of the inhabitants were, and why visitors should return to the museum again and again.

My topic today is the use of virtual reality in museums, such as our Virtual Site Museum (or VSM), a concept we developed collaboratively with the University of Buffalo (hover over the image at the left to enlarge). The VSM is a fully interactive, virtual environment for museum gallery visitors, educational and promotional outreach, and curatorial research. The virtual world contains precise, authoritative, and integrated archaeological and historical data culled from published and unpublished excavation records and holdings at various museums and archaeological sites. Objects in the virtual world are linked to 2D images, explanatory text, and online resources to produce an engaging and educational participatory experience. The resulting real-time module can be viewed either in full-body immersion or on gallery-based computer kiosks.

Once a precise, accurate, and detailed 3D computer model has been built, there are several possible output visualization types (you've seen some already; see some definitions at the left; hover over to enlarge).

There are individual static renderings, extracted from virtual cameras in the 3D model (see the example at the left, showing the 3D modeling software and the resulting rendering; hover over to enlarge)...

    and computer animations which are like movies in that they are linear presentations often with voiceovers, sounds, or overlying explanatory text, but these, too, are passive providing no user interaction (as in the short snippet at the left, from the Learning Sites 3D model of the Northwest Palace, Nimrud; hover over the image for controls).

Then, there is QTVR, which requires the viewer to navigate by looking around in a scene as if standing in the center of a bubble the interior surface of which is covered with flat images. QTVR gives the impression that you can look everywhere. But since there is no real 3D space, you cannot walk around, or look behind anything, because your viewpoint is fixed at the center of the bubble (see the example below, from the Learning Sites 3D model of the palace at Til Barsib; ed. note 10/2019: the original QTVR file format is no longer supported by Web browsers, instead, we have converted the scene into a similar bubble-like environment using WebGL / HTML5; left click and use your mouse to turn and zoom).

In contrast, true virtual reality is a digital replica of actual 3D space, allowing the viewer to go anywhere and closely examine objects from all sides (as for example, the Roman ramming prow seen at the left; the photo of the object in the museum in Israel, and the interactive VR environment below; ed. note 10/2019: the original presentation showed a VRML version of the ram, but that file format is no longer supported by Web browsers, so we have recoded the model and exported as a WebGL virtual world; left click and use the mouse to rotate and zoom). Virtual environments can be further enhanced with sounds and animated characters. VR allows users to explore at their own pace, see, hear, and touch things in ways that come very close to a first-person experience

Virtual reality is an ideal visualization and storytelling technique for museums, because it can so artfully evoke any time period or situation (ed. note 10/2019: in the original presentation, the virtual world of the double house from al-Meragh, Sudan, was a VRML world, but that format is no longer supported by browsers, so Learning Sites recoded the world for the current standard, WebGL; that world can be enjoyed below; use your mouse to navigate the world, 'c' freezes (and unfreezes) the world so that your cursor can click on green-highlighted hotspots; click on the 'i' in the upper right corner to access additional interactive content).

Most visitors to museum galleries or living history or archaeological site venues do want to know more about the artifacts on display than can be gained by just looking. Sure, by just looking they can marvel at the beauty of the objects and at the ingenuity of the makers. But without some information about the historical and cultural contexts of the things exhibited, the experience will be limited.

Integrating computer reconstructions into galleries to show artifacts, buildings, and even entire sites in context offers unprecedented possibilities for showing how the past originally looked and how objects on display were used (see for example the images at the left; a photo of Area G, City of David, Jerusalem showing the excavated remains; and a Learning Sites reconstruction of one of the houses found there; hover over to enlarge each). We've all used reconstructions for a long time already. Whether as drawings or dioramas, they help visitors understand what they're looking at.

Digital reconstructions can be so much more immediate, vital, and evocative than traditional paper-based depictions (the drawing at the left illustrates the same house as is rendered above; hover over to enlarge).

Which brings us to storytelling. Stories are a nourishment for hungry minds and souls. When we visit historical sites or museums, we don't want just dates and names; those are the dry statistics of our school days. We want to be engaged, challenged, entertained; we like being told stories; the more vivid the better, and so our job is to visually re-create history's tapestries in ways that connect a diverse set of visitors with specific places, times, and events (such are the storylines listed to the left; hover over to enlarge).

We can do all this very well with computer graphics. The very nature of the simulated realities presented by the 3D models can focus attention on a single storyline, present multiple storylines, or personalized storylines, and supplement and complement the other themed installations (see the virtual interview at the left; hover over to enlarge).

Such applications are not new. Museums have used interactive computer graphics since the beginning of 3D graphics themselves. For example:

• Natural History Museum, London, had VR kiosk showing an interactive 3D model of Captain Cook's ship Bark Endeavor (late 1990s).
• San Diego Natural History Museum has a current exhibit on the Dead Sea scrolls that includes an interactive reconstruction of Khirbet Qumran. "The result is a journey back into a world that influenced the birth of modern Judaism and Christianity. There are two important benefits of modeling archaeological remains in a virtual environment. First, the model allows us to visualize ancient sites in a way that assists the researchers in communicating a vision of the site, thus revealing to laypeople, who often can't easily visualize three-dimensional structures from site maps or floor plans, what the site looked like in antiquity. And second, and perhaps more valuable, 3D models allow researchers to test new theories, ideas, and reconstructions, much like an automobile designer tests certain designs for structural and performance flaws in a virtual setting before producing the actual vehicle."
• Canadian Museum of Civilization had objects scanned into 3D models and made available for study at museum and on their Website.
• PowerHouse Museum, Sydney, Australia, created a virtual Olympia for display and visitor exploration during the Athens Olympics 2004; currently has huge 25-set interactive exhibit on special effects that make extensive use of interactive 3D graphics.
• Canterbury Museum, built a VR reconstruction of Roman Canterbury for museum kiosks and online.
• Davidson Center, Jerusalem, displays a VR re-creation of the Temple Mount.
• Foundation of the Hellenic World museum and gallery houses several immersive and non-immersive VR exhibits.
• National Museum of Marine Biology and Aquarium, Taiwan, has personal computer-based visualization systems simulating life-sized rare or extinct aquatic life as interactive 3D graphics.

Growing interest in complementing gallery visits with computer graphics has prompted Acoustiguide to build Opus, a video-enhanced wand integrating real-time images and audio. Acoustiguide also designed a special digital guide for the Museum of Modern Art, NYC, offering multiple learning styles to choose from, architectural history of the building, and a guide to its collection.

Further, SecondLife and other perpetual online 3D worlds provide a real-time graphics environment that can be used by museums for education, collaboration, innovative displays, and publication, just as these worlds are used by businesses.

The first of our sites to be documented in our Virtual Site Museum module was the 9th c. BCE Northwest Palace of King Assur-nasir-pal II at Nimrud (ancient Assyria, now in northern Iraq). We chose this palace as our first VSM project: (1) because it is in relatively good (though seriously endangered) condition; (2) because it is the paradigm palace for the Assyrian empire and all subsequent Near Eastern palaces up to and including the former palace of Sadaam Hussein; (3) because artifacts and decorative reliefs from the palace can be found in dozens of museums around the world; (4) because the original sculptural program and architectural complexity of the building can no longer be studied or understood with traditional visualization media; and (5) because we believe that this kind of real-time virtual exposure serves as a valid proxy experience in an age when actual site visits have become either too expensive for scholars and interested laypersons, impossible if a site no longer exists, or, as in the case of Nimrud, too life-threatening (see the compendium of images of the palace at the left; hover over to enlarge; ed. note 10/2019: the Northwest Palace was completely dynamited by ISIS/Daesh in 2016; so the Learning Sites virtual reality re-creation of the site remains the only visual survivor of the Assyrian architectural masterpiece). 

The Hood Museum of Art, Dartmouth College, was the last venue to host this Northwest Palace museum package, from late 2006 to earlier this year (see images at the left from the installation; hover over to enlarge). 

The Northwest Palace is so-named simply because it was found near the northwest corner of Nimrud citadel mound (see the plan at the left; hover over to enlarge). During excavations in the mid-19th century, the stone reliefs found all over the Palace's walls were removed and distributed to the excavator's friends, family, and the British Museum (see a drawing at the left showing removal of the wall panels in the 1840s; hover over to enlarge).

Later, visitors to Nimrud received permission, literally, to mine the site of what were considered duplicate images, some of them among the finest examples of ancient bas-relief sculpture. Tracing the buying and selling of these reliefs and fragments of reliefs over the last 150 years has become a daunting task. Examples of sculpture from the Northwest Palace have been identified in 65 museums and private collections across the world. The site is particularly difficult to visit these days, even by locals, let alone for school kids from the United States or scholars from around the world. Even if you could go to Nimrud, what you would see there would not be very helpful in understanding the sculptural program, the scale of the building complex, the decorative details, nor the original sightlines, lighting, or circulation patterns (see the composite image at the left, showing, inset, a view of the palace throne room as of c.2002 [image courtesy of Michael Weigl] and our reconstruction of the room from the same vantage point; hover over to enlarge).

   The situation is an ideal candidate for creating in a virtual site museum. The Hood Museum came to us early in their exhibit-design process that would celebrate the 150th anniversary of their purchase of five reliefs from the Palace. The Hood arranged several meetings and presentations with me and their staff, curators, and college professors to outline the broad themes of the visualization modules, upon which were based the scope of the 3D environment. The main thrust of the Hood's desire to use computer graphics was to show visitors the original contexts of their massive reliefs, to explain their iconography, and to offer far more information than traditional brochures, labels, or gallery guides could. The cost of the final deliverables was covered by the Museum from grants, donations, and in-house research budgets. The Hood team received from us interim versions of this animation and the real-time environment for testing and comment (view a snippet of the animation at the left; once it has begun, move the cursor off the image to see the text).

Let's look at the issues another way. Creating images intended to reproduce visual experience has been part of human activity for millennia. From the earliest known cave paintings, humankind has exhibited an innate drive to represent the world around it. In addition to visual images, theater and literature have all offered views of other places, other experiences, other beliefs, and other times, to stimulate the imagination, to instill wonder about the fantastic, and to speculate about the spiritual. As technologies have changed, so have the means for stimulating the eye and the imagination, from mathematical perspective in the 15th century to motion pictures in the 20th century. People are fascinated by the illusion of being there.

Representation has also been used throughout human history to educate and entertain. Visualization is a recognized means of making data and concepts more easily comprehended and assimilated. So, instructional books are illustrated, and audiovisual materials are widely used in classrooms. And now along come computers and interactive 3D digital visualizations. VR is just another step in the age-old drive toward heightened visual experience based on reality.

In conclusion, the many benefits of the Virtual Site Museum and the integration of 3D computer graphics into museum operations underscore how important visualization and context are for comprehension; for education; and for presenting much more content, all of which leads to increased exposure and visitorship, plus that added bonus of tapping into and attracting younger audiences. The various stories that curators and educators tell with gallery design and didactic materials can all be enhanced through the use of interactive 3D computer models.

The Virtual Site Museum is but one example of how virtual worlds can inform, engage, and excite. Once the models are built, there are other supplementary uses of the 3D data, such as we have seen in the flythrough of the Palace.

But VR is merely one of the current visualization options. Soon, you'll be able to offer visitors individualized hand-held devices, with GPS locators, real-time holographic screens, and virtual tour guides for personal and personalized tours or group experiences, able to link museum to museum for a global merging of collections. Nothing can replace seeing the real site or real artifact, but think how much more visitors would get out of coming to your museum once we realize that the study and display of the past need not be constrained by the methods and technologies of the past.