Brown Computer Graphics Group Brown Computer Graphics Group

Tele-Immersion Applications -
An Electronic Book for Surgical Training

demo demo
Scene from original surgery
Captured scene rendered from reconstructed 3D geometry
with annotations and trainee's hands mimicing the surgeon (both images by Andrei State, UNC)

Project Overview

Tele-immersion provides a dramatic new medium for groups of people widely separated from each other to work and share experiences together in an immersive 3D virtual environment, much as if they were co-located in a shared physical space. Immersive time machines add an additional important dimension, that of being able to record experiences in which a viewer, immersed in the 3D reconstruction, can literally walk through the scene or move backward and forward in time. While there are many potential application areas for such novel technologies such as maintenance and repair, design and virtual prototyping, and paleontological and archaeological reconstruction, our focus is on teaching the surgical management of difficult, potentially lethal, injuries.

Today, the pace of surgical innovations has increased dramatically, as have the societal demands for safe and effective practices. The mechanisms for training and retraining suffer from inflexible timing, extended time commitments, and limited content. Traditional videotaped instruction has also long been available to help surgeons learn new procedures, but these videos are universally seen as marginally effective at best for a number of reasons: the fixed point of view that is integral to the narration, lack of depth perception and interactivity, and missing information. In short, the experience of watching a video is not sufficiently close to being there and seeing the procedure.

We are exploring a new paradigm for teaching surgical procedures immersive electronic books that in effect blend a "time machine" with 3D hypermedia. Our goal is to allow surgeons to witness and explore (in time and space) a past surgical procedure as if they were there, with the added benefit of instruction from the original surgeon or another instructor, as well as integrated 3D illustrations, annotations, and relevant medical metadata. The trainees should be able to freely and naturally walk around a life-sized, high-fidelity, 3D graphical reconstruction of the original time-varying events, pausing or stepping forward and backward in time to satisfy curiosity or allay confusion. Existing tools for time-varying 3D scene capture are crude at best, and the authoring tools, interaction metaphors, and "cinematic" techniques for this fundamentally new medium are unknown. We are bringing together experts in the respective disciplines and leverage our prior work in tele-immersion, to achieve our goals.


Brown University Computer Graphics Group, Prof. Andries van Dam

University of North Carolina, Chapel Hill, Office of the Future, Prof. Henry Fuchs and Prof. Greg Welch


Loring Holden and Bob Zeleznik

Back to tele-immersion home

home people information publications research outreach sponsors search
Brown CS Dept. STC HomePage
Search the Graphics Group Web Pages

graphics web master