Virtual Reality and Physically-Based Simulation WS 15/16
Virtual Reality (VR) is a research area at the intersection of computer graphics, physically-based simulation, and 3D human-computer interaction (HCI). VR is concerned with novel interaction technologies and interaction metaphors in virtual environments (in particular intuitive and direct metaphors), immersion and presence, and real-time rendering. Another important topic is physically-based simulation in real-time, which tries to simulate real-world phenomena such as fire, cloth, the behavior of rigid objects when colliding with each other, fluids, or objects made of deformable material.
Over the past one or two decades, VR has established itself as an important tool in several industries, such as manufacturing (e.g., automotive, airspace, ship building), architecture, and pharmaceutical industries. In addition, a lot of the techniques and solutions developed in the area of VR can be applied directly to computer games.
In this course, we will first look at the fundamental methods, and then go on to more advanced algorithms that are needed to build complex and full-fledged VR systems or real-time computer games. Example topics are object behavior, acoustic rendering, haptics, and collision detection.
The assignments will be mostly practical ones, based on the cross-platform
Participants can choose the programming language from
You are encouraged to work on assignments in small teams.
Some of the topics to be covered (tentatively):
- Introduction, basic notions of VR, a few applications
- VR technologies: displays, tracking, input devices, software design
- Stereo rendering
- Error correction: tracking correction, filtering
- Techniques for real-time rendering
- Fundamental immersive interaction techniques: gesture recognition, navigation, selection, grasping, menues in 3D
- Complex immersive interaction techniques: world-in-miniature, action-at-a-distance, magic lens, etc.
- Particle systems
- Spring-mass systems
- Force feedback: rendering forces
- Collision detection
- Acoustic rendering
Note: this list is just tentative and subject to change during the semester.
The following table will, eventually, contain all the topics that were covered in this class, the accompanying slides, exercise sheets, and frameworks for solving the programming exeercises. (This table will be filled week by week.)
Introduction (definitions, immersion, fidelity, presence, history, Milgram's continuum),
|2.||Scenegraphs: immediate / retained mode, semantics of nodes and edges, special issues with light sources, shared geometry (instancing), thread-safe scenegraphs, distributed rendering, fields & routes concept,types of nodes, specification of the material using the Phong model, indexed face set,|
|3.||Stereo rendering 1: depth cues, human visual system and stereopsis, horopter, stereo projection, multiplexing techniques for stereo images, immersive projection technologies|
|4.||Stereo rendering 2: issues with stereo rendering (depth aliasing, convergence-focus incongruity, in-correct viewpoint, stereo violation), hypo- and hyper-stereo, automultiscopic displays, pre-distortion for HMDs, low and full persistence,|
|5.||Real-time rendering 1: simulator sickness, latency and its sources, view-independent rendering, prioritized rendering, level-of-detail techniques, static/dynamic/psychophysiological selection, predictive LOD selection, progressive meshes, portal culling,|
Real-time rendering 2: state sorting using online sorting buffer algorithms,
stereoscopic image warping,
latencies with conventional rendering,
reducing latency using time warping
Input Devices: degrees of freedom, multimodality, virtual trackball,
directness continuum, isotonic vs isometric,
tracking, data gloves, locomotion devices
|9.||Interaction metaphors 1: Universal Interaction Tasks, design of user interfaces, gesture recognition, the taxonomy, abstract representation of the user, navigation metaphors (point-and-fly, scene-in-hand, two-handed, hands-free, et al.), user models (power law of practice, Hick's law, Fitts's law), task decomposition, control-display ratio, selection techniques (go-go, ray-based techniques, flexible pointer), bent ray, cone-based technique with ranking, balloon selection,|
Interaction metaphors 2:
object manipulation (grasping and moving),
Natural User Interaction,
principles for interaction design: action-at-a-distance,
image plane interaction, proprioceptive interaction,
tangible user interfaces.
|11.||Particle systems: dynamics/kinematics, Euler integration, phase space, definition of particle systems, physical effects, non-physical effects, collision handling, rendering, rendering transparent objects, flames & fire, procedural modeling of plants, the concept of streaming architectures|
|12.||Mass-spring systems: Newton's Laws, single spring-damper, explicit Euler integration, instability and error accumulation with explicit Euler integration, Runge-Kutta, Verlet integration, constraints,|
Mass-spring systems 2:
implicit integration, tangent stiffness matrix, comparison to explicit integration,
mesh creation for volumetric objects, consistent collision response for volumetric
Haptics: applications, devices, the haptic loop, human factors, haptic textures, buzzing, intermediate representations, impendance/admittance approaches,
motivation, definitions, collision detection pipeline, broad/narrow phase,
separating planes algorithm (for convex objects),
Mikowski sums, intersection test based on Minkowski sums, hierarchical coll.det.,
bounding volume hierarchies, types of BV's,
overlap test for k-DOP's,
construction of BVH's,
inner sphere trees, sphere packings, proximity computation using ISTs.
- Ralf Dörner, Wolfgang Broll, Paul Grimm, Bernhard Jung (Hrsg.): Virtual und Augmented Reality (VR/AR), Grundlagen und Methoden der Virtuellen und Augmentierten Realität. Springer 2013. You can read the e-book from within the university's network.
- Kay M. Stanney (Ed.): Handbook of Virtual Environments. Lawrence Erlbaum Associates, 2002
- William R. Sherman, Alan B. Craig: Understanding Virtual Reality. Morgan Kaufmann.
- Don Brutzman, Leonard Daly: X3D: Extensible 3D Graphics for Web Authors. Morgan Kaufmann, 2007.
- Daniel Fleisch: A Student's Guide to Vectors and Tensors. Cambridge University Press
- Kenny Erleben et al.: Physics Based Animation. Charles River Media, 2005.
- Mario Gutiérrez, Frédéric Vexo, Daniel Thalmann: Stepping into Virtual Reality. Springer, 2008. You can read the e-book from within the university's network.
- Anthony Steed, Manuel Oliveira: Networked Graphics: Building Networked Games and Virtual Environments. Morgan Kaufman, 2009. From this book, only chapters 7, 10, and 11 are relevant to this course.
Warning: these text books can only give you a general introduction to the field of VR! Most of the topics taught in class will not be covered by any of these text books directly -- in fact, AFAIK there are no text books that cover these topics. Therefore, I recommend to attend class.
If you are thinking of buying some of these books, then I suggest to consider buying a used copy of them -- very often, you can find them at a fraction of the price of a new copy. The following are two very good internet sites for finding inexpensive used copies of books: Abebooks and BookButler.
In order to acquire the "Schein", you will have to do a number of assignments. These will be mostly small practical programming assignments (based on a freely available VR system).
Online Literature and Resources on the Internet
- The original article praising the panorama in Blackwood's Edinburgh Magazine (vol. 15, 1824)
- Another visionary article by Vannevar Bush: As We May Think (1945)
- Interview with Jaron Lanier (it was part of the program "Druckfrisch" and aired on 5.10.2014 on the German ARD)
- Literature on stereoscopic ("3D") rendering:
- Implementing Stereoscopic 3D in Your Applications by Samuel Gateau and Steve Nash from NVidia at GPU Technology Conference 2010 (Source)
- Rendering 3D Anaglyph in OpenGL by Animesh Mishra, 2011 (Source)
- A very nice chapter on Binocular Vision and Space Perception. A pretty accessible read.
- A very good conference tutorial talk on factors and limits of human stereoscopic vision from a perception and neuroscientific perspective, by Martin Banks, 2013.
- More literature on the topic of user interface design:
- Here are the -- not quite serious -- Cartoon Laws of Physics ;-) .
- Literature on particle and on spring-mass systems:
- William T. Reeves: Particle Systems - A Technique for Modeling a Class of Fuzzy Objects;
- More advanced particle systems (e.g., n-body systems) are explained in Real-Time Particle Systems on the GPU in Dynamic Environments by Shannon Drone (Source)
- Georgii, Westermann: Mass-Spring Systems on the GPU, Simulation Practice and Theory 2005. (Source)
- Literature on physically-based simulation in general:
- Introductory Vector Calculus by Norman Wittels (Source)
- Siggraph 2008 course notes: Real Time Physics
- Siggraph 2001 course notes: Physically Based Modeling
- Survey paper over the field of deformation simulation methods: Physically Based Deformable Models in Computer Graphics by Andrew Nealen, Mathias Muller, Richard Keiser, Eddy Boxerman and Mark Carlson, EG 2005
- A bit on the history of Virtual and Mixed Reality:
- An article from The Verge: The Rise and Fall and Rise of Virtual Reality by Adi Robertson and Michael Zelenko (Source, in particular here)
- Highlights from an interview with from The Verge: Digital Natives, A conversation between virtual reality visionaries Jaron Lanier and Kevin Kelly by Casey Newton (Source, in particular here)
- The Ultimate Display by Donald Sutherland
- The short novel
Gegen den Strich
by Joris-Karl Huysmans.
(Sorry, I have it in German only)
Denkanregung: was hat das mit VR zu tun?
- Help and documentation for the Unreal Engine:
- For general help, take a look at the Unreal Engine 4 Documentation and in case of problems also the UE4 AnswerHub
- For quick help you can visit the IRC channels #unrealengine and #ue4linux on Freenode (web client)
- Compiling and running Unreal on Linux is explained in the wiki
- To setup c++ debugging this Youtube video is highly recommended
- For people interested in modern real time rendering pipelines, take a look at this GTA V - Graphics Study
Literature and Resources on X3D/VRMLSince X3D/VRML is no longer the platform for the practical exercises in this course, I have demoted the links to X3D/VRML to this place.
- Tools that can "play" X3D- / VRML97 files:
- Introduction to VRML / X3D:
- A "Cheat Sheet" for VRML.
- SIGGRAPH 2008 Class Notes: Don't be a WIMP; These course notes not only discuss some post-WIMP interaction techniques, but also explain how to use some of the more advanced features of InstantReality, such as different stereo rendering modi, special interaction devices, clustering, scripting, animations, etc. (Source)
- The Annotated VRML97 Reference Manual (Source)
- The VRML Primer and Tutorial; those chapters that deal with HTML are not relevant for this course. In addition, the chapter on tools is outdated; but otherwise it is still a veritable introduction for VRML novices. (Source)
- The most important documents on the X3D standard:
- Examples for X3D / VRML:
- A large collection of materials,
both in VRML, as well as in XML encoding.
And here it is again as a ZIP archive.
- Links on socalled authoring tools; for VRML/X3D; if you want, you are welcome to use them. But if you have a decent programming editor (preferrably in ASCII), then you are probably more efficient with your editor.
- A handy little tool for
Calculating VRML Viewpoints.
Takes as input viewpoint, look-at, and up vector, outputs rotation as axis + angle to achieve the rotation from world coordinate frame into (viewpoint, look-at, up). From there it is trivial to compute the quaternion.
Last modified: Wed Sep 21 11:40:55 CEST 2016