Virtual Reality and Physically-Based Simulation WS 15/16


Virtual Reality (VR) is a research area at the intersection of computer graphics, physically-based simulation, and 3D human-computer interaction (HCI). VR is concerned with novel interaction technologies and interaction metaphors in virtual environments (in particular intuitive and direct metaphors), immersion and presence, and real-time rendering. Another important topic is physically-based simulation in real-time, which tries to simulate real-world phenomena such as fire, cloth, the behavior of rigid objects when colliding with each other, fluids, or objects made of deformable material.

Over the past one or two decades, VR has established itself as an important tool in several industries, such as manufacturing (e.g., automotive, airspace, ship building), architecture, and pharmaceutical industries. In addition, a lot of the techniques and solutions developed in the area of VR can be applied directly to computer games.

In this course, we will first look at the fundamental methods, and then go on to more advanced algorithms that are needed to build complex and full-fledged VR systems or real-time computer games. Example topics are object behavior, acoustic rendering, haptics, and collision detection.

The assignments will be mostly practical ones, based on the cross-platform VR system InstantReality. Participants can choose the programming language from Java, Javascript, and C++.
You are encouraged to work on assignments in small teams.

Some of the topics to be covered (tentatively):

  1. Introduction, basic notions of VR, a few applications
  2. VR technologies: displays, tracking, input devices, software design
  3. Stereo rendering
  4. Error correction: tracking correction, filtering
  5. Techniques for real-time rendering
  6. Fundamental immersive interaction techniques: gesture recognition, navigation, selection, grasping, menues in 3D
  7. Complex immersive interaction techniques: world-in-miniature, action-at-a-distance, magic lens, etc.
  8. Particle systems
  9. Spring-mass systems
  10. Force feedback: rendering forces
  11. Collision detection
  12. Acoustic rendering

Note: this list is just tentative and subject to change during the semester.

News


Slides

The following table will, eventually, contain all the topics that were covered in this class, the accompanying slides, exercise sheets, and frameworks for solving the programming exeercises. (This table will be filled week by week.)

-->
Week Topics Slides Exercise
sheets 		Frameworks
1. Orga stuff
Introduction (definitions, immersion, fidelity, presence, history, Milgram's continuum),
PDF0 PDF1
2. Scenegraphs: immediate / retained mode, semantics of nodes and edges, special issues with light sources, shared geometry (instancing), thread-safe scenegraphs, distributed rendering, fields & routes concept,types of nodes, specification of the material using the Phong model, indexed face set, PDF Assignment 1
3. Stereo rendering 1: depth cues, human visual system and stereopsis, horopter, stereo projection, multiplexing techniques for stereo images, immersive projection technologies PDF
4. Stereo rendering 2: issues with stereo rendering (depth aliasing, convergence-focus incongruity, in-correct viewpoint, stereo violation), hypo- and hyper-stereo, automultiscopic displays, pre-distortion for HMDs, low and full persistence, PDF Assignment2
5. Real-time rendering 1: simulator sickness, latency and its sources, view-independent rendering, prioritized rendering, level-of-detail techniques, static/dynamic/psychophysiological selection, predictive LOD selection, progressive meshes, portal culling, PDF
6. Real-time rendering 2: state sorting using online sorting buffer algorithms, stereoscopic image warping, latencies with conventional rendering, reducing latency using time warping
PDF Assignment3 Jump and lag
7. Dies Academicus
8. Input Devices: degrees of freedom, multimodality, virtual trackball, directness continuum, isotonic vs isometric, tracking, data gloves, locomotion devices
 PDF Assignment4 Assignment4-hints
9. Interaction metaphors 1: Universal Interaction Tasks, design of user interfaces, gesture recognition, the taxonomy, abstract representation of the user, navigation metaphors (point-and-fly, scene-in-hand, two-handed, hands-free, et al.), user models (power law of practice, Hick's law, Fitts's law), task decomposition, control-display ratio, selection techniques (go-go, ray-based techniques, flexible pointer), bent ray, cone-based technique with ranking, balloon selection, PDF
10. Interaction metaphors 2: object manipulation (grasping and moving), Natural User Interaction, PRISM metaphor, principles for interaction design: action-at-a-distance, image plane interaction, proprioceptive interaction, world-in-miniature, magic lenses, redirected walking, system control, menus, tangible user interfaces.
PDF Assignment5 Stats:Jump and lag
11. Particle systems: dynamics/kinematics, Euler integration, phase space, definition of particle systems, physical effects, non-physical effects, collision handling, rendering, rendering transparent objects, flames & fire, procedural modeling of plants, the concept of streaming architectures PDF
12. Mass-spring systems: Newton's Laws, single spring-damper, explicit Euler integration, instability and error accumulation with explicit Euler integration, Runge-Kutta, Verlet integration, constraints, PDF Assignment4 Mass-spring framework
13. Mass-spring systems 2: implicit integration, tangent stiffness matrix, comparison to explicit integration, mesh creation for volumetric objects, consistent collision response for volumetric mass-spring systems.
Haptics: applications, devices, the haptic loop, human factors, haptic textures, buzzing, intermediate representations, impendance/admittance approaches, 		PDF1 PDF2
14. Collision detection: motivation, definitions, collision detection pipeline, broad/narrow phase, separating planes algorithm (for convex objects), Mikowski sums, intersection test based on Minkowski sums, hierarchical coll.det., bounding volume hierarchies, types of BV's, overlap test for k-DOP's, construction of BVH's, inner sphere trees, sphere packings, proximity computation using ISTs.
 PDF

Literature

Warning: these text books can only give you a general introduction to the field of VR! Most of the topics taught in class will not be covered by any of these text books directly -- in fact, AFAIK there are no text books that cover these topics. Therefore, I recommend to attend class.

If you are thinking of buying some of these books, then I suggest to consider buying a used copy of them -- very often, you can find them at a fraction of the price of a new copy. The following are two very good internet sites for finding inexpensive used copies of books: Abebooks and BookButler.

Assignments

In order to acquire the "Schein", you will have to do a number of assignments. These will be mostly small practical programming assignments (based on a freely available VR system).

Online Literature and Resources on the Internet

Literature and Resources on X3D/VRML
Since X3D/VRML is no longer the platform for the practical exercises in this course, I have demoted the links to X3D/VRML to this place.
Gabriel Zachmann
Last modified: Wed Sep 21 11:40:55 CEST 2016