Design and development of hardware and software for wireless Head-Mounted Displays

With the introduction of a new head-mounted displays models on the consumer market, the interest to those devices raised dramatically in the recent past. Although only a few wireless head-mounted display models are available today, there is a certain range of Vitrual- and Augmented Reality applications where those are needed. Existing solutions that allow using conventional HMDs for such applications wirelessly are either very expensive or too cumbersome.

This thesis is aimed to overcome those drawbacks with a new solution of a distributed design. Wired HMD is being connected to a compact, low-cost, portable computer that displays a real-time compressed video stream, transferred from a computer performing the actual rendering of an HMD-enabled application.

On the Figure a prototype is depicted: 'A' - USB 802.11n 5GHz WiFi adapter; 'B' - Single-board ARMv7 computer (Banana Pi); 'C' - Battery Pack (5V, 37Wh); 'D' - HMD input connectors; 'E' - Oculus Rift Dev. Kit 1 HMD running demo application.

Description

The system concept can be described by the following points:

1. There is no modification of the head-mounted display hardware, the HMD remains wired, but being connected instead to a secondary, small, portable computer.

2. The frame buffer of the main computer that performs the rendering of the HMD-enabled application is captured with a frame rate that is equal or higher, than the HMD screen refresh rate is (60Hz for Oculus Rift DK1).

3. The captured frame buffer information is being encoded according to H.264 AVC video coding standard.

4. A video stream of encoded video is created and sent to the secondary computer over the 5GHz IEEE 802.11n Wi-Fi network where it is decoded and played on the attached head-mounted display.

5. The USB communication with HMD is encapsulated into TCP payload and transferred between main and portable computers in real-time.

Software used on portable (client) computer: Debian "Wheezy" GNU/Linux OS, Mplayer2 (VDPAU accelerated H.264 decoding), USB over IP.

Software used on main (server) computer: Windows family (7+) OS, Lib x264, FFmpeg, USB over IP.

Results

On this initial stage the added "streaming" latency of the developed prototype was 94 ms (avg.). This is high for typical VR applications, but might be suitable for a certain range of AR applications. With further hardware and software optimizations it is expected to reach the level of 30-40 ms.

The design of the system allows use of various HMDs, supporting different video modes, refresh rates and operating over an unlicensed WLAN network. All software utilized is free and mostly open source. The main (server) computer part can be easily ported to Linux and MacOS.

Developed prototype can be easily worn due to compact size and low weight (361 gr). System start is fully automated and takes about 20 seconds. Battery operation time is approx. 4 hours.

Files

Full text of the Master's thesis

A running wireless HMD prototype:

License

This original work is copyright by University of Bremen.
Any software of this work is covered by the European Union Public Licence v1.2. To view a copy of this license, visit eur-lex.europa.eu.
The Thesis provided above (as PDF file) is licensed under Attribution-NonCommercial-NoDerivatives 4.0 International.
Any other assets (3D models, movies, documents, etc.) are covered by the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit creativecommons.org.
If you use any of the assets or software to produce a publication, then you must give credit and put a reference in your publication.
If you would like to use our software in proprietary software, you can obtain an exception from the above license (aka. dual licensing). Please contact zach at cs.uni-bremen dot de.