HIPS - Computergraphics and Virtual Reality

Hip-Replacement Implantation Preparation Simulator

HIPS is a joint project of University of Bremen (Computer Graphics and Virtual Reality Research Lab (CGVR)), Chemnitz University of Technology (department of mechanical engineering), CAT PRODUCTION GmbH and FAKT Software GmbH that is funded by the VDI/VDE within the Central Innovation Programme for SMEs (ZIM) by German Federal Ministry for Economic Affairs and Energy (BMWi) under the grant KF2039721AK2, and additionally by the German Federal Ministry of Education and Research (BMBF) under the grant 16SV8356. This project is partially supported by the Bremen Spatial Cognition Center (BSCC).

Our vision is to enable students of surgery to learn the neccessary skills to perform a hip replacement surgery, called arthroplasty, in a fun, safe and effective virtual reality simulation using haptic feedback or force feedback. Actually, virtual reality is one way to lower training costs, as students don't need to practice on donor hips, or at least not exclusively. The first iteration (completed in 2020) focuses on a single however very difficult step in the whole procedure, the so called acetabular reaming, where the hip socket is enlarged and cleared of old joint tissue and bone in order to facilitate enough room to fit a prosthetic hip socket. In the second iteration (completed in 2023), we have implemented all five steps of the procedure:

Cutting off the head of the femur bone
Reaming of the acetabulum of the pelvic bone to create a round cavity of desired dimension
Implantation of the artificial hip socket into the acetabulum by hammer impact
Rasping a cavity into the femur for the implant (also controlled with a hammer)
Implantation of the artificial joint into the femur by hammer impact

The difficulty when reaming the hip socket is that the amount of force and the duration with which to ream can only be felt. If the surgeon were to ream too far, he could penetrate and damage the bone, or even worse damage nearby major arteries. Surgeons rate this step in the procedure as the most difficult and studies showed that the successrate is directly correlated to the surgeons experience. However, how do you teach a skill that can not be fully expressed by words or images? The key component in learning this skill is haptics or force feedback.

Consequently, we developed a novel collision detection and force feedback method using inner sphere trees that enables haptic force rendering of high forces with stable continuous behaviour. We use a KUKA LBR iiwa robot to render extremely hard contacts and really high forces (up to 137 N).

For further informations about our new geometric data structure that is based on sphere packings, the "Inner Sphere Trees", please visit our corresponding project page.

We successfully demonstrated a working prototype of an intermediate version of our simulator at the European Robotics Forum 2018 and at IEEE VR 2018.
As part of the Deutschen Institut für Virtual Reality (DIVR) competition of 2019, we were awarded the 'best tech' DIVR-Award among about 50 participants for our VR demo presentation.
The project was also presented in July 2020 on MDR Aktuell in the report "OP-Training dank VR: Chemnitzer Forscher entwickeln virtuelle Hüftoperationen".
The "Marburger Bund Zeitung" published in June 2020 in its column "Wissenschaft & Technik" the article "Für ein realistisches Gefühl / Virtuelle Realität mit einem Computer" about our project.
We won the DIVR-Award 2023 in best tech among over 100 participants from all over Europe. Here, we presented our extended version of the simulator, that haptically simulates 5 individual steps in the hip surgery procedures (femur cutting, hip acetabular reaming, inserting hip implant, femur rasping, inserting femur implant).

Our patented hammering force feedback device in action at DIVR 2023.

Our patented hammering force feedback device in action at DIVR-Award 2023. (Bildquelle: divr.de)

We received the DIVR-Award 2023 for best tech among >100 participants. (Bildquelle: divr.de)

Visualization of the found density distribution in the acetabulum. Red parts are hard, green parts are soft, white have not been analyzed.

Demo presentation within the DIVR-Award assessment.

Demo presentation within the DIVR-Award 2019 assessment. (Foto: Mediennetzwerk.NRW / Juliane Herrmann)

We received the DIVR-Award 2019 for best tech!

Us receiving the DIVR-Award 2019 for best tech among 50 participants. (Foto: Mediennetzwerk.NRW / Juliane Herrmann)

Visualization of a poorly reamed hip socket. Colored parts are errors.

Blue shaded parts visualize under-reamed material, such as unhealthy bone or cartilage.

Red translucent parts show material that was over-reamed, these weaken the bone structure.

Our demo setup while meeting with experienced surgeons to evaluate our simulator in May 2018. We mounted the real surgeon tool to the end-effector of the KUKA iiwa.

An actual acetabular reamer mounted to the robot.

Surgeon testing our simulator in Chemnitz, May 2018.

More people trying the simulator in Chemnitz.

Demonstration at European Robotics Forum 2018

Demonstration of our simulator at the European Robotics Forum 2018 in Tampere, Finnland. The demo was sponsored by KUKA.

The setup for our demo, which we also presented at the IEEE VR 2018 in Reutlingen, Germany.

Awards

DIVR-Award 2023 for best tech
Best Application Paper Award at EuroVR 2020
DIVR-Award 2019 for best tech [Pressemitteilung der TUC], [Medien Netzwerk NRW]

Publications

Perceived Realism of Haptic Rendering Methods for Bimanual High Force Tasks: Original and Replication Study, Nature Scientific Reports 13, 2023.
The Impact of 3D Stereopsis and Hand-Tool Alignment on Effectiveness of a VR-based Simulator for Dental Training, IEEE ICHI, 2022.
Immersive Anatomy Atlas: Learning Factual Medical Knowledge in a Virtual Reality Environment, American Association for Anatomy, 2021.
A Shared Virtual Environment for Dental Surgical Skill Training, IEEE VR, 2021.
Review of Haptic Rendering Techniques for Hip Surgery Training Best Application Paper Award, EuroVR, 2020.
A cadaver-based biomechanical model of acetabulum reaming for surgical virtual reality training simulators, Nature Scientific Reports 10, 2020.
Realistic Haptic Feedback for Material Removal in Medical Simulations, Haptics Symposium, 2020.
A Continuous Material Cutting Model with Haptic Feedback for Medical Simulations, IEEE VR, 2019.
HIPS - A Virtual Reality Hip Prosthesis Implantation Simulator, IEEE VR, 2018.
A Virtual Hip Replacement Surgery Simulator with Realistic Haptic Feedback, IEEE VR, 2018.
A Volumetric Penetration Measure for 6-DOF Haptic Rendering of Streaming Point Clouds, IEEE World Haptics, 2017.

Reports

Interview "Hüftendoprothesen: OP virtuell und mit Gefühl trainieren", Online-Magazin MEDICA, 1st of April, 2021
OP-Training dank VR: Chemnitzer Forscher entwickeln virtuelle Hüftoperationen, MDR, 16th of July, 2020

Videos

DYNAMIC HIPS Project

Presenting the research project DYNAMIC HIPS (German version).

Dental Simulator Demonstration with Eye tracking

Layman demonstrating VR simulator features during root-canal access opening on tooth #26 on upper jaw (with eye tracking).

Rigid-Body-based Hip Reaming

Rigid-Body-based simulation of hip reamer and pelvic bone with material removal, visualized in Unreal Engine.

Hip Reaming

Performing acetabular reaming to prepare for prosthetic fitting, visualized in Unreal Engine.

Dental Milling

Performing dental caries removal, visualized in Unreal Engine.

Short movie

Short movie demonstrating our project. (German version)

Orthopedic surgeon trying out the simulator

Orthopedic surgeon testing our application.

Recording of the first working version

Work-in-progress video of our application (with english subtitles).

Ackonwledgements

This work is supported by the Central Innovation Programme for SMEs (ZIM) by German Federal Ministry for Economic Affairs and Energy (BMWi) under the grant KF2039721AK2, and additionally by the German Federal Ministry of Education and Research (BMBF) under the grant 16SV8356.