3D Virtual Navigation with 'In the Air' Real Control of Surgeon's Hands

Short summary

In the collaboration of 1RIT Croatia and 2Bitmedix, we have developed a 3D hardware sensor device controller that supports hand/finger motions as input, with no hand contact – an approach that would enable surgeons to get complete and aware orientation in the operative field, where ‘overlapping’ of the real and virtual anatomic models is inevitable. Our solution for 3D virtual navigation is offering an alternative to closed software systems for visual tracking, where we developed the software framework that could interface with high resolution stereo depth cameras and provide a set of standardized methods for medical applications, such as hand gestures. Motion tracking enables more precise virtual movement, rotation, cutting, spatial locking and measuring as well as slicing through datasets. In order to provide the most immersive experience, we used a camera for depth and motion tracking that has active stereo depth resolution with precise shutter sensors for depth streaming with a range up to 2-3 meters which gives a sense of freedom to the surgeon during the surgery.