“You quickly forget about everything around you and become completely immersed into the virtual world” – When we hear about Virtual Reality, we often think about hardcore gamers with their headsets on, immersed in a fantasy world. But the different kinds of realities have the potential to revolutionise healthcare: they could be used in trainings, for remote assistance or even during surgery – explained at the latest episode of Rise to the Challenge! Leon Rizner, scientist at Novartis (Pharmaceutical Development) in Slovenia.

Virtual Reality, Mixed Reality, Augmented Reality… let’s be honest, it is quite easy to get confused about the characteristics of the different realities! That is why Leon Rizner started the talk with setting things straight. As the scientist explained, the best way to understand ‘realities’ is to imagine them on a spectrum, where on the far-left side we find our “boring” reality.

“We know how to describe it with physical laws… We know we can’t see through the wall… We know how to interact with objects. We know how to hold them, what happens if we let them go… we experience this reality with all our five senses” – he said in the interview which is available on EIT Health’s Facebook page. If we move to the far-right side of the spectrum we find Virtual Reality. It affects two of our main senses: we see and hear a virtual world.

“Even though only two senses are moved from our reality to this virtual one, you quickly forget about everything around you and become completely immersed into the virtual world” – he explained to his talking partner, Tamás Békási, RIS Business Creation Manager at EIT Health InnoStars, providing a little background about VR: although many people think that it is a recent technology, the truth is that the NASA already used it in the 90s to train astronauts. It is true that it skyrocketed in the last few years, and according to the Slovenian scientist, it could be useful in healthcare as well.

Tools and equipment in health care are getting more and more advanced. The training of people who operate such complex tools safely and efficiently can be costly and complicated. “With VR you can put trainees in front of such equipment in a completely virtual world and give them opportunity to get familiar with the equipment” – gave Rizner a practical example. According to the scientist, the real training could become more efficient and could lead to less errors if it was preceded by a virtual training. He admitted, however, that such completely virtual trainings can be expensive. “You need to bring this hole device or machine that you want to train people on in the 3D world, so this is still a little bit unavailable.”

If we take another look at the ‘spectrum of realities’, somewhere in the middle we find augmented reality, which can bring virtual elements to our reality. However, these elements do not interact, or they don't understand our reality. Rizner, who described himself as a “bit of data nerd”, gave an everyday example of how augmented reality works – through smart watches. “If you want to see the data that is gathered, you have to either look at your watch or you have to take out your phone and synchronise it with the watch and check the data on the phone. But let's imagine that we have this augmented possibility, and we have this data always available, right in front of us. On the left side there could be the heartbeat, and on the right, you could see how many steps you took… But this technology does not know that you are about to step on the road, that this is dangerous. So, this technology just augments or enriches our own reality, but it does not understand the world around us” – explained the scientist, adding that AR is already in use in healthcare as well. A Chicago-based company, Augmedics, for example, uses an Augmented Reality navigation technology during surgeries which “allows surgeons to see the patient’s anatomy through skin and tissue as if they have ‘x-ray vision’, allowing them to more accurately navigate instruments and implants during procedures”.

After describing these technologies, Mixed Reality is easy to understand – Rizner pointed out. “It just combines all of these (realities) into one approach... We see the world around us like it is, but we can bring virtual elements, objects or information and it understands its surroundings. So, it's more advanced than augmented reality.”

He thinks that MR could be used in healthcare trainings as well, and it would be less expensive than in the case of the before mentioned VR. “You can put trainees in front of a machine and give them step-by-step instructions. You can call the expert, who is working from home or the other side of the world, and they can help them and guide them.” Visualization could also be a potential field where MR could help. “Imagine, that in the hospital the doctor needs to explain something to the patient who broke his arm. It is very difficult to explain what is going to be done through 2D scans. But we could use this technology to bring 3D models in… it would be much easier to explain a patient how a surgery will be done… with such 3D models it could be much easier” – Rizner said mentioning the case of medicine students who could learn anatomy much easier this way.

The scientist is very hopeful regarding the future of VR, AR, and MR, comparing their evolution to the evolution of mobile phones: with each generation and iteration, they will become better, easier to use, and the industry will also become more aware of what users need.

With some colleagues, Rizner, a pharmacist by profession who is a self-described geek, raised the issue of the need of Mixed Reality in pharmaceutical development to the management at Novartis in Slovenia. “Just in two years we managed to bring this technology from not having it at all to being able to order it as simple as ordering a laptop on every location in Novartis” – he said. As he promised, he will talk more about the possible uses of VR, AR and MR in healthcare during the ‘Augmented / Mixed Reality in Healthcare Hackathon’, on November 12-14. At the event Novartis and EIT Health “will be looking for fresh ideas and pioneers to create new augmented / mixed reality solutions for the healthcare industry”. Participants will not only receive practical trainings on idea incubation and design thinking but will demonstrate their idea in front of senior experts and executives of Novartis and EIT Health.

As Rizner explained, participants will be mentored by early adaptors of the technology at Novartis who have tons of experience. They know what works and can guide the ideas into the right direction. The winners will receive financial support – 1st: 5000€, 2nd: 3000€, 3rd: 2000€ – and a chance for co-operation with Novartis after the hackathon.

“I’m super excited about the Hackathon, about its pure scale and what it can bring to Novartis. We already use Mixed Reality glasses in Novartis in our work but as typical eager scientists, we are always looking for exciting new ideas. This is what we think you can bring to us!”

You can watch the previous episode of ‘Rise to the Challenge!’ about healthcare investors and fundraising here.

’Rise to the Challenge!’ can be followed on the EIT Health InnoStars Facebook account and on LinkedIn.