While the three technologies are often mixed up, there are significant differences between VR, AR and Mixed Reality (MR). AR lets users see the real world and projects digital information onto the existing environment. VR shuts out everything else completely and provides an immersive simulation. Mixed reality is closer to AR in the sense that it also projects synthetic content on the environment that is anchored in reality. However, unlike AR, MR technology interacts with the world. It means that while AR will soon be able to project the clinical data of a patient in front of me, MR first senses what is around and projects the requested data adjusted to the given environment.

Mixed reality is provided through Microsoft’s newly released low cost Mixed Reality Headset as well as the advanced HoloLens. The latter is the first self-contained, holographic computer which enables the user to engage with digital content and interact with holograms around.  Microsoft HoloLens scans the environment first and one can actually place the appearing window on the objects that it sees.

Interestingly, the HoloLens uses voice commands to search the web or start certain apps. The gadget uses speakers which do not obstruct external sounds to allow you to hear both virtual and natural sounds. The user is also able to command the entire system through different hand gestures or through a clicker. Interacting with the apps is quite straight-forward once you get used to the clicking hand gesture – it is just like using Microsoft Windows without a mouse or a keyboard.

Applications of Mixed Reality:

  • Medical & Nursing Education & Training: HoloLens has opened up radically new ways for medical education as, unlike through Smartphone based AR, it is able to project a full sized human body in front of medical students. Thus, the organs, veins or bones will be visible accurately in 3D, and future medical professionals will be able to analyze organ systems and remember much better than it is possible while studying from a book. There are already universities that plan to introduce this new technology: Case Western opens its new health education campus in collaboration with the Cleveland Clinic in 2019, where students won’t learn anatomy from cadavers – they’ll learn it from virtual reality.
  • HoloLens in Diagnostic Imaging: Visualizing radiological data such as DICOM data from MRI/CT scans reconstructed into virtual 3D Holograms makes a powerful diagnostic tool, which aids physicians and surgeons to make an accurate diagnosis and plan treatment. A doctor can a HoloLens and actually “see” the pathology superimposed inside the patient from all sides.
  • Surgical Planning and Navigation: HoloLens is being applied by hospitals in doing pre-operative planning of operations. By projecting actual reconstructed scans as 3D holograms over patients, surgeons could plan an entire intervention using such holograms and they could accurately see the places for making incisions and also clearly envisage the consequences of their moves.

Immertive MR

Developing HoloLens and Mixed Reality applications requires a blend of different talents – medical domain, visual designers and design thinkers, 3D graphic artists and software engineers and programmers who have worked on Unity and HoloLens. The team at Immertive brings together:

  • Healthcare Domain Knowledge: Our in-house team of medical & medical education experts collaborating with our panel of external specialty-wise subject matter experts help in creating Holograms that can be superimposed within your own environment.
  • 3D Art: Immertive’s expert 3D graphics team creates compelling 3D graphics and can even scan objects and make them into 3D Holograms with high fidelity.
  • Design Thinking Approach: Our design team covers all aspects of Visual Design including Information design, Interactivity design, Navigation design, Interface design, Identity design and Graphic design. The UX for HoloLens is unique in that the user can interact with 3D Holograms being seen through the device by using voice commands, gestures and a clicker. Our design experts create unique interfaces that make the HoloLens experience to be something that’s long lasting in impact and outcome.
  • Software Development: Our software engineers and programmers are adept at using Unity and HoloLens SDKs to build 3D immersive interactive HoloLens experiences. This allows scanning and mapping the environment and placing Holograms within the scanned spaces and interactivity through voice and gesture commands.
  • Technology: We build applications for Microsoft’s HoloLens as well as for the lower costing Mixed Reality Headset.