There's a phrase I often hear: "VR, XR, it's all the same thing, right?" That's a bit like saying "cheese and cottage cheese are the same." They sound similar, but they're not.
VR is only one part of a much broader umbrella called XR, Extended Reality. XR includes all the "realities" we know: VR, AR, and MR. Behind those letters lies a story about the evolution of reality itself, about how we experience, learn, and shape the world around us.
One reality, several stages
Over the past few decades we've gone through technological revolutions that didn't just change our tools, they reshaped our understanding of reality:
- Classic Reality (CR), our physical, screen-free world.
- Portable Reality (PR), the mobile era, when the phone became a window to another world.
- Virtual Reality (VR), a fully computer-generated environment.
- Augmented Reality (AR), a digital layer projected onto the real world.
- Mixed Reality (MR), an interactive blend, where virtual objects respond to real-world surfaces and lighting.
- Extended Reality (XR), the umbrella that connects all these worlds together.
VR: when we step into reality
Virtual Reality lets us experience an environment that exists entirely in the computer, whether real or imagined. Through vision, motion, and even touch (via controllers, gloves, or haptic devices), we become part of a virtual world.
In medicine, VR is far from a gimmick. It enables interactive 3D learning of anatomy, trains surgeons and prepares patients for procedures, helps in trauma recovery and rehabilitation, and transforms dry CT scans into living, explorable models.
In VR, we don't just see the world, we exist inside it.
Yet with all its power, VR has limits: bandwidth constraints, motion sickness, heavy headsets, and sometimes social isolation. In a world already filled with screens, even virtual reality needs boundaries.
AR & MR: digital layers over the real world
Augmented Reality isn't new, it's simply evolved. Think of the weather graphics on TV, or the yellow offside line in a soccer match: that's already AR. Today it's everywhere, in Google Translate's live sign translation, in navigation apps, in jewelry fitting tools, or IKEA's furniture-placement app.
In medicine, AR and MR let us project anatomy directly onto the patient's body, navigate through organs in real time, receive remote expert guidance from anywhere, and perform precise surgical simulations using systems like Surgical Theater, Medivis, or Novarad VisAR.
The thin line between AR and MR
The difference is subtle but meaningful. In AR, the digital layer "floats" above the real world, projected onto reality but not interacting with it. In MR, the digital layer responds to the physical environment, recognizing surfaces, reflecting light, and casting shadows.
In AR, a digital ball passes through a table. In MR, it stops on top of it.
Registration: aligning the virtual and the real
Here comes the crucial concept of registration: the precise alignment between the digital model and the physical body. Common methods include:
- Markers, at least three uniquely identifiable codes detected by the system.
- Landmarks, at least three real-world points recognized in both digital and physical space.
- Contour recognition (2D/3D), matching camera-captured outlines to the 3D model.
- Automatic scanning, surface mapping that aligns the digital model with the real object.
Without accurate registration, even the most advanced technology won't be precise enough for medical use. It's not plug & play, every millimeter of deviation in the virtual world can become critical in the real one.
XR: the grand synthesis
For me, XR isn't just technology, it's a new way to see the human body. It lets me move between worlds seamlessly: plan a surgery in VR, analyze data through AR, interact with instruments in MR, and then evaluate outcomes in the physical world. With a single click I can move from a 3D screen to standing beside the operating table, where the data overlays reality itself.
What's next?
Here come the Metaverse and the Internet of Things (IoT). When every device, from printers to ventilators, is connected, and live data updates in real time within virtual environments, we approach a world where every layer of reality communicates with the others. When the ventilator, catheter, and AR display all "speak the same language," it's no longer science fiction, it's a smart, real-time ecosystem that enables faster, safer, and more human decisions.
But let's remember: technology is incredible, yet it's only a tool. To use it wisely, we must understand both the human body and its limits.
In summary
XR isn't the future, it's the evolving present. It's already transforming how we learn, train, operate, and design the future of healthcare. But more than anything, it's changing how we understand ourselves.
This article was inspired by a lecture I delivered at HIT, the Holon Institute of Technology, and by collaboration with experts at the Levin Center for Surgical Innovation at Ichilov Medical Center, Tel Aviv.
