Ah, there are so many cool things happening at the intersection of technology and healthcare at the moment! Previously on the blog, we’ve investigated medical applications for virtual reality, from cutting-edge treatments for chronic pain to its use in experiential medicine. But the applications don’t end there – VR can also be used as a method of rehabilitation for those with fibromyalgia and other conditions. But wait, there’s more! Not only does VR have a variety of benefits for all sorts of patients, but VR is an excellent training tool for medical professionals as well. As you can tell, we’ve long established how useful virtual reality is in healthcare – but what about augmented reality?
Augmented reality isn’t quite mainstream beyond TikTok and SnapChat filters (at least not yet). When you think of AR, you might think of a game like Pokémon Go (which may have caused an uptick in diligent Pokémon trainers running around your local park). Or, you may think of Houzz, a popular home design mobile app with over a quarter million ratings on the App Store. Neither of these apps is remotely healthcare-related (with the exception of stocking up on potions to heal your Pokémon after grueling battles, of course), so you may be surprised to learn that AR is providing medical professionals with groundbreaking new ways to do their jobs, and we’re here to fill you in on all of the details!
X-ray vision, time travel, and other AR superpowers
via Giphy
Even though it isn’t commonplace to see AR used in doctor’s offices right now, Louis Rosenberg, PhD., believes it will be sooner rather than later. In an article for VentureBeat, Rosenberg describes his experience while attending the 2022 Digital Orthopaedics Conference San Francisco. As part of his work on a panel, he and his fellow panelists reviewed recent studies and research on AR applications in medicine. He was surprised by what he found, writing “I was deeply impressed by how far augmented reality (AR) has progressed over the last eighteen months for use in medicine. So much so, that I don’t expect we’ll need to wait until 2037 for AR to have a major impact on the field. In fact, I predict that by the end of this decade augmented reality will become a common tool for surgeons, radiologists, and many other medical professionals. And by the early 2030s, many of us will go to the family doctor and be examined by a physician wearing AR glasses.” Why does he believe this? Well, he writes that AR could give doctors superpowers.
While AR may not give doctors x-ray vision akin to Superman, it could allow them to take MRI and CT scans of patients and then view the sites of interest or injury as a 3D model. Since human bodies are, of course, 3D, while scans are rendered on flat screens, this would allow doctors a more accurate, detailed image to work with while also being less invasive to patients. In the near future, AR could also allow doctors another superpower – vision into the past. Past scans or 3D images rendered in AR could be compared to or superimposed over a patient’s present body, allowing professionals a more intuitive and efficient examination than glancing back and forth between 2D images and a person.
Beyond scans and examinations, AR can help surgeons during procedures in a variety of ways. For instance, AR headsets can overlay important information and signals in a doctor’s environment. Rosenberg provides an example of how this might work: “…surgeons performing a delicate procedure will be provided with navigational cues projected on the patient in real-time, showing the exact location where interventions must be performed with precision. The objective is to increase accuracy, reduce mental effort, and speed up the procedure.” To see what some of these superpowers look like in development and in action, check out the video below, featuring professionals at the Cleveland Clinic talking about the past, present, and future of the Microsoft Hololens (note that this video is from 2018, so at the time of writing this blog, we already have four more years of progress beyond what is shown here!).
If at this point you’re wondering why Rosenberg, a CEO of an AI company, has a vested interest in AR, the answer is simple: he helped develop the first AR system in the early 90s with the Air Force Research Laboratory (AFRL).
via IEEE Spectrum: “Louis Rosenberg tests Virtual Fixtures, the first interactive augmented-reality system that he developed at Wright-Patterson Air Force Base, in 1992”
Rosenberg writes that today, he is impressed by the progress of the technology: “…when testing that first AR system with human subjects in 1992, we required users to move metal pegs between holes spaced two feet apart in order to quantify if virtual overlays could enhance manual performance. Now, thirty years later a team at Johns Hopkins, Thomas Jefferson University Hospital, and Washington University, performed delicate spinal surgery on 28 patients using AR to assist in the placement of metal screws with precision under 2-mm.” If you’re curious about the details of that particular surgery, you can learn all about it in the video below!
Overall, we’re excited that Rosenberg, a seasoned AR expert, is predicting AR to have a larger role in healthcare in the coming years. From easy 3D modeling to providing a window into the past, augmented reality could serve as a major asset to both doctors and their patients. Even though researchers are still unlocking the full potential of AR in healthcare, it is evident from the video above that AR is already helping professionals provide much-needed care to those who need it.
AR for the heart – and eyes, and beyond
via CUNY Academic Commons
If you’ve read this far, you know that the medical applications for augmented reality are brimming with potential for all sorts of procedures. The Microsoft Hololens and Hololens 2 aren’t the only headsets providing doctors with an AR boost. For example, cardiothoracic surgeon Mark Katz has partnered with an AR platform dedicated solely to surgical imaging called ClearView Surgical. Like the “x-ray vision” we mentioned above, Dr. Katz and the ClearView Surgical team are working on developing the tool for complex surgeries by building AR 3D models of patients’ hearts.
According to Bryce Donovan writing for the Medical University of South Carolina, “…the engineers at ClearView Surgical build exact 3D models of patients’ hearts (though it could be any organ; Katz said the heart is one of the more complicated ones because it is always moving), using the information from CT scans. In turn, these models then allow the surgeon to manipulate the heart in any direction. Need to remove a valve? No problem. What’s behind that ventricle? Let’s take a look. By marrying technology with medicine, surgeons like Katz will one day be able to look at – and touch – a patient’s heart without actually opening them up. The result: zero risk, tons of reward.” Much like medical training through VR, this use of AR could significantly lower the chances of error by increasing a doctor’s preparedness and understanding of each patient’s unique heart, long before surgery has even begun.
AR technology is also being developed and tested for use during complex retina surgeries that use a combination of 3D digital microscopes and AR headsets, called the Beyeonics One system.
From commonplace to complex surgeries, AR has medical applications that could positively impact the future of the operating room. It’s an exciting time to keep up with augmented reality in healthcare. Our future likely holds more precise and quick procedures, more efficient ways to monitor injuries and other conditions – and who knows where else medical AR innovation will lead?
If you’d like to explore the possibilities of VR or AR for learning, let us know! As an educational game developer with over 17 years of experience, we’re here to help you create playful experiences that improve people’s lives.
More on XR and medicine: