Augmented, virtual realities pose the next step in patient care
Click Here to Manage Email Alerts
Published literature has shown increased interest in computer-assisted orthopedic surgery among researchers, with the technology being used for surgical planning, simulation and navigation.
As the most advanced part of computer-assisted surgery, navigation and robotic technology have continued to evolve, leading augmented and virtual realities to become an interest among surgeons.
Source: Cory L. Calendine, MD
“Traditionally, surgeons would use their eyes and their intuition to help guide them as far as resecting bones and placing implants,” Matthew D. Saltzman, MD, orthopedic surgeon at Northwestern Medicine, told Healio | Orthopedics Today. “Overtime, things have evolved and there are robots, which are different; there is navigation, which has been around for a while. And augmented reality is a unique technology that allows you to have augmented computer graphics that overlay the patient’s anatomy via glasses. These graphics allow surgeons to have some additional guidance based on what they planned preoperatively while looking at the patient’s anatomy.”
Although the terms augmented reality and virtual reality are often used interchangeably, these are not the same applications, according to Joseph P. Iannotti, MD, chief research and academic officer at Cleveland Clinic Florida Market. He said augmented reality combines “a physical representation of the bone or the surgical site with something that is completely digital.”
“Where that is being used today is as a heads-up screen,” Iannotti said. “You may have a set of goggles on that, if you are looking through the bottom of the goggles, you are in the real world and if you gaze up, you are in the digital world.”
However, he said virtual reality is being applied “as an immersive learning environment that allows the learner to feel like they are in a virtual space that simulates the learning environment.”
“I learn more visually and, to some degree, maybe in a bit more of a ... gaming experience,” Iannotti said. “I think a lot of people learn better that way than reading it in a book or looking at 2D or a video.”
Resident training
According to Justin Barad, MD, pediatric orthopedic surgeon and co-founder and chief strategy officer at Osso VR, virtual reality has most frequently been used in resident education and for training on emerging medical technologies.
“Right now, the problem at its core from a training standpoint is that there is too much to learn. Accelerating science and innovation is expanding the library of procedures we are expected to know how to do on demand, and all the introduction of these new technologies and devices is becoming rather overwhelming, both in training and beyond,” Barad told Healio | Orthopedics Today.
Although augmented and virtual realities are still in stages of infancy, Cory L. Calendine, MD, of the Bone and Joint Institute of Tennessee, said the use of virtual reality has been pushed to the forefront by allowing residents to increase the number of surgical repetitions in training on an 80-hour work week and during the COVID-19 pandemic.
It also allows skilled orthopedic surgeons “to extend their reach as educators to other surgeons” and conveys the nuances of a surgery or of a particular procedure, implant or device to early and mid-career surgeons, according to Danny P. Goel, MD, MBA, MSc, FRCSC, orthopedic surgeon at the University of British Columbia and the CEO and founder of PrecisionOS.
“Importantly, [virtual reality allows residents and surgeons] to make mistakes in the system [and] avoid making those errors in real life,” Goel said.
Increased accuracy
Goel also said augmented reality provides insights to a surgeon’s limitations during surgery.
“This will not replace the surgeon but will significantly empower them during surgery to make better and more informed decisions,” he said.
If surgeons know their limitations prior to surgery, Saltzman said they can utilize augmented and virtual realities to understand a patient’s anatomy and preoperatively plan the surgery by using a CT scan to create a 3D model of the anatomy and use it to virtually implant a prosthesis.
“People have been doing this for the last 10-plus years where they can come up with a plan on the computer ahead of time, go to the operating room and try to execute that plan and try to put the implant in an optimal position,” Saltzman said. “But the way virtual reality is different, and the pro to it, is that it will help guide you while you are doing the surgery. It is not just planning it before, but it is the actual execution in the operating room.”
Use of augmented reality to more accurately perform orthopedic surgery may also lead to better range of motion and functional outcomes, according to Saltzman. And because it runs on software, he said updates can make the technology run more efficiently and make data points more accurate.
“The nice thing about this technology compared to a robot is it is not a capital piece of equipment that has been manufactured and that is what you get,” Saltzman said. “It is software. The software can also be updated if new technology and new versions of it are available, and you can still use the same glasses and the same computer programs to run the updated technology.”
Issues with headsets
These technologies also make a surgical plan easier to accurately carry out in terms of capital costs, resources, space needed for surgery and ability to perform surgery in an ASC vs. hospital setting, according to Jonathan C. Levy, MD, director of the Levy Shoulder to Hand Center at the Paley Orthopedic & Spine Institute and program director at the Paley Institute – Holy Cross Shoulder and Elbow Fellowship.
“In its ideal form, virtual or mixed reality will involve [a surgeon] putting on a light pair of glasses or a headset and having it integrate with a computer,” Levy, a Healio | Orthopedics Today Editorial Board Member, said. “[It would be] easy use, at a lower cost and, if it can be highly accurate, why not use it?”
However, Levy said the headsets worn for augmented reality can often be “burdensome and heavy.”
“It can make a surgeon uncomfortable if they are doing six cases in a day and they are wearing a heavy headset for the entire time,” he said. “There are some drawbacks currently to the size of the technology. Then, of course, we are not at the point where the technology has been proven to have clinical impacts in terms of better outcomes and precision accuracy.”
But as the technology continues to evolve, Barad said the headsets will continue to get smaller and lighter.
“Everybody wants a world where we are just wearing a pair of sunglasses and we are seeing this augmented reality view,” Barad said. “I think that will be great and I think it will come, but I do not think we need it.”
Layer of complexity
One of the limitations of virtual and augmented realities is some surgeons’ “belief that we do not need augmented or additional information to succeed as surgeons,” Goel said.
“We are also traditionalists and skeptical of certain technologies that we believe may not impact patient outcomes and, therefore, require evidence of its effectiveness,” he said.
The technology also adds a layer of complexity to a resident’s education, according to Saltzman.
“Traditionally, when you are learning orthopedic surgery, a lot of it is based on your mentor’s experience, intuition and tricks, and now you have added another layer of complexity – a computer screen, glasses or goggles that have the virtual images on there,” Saltzman said. “It adds another layer of complexity to the learning experience.”
But Saltzman said augmented and virtual realities have a short learning curve and “can quickly become second nature.”
“It is not something that is a long-term hinderance or a lot of added steps,” he said.
Even without a steep learning curve, Levy said one limitation of virtual reality is not being able to touch the patient or see “the push-pull of retractors.”
“[Virtual reality is] good at learning the steps of the procedure, knowing where to put things and knowing what order things should happen, what is the next step, etc., but it does not quite replace the true hands-on experience,” Levy told Healio | Orthopedics Today.
Patient education
Besides its use for surgeon training and education, virtual reality can also help educate patients, according to sources who spoke with Healio | Orthopedics Today. Educating patients prior to surgery can not only improve outcomes from “a mental health point of view but also from a compliance perspective,” Goel said.
“Enabling one to understand the 3D world using 3D technology is a powerful value proposition across the patient care lifecycle,” Goel told Healio | Orthopedics Today. “Implementing these technologies can have a dramatic impact without much more additional cost.”
In addition, virtual reality can benefit patients by guiding them through physical therapy and rehabilitation.
“There will likely be an evolution of some form of mixed reality to allow patients to be able to have assessments to make sure they are doing the exercises correctly, either currently with avatars that allow you to watch the exercises being done or, at some point, integrating your range of motion to see what you should be doing to what you are actually doing,” Levy said.
Source: PrecisionOS
Better patient outcomes
However, more research is needed to prove that use of virtual and augmented realities translates to better patient outcomes, according to Calendine.
“This is not about what is new or shiny. This is about how do we provide better health care,” Calendine told Healio | Orthopedics Today. “In education, it is probably more virtual reality. In execution, it is more augmented reality. But both must be connected to our goal, which is improvement for patients.”
Iannotti said the continued digitization of real-life information to identify common mistakes and pathways to optimal results may develop better systems and reliable processes in patient care, leading to safe and reproducible outcomes.
“How do you take real-life observational data and digitize it in an AI format to say this is how you avoid problems, this is how you build more reliable systems and this is how you get better outcomes?” Ianotti said.
But Levy said the “holy grail” of evolution of augmented reality is when the technology is sophisticated enough that it will allow surgeons to immediately navigate the case without placing anchor points to bones.
“That is going to be the evolution that will be the major game changer, but I do not think any company currently has that technology,” Levy said.
Cusp of change
Overall, Iannotti said it takes most new technology 5 to 10 years to become more widely adopted and have adequate market penetration. However, once higher market penetration begins to improve, “most of the industry figures out how to improve upon it, make it less expensive [and] easier to use,” according to Iannotti.
“All of that is going to happen with navigation, augmented reality and virtual reality in orthopedics,” he told Healio | Orthopedics Today. “We are just seeing the cusp of this.”
According to Barad, augmented and virtual realities are in a “change management phase where people are getting used to this technology.” He said elimination of any “points of friction” and dialing in the user experience may help improve the technology. This includes physicians not worrying about having to set up the training space or logging into their accounts.
Barad added it is also important for surgeons to be able to access the content they need for a particular surgery.
“Making it easy for surgeons to build and access the content that they need, that is another area that we have a lot of exciting things we are working on,” he said.
Expansion of technology
As surgeons become more comfortable with visually engaging with the digital world, Calendine said use of augmented and virtual realities will expand. However, he said the technology needs “to be easy to use, the cost has to be manageable and [surgeons] have to be directly connected to the results that we care about. In medicine, of course, it is patient care.”
“[Are augmented and virtual realities] going to change health care tomorrow? No, but will health care look anything like it does today 10 years from now? I certainly hope not because if it does, it means we did not leverage technology to improve,” Calendine said.
As this technology continues to evolve and become more dominant in the orthopedic specialty, Barad said it is important to make the changes “in a manner that does not feel overwhelming or chaotic.”
“In health care, there is a lot of new stuff happening and it is overwhelming,” Barad said. “Even when things work better and solve big problems for us, there is this component of getting used to things and getting accustomed to things.”
He continued, “As all of those changes are happening with virtual reality, augmented reality and artificial intelligence, we need to be mindful of health care professionals and stakeholders and roll those changes out in a thoughtful way that takes this human element into account.”
- References:
- Berezowsky CA, et al. Journal of Musculoskeletal Surgery and Research. 2022;doi:10.25259/JMSR_129_2022.
- Bian D, et al. Front Surg. 2024;doi:10.3389/fsurg.2024.1336703.
- Blumstein G, et al. J Surg Educ. 2020;doi:10.1016/j.jsurg.2020.01.002.
- Chytas D, et al. Front Surg. 2019;doi:10.3389/fsurg.2019.00038.
- Dinizo M, et al. Bull Hosp Jt Dis (2013). 2022;80:75-79.
- Gupta A, et al. Cureus. 2022;doi:10.7759/cureus.29099.
- Hasan S, et al. Cureus. 2023;doi:10.7759/cureus.43964.
- Lohre R, et al. JSES Int. 2020;doi:10.1016/j.jseint.2020.02.005.
- Longo UG, et al. Appl Sci (Basel). 2021;doi:10.3390/app11073253.
- Matthews JH, et al. Curr Rev Musculoskelet Med. 2021;doi:10.1007/s12178-021-09713-8.
- McKinney B, et al. J Surg Educ. 2022;doi:10.1016/j.jsurg.2022.06.008.
- Orland MD, et al. Clin Orthop Relat Res. 2020;doi:10.1097/CORR.0000000000001362.
- Science & tech spotlight: Extended reality technologies. https://www.gao.gov/products/gao-22-105541. Published Jan. 26, 2022. Accessed July 19, 2024.
- Sugano N. J Orthop Sci. 2003;doi:10.1007/s10776-002-0623-6.
- For more information:
- Justin Barad, MD, of Osso VR, can be reached at jhbarad@ossovr.com.
- Cory L. Calendine, MD, of the Bone and Joint Institute of Tennessee, can be reached at cory.calendine@bjit.org.
- Danny P. Goel, MD, MBA, MSc, FRCSC, of PrecisionOS, can be reached at danny@precisionostech.com.
- Joseph P. Iannotti, MD, of Cleveland Clinic Florida, can be reached at allena@ccf.org.
- Jonthan C. Levy, MD, of the Levy Shoulder to Hand Center at the Paley Orthopedic & Spine Institute, can be reached at jonlevy123@yahoo.com.
- Matthew D. Saltzman, MD, of Northwestern Medicine, can be reached at mark.rudi@nm.org.
Click here to read the Point/Counter to this Cover Story.
Collapse
Read more about
Click Here to Manage Email Alerts