TROY, N.Y. — A surgeon makes an incision on a virtual patient with support from a perioperative nurse, while an anesthesiologist monitors the patient’s vital signs. As the procedure continues, the team members navigate together through any challenges that arise — even though each of them may be participating from different rooms, buildings, or even cities.

A new $2.3 million grant from the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health will support a research effort led by Rensselaer Polytechnic Institute to make that virtual scenario — and others like it — a reality.

“People will be wearing head-mounted displays, and they will be immersed in a virtual operating room working on a virtual patient as a team,” said Suvranu De, the director of the Center for Modeling, Simulation, and Imaging in Medicine at Rensselaer, who is heading up this effort. “We want to have an expert team in the operating room focused on the treatment of a patient, and not just a team of experts.”

De and his team have dedicated more than a decade of research to making surgery safer by developing virtual reality-based surgical simulations that closely mimic the optics and haptics a surgeon may encounter in the operating room.

Those simulations have allowed surgeons to practice essential technical skills — especially those associated with hand-eye coordination — in a no-risk environment. Now, the Rensselaer researchers are developing a collaborative virtual reality-based surgical simulation environment that allows medical professionals to practice technical, cognitive, and interpersonal skills as a team.

Conceptually, this approach is similar to crew resource management practiced by aviation pilots, which has led to a significant reduction in aircraft accidents. The Virtual Operating Room Team Experience (VORTeX) simulation system will provide realistic distractions, interruptions, and other stressors that medical professionals may encounter in an operating room.

Traditionally, this type of simulation training has required mannequins, instructors, and a dedicated space, as well as significant coordination and resources. In contrast, the VORTeX system will be both distributed and asynchronous – allowing participants to join the simulation from different locations, and instructors to review the simulation and provide feedback at their convenience. Machine learning algorithms will be used to crunch the data and provide feedback to participants, who will be able to return to the virtual environment to review their performance.

The COVID-19 pandemic has highlighted the importance of such a flexible approach. Educators and professionals are looking for new ways to train and complete procedures while maintaining social distance.

“Teams have to train in a very different way because of COVID-19,” De said. “These are very complicated processes, which actually expose the clinicians to threats, and there is no second chance. So being able to perform these kinds of things in the virtual environment does open up significant possibilities.”

De is working with Alhussein Abouzeid, a professor of electrical, computer, and systems engineering at Rensselaer, as well as teams from Beth Israel Deaconess Medical Center, Albany Medical Center, and the University of Central Arkansas to develop the VORTeX. A steering committee of the American College of Surgeons (ACS) will advise the team. The technology developed will have the potential to positively impact training programs offered through the national network of ACS-accredited simulation centers.

“We hope this spreads through the country and the world and that it has the same effect this kind of training has had on the aerospace industry in enhancing patient safety,” De said.