Mixed Reality Using Microsoft's HoloLens is Changing the Future of Medicine Studies

🗣 At Microsoft’s annual Build Conference, Professor Mark Griswold told a global audience about the potential of Microsoft HoloLens to transform learning. Twelve months later, Griswold showed the world exactly how – or at least his hologram did –.

Speaking for the second consecutive year, Griswold was joined by School of Medicine’s Dean, Pamela Davis and two university developers, Jeff Mlakar and Henry Eastman. All three of them were actually on stage in person at San Francisco’s Moscone Center. The group’s task? To demonstrate the immense advantage that three-dimensional images can give to students learning anatomy.

Six months ago, Case Western Reserve and Cleveland Clinic broke ground on a 485,000-square-foot Health Education Campus, a space designed to support interprofessional learning and offer the most advanced technology available. As part of creating what Davis called a “state-of-the-future” building, the new space will not include any of the traditional cadaver-filled laboratories that for decades have housed anatomy classes. Instead, just as Mlakar and Eastman did, medical students will use HoloLens headsets to see the body’s organs and systems.

The pair began with a hologram of the digestive system, with labels on organs including the stomach, gall bladder and liver. Davis asked them about the pancreas, which, nestled behind the stomach, cannot be seen from the front of the body. “Thankfully with HoloLens, it’s really easy to get the best view of things,” Mlakar said, making a small hand gesture that caused the holographic body to rotate 180 degrees and reveal the organ.

It’s also easy to engage with an instructor hundreds of miles away, as Professor Griswold demonstrated when a simple 3-D head and arm represented him entering the sold-out auditorium. “This is our new system, which allows me to teach and interact with you, even though I’m not there. … This is really changing what it means to be in class”, added Mlakar.

Based on an actual patient’s MRI from Case Western Reserve Professor Cameron McIntyre’s lab, Griswold guided Eastman and Mlakar through an examination of white-matter tracts of the brain. The hologram color-coded the fibers by direction, and included a bright red mass that Griswold identified as a tumor.  “Can you see how this tumor intersects the light blue track, but not the green track or the yellow one?” Griswold’s hologram asked, and indeed they could, illuminating the two lobes whose fibers the tumor crossed.

Griswold, a world-renowned radiology researcher, explained that he had worked with datasets of brain MRIs for more than a decade, “and I never fully understood their 3-D structure until I saw them in HoloLens.” Davis added that students who had used the HoloLens devices reported that 15 minutes with the three-dimensional images could have saved them dozens of hours in their traditional anatomy labs: “The quicker our students learn facts like these, the more time they have to think with them. We are teaching them to think like doctors.”

Case Western Reserve was the only university among 30 organizations that partnered with the company for initial pilot testing of the HoloLens’ potential. Others include NASA, Japan Airlines, Volvo and the Dubai Electricity and Water Authority. Case Western Reserve also was the only organization to give a HoloLens demonstration on the Build stage and the university’s team was introduced by Microsoft’s Alex Kipman, the lead creator of the HoloLens device.

In the speaker’s words, technology not only can match existing educational methods—it can actually improve upon them. In many ways, this is why Cleveland Clinic CEO Toby Cosgrove, contacted then-Microsoft executive Craig Mundie in 2013 after the hospital and university agreed to partner on a new building that has become a 485,000-square-foot Health Education Campus project: “We launched this collaboration to prepare students for a health care future that is still being imagined, by combining a state-of-the-art structure, pioneering technology, and cutting-edge teaching techniques, we will provide them the innovative education required to lead in this new era.”

As Cosgrove, the more other academic leaders engaged extensively with Microsoft, the more potential everyone saw. After they first experienced HoloLens in december, the faculty returned to Cleveland to create a core team dedicated to exploring the technology’s academic potential. A couple of months later, ten members of the team returned to Microsoft for a HoloLens programming deep dive.

Case Western Reserve was the only university represented during the three-day event, a distinction Griswold attributes in part to the core team’s breadth of expertise and collegial approach. “Without all of those people coming together, this would not have happened,” he said.

Because the technology is relatively easy to use, students will be able to build, operate and analyze all manner of devices and systems, “It will encourage experimentation leading to deeper understanding and improved product design.” said one of the faculty members.

HoloLens ultimately could have applications for dozens of Case Western Reserve’s academic programs. NASA’s Jet Propulsion Laboratory already has worked with Microsoft to develop software that will allow Earth-based scientists to work on Mars with a specially designed rover vehicle. A similar collaboration could enable students here to take part in archeological digs around the world, or allow astronomy students to stand in the midst of colliding galaxies, securing a front-row view of the unfolding chaos. Art history professors could even present masterpieces in their original settings—a centuries-old castle, or the Sistine Chapel.

For now, however, the top priority is creating a full digital anatomy curriculum, a process launched with the advent of the Health Education Campus, and now experiencing even greater momentum. Something essential, Griswold said, has been the advice and assistance of Microsoft’s HoloLens team and executives: “It’s been a joy to work with them. They have been so collaborative and we’re going to do incredible things together.”

👉 Case Western Reserve University has already taken the next step to ensure an A-class education using HoloLens and mixed reality technology 💯.

Are you ready to do the same for your company? Check out our services here.