A virtual reality simulator in development for training, teaching intraocular surgery

Ophthalmologists can learn surgery without any risk to the patient.

Today, virtual reality has gained acceptance in many different areas of education. The use of flight simulators for the training of pilots has shown that virtual reality can improve common forms of training. Until today, the high complexity of medical operations prevented computer simulators from being applied broadly in clinical training. Increasing computational power and the development of new algorithms for the simulation of biomechanical behavior will lead to an increasing use of simulators in the medical sector.

Training in ophthalmic surgery involves high cost. A 2-year-period is needed to prepare the future surgeon for his profession. At the beginning, it is mainly the hand-eye coordination that is difficult to learn. Because eye surgeries are performed using a stereomicroscope, hand movement is decoupled from natural vision, and the coordination is different from the accustomed way. In the limited space of the surgical field, tiny hand movements can have fatal consequences. Hand-eye coordination can be well-learned using virtual reality. The future surgeon can experience, explore and get used to the small dimensions of such operations.

EyeSi2, a virtual reality simulator for eye surgery training, is not only for beginners. Staying current with new surgical techniques requires life-long learning for eye surgeons. After implementation as a virtual reality, new techniques can easily be explained and taught through systems like EyeSi2.

EyeSi2 can also be used for the development of new surgical instruments. In virtual prototyping, a new surgical instrument is designed and immediately tested in the simulator. Shorter development cycles and smaller risk for the patient are essential advantages of the simulator.

Surgery in cyberspace

Simulators using virtual reality must be constructed in a way so that the user feels as if he or she has been transported into the situation that is simulated. The user should forget his or her actual surroundings and operate completely naturally in the virtual reality. All necessary senses of the user must be involved in order to achieve this feeling. Virtual reality aims at merging real sensory perception with virtually presented images, creating complete involvement. To simulate an intraocular operation, the stimulation of the visual and acoustic senses is feasible.

In the case of EyeSi2, stereoscopic images of the virtual operation scenario are shown on two small LCD displays that are fixed in the shape of a microscope eyepiece. Through these displays, the surgeon watches the computer-generated operation scenario three dimensionally. The original surgical instruments are introduced in a mechanical model of the eye. The artificial eye is constructed in a way that its characteristics of movement correspond to those of the human eye. It is fixed on an operating table covered by a mask modeling a human face. The artificial eye serves as the interface to the virtual world. Instruments can be moved in the same way as they are moved when operating on a human eye. Rotating the eye out of its rest position generates backdriving forces corresponding to the muscles of the eye. An optical tracking system observes the eye model from below. It sends data about the current position and direction of the instruments as well as the orientation of the eye to a computer. The movements of the instruments and the mechanical eye cause their virtual counterparts to move correspondingly.

When a virtual instrument collides with a pathological membrane of the virtual world, the computer calculates the reaction of the membrane with the help of biomechanical models. Using such mathematical models, interactions with a broad variety of pathological structures can be simulated. After having calculated the reaction, the computer displays the updated virtual scenario. These changes happen within fractions of seconds to generate a realistic impression.

---An operation in the eye is performed under the stereomicroscope (left side). The simulation (right side) takes all relevant aspects of the operation into account. With the help of two small LCD displays, the three-dimensional view of the microscope is presented to the user who works with original instruments in a mechanical model of the eye.

The mechanical eye

The mechanical eye is a metal hemisphere offering the same rotational degrees of freedom as the human eye in the eye socket. The effect of the eye muscles is modeled by springs with appropriate forces. The mechanical eye has two puncture marks through which the surgical instruments can be introduced.

At the equator of the eye, two marks are visible from below. These marks and the fixed sphere’s center give a plane that defines the rotational state of the eye. The surgical instruments are also marked at the tip. Their exact position in space can be determined by the fixed puncture marks in the mechanical eye.

Optical tracking

The optical tracking system is used to determine the current position and direction of the manipulated real instruments and the orientation of the mechanical eye. The computer uses these coordinates to model their virtual counterparts at corresponding positions in the virtual world. The tracking system constitutes the connection between the instruments, which are manipulated by the user’s hands, and the virtual instruments. The tracking system consists of three CCD cameras, which provide a tracking rate of 50 Hz for all four markers.

The cameras are fixed below the mechanical eye, which is observed under different angles. The position of the instrument’s tip and the orientation of the eye are calculated from two camera images by means of stereoscopic transformation. The tips of the instruments can be defined easily with the help of the marks.

---Comparison between operation and simulation. The left side shows video captures from a real operation; the right side shows screen shots of the simulation.

Computer graphics

The optical tracking system registers the positions of the instruments as well as the orientation of the mechanical eye and transmits the information to the computer. The computer modifies the virtual scene correspondingly to the new tracking data. Based on anatomical information, an OpenGL model of the eye was developed for the virtual scene. Texture mapping is used to project real photos of the eye’s background and an iris on the appropriate positions in the computer-graphic eye. Great attention was given to the modeling of the spotlight from the cold light source and the shadow produced by the instruments, as they are important navigational means. When simulating intraocular operations, the distance between the shadow and the tip of the instrument gives cues about the depth of the instrument in addition to the three-dimensional view through the stereo-microscope. As the distance between the tip of the instrument and the shadow reduces, the distance between the instrument and the retina decreases.

---This illustration shows the mechanical eye that serves as the interface to the virtual world. The eye is constructed so that its characteristics of movement correspond to those of the human eye. Springs are fixed on three rotational axles, exerting forces on the eye when it is rotated from its rest position. The instruments are introduced into the eye through two small punctures.

Biomechanical simulation

When the instruments are moved inside the eye, collisions with structures of the eye may occur. If, for example, an instrument touches a pathological membrane, this membrane will be modified correspondingly to the forces exerted by the instruments and the membrane itself. Normally, a slight touch of the retina causes immediate bleeding, which the surgeon perceives as a red spot on the retina. Such events must be visualized immediately. Therefore, it is important that the computer can quickly determine whether a collision occurred in the virtual world and, if so, which structures were concerned and how the virtual world has to be modified.

The interaction with a pathological membrane makes high demands on the computer. The first steps consist of determining displacements and the forces that are having an effect on the membrane. The forces are used to calculate the reaction of the membrane, which is described by a biomechanical model. For simulators like the EyeSi2, it is important for the calculation of the biomechanical reaction to be fast, because otherwise the simulation begins to flag and becomes unrealistic. The speed of the calculation depends on the complexity of the membrane’s mathematical description, which again depends on the approach chosen for the biomechanical model of the membrane.

As far as the EyeSi2 is concerned, two different approaches are applied. Mass-spring-models are based on the idea that mass points are connected to each other by springs. Moving one mass point generates forces on adjacent masses through the connecting springs. Mass-spring-models are used to simulate membranes (two-dimensional objects in space). ChainMail is an approach in which an object is considered to consist of many chain-links that are crossed over like a coat of chain-mails. In this case, a singular chain-link can be moved freely within certain borders. If these borders are passed, the modification will be transmitted to the neighboring chain-links as well. ChainMail is applied for the simulation of two-dimensional objects (membranes) as well as for the simulation of three-dimensional structures like the vitreous humor.

---This illustration shows the reconstruction of the three-dimensional position of an instrument’s tip. The three cameras observe the scenario from different angles. The instrument’s position in space is calculated from the position of the image of the instrument’s tip on the cameras’ sensors using a stereoscopic transformation.

Future development

EyeSi2 will be further developed by VRmagic. A new version of the mechanical setup is almost finished. It will be essentially smaller and tougher than its predecessors and provide an even higher level of realism.

VRmagic’s current work on software development aims at accelerating the simulation algorithms, which will allow a further increase of the complexity and realism of the virtual reality.

In cooperation with the MAD-Lab (Microsurgery Advanced Design Lab) at the Wilmer Eye Institute of Johns Hopkins University in Baltimore, Md., VRmagic is working on the simulation of complete surgeries.

Anatomical model of the eye. The rear part of the eye is semitransparent to allow a view of the retina (left). The eye from the front with surgical instruments inserted (right). The cold light source is introduced from the left side, a vitrector from the right side.
Comparison of operation and simulation. Interaction with a membrane during a “real” operation on a patient (left) and simulated membrane interaction (right).

For Your Information:

• Michael C. Knorz, MD, can be reached at the department of ophthalmology, Klinikum Mannheim, Mannheim, 68135 Germany; 49-621-383-2242; fax: 49-621-383-3803; e-mail: knorz@eyes.de. For further and up-to-date information, go to VRmagic’s Web site at www.vrmagic.de. Dr. Knorz has no direct financial interest in any products mentioned in this article, nor is he a paid consultant for any companies mentioned.