At any one time, the effect of the stylus is signalled by its appearance; ... The stylus, held by the user's invisible hand, can move brain structures, data plane slices ...
Appeared in: Visualization in Biomedical Computing 1996, Hamburg, Germany, pp. 491-500.
The Brain Bench: virtual stereotaxis for rapid neurosurgery planning and training Tim Poston, Wieslaw L. Nowinski, Luis Serra, Chua Beng Choon, Ng Hern CIeMed1 Institute of Systems Science, National University of Singapore Kent Ridge, Singapore 119597 Prem Kumar Pillay The Brain Centre, Singapore General Hospital College Road, Singapore 169608
Abstract This surgical planning and training system integrates an electronic atlas of brain structure with a virtual stereotactic frame and patient data, for intuitive, reach-in manipulation. The objective is plans prepared faster; better, more accurate choice of target points; improved avoidance of sensitive structures; fewer sub-optimal frame attachments; and speedier, more effective training. If validated by clinical study now under way, this will improve medical efficacy and reduce costs. Keywords: Brain atlas, image-guided surgery, neurosurgery planning, frame-based stereotaxis, virtual reality, virtual workbench
1 Introduction 3D views have become common in medical software, though many scans are still displayed as a spread of 2D slices. Stereo display adds to the sense of depth, but—in most software—manipulation is still done as through a glass, clumsily; an object that does not move and turn with the hand is awkward to control. 3D display demands a 3D interface, but is not in itself such an interface. Merging the display space and the user's personal space constitutes Virtual Reality, but VR alone is not enough for a clinical tool fit for routine use. Dextrous tasks require a high resolution display, a well calibrated relation between hand and displayed object, and minimal conflict between depth cues. Our collaboration, between a neurosurgeon and the creators of a hand-eye coordinated interface and of an electronic successor to clinically standard brain atlases, aims to achieve this in a lifecritical area of high-precision surgery. We first describe the standard approach to planning brain probe insertion, and some of its problems. We outline the `Virtual Workbench' general purpose 3D interface scheme and the electronic Brain Atlas, then the Brain Bench tool that incorporates them. An account of some planning procedures for particular types of intervention is followed by summaries of the advantages we see in the new system and of our approach to validating these advantages, and future plans.
2 Background Stereotactic neurosurgery dates from 1906. Using X-rays, surgeons hand-computed the (x,y,z) coordinates of target lesions and adjusted a stereotactic frame to guide the delivery of a penetrating instrument (biopsy needle, 1. The Centre for Information-enhanced Medicine, jointly created by the Institute of Systems Science of the National University of Singapore and Johns Hopkins University.
1
Appeared in: Visualization in Biomedical Computing 1996, Hamburg, Germany, pp. 491-500. electrode,...; we will merely say `probe') to the target location chosen. The guiding frame is movably—and precisely controllably—attached to a fixation device, rigidly screwed to the skull, in a position established with millimetric precision. Only a narrow hole is opened in the skull, and no sight path opened in the brain. Volume scans such as CAT and MR have made target localization far more accurate, increasing the range of cases treatable, and reducing morbidity and cost. Software support has still been based on fundamentally 2D display paradigms, displaying volume data as a set of slices (a surface reconstruction of the lesion itself may appear in one or more monoscopic 3D views, with shading techniques as the principal depth cue). In a typical interface the surgeon chooses a target in one slice, an entry point on another, and studies slice by slice the surroundings of the point where the line joining them passes through. No planning software has included a 3D brain atlas to assist the surgeon in locating brain structures unrevealed by the scan. When 2D (slice by slice) brain atlases have been included, they have used a purely linear fit based on the two inter-commissure landmarks of the Talairach-Tournoux (TT) scheme [11], rather than the whole piecewise-linear TT proportional grid. No software has assisted in the initial placement of the fixation device for frame-based surgery which during scan holds fiducial markers, and during surgery the guiding frame. Some disadvantages have been: • Spatial data have the six degrees of freedom of Euclidean motion. Control by a two-degree-of-freedom mouse is necessarily indirect, making any interface laborious to use, and gives it a slow learning curve. Long training is costly, as are errors by a surgeon who is trained but not yet long in experience. • When a target point should be deep in an asymmetric lesion, where the deepest point does not appear central in any one slice, selection can be hard. • If the selected path is oblique to the displayed slices, it is not easy to judge by how far (obliquely to the slice planes) the probe will miss a sensitive structure, such as a nerve, blood vessel or eloquent brain area; particularly for unseen structures, whose position is anatomically inferred. This makes it hard to estimate dangers due to errors in MR localization or registration. • Structures such as blood vessels are hard to perceive as 3D networks from the pointwise way they meet separately displayed slices, with gaps easily crossed in 3D. • The initial placement of the fixation device on the skull involves directly all six Euclidean degrees of freedom, so that nobody has yet designed a software planner for it that surgeons find better to use than placement `by eye'. Mouse-based planning tools all start with data from a scan with the fixation device (and fiducials) already in place. For a less experienced surgeon, in a centre new to these techniques, a poor fixation placement can force a sub-optimal probe path, or even rescheduling the entire procedure. Goble et al. [2] describe a 3D interface for image exploration, using hand-held props, though these are not used in their probe path selection, which does not include an atlas. Davey et al. [1] display insertion of a probe in a stereo view of patient data, clarifying its relation with DSA-revealed arteries in 3D, but manipulation is by 2D mouse and atlas work is not yet integrated with their planning system. Höhne et al. [3] gives a mouse-based interface to a probe entering a 3D brain atlas, not registered with patient data or frame position. we describe the Brain Bench, created at ISS and now starting further development and clinical testing at the Brain Centre of Singapore General Hospital. It is based on the Virtual Workbench [7] general environment for careful work in 3D, and the Electronic Brain Atlas [5], a combined electronic version of brain atlas books standard in stereotaxis, reorganized as a system of 3D structures. We describe these briefly below, with an account of their combination in the Brain Bench, and validation plans.
3 The Virtual Workbench Conventional VR immerses the user in a computer-constructed environment via a head-mounted display. We have found it more productive for the user to `reach in' to a 3D display where hand-eye coordination allows careful, dextrous work in examination of data and manipulation of computer constructs. The user reaches behind a mirror in which a 3D display shows both the patient data and a visual echo of the physical tool handle that the user's hand controls, in the same apparent place. This ‘virtual’ tool handle interacts with the data as a grasper, cutter, marker... as arranged by software. The ‘see it where you feel it’ coordination contrasts with the remote props in [2]. A more technical account is in [7]. Informally, the Virtual Workbench is a box one reaches into, with invisible hands holding visible tools. The left tool typically grasps objects and moves them (using the motor habits learned in childhood), while the dominant hand performs delicate work. A curve or other shape may be created, cut, bent,..., or one can make measurements with a virtual ruler; just place it between points of interest, as with a real one. The ruler typifies the contrast between the Virtual Workbench and a slice-by-slice 2D display; for the distance between a point P and an extended object E, a stereo view makes it easy to select the point in E nearest to P. For the mutually nearest
2
Appeared in: Visualization in Biomedical Computing 1996, Hamburg, Germany, pp. 491-500. points in E and another object E’— usually best estimated as between data slices—with a reach-in tool one can measure their distance directly. Finding these nearest points can be hard in a 2D system. This contrast increases when real-time volume display of data becomes possible, for instance by the use of ‘3D textures’ (see, e.g., [8]) to display a structure. Instead of contour-outlining it slice by slice, one directly displays it as a 3D solid (not an extracted surface) by making the material around it transparent. Picking a point on the (easily perceived) surface is hard with a 2D mouse, easy by reaching in.
4 The Electronic Brain Atlas The brain atlas database [4] developed at CIeMed contains electronic versions of several paper brain atlases used in day-to-day clinical practise: • Atlas of Stereotaxy of the Human Brain; Schaltenbrand and Wahren [9] • Co-Planar Stereotactic Atlas of the Human Brain (TT); Talairach and Tournoux [10] • Referentially Oriented Cerebral MRI; and Anatomy. Atlas of Stereotaxic Anatomical Correlations for Gray and White Matter; Talairach and Tournoux [11] • Atlas of Cerebral Sulci; Ono, Kubik and Abernathey [6] The atlas data from the paper atlases are digitized, enhanced, colour coded, labelled, and organized into volumes. Various 3D extensions of them are in effect, and more are in progress. The workstation system [5] supports registration, 3D display and real-time manipulation, object extraction/editing, quantification, image processing and analysis, reformatting, anatomical index operations, and file handling. Its primary goal is automatic labelling and quantification of human cerebral structures. Its main applications are (3D) neuroeducation, (quantitative) neuroradiology, (stereotactic) neurosurgery, and neuroscience. Each source atlas has its own strengths. The TT holds anatomical and functional information, with axial, coronal, and sagittal sections taken from a single brain specimen. Its proportional grid system lets one localize cortical and subcortical structures in 3D space, and to compare different brains. The TT and its grid are a clinical standard, mainly used for functional MRI and PET studies. The Schaltenbrand-Wahren atlas [9], based on 111 brains, contains macroscopic and microscopic frontal, sagittal and horizontal sections through the hemispheres and the brain stem. The macroscopic sections give the extent of brain structure variation. The microscopic ones detail cerebral deep structures rarely displayed well by MRI or CT, such as the thalamic nuclei. The Atlas of Cerebral Sulci [6] studies 25 brain specimen for variation and consistencies in location, shape, size, dimensions, and relationships to the internal structures. It contains two types of information: drawings of the sulcal patterns (showing variability of side branches, etc.), and variation incidence rates. The mutual pre-registration of these atlases merges their information (such as different parcellations of the thalamus) and makes them all applicable to a scan when the user brings any one of them into register with it.
5 The Brain Bench We have constructed an initial version of a system in which a Virtual Workbench user can manipulate brain scan data, with a model of the frame by which probes are controlled (Figure 1), and brain structures from the electronic Brain Atlas (coloured surfaces), registered by the standard surgical landmarks with the scan data. This ‘Brain Bench’ currently contains the TT deep brain structures, where patient variability is relatively small once adjustment to its standard coordinates is made. We are working on support for the other datasets of the multi-source atlas above, with its tools for analysis and managing variation. We have aimed to make this environment straightforward, intuitive and quick to use and learn. The user's grasping hand controls the entire displayed complex, turning and placing it as one does an object held in real space. The other does detailed manipulation, using 3D widgets (interaction gadgets) to interact with the surgical planning tools. In all interactions, `reach in for the object of interest and press the switch on the stylus to interact with it'. When the stylus tip enters the volume of influence around an object, the object signals ‘Reactive!’ by a colour change or other highlighting. At any one time, the effect of the stylus is signalled by its appearance; a slice-selector, a point-mover, a volume-of-interest selector, etc. When it enters a widget's volume of influence, it becomes the manipulator associated with that widget, that knows how to interact with it. For example, while it is in the volume of the slider, the pointmover transforms to the slider-dragger; it will not point-move the slider widget but drag its bead.
3
Appeared in: Visualization in Biomedical Computing 1996, Hamburg, Germany, pp. 491-500.
arc
scan slices stylus slider
drag directions
toolrack
Fig 1 The Brain Bench. The stylus, held by the user's invisible hand, can move brain structures, data plane slices, the zoom slider, and the arc (a preliminary model of a stereotactic frame). The button rack controls its mode of action. The deep brain structures from the atlas behave like widgets. When the stylus moves near them, the structure nearest it remains opaque and is labelled with its abbreviated TT name; the others turn translucent, revealing its shape in context. The user can then press the stylus switch to grip it, and examine it as if on a skewer held by the stylus hand. Two sets of widgets control the stereotactic frame: one controls the target and the other the entry point. The arc's hinge direction is fixed by the base frame, whose virtual form can only be dragged without turning; selecting one or more of the arms at the end of the arc in Figure 1 and Figure 3 allows dragging it with those degrees of freedom. The entry path is adjusted by moving the arc on its hinge, and the probe guide along the arc, or both at once. The patient's data mainly appear as tri-planar cross-sections. The cross-sections are controlled by reaching to one or more of them, and dragging them to other, parallel planes (Figure 2). Another cross-section can be enabled, orthogonal to the path of the probe (Figure 3). Sliding it up and down this path shows a non-oblique view of the path's passage through the scan data. One can also enable a cross-section plane containing the path, and turn it with this path as axis.
4
Appeared in: Visualization in Biomedical Computing 1996, Hamburg, Germany, pp. 491-500.
Fig 2 Dragging data planes, by their meeting point, through the thalamus and hippocampus.
5
Appeared in: Visualization in Biomedical Computing 1996, Hamburg, Germany, pp. 491-500.
Fig 3 A data slice orthogonal to the probe, dragged along it to study impact.
6
Appeared in: Visualization in Biomedical Computing 1996, Hamburg, Germany, pp. 491-500. A ‘toolrack’ menu at the bottom switches between stylus modes. As the stylus moves over it, the button nearest the tip is highlighted. The rack announces the button's function, enabled by clicking. A slider scales the whole frame/ data/atlas ensemble, allowing detailed or overall inspection and manipulation. A scroll list widget lets one select, identify and highlight chosen brain structures (often hard to find among the rest). We also provide such generic tools as a volume-of-interest widget (which lets one volume render only what is in a box with draggable corners), and a hand-held slicing tool for viewing arbitrary cross-sections. Modelling the initial attachment of the fixation device to the patient's skull, a natural extension of this toolkit, will allow the surgeon to test the feasibility of a proposed combination of placement and adjustment.
6 Surgery Planning Using the Brain Bench A stereotactic procedure is usually performed in four stages: (i) frame fixation, (ii) diagnostic imaging, (iii) computerized neurosurgery planning, and (iv) neurosurgery. The Brain Bench addresses (i) and (iii). The frame model shown, a simplified version of the Leksell system, will be replaced by exact models of several frames in clinical use. A semi-circular arc can rotate about to an axis, which slides in three dimensions relative to the fixation device, controlled by the widget at left in Figure 1. The probe guide moves along the arc, and constrains the probe to move along a straight line; in several systems, along a radius through the arc centre (often placed at the chosen target). A procedure involves two steps, placement of the fixation device and of the probe path. Where the scan data include a system of fiducials attached to an already fixed device (the current normal procedure), locating it can be automated. When planning support begins earlier, as we propose here, a virtual version of the device can be placed freehand over any 3D model of the patient's head (as volume or surface) in which the target can be moderately well localized. If an approximate plan for the second step gives a path securely far from obstruction by screws or other frame constraints, the real fixation device can be similarly attached and the fiducial-equipped scan taken, with a much reduced likelihood of obstruction later. After this scan, the normal precise registration of the device position with the scan data is done. Next, the probe path is selected (Figure 4). The chosen target becomes the arc centre, then the probe path changes with the angle of the arc and, on it, of the probe holder. The tools already described help in judging candidate targets and paths, as one varies the choice by dragging around the target (select the robe tip) or the approach angle (select the guide tip). The entry path is chosen to minimize damage to the structures along the penetrated path (eloquent brain, blood vessels, critical deep structures, etc.). Beside visual observation of the data with the tools above, and of the path's relation to brain atlas information, the system lists (and optionally highlights) the atlas structures intersected. A cortical surface labelled with the Brodmann (functional) areas is under development.
7
Appeared in: Visualization in Biomedical Computing 1996, Hamburg, Germany, pp. 491-500.
Fig 4 Swinging the arc and dragging the probe around it. Impacted structures are highlighted. Once a path is selected, the system prints out the mechanical adjustment values to which the frame must be set.
8
Appeared in: Visualization in Biomedical Computing 1996, Hamburg, Germany, pp. 491-500. Two major Brain Bench applications are tumour stereotaxis and functional neurosurgery. In tumour stereotaxis, the target is selected using the three orthogonal views; if the tumour shape has been extracted from the scan as a set of voxels, a set of contours, or a surface, this can be displayed also. These cues guide choice of target(s) within the tumour. In functional neurosurgery, e.g., thalamotomy, pallidotomy, or hippocampoerectomy, the target is an anatomically defined point in the structure to be treated, chosen on functional grounds. The atlas acts as a guide to the target structure in the patient data (as well as segmenting and labelling the structures surrounding it), for a more informed initial guess in placing an electrode to locate the target by functional response (along a good line for sampling at several depths), hopefully leading to a good placement in fewer steps.
7 Projected Advantages • The reach-in interface allows natural manipulation, easier to learn and faster to use than moving 3D objects by pushing a 2D mouse around a pad. • The surgeon adjusts the whole probe trajectory as a visual object, among atlas structures and volume displayed scan data. • Good visual selection of a deep point within a structure becomes easier. • An easily-used placement planner will reduce the effort, time and patient discomfort now caused by misplaced fixation devices. • The 3D system will provide an improved learning tool for the mental 3D anatomy model that trainees must still acquire, creating a better one faster than 2D images can provide.
8 Validation plan The first test of the system will be of its use in the education and training of neurosurgeons. We will compare • surgeons trained with the traditional method, involving hands-on experience with the stereotactic system, observation of patient treatment, and finally performing an actual case under supervision • surgeons trained by combining these with the Brain Bench for planning. Each will be tested both on the ability to carry out a procedure efficaciously and safely, and on the time taken. We will also test the young neurosurgeons on the accuracy with which they identify anatomical structures, and their localization of target centres, with traditional methods and with the Brain Bench. We will compare target and trajectory selection with that of trainees using traditional methods, against a ‘gold standard’ best choice provided by the senior neurosurgeon in charge, with experience of more than 1,000 cases. Accuracy testing will be performed with sharply defined landmarks. Choosing a target point that can be anatomically located with submillimetric precision, we will test the frame settings found against those found with traditional software, for access at the same angle. Predictions of the point at which the probe will meet different data slices will be compared for the two systems. Finally, using an established neurosurgical team with a good track record in stereotactic neurosurgery, we will monitor the extent to which use of the Brain Bench can improve their surgical results. This will be carried out in two stages. In the first, the team will perform their surgery using the traditional planning methods, while in parallel making a plan using the Brain Bench Target and trajectory plans will be compared for accuracy and appropriateness with those made by traditional methods; in the case of any difference (for instance, in the choice of a central point), the Brain Bench plan must be considered “at least as good” by reviewers who examine it only with the established software. This will be carried out for a total of 15 patients. Once accuracy has been verified and the choices made with it found satisfactory, increased reliance will be placed on the Brain Bench for surgical plans actually executed. Measurements to compare it with traditional methods will require a study of 50 patients. Specifically, we will measure • time taken to identify the anatomical target. • number of electrode insertions before a target is functionally confirmed • time taken to identify an appropriate trajectory. • time taken to identify alternate targets and trajectories. • the accuracy with which a target volume centre is selected. • a surgeon's evaluation from 0 to 10, for each case, of the improvement in the surgeon's ability to carry out the operation.
9
Appeared in: Visualization in Biomedical Computing 1996, Hamburg, Germany, pp. 491-500. • a surgeon's evaluation from 0 to 10, for each case, of user-friendliness. • a surgeon's evaluation from 0 to 10, for each case, of the convenience in teaching other members to use the system, and to supply relevant information during the procedure.
9 Further Work The Brain Bench promises to be an important tool in neurosurgery time and cost reduction, in both training and clinical practise. Further developments will depend strongly on clinician input, but we anticipate adding features such as: Local adaptive registration: such fitting is now too slow for clinical use on a whole brain, but can be much faster if applied to a subregion of interest and supplied with a good initial linear position adjustment; such selection and adjustment tools become natural to design in the Virtual Workbench. Atlas extension tools, by which structures not easily extracted from scan data can be added and edited by expert neuroanatomists. (In this environment it is easy [7] to hand map and edit 3D curves within a volume data set; work continues on intuitive hand shaping of surfaces and solids.) With such tools we will extend the basic electronic brain atlas to major blood vessels and cranial nerve pathways, and improve the existing account of the white matter `cabling' within and from the `processor' grey matter of the cortex; now sketchily indicated by dotted lines within the planes of the atlas pages, suppressing the aspect of between-plane connectivity. Brain structures in the atlas, will be coded for sensitivity to particular probe types (a fine probe being in general less damaging). The system can then not merely report contact, but issue proximity warnings. Frameless stereotaxis presents closely related planning problems, and the Brain Bench can straightforwardly extend to providing numerical guidance to such systems as well as settings for frame positions. Planning stereotactic radiotherapy (targeted X or gamma radiation with sub-millimetric precision) can similarly benefit from combining sensitivity-coded brain atlas guidance with dextrous, intuitive control.
10 References [1]
B. Davey, R. Comeau, C. Gabe, A. Olivier & T. Peters, Interactive Stereoscopic Image-Guided Neurosurgery, SPIE vol. 2164, 1994, pp.167--176. [2] J. C. Goble, K. Hinkley, R. Pausch, J. W. Snell & N. F. Kassell, Two-handed Spatial Interface Tools for Neurosurgical Planning, IEEE Computer, July 1995, pp. 20--26. [3] K. H. Höhne, M. Bomans, M. Riemer, R. Schubert, U. Tiede, W. Lierse, A 3D anatomical atlas based on a volume model, IEEE Comput Graphics Appl. 12, 4 (1992), pp. 72--78. [4] 4. W. L. Nowinski, R. N. Bryan & R. Raghavan, eds., The Electronic Clinical Brain Atlas. Three-Dimensional Navigation of the Human Brain. Thieme, NY, 1996. [5] W. L. Nowinski, A. Fang, B. T. Nguyen, J. K. Raphel, L. Jagannathan, R. Raghavan, R. N. Bryan & G. Miller, Multiple brain atlas database and atlas-based neuroimaging system, Journal of Image Guided Surgery (in press). [6] M. Ono, S. Kubic & C. D. Abernathey, Atlas of the Cerebral Sulci. Georg Thieme Verlag, Stuttgart 1990. [7] T. Poston & L. Serra, Dextrous Virtual Work, Communications of the ACM, May 1996, 29:5, pp. 37-45. [8] T. Poston, L. Serra, M. Solaiyappan & P. A. Heng, The Graphics Demands of Virtual Medicine, Computers & Graphics, vol. 20 (1996), pp. 61--68. [9] G. Schaltenbrand & W. Wahren, Atlas of Stereotaxy of the Human Brain. Georg Thieme Verlag, Stuttgart 1977. [10] J. Talairach & P. Tournoux, Co-Planar Stereotactic Atlas of the Human Brain. Georg Thieme Verlag, Stuttgart 1988. [11] J. Talairach & P. Tournoux, Referentially Oriented Cerebral MRI Anatomy. Atlas of Stereotaxic Anatomical Correlations for Gray and White Matter. Georg Thieme Verlag, Stuttgart 1993.
10
