Journal of Scientific & Industrial WEE et Research al: SURFACE RENDERING OF 3D ULTRASOUND IMAGES USING VTK Vol. 70, June 2011, pp. 421-426
421
Surface rendering of three dimensional ultrasound images using VTK Lai Khin Wee1,2*, Hum Yan Chai1 and Eko Supriyanto1 1
Department of Clinical Science and Engineering, Faculty of Health Science and Biomedical Engineering, Universiti Teknologi Malaysia, Skudai, 81310 Johor, Malaysia 2 Biomedical Engineering Group, Institute of Biomedical Engineering and Informatics, Faculty of Computer Science and Automation, Technische Universitat Ilmenau, POB 100565 98684 Ilmenau, Germany Received 11 January 2011; revised 27 April 2011; accepted 28 April 2011 This study utilized VTK visualization tool to demonstrate reconstruction of medical image using surface rendering technique. Simulation result shows three dimensional surface rendering of multiple slices ultrasound fetal phantom images in orthogonal view, upper left and upper right for reconstructed coronal and sagittal images and also internal view by changing position. A three dimensional reconstruction using surface rendering via VTK library is accomplished and it is found effective in terms of cost and performance in medical image. Keywords: Medical image, Reconstruction, Surface rendering, Three dimensional, Visualization toolkit (VTK)
Introduction Three-dimensional (3D) medical image (MI) reconstruction and visualization has great significance in diagnosis of medical, surgical planning and simulation, surgery, prosthetic surgery, radiation planning and anatomy. A number of studies1-3 on computer visualization of 3D reconstruction [Marching Cube (MC) algorithm, Marching Tet rahedra (MT) algorithm, and Delaunay tetrahedron] has been carried out. From two types (surface rendering and volume rendering) of 3D reconstruction data field, classical method in surface rendering algorithm is MC algorithm, which detects position of triangle vertices by linear interpolation, using a large number of triangles to represent isosurface, and then reconstructs 3D triangle mesh surface. VTK4 - 6 (Visualization Toolkit), an object-oriented, powerful visualization and graphics, image processing toolkit, providing more than 300 C++ classes, has provided required 3D visualization technology for MI processing and analysis. It is an open source cross-platform MI processing and analysis library that can process DICOM file, render 3D image that encapsulates MC algorithm and image processing7,8 algorithms. Among surface reconstruction algorithms, boundary contour method is a simple, consuming only relatively *Author for correspondence E-mail:
[email protected];
[email protected]
small space and initial version of surface rendering technique, characterized by decomposition of data into slices in order to obtain closed contours of each slice, and then render surface by using the combination of contour between slices. Among 3D reconstruction methods in MI, classic representative is MC algorithm. Basic idea of MC algorithm9-11 is to form a cube located between two slices on 2x2x2, total 8 neighbouring voxels. All different cubes must include object surface, where object surface in different cutting side of cube will be obtained using interpolation method12,13. Finally, all slices join into triangle by following certain rules to represent outer and inner object on cube surface through density gradient computing to get surface direction. This algorithm can be realised easily and is commonly used. However, method drawbacks include result will contain a lot of scattered triangle facets, formation centre layer and some ambiguity cases in data structure. This study applied VTK and MC algorithm together to construct a crossed scalable platform for 3D reconstruction of MIs. Experimental Section In 3D image processing, OpenGL is an efficient 3D mechanism and relatively easy to be implemented.. As 3D models and visualization 14 is getting widely implemented in scientific computing applications and intense development of object-oriented programming,
422
J SCI IND RES VOL 70 JUNE 2011
vtkSource vtkFilter vtkMapper vtkProperty vtkActor
vtkCamera
vtkRenderer Fig. 1—Basic flow of VTK pipeline mechanism
there is a need to have a simple, object-oriented features and easy-to-transplant 3D graphics and extendable visualization function module library. Therefore, VTK was developed for this reason and it is strictly designed according to object-oriented concept, and in order to take into account simplicity of compiled programming and interpreted programming. Also, to consider compatibility and ease of maintenance and upgrades, VTK uses standard programming language development, and visualization algorithms, which is commonly used to encapsulate so that user need only to set interface parameters to implement algorithm. Basic Process of Reconstruction
In VTK image processing15-20, VTK image data is classified into a number of categories (structural point, line grid, structured grid, unstructured points, polygon data and unstructured grids). Different filter will be allocated for each type of structure for processing. However, most of the basic flow of process is similar (Fig. 1). vtkObject is basic class in VTK library, which provides basic visualised method flow. Source data is used to generate data layer. vtkSource is a derived class from vtkObject and is parent class of vtkFilter, which is a derived class from vtkSource. vtkFilter implements data processing on raw data through filters, converting it into module that can be directly processed using an algorithm model. vtkMapper is also a derived class of vtkObject; it will undergo different filtering process and be converted into geometric data as interface in connecting original data and image data. Mapper can achieve a mapping from data to graphics. Actor plays role of realisation in visual using Mapper results to show resultant image. Actor can manipulate image display properties through vtkproperty to make shown image realistic. Next step is to display an image on computer screen; this task is accomplished
by Render & Render Window. Render Window Interactor is used to achieve interoperability between image and user, and by controlling this method, image can be rotated by mouse to see images from all angles. VTK21 is also designed to use in pipeline mechanism; it can process almost all structure type of data, and provides a various type of data conversion or processing for a number of corresponding classes. In present study, vtkSource class object creates a data source; then, vtkFilter class or its derived class processes data source object. It follows by vtkMapper class or its derived classes mapped pixel data, and then vtkActor class or a derived class represents a drawing entity. Finally,f: image is rendered by vtkRenderer. By using VTK for 3D reconstruction developments, main aim is to design appropriate lines, and set parameters to meet program needs. Surface Rendering
Basic principle of MC algorithm is to construct 3D data field and find out isosurface voxels. Position of triangle vertices with equivalent large number of triangles face will be determined through linear interpolation method. In functional processes of MC 3D reconstruction (Fig. 2), program will read DICOM image file format in series, and then it will cut pictures into a collection of small cubes from volume element. MC algorithm will access each cube to generate equivalent collection of triangles. Unfortunately, standard MC algorithm, used to generate an excessive number of triangles, causes followup treatment become a bottleneck, so simplification of triangular grid appears essentially vital. Generation of Small Cubes
Input of MC algorithm is a collection of small cubes, so it must split whole image into collection of small cubes set in advance before MC algorithm execution. Before edge cutting, length of cube and image spacing need to be set. Normally, CT image acquisition thickness is 1 ~ 10 mm, and default value is set to 8. In present ultrasound images, spacing value is set to 3.57. Collected DICOM file is stored in 8 bit, and digital unsigned characteristic with gray scale value (0 - 255). Maximum numbers of stored frames (256) are arranged in advance according to priority. Scanned object in present studies is fetal phantom, with resolution 641 x 598 using transabdominal 3.5 MHz ultrasound transducer (Fig. 3). There is a trade off for appropriate selection between quality of reconstruction and time needed. Layer spacing
423
WEE et al: SURFACE RENDERING OF 3D ULTRASOUND IMAGES USING VTK
VTK Sources Data (DICOM Slices)
VTK Contour Filter
Call Back Command
VTK Poly Data Normal’s
(a)
(b)
Fig. 3—Fetal phantom: a) For ultrasound scanning; and b) Snapshot used for simulation VTK Poly Data Mapper
enhancement will also improve storage structure and reduce storage capacity. This study has taken merger of vertex and vertex index reduction methods. VTK Camera
VTK Actor
Renderer
Merge of Triangle Vertices VTK Render Window Interactor
Exit Fig. 2—MC three dimensional reconstruction processes
and grid side length, measured in pixels, can be set into standard size conversion in the shell. Overall volume element cutting processes are described as follows: 1) All known tomography images need to be arranged in accordance with intervals and order of priority by assuming that scan interval is small enough, this arrangement can be seen as a 3D entity; 2) Define needs for speed and accuracy of 3D reconstructions, a reasonable choice for cube X and Y interval (Z interval is gap interval) and coordinates with 8 groups of a cube must be chosen; and 3) Obtain coordinates of all vertices of cube, and store in the array. It will serve as MC algorithm input data. MC Algorithm Enhancement
Standard MC algorithm generates excessive number of triangles, including a large number22,23 of redundant information, resulting in occupying expensive storage space. Also, too long time for reconstructed image makes difficulty to achieve real-time interactive display. This requires simplification of triangular grid to produce a significant reduction in number of triangles. This
Vertex merging algorithm24,25 is to reduce generated triangle number. Basic principle of vertex merging algorithm is to replace representative of verticals, which can be combined to form a new vertex in next round of consolidation. If every combination of three vertices of a triangle is possible, it is known as triangle contraction. Besides, if a time merges two shares vertices of an edge, then it will be called edge contraction. In this study, method of edge contraction of vertex edge is applied. When two vertices satisfy the conditions, edges of two vertices will shrink to a point, where triangle vertices are deleted. If conditions are not met, triangle is retained and it will continue to determine next vertex of a triangle situation. Assuming that two vertices of triangle Vi ,Vj , angle between vector α(V i ,V j ), a minimum angle constraint MAC (M in. Angle Constraint) is set as (1) where Ni ,Nj denote normal vector of vertex Vi ,Vj . If triangle vertices Vi ,Vj denote distance between geometric d (Vi ,Vj ), set a maximum distance constraint MDC, then distance constraint is given as d (Vi ,Vj ) < MDC. Continuity of isosurface is equivalent to a collection of triangles in presence of a large number of common vertices of triangle. Due to the large number of these duplicate vertices, it will increase burden of additional space storage. In order to solve this problem, index of triangle vertex representation was applied to separate each vertex of triangle in vertex array with its index and
424
J SCI IND RES VOL 70 JUNE 2011
Fig. 4—Simulation result of 3D surface rendering, closed view (RHS) shows surface composed by triangles strips
stored into an array of all vertices. Index of triangle vertices can greatly reduce storage space. However it might introduces another problem, where new vertex obtained might already exists in array and causing added time consumption with a rough estimate (1 + n) * (n / 2), where n is number of vertices with repeated vertices. Therefore, in this study, hash map using STL was applied to improve speed of search point, so time complexity is close to 1, reaching ideal point of search speed. Results Medical doctor always need to find out the best sagittal plane during prenatal ultrasound screening, to access certain ultrasound markers (nuchal translucency, nasal bone, or ductus venous) for trisomy 21 assessment. If analyzed image planes are not in appropriate position, ultrasound marker will not be shown, or it might be measured longer or shorter than its real length. Therefore, it is crucial to identify and display the best 2D image plane from 3D model. Fig. 4 shows 3D surface rendering of multiple slices ultrasound fetal phantom images. When whole volume image data are stored in computer’s memory, image reorganization can be achieved interactively with some others external library (vtkBoxWidget), which provide great 3D features like cutting slider (Fig. 5). Besides, Multiplanar Reconstruction (MPR) is sequence images formed from original axial images to display human tissues and organs in coronal, sagittal, axial, and inclined 2D orientation image. When whole volume data are stored in computer’s
a)
b) Fig. 5—(a) Simulation result surface rendering (b) Internal view by changing position of vtkBoxWidget
WEE et al: SURFACE RENDERING OF 3D ULTRASOUND IMAGES USING VTK
425
Fig. 6— Simulation results of multiplanar reconstruction (MPR), Lower left cross-sectional imaging device and lower right orthogonal view, upper left and upper right for restructured coronal and sagittal images
memory image, this reorganization can be achieved interactively (Fig 6). Discussion Variable spacing is key problem in three-dimensional image reconstruction system. MC algorithm generates a triangle mesh, which has a lot of unused nodes that occupy a huge number of memory spaces. Triangle meshes create shared nodes causing problem of duplicating memory storage repetitively. This causes a tremendous impact in image storage and display speed, and this problem further arise when system is to process an increasing number of medical images. For practical utility, these constraints must be resolved. In this study, triangle meshes MC are simplified and storage structures are improved with intent of reducing storage amount and thus improve display speed of reconstructed image. Conclusions A three dimensional reconstruction method is proposed for surface rendering using VTK library. This technique has great advantages in cases of virtual endoscopy and surgery simulation for surface rendering reconstruction for internal organs. It also helps medical personnel to select and analyze appropriate plane of ultrasound images based on three dimensional model and it has a greater significance in practical clinical applications.
Acknowledgements Authors thank Prof Ing Habil, Jens Haueisen and Dr Rer Nat Uwe Graichen for support and guidance throughout the project. Authors also thank DAAD for financial support to the student who conducted project at TU-Ilmenau, Germany. Authors also thank members of BMTI, Technische Universitat Ilmenau, Germany, for ideas and support to the project. References 1
2
3
4
5 6
7 8
Al Bayati A Y, Saeed R K & Hama-Salh F K, The Existence, Uniqueness and error bounds of approximation splines interpolation for solving second-order initial value problems, J Math Stat, 5 (2009) 123-129. Asamwar R S, Bhurchandi K & Gandhi A S, Successive image interpolation using lifting scheme approach, J Comput Sci, 6 (2010) 968-978. Youness E A, Hassan S Z & El-Rewaily Y A, Quadratic interpolation algorithm for minimizing tabulated function, J Math Stat, 4 (2008) 217-221. AbuBaker A A, Qahwaji R S, Aqel M J, Al-Osta H & Saleh M H, Efficient pre-processing of usf and mias mammogram images, J Comput Sci, 3 (2007) 67-75. Alsultanny Y A, Laser beam analysis using image processing, J Comput Sci, 2 (2006) 109-113. Selvan S, Kavitha M, Shenbagadevi S & Suresh S, Feature extraction for characterization of breast lesions in ultrasound echography and elastography, J Comput Sci, 6 (2010) 67-74. Ouadfel S & Batouche M, An efficient ant algorithm for swarmbased image clustering, J Comput Sci, 3 (2007) 162-167. Ali A D, Swami P D & Singhai J, Modified curvelet thresholding algorithm for image denoising, J Comput Sci, 6 (2010) 18-23.
426 9
10
11
12
13
14
15
16
17
18
J SCI IND RES VOL 70 JUNE 2011
Tianqi J, Xue F & Xijuan G, Improvement of marching cubes algorithm based on sign determination, in 2nd Int Workshop on Education Technology & Computer Science (ETCS), vol 1, 2010 194-197. Gong F & Zhao X, Three-dimensional reconstruction of medical image based on improved marching cubes algorithm, in Int Conf on Machine Vision and Human-Machine Interface (MVHI), 2010, 608-611. Lopes A & Brodlie K, Improving the robustness and accuracy of the marching cubes algorithm for isosurfacing, IEEE Trans Visualization & Compu Graphics, 9 (2003) 16-29. ]Bartsch M, Weiland T & Witting M, Generation of 3D isosurfaces by means of the marching cube algorithm, IEEE Trans Magnetics, 32 (1996) 1469-1472. Nico P, Fabio G & Andrea G, Robust segmentation of anatomical structures with 3D active contours and marching cubes in 19th Int Congr Ser, 1281 (Berlin, Germany) 2005, 1278. Carr J C, Fright W R & Beatson R K, Surface interpolation with radial basis functions for medical imaging, IEEE Trans Med Imaging, 16 (1997) 96-107. Wang Q, Zhigeng P, Chen C & Bu J, Surface rendering for parallel slices of contours from medical imaging, Comput Sci & Engg, 9 (2007) 32-37. Jacq J-J & Roux C J, A direct multi-volume rendering method aiming at comparisons of 3D images and models, IEEE Trans Inform Technol Biomed, 1 (1997) 30-43. Weng N, Yang Y.-H & Pierson R, Three-dimensional surface reconstruction using optical flow for medical imaging, IEEE Trans Med Imaging, 16 (1997) 630-641. Wee L K, Arooj A & Supriyanto E, Computerized automatic nasal bone detection based on ultrasound fetal images using
19
20
21
22
23
24
25
cross correlation techniques, in WSEAS Trans on Information Science and Applications, 2010, 1068-1077. Wee L K, Min T Y, Arooj A & Supriyanto E, Nuchal translucency marker detection based on artificial neural network and measurement via bidirectional iteration forward propagation, in WSEAS Trans on Information Science and Applications, 2010, 1025-1036. Wee L K & Supriyanto E, Automatic detection of fetal nasal bone in 2 dimensional ultrasound image using map matching, in 12th WSEAS Int Conf on Automatic Control, Modelling & Simulation (ACMOS), 2010, 305-309. Demangeat J-L, Gries P, Bessekri A & Fellinger F, Surface rendering of three-dimensional myocardial SPECT: Clinical usefulness compared with bull’s-eye and conventional tomograms, J Nucl Cardiol, 4 (1997) 349-357. Dietrich C A, Scheidegger C E, Schreiner J, Comba J L D, Nedel L P & Silva C T, Edge transformations for improving mesh quality of marching cubes, IEEE Trans Visualization & Compu Graphics, 15 (2009) 150-159. Deschamps T, Malladi R & Ravve I, Fast evolution of image manifolds and application to filtering and segmentation in 3D medical images, IEEE Trans Visualization & Compu Graphics, 10 (2004) 525-535. Arata L K, Simplification of triangle meshes for fast surface rendering of tomographic data, in Nuclear Science Symp & Medical Imaging Conf, vol 3, 1994, 1438-1442. Lee B, Tian L-F, Ping C, Mo H-Q & Mao Z-Y, A fast accurate 3D surface reconstruction method of medical image based on modularization, in Proc Int Conf on Machine Learning & Cybernetics, vol 8, 2005, 4942-4945.
