EXTRACTING CORN GEOMETRIC STRUCTURAL PARAMETERS USING KINECT Yiming Chen 1, Wuming Zhang1,*, Kai Yan1, Xiaowen Li2 ,Guoqing zhou1 Beijing Normal University, 200875 Beijing, China –
[email protected],
[email protected] 2 Guilin University of Technology, 541004 Guilin, Guangxi, China –
[email protected] 1
ABSTRACT In remote sensing and agriculture, corn is a common crop which is often studied. In both cases, it is important to measure the geometric structural parameters such as Leaf Area Index (LAI) and Leaf Angle Distribution (LAD). They are useful indicators that affect corn growth. Kinect is a sensor that can be used to get the distance between the object and Kinect itself. It costs little but offers high accuracy. We use Kinect to obtain point clouds of the corn and build a 3D model of the leaves in order to measure structural parameters. The current results show the proposed method is feasible. But more efforts should be made to improve the automation and practically of this method. Index Terms—Kinect, Depth Data, Corn, Geometric Structural Parameters, 3D reconstruction 1. INTRODUCTION Geometrical structural parameters of Plants such as Leaf Area Index (LAI) and Leaf Angle Distribution (LAD) are of great importance in ecology and biology. They control the solar radiation transmission through canopies. Therefore, these parameters are important indicators that affect plant growth. Although, with the development of remote sensing techniques, we can use different ways to measure these parameters, the results usually require ground-based validation. However, in the validation, the acquirement of structural parameters is a difficult task. All of the existing methods for measuring LAI and LAD on the ground can be categorized into direct methods and indirect methods [1]. The direct method gives the credible values of LAI and LAD, but it usually requires destructive measurement and is time-consuming and labor-intensive. The indirect method uses radioactive transfer model or geometric optical model such as Li-Strahler model [2]. This kind of method does no harm to the plants, but needs to be validated by the direct method. There is another kind of indirect method to extracts structural parameters from a 3D model. Considering the main ways to generate a 3D model, images and point clouds can be used in this method. In our previous work, we used *
photos from different angles to extract these parameters [3]. In recent years, LiDAR(Light Detection and Ranging) is widely used to generate a 3D model and obtain these parameters since it can get 3D coordinates directly. It is simpler and offers high accuracy. However, the LiDAR system is still very expensive to private organizations and not easy to carry. Kinect is another sensor that can be used to build a 3D model. It is a motion sensing input device by Microsoft for Xbox 360 and enables users to control the Xbox 360 without touching a game controller. It can capture depth images and obtain 3D points in order to generate 3D models. Since Kinect is cheaper and has a small size, it is useful in close-range digitalization. The corn is selected as study case for the following reasons. First of all, it is the most prevalent crop in many parts of China. In this project, effective measurement of corn leaf area and leaf direction is requested for the planned field work. Second, corn is a plant with moderately complex shapes, providing a test case that will neither result in overly optimistic accuracy, nor prove too complex for methodological development. Third, the leaves are curved and cannot be represented by a simple plane. Moreover, the appearance of each leaf is similar, so it is hard for a human being to distinguish between them. However, corn has some distinct features which make it possible to be reconstructed [4]. There are two main parts of a corn model: the stem and leaves. The reconstruction of the stem is relatively easier, as it can be modeled using several cylinders. Leaf shape is determined by its boundary and central vein, so the difficulty of corn modeling mainly lies in reconstructing these key boundary curves. So it is a challenge to reconstruct a 3D model of corn leaves. 2. MATERIALS AND METHODS 2.1. Data Collection 2.1.1 Kinect data The Kinect data was acquired April 24th, 2012. The study Area is an individual corn that grows in Zhangye, Gansu Province. The raw data contains two parts: the RGB image of the corn, whose resolution is 640*480 and the depth
Corresponding author.
978-1-4673-1159-5/12/$31.00 ©2012 IEEE
6673
IGARSS 2012
image, with resolution of 320*240. The data is used to reconstruct the 3D corn leaf model and obtain LAI and LAD. 2.1.2 Field measurements
Kinect is a short range measuring sensor, and is sensitive to the light. Thus, we remove an individual corn to the inside and put it closer to Kinect (0.5-3m available). Figure 2 and Figure 3shows the RGB and depth image that is taken from nadir view of the corn.
Filed measurements are the geometrical structural parameters of a single corn, including Leaf Area Index (LAI) and Leaf Angle Distribution (LAD). These parameters are measured using rulers and compasses, and they are used to provided accuracy information for structure parameters measurements obtained from Kinect sensor. 2.2. Corn Leaf Reconstruction Strategy In this study, we use the image captured by Kinect sensor of a single view to obtain point clouds of a corn leaf and reconstruct it with computer vision method. The work flow of modeling an individual corn leaf is described as follows: 1. Kinect sensors are used to capture color images and depth images of the corn. 2. A point cloud is obtained by the depth image. 3. Segment the point cloud of an individual leaf, and reconstruct the 3D corn model. 4. Based on the 3D corn model, LAI and LAD of individual corn plants can be calculated. Figure 1 illustrates the work flow of the proposed method for corn 3Dreconstruction.
Figure 2. RGB image of the corn
Figure 3. Depth image of the corn Since the depth contains the distance between every pixel to the sensor, the point cloud can be obtained with Windows SDK and C#. In this study, we only retrieve the points which are 0.6 to 1.2m from the sensor. Then the cloud of every corn leaf can be easily segmented.
Figure 1. Flow chart of Kinect-based 3D corn reconstruction Figure 4. Point cloud of the corn
2.3. Point Cloud Extraction Algorithm
6674
Table 1. LAI results using direct and Kinect method
2.4. 3D Leaf Reconstruction and Structural Parameters Acquisition
Number
The 3D leaf reconstruction algorithm is based on computer vision. The leaves of the corn are complex to describe by a single line or curve. In our work, TIN(Triangular Irregular Network) is used to construct the 3D model. With the help of Visual C++ and OpenGL, the point cloud can be shown and triangularization. The result of leaf 3D reconstruction has been shown in Figure 5. As we compute all the triangles in the point cloud, we can sum all these triangles’ areas to get the total leaf area. Moreover, we can also know each normal vector of an individual leaf. Then the total LAD is easy to get.
1 2 3 4 5
taped-measured LAI/cm2 32.5 54 85 76.3 28
Kinect-measured LAI/cm2 34.4 46.6 60.4 75.8 25.2
Table 2. LAD results using Kinect method of an individual leaf LAD Leaf area 0-10 9.28 10-20 0.34 20-30 0.51 30-40 12.78 40-50 13.39 50-60 13.52 60-70 8.14 70-80 4.18 80-90 6.17 3.2. Automation and Practicality At present, the segmentation of the point cloud still needs human computer interaction. We should pay more attention on improve the automation and pragmatically of this method. The edges of leaves also need to be automatically extracted.
Figure 5. 3D Leaf Reconstruction
4. CONCLUSION AND FUTURE WORKS
3. RESULTS AND DISCUSSIONS 3.1. Current Results In our study, we choose a early-spring corn to reconstruct 3D model. Since only a one-view-image is obtained, the LAI and LAD results are partly underestimated. If we regard the tape-measure method as the accurate result, the LAI values measured by the 3D corn model based-on Kinect sensor were evaluated. LAD is not easy to be retrieved by destructive method, so it is not evaluated. Table 1 and Table 2 shows the LAI and LAD results. From the tables, the Leaf Area is underestimated. The reasons are illustrated as follows: 1. The image is obtained from only one view. Some layers of the corn may be sheltered. 2. The light of the sensor shoot at the boundary of the leaf cannot be obtained from Kinect. 3. Not all triangles should be taken into calculation.
This paper presents a method using Kinect to obtain a 3D corn model from the depth image. Then the geometric structural parameters such as LAI and LAD can be easily measured on the 3D model. This method has some advantages. Comparing with the image-based reconstruction, Kinect can obtain compact 3D point clouds directly. It is also much cheaper than LiDAR and other similar systems. And since the only data we need is depth image, the plant would not be destroyed. Furthermore, the error of this method is acceptable. The method in this paper needs to be further investigated. In particular, a more accurate boundary of a leaf should be found by using both RGB and depth image. The accuracy of the parameters should be thoroughly evaluated. More samples should be used, in order to estimate the accuracy of calculate LAI and LAD.
6675
5. ACKNOWLEDGEMENTS
This work is supported by National Natural Science Foundation of China Grant No. 41171265 and 40801131, and the National High-Tech Research and Development Plan of China Grant No. 2010AA122202. This work is also supported by major project of State Key Laboratory of Remote Sensing Science of China and open foundation of Guangxi Key Laboratory of Spatial Information and Geomatics Grant No. 1103108-04. 6. REFERENCES [1] H. P. White and E. R. Young, "Comparison of in situ LAI retrieval of two instruments of four mature agricultural crops," Geomatics Canada, 2007. [2] X. W. Li and A. H. Strahler, "Geometric-Optical Modeling of a Conifer Forest Canopy," IEEE Transaction on Geoscience and Remote Sensing, vol. GE-23, pp. 705-721, 1985. [3] H. X. Wang, W. M. Zhang, G. Q. Zhou, G. J. Yan, and N. Clinton, "Image-based 3D corn reconstruction for retrieval of geometrical structural parameters," International Journal of Remote Sensing, vol. 30, pp. 5505-5513, 2009. [4] C. Fournier and B. Andrieu, "A 3D architectural and process-based model of maize development," Annals of Botany, vol. 1, pp. 695-702, 1998. [5] X. Y. Guo, C. J. Zhao, B. X. Xiao, S. L. Lu, and C. F. Li, "Study on maize leaf morphological modeling and mesh simplification of surface," Computer and Computing Technologies in Agriculture, vol. 1, pp. 695-702, 2008.
6676
