A method for 3D reconstruction of tree crown volume from ... - CiteSeerX

Tree Physiology 25, 1229–1242 © 2005 Heron Publishing—Victoria, Canada

A method for 3D reconstruction of tree crown volume from photographs: assessment with 3D-digitized plants J. PHATTARALERPHONG1–3 and H. SINOQUET1 1

UMR PIAF INRA-UBP, Site de Crouelle, 234 Avenue du Brézet, 63039 Clermont-Ferrand Cedex 2, France

2

Department of Botany, Faculty of Science, Kasetsart University, Bangkok, 10900, Thailand

3

Corresponding author ([email protected])

Received October 5, 2004; accepted February 18, 2005; published online August 1, 2005

Summary We developed a method for reconstructing tree crown volume from a set of eight photographs taken from the N, S, E, W, NE, NW, SE and SW. This photographic method of reconstruction includes three steps. First, canopy height and diameter are estimated from each image from the location of the topmost, rightmost and leftmost vegetated pixel; second, a rectangular bounding box around the tree is constructed from canopy dimensions derived in Step 1, and the bounding box is divided into an array of voxels; and third, each tree image is divided into a set of picture zones. The gap fraction of each picture zone is calculated from image processing. A vegetated picture zone corresponds to a gap fraction of less than 1. Each picture zone corresponds to a beam direction from the camera to the target tree, the equation of which is computed from the zone location on the picture and the camera parameters. For each vegetated picture zone, the ray-box intersection algorithm (Glassner 1989) is used to compute the sequence of voxels intersected by the beam. After processing all vegetated zones, voxels that have not been intersected by any beam are presumed to be empty and are removed from the bounding box. The estimation of crown volume can be refined by combining several photographs from different view angles. The method has been implemented in a software package called Tree Analyzer written in C++. The photographic method was tested with three-dimensional (3D) digitized plants of walnut, peach, mango and olive. The 3D-digitized plants were used to estimate crown volume directly and generate virtual perspective photographs with POV-Ray Version 3.5 (Persistence of Vision Development Team). The locations and view angles of the camera were manually controlled by input parameters. Good agreement between measured data and values inferred from the photographic method were found for canopy height, diameter and volume. The effects of voxel size, size of picture zoning, location of camera and number of pictures were also examined. Keywords: crown dimension, gap fraction, image processing, perspective image, Tree Analyzer.

Introduction The spatial distribution of leaf area determines resource capture and canopy exchanges with the atmosphere. Measuring the spatial distribution of leaf area is generally tedious and time consuming, even when three-dimensional (3D) digitizing techniques are employed (Lang 1973, Sinoquet et al. 1991, Sinoquet and Rivet 1997, Takenaka et al. 1998). Many tree models, e.g., light models, therefore abstract individual canopies as a volume filled with leaf area. Simple shapes like ellipsoids or frustrums have been extensively used to model tree shape (e.g., Norman and Welles 1983, Oker-Blom and Kellomaki 1983). More sophisticated parametric envelopes have been proposed by Cescatti (1997) to extend the range of modeled canopy shapes, and non-parametric envelopes like polygonal envelopes are expected to fit any tree shape (Cluzeau et al. 1995). However, Nelson (1997) and Boudon (2004) showed that different shape models for the same tree may lead to large differences in crown volume. Moreover, because of the fractal nature of plants (Prusinkiewicz and Lindenmayer 1990), the definition of crown volume is rather subjective (Zeide and Pfeifer 1991, Nilson 1992) as it depends on the way space unoccupied by phytoelements is classified, namely as canopy space or outer space (Fuchs and Stanhill 1980). The estimation of crown volume therefore depends on scale (Nelson 1997). Several field methods have been proposed for estimating crown volumes. When simple parametric envelopes are used, tree height and diameter can be determined from dendrometric measurements, although Brown et al. (2000) used fisheye photographs to estimate tree crown size. To estimate the non-parametric envelope of crown volume, Giuliani et al. (2000) monitored the shadow cast by the tree crown with an array of light sensors at the ground surface, and used tomography techniques to infer the 3D volume from 2D projections of the crown shadow. Photographs can also be used to reconstruct the 3D volume of an object by computer vision techniques such as voxel coloring (Seitz and Dyer 1997), space carving (Kutulakos and Seitz 2000) and visual hull (Laurentini 1999). The photographic method was first developed for solid object with well-defined opaque contours, but some work was devoted to

1230

PHATTARALERPHONG AND SINOQUET

tree canopies, e.g., Shlyakhter et al. (2001) and Reche et al. (2004). Shlyakhter et al. (2001) computed crown volume from tree photographs by the silhouette method, which is based on the visual hull technique. The silhouette area seen on each photograph is used to compute a solid angle made by the tree viewed from the camera location; this is a cone in which crown volume is included. Crown volume is thus estimated as the intersection of the cones provided by a set of photographs. Reche et al. (2004) reconstructed crown volume from a set of voxels that were considered semi-transparent. The opacity of each voxel was solved using information on pixel color. Neither of these methods for tree crown volume estimation has been evaluated by comparison with direct measurements. Moreover, neither method accounts for the fractal nature of plants, because only one value of crown volume is computed (i.e., at the observation scale) and changes in crown volume with measurement scale are ignored. In this study, we describe a photographic method for estimating individual tree dimensions and crown volume. In this method, the canopy space is described as an array of 3D cubic cells (e.g., Kimes and Kirchner 1983, Reche et al. 2004). In computer graphics jargon, the 3D cells are called voxels—a nickname for “volume element” or “volume pixel.” Crown volume is defined as the volume of the set of voxels containing phytoelements. Changing voxel size allows one to explore the scale-dependence of crown volume. The method was tested with 3D-digitized plants, i.e., plants for which the location, orientation and size of all leaves were recorded by a 3D-digitizing technique (Sinoquet et al. 1998). The 3D-digitized data sets allowed us to (1) synthesize plant images with graphics software mimicking any camera; and (2) compute the actual crown volume at any scale, to assess the quality of the proposed photographic method. Materials and methods The photographic method is based on a set of digital photographs of a tree (e.g., eight images taken from N, S, E, W, NE, NW, SE and SW). Photographs must be taken so that image processing allows classification of pixels as vegetation or

background, i.e., to develop a binary image as in fisheye photographic methods (e.g., Frazer et al. 2001, Mizoue and Inoue 2001). In addition to photographs, the method involves geometric parameters associated with each photograph; namely, the distance between the camera and the tree trunk Dc, camera height H c, camera elevation β c, camera azimuth α c around the tree and focal length f. The use of digital cameras requires a calibration procedure to convert focal length to view angle (see Appendix 1). Computation from binary photographs includes three steps: (1) estimation of tree size; (2) construction of a 3D array of voxels; and (3) removal of empty voxels from the array. The method has been implemented as a software package written in C++.Net 2003 (Microsoft, Redmond, WA) and is called Tree Analyzer. Estimation of tree size For each image, canopy height and diameter are estimated from the topmost, rightmost and leftmost vegetated pixels as shown in Figure 1. A canopy plane (Pt ) is defined as the vertical plane including the base of the tree trunk and facing the camera; the normal vector of the canopy plane has the same azimuth α c as the camera. Each pixel in the image corresponds to a line originating from the camera location in 3D space. The equation of the line of each pixel is computed from the camera parameters and the location of the pixel on the image, as a function of the focal length ( f ) of the camera (see Appendix 2). The 3D position of the intersected point between the line and the canopy plane is then calculated by a ray-plane intersection algorithm (Glassner 1989). Tree height is computed as the height of the intersected point of the topmost pixel in the canopy plane. Similarly, crown height and diameter are inferred from the difference between the projections on the canopy plane of the topmost and bottommost pixels, and the rightmost and leftmost pixels of a tree crown, respectively. A set of values is computed for each photograph. Construction of a 3D array of voxels The origin of the system is the tree trunk at ground level. A

Figure 1. Estimation of tree dimension from an image. Canopy height and diameter are estimated from the intersection point of the beam line (of the topmost, rightmost and leftmost vegetated pixels) and the canopy plane (Pt ), where Pt is the vertical plane including the tree base and facing the camera.

TREE PHYSIOLOGY VOLUME 25, 2005

3D RECONSTRUCTION OF CROWN VOLUME FROM PHOTOGRAPHS

1231

Figure 2. Construction of a voxel array: (A) construction of the rectangular bounding box; (B) the bounding box must be larger than the real canopy; and (C) division into a voxel array.

rectangular bounding box is constructed around the tree with the canopy dimensions derived from the previous stage (Figures 2A and 2B). The highest values found for tree height and crown diameter are used to ensure that all of the tree is included in the box. Then the bounding box is divided into an array of voxels (Figure 2C). Voxel size along the x-, y- and z-axes (dx, dy, dz) is user-defined. Each voxel is defined by the coordinates (xv, yv, zv) of its point of origin. The division process starts from the origin of the system (0,0,0). The first voxel is centered on the point of origin. Other voxels are created until the border of the bounding box is reached. Removing empty voxels from the array Each tree photograph is divided into a set of picture zones, the size of which is user-defined (e.g., 10 × 10 pixels). Each zone is associated with a beam originating from the camera location and passing through the center of each picture zone: the smaller the picture zone, the higher the density of beams in the picture. Gap fraction is computed for each zone as the proportion of white (i.e., background) pixels. For each vegetated zone, i.e., where the gap fraction is < 1, the beam line equation is computed for the pixel in the zone center as described in Appendix 2. Then the ray-box intersection algorithm (Glassner 1989) is used to compute the list of voxels intersected by the beam line. After the beam line equations for all vegetated pic-

ture zones have been computed, the voxels that have not been intersected by any beam are assumed to be empty and are removed from the bounding box. This process is iterated for each photograph. After processing a set of photographs, the crown volume is estimated as the volume of the remaining voxels (Figure 3). Software output also includes the list of remaining voxels as a VegeSTAR Version 3.0 file (Adam et al. 2002), allowing further visualization of the tree canopy shape. Testing the method Digitized trees Three dimensional digitized trees were used to assess the quality of the photographic method. A 2-year-old mango tree (Mangifera indica L. cv. Nam Nok Mai), a 1-year-old olive tree (Olea europaea cv. Manzanillo) and a 3-year-old hybrid walnut tree (Juglans NG38 × RA) were 3D-digitized at the leaf scale, as described by Sinoquet et al. (1998), in November 1997, August 1998 and December 1999, respectively. The mango tree was grown on a commercial farm in Ban Bung, 150 km southeast of Bangkok, and the olive tree was grown in Pathum Thani, 40 km north of Bangkok, Thailand. The walnut tree was grown on an experimental plot at the INRA in Clermont-Ferrand, France. For each tree, the location and orientation of each leaf was recorded with a magnetic digitizer (Fastrak 3Space, Polhemus, VT), and the length and width of each leaf was measured with a ruler. A sample of leaves was

Figure 3. Visualization of the reconstruction process using a set of images. The process starts from the bounding box and iterates by using each image. The arrow shows the camera direction.

TREE PHYSIOLOGY ONLINE at http://heronpublishing.com

1232

PHATTARALERPHONG AND SINOQUET

harvested on similar trees to establish an allometric relationship between individual leaf area and the product of leaf length and width. The area of each sampled leaf was measured with a Li-Cor 3100 leaf area meter. Thus, the data sets consisted of a collection of leaves, the size, orientation and location of which were measured in the field. A 4-year-old peach tree (Prunus persica cv. August Red) at the CTIFL Center, Nîmes, France, was digitized in May 2001 at the current-year shoot scale, 1 month after bud break. Given the large number of leaves (~14,000), digitizing at the leaf scale was impossible. The magnetic digitizing device was used to record the spatial coordinates of the bottom and top of each leafy shoot for the reconstruction of leaves on the shoot. Thirty shoots were digitized at the leaf scale to derive leaf angle distribution and allometric relationships between numbers of leaves, shoot leaf area and shoot length. Leaves of each shoot were then generated from (1) allometric relationships, (2) sampling of leaf angle distribution, and (3) the additional assumptions of constant internode length and leaf size within a shoot (Sonohat et al. 2004). Computation of actual crown volume The actual crown dimensions and volume of the 3D-digitized trees were computed from the 3D-digitizing data using the Tree Box software. The canopy space was divided into an array of voxels, by using the same bounding box and voxel definition as in the Tree Analyzer software. For each leaf in the canopy, spatial coordinates of seven points (six points on the leaf margin plus the leaf

center point) were computed. Voxels containing at least one leaf point were classified as vegetated voxels. Because of the fractal nature of plants, defining tree volume is rather subjective (Nilson 1992, Farque et al. 2001). For this reason, six types of crown volumes were defined: (1) comprising vegetated voxels only; (2) including empty voxels making a closed cavity within the crown; (3) including empty voxels located between vegetated voxels along the three directions of the 3D space. Volume Definitions 1, 2 and 3 lead to the same external canopy volume (Figure 4A), but differ according to the presence or absence of internal (invisible) voxels, giving rise to the following definitions; (4) including empty margin voxels to remove concavity in each horizontal layer (Figure 4B); (5) including empty margin voxels to remove concavity in each vertical stack (Figure 4C); and (6) comprising simply the bounding box of the canopy (Figure 4D). Synthesis of plant photographs Virtual undistorted photographs of the 3D-digitized plants were synthesized with the freeware software package POV-Ray Version 3.5 (Persistence of Vision Development Team, www.povray.org), as described by Sinoquet et al. (1998), to synthesize orthographic images of the digitized plants. In this experiment, perspective images were used to generate photograph-like images. This requires the calibration parameter (k) of the camera, which accounts for the relation between metric unit and pixel unit in the image at different focal lengths (see Appendix 1). Focal length and camera calibration parameters were therefore used to calculate the

Figure 4. Six types of crown volume defined by the 3D-digitizing data set and computed with Tree Box software using a voxel size of 20 cm. (A) Crown volume Definition: (1) vegetated voxels only; (2) addition of empty voxels making a closed cavity within the crown); and (3) addition of empty voxels located in between vegetated voxels along all three spatial dimensions. Although similar, they differ in the presence or absence of internal (invisible) voxels. (B) Crown volume Definition: (4) addition of empty margin voxels to remove concavity in each horizontal layer. (C) Crown volume Definition: (5) addition of empty margin voxels to remove concavity in each vertical stack. (D) Crown volume Definition: (6) bounding box of the canopy.

TREE PHYSIOLOGY VOLUME 25, 2005

3D RECONSTRUCTION OF CROWN VOLUME FROM PHOTOGRAPHS

1233

view angle of the camera by POV-Ray (see Appendix 2). Here we used the calibration parameter of a Fuji Finepix1400Z camera. Black and white perspective images with a size of 640 × 480 pixels were synthesized. Spatial location, orientation angles and focal length of the camera were simulated in POVRay software. The camera was pointed to the central axis of the tree. Camera distance was set to about twice canopy height. Elevation and focal length were set so that the entire canopy was included in the image. Image output files were stored as bitmap files.

Table 1. Sets of photographs to test the effect of the number of pictures on canopy volume estimates.

Sensitivity analysis

100

Effect of voxel size The effect of voxel size on estimated crown volume was tested on the 3D-digitized mango, olive, peach and walnut trees. One hundred virtual photographs of each plant were synthesized from the 3D-digitizing data sets with the POV-Ray software: 46 virtual photographs were taken from a set of evenly distributed sky directions (i.e., according to the Turtle sky discretization proposed by den Dulk 1989); 46 photographs were taken from directions opposite to the 46 sky directions (i.e., virtual photographs from belowground); and eight photographs were taken from the main horizontal directions (N, S, E, W, NE, SE, NW and SW). Such a set of photographs could not be used in real experiments or for practical application of the method, not only because of the large number of images, but also because it is, in reality, impossible to photograph from belowground. However, this set of images allowed a theoretical evaluation of the photographic method. Size of picture zoning was set to 3 × 3 pixels and camera distance was set to twice canopy height. To compute the fractal dimension of the tree crown, crown volume V as a function of voxel size dx was fitted with the power law: V = adx b. According to the counting-box method used to derive the fractal dimension (Falconer 1990), exponent b is related to the fractal dimension d: d = 3 – b.

No. of images 3 4 6 8 9 24 54

Directions (East = 0°, North = 90°) 3 horizontal directions (0, 120 and 240) 4 horizontal directions (N, S, W and E) 6 horizontal directions (0, 60, 120, 180, 240 and 300) 8 horizontal directions (N, S, W, E, NE, NW, SE and SW) 8 horizontal directions + top image 16 Turtle sky (den Dulk 1989) + 8 horizontal directions 46 directions of turtle sky (den Dulk 1989) + 8 horizontal directions 54 directions + 46 opposite directions of turtle sky

height and focal length was set so that the whole tree could be imaged. Effect of size of picture zoning Photographs of the walnut tree taken from 100 directions were used. Camera distance from the canopy was set at twice canopy height. The size of picture zoning was varied between 1 × 1 to the maximum size allowed by voxel size. For the sake of consistency, the upper limit to picture zone size was defined so that the projection of the picture zone onto the canopy plane was kept smaller than the voxel size. Crown volume was computed by setting voxel size to 10, 20 and 40 cm. Effect of camera distance Eight virtual photographs, synthesized from the 3D-digitizing data set with the POV-Ray software, of the mango, olive, peach and walnut trees taken from the main horizontal directions (N, S, E, W, NE, SE, NW and SW) were used. Horizontal distances to the tree base varied from 1 to 5 times tree height to test the effect of camera distance on estimated crown volume. Voxel size was set at 20 cm, with size of the picture zone equal to 3 × 3 pixels. Results

Effect of number of pictures Seven sets of photographs were used (Table 1). The larger set included 100 images, as described above. Other sets included images taken in the horizontal directions and from above the canopy, according to the Turtle sky discretization in 46 or 16 directions (den Dulk 1989). The number of photographs in the other sets ranged from 54 to 3. Camera distance was set at about twice canopy

Canopy structure Table 2 shows the large variations in canopy structure parameters among the 3D-digitized trees. Number of leaves ranged from 1558 for mango to 14,260 for peach; leaf size ranged from 1.52 cm2 for olive to 47.2 cm 2 for walnut; total leaf area ranged from 0.83 m2 for olive to 28.11 m 2 for peach; whereas

Table 2. Canopy structure parameters of 3D-digitized plants. Plants

Height (m)

Diameter (m) 1

No. leaves

Mean leaf area (cm 2 )

Total leaf area (m 2 )

Bounding box volume (m 3 )

Mango Olive Peach Walnut

1.7 2.3 2.5 2.8

1.7 1.4 3.0 1.8

1636 5490 14260 1558

39.58 1.52 19.64 47.2 2

0.83 6.48 28.11 7.35

3.1 3.0 22.2 8.2

1 2

Mean diameter from N–S and E–W. Mean leaflet area.

TREE PHYSIOLOGY ONLINE at http://heronpublishing.com

1234

PHATTARALERPHONG AND SINOQUET

bounding box volume ranged among species from 3 to 22 m 3. Synthesized side views of the 3D-digitized trees showed that canopy shape differed with species: it approximated to a sphere for mango, a cylinder for walnut, a frustrum for peach and an asymmetric shape for olive (Figure 5). Estimation of canopy dimension from tree photographs The maximum, minimum and mean values of estimated tree height and crown diameter computed from the set of 100 images taken around the tree showed good correlations to measured values obtained from the digitized data: r 2 = 0.58, 0.91 and 0.98 for tree height and r 2 = 0.99, 0.99 and 0.98 for crown diameter, respectively. The photographic method slightly overestimated mean value of tree height and crown diameter (Figures 6A and 6B). The minimum value was always underestimated, whereas the maximum value was always overestimated (Figures 6A and 6B). Because of smaller errors related to perspective, values computed from eight photographs taken in horizontal directions showed higher correlations with the measured data (Figures 6C and 6D). Again maximum values for tree height and diameter were slightly higher than the values estimated from the 3D-digitized data. Maximum values obtained by the photographic method were therefore used to build the tree canopy bounding boxes. Estimation of crown volume from tree photographs Canopy shape and volume, as inferred from the photographic method, strongly depended on voxel size (Figure 7, for the walnut tree). A smaller voxel size (i.e., 5 cm) allowed better fitting of the canopy outlines, so that the reconstructed canopy more closely approximated the 3D-digitized plant. As a result

of the fractal nature of plants, crown volume—estimated from 3D-digitized data and by the photographic method—increased with voxel size (Figure 8). For voxel sizes ranging from 10 to 40 cm, crown volume estimated from a set of 100 photographs was close to the values computed from the 3D-digitized data. Regression analysis for all canopy volume estimates made with voxels of 10–40 cm showed an r 2 of 0.99. For voxels greater than 40 cm, discrepancies between the two crown volume estimation methods emerged, and the discrepancies generally increased with voxel size. With voxels between 10 and 60 cm, crown volume was closely related to voxel size by a power law, because the coefficients of determination (r 2 ) were between 0.965 and 0.998, which demonstrates the fractal behavior of the tree canopies. The fractal dimension, as derived from the exponent of the power regression analysis between voxel size and canopy volume, was about 2.2 for all trees, but the olive tree showed a smaller value of 1.88. As a result of the good correlation between crown volumes computed from Tree Box and Tree Analyzer, regression analysis showed a good agreement between fractal dimensions estimated by the two methods, with r 2 = 0.94. The values of fractal dimension computed from the photographic method were, however, slightly higher (+4%, data not shown) than the values obtained from the 3D-digitized data. Figure 9 shows crown volumes of all tree canopies at a given voxel size of 20 cm, from direct estimation from the 3D-digitizing data based on six possible volume definitions, and from the photographic method based on various sets of photographs. For all tree canopies, direct estimation of crown volume showed small variations, except for the canopy bounding box (volume Definition 6), which was 2.5 to 4 times the crown

Figure 5. Virtual images of the trees viewed from the horizontal direction, synthesized from the 3D-digitizing data set with the POV-Ray software: (A) mango tree; (B) peach tree; (C) walnut tree; and (D) olive tree.

TREE PHYSIOLOGY VOLUME 25, 2005

3D RECONSTRUCTION OF CROWN VOLUME FROM PHOTOGRAPHS

1235

Figure 6. Comparison between tree dimensions as measured from the 3D-digitizing data set and estimated from the photographic method. (A) Tree height from a set of 100 photographs; (B) crown diameter from a set of 100 photographs; (C) tree height from a set of eight photographs taken in the horizontal directions; and (D) crown diameter from a set of eight photographs taken in the horizontal directions. Measured crown diameter is a mean value from N–S and E–W directions.

volume estimated from the vegetated voxels only. Including internal empty voxels (volume Definitions 2 and 3) slightly increased crown volume (0.2–0.6%); including external empty voxels by filling concavities in the horizontal plane (volume Definition 4) and along the vertical direction (volume Definition 5) also increased volume estimation slightly (4–12%), except for the peach canopy, where the volume difference was greater (35%).

Crown volume estimated by the photographic method decreased with increasing number of photographs (Figure 9). As shown in Figure 8, crown volume estimated from 100 photographs closely approximated the direct estimate based on vegetated voxels only. Use of only eight photographs made in the horizontal direction—a convenient approach for field applications—led to a slight increase in crown volume (13–31%) compared with the direct estimate of crown volume based on

Figure 7. Visualization of the walnut tree canopy as computed from a set of 100 photographs using picture zoning 3 × 3 pixels at a range of voxel sizes, and comparison with the image synthesized from the 3D-digitizing data.

TREE PHYSIOLOGY ONLINE at http://heronpublishing.com

1236

PHATTARALERPHONG AND SINOQUET

Figure 8. Crown volume as a function of voxel size dx: comparison between the photographic method (× = Tree Analyzer, using a set of 100 photographs, picture zoning 3 × 3 pixels) and direct estimation (䊐 = Tree Box, Definition 1, i.e., computation from the vegetated voxels only).

vegetated voxels, but was quite similar to crown volume based on Definition 5 (Figure 10). By using another set of eight horizontal photographs, namely with different camera azimuth angles, variations in estimated volume were small (Figure 10). Use of less than eight photographs led to a large overestimation of crown volume (Figure 9) and large variation in crown volume estimates (Figure 10), even when based on the direct volume Definition 5. The largest overestimation of crown volume by the photographic method was found with the

peach tree; this could be related to crown concavity at the crown apex as a result of goblet training (see Figure 5). In the range of 1 × 1 to 5 × 5 pixels, the picture zone size (i.e., density of beam sampling) on the photographs had a minor effect on crown volume computations (Figure 11A). Greater picture zone size (from 10 × 10 pixels) led to underestimates of crown volume. In the range of 1 × 1 to 5 × 5 pixels, computation time was markedly influenced by picture zone size, whereas computation time was unaffected by picture

Figure 9. Comparison of crown volumes computed by direct estimation (six volume definitions computed from the 3D-digitizing data sets with software Tree Box) and by the photographic method using different sets of photographs and picture zoning 3 × 3 pixels. Volume unity is crown volume computed from the vegetated voxels only (Definition 1).

TREE PHYSIOLOGY VOLUME 25, 2005

3D RECONSTRUCTION OF CROWN VOLUME FROM PHOTOGRAPHS

1237

Figure 10. Comparison between crown volumes computed from the 3D-digitizing data sets (Definition 5) and by the photographic method, for different numbers of photographs, using a voxel size of 20 cm. The error bars show the standard deviation of crown volume from three different sets of images. The images were synthesized by setting horizontal camera elevation and camera distance at twice canopy height. Camera height (1.2–1.5 m) and focal length (7–9 mm) were set so that the entire canopy was included in the image.

zone size greater than 5 × 5 pixels (Figure 11B). As a result, setting the picture zone size at 3 × 3 pixels provided a good estimate of crown volume within a reasonable computation time. For all trees, the effect of camera distance on crown volume estimation was small; the distance was in the range of one to five times canopy height (Figure 12). Compared with estimates of crown volume based on the direct volume (Definition 5), estimated crown volume was slightly greater when camera distance was set equal to canopy height (1% in peach to 11% in walnut) and was minimal when set at two or three times canopy height for all trees.

Discussion In this study, we described and evaluated a photographic method to estimate the crown volume of isolated tree canopies. At a given scale, space occupied by the tree canopy has been defined by parametric shapes (e.g., Norman and Welles 1983, Cescatti 1997) or convex envelopes (Cluzeau et al. 1995). Here we have defined crown volume as the volume of voxels classified as canopy space, where voxels were regarded as either empty (gap fraction = 1) or vegetated (gap fraction < 1), and the transparency information was not used further. This kind of binary information is suitable for volume computation; however, it is unsuitable if the aim is to compute vegetation density within the voxels. The first step in the photographic method is to estimate plant size in order to define a bounding box. The principle is similar to that adopted when using a dendrometric clinometer to determine tree height and crown diameter. Tree size values averaged from the set of photographs closely approximated the

Figure 11. Effect of picture zone size on crown volume (A) and computation time (B) for a walnut tree. Volume was computed from a set of 100 photographs, for different voxel sizes. Maximum picture zone size was defined so that the image of the picture zone on the canopy plane is smaller than voxel size. The computations were done on a personal computer with CPU Intel Pentium III 1.06 GHz.

value computed from the 3D-digitized data set (Figure 6). For volume computation, the bounding box was built from maximum values found for tree height and diameter, thus ensuring that the whole tree crown was included within the bounding box. Finally, crown volume was computed from iterative erosion of the bounding box, according to plant silhouettes provided by the photographs. This procedure differs from, and is simpler than, other photographic methods. For example, Shlyakhter et al. (2001) computed the intersection of solid angles defined by plant silhouettes from camera location, whereas Reche et al. (2004) used a method derived from medical tomography. We tested our photographic method quantitatively by comparing crown volume computed from photographs with crown volume derived from the 3D-digitizing data set. This may be regarded as a virtual experiment because it allowed us to as-

TREE PHYSIOLOGY ONLINE at http://heronpublishing.com

1238

PHATTARALERPHONG AND SINOQUET

Figure 12. Estimation of walnut tree crown volume from the photographic method with a set of eight photographs (N, S, E, W, NE, SE, NW and SW), with camera distance directly related to canopy height (picture zoning 3 × 3 pixels, voxel size 20 cm).

sess the photographic method, but avoided the constraints associated with field experiments. This test helped define the optimal configuration of the photographic method in the field. The choice of view points and the number of photographs influence the accuracy of 3D-reconstructed objects (Laurentini 1996, 1997) and these factors depend on the shape or structure of the object. With a large number of pictures, the photographic method gave an accurate estimate of the smallest crown volume. Using more photos led to more accurate estimates of crown volume, owing to the algorithm of progressive erosion of the bounding box. Although a set of 100 photographs per tree is unsuitable for field applications because of the time needed for setting up the experiment and image processing, such a large set was useful in demonstrating the overall suitability of the method. Previous studies used 14–22 tree photographs (Shlyatkhter et al. 2001, Reche et al. 2004). We found that a set of eight photographs taken in the main horizontal directions allowed computation of crown volume when internal empty voxels and some external ones (i.e., Definition 5) were included. The use of eight photographs appears to be a good compromise between accuracy and practical applicability (Figure 10). In the study by Shlyakhter et al. (2001), the envelope of the plant silhouette seen in each photograph was approximated by a polyline, at an arbitrary scale, whereas, like us, Reche et al. (2004) used a voxel method, but with very small voxels. In our method, voxel size can be varied, so that the method provides information on the fractal behavior of individual tree crowns, e.g., by the box counting method (Falconer 1990), or the twosurface method (Zeide and Pfeifer 1991). Similar results, including the estimate of fractal dimension, were also found with the direct and the photographic methods (Figure 8). This could be used to further study the fractal behavior of leaf canopies, which might be useful in certain studies, e.g., in assessing light capture properties (Fouroutan-Pour et al. 2001, Mizoue 2001) and animal size distribution in vegetation canopies (Morse et al. 1985). We performed a sensitivity analysis to identify the optimal

configuration for field application and algorithm parameterization. A satisfactory comparison between crown volume estimated by the direct and photographic methods was found for dense picture zoning. Because estimation of crown volume was relatively insensitive to camera distance from the tree, our method could be used in open orchards where tree spacing and tree height are about the same. We did not test the effect of image resolution because we used virtual photographs synthesized by POV-Ray software. The photographic method was tested with undistorted, computer-generated photograph-like images synthesized by POVRay. Actual photographs may be distorted, depending on the characteristics of the camera. For the calibration of actual cameras (see Appendix 1), we proposed a linear parameter estimation method (Heikkila and Silven 1997) based on the direct linear transformation method (DLT) originally developed by Abdel-Aziz and Karara (1971). The calibration method does not explicitly include image distortion; however, the calibration procedure uses several photographs taken along the focal range, so that image distortion is, in part, implicitly taken into account. As it shows high r 2 coefficients (Table A1.1), this approximate calibration method should be adequate for field application. For greater accuracy, Tsai’s calibration algorithms (Tsai 1987; used by Reche et al. 2004) could be applied, although Tsai’s method is more complicated and involves more parameters (e.g., radial distortion and uncertainty). In addition, modern zoom lens do not work exactly as assumed in Tsai’s algorithms (Tapper et al. 2002). We did not test our photographic method in the field; therefore, there may be additional difficulties related to the measurements of camera parameters and photographic processing under field conditions. A digital compass and clinometer can be used to control camera angles. Camera location can be monitored with (laser) distance meters and a level. Photographs must allow background separation. Although pixel separation methods for digital images are available (Mizoue and Inoue 2001), a uniform background is desirable (Reche et at. 2004). This can be achieved by using red cloth as a background (e.g., Andrieu and Sinoquet 1993). Windy conditions could also be limiting due to plant movements introducing noise in the location of phytoelements seen on the different photographs. In conclusion, we have described a fast and nondestructive photographic method, implemented in the Tree Analyzer software, for estimating crown volumes of isolated trees. Estimates of crown volume made by the photographic method were compared with values computed directly from 3D-digitized plants. Satisfactory estimates of crown volume were obtained based on a set of eight photographs taken around the tree in the main horizontal directions. Field application will require that the user is able to separate tree vegetated pixels from the picture background (Mizoue and Inoue 2001), as in processing fisheye photographs of crown projected area (e.g., Frazer et al. 2001). Further development of the photographic method will include estimation of leaf area and leaf distribution within the crown volume based on inversion methods (Lang and Yueqin 1986, Chen and Cihlar 1995).

TREE PHYSIOLOGY VOLUME 25, 2005

3D RECONSTRUCTION OF CROWN VOLUME FROM PHOTOGRAPHS Acknowledgments The authors are grateful to D. Combes (INRA-Lusignan, France) for the walnut tree 3D-digitizing data and for assistance in acquiring the peach tree digitizing data, to P. Kasemsap, S. Thanisawanyangkura and N. Musigamart (Kasetsart University, Bangkok, Thailand) for assistance with acquisition of the mango and olive tree digitizing data, and to the POV-Ray Team who provided POV-Ray freeware and its documentation. Acquisition of the peach tree digitizing data was supported by project “Production Fruitière Intégrée” funded by INRA. Peach trees were made available by CTIFL, Balandran.

References Abdel-Aziz, Y.I. and H.M. Karara. 1971. Direct linear transformation into object space coordinates in close range photogrammetry. Proc. Symp. on Close-Range Photogrammetry, Urbana, IL, pp 1–18. Adam, B., N. Donès and H. Sinoquet. 2002. VegeSTAR: software to compute light interception and canopy photosynthesis from images of 3D digitised plants. Version 3.0. UMR PIAF INRA-UBP, Clermont-Ferrand, France. Andrieu, B, and H. Sinoquet. 1993. Evaluation of structure description requirements for predicting gap fraction of vegetation canopies. Agric. For. Meteorol. 65:207–227. Boudon, F. 2004. Représentation géométrique multi-échelles de l’architecture des plantes. Ph.D. Thesis, Université Montpellier II, Montpellier, France, 176 p. Brown, P.L., D. Doley and R.J. Keenan. 2000. Estimating tree crown dimensions using digital analysis of vertical photographs. Agric. For. Meteorol. 100:199–212. Cescatti, A. 1997. Modelling the radiative transfer in discontinuous canopies of asymmetric crowns. I. Model structure and algorithms. Ecol. Model. 101:263–274. Chen, J.M. and J. Cihlar. 1995. Plant canopy gap-size analysis theory for improving optical measurements of leaf-area index. Appl. Optics 34:6211–6222. Cluzeau, C., J.L. Dupouey and B. Courbaud. 1995. Polyhedral representation of crown shape. A geometric tool for growth modelling. Ann. Sci. For. 52:297–306. den Dulk, J.A. 1989. The interpretation of remote sensing, a feasibility study. Ph.D. Thesis, Wageningen Agricultural Univ., Wageningen, The Netherlands, 173 p. Falconer, K. 1990. Fractal geometry: mathematical foundation and applications. John Wiley and Sons, Chichester, UK, 337 p. Farque, L., H. Sinoquet and F. Colin. 2001. Canopy structure and light interception in Quercus petraea seedlings in relation to light regime and plant density. Tree Physiol. 21:1257–1267. Foroutan-Pour, K., P. Dutilleul and D.L. Smith. 2001. Inclusion of the fractal dimension of leafless plant structure in the Beer-Lambert law. Agron. J. 93:333–338. Frazer, G.W., R.A. Fournier, J.A. Trofymow and R.J. Hall. 2001. A comparison of digital and film fisheye photography for analysis of forest canopy structure and gap light transmission. Agric. For. Meteorol. 109:249–263. Fuchs, M. and G. Stanhill. 1980. Row structure and foliage geometry as determinants of the interception of light rays in a sorghum row canopy. Plant Cell Environ. 3:175–182. Giuliani, R., E. Magnanini, C. Fragassa and F. Nerozzi. 2000. Ground monitoring the light–shadow windows of a tree canopy to yield canopy light interception and morphological traits. Plant Cell Environ. 23:783–796. Glassner, A.S. 1989. An introduction to ray tracing. Morgan Kaufmann Publishers, San Francisco, CA, 119 p.

1239

Heikkila, J. and O. Silven. 1997. A four-step camera calibration procedure with implicit image correction. Proc. of the 1997 Conference on Computer Vision and Pattern Recognition. IEEE Comput. Soc., Washington, DC, pp 1106–1112. Kimes, D.S. and J.A. Kirchner. 1983. Diurnal variation of vegetation canopy structure. Int. J. Remote Sens. 4:257–271. Kutulakos, K.N. and S.M. Seitz. 2000. A theory of shape by space carving. Int. J. Comput. Vision 38:199–218. Lang, A.R.G. 1973. Leaf orientation of a cotton plant. Agric. Meteorol. 11:37–51. Lang, A.R.G. and X. Yueqin. 1986. Estimation of leaf area index from transmission of direct sunlight in discontinuous canopies. Agric. For. Meteorol. 37:229–243. Laurentini, A. 1996. Surface reconstruction accuracy for active volume intersection. Pattern Recogn. Lett. 17:1285–1292. Laurentini, A. 1997. How many 2D silhouettes does it take to reconstruct a 3D object? Computer Vision and Image Understanding 67: 81–87. Laurentini, A. 1999. Computing the visual hull of solids of revolution. Pattern Recogn. 32:377–388. Mizoue, N. 2001. Fractal analysis of tree crown images in relation to crown transparency. J. For. Plan. 7:79–87. Mizoue, N. and A. Inoue. 2001. Automatic thresholding of tree crown images. Jpn. J. For. Plan. 6:75–80. Morse, R., J. Lawton, M. Dodson and M. Williamson. 1985. Fractal dimension of vegetation and the distribution of arthropod body lengths. Nature 314:731–732. Nelson, R. 1997. Modeling forest canopy heights: the effects of canopy shape. Remote Sens. Environ. 60:327–334. Nilson, T. 1992. Radiative transfer in nonhomogeneous plant canopies. Adv. Bioclimatol. 1:59–88. Norman, J.M. and J.M. Welles. 1983. Radiative transfer in an array of canopies. Agron. J. 75:481–488. Oker-Blom, P. and S. Kellomaki. 1983. Effect of grouping of foliage on the within-stand and within-crown light regime. Comparison of random and grouping canopy models. Agric. Meteorol. 28: 143–155. Prusinkiewicz, P. and A. Lindenmayer. 1990. The algorithmic beauty of plants. Springer-Verlag, New York, 228 p. Reche A., I. Martin and G. Drettakis. 2004. Volumetric reconstruction and interactive rendering of trees from photographs. ACM Transactions on Graphics (SIGGRAPH Conference Proceedings) 23: 1–10. Seitz, S.M. and C.R. Dyer. 1997. Photorealistic scene reconstruction by Voxel Coloring. Proc. IEEE CVPR, pp 1067–1073. Shlyakhter, I., M. Rozenoer, J. Dorsey and S. Teller. 2001. Reconstructing 3D tree model from instrumented photographs. IEEE Comput. Graph. Appl. 21:53–61. Sinoquet, H. and P. Rivet. 1997. Measurement and visualization of the architecture of an adult tree based on a three-dimensional digitising device. Trees 11:265–270. Sinoquet, H., B. Moulia and R. Bonhomme. 1991. Estimating the 3D geometry of a maize crop as an input of radiation models: comparison between 3D digitizing and plant profiles. Agric. For. Meteorol. 55:233–249. Sinoquet, H., S. Thanisawanyangkura, H. Mabrouk and P. Kasemsap. 1998. Characterisation of light interception in canopies using 3D digitising and image processing. Ann. Bot. 82:203–212. Sonohat, G., H. Sinoquet, V. Kulandaivelu, D. Combes and F. Lescourret. 2004. Three-dimensional reconstruction of partially 3D digitised peach tree canopies. In Proc.: 4th Int. Workshop on Functional–Structural Plant Models. Eds. C. Godin, J. Hanan, W. Kurth, A. Lacointe, A. Takenaka, P. Prusinkiewicz, T. DeJong, C. Bever-

TREE PHYSIOLOGY ONLINE at http://heronpublishing.com

1240

PHATTARALERPHONG AND SINOQUET

idge and B. Andrieu. UMR AMAP, Montpellier, France, pp 6–8. Takenaka, A., Y. Inui and A. Osawa. 1998. Measurement of three-dimensional structure of plants with a simple device and estimation of light capture of individual leaves. Funct. Ecol. 12:159–165. Tapper, M., P.J. McKerrow and J. Abrantes. 2002. Problems encountered in the implementation of Tsai’s algorithm for camera calibration. Proc. 2002 Australasian Conference on Robotics and Automation, ARAA, Auckland, pp 66–70.

Tsai, R.Y. 1987. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-self TV cameras and lenses. IEEE J. Robotics Autom. 3:323–344. Zeide, B. and P. Pfeifer. 1991. A method for estimation of fractal dimension of tree crowns. For. Sci. 37:1253–1265.

Appendix 1

where L is the length of the image diagonal. Note that L and D can be expressed in both metric (subscript m) and pixel (subscript p) units. In Equation A1.2, k is the unknown to be inferred, values of f and L p both change according to zooming, and D is a constant defined by the experimental layout. In digital cameras, the value of f is stored as an image property in the image file and can be displayed with any imaging software. For each image, the length of the image diagonal in metric units, L m, can be computed from the length of the photographed line, both in metric and pixel units, i.e., l m and l p, respectively, and the length of image diagonal L p in pixel units:

Derivation of calibration parameter for digital cameras In this study, the calibration parameter (k) of the camera is needed to compute the beam line equation associated with each pixel of the photograph (see Appendix 2), and to compute the view angle of the virtual camera used in POV-Ray software to synthesize photograph-like images. The view angle γc of the photograph is defined as the angle made by the diagonal of the picture. It depends on the camera model (i.e., type of lens) and focal length f (Figure A1.1). The calibration parameter k is the diagonal length of the projected image onto the receptor and has the same unit as f (usually mm). The receptor is either the film in classical cameras or a CCD array (charge-coupled device) in digital cameras. The relationship between γc, f and k is: k γ  tan c  =  2  2f

(A1.1)

Here we propose a method to derive k from a set of pictures of the same object taken at a range of focal lengths. The camera is assumed to be a pinhole camera (Figure A1.1) and image distortion due to lens properties is neglected. The object is usually a horizontal line of known length (l) drawn on a vertical plane. The camera is located at the same level as the object at a fixed distance D from the vertical plane, where D is chosen so that the line is entirely viewed on the image when using maximum zooming (D is about 2–3 m for l = 50 cm). From geometrical considerations: k L = f D

(A1.2)

l  L m = L p  m   lp 

(A1.3)

In Equation A1.3, l p can be derived from the pixel location (xp, yp) on the x and y axes defining the image plane, whereas L p can be derived from image resolution. Finally, the calibration parameter k is inferred from Equation A1.2 as the slope of the regression line between variables Dm /L m and f. Figure A1.2 shows the regression line for the Minolta DiMAGE 7i digital camera, and Table A1.1 gives the values of parameter k for various camera types. The high r 2 coefficient found in the regression analysis used to derive the calibration parameter shows that the calibration procedure is valid, even though image distortion is neglected (Table A1.1). The value of k changes markedly with camera type, from 6.5 to 10.9 mm for the Canon PowerShot A75 and Minolta Dimage 7, respectively. In contrast, k values of two cameras of the same type (here Canon PowerShot75 and NikonCoolPix885) show only small variations.

Figure A1.1. Simple camera model (pinhole camera) showing the relationship between view angle (γ c ) of the camera, focal length ( f ), camera distance (D) and size of the image projected onto the camera receptor (k).

TREE PHYSIOLOGY VOLUME 25, 2005

3D RECONSTRUCTION OF CROWN VOLUME FROM PHOTOGRAPHS

1241

cation; and u is a unit vector defining the direction associated with each pixel: u = [ a , b, c], with a 2 + b2 + c 2 = 1

(A2.2)

λ is scalar distance from the beam origin to the point. The beam line equation is defined by vectors C and u, which are known. For a given image, C is fixed whereas u changes according to pixel location in the image. Computation of camera location (C)

Figure A1.2 Relationship between focal length ( f ) and variable Dm /L m for the Minolta DiMAGE 7i digital camera. The calibration parameter (k = 10.931) is computed as the slope of the regression line.

For each photograph, information about camera location and orientation has to be recorded by the operator: camera height (Hc), horizontal distance from tree base (Dc), azimuth (α c), elevation (βc) and rolling (θc). Then C is derived as: C = (Dc cos( α c ), Dc sin( α c ), H c )

(A2.3)

Calculation of unit vector (u) Appendix 2 Derivation of the beam line equation associated with a pixel on the photograph Each pixel on the photograph is associated with a beam line originating from the camera location. The line equation depends on camera parameters and on pixel location on the image. The line equation is needed to compute the list of voxels associated with a pixel, i.e., crossed by the beam line. The origin of the system is located at the tree base. The axis X+ points to the East, axis Y+ points to the North and axis Z+ points upward. The camera is located at C and points to Z+. Image plane (Pi ) is the back projection of the image at a distance equal to focal length ( f ) perpendicular to camera view direction (Figure A2.1). The equation of the beam line can be written: r = C + λu

(A2.1)

where: r is any point (x, y, z) on the beam line; C is camera lo-

We can derive u from the spatial coordinates of two points: P1(x1, y1, z1) and P2(x2, y2, z2):

u=

P1 − P2 where λ = ( x1 − x 2 )2 + ( y1 − y 2 )2 + ( z1 − z2 )2 λ (A2.4)

Here, P1 is camera location and P2 represents the spatial coordinates of a given pixel. To make the calculation simpler, the reference origin is translated to camera location. Thus P1 = (0, 0, 0) and Equation A2.4 becomes: u = − P2 / λ where λ =

x 22 + y 22 + z22

(A2.5)

Derivation of u reduces to the calculation of P2 for any pixel (xp, yp) as follows: (1) transformation of 2D coordinates (xp, yp) into 3D coordinates (xi, yi, zi); and (2) rotation of the 3D coordinates according to camera Euler angles.

Table A1.1. Calibration parameter (k) of several camera models. Camera model

Maximum resolution

Focal length (mm)

View angle (°)

k (mm)

r2

Canon PowerShot A75 Canon PowerShot A75 Casio QV-3500EX Epson PhotoPC 3100Z Fuji FinePix1400Z Minolta DiMAGE 7i Nikon CoolPix4500 Nikon CoolPix885 Nikon CoolPix885 Nikon E995 Olympus C-2020Z Sony DSC-P50

2048 × 1536 2048 × 1536 2544 × 1904 2048 × 1536 1280 × 960 2560 × 1920 2272 × 1704 2048 × 1536 2048 × 1536 2048 × 1536 1600 × 1200 1600 × 1200

5.41–13.4 5.41–13.4 7–21 7–20.7 6–18 7.2–50.8 7.85–32 8–24 8–24 8–32 6.5–19.5 6.4–19.2

27.5–62.4 27.4–62.2 26.1–66.3 24.4–65.2 21.7–59.8 12.3–74.4 16.1–60.0 20.9–57.9 21.1–58.5 15.7–57.8 22.6–61.9 19.2–53.8

6.5598 6.5295 9.3196 8.9623 6.903 10.931 9.0602 8.8532 8.9577 8.8481 7.8036 6.4985

0.9976 0.9964 0.9851 0.9994 0.9917 0.9983 0.9995 0.9844 0.9894 0.9998 0.9970 0.9995

TREE PHYSIOLOGY ONLINE at http://heronpublishing.com

1242

PHATTARALERPHONG AND SINOQUET

Figure A2.1. Reference axes and camera angles used to derive the beam line equation from pixel location in the image.

Transformation of 2D (xp, yp) coordinates into 3D coordinates (xi, yi, zi) The image plane is first assumed vertical at a focal length distance f from camera location on the x-axis. After Appendix A1, focal length fp in pixel units is: fp =

fm L p k

 1 0 0   Rx = 0 cos( θ c ) – sin( θ c )  0 sin( θ c ) cos( θ c ) 

(A2.6)

where fm is focal length in metric unit, L p is image diagonal in pixels, and k is the calibration parameter of the camera. For each pixel (xp, yp) counted from the top left corner (i.e., standard coordinates in bitmap images) with image resolution of wp by h p pixels, 3D coordinate (xi, yi, zi) of pixel location in the image plane is:   xi   f p      y i  =  x p − ( wp / 2 )  z i  ( h / 2 ) − y  p   p

camera elevation (β c); and (3) rotation around z-axis (R z) due to camera azimuth (α c):

cos(β c ) 0 – sin(β c )   Ry =  0 1 0   sin(β c ) 0 cos(β c ) 

(A2.8)

cos( α c ) – sin( α c ) 0   R z =  sin( α c ) cos( α c ) 0  0 0 1 

(A2.7) Finally P2 can be written:

Rotation of (xi, yi, zi) according to camera Euler angles The effect of camera orientation on 3D pixel coordinates is accounted for by applying three rotation matrices according to the Euler angles of the camera: (1) rotation around x-axis (Rx) due to camera rolling (θc); (2) rotation around y-axis (Ry) due to

 xi    P2 = Rz R y Rx  y i   z i  and u is computed from P2 with Equation A2.5.

TREE PHYSIOLOGY VOLUME 25, 2005

(A2.9)