Precision Agric DOI 10.1007/s11119-015-9398-5
Application of the Kinect sensor for dynamic soil surface characterization F. Marinello1 • A. Pezzuolo1 • F. Gasparini1 • J. Arvidsson2 L. Sartori1
•
Ó Springer Science+Business Media New York 2015
Abstract Agricultural soil roughness is pertinent to important agricultural phenomena, such as evaporation, infiltration or compression. Monitoring roughness variations would make possible the improvement of tillage operations. In the present work, implementation of the Microsoft KinectTM RGB-depth camera for dynamic characterization of soil microrelief is proposed and discussed. The metrological performance and the effect of the operating conditions on three-dimensional reconstruction was analyzed considering both laboratory tests on calibrated reference surfaces and field tests on different agricultural soil surfaces. Data set analysis was made on the basis of surface roughness parameters, as defined by ISO 25178 (2012) series: average roughness, root mean square roughness, skewness and kurtosis. Correlation between different tillage conditions and roughness parameters describing soil morphology was finally discussed. Keywords
Surface roughness Dynamic measurement Micro-relief Feedback
Introduction Soil roughness, mainly due to the presence of secondary soil units (such as peds, clods or aggregates), plays an important role in many agricultural phenomena, such as aggregate stability, infiltration, runoff, water stored in depressions, erosion, heat flux and evaporation (Helming et al. 1998; Kamphorst et al. 2000). These processes are essential for soil conservation, affecting germination and growth of seeds, plant available water and
& F. Marinello
[email protected] 1
TeSAF, Dipartimento Territorio e Sistemi Agro-Forestali, University of Padova, Viale dell’Universita` 16, Legnaro, Italy
2
Department of Soil and Environment, Swedish University of Agricultural Sciences, Box 7014, 750 07 Uppsala, Sweden
123
Precision Agric
eventually influencing fertility and productivity (Sandri et al. 1998). On the other hand, soil macro- and micro-topography are continuously and rapidly evolving, due to the combined impact of many factors, such as wind and rainfall erosion, solar heating, vegetation growth and tillage (Gessesse et al. 2010). Large differences in soil roughness are commonly found not only within the same field (macro-roughness), but also locally within a few meters range (micro-roughness). Monitoring cloddiness and roughness variations would make possible the equipping of farm machinery for tillage with an electronic feedback or feed-forward system, as depicted in Fig. 1. In the feed-forward configuration, the system evaluates the state of the soil ahead of the machine and correspondingly modulates the operation of the working parts, e.g. forward speed, rotating speed of the rotors, tillage depth, angle of tines, discs or deflectors, etc. Conversely, in the feed-back configuration, the instrument carries out a real time analysis on the cloddiness of the soil after machine operations and instantly modifies working parameters, in order to enhance soil finishing homogeneity or keep the soil roughness close to a predetermined optimal value. Such an approach is interesting: indeed taking advantage of the recently introduced ISOBUS communication protocol (ISO 117839:2012) which enables quick and simple data transfer between tractors and implements, it can allow optimization of operations in terms of: – machine functioning: the performance can be exploited acting on different parameters such as forward speed, mechanical parts positions, pto speed, etc.; – seedbed preparation: to obtain an optimal aggregate size distribution, with ideally no variations due to differences in soil texture, headlands, etc.; – time: the feedback system slows up the operation only when reference parameters exceed given thresholds; – investments: reduced working time leads to reduction of fuel consumption and wear of mechanical parts; – pollution: avoiding excessive soil pulverization and minimizing power overuse, release of pollutants in the atmosphere can be limited; To allow such a ‘‘dynamic calibration’’ approach, instruments are needed for fast threedimensional characterization of soil roughness. To this end, contact instruments cannot be applied due to their typically slow sampling rate. On the other hand, several non-contact
Fig. 1 Schematic representation of a three-dimensional sensor unit, positioned in front of the tractor for a feed-forward loop or alternatively behind the tractor for a feed-back system
123
Precision Agric
approaches potentially suitable for fast soil sensing have been proposed in the past and in recent years (Viscarra Rossel et al. 2011), mainly based on image analysis techniques (Stafford and Ambler, 1990), laser scanning (Bertuzzi et al. 1990; Darboux and Huang, 2003; Chatzinikos et al. 2013) and photogrammetry (Aguilar et al. 2009). However such techniques suffer from various impediments. Image analysis is not suitable in the case of geometric clod features and may need human intervention for proper segmentation execution (Chimi-Chiadjeu et al. 2014). Laser scanning is a very interesting choice thanks to resolution and fast sampling performances. However, it is better suited to a single profile rather than for whole area analyses and, as costs are still relatively high, it is difficult to justify its implementation in agricultural machines (Draelos et al. 2012). Photogrammetry is also an interesting solution, benefitting from low cost sensors. However, computational time for 3D reconstruction is still an issue. However, in recent years the advent of faster computer processors is pushing the introduction of new inexpensive sensors, which make the possibility of integration with agricultural machines more realistic. In the present work, a suitable low-cost sensor based on 3D depth camera technology for dynamic characterization of soil micro-relief is proposed, discussing possibilities and limitations for agricultural applications. The main objectives were to (1) study the performance of the camera under controlled conditions in the laboratory, and (2) study if differences in soil micro-relief due to tillage could be determined under field conditions.
Materials and methods Depth sensing camera technology The present work takes advantage of depth sensing camera technologies. Such technology is not a new invention but, in the last couple of years, it has raised great interest and widespread diffusion mainly thanks to recent applications in the video games industries. As a consequence, this technology has rapidly reached a mature level and low-cost depth cameras are currently available on the market. For the present study, a Microsoft KinectTM RGB-depth camera was implemented (Zhang 2012), which is already raising significant interest and growing attention for
Fig. 2 a Kinect three-dimensional sensing is based on a triangulation principle. Actual (b) and schematic view (c) of the Kinect sensor: mainly consisting of an infrared (IR) projector, an IR camera and an RGB camera
123
Precision Agric
agricultural applications because of its high quality and low cost (Che´ne´ et al. 2012; Tanke et al. 2012). This sensor includes an infrared laser emitter, an infrared camera and an RGB camera. Depth information is achieved through a triangulation process: a diffraction grating splits the infrared laser into a given pattern which is projected onto the scene. The pattern is collected by the infrared camera and compared with the projected one: local shifts generate a disparity map, where larger shift values correspond to positions further from the sensor and, conversely, lower values correspond to positions closer to the sensor. The sensor was priced around US$150 at launch and lower thereafter. From the experience of the author, a single sensor can be mounted and cabled on a machine for less than US$ 1000. A picture of the sensor is shown in Fig. 2.
Experimental tests The main improvement allowed by the Kinect depth sensing sensor is the possibility of exploiting static and dynamic characterization of three-dimensional surfaces. The performance in terms of range, resolution, linearity, repeatability and maximum detectable slope has already been exhaustively discussed (Marinello et al. 2013). The present work focuses on the ability of the sensor to properly quantify soil roughness. For this reason, investigations were carried out, comprising three set of experiments: – laboratory tests on calibrated reference surfaces – field tests on different agricultural soil surfaces – field tests on a soil surface harrowed at different speeds.
Reference parameters The Kinect sensor allows depth data transfer to a PC or notebook using a USB cable via a USB port. The signal can then be processed taking advantage of different open libraries which are accessible mainly on three platforms: OpenKinect (an open community of developers sharing libraries and projects for Windows, Linux and Mac), OpenNI (an open source standard interface for 3D sensor data processing algorithms) and Microsoft SDK (the Software Development Kit allowing user to code and run specific analysis tools). The latter platform was the one implemented to allow processing of collected data and calculation of reference parameters. In particular for the present work, as a reference for the tests, four parameters were chosen, namely Sa, Sq, Ssk and Sku, as defined by standard ISO 25178-2 (2012). Sa is the arithmetical mean height of the ordinate values within a definition area A, and is defined by (1): ZZ 1 ð1Þ Sa ¼ jzðx; yÞjdxdy A A
where x, y are the co-ordinates in the horizontal axis and z the ordinate in the vertical axis. Sq is the root mean square value of the ordinate values, and is defined by (2): vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u ZZ u1 z2 ðx; yÞdxdy ð2Þ Sq ¼ t A A
123
Precision Agric
Ssk is the skewness of the scale-limited surface and is quantified by the quotient of the mean cube value of the ordinate values and the cube of the root mean square roughness Sq: 2 3 ZZ 1 41 z3 ðx; yÞdx dy5 ð3Þ Ssk ¼ 3 Sq A A
Sku is the kurtosis of the scale-limited surface and is defined as the quotient of the mean quartic value of the ordinate values and the fourth power of the root mean square roughness Sq: 2 3 ZZ 1 41 4 z ðx; yÞdx dy5 ð4Þ Sku ¼ 4 Sq A A
Such parameters not only quantify the main amplitude of surface roughness distribution (Sa and Sq), but also the level of asymmetry (Ssk) and peakedness (Sku) of the distribution itself about its mean. Taken together, such parameters can describe in a comprehensive way, the different aspects of soil condition.
Laboratory tests on calibrated surfaces Laboratory tests had the goal of verifying the actual performance of the sensor in roughness parameters estimation. For the tests, reference surfaces featuring hemispherical elements were chosen. The surface had a density of about 92 hemispheres per square meter, with radii ranging between 25 and 60 mm. The sensor was positioned at a relative height of 1100 mm from the reference surface (i.e. the plane where the hemispheres lie), to capture an area of 992 9 750 mm with a sampling resolution of about 1.5 9 1.5 mm. Each surface was measured 5 times, with different hemisphere distributions.
Field tests with different tilling operations A first set of tests was performed to verify the performance of the sensor when dealing with different tilling operations. All operations were done on the same area situated at the Experimental Farm of the Faculty of Agricultural Science of Padova situated at Legnaro (45°210 N, 11°580 E) in a flat area of the Po Valley. The soil can be defined, according to the USDA, as loamy and containing 46 % sand, 30 % silt and 24 % clay: Table 1 gives the main physico-chemical characteristics. Table 1 Main physico-chemical characteristics of the soil in the trial
Sand (%)
47
Silt (%)
40
Clay (%)
13
Total CaCO3 (%)
24
Organic matter (%)
1.0
Total nitrogen (%)
0.1
C/N ratio
7.5
pH (H2O)
8.0
Soil moisture (w/w) (%)
16
123
Precision Agric
Measurements were carried out with the same scanning parameters reported for the laboratory tests, after four different tillage operations: plough, harrow, vibro-tiller and rotarytiller. The sensor was then positioned at an average height from the soil of 1100 mm, measured with reference to the tractor wheels bottom surface allowing mapping of an area of about 0.74 m2/frame. Continuous scanning allowed mapping of a larger area by using stitching procedures (Marinello et al. 2007). A detailed view of the operations is given in Table 2.
Field tests with different tillage speeds A set of field tests was also carried out on the soil tilled at different speeds. The soil was harrowed as in the first set of tests, but at six different speeds ranging between 1.1 and 7.3 km/h. Three measurements were taken for each treated portion by means of the Kinect sensor, for a total of 18 measurements.
Results and discussion Laboratory tests on calibrated surfaces A three-dimensional rendering of one of the reference samples, as revealed by the Kinect sensor, is shown in Fig. 3. Results from laboratory tests are summarized in the four graphs of Fig. 4. The first two graphs refer to measured average roughness (Sa) and root mean square roughness (Sq), compared with corresponding reference values. The graphs show the sensor non-linearities. In particular, they show that the sensor in general overestimates relative heights especially in the range 0–15 mm for Sa and 0–25 mm for Sq, corresponding to hemispheres with a radius smaller than 40 mm. Conversely for the same reference surfaces, skewness and kurtosis analyses exhibit a slight underestimation, most probably due to root mean square roughness overestimation. However, it is worth noting that such behavior is very repeatable: indeed repeated measurements on a range of different surfaces produced small standard deviations, which allowed for calibration and correction of the sensor.
Table 2 Details for soil tillage operations considered during field tests Machine
Furrow reversible plough with cylindrical mouldboard
Working width (mm) 900
Working depth (mm) 350–400
Folding power harrows with cage roller
3600
250
Seedbed cultivator (vibro-tiller)
3000
250
Rotary tiller with levelling bonnet
2600
250
123
PTO connection
X
X
Universal 3 point hitch
Tools
X
2 bodies
X
25 blades 350 rpm
X
29 tines
X
60 blades 540 rpm
Precision Agric
Fig. 3 Three dimensional rendering of surfaces used for sensor calibration and testing: a an ideally flat plane and b a surface featuring 33 hemispheres: some void pixels (i.e. missing data) are visible on hemispheres portions exceeding Kinect maximum detectable slope
Fig. 4 Measured values versus reference values respectively for average roughness Sa, root mean square roughness Sq, skewness Ssk and kurtosis Sku. Bars indicate standard deviation on 5 measurements
123
Precision Agric
Field tests on different soils An example of three dimensional rendering for each of the scanned soils is shown in Fig. 5. Five measurements were chosen and analyzed for each operation, and the same roughness parameters previously discussed were considered, together with the amount of void pixels. Average values together with standard deviations are shown in Fig. 6. Analyses in terms of average and root mean square roughness obviously show that a ploughing results in a soil which is more uneven, quantitatively by a factor of 4–5 compared with other operations. Fig. 5 Three dimensional rendering achieved through Kinect sensing on a ploughed, b harrowed, c vibro-tilled and d rotary-tilled soils
123
Precision Agric
Conversely vibro-tilling, harrowing and rotary-tilling are comparable in terms of roughness amplitude. However differences can be recognized when analyzing the other two parameters. In particular, the symmetry (inversely quantified by skewness) reaches the highest value (i.e. lowest Ssk rate) with the harrowed soil where the packer produces a sinusoidal profile with a symmetrical number of valleys and peaks (see Fig. 5b). On the other hand, higher Ssk is found for vibro-tilled or rotary-tilled soil: indeed, in these two last cases, the soil appears more like a flat surface with isolated clods, rather than a homogeneous surface. In particular, for the rotary tiller, such morphology can be associated with the presence of a leveling board which produces a flattening of the tilled surface. Kurtosis is also interesting for quantifying local height distribution. Low kurtosis values are indeed indicative of wide spread height distributions: and this is the case for ploughing (which leads to deep holes as well as large clods) and for harrowing, with its sinusoidal profile. High kurtosis values for the vibro-tiller and rotary-tiller operations are, on the other hand, indicative of a soil where most of the surface has a limited height range, while only a few clods (or holes) interrupt such a distribution.
Fig. 6 Measured roughness values on different soils: graphs show respectively average roughness Sa, root mean square roughness Sq, skewness Ssk and kurtosis Sku. Bars indicate standard deviation on 5 measurements
123
Precision Agric
Fig. 7 a The harrow-packing operation and measurement results: b average roughness Sa, c skewness Ssk and d kurtosis Sku. Bars indicate standard deviation on 3 measurements
Field tests with different tillage speeds Mean results in terms of average roughness and kurtosis are reported in Fig. 7. Measurements revealed that increasing speeds were associated with lower average roughness values and higher kurtosis and skewness values. This trend confirms visual inspections. Indeed on areas operated at lower forward speed, the soil appeared with a clear and large waviness left by the packer: this results in high average values and low kurtosis. On the other hand, on areas where the machine had been operated at higher speeds, such waviness was less pronounced, with a predominance of clods on the soil surface: conversely this results in lower average roughness and higher skewness and kurtosis. It is interesting to note the high correlation detected between average roughness and forward speed. Computing the coefficient of determination after linear regression, an R2 = 0.88 was found, corresponding to a correlation coefficient of r = 0.939. Whenever such a condition is fulfilled or, in other words, whenever machine operating settings (such as power or speed) directly influence soil condition in a repeatable way, a three dimensional sensor such as Kinect can be used to better foresee final soil roughness and help better machine calibration.
123
Precision Agric
Conclusions In the present paper, the possible use of a Microsoft KinectTM RGB-depth camera is proposed as a cost effective solution for three-dimensional characterization of soil microrelief. Tests on reference surfaces demonstrated how the sensor could give repeatable roughness characterization on surfaces with average roughness down to 1 mm. Field tests were carried out on soils which had been subjected to different tillage operations (ploughing, harrowing, vibro-tilling and rotary-tilling): in all cases, different roughness parameter values could be associated with different characteristics of tilled soils. Tests also included harrowing at different speeds. 3D characterization could discriminate between these different speeds, with a linear correlation coefficient higher than 0.9 between average roughness Sa and speed variations.
References Aguilar, M. A., Aguilar, F. J., & Negreiros, J. (2009). Off-the-shelf laser scanning and close-range digital photogrammetry for measuring agricultural soils microrelief. Biosystems Engineering, 103(4), 504–517. Bertuzzi, P., Caussignac, J. M., Stengel, P., Morel, G., Lorendeau, J. Y., & Pelloux, G. (1990). An automated, noncontact laser profile meter for measuring soil roughness in situ. Soil Science, 149(3), 125–184. Chatzinikos, A., Gemtos, T.A., & Fountas, S. (2013). The use of a laser scanner for measuring crop properties in three different crops in Central Greece. In J.V. Stafford (Ed.), Precision Agriculture’13: Proceedings of the 9th European Conference on Precision Agriculture pp. 129–136. The Netherlands: Wageningen Academic Publishers. Che´ne´, Y., Rousseau, D., Lucidarme, P., Bertheloot, J., Caffier, V., Morel, P., et al. (2012). On the use of depth camera for 3D phenotyping of entire plants. Computers and Electronics in Agriculture, 82, 122–127. Chimi-Chiadjeu, O., Le He´garat-Mascle, S., Vannier, E., Taconet, O., & Dusse´aux, R. (2014). Automatic clod detection and boundary estimation from Digital Elevation Model images using different approaches. Catena, 118, 73–83. Darboux, F., & Huang, C. (2003). An instantaneous-profile laser scanner to measure soil surface microtopography. Soil Science Society of America Journal, 67(1), 92–99. Draelos, M., Deshpande, N., & Grant, E. (2012). The Kinect up close: Adaptations for short-range imaging, 2012 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), pp. 251–256. Gessesse, G. D., Fuchs, H., Mansberger, R., Klik, A., & Rieke-Zapp, D. H. (2010). Assessment of erosion, deposition and rill development on irregular soil surfaces using close range digital photogrammetry. Photogrammetric Record, 25(131), 299–318. Helming, K., Romkens, M. J. M., & Prasad, S. N. (1998). Surface roughness related processes of runoff and soil loss: A flume study. Soil Science Society of America Journal, 62(1), 243–250. ISO 11783-9:2012. Tractors and machinery for agriculture and forestry—serial control and communications data network—part 9: Tractor ECU. International Standards Organization, Switzerland. ISO 25178-2. (2012). Geometrical product specifications (GPS)—surface texture: Areal—part 2: Terms, definitions and surface texture parameters. Switzerland: International Standards Organization. Kamphorst, E. C., Jetten, V., Gue´rif, J., Pitkanen, J., Iversen, B. V., Douglas, J. T., et al. (2000). Predicting depressional storage from soil surface roughness. Soil Science Society of America Journal, 64(5), 1749–1758. Marinello, F., Bariani, P., De Chiffre, L., & Hansen, H. N. (2007). Development and analysis of a software tool for stitching three-dimensional surface topography data sets. Measurement Science & Technology, 18(5), 1404–1412. Marinello, F., Pezzuolo, A., Gasparini, F., & Sartori, L. (2013). Three-dimensional sensor for dynamic characterization of soil microrelief. In: J.V. Stafford (Ed.), Precision Agriculture’13: Proceedings of the 9th European Conference on Precision Agriculture (pp. 71–78). The Netherlands: Wageningen Academic Publishers.
123
Precision Agric Sandri, R., Anken, T., Hilfiker, T., Sartori, L., & Bollhalder, H. (1998). Comparison of methods for determining cloddiness in seedbed preparation. Soil and Tillage Research, 45, 75–90. Stafford, J.V., & Ambler, B. (1990). Computer vision as a sensing system for soil cultivator control. Proceedings of the Institution of Mechanical Engineers C419/041 (pp. 123–129). Tanke, N., Long, G.A., Agrawal, D., Valada, A., & Kantor, G.A. (2012). Automation of hydroponic installations using a robot with position based visual feedback. In: F. Rovira-Ma`s (Ed.) Proceedings of the international conference of agricultural engineering CIGR-Ageng. Spain: Diazotec S. L. Publisher, paper no. C1429. Viscarra Rossel, R. A., Adamchuk, V. I., Sudduth, K. A., McKenzie, N. J., & Lobsey, C. (2011). Proximal soil sensing: An effective approach for soil measurements in space and time. Advances in Agronomy, 113, 243–291. Zhang, Z. (2012). Microsoft kinect sensor and its effect. IEEE Multimedia, 19(2), 6190806 4-10.
123
