CONVOLUTIONAL NEURAL NETWORK FOR MULTI-SOURCE DEEP LEARNING CROP CLASSIFICATION IN UKRAINE Mykola Lavreniuk Space Research Institute NASU-SSAU, Kyiv, Ukraine National Technical University of Ukraine “Kyiv Polytechnic Institute”, Kyiv, Ukraine 1
Introduction • Agriculture in Ukraine – 1st world largest sunflower producer and exporter (in 2013-2014) – 8th world largest wheat producer (in 2014) [source: FAS USDA]
• Crop mapping – important input within many problems solving • crop area estimation • crop yield forecasting • environmental impact analysis
AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
Big Data Era in Satellite Monitoring - SAR Radarsat-2: Nominal Scene Size - 50x50 km2 Repeat cycle - 24 days Cost - more than 4000$ per scene
Sentinel-1: Nominal Scene Size - 250x250 km2 Repeat cycle - 12 days More than 500 scenes in 2015 for Ukraine territory Cost - free
AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
3
Big Data Era – optical RapidEye: Swath Width – 77 km Revisit time: Daily (off-nadir) / 5.5 days (at nadir) Cost – non free
Sentinel-2: Swath Width – 100 km Revisit time: 10 days Cost - free
Landsat-8: Scene Size - 185x180 km2 Repeat cycle - 16 days Cost – free
Proba-V: Swath Width – 2250 km (full field of view) Cost – free AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
4
Objectives To develop classification methodology for Ukraine with multitemporal optical images and multi-temporal multipolarization SAR images using deep learning and convolutional neural network ensemble
+ Multi-temporal optical images
Multi-temporal SAR images
Crop maps AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
Deep Learning Deep Learning – is a set of machine learning algorithms based on multi-layer networks CAT
DOG
Training
© Deep Learning Summer Workshop AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
Challenges for EO Imagenet database: 14 mln labeled images, 20K categories, 200x200 size
Satellite data specifics: Big size of imagery (size 10K x 10K) Non equal shape of imagery Non full overlapping of different satellite
No data pixels (cloudiness) Relatively small size of labeled data
There is no case (ready to use) solution for satellite data classification! AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
Core expertise: Deep Learning 4. Data Fusion and geospatial intelligence, geospatial socioeconomic analysis
3. Map filtering (weighted voting approaches with division parcels into the fields) 2. Universal deep learning approach for time series classification at regional level
1. Clustering and no-data pixels restoration (clouds and shadows) using self-organized Kohonen maps AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
Ground Observations Along the roads surveys, 2013-2015 2013 Polygons № Class No. % 1 Artificial 6 1.6 2 Winter wheat 51 13.2 3 Winter rapeseed 12 3.1 4 Spring crops 9 2.3 5 Maize 87 22.5 6 Sugar beet 8 2.1 7 Sunflower 30 7.8 8 Soybeans 60 15.5 9 Other cereals 32 8.3 10 Forest 17 4.4 11 Grassland 48 12.4 12 Bare land 10 2.6 13 Water 16 4.1 Total
386
2014 2015 Polygons Polygons No. % No. % 0 0 17 2.8 126 20.4 102 18.6 36 5.8 22 4.0 45 7.3 11 2.0 76 12.3 98 17.9 8 1.5 18 2.9 31 5.0 53 9.7 109 17.6 87 15.9 0 0 12 1.9 35 5.7 49 9.0 68 11.0 64 11.7 11 1.8 10 1.8 34 5.5 43 7.9 618
547
AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
Data randomly divided into training (50%) and testing (50%) sets
2014
Kyiv oblast (area 28,131 km2)
Kyiv Region Classification Map 2015 Satellite
OA, %
L8 + S1 SENTINEL-1 LANDSAT-8
92.7 91.4 85.4
AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
Validation: JECAM experiments Ukrainian JECAM site
Cropland mask Kyiv region
AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
11
Deep Convolutional Neural Network Overall Accuracy Landsat-8 + Sentinel-1 = 93.5% RF NN CNN PA, % UA, % PA, % UA, % PA, % UA, % 95.2 95.3 95.1 95.9 1 Winter wheat 95.8 92.2 97.5 78.4 97.1 78.3 2 Winter rapeseed 78.2 97.3 50.6 96.7 48.6 93.6 3 Spring crops 96.1 49.1 90.6 90.8 90.6 94.1 4 Maize 88.2 79.3 97.5 98.4 97.4 99.1 5 Sugar beet 95.7 91.3 99.2 96.3 99 96.4 6 Sunflower 96.5 98.9 81 83.9 87.3 84.5 7 Soybeans 69.8 79.0 99.6 99.7 99.4 99.8 8 Forest 99.5 99.5 94.5 91.6 94.6 91.8 9 Grassland 82.0 92.0 90.6 82.8 92.6 74.9 10 Bare land 47.4 89.6 99.8 99.9 99.9 100 11 Water 100.0 99.8 OA, % 88.7 92.7 93.5 # Class
AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
Crop rotation violations in 2013-2015
AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
13
GEE Award, 2016 Google Earth Engine Research Award 2016 “Large scale crop mapping in Ukraine using SAR and optical data fusion”
AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
14
Conclusions & Further work • Big data and deep learning is challenge for satellite data classification; • Data fusion of SAR and optical data (Sentinel-1A + Landsat-8) for Kyiv region brings +1.3% and +7.3% comparing to SAR and optical data, respectively; • Deep convolutional neural network allows to increase OA by +4.8% and +0.8% (to 93.5%) comparing to Random Forest and Ensemble of Neural Networks; • We are going to utilize deep learning technique with CNNs within Google Earth Engine Research Award using Sentinel-1 and Sentinel-2 data. AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
References 1. Skakun S. Efficiency Assessment of Multitemporal C-Band Radarsat-2 Intensity and Landsat-8 Surface Reflectance Satellite Imagery for Crop Classification in Ukraine / Skakun, S., Kussul, N., Shelestov, A., Lavreniuk, M., Kussul, O. // IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. – 2016. – vol. 9, no. 8. – P. 3712-3719. DOI: 10.1109/JSTARS.2015.2454297.
2. Kussul N. Regional scale crop mapping using multi-temporal satellite imagery / Kussul, N., Skakun, S., Shelestov, A., Lavreniuk, M., Yailymov, B., & Kussul, O. // The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences. – 2015. vol. XL-7/W3. – P. 45-52. DOI:10.5194/isprsarchives-XL-7-W3-452015. 3. Kussul N. Parcel-based Crop Classification in Ukraine Using Landsat-8 Data and Sentinel-1A Data / N. Kussul, G. Lemoine, J. Gallego, S. Skakun, M. Lavreniuk, A. Shelestov // IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. – 2016. – vol. 9, no. 6. – P. 2500 – 2508. DOI: 10.1109/JSTARS.2016.2560141. AGU Fall Meeting 2016, San Francisco, United States of America, 16 December 2016
Thank you!
[email protected]
