APPLICATION OF SfM-MVS PHOTOGRAMMETRY IN

2 downloads 0 Views 2MB Size Report
Page 13 .... Cherub Statue, https://skfb.ly/CMIG (accessed Feb 17, 2017). 3DFlowb. 3DF Zephyr Photography ... sgmccann.pdf (accessed Feb 14, 2017). nVidia.
APPLICATION OF SfM-MVS PHOTOGRAMMETRY IN GEOLOGY VIRTUAL FIELD TRIPS Youngwoo Cho and Renee, M. Clary

OBJECTIVES  Research a procedure for reconstructing 3D models of geologic targets for Virtual Field Trips (VFTs) using Structure from Motion (SfM) and Multi-View Stereo (MVS) Photogrammetry.  Research techniques for achieving high enough quality of the models for VFTs, based on the photography and post-production techniques.

https://sketchfab.com/yow

INTRODUCTION: 3D Models  The product of a mathematical representation of an object using surface features of that object.

WikiVisually

3D FLOWa

INTRODUCTION: SfM-MVS  SfM-MVS; Structure from Motion – Multi-View Stereo  Point Cloud Data Acquisition  SfM is the process to calculate a sparse 3D point cloud model from the overlapping pictures taken for the surface of an object from different camera locations. (Carrivick et al. 2016)

INTRODUCTION: SfM-MVS  SfM-MVS; Structure from Motion – Multi-View Stereo  MVS is a refinement of 3D model from the SfM method.  Algorithms based on the measurement of a consistency function are used to generate a dense 3D reconstruction of an object. (McCann)  Limitations given by noise and calibration errors, in terms of colors and texture. (McCann)

PROBLEM DESCRIPTION  Problems in taking pictures of geologic targets, in terms of reconstruction quality.  Geologic outcrop  Under the Sun 1. High-contrast, Shadows, Time-variant 2. Too large to control the photographic environment.

PROBLEM DESCRIPTION  Small geologic samples  Uniform lighting  Dealing with shadows and glossy surface

METHODS: PHOTOGRAPHY  Lighting Considerations  Geologic outcrop: Low-Contrast Environment  Cloud covers  Early morning or late afternoon shots  Canopy  Limited use of camera flash  Small geologic samples  Dealing with shiny or bright spots  Diffusers  Polarizers

METHODS: PHOTOGRAPHY  Dynamic Range Considerations  High Dynamic Range Photography

Johnston

METHODS: PHOTOGRAPHY  Dynamic Range Considerations  High Dynamic Range Photography  Auto Exposure Bracketing

METHODS: PHOTOGRAPHY  Dynamic Range Considerations  High Dynamic Range Photography  High dynamic range blending

METHODS: PHOTOGRAPHY  Dynamic Range Considerations  Low ISO shooting  RAW capture

METHODS: PHOTOGRAPHY  RAW capture  Higher dynamic range  Lower noise

Photography course

METHODS: PHOTOGRAPHY  Field-of-Depth Considerations  Field of Depth consideration  Focus stacking  Small aperture photography  Stable support for long exposures  Tripods or Jibs  Shock Damping

Professional Photography Simplified

METHODS: PHOTOGRAPHY  Noise Suppression  Low ISO  Sufficient illumination  Dealing with shadows while shooting  RAW image capture (High Dynamic Range, low noise)  Reasonable noise sampling and application

METHODS: PHOTOGRAPHY  Considerations for Photographic Equipment  Cameras  Sufficient sensor size  Good noise suppression  Good dynamic range  RAW capture capability  Image stabilizer  Mirror lock-up

METHODS: PHOTOGRAPHY  Considerations for Photographic Equipment  Lenses  Low geometric distortion  No wide-angle lenses  Large maximum-aperture  Good optical quality

METHODS: PHOTOGRAPHY  Considerations for Photographic Equipment  Accessories  Heavy-duty tripod with low- to high-angle capture  Portable Jimmy Jibs or Cranes  Cable release or WIFI/Bluetooth Shooting  Camera flash  Diffuser, Bouncer, or Reflection plate

METHODS: PHOTOGRAPHY  Cameras used Canon 5D Mark II

Canon 350D

Sufficient sensor size

21.1 MP

8 MP

Good noise suppression

100-6400

100-1600

Good dynamic range



×

RAW capture capability





Image stabilizer

×

×

Mirror lock-up





METHODS: PHOTOGRAPHY  Cameras used Canon 5D Mark II

Canon 5D Mark IV

Sufficient sensor size

21.1 MP

30.1 MP

Good noise suppression

100-6400

100-32000

Good dynamic range





RAW capture capability





Image stabilizer

×

×

Mirror lock-up





METHODS: PHOTOGRAPHY  Lenses used Canon 24mm f/1.4 L II

Canon 70-200mm f/2.8 L

Low geometric distortion





No wide-angle lenses





Large maximumaperture





Good optical quality





METHODS: PHOTOGRAPHY  Cameras and Lenses used

METHODS: PHOTOGRAPHY  Accessories used Heavy-duty tripod with low- to high-angle capture Manfroto 055X PROB with 410 Gear Head Cable release



Camera flash

×

Diffuser or Reflection plate ○

METHODS: RECONSTRUCTION  Constructing 3D models of objects for VFTs  Building Sparse Point Clouds  Building Dense Point Clouds  Building Mesh  Building Texture

METHODS: COMPUTING SYSTEM  CPU: Intel Core i7-4790 CPU @3.60 GHz  RAM: 32.0 GB  Video Card: nVidia Geforce GTX 1060 6GB CUDA: “A parallel computing platform and application programming interface (API) model created by Nvidia. It enables dramatic increases in computing performance by harnessing the power of the graphics processing unit (GPU).” (nVidia)

RESULTS  Small geologic samples  Geode

https://sketchfab.com/yow

RESULTS  Small geologic samples  Geode

RESULTS  Small geologic samples  Shark Tooth fossil

RESULTS  Geologic outcrops  Petrified wood

RESULTS  Geologic outcrops  Smith County

RESULTS  Geologic outcrops  Smith County

RESULTS  Geologic outcrops  Smith County

CONCLUSION  Photographic techniques enhanced the quality of the 3D model, unless otherwise fail to do or generates weird results.  High dynamic range photography and noise suppression helps decrease the photographic and reconstruction noise.  Good selection of photographic equipment is very important.

CONCLUSION  Lighting control is important for the reconstruction of the small geologic samples, particularly for that having glossy surface. It is also very important in outcrop shooting although it is sometimes not easy to control.  Choosing a cloudy day, early morning or late afternoon for outcrop shooting helps reduce the lighting contrast. But the weather can not be controlled and early-morning/late-afternoon shooting gives an additional challenge due to the sunlight coming into the field-of-view.

CONCLUSION  Using camera flash is a good choice to decrease the contrast in the field, but it sometimes may cast additional shadow. It also has a limitation in the case the outcrop is very large.  RAW capture is not an option.  Need to keep the camera stable during a long exposure with large f-ratio. A plan for additional stability is an option.  Need to suppress the photographic noise as low as possible.

CONCLUSION  Field of Depth should be large enough to have all points in the image be well focused. The f-ratio of 22 or larger was used. Focus stacking could be an alternative.  Reconstructing a 3D model could take very long time, especially in case of large number of pictures and higher precision settings in the software, often more than a week. So, having a higher performance computer is vital.  Need to choose a good graphics card with a good CUDA performance.

DISCUSSION  (COST) It requires good photographic equipment to build 3D models to be used in VFTs, though it is still cheaper compared to the other point cloud data acquisition method.  (COMPUTING TIME) It takes a substantial amount of time to build a 3D model; a factor increasing the cost of this method.  (SOFTWARE PERFORMANCE) It still looks a long way to go, especially in the software performance.

REFERENCES 3DFlowa. Cherub Statue, https://skfb.ly/CMIG (accessed Feb 17, 2017) 3DFlowb. 3DF Zephyr Photography guide, http://www.3dflow.net/technology/documents/photogrammetry-how-to-acquirepictures/ (accessed Feb 17, 2017) Agisoft (2016). Agisoft Photoscan User Manuals, http://www.agisoft.com/downloads/user-manuals/ (accessed Feb 14, 2017) Carrivick, J.L., Smith, M.W., Quincey, D.J. (2016). Structure from Motion in the Geosciences (Analytical Methods in Earth and Environmental Science), Wiley. Kindle Edition. Johnston, M. More on Dynamic Range, http://theonlinephotographer.typepad.com/the_online_photographer/2009/02/dyna mic-range.html (accessed Feb 20, 2017) LuptidStyle.com. Professional Photography Simplified, http://luptidstyle.com/professional-photography-simplified/ (accessed Feb 21, 2017)

REFERENCES McCann, S. 3D Reconstruction from Multiple Images, https://web.stanford.edu/class/cs231a/prev_projects/CS231a-FinalReportsgmccann.pdf (accessed Feb 14, 2017) nVidia. What is CUDA?, https://www.nvidia.com/object/cuda_home_new.html (accessed Feb 21, 2017) Photography course. RAW vs JPG, http://photographycourse.net/raw-vs-jpeg/ (accessed Feb 21, 2017) WikiVisually.com. Structure from Motion, http://wikivisually.com/wiki/Structure_from_motion (accessed Feb 21, 2017)