Rapid Texture Optimization of Three-Dimensional

sensors Article

Rapid Texture Optimization of Three-Dimensional Urban Model Based on Oblique Images Weilong Zhang 1 , Ming Li 1,2, *, Bingxuan Guo 1, *, Deren Li 1,2 and Ge Guo 1 1

2

*

State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China; [email protected] (W.Z.); [email protected] (D.L.); [email protected] (G.G.) Collaborative Innovation Center for Geospatial Technology, Wuhan University, Wuhan 430079, China Correspondence: [email protected] (M.L.); [email protected] (B.G.); Tel.: +86-138-7102-5925 (M.L.); +86-138-7115-8036 (B.G.)

Academic Editors: Changshan Wu and Shawn (Shixiong) Hu Received: 10 March 2017; Accepted: 14 April 2017; Published: 20 April 2017

Abstract: Seamless texture mapping is one of the key technologies for photorealistic 3D texture reconstruction. In this paper, a method of rapid texture optimization of 3D urban reconstruction based on oblique images is proposed aiming at the existence of texture fragments, seams, and inconsistency of color in urban 3D texture mapping based on low-altitude oblique images. First, we explore implementing radiation correction on the experimental images with a radiation procession algorithm. Then, an efficient occlusion detection algorithm based on OpenGL is proposed according to the mapping relation between the terrain triangular mesh surface and the images to implement the occlusion detection of the visible texture on the triangular facets as well as create a list of visible images. Finally, a texture clustering algorithm is put forward based on Markov Random Field utilizing the inherent attributes of the images and solve the energy function minimization by Graph-Cuts. The experimental results display that the method is capable of decreasing the existence of texture fragments, seams, and inconsistency of color in the 3D texture model reconstruction. Keywords: oblique images; 3D texture reconstruction; radiation correction; occlusion detection; Graph-Cuts; texture mapping

1. Introduction Low-altitude oblique images can be photographed from the side and the top of the city, which can not only provide a precise 3D surface model, but also obtain reliable texture information, becoming one of the principal data resources in urban 3D texture reconstruction and updating. Now, some reliable methods in the reconstruction of urban 3D surface models from oblique images have been proposed [1–5], especially for the commercialization of some software like Smart3D Capture. However, problems existed in low-altitude oblique images—such as great changes in field of view, large volume of data, inconsistency of light, perspective distortion, and occlusion—will the vast texture fragments and the discontinuity of color common, which needs to be solved. Research about these problems has been carried out in numerous of papers. Take the vertex rendering method epidemic in the early days, [6,7] assign the vertexes of the triangular by rear projection after color components of RGB of the aerial images. The texture generated lost great details with low resolution. References [8,9] utilized the method of weighting different images to generate texture, which did not work well with satisfactory consistency of the color of texture in many cases. In order to get a superior solution of these problems, many studies started to address how to choose the priority images as the texture resource for seamless texture mapping in a triangular mesh model. References [10,11] chose the priority image for each triangular facet as the texture resource by taking the visibility condition between the images and 3D surface model in to consideration. Sensors 2017, 17, 911; doi:10.3390/s17040911

www.mdpi.com/journal/sensors

Sensors 2017, 17, 911

2 of 15

Sensors 2017, 17, 911

2 of 15

the priority image for each triangular facet as the texture resource by taking the visibility condition between the images and 3D surface model in to consideration. References [12,13] attempted to choose the highest [12,13] quality attempted image for every triangular facet as its texture after the corresponding References to choose the highest quality image for matching every triangular facet as its line segments in the images and 3D surface andinmaking overall of image texture after matching the corresponding line model segments the images andconsideration 3D surface model and occlusion, resolution, as well as the normal vector of the model. Reference [14] selected the priority making overall consideration of image occlusion, resolution, as well as the normal vector of the model. texture for[14] indoor small close utilizing the Markov random (MRF) Reference selected theobjects prioritywith texture forshots indoor small objects with close shotsfield utilizing the method Markov after overall which obtained results. Nevertheless, neither these could address random fieldoptimization (MRF) method after overall good optimization which obtained goodofresults. Nevertheless, the problem of texture fragments inconsistency color in big texture mapping. neither of these could address theand problem of textureoffragments andurban-scene inconsistency of color in big Moreover, thetexture methods above require excessive computer resources low efficiency. On urban-scene mapping. Moreover, the methods above requireresulting excessiveincomputer resources account of these problems, a method of rapid texture optimization will be presented in this paper to resulting in low efficiency. On account of these problems, a method of rapid texture optimization accelerate and improve quality of seamless urban the texture reconstruction. paper, the will be presented in thisthe paper to accelerate and3D improve quality of seamless In 3Dthis urban texture experimental oblique been corrected for distortion, we will a radiation procession reconstruction. In thisimages paper, have the experimental oblique images have beenadopt corrected for distortion, we algorithm to preprocess the oblique images first; and then, an efficient occlusion detection algorithm will adopt a radiation procession algorithm to preprocess the oblique images first; and then, an efficient will be proposed foralgorithm creating awill priority list of visible images; at last, list a Graph-Cuts based texture occlusion detection be proposed for creating a priority of visible images; at last, clustering andbased selecting algorithm willand be put forward. a Graph-Cuts texture clustering selecting algorithm will be put forward. 2. 3D 3D Urban Urban Surface Model thepaper, paper,the theprocedure procedure urban terrain surface model reconstruction is as follows. In the of of 3D3D urban terrain surface model reconstruction is as follows. First, First, extracting and matching feature points in the multi-view obliqueimages imagesutilizing utilizingSIFT SIFT (Scale extracting and matching feature points in the multi-view oblique calculating thethe geometrical relationship between the Invariant Feature Feature Transform) Transform)[3] [3]isiscombined combinedwith with calculating geometrical relationship between multiple views. Then, the matching result by SFM (Structure from Motion) [1] is calculated to get scene the multiple views. Then, the matching result by SFM (Structure from Motion) [1] is calculated to get information such as the as camera position and then sparsesparse by theby obtained 3D point scene information such the camera position andreconstruct then reconstruct the obtained 3D cloud. point After that, wethat, combine the CMVS (Cluster Stereo) andStereo) PMVS and (Patch-based Multi-View cloud. After we combine the CMVSMulti-View (Cluster Multi-View PMVS (Patch-based Stereo) technique acquire a to reconstructed 3D dense point cloud point [4,5]. The result a constructed Multi-View Stereo)totechnique acquire a reconstructed 3D dense cloud [4,5].isThe result is a 3D urban surface modelsurface (expressed by Triangulated Irregular Network) for texture optimization and constructed 3D urban model (expressed by Triangulated Irregular Network) for texture mapping. Figure shows a local 3D1 model the experimental optimization and 1mapping. Figure shows of a local 3D model of area. the experimental area.

(a)

(b)

Figure 1. 1. Local model of of the the experimental experimental area. area. (a) (a) 3D 3D fitting fitting model; model; (b) (b) 3D 3D mesh mesh model. model Figure Local 3D 3D model

3. 3. Image Image Radiation Radiation Preprocessing Preprocessing On On account account of of the the time time difference, difference, environmental environmental implications, implications, and and camera camera system system error error existing existing when shooting in multi-camera oblique photograph systems, there is a difference in brightness and when shooting in multi-camera oblique photograph systems, there is a difference in brightness and color color in different images shot on the same object. If the difference is left alone, it could result in color in different images shot on the same object. If the difference is left alone, it could result in color jump jump reconstructing texture. Therefore, this paper, a new algorithm is proposed which can when when reconstructing texture. Therefore, in this in paper, a new algorithm is proposed which can correct correct the radiation inconsistency in multi-view images. In our algorithm, the local and overall the radiation inconsistency in multi-view images. In our algorithm, the local and overall environmental environmental impacting factors systemic between and is inside the camera impacting factors are adjusted andare theadjusted systemic and errorthe between anderror inside the camera corrected, which is corrected, which could recover the mean value and the variance of the image to the initial radiation could recover the mean value and the variance of the image to the initial radiation brightness value of brightness the scene far as the possible and improve brightness andfor color of the image the scene asvalue far as of possible and as improve brightness and colorthe of the image used texture extraction. used for texture extraction. 3.1. Correction of Environmental Radiation Error The uniform color and light mechanism provided by Wallis filter is a relative radiation correction algorithm [15,16]. The algorithm can remove environmental radiation errors caused by global multi view by adjusting the mean value and variance of the image, which obtains a considerable result in uniform

Sensors 2017, 17, 911

3 of 15

color and bright in aerial images. Formula (1) [15,16] is Wallis’ mapping function. However, during the processing of oblique photographs, on account of the vast gap of shooting distance on the oblique surface, a resultant effect of white mist appeared in some images caused by the inconsistency of atmospheric influence. As for the haze phenomena which has shrouded the cities of China for a long time; a superior result is not attainable by Wallis’ mapping function alone. Therefore, in this paper, the algorithm based on the dark channel in [17] is brought in to remove the phenomena of white mist caused by the inconsistency of depth. The experiment proved that the environmental radiation problem could be solved from the depth information of the images. Formula (2) [17] is the recovery function of the scene based on dark channel. h i cs f f ( x, y) = g( x, y) − m g + bm f + (1 − b)m g (1) cs g + (1 − c)cs f J ( x, y) =

I ( x, y) − A h i+A max t0 , t( x, y)

(2)

In Formula (1), f (x,y) is the image after conversion and g(x,y) is the raw image. sf , sg and mf , mg are the standard deviation and mean value before and after conversion respectively. c ∈ [0, 1], which is the expansion constant of the variance of the image. b ∈ [0, 1], which is the brightness coefficient of the image. In Formula (2), J(x,y) is the image under ideal consideration; A is the intensity of atmospheric light; t(x,y) is the transmittance of transmission medium; t0 is the threshold of transmission. 3.2. Correction of Camera System Error After the processing above in Section 3.1, the effect of environmental factor in different images is removed fundamentally, recovering their radiance to the level of a single image. Nevertheless, on account of the existing of camera system error, the phenomenon of the difference in brightness and color still appears in different images photographing the same object, which needs correcting. Generally, camera system error could be divided into exposure factor, white balance coefficient, camera response function, and the influence of halo. In the paper, other than the rest algorithms that only the camera response function taken into consideration to correct the camera system error, our proposed generalization camera response function including halo function, white balance coefficient, and exposure factor to correct the mapping from scene radiance to brightness value for color images. Formula (3) [18] is the mapping relation of the generalized camera response estimation function in the paper. g( B1 ) g( B2 ) = w1i e1 M ( x1 ) w2i e2 M( x2 )

(3)

In Formula (3), g(·) represents the inverse function of camera response function; B represents the brightness value of the image; e represents the exposure factor; M represents the halo function; wi represents the white balance coefficient in different channel; and i represents different channel. 4. Establishing a Visible Images List Before the texture clustering and selection processed, it is required to know every triangular facet on the 3D surface model is visible in which images. Therefore, a list of visible images for each triangular facet in a 3D surface model needs to be established. In the paper, occlusion detection of triangular facets is implemented at first; then, according to the idea of texture filtering in [19], the occlusion area and normal vector are used as the evaluation criteria and a priority list of visible images is created. 4.1. OpenGL-Based Occlusion Detection Different from the geometric model obtained by scanning a Lidar point cloud, it is uncertain to reconstruct all the surface features when utilizing oblique-image-based 3D reconstruction, which could

Sensors 2017, 17, 911

4 of 15

lead to the inconsistency of occlusion between images and the 3D surface model. Therefore, algorithms of occlusion detection analogous to Z-buffer do not work well. In order to solve this problem, according to the transformation matrix of OpenGL in relation with the interior and exterior orientation elements in photogrammetry in [20], we completed the transformation matrix expression of OpenGL. Achieving rapid, efficient, and precise occlusion detection.

[ u v 1 ]T = V3×4 · P4×4 · M4×4 [ X Y

Z

1 ]T

(4)

Figure 2 shows three basic transformation of the procedure when OpenGL display 3D objects in window, they are model transformation (M), projection transformation (P), and viewport transformation (V). The object space coordinate system can be transformed into the screen coordinate system through these three transformation matrixes. The mathematical expressions of the matrixes are as seen in Formula (4). Formulas (5)–(7) are the specific expression of M, P, and V respectively, in which the model transformation equals the rotation and translation of the exterior orientation elements, R and Ts represent the rotation matrix and translation matrix, respectively. Projection transformation equals to 3D cutting + similarity transformation + affine transformation, in which (f,x0 ,y0 ) is the interior orientation elements and (w,h) is the width and height of the camera. Viewport transformation is equal to 2D Euclidean transformation. " # R T − R T Ts M= (5) 0 1 

2f w

2f h

  P=  

− xw0 − yh0 N − FF+ −N −1

w 2

0

 V= 0 0

0

h 2

0 0 0

w 2 h 2



− F2FN −N

   

 + x0  + y0  1

Sensors 2017, 17, 911

(6)

(7) 5 of 15

Figure 2. 2. OpenGL OpenGL transform transform procession. procession. Figure

4.2. Adaptive Partial Occluded Triangular Facet Subdivision After occlusion detection, the images can be divided into three types for each triangular facet: fully occluded images, partially occluded images, and fully visible images. The fully occluded images are removed. However, if we do not process the triangular facets we only have partially occluded images, the texture inconsistencies might appear because of the 3D triangular mesh surface model

Sensors 2017, 17, 911

Figure 2. OpenGL transform procession.

5 of 15

4.2. Adaptive Partial Occluded Triangular Facet Subdivision 4.2. Adaptive Partial Occluded Triangular Facet Subdivision

After occlusion detection, the images can be divided into three types for each triangular facet: After occlusion detection, images can be divided into visible three types for each facet: images fully fully occluded images, partiallythe occluded images, and fully images. Thetriangular fully occluded occluded images, partially occluded images, and fully visible images. The fully occluded images are are removed. However, if we do not process the triangular facets we only have partially occluded removed. if we do not process theappear triangular facetsofwe only partially occluded images, images, theHowever, texture inconsistencies might because the 3Dhave triangular mesh surface model the texture inconsistencies might appear because of the 3D triangular mesh surface model subdivision is subdivision is insufficient. In order to solve the problem and improve the occupancy of the noninsufficient. In order to solve the problem and improve the occupancy of the non-occluded information occluded information in a single triangular facet, a local quartering algorithm is applied in adaptive in a single triangular facet, a local quartering algorithm is applied in adaptive partial occluded partial occluded triangular facet subdivision in the paper. The new subdivision algorithm only triangular facet subdivision in the paper. The new subdivision algorithm only partially occluded partially occluded triangular facets and dichotomized the adjacent triangular facets (as Figure 3). In triangular facets and dichotomized the adjacent triangular facets (as Figure 3). In this way, not only this way, not only the efficient of partial occluded triangular facets subdivision can be guaranteed, it the efficient of partial occluded triangular facets subdivision can be guaranteed, it can also decrease can also decrease the time consumption and data size of segmenting non-occluded triangular facets. the time consumption and data size of segmenting non-occluded triangular facets. At the same time, it At the same time, it can also guarantee the invariability of half-edge data structure and the quality of can also guarantee the invariability of half-edge data structure and the quality of manifold surfaces of manifold surfaces of the net form on the 3D surface model. the net form on the 3D surface model.

Figure3.3.Triangle Trianglefacet facetsubdivision: subdivision: (a) (a) original (c)(c) our method. Figure originalmesh; mesh;(b) (b)global globalquartering; quartering; our method.

The triangularfacet facetwhich whichisispartially partiallyoccluded. occluded. Figure shows Theshadow shadowarea areain inFigure Figure 33 is is the triangular Figure 3c 3c shows the subdivisionresult resultof oftwo twodifferent different algorithms. algorithms. Compared global quartering in in the subdivision Comparedwith withthe theresult resultofof global quartering Figure3b, 3b, only only in situation, the number of triangular facets infacets Figurein 3c Figure is equal 3c to Figure 3b. to Figure inthe theworst worst situation, the number of triangular is equal Generally, the amount of the triangular produced is 4a + 6a by our algorithm, which is smaller than 4b. is Figure 3b. Generally, the amount of the triangular produced is 4a + 6a by our algorithm, which Sensors 2017, 17, 911 6 of 15 a is the number of partially occluded triangular facets, b is the number of all of the triangular facets. smaller than 4b. a is the number of partially occluded triangular facets, b is the number of all of the triangular 4.3. Priorityfacets. Principle of Visible Images 4.3. Priority Principle of Visible Images

InIn the thevisible visible images priority list following principles thepaper, paper, we we establish establish the images priority list following threethree principles below:below: (1) area(1) area maximum principle. That is to say, the larger area of the image on the triangular facet, the better; maximum principle. That is to say, the larger area of the image on the triangular facet, the better; (2)(2) angle principle.InIna anon-occluded non-occluded image, image deformation is proportional to the angleminimum minimum principle. image, image deformation is proportional to the angle angle between photography and terrain normal n. As shown in Figure 4, for surface abcd,Oimage between photography light light and terrain normal n. As shown in Figure 4, for surface abcd, image 1 is image, for abcd surface in theincube, imageimage O3 hasOhigher priority than image O2 . O1occlusion is occlusion image, for abcd surface the cube, 3 has higher priority than image O2.

Figure 4. 4. Occlusion Occlusion conditions. Figure conditions.

5. Texture Clustering and Selection Based on Graph-Cuts Selecting the texture with the highest priority as the texture source for every triangular facet in the established priority list of visible images often cannot meet the global optimization condition. Therefore, how to choose the optimal image in the priority list of visible images as the texture

angle between photography light and terrain normal n. As shown in Figure 4, for surface abcd, image O1 is occlusion image, for abcd surface in the cube, image O3 has higher priority than image O2.

Sensors 2017, 17, 911

6 of 15

5. Texture Clustering and Selection Based on Graph-Cuts Selecting the texture with the highest priority as the texture source for every triangular facet in the established priority list of visible images often cannot meet the global optimization condition. Therefore, how to choose the optimal image in the priority list of visible images as the texture mapping resource is still an intractable problem. In computer vision, a discrete multi-label problem like texture mapping can be regarded as a typical MRF model and Graph-Cuts is one of the most efficient global optimization tools to solve the energy function [21,22]. Thus, the Graph-Cuts based texture clustering algorithm under the frame of MRF Figure is proposed to solveconditions. the problem of texture selection optimization 4. Occlusion in the paper.

5. Texture Clustering and Selection Based on Graph-Cuts 5.1. Label Graph Construction Selecting the texture with the highest priority theword texture source for everybetriangular Graphing is an intuitive representation of theas real which is composed a point setfacet V in the established priority list of between visible images often cannot meet global as optimization condition. and a set of sides connected the nodes under a certain rulethe indicated G = . In this Therefore, howtriangular to choose theinoptimal imagemesh in the priority listis of visibleasimages asgraph the texture paper, each facet a 3D triangular surface model regarded a node in G. Every two adjacent triangles with a common side respond an edge constructed between the nodes mapping resource is still an intractable problem. In computer vision, a discrete multi-label problem graph mapping G. In this way, constructing the graph we need. Graph-Cuts isisa one subset setmost like in texture can we be finish regarded as a typical MRFthat model and Graph-Cuts ofinthe E of graph G. Graph-Cuts is a two-labeling problem, which could link the minimum of the energy efficient global optimization tools to solve the energy function [21,22]. Thus, the Graph-Cuts based function with the minimum cut of an image containing source and sink. As seen in the labeled graph texture clustering algorithm under the frame of MRF is proposed to solve the problem of texture constructed in this paper, not all the images could be the candidate texture resource to some certain selection optimization in the paper.

triangular facets, as every triangular facet may be occluded in some image. That is to say, the mapping relation between triangular facets and images is not a complete graph, which needs to be processed. 5.1. Label Graph Construction As shown in Figure 5, Figure 5a is a general label graph, in which the top and bottom special red terminal are the source s and sinkof t in thereal graph, respectively. The rest nodes different Graphing isnodes an intuitive representation the word which is composed be aare point set V and labeled point to each triangular facet. The nodes are connected by side t-link and side n-link which a set of sides connected between the nodes under a certain rule indicated as G = . In this paper, measured facet the probabilistic relation mesh of Energy weight. t-link connects the s,t,graph and other eachistriangular in a 3D triangular surface model is regarded as asides nodeofin G. Every points which contains the energy between different labels. n-link connects the sides of middle points two adjacent triangles with a common side respond an edge constructed between the nodes in graph which contains the energy relation between adjacent nodes. The relations mainly reflect the probability G. In this way, we finish constructing the graph that we need. Graph-Cuts is a subset in set E of graph of selecting the same texture label between the adjacent nodes. All the nodes constitute label space. G. Graph-Cuts is asurfaces two-labeling problem, which could minimum the energy function Moreover, the in the same label layer have the link samethe texture source.ofFigure 5b is the labeled with the minimum cut of an containing source sink. seen in the labeled graph constructed in image the paper. Compared with and Figure 5a, As some green blank nodes graph appear,constructed which in this paper, all the images could be the candidate texture resource some certain means thatnot texture nodes are undesirable in the texture space of this layer. to Therefore, all thetriangular links facets, as every may occluded in some image. Thatsize is toofsay, the mapping relation between the triangular nodes need facet to avoid thebe green blank nodes so that the data Graph-Cuts algorithm involved will be decreased the overall of the algorithm can be improved. between triangular facets andand images is notefficiency a complete graph, which needs to be processed. s

s

n-link

n-link t-link

t-link

t

t

(a)

(b)

Figure 5. Labeled graph: (a)(a)shows labeledgraph; graph;(b)(b) shows labeled graph. Figure 5. Labeled graph: showsthe thegeneral general labeled shows ourour labeled graph.

5.2. Energy Function Construction The energy function is the bridge between Graph-Cuts theory and a specific problem, it consists of two parts: data item and smooth item, which have one-to-one correspondence to weight of Graph-Cuts

Sensors 2017, 17, 911

7 of 15

sides. The minimum cut in Graph-Cuts equals to the minimum energy function. Formula (8) is the functional expression of the energy function in the paper. In graph G, each node has a label set that contains k factors. The optimization algorithm selects the optimal label in the label set for each node. To guarantee calculation in the valid real range by the Graph-Cuts algorithm and the ability to process big data, the energy function construction needs to meet the following principles: (A) The energy function needs to possess upper and lower bounds; (B) The value range of the energy function needs to be no more than limit times of the number of the data calculated; (C) The data item, smoothed value, and fluctuating value of the energy are supposed to be in the same order of magnitude. In order to accurately reflect the identity of the corresponding texture resources in adjacent triangular facets, we constructed the data item and smooth item for energy functions using the inherent attributes of the candidate images, to achieve selecting optimization and clustering for textures in adjacent triangular facets. E( L) = αD ( L) + βN ( L) (8) M = {mi |mi = AVERAGE( Ik )}, i ∈ [0, n − 1], k ∈ [0, O − 1]

(9)

D = {di |di = STDEV ( Ik )}, i ∈ [0, n − 1], k ∈ [0, O − 1]

(10)

F = { f i |i ∈ [0, n − 1]}

(11)

L = {li |li ∈ [0, O − 1], i ∈ [0, n − 1]}

(12)

In Formula (8), α and β are the control coefficients of the data item D(L) and smooth item N(L), respectively. Formulas (9–12) indicate the means value M, variance D, terrain triangular facet F, and the range of the label L, respectively. O is the number of the images and n is terrain surface. Formulas (13)–(14) are the expressions of the data item D(L) and smooth item N(L) in the energy function, in which αd and βd are internal factors of data items; αn and βn are internal factors of smooth item; V fi li is the included angle between the terrain triangular surface fi and the responding image li ; W f l is the visual ratio, which is the ratio of visible area and real area. i i  D ( L) =

∑ DP = ∑ exp

h







αd π/2 − v f i li + β d w f i li  h   i  αd π/2 − v f k lk + β d w f k lk ∑

i  1 − Pf i (li ) − 1, Pf i (li ) =  

(13)

k ∈[0,n−1]

     N ( L) =     Nhp,qi

(

0 l p = lq ∑ Nhp,qi · δ l p , lq , δ l p , lq = 1 l p 6 = lq { p,q}∈ N      2 2  (m p −mq ) (d p −dq ) = exp min max sqrt αn 2σ2 + β n 2σ2 , n _t −1





1

(14)

2

The energy function constructed in the paper satisfies not only the basic energy construction principle but also the normal distribution. Part of the parameter values in the formula above are as follows: α = 0.8, β = 0.2; αd = 0.8, βd = 0.2; αn = βn = 0.5. The parameters control the mean value and variance of the image directly. The n_t is the threshold value of the smooth item and here n_t = 1. 5.3. Energy Minimization Solution In the paper, α-β swap optimization algorithm is utilized to solve energy function, which is an efficient Graph-Cuts based dichotomy optimization. Not only can the initial data set be optimized and segmented, it can also transform the multidimensional digraph to 2D simple digraph which can avoid bounded uncertainty [21–23]. The basic idea of the algorithm supposes that there is a known label set L and segmented set P. If α, β ∈ L, exchange the label set α and β to form a new label set Lnew . To make sure that the cut corresponding to the Graph-Cuts in the new label set is smaller than the original ones, the new label set is assigned to L. The procession is circulated in turn until the minimum cut of the Graph-Cuts appears. Any label set L to segmenting set P can be equal to P = {Pl |l ∈ L} with

avoid bounded uncertainty [21–23]. The basic idea of the algorithm supposes that there is a known label set L and segmented set P. If α, β ∈ L, exchange the label set α and β to form a new label set Lnew. To make sure that the cut corresponding to the Graph-Cuts in the new label set is smaller than the original ones, the new label set is assigned to L. The procession is circulated in turn until the minimum Sensors 2017, 17, 911 8 of∈15L} with cut of the Graph-Cuts appears. Any label set L to segmenting set P can be equal to P = {Pl|l no difference. L is the definitional domain of the label set. Pl = {p ∈ P|lp = l} is the subset label of l difference. is the definitional domain of the set. Pl = {p ∈ the subset of l which is a no label in set P.L Given a set of labels α and β,label α-β exchange is P|l applied. is to label say, exchange p = l} is That which is a label in set P. Given a set of labels α and β, α-β exchange is applied. That is to say, exchange α and β to form a new segmented set Pnew, in which the set l ≠ α, β remains unchanged. α and β to form a new segmented set Pnew , in which the set l 6= α, β remains unchanged. Figure 6 shows the construction way of α-β graph. α and β are connected through the Pα set and Figure 6 shows the construction way of α-β graph. α and β are connected through the Pα set and Pβ set in the graph, αβ = Pα∪Pβ to define the weighted value of the corresponding t-link and Pβα-β set in the α-βwhere graph, P where Pαβ = Pα ∪Pβ to define the weighted value of the corresponding t-link n-link. Np is the neighborhood of node dissimilarity degree isislabeled byDDp (p·(· guides the and n-link. Np is the neighborhoodp, of and nodethe p, and the dissimilarity degree labeled by ),),it it guides the energy function space of Graph-Cuts; V(·)diversity is the diversity measure theneighborhood neighborhood nodes, it it has energy function space of Graph-Cuts; V(· ) is the measure of of the nodes, has effect. a smoothing effect. a smoothing

α tαb

tαa

tαc

tαd

Pα a tβa

tαf

tαg

Pα f

Pβ g e

Pβ b

eab

tαe

tβb

ebc

tβc

c

ecd

d

e

ede

tβd

tβe

eef

fg

tβf

tβg

β Figure 6. The α-βgraph. graph. Figure 6. The α-β

6. Experiments and Analysis

6. Experiments and Analysis

6.1. Experimental Data and Environment

6.1. Experimental Data and Environment In order to objectively evaluate and verify the validity and superiority of the proposed method in the paper, we chose the real oblique images of an experimental area to test the method. Theproposed experimental In order to objectively evaluate and verify the validity and superiority of the method images acquisition platform is SWDC-5, which consists of an upright camera and four cameras in the in the paper, we chose the◦ real oblique images of an experimental area to test the method. The four directions in a 45 angle layout. The experimental computer environment is Windows 7 operating experimental images acquisition platform SWDC-5, consists upright camera system, with 32 G computer memory, is intel core 7 (3.6which GHz, four cores of andan eight threads). Figureand 7 four cameras inshows the 17, four directions inofathe 45° angle layout. the five-view images experimental area. The experimental computer environment Sensors 2017, 911 9 of 15 is Windows 7 operating system, with 32 G computer memory, intel core 7 (3.6 GHz, four cores and eight threads). Figure 7 shows the five-view images of the experimental area.

Figure7.7.Experimental Experimental oblique Figure obliqueimages. images.

6.2. Pyramid Design of PagedLoad 6.2. Pyramid Design of PagedLoad Large-scale urban texturemodel modelreconstruction reconstruction requires of of computing resources and and Large-scale urban 3D3D texture requiresa alotlot computing resources storage space. In order alleviatethe thelimitations limitations and hardware andand massive storage space. In order to to alleviate and conflicts conflictsofofcomputer computer hardware massive data, we designed a ready-to-use PageLoad pyramid model during the execution of the algorithm. data, we designed a ready-to-use PageLoad pyramid model during the execution of the algorithm. The main idea of the PageLoad pyramid model is to layer and downsample the reconstruction scene The main idea of the PageLoad pyramid model is to layer and downsample the reconstruction scene according to the different visual distances and ranges of the observer to the scene. As shown in Figure 8, according to the different visual distances and ranges of the observer to the scene. As shown in Figure 8, the grid of the top of the pyramid is the simplest, and the lower layers of the pyramid become more and more complex.

Large-scale 3D texture model reconstruction requires a lot of computing resources and 6.2. Pyramid urban Design of PagedLoad storage space. In order to alleviate the limitations and conflicts of computer hardware and massive Large-scale urban 3D texture model reconstruction requires a lot of computing resources and data, we designed a ready-to-use PageLoad pyramid model during the execution of the algorithm. storage space. In order to alleviate the limitations and conflicts of computer hardware and massive The main of the PageLoad pyramid model is to layer and downsample theof reconstruction data,idea we designed a ready-to-use PageLoad pyramid model during the execution the algorithm. scene Sensors 2017, 17, 911 9 of 15 according to theidea different visual distances ranges of theand observer to the the scene. As shown in Figure 8, The main of the PageLoad pyramidand model is to layer downsample reconstruction scene according theofdifferent visual distances and ranges of the to the scene. Aspyramid shown in become Figure 8, more the grid of the to top the pyramid is the simplest, and theobserver lower layers of the the grid of the top of the pyramid is the simplest, and the lower layers of the pyramid become more the grid of the top of the pyramid is the simplest, and the lower layers of the pyramid become more and more complex. and more complex. and more complex.

(a)

(a)

(b)

(b)

Figure 8. Mesh simplification. (a) Pyramid simplified model; (b) Scene simplified instance.

Figure 8.Figure Mesh8.simplification. (a) (a) Pyramid model; (b) Scene simplified Mesh simplification. Pyramidsimplified simplified model; (b) Scene simplified instance. instance. 6.3. Results Analysis Results Analysis 6.3. Results6.3. Analysis

6.3.1. Comparison of Image Radiometric Correction 6.3.1. Comparison of Image Radiometric Correction

6.3.1. Comparison of Image Radiometric Correction Figure 9 shows the defog effect using dark channels. From left to right in order are the original Figure 9 shows the defog effect using dark channels. From left to right in order are the original

oblique image, Wallis processing result, dark channel processingresult, result, and depth map estimated by oblique image, Wallis processing channel processing depth map estimated Figure 9 shows the defog effect result, usingdark dark channels. From leftand to right in order are by the original darkdark channel. It can be easy seen from Figure 9 that the dark channel method has a better defog effect channel. It can be easy seen from Figure 9 that the dark channel method has a better defog effect oblique image, Wallismethod, processing result, dark channel processing result, and depth map estimated by thanthan the the Wallis cansignificantly significantly enhance the contrast and color saturation of the Wallis method, which which can enhance the contrast and color saturation of the image, dark channel. It can be dark easy seen Figure 9 that the dark channel method has a better defog andand because the method can estimate the approximate range of the original image, image, because thechannel dark from channel method can estimate thedepth approximate depth range of the effect we can achieve defog without depth than the Wallis method, which can significantly enhance theinformation. contrast depth and color saturation of the original image, wethe candesired achieve the effect desired defogadditional effect without additional information.

image, and because the dark channel method can estimate the approximate depth range of the original image, we can achieve the desired defog effect without additional depth information.

(a)

(b)

(c)

(d)

Figure 9. Contrast effect. Original image; (b) processing Wallis processing result; (c) Dark Figure 9. Contrastofofdefogging defogging effect. (a)(a) Original image; (b) Wallis result; (c) Dark channel channel processing (d) Depth processing result;result; (d) Depth map. map.

(a)

(b)

(c)

(d)

10 shows the comparison theOriginal authenticity of our method and result; Hugin [18] Figure 9. Figure Contrast of defogging effect. of(a) image; (b)proposed Wallis processing (c) Dark for the camera response curve estimation. The test data set is composed of the experimental oblique channel processing result; (d) Depth map. images in this paper. There are many ladder-like fluctuations in the camera response curve estimated by Hugin, the main reason is because it did not constrain the local constant function, that leads to a deformed camera response curve. Since the Wallis has a good global radiometric correction effect in the absence of fog, and the dark channel algorithm is only applied to single image defogging, and the overall image is darker, therefore, in order to avoid the processed image still displaying radiation inconsistency after dark channel algorithm correction and our camera response function. The experimental oblique images are processed with the camera response function and the dark channel algorithm, and then Wallis processing, to make up for the defects of the dark channel algorithm without image-related information.

Sensors 2017, 17, 911

10 of 15

Figure 10 shows the comparison of the authenticity of our proposed method and Hugin [18] for Figure 10 shows the comparison the authenticity of our proposed method Hugin [18] for the camera response curve estimation.ofThe test data set is composed of the and experimental oblique the in camera response curve The test fluctuations data set is composed of the response experimental oblique images this paper. There areestimation. many ladder-like in the camera curve estimated images in this paper. There are many ladder-like fluctuations in the camera response curve estimated by Hugin, the main reason is because it did not constrain the local constant function, that leads to a Sensors 2017, 17, 911 10 of 15 by Hugin, the main reasoncurve. is because it did not constrain the local constant function, that leads to a deformed camera response deformed camera response curve.

Figure 10. Contrast of camera response curves.

Figure of camera cameraresponse response curves. Figure10. 10.Contrast Contrast of curves. Since the Wallis has a good global radiometric correction effect in the absence of fog, and the dark channel onlyeffects applied to single defogging, and the overall image darker, Since the Wallis hasthe aislocal good globalcontrast radiometric correction effect in the absence ofisafter fog, and the Figure 11algorithm shows of image the 3D texture reconstruction before and the therefore, in order to avoid the processed image still displaying radiation inconsistency after dark radiation pretreatment. The left side of Figure 11 shows the effect of radiation pretreatment without dark channel algorithm is only applied to single image defogging, and the overall image is darker, channel correction ourwhere camera function. images our method; the to right sidethe areand ours, weresponse canstill see displaying that there The isradiation noexperimental textureinconsistency coloroblique jump problem. therefore, inalgorithm order avoid processed image after dark are processed with the camera response function and the dark channel algorithm, and then Wallis After pretreatment of our radiometric correction, mapping textures not only maintain the high contrast channel algorithm correction and our camera response function. The experimental oblique images processing, tobut make up image for thesaturation defects of dark channel algorithm without image-related of the image, alsocamera the andthe white have been significantly and are processed with the response function andbalance the dark channel algorithm,adjusted, and then Wallis information. the visual effect of texture model is more natural and clear.

processing, to make up for the defects of the dark channel algorithm without image-related information.

Figure 11. Contrast of texture radiation correction.

11 shows the of local effects contrast 6.3.2.Figure Efficiency Analysis Occlusion Detectionof the 3D texture reconstruction before and after the radiation pretreatment. The left side of Figure 11 shows the effect of radiation pretreatment without Concerning occlusionFigure detection efficiency,ofour use of the occlusion detection algorithm of Sparse 11. Contrast texture radiation correction. Grid is based on Z-Buffer [24] as the reference object. The time complexity of traditional Z-Buffer is m ), the improved algorithm of Sparse Grid can be reduced to O(n), where m is the number of O(n Figure 11 shows the local effects contrast of the 3D texture reconstruction before and after the average pixels corresponding to each triangle facet, and n is the number of triangle facets. For the radiation pretreatment. The left side of Figure 11 shows the effect of radiation pretreatment without data set of massive triangular facets in a large-scale city scene 3D texture reconstruction, the spatial

adjusted, and the visual effect of texture model is more natural and clear. 6.3.2. Efficiency Analysis of Occlusion Detection Concerning occlusion detection efficiency, our use of the occlusion detection algorithm of Sparse 11 of 15 Grid is based on Z-Buffer [24] as the reference object. The time complexity of traditional Z-Buffer is O(nm), the improved algorithm of Sparse Grid can be reduced to O(n), where m is the number of complexity of the improved algorithm above is non-uniform linearofgrowth. shows average pixels corresponding to eachmentioned triangle facet, and n is the number triangleFigure facets.12For the the comparison result of the Sparse Grid algorithm and our occlusion detection algorithm proposed data set of massive triangular facets in a large-scale city scene 3D texture reconstruction, the spatial in the paper.ofItthe canimproved be seen from Figurementioned 12 that although thenon-uniform algorithm islinear not dominant when the complexity algorithm above is growth. Figure 12 amount of data is small, result due to of thethe optimization of OpenGL shows the comparison Sparse Grid algorithmoff-screen and our rendering occlusiondetection detectionmechanism algorithm and GPU acceleration, linear growth of processing time canthe be ensured with by increasing proposed in the paper.the It can besteady seen from Figure 12 that although algorithm is not dominant the number of triangular When thethe number of triangular facets off-screen is more than 300,000,detection the time when the amount of data facets. is small, due to optimization of OpenGL rendering consumption of the algorithm basedthe on linear Sparsesteady Grid shows abnormal jump growth with an increasing mechanism and GPU acceleration, growth of processing time can be ensured with number of triangular facets,ofwhich demonstrates that our detection has by increasing the number triangular facets. When theproposed number occlusion of triangular facetsalgorithm is more than Sensors 2017, 17, x 12 of 16 a300,000, significant advantage in dealing with large data volumes. the time consumption of the algorithm based on Sparse Grid shows abnormal jump growth with an increasing number of triangular facets, which demonstrates that our proposed occlusion detection algorithm has a significant advantage in dealing with large data volumes. Sensors 2017, 17, 911

Figure 12. Efficiency contrast of occlusion detection.

Figure 12.12.Efficiency of occlusion occlusiondetection. detection. Figure Efficiencycontrast contrast of

Figure 13 compares compares the effects the occlusion detection. Itcan can beseen seen from the red red boxes boxes Figure 13 effects the occlusion detection. ItItcan bebe seen from thethe red boxes that that Figure 13 compares thethe effects ofofof the occlusion detection. from that the occlusion phenomenon has been effectively removed in the 3D texture model on the right, the occlusion phenomenon hasbeen beeneffectively effectively removed removed ininthe model on the right,right, the occlusion phenomenon has the3D3Dtexture texture model on the guaranteeing guaranteeing the the texture texture disparity disparity coherence coherence and and consistency consistency of of the the reconstructed reconstructed 3D 3D texture texture model. model.

guaranteeing the texture disparity coherence and consistency of the reconstructed 3D texture model.

Figure 13. Results comparison of occlusion detection.

FigureClustering 13. Resultsand comparison 6.3.3. Comparison of Texture Selection of occlusion detection.

Figure 14 shows the effects of texture clustering and selection, the above is the effect of our 6.3.3. Comparison of Texture Clustering and Selection algorithm, the below effect is not optimal. The zoom view of four corners are the enlarged pictures corresponding to red frame, where the different colors represent the triangular texture taken from the different images. The comparison of the effects in Figure 14 is obvious, after texture clustering optimization, more adjacent triangular facets in the 3D triangular mesh surface model are given the same texture source, which can reduce the problem of texture fragments and seams. Besides, the results

Figure 14 shows the effects of texture clustering and selection, the above is the effect of our algorithm, the below effect is not optimal. The zoom view of four corners are the enlarged pictures Sensors 2017, 17, 911corresponding to red frame, where the different colors represent the triangular texture taken 12 of from 15 the different images. The comparison of the effects in Figure 14 is obvious, after texture clustering optimization, more adjacent triangular facets in the 3D triangular mesh surface model are given Sensors 17, 911 12 ofthe 15 6.3.3. Comparison of2017, Texture Clustering and Selection same texture source, which can reduce the problem of texture fragments and seams. Besides, the Figure 14 results shows can the also effects of texture clustering and selection, theimprove above the is the effect of of texture our be used in the texture mapping operation to efficiency can also be used in the texture mapping operation to improve the efficiency of texture mapping mapping by making same clustering texture andcorners to obtainare thethe corresponding texturesby at algorithm, the below effect is not the optimal. The zoom viewblock of four enlarged pictures making the same clustering texture block and to obtain the corresponding textures at one time. one time. corresponding to red frame, where the different colors represent the triangular texture taken from

the different images. The comparison of the effects in Figure 14 is obvious, after texture clustering optimization, more adjacent triangular facets in the 3D triangular mesh surface model are given the same texture source, which can reduce the problem of texture fragments and seams. Besides, the results can also be used in the texture mapping operation to improve the efficiency of texture mapping by making the same clustering texture block and to obtain the corresponding textures at one time.

Figure 14. Contrast of clustering optimization.

Figure 15 shows the clustering statistic results of the triangular facets in the 3D triangular mesh surface model (3117 triangular facets). Figure 15a shows the texture selection distribution before clustering, Figure 15b is the result after clustering. According to the specific statistical data of clustering results, before clustering and selection optimization the isolated clustering (the number of adjacent triangular facets selecting the same texture source is 1 called isolated clustering) is about 35% of the total number of triangular facets, and the maximum clustering capacity is less than 20. After optimization, the isolated clustering is reduced to 15%, and the maximum clustering capacity is more than 40, which shows our optimization algorithm has significantoptimization. effect. Figure 14. Contrast ofaclustering

(a)

(b)

Figure 15. Comparison of the statistical results of clustering: (a) Before clustering; (b) After clustering.

Figure 15 shows the clustering statistic results of the triangular facets in the 3D triangular mesh surface model (3117 triangular facets). Figure 15a shows the texture selection distribution before clustering, Figure 15b is the result after clustering. According to the specific statistical data of clustering results, before clustering and selection optimization the isolated clustering (the number of adjacent triangular facets selecting the same texture source is 1 called isolated clustering) is about 35% of the total number of triangular facets, and the maximum clustering capacity is less than 20. After optimization, the isolated clustering is reduced to 15%, and the maximum clustering capacity is more than 40, which shows our optimization algorithm has a significant effect. (a) Table 1 is the efficiency comparison of energy minimization(b) by Graph-Cuts with four different simplification mesh surface models. It be seen from Table that the processing time Figure 15. Comparison of3D thetriangular statistical results of clustering: (a)can Before (b)1After clustering. Figure 15. Comparison of the statistical results of clustering: (a)clustering; Before clustering; (b) After clustering. is directly proportional to the number of triangular facets, and it can converge with the global minimization quickly, the number of iterations is around 5, meanwhile, the adjacent iterative energy Figure 15 shows the1clustering statistic results of ofenergy the triangular facets the 3D triangular mesh Table is the efficiency comparison minimization byin Graph-Cuts with four different shows a decreasing trend. It should be noted that the local energy minimum value perhaps appears 3D triangular mesh surface models.the It can be seen from Table 1 that the processing surface model simplification (3117 triangular facets). Figure 15a shows texture selection distribution before because the observed value is insufficient when the data set of the triangular facets is small. However, time is directly proportional to the number of triangular facets, and it can converge with the clustering, Figure 15b is the result after clustering. According to the specific statistical dataglobal of minimization quickly, the number of iterations is around 5, meanwhile, the adjacent iterative energy clustering results, before clustering and selection optimization the isolated clustering (the number of shows a decreasing trend. It should be noted that the local energy minimum value perhaps appears adjacent triangular facets selecting the same texture source is 1 called isolated clustering) is about because the observed value is insufficient when the data set of the triangular facets is small. However, 35% of the totalour number of triangular facets, and the maximum clustering capacity is less than 20. constructed energy function can skip the local minimum value, and finally obtains the global After optimization, the isolated reduced to 15%, andand thehas maximum clustering capacity minimization. Thisclustering shows thatisour algorithm is stable high convergence efficiency and is more than 40,strong which shows our optimization algorithm has a significant effect. robustness.

Table 1 is the efficiency comparison of energy minimization by Graph-Cuts with four different simplification 3D triangular mesh surface models. It can be seen from Table 1 that the processing time is directly proportional to the number of triangular facets, and it can converge with the global minimization quickly, the number of iterations is around 5, meanwhile, the adjacent iterative energy shows a decreasing trend. It should be noted that the local energy minimum value perhaps appears

Sensors 2017, 2017, 17, 17, 911 911 Sensors

13 of of 15 15 13

our constructed constructed energy energy function function can can skip skip the the local local minimum minimum value, value, and and finally finally obtains obtains the the global global our minimization. This shows that our algorithm is stable and has high convergence efficiency and strong minimization. This shows that our algorithm is stable and has high convergence efficiency and strong Sensors 2017, 17, 911 13 of 15 robustness. robustness. Table 1. 1. Efficiency Efficiency comparison comparison of of energy energy minimization minimization by by Graph-Cuts. Graph-Cuts. Table Table 1. Efficiency comparison of energy minimization by Graph-Cuts. The Number Number of of Convergent Rate Rate of of Number of of Energy Difference Difference of of The Convergent Number Energy Triangular Facets Energy Function Iterations Adjacent Iteration Iteration The Facets Number of Convergent Rate of Number Energy Difference of Triangular Energy Function Iterations Adjacent 3117 7.678Function 58.4, 5.8, 6.7, 6.7, 4.2 4.2 Triangular Facets Energy of Iterations Adjacent Iteration 3117 7.678 ss 55 58.4, 5.8, 7643 3117 12.488 s 6 97.2, 68.4, 30.0, 3.6, 1.0 7643 12.488 s 6 97.2, 68.4, 30.0, 3.6, 1.0 7.678 s 5 58.4, 5.8, 6.7, 4.2 32,522 35.553 s 5 351.1, 66.8, 58.5, 20.2 7643 12.488 s 6 97.2, 68.4, 30.0, 3.6, 1.0 32,522 35.553 s 5 351.1, 66.8, 58.5, 20.2 136,83432,522 162.729 1178.2, 278.3, 36.3 35.553ss s 5 44 351.1, 66.8, 58.5,278.3, 20.2 36.3 136,834 162.729 1178.2, 136,834 162.729 s 4 1178.2, 278.3, 36.3

6.3.4. Effect Effect Analysis Analysis of of 3D 3D Texture Texture Reconstruction Reconstruction 6.3.4. 6.3.4. Effect Analysis of 3D Texture Reconstruction Figure 16 16 shows shows the the experimental experimental results results of of 3D 3D texture texture reconstruction reconstruction of of our our proposed proposed method. method. Figure The left left side16 ofshows Figurethe 16 is isexperimental the global global view view of the the 3D texture model in the the experimental experimental area, the the right Figure results of 3D 3D texture texturemodel reconstruction of our proposed method. The side of Figure 16 the of in area, right side of Figure 16 is the zoom view of the local 3D texture model. We can easily see the experimental The left side of Figure 16 is the global view of the 3D texture model in the experimental area, the right side of Figure 16 is the zoom view of the local 3D texture model. We can easily see the experimental results are better better tothe maintain the consistency consistency of3D color, realistic effects, andeasily rich details. details. Furthermore, side of Figure 16 is zoom view of the localof texture model. We can see the Furthermore, experimental results are to maintain the color, realistic effects, and rich in order order tobetter more to intuitively demonstrate theofeffectiveness effectiveness and superiority ofdetails. the proposed proposed method, results are maintain demonstrate the consistency color, realistic effects, and rich Furthermore, in in to more intuitively the and superiority of the method, our method is compared with the famous 3D reconstruction software Photomodeler. As shown in order to more intuitively demonstrate the effectiveness and superiority of the proposed method, our method is compared with the famous 3D reconstruction software Photomodeler. As shown in Figure 17, there is some local amplification of the 3D texture model reconstructed by Photomodeler our method is compared withamplification the famous 3D reconstruction software Photomodeler. As shown in Figure 17, there is some local of the 3D texture model reconstructed by Photomodeler and our proposed method. From the areas indicated by the arrows in Figure 17, we can see that there there Figure 17, there is some local amplification of the 3D texture model reconstructed by Photomodeler and our proposed method. From the areas indicated by the arrows in Figure 17, we can see that are lots of texture inconsistencies, mapping errors, and seams in the results which are reconstructed and our proposed method. From the areas indicated by the arrows in Figure 17, we can see that there are lots of texture inconsistencies, mapping errors, and seams in the results which are reconstructed by Photomodeler. Photomodeler. After our our texture texture optimization, the problems suchresults as texture texture fragments and texture texture are lots of texture inconsistencies, mapping errors, the andproblems seams in such the which are reconstructed by by After optimization, as fragments and seams are significantly reduced, and the texture color continuity and integrity have been well Photomodeler. After our texture optimization, the problems such as texture fragments and texture seams seams are significantly reduced, and the texture color continuity and integrity have been well maintained. are significantly reduced, and the texture color continuity and integrity have been well maintained. maintained.

Figure 16. 16. Texture Texture reconstruction reconstruction results. results. Figure reconstruction results.

Figure 17. 17. Contrast Contrast of of local local zoom zoom views views of of texture texture optimization. optimization. Figure

7. Conclusions In this paper, the experimental images are obtained by oblique photography technology which is used for 3D surface model reconstruction and as a texture extraction source. The main innovations are as follows: with respect to image radiation correction, the camera response function is introduced

Sensors 2017, 17, 911

14 of 15

to deal with image reprocessing; with respect to occlusion detection, an efficient OpenGL detection method and a partial quartering algorithm are proposed; with respect to texture extraction, the discrete multi-label problem is solved by Graph-Cuts under the frame of MRF in texture selection optimization. Through the objective comparison of experimental results, the novel method of 3D urban texture model reconstruction is proved effective; can improve clustering expression and radiation consistency of texture mapping; and demonstrates advantages of high efficiency, fine automation level, realistic effects, and low cost. Acknowledgments: This study is supported by the National Key Research and Development Program of China (2016YFB0502202), the NSFC (91638203), the State Key Laboratory Research Expenses of LIESMARS. Author Contributions: W.Z. and M.L. proposed the idea and wrote the manuscript; B.G. and D.L. conceived and designed the experiments; G.G. performed experiments. Conflicts of Interest: The authors declare no conflict of interest.

References 1. 2. 3. 4. 5. 6. 7.

8. 9. 10. 11.

12. 13.

14. 15. 16. 17.

Goesele, M.; Curless, B.; Seitz, S. Multi-view stereo revisited. In Proceedings of the Computer Vision and Pattern Recognition, New York, NY, USA, 17–22 June 2006; Volume 2, pp. 2402–2409. Kazhdan, M.; Bolitho, M.; Hoppe, H. Poisson surface reconstruction. In Proceedings of the Symposium on Geometry Processing, Cagliari, Italy, 26–28 June 2006; pp. 61–70. Brown, M.; Lowe, D. Automatic Panoramic Image Stitching using Invariant Features. Int. Comput. Vis. 2007, 74, 59–73. [CrossRef] Furukawa, Y.; Ponce, J. Accurate, dense, and robust multi-view stereopsis. IEEE Trans. Pattern Anal. Mach. Intell. 2010, 32, 505–512. [CrossRef] [PubMed] Schoning, J.; Heidemann, G. Evaluation of multi-view 3D reconstruction software. In Proceedings of the Computer Analysis of Images and Patterns, Valletta, Malta, 2–4 September 2015; pp. 450–461. Soucy, M.; Godin, G.; Rioux, M. A texture-mapping approach for the compression of colored 3D triangulations. Vis. Comput. 1996, 12, 503–514. [CrossRef] Debevec, P.; Yu, Y.; Boshokov, G. Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping. In Proceedings of the Eurographics Symposium on Rendering Techniques, Vienna, Austria, 29 June–1 July 1998; pp. 105–116. Bernardini, F.; Martin, I.; Rushmeier, H. High-quality texture reconstruction from multiple scans. IEEE Trans. Vis. Comput. Graph. 2001, 7, 318–332. [CrossRef] Baumberg, A. Blending images for texturing 3D models. In Proceedings of the British Machine Vision Conference, Cardiff, UK, 2–5 September 2002; pp. 404–413. Kim, C.; Rhee, E. Realistic Facade Texturing of Digital Building Models. Softw. Eng. Appl. 2014, 8, 193–202. [CrossRef] Luo, Q.; Zhou, G.; Zhang, G.; Hang, J. The texture extraction and mapping of buildings with occlusion detection. In Proceedings of the International Geoscience and Remote Sensing Symposium, Milan, Italy, 26–31 July 2015; pp. 3002–3005. Frueh, C.; Jain, S.; Zakhor, A. Data Processing Algorithms for Generating Textured 3D Building Facade Meshes from Laser Scans and Camera Images. Int. J. Comput. Vis. 2005, 61, 159–184. [CrossRef] Iwaszczuk, D.; Hoegner, L.; Stilla, U. Quality-based building-texture selection from different sensors. In Proceedings of the Urban Remote Sensing Event (JURSE), Lausanne, Switzerland, 30 March–1 April 2015; pp. 1–4. Gal, R.; Wexler, Y.; Hoppe, H.; Cohenor, D. Seamless Montage for Texturing Models. Comput. Graph. Forum 2010, 29, 479–486. [CrossRef] Li, D.; Wang, M.; Pan, J. Auto-dodging processing and its application for optical RS image. Geomat. Inf. Sci. Wuhan Univ. 2006, 31, 753–756. Luo, S. Improved dodging algorithm based on Wallis principle. Geomat. Sci. Technol. 2015, 3, 51–58. [CrossRef] He, K.; Tang, X. Single Image Haze Removal Using Dark Channel Prior. IEEE Tran. Pattern Anal. Mach. Intell. 2011, 33, 2341–2353.

Sensors 2017, 17, 911

18. 19. 20. 21. 22. 23. 24.

15 of 15

Dangelo, P. Radio metric alignment and vignetting calibration. In Proceedings of the International Conference on Computer Vision System, Bielefeld, Germany, 21–24 March 2007. Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision, 2nd ed.; Cambridge University Press: London, UK, 2000. Zhang, Z.; Su, G.; Zhen, S.; Zhang, J. Relation OpenGL imaging process with exterior and interior parameters of photogrammetry. Geomat. Inf. Sci. Wuhan Univ. 2004, 29, 570–574. Boykov, Y.; Veksler, O.; Zabih, R. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 2001, 23, 1222–1239. [CrossRef] Schmidt, M.; Alahari, K. Generalized Fast Approximate Energy Minimization via Graph Cuts: Alpha-Expansion Beta-Shrink Moves. In Proceedings of the Uncertainty in Artificial Intelligence, Barcelona, Spain, 14–17 July 2011. Hussmann, S.; Ringbeck, T.; Hagebeuker, B. A Performance Review of 3D TOF Vision Systems in Comparison to Stereo Vision Systems; InTech: Vienna, Austria, 2008. Zhang, C.; Zhang, W.; Guo, B.; Liu, J.; Li, M. Rapidly 3D texture reconstruction based on oblique photograph. Acta Geod. Cartogr. Sin. 2015, 44, 782–790. © 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).