remote sensing Article
Frescoed Vaults: Accuracy Controlled Simplified Methodology for Planar Development of Three-Dimensional Textured Models Marco Giorgio Bevilacqua 1,† , Gabriella Caroti 2,† , Isabel Martínez-Espejo Zaragoza 2,† and Andrea Piemonte 2, *,† 1 2
* †
Energy, Systems, Territory and Construction Engineering Department (DESTeC), University of Pisa, Largo Lucio Lazzarino 1, 56122 Pisa, Italy;
[email protected] Civil and Industrial Engineering Department (DICI), University of Pisa, Largo Lucio Lazzarino 1, 56122 Pisa, Italy;
[email protected] (G.C.);
[email protected] (I.M.-E.Z.) Correspondence:
[email protected]; Tel.: +39-050-221-7773 These authors contributed equally to this work.
Academic Editors: Diego Gonzalez-Aguilera, Fabio Remondino, Norman Kerle and Prasad S. Thenkabail Received: 30 November 2015; Accepted: 7 March 2016; Published: 14 March 2016
Abstract: In the field of documentation and preservation of cultural heritage, there is keen interest in 3D metric viewing and rendering of architecture for both formal appearance and color. On the other hand, operative steps of restoration interventions still require full-scale, 2D metric surface representations. The transition from 3D to 2D representation, with the related geometric transformations, has not yet been fully formalized for planar development of frescoed vaults. Methodologies proposed so far on this subject provide transitioning from point cloud models to ideal mathematical surfaces and projecting textures using software tools. The methodology used for geometry and texture development in the present work does not require any dedicated software. The different processing steps can be individually checked for any error introduced, which can be then quantified. A direct accuracy check of the planar development of the frescoed surface has been carried out by qualified restorers, yielding a result of 3 mm. The proposed methodology, although requiring further studies to improve automation of the different processing steps, allowed extracting 2D drafts fully usable by operators restoring the vault frescoes. Keywords: frescoed vault; vault development; restoration
1. Introduction Planar development of more or less complex curved surfaces, also possibly decorated, is of great interest for experts working in the field of documentation and preservation of cultural heritage. The scientific community has addressed the problem in a rigorous way, with both analytical [1] and digital [2–6] methodologies. These investigations have led to the development of software applications that have only partially automated the procedures required for planar development of curved surfaces. Studies carried out by the authors have not found as yet any commercial tool allowing automated and controlled development of geometry and texture. The goal of this research was to study a simplified methodology enabling both planar development of geometry and texture of frescoed vaults (surveyed with geomatics techniques) and checking of the errors related to the different operating steps. In particular, this methodology has been developed in the case of the “a schifo” vault typology [7,8] and does not provide any kind of cartographic projection. It uses very common commercial software and includes some processing steps requiring user operation.
Remote Sens. 2016, 8, 239; doi:10.3390/rs8030239
www.mdpi.com/journal/remotesensing
Remote Sens. 2016, 8, 239
2 of 16
2. State of the Art 3D photorealistic environments allow engineers, historians and restorers to research, investigate, and simulate outcomes of restoration projects before these are executed. For all these aspects, 3D-textured metric modeling is currently the most sought after tool for cognitive evaluation and operating approach in the field of cultural heritage [9]. Creation of a 3D-textured model includes three steps: geometry modeling, parameterization and texture creation. 2.1. Geometry Survey Laser scanning is a well-established surveying methodology, whose output is readily usable for representing historical and architectural heritage [10–17]. Accuracy and resolution attainable in comparatively short times (last-generation scanners for architectural surveys can acquire millions of points per second with sub-centimeter accuracy) are the main strengths of these systems although prices are still quite high. The new approach to softcopy photogrammetry realized by Structure from Motion (SfM) and MVS (Multi-View Stereo) algorithms generates very dense 3D color point clouds quite similar in size to those produced from laser scanning surveys [18–23]. However, even if software evolution in this field is very fast and performance is good in terms of processing time, and the amount of manageable data and obtainable precisions are gradually improving, these procedures may not always be considered reliable. In fact, matching algorithms can be very sensitive to recording and illumination differences and not reliable in poorly textured or homogeneous regions. This can result in noisy point clouds and/or difficulties in feature extraction [24]. These matching algorithms could suffer from variable precision, strongly dependent on the pattern present on surveyed objects, as well as the difficulty of having control of the achievable accuracy at the geometric and morphological levels [20,25]. 2.2. Texture Mapping In large-scale 3-D models used as supporting documentation in restoration works, textures are not a mere aesthetic complement. In fact, besides supporting construction, material and chromatic studies, they also act as metric surveying tools, providing, once applied to the models, a guideline for measurements. Therefore, if textures have to meet these requirements, their positioning accuracy must be consistent with the scale used, besides having the necessary chromatic precision [26]. Many laser scanners have built-in cameras, whose relative orientation is calibrated by the manufacturer, which allow direct true coloring of point clouds. These textures are characterized by a high geometric accuracy, but the systems used for the photographic takes usually do not achieve good results in terms of resolution and color fidelity [27]. Simplified, realistic-looking models may not suffice for restorers, who require rigorous texture mapping for both morphology and color information. In these cases, it is essential to resort to a dedicated photographic campaign, performed with high quality cameras as regards optics, sensor size and image post-processing. In the case of SfM software, the creation of models and textures is almost contextual, and the procedure usually involves self-calibration of the camera, which also takes account of characteristic distortion parameters. In these models, although textures usually have good photographic quality, it is necessary to check the overall morphological reliability. 2.3. Vault Development Commercial and open-source software currently available are capable to render architectures in 3D as regards formal appearance and color. On the other hand, operative steps of restoration interventions still require large-scale, 2D metric surface representations. The transition from 3D to 2D representation, with the related geometric transformations, has not yet been fully formalized and still features open issues, e.g., in the case of planar development of frescoed vaults [28]. Methodologies
Remote Sens. 2016, 8, 239
3 of 16
proposed so far on this subject provide transitioning from point cloud models to known mathematical surfaces (developed on plane, or not), and afterwards seeking an ideal representation of the actual surface, losing some architectural and building details in this process [29–33]. To the best knowledge of the authors, modeling and reverse engineering software commonly used do not have dedicated tools that enable automatic development of geometry and textures. Moreover, the tools that only partially solve the problem do not take account of the introduced deformations. 3. Materials and Methods In order to achieve planar development of frescoed vaults, textured 3D models of the vaults are required. These should feature good geometric accuracy to identify the ideal geometric root that best fits the actual vault. Textures associated with the models will have good radiometric quality and true colors, and will also faithfully reproduce position and dimension of any fresco detail. 3.1. The Case Study The object of this investigation is a vault in Palazzo Roncioni (Pisa, Italy). Its entire surface is covered by a XVIII century fresco, painted by Tuscan painter G.B. Tempesti, which has over time undergone extensive damage (cracks, plaster gaps, etc.). Currently, the vault is the subject of safety and restoration work. A laser scanning survey of the vault has been performed with the pulse shift-based laser scanner Leica Geosystems C10 ScanStation, with a point cloud density of about 70 pts/cm2 on average. Use of a phase-based laser scanner would have allowed for more accurate results at short distances and therefore less noisy reference data. The photogrammetric survey was performed with a Nikon D700 SLR camera (f = 20 mm lens) at about 4.5 m range, ensuring a roughly 2-mm pixel covering on the vault surface. It has been subjected to image processing via SfM algorithms, granting an overlap ě70%. 3.2. 3D Modeling and Texturing A separate 3D TIN model has been built by means of each surveying methodology. Both models have been rigorously registered in the same reference system due to the extrapolation, from the colored point cloud, of the coordinates of 12 Ground Control Points (GCPs). These have been chosen as easily identifiable fresco details, spread evenly across the entire vault, and have been used in the processes of scaling and rototranslation of the photogrammetric model. The model obtained via laser scanning survey, henceforth referred to as “model LASER” (Figure 1c), features homogenous geometric precision and a high enough resolution to show elements in the sub-centimeter range (cracks, plaster displacements, etc.). As regards textures, images collected via the on-board camera (single image 17˝ ˆ 17˝ , 1920 ˆ 1920 pixel) do not grant adequate radiometric quality. The model obtained by means of SfM/MVS methodology, henceforth referred to as “model SfM/MVS”, features uneven geometric precision, mostly in the areas where radiometric uniformity reduces the performance of the SfM algorithms (Figure 1b). On the other hand, it has been obtained by a photographic campaign executed with a high quality camera, so that despite some local errors in detail rendering, image orientation is substantially fit to the needs of the application, as detailed later. A new textured model (“model SfM/LASER”) has been generated via SfM software from the geometry of model LASER and the images orientated for creation of model SfM/MVS (Figure 1a). Model SfM/LASER is the best result for geometry and texture quality, starting from collected data. As regards processing time, this texturing process definitely has lower requirements than single image orientation based on GCPs [26].
Remote Sens. 2016, 8, 239 Remote Sens. 2016, 8, 239
44of of16 16 4 of 16
Figure 1. (a)Model Model SfM/LASER; (b) Model SfM/MVS; SfM/MVS; (c) Model LASER. Figure (c)Model ModelLASER. LASER. Figure1.1.(a) (a) ModelSfM/LASER; SfM/LASER; (b) Model SfM/MVS; (c)
3.3. Vault Development 3.3. Vault Development Development 3.3. Vault Point clouds, either either collected collected by by laser laser scanning scanning (Cloud (Cloud LASER) LASER) or obtained by photogrammetry Point or obtained obtained by by photogrammetry photogrammetry Point clouds, clouds, either collected by laser scanning (Cloud LASER) or (Cloud SfM/MVS), have been framed in a single reference system. The Xand Y-axes lie in the the (Cloud SfM/MVS), have been framed in a single reference system. The Xand Y-axes lie lie in in (Cloud SfM/MVS), have been framed in a single reference system. The X- and Y-axes the speculated vault impost impost plan, which which is not not horizontal horizontal (axes (axes origin origin in in a corner, X-axis on the the long side side speculated corner, X-axis speculated vault vault impost plan, plan, which is is not horizontal (axes origin in aa corner, X-axis on on the long long side and Y-axis on the short side) and the Z-axis completes the orthogonal triplet. In order to proceed with and the short short side) side) and and the the Z-axis Z-axis completes completes the the orthogonal orthogonal triplet. In order order to to proceed proceed with with and Y-axis Y-axis on on the triplet. In the 2D 2D vault vault development, development, the the 3D 3D model model has has been been analyzed. analyzed. the the 2D vault development, the 3D model has been analyzed. Analysis and Preliminary Processing of Laser Laser Data 3.3.1. 3.3.1. Analysis and Preliminary Processing of Laser Data Data order todefine define thegeometric geometric components that constitute the vault, a dense contour (step =2 In components that constitute thethe vault, a dense contour (step(step = 2 cm) In order orderto to definethe the geometric components that constitute vault, a dense contour =2 cm) representation of model LASER has been generated according thethree threecoordinate coordinateplanes planes(Figure (Figure 2). representation of model LASER has been generated according the cm) representation of model LASER has been generated according the three coordinate planes (Figure 2).
Dense contour model (isometric view). Figure 2. Figure 2. Dense contour model (isometric view).
The study of this contour representation (Figure 3) has allowed identification of nine studyof of contour representation 3) hasidentification allowed identification of nine The study thisthis contour representation (Figure(Figure 3) has allowed of nine discontinuity discontinuity directions that divide the vault in 6 areas, each featuring its own section profiles with discontinuity divide the vault 6 areas, its each featuring its own with section profiles with directions thatdirections divide thethat vault in 6 areas, eachin featuring own section profiles almost constant almost constant radius: areas 2, 4 and 6, close to the vault impost, have greater section profile radii almost constant 2, the 4 and 6, close to the vault impost, have greater section radius: areas 2, 4radius: and 6, areas close to vault impost, have greater section profile radii than profile those inradii the than those in the upper part of the vault (areas 1, 3 and 5). Separation between the lower and upper than those in the upper part of the vault (areas 1, 3 and 5). Separation between the lower and upper upper part of vault (areas 1, 3 and 5). Separation between the lower and upper parts of the vault is parts of the vault is located at about one third of the vault height above the impost plane. parts of at the vaultone is located about one thirdabove of thethe vault height above the impost plane. located about third ofatthe vault height impost plane.
Remote 2016, 8, Remote Sens. Sens. 2016, 8, 239 239 Remote Sens. 2016, 8, 239
55of of16 16 5 of 16
Figure 3. Discontinuity directions and vault areas (bottom view). Figure view). Figure 3. 3. Discontinuity Discontinuity directions directions and and vault vault areas areas (bottom (bottom view).
The interpretation of these results suggested that the vault could be generated from the The interpretation of these results suggested that the vault could be generated from the combination of several elements to different cylindrical surfaces, could be part of the the The interpretation of thesebelonging results suggested that the vault could and be generated from combination of several elements belonging to different cylindrical surfaces, and could be part of the “a schifo” type. includes a lower portion, to acylindrical section of surfaces, a pavilionand vault, andbe anpart upper combination of This several elements belonging tosimilar different could of “a schifo” type. This includes a lower portion, similar to a section of a pavilion vault, and an upper one, named “specchio” (mirror), which features so wide a curvature to appear almost planar. This the “a schifo” type. This includes a lower portion, similar to a section of a pavilion vault, and an upper one, named “specchio” (mirror), which features so wide a curvature to appear almost planar. This vaultnamed type has been widely used in features architecture since Renaissance exactly in the case This of fresco one, “specchio” (mirror), which so wide a curvature to appear almost planar. vault vault type has been widely used in architecture since Renaissance exactly in the case of fresco decorations. type has been widely used in architecture since Renaissance exactly in the case of fresco decorations. decorations. Once the the hypotheses hypotheses about type of Once about the the building building type of the the vault vault have have been been substantiated, substantiated, the the values values of of Once the hypotheses about the building type of the vault have been substantiated, the values of the geometric geometricparameters parameters(axis (axisand and radius) of the elementary cylindrical surfaces that fit best the the radius) of the elementary cylindrical surfaces that best thefit point the geometric parameters (axis and radius) of the elementary cylindrical surfaces that best fit the point clouds ofof each six areas detected were computed by means of approximation algorithms. clouds of each theof sixthe areas detected were computed by means of approximation algorithms. point clouds of each of the six areas detected were computed by means of approximation algorithms. As an example, approximation by cylindrical surfaces of the long side (Figure 4) yielded the As an example, approximation by cylindrical surfaces of the long side (Figure 4) yielded the As an example, approximation by cylindrical surfaces of the long side (Figure 4) yielded the following results results (Table (Table 1). 1). following following results (Table 1).
Figure 4. Approximation by cylindrical surface. Figure 4. Approximation by cylindrical surface.
Remote Sens. 2016, 8, 239
6 of 16
Remote RemoteSens. Sens.2016, 2016,8, 8,239 239
of16 16 66of
Table 1. Approximation by cylindrical surface—radius. Table Table 1. 1. Approximation Approximation by cylindrical surface—radius.
Area Area11 Radius Radius (m) Radius (m) 7.349 7.349 STDV (m) 0.007 STDV 0.007 STDV (m)
Area 2 Area 2 5.280 0.008
5.280 0.008
Analysis Standard Deviation (STDV) should should take vault does not actually Analysis of StandardDeviation Deviation(STDV) does not actually Analysis ofof Standard takein inaccount accountthat thatthe thevault vault does not actually show neat transitions between contiguous cylindrical surfaces, but rather the curvature radius show transitions between contiguous the curvature show neatneat transitions between contiguous cylindrical surfaces, but rather the curvature radiusradius changes changes gradually. In fact, the higher values of the difference and actual surfaces are in changes gradually. In fact, the higher between ideal and actual surfaces are gradually. In fact, the higher values of the difference between ideal and actual surfaces are found found foundin inthese these transition transition areas areas (Figure (Figure 5). these transition areas (Figure 5).
Figure5.5. 5.Actual Actual3D 3D model—Cylindrical model—Cylindrical surface Figure surfaceerror. error. Figure Actual 3D model—Cylindrical surface error.
3.3.2. Analysis of the DevelopmentMethodology Methodology 3.3.2. Analysis ofof the Development 3.3.2. Analysis the Development Methodology In order order to achieve planardevelopment development of of vault vault geometry geometry well known and achieve planar development and texture using well known and In In order toto achieve a aaplanar geometryand andtexture textureusing using well known and easily available software tools, a methodology using model representation by contours, rather than easily available software tools, a methodologyusing usingmodel modelrepresentation representationby bycontours, contours,rather ratherthan than by easily available software tools, a methodology byideal idealshapes shapes such such as as cylinders, cylinders, has has been investigated investigated and applied to this case study. by applied this case study. ideal shapes such as cylinders, has beenbeen investigated andand applied to to this case study. For this purpose, just the contour lines lying in the XY coordinate plane have been used in aa this purpose, just contour lines lying in the coordinate plane have been used ForFor this purpose, just thethe contour lines lying in the XYXY coordinate plane have been used in ainCAD CAD environment, with a 20-cm step (Figure 6). CAD environment, with a 20-cm step (Figure 6).
environment, with a 20-cm step (Figure 6).
Figure 6. 20-cm step contour lines model.
Figure6.6.20-cm 20-cm step contour Figure contour lines linesmodel. model.
Remote Sens. 2016, 8, 239 Remote Sens. 2016, 8, 239
77of of16 16 7 of 16
It has been assumed that vault sections between adjacent contours were planar. Contours have that vault between adjacent contours werethe planar. Contours have Ithas hasbeen been assumed thatbetween vaultsections sections between adjacent contours were planar. have beenItassumed asassumed connections adjacent planes. In order to appraise errorContours introduced by been assumed as connections between adjacent planes. In order to appraise the error introduced been assumed asanconnections In order to appraise the checked. error introduced byby this assumption, orthogonalbetween section adjacent of the interpolating cylinders has been In the most this an of cylinders has beenchecked. checked. the most thisassumption, assumption, anorthogonal orthogonal section of the the interpolating interpolating has been the most unfavorable situation (Figure 7),section the difference between thecylinders section arc Equation (1) andInIn the related unfavorable situation (Figure 7), the difference the section arc Equation (1) and the related unfavorable situation (Figure 7), the difference between the section arc Equation (1) and the related chord Equation (2), bounded by two adjacent contours, has been computed. chordEquation Equation(2), (2),bounded boundedby bytwo twoadjacent adjacentcontours, contours, has has been computed. chord computed. (1) AB = β∙R (1) AB = β∙R x AB “ β¨ R (1) (2) AB = 2∙R∙ sin β⁄2 (2) AB = 2∙R∙ sin β⁄2 AB “ 2¨ R¨ sin pβ{2q (2)
Figure 7. Section arc—chord arc—chord comparison. Figure comparison. Figure7. 7.Section Section arc—chord comparison.
On was 33 mm, mm, with with aa sub-millimeter sub-millimeterrelative relative Onananarc arclength length==1.718 1.718m, m,the themaximum maximum difference difference was error. approximation has been deemed acceptable. planar development the XY contour error. This approximation has been deemed acceptable. For the of error. This approximation has been deemed acceptable. For the planar development of the XY contour model with 20-cm step (the same used detect thedifferent differentportions portions of the vault), the crown lines ofof the vault), the crown of linesmodel modelwith withaaa20-cm 20-cmstep step(the (thesame sameused usedtoto todetect detectthe the different portions the vault), the crown ofofthe vault has been outlined in CAD at 1:1 scale (line AB in Figure 8). the vault has been outlined in CAD at 1:1 scale (line AB in Figure 8). the vault has been outlined in CAD at 1:1 scale (line AB in Figure 8).
Figure8.8.Geometric Geometric development development of Figure of contours contours model. model. Figure 8. Geometric development of contours model.
Remote Sens. 2016, 8, 239 Remote Sens. 2016, 8, 239
8 of 16 8 of 16
Subsequent contour lines have been separately developed by trilateration for each surface of the vault. Assumingcontour the extremes of thebeen previous contour as fixed points, the extremes of the next contour Subsequent lines have separately developed by trilateration for each surface of the have Assuming been plotted. Figure 8 shows how planar geometric development of the actual the vault. the extremes of the previous contour as fixed points, the extremes of thevault next lacks contour regular course found in the development of angeometric ideal surface constitutedofjust portions cylinder have been plotted. Figure 8 shows how planar development the by actual vaultof lacks the with parallel regular courseaxes. found in the development of an ideal surface constituted just by portions of cylinder Sectionsaxes. defined by ZX (blue lines) and ZY (red lines) planes have then been superimposed on with parallel the Sections development of the (green defined by XY ZX sections (blue lines) andlines). ZY (red lines) planes have then been superimposed on Besides geometric vault development, sections are also required as a reference for correct texture the development of the XY sections (green lines). placement the developed model. Besideson geometric vault development, sections are also required as a reference for correct texture In order to developed apply textures to the developed geometric model, the following methodology has placement on the model. been Inchosen. order to apply textures to the developed geometric model, the following methodology has For each surface, eight directions have been identified to set orthogonal views of 3D model been chosen. SfM/LASER. directions are orthogonal to theidentified axis of theto theoretical cylinders and,ofstarting from For each These surface, eight directions have been set orthogonal views 3D model the horizontal view, tilted byare 15°orthogonal relative to to thethe previous view. SfM/LASER. These directions axis of the theoretical cylinders and, starting from For eachview, viewing direction, imagestoofthe theprevious model ofview. vault portion bounded by a 15° cylindrical the horizontal tilted by 15˝ relative ˝ cylindrical arc For have been collected. Theseimages are anoforthogonal of thebounded vault texture an orthogonal each viewing direction, the model projection of vault portion by a 15on arc plane relative to the viewing direction [34]. Fresco elements projected in this way are obviously have been collected. These are an orthogonal projection of the vault texture on an orthogonal plane deformed. relative to the viewing direction [34]. Fresco elements projected in this way are obviously deformed. Acceptingthe thesimplification simplificationthat thatthe thevault vaultisisrepresented representedby bythe thesurfaces surfacesofofthe theinterpolating interpolating Accepting cylinders,ititisispossible possibletotoquantify quantifythis thisdeformation. deformation.As Asfor forthe theorthogonal orthogonalprojection projectionon onaaplane plane cylinders, tangenttotothe thecylinder, cylinder,there thereisisno nodeformation deformationalong alongparallel paralleldirections directionsrelative relativetotothe thetangency tangencyline, line, tangent while deformation is highest along orthogonal directions. Linear deformation module (m l ) at the while deformation is highest along orthogonal directions. Linear deformation module (ml ) at the extremesofofthe theorthogonally orthogonallyprojected projectedarea areaisisdefined definedbybyEquation Equation(3). (3). extremes ⁄2 AB' 2∙ ∙ ml =AB1 = 2¨ R¨ sin pβ{2q R ∙ β ml “ AB “
(3) (3) x R¨ β AB In the projection used, deformation is highest in the furthest point relative to the tangency line the projection used, deformation highest furthest point relative the (Figure tangency (forIn breadth = 15°, distance is about 1 m),iswhere min l = the 0.9970 and deformation = 3tomm 9).line (for breadth = 15˝ , distance is about 1 m), where ml = 0.9970 and deformation = 3 mm (Figure 9).
Figure 9. Projection deformation. Figure 9. Projection deformation.
InInaccordance accordancewith withthe theoperators operatorsdealing dealingwith withthe therestoration restorationofofthe thefresco, fresco,this thisdeformation deformationhas has been beendeemed deemedacceptable. acceptable. Each view of model SfM/LASER has been performed in twoinconfigurations and saved Eachorthogonal orthogonal view of model SfM/LASER has been performed two configurations and insaved two separate image files. Configuration 1 provided superimposing the section the linessection to the model in two separate image files. Configuration 1 for provided for superimposing lines to (Figure 10a).(Figure Configuration 2 viewed the2model with the high applied (Figure 10b). the model 10a). Configuration viewed thejust model with quality just thetexture high quality texture applied (Figure 10b).
Remote Sens. 2016, 8, 239 Remote Sens. 2016, 8, 239 Remote Sens. 2016, 8, 239
9 of 16 9 of 16 9 of 16
Figure10. 10.(a) (a)Orthogonal Orthogonal view view with with section lines; (b) Figure (b) Orthogonal Orthogonalview viewwith withhigh highquality qualitytexture. texture. Figure 10. (a) Orthogonal view with section lines; (b) Orthogonal view with high quality texture.
Thefollowing following processingsteps steps have therefore therefore been run run for each image The image pair: pair: The followingprocessing processing stepshave have therefore been been run for for each each image pair: • Image pairs and the geometric vault development frame (Figure 8) have the ‚ • Image frame (Figure (Figure8) 8)have havebeen beenimported importedin the Imagepairs pairsand andthe thegeometric geometric vault vault development development frame been imported ininthe same photo editing software environment. same photo editing software environment. same photo editing software environment. singleblock block hasbeen been createdwith with both images, images, so any ‚ •• AA so that that any anytransformation transformationapplied appliedto anyone one Asingle single blockhas has beencreated created with both both images, so that any transformation applied totoany one image was similarly applied to the other. image was similarly applied to the other. image was similarly applied to the other. • The layer containing the image with just the texture has been turned off, leaving visible just the ‚ • The been turned turnedoff, off,leaving leavingvisible visiblejust justthe the Thelayer layercontaining containingthe theimage imagewith with just just the the texture has been image with the section lines. image imagewith withthe thesection sectionlines. lines. • The image has been scaled and moved on the geometric frame, assuming the section lines Theimage image been scaled and moved ongeometric the geometric frame, assuming the section lines ‚ • The hashas been scaled moved on the frame, the section lines obtained obtained with XZ and YZand planes (vertical lines in Figure 10) asassuming reference. obtained with and YZ planeslines (vertical lines 10) in Figure 10) as reference. with XZ and YZXZ planes (vertical in Figure as reference. As proof of the small deformation of the images, it has been noticed that after scaling the image Asproof proofofofthe thesmall smalldeformation deformation of of the the images, images, it has been As been noticed noticedthat thatafter afterscaling scalingthe theimage image in the direction of the axis of the interpolating cylinders for a single projection direction, it aligns with thedirection directionofofthe theaxis axisof ofthe the interpolating interpolating cylinders for aa single ininthe singleprojection projectiondirection, direction,ititaligns alignswith with images derived from other projection directions at less than the computed deformation (Figure 11). imagesderived derivedfrom fromother otherprojection projectiondirections directions at at less less than than the computed computed deformation images deformation(Figure (Figure11). 11).
Figure 11. Superimposition of orthogonal view on sections model development. Figure11. 11.Superimposition Superimposition of of orthogonal orthogonal view Figure view on on sections sections model modeldevelopment. development.
Remote Sens. 2016, 8, 239 Remote Sens. 2016, 8, 239
10 of 16 10 of 16
4. 4.Results Resultsand andDiscussion Discussion ordertotovalidate validatethe themethodology methodology used, used, results must precision InIn order must be bechecked checkedfor forboth bothgeometric geometric precision the differentmodels modelsobtained obtainedand and precision precision of placement, placement, dimension ofof the different dimensionand andshape shapeofofthe theapplied applied textures. Finally, the quality of the planar development of the vault was assessed (Figure 12). textures. Finally, the quality of the planar development vault was assessed (Figure 12).
Figure12. 12.High High quality quality textured textured model Figure modeldevelopment. development.
4.1. AssessmentofofModel ModelGeometrical GeometricalAccuracy Accuracy 4.1. Assessment Thelaser laserscanning scanningcolored coloredpoint point cloud cloud model model (Cloud The (Cloud LASER) LASER)can canbe beassumed assumedasasthe theabsolute absolute geometric reference in this application. It has very high point density, and allows extraction geometric reference in this application. It has very high point density, and allows extractionofof coordinates of features for both geometry (cracks, gaps, etc.) and painting (boundary lines, color coordinates of features for both geometry (cracks, gaps, etc.) and painting (boundary lines, color transitions, etc.) with a sub-centimeter resolution. transitions, etc.) with a sub-centimeter resolution. Standard deviation obtained by comparing Cloud LASER with Model LASER is 1 mm, with Standard deviation obtained by comparing Cloud LASER with Model LASER is 1 mm, with peaks peaks in the 3 mm range. These results highlight that the transition from point cloud to surface model in the 3 mm range. These results highlight that the transition from point cloud to surface model entails entails a small decay of geometric precision. a small decay of geometric precision. A second check has been performed comparing Cloud LASER with Cloud SfM/MVS; the A second check has been performed comparing Cloud LASER with Cloud SfM/MVS; the standard standard deviation averaged at 3 mm, with peaks of about 6 mm. deviation averaged 3 mm,has with peaks of about 6 mm.Model SfM/MVS; the standard deviation was Finally, CloudatLASER been compared against Finally, Cloud LASER has been compared against Model SfM/MVS; the standard deviation was 3 3 mm on average, peaking at about 10 mm. mm onThese average, peaking about 10orientation mm. results show at that image in SfM is substantially accurate and confirm the mean These results show that image orientation in SfM is substantially and confirm the mean reprojection error involved with orientating each image via SfM to beaccurate 0.70 pixel with an average of reprojection error involved with orientating each image via SfM to be 0.70 pixel with an average 9000 tie points per image. On the other hand, maximum deviation values are in the range of 7–10 mmof 9000 points per image. On thecollapse other hand, maximum values are in theof range of 7–10 mm andtie refer to cracks and plaster borders. Figuresdeviation 13–17 show an overview the fresco and and refer to cracks and plaster collapse borders. Figures 13–17 show an overview of the fresco and some details on local deviations. some details local deviations. Takingon into account all these cases, greater deviations are found when surveyed surfaces are Taking into account all these cases, greater does deviations are found whenthe surveyed surfaces are orthogonal to the vault. SfM/MVS methodology not correctly represent transitions typical of deep cracks delamination. This result isdoes in the duethe to transitions the fact that theseof orthogonal to the and vault. SfM/MVS methodology notauthors’ correctlyopinion represent typical surface regions are acquired by inclined with different inclinations sometimes with the deep cracks and delamination. This result views is in the authors’ opinion due to and the fact that these surface camera axis parallelby to inclined the surface. This fact,different reportedinclinations in the literature, leads to worse regions are acquired views with and sometimes withperformance the camera of axis the matching algorithms parallel to the surface. This[24]. fact, reported in the literature, leads to worse performance of the matching algorithms [24].
Remote Sens. 2016, 8, 239 Remote Sens. 2016, 8, 239 Remote Sens. Sens. 2016, 2016, 8, 8, 239 239 Remote
11 of 16 11 of 16 11 of of 16 16 11
Figure 13.Regions Regionschecked checkedfor fordeviations deviations between between cloud Figure 13. cloud LASER LASERand andmodel modelSfM/MVS. SfM/MVS. Figure 13. Regions Regions checked checked for for deviations deviations between between cloud cloud LASER and model SfM/MVS. Figure 13. LASER and model SfM/MVS.
Figure 14. Region A: total plaster collapse borders. Figure 14. 14. Region Region A: A: total total plaster collapse collapse borders. Figure Figure 14. Region A: total plaster plaster collapseborders. borders.
Figure 15. Region B: total plaster collapse borders. Figure15. 15. Region Region B: B: total total plaster plaster collapse borders. Figure plaster collapse collapseborders. borders. Figure 15. Region
Remote Sens. 2016, 8, 239 Remote Sens. 2016, 8, 239
12 of 16 12 of 16
Figure16. 16.Region RegionC: C:gap gapin inthe thefresco. fresco. Figure
Figure Figure17. 17.Region RegionD: D:crack crackin inthe thetopmost topmostregion regionof ofthe thevault. vault.
Hence, the the overall overall accuracy accuracy of of the the SfM-derived SfM-derived model model is is good good (3 (3 mm), mm), but but shows shows some someflaws flaws Hence, preciselyin inthe theregions regionsof ofmost mostinterest interestto to restorers. restorers. This This processing processing methodology, methodology,on onthe the other other hand, hand, precisely has the theadvantage advantage of ofsignificantly significantlylower lowerresource resourcerequirements: requirements: manual manual intervention intervention isis limited limited to to has inputting the support points to orient and scale the model. inputting the support points to orient and scale the model. Theseconsiderations considerationson ongeometric geometricprecision precisionled ledtotothe thechoice choiceofofModel ModelSfM/LASER SfM/LASERas asaastarting starting These modelfor forvault vaultdevelopment. development. model 4.2. 4.2.Texture TextureDimension Dimensionand andPositioning PositioningAccuracy AccuracyAssessment Assessment After After geometric geometric accuracy accuracy of of the the models models has has been been checked, checked, texturing texturing precision precision has has also also been been monitored. monitored.For Forthis thispurpose, purpose,the thecoordinates coordinatesof of36 36Control ControlPoints Points(CPs) (CPs)have havebeen beenextracted extractedby byCloud Cloud LASER. compared with those obtained by by digitizing the the points on LASER.These Thesecoordinates coordinateshave havebeen beenfirstly firstly compared with those obtained digitizing points the and obtaining their 3D position in CloudinSfM/MVS (Figure 18). The comparison provided on images the images and obtaining their 3D position Cloud SfM/MVS (Figure 18). The comparison the statistics Table 2. in Table 2. provided thedisplayed statistics in displayed
Remote Sens. 2016, 8, 239 Remote Sens. 2016, 8, 239
13 of 16 13 of 16
Figure 18. 18. CPs the vault. vault. Figure CPs on on the Table 2. 2. CP CP coordinates coordinates comparison comparisonCloud CloudLASER—Cloud LASER—CloudSfM/MVS. SfM/MVS. Table
mean (m) mean (m) max (m) max (m) STDVSTDV (m) (m)
X Y Y Z 0.000 0.000 0.000 0.000 0.005 0.000 0.005 0.005 0.005 0.010 0.002 0.003 0.003 0.002 0.004 X
Z 0.000 0.010 0.004
Note: The values are in line with the geometric comparison between point clouds LASER and SfM/MVS. Note: The values are in line with the geometric comparison between point clouds LASER and SfM/MVS.
Subsequently, the same points have been digitized directly on Model SfM/LASER. A comparison Subsequently, the same points have been digitized directly on Model SfM/LASER. A comparison with the reference CP coordinates yielded the results displayed in Table 3. with the reference CP coordinates yielded the results displayed in Table 3. Table 3. CP coordinates comparison Cloud LASER—Model SfM/LASER. Table 3. CP coordinates comparison Cloud LASER—Model SfM/LASER.
mean (m) max (m) mean (m) max (m) STDV (m)
STDV (m)
X 0.000 Y 0.000 0.019 0.000 0.019 0.007 0.023 X
0.007
0.007
Y Z 0.000 0.023 0.000 0.026 0.007 0.008
Z 0.000 0.026 0.008
This comparison shows that precision checks on texture yielded a slightly worse result relative Thison comparison precision on texture yielded slightly worse result to to those geometry. shows Such anthat outcome waschecks predictable, assuming thea addition of errors for relative geometry those on geometry. an outcome was predictable, addition of errors for geometry with those for imageSuch orientation and texture projection,assuming as well asthe those for direct CP collimation on with those for image orientation and texture projection, as well as those for direct CP collimation on Model SfM/LASER. Model SfM/LASER. 4.3. Vault Development Accuracy Assessment 4.3. Vault Development Accuracy Assessment Besides the 3D comparison between Model SfM/LASER and Cloud LASER, planar development Besides the validated 3D comparison between Model SfM/LASER andthe Cloud LASER, planar development has also been at actual scale. Some portions of image, representing the vault has also been validated at actual scale. Some portions of the image, representing the vault development, development, have been printed at 1:1 scale on A0 tracing paper. Subsequently, restorers checked the have printed 1:1represented scale on A0fresco tracingportions paper. Subsequently, restorers the prints directly printsbeen directly withatthe (Figure 19), noticing thechecked accordance of shapes and with the represented fresco portions (Figure 19), noticing the accordance of shapes and dimension dimension of the checked portions in line with the deformations already expected and accepted of in the checked portions in line with the deformations already expected and accepted in theofprocessing the processing steps. On the same tracing paper sheet, restorers have drafted the outlines the actual steps. the same tracing paper sheet, restorers have drafted the outlines of the actual fresco paintings; fresco On paintings; the resulting accuracy is 3 millimeters. the resulting accuracy is 3 mm.
Remote Sens. 2016, 8, 239 Remote Sens. 2016, 8, 239
14 of 16 14 of 16
Figure 19. Development accuracy assessment at 1:1 scale.
5. Conclusions 5. Conclusions The methodology methodology discussed simplified solution problem of of aa metrically metrically The discussed has has proposed proposed aa simplified solution for for the the problem correct planar representation of a frescoed “a schifo” vault. The processing steps shown can be carried correct planar representation of a frescoed “a schifo” vault. The processing steps shown can be carried out even even by by relatively relatively inexperienced inexperienced users users and and do do not not require require specific specificsoftware. software. out A peculiar peculiar feature feature of of this this methodology methodology is is the the creation creation of of collages collages of of several several orthogonal orthogonal views views of of A the textured 3-D model, thanks to geometrical references provided by the section lines of the model. the textured 3-D model, thanks to geometrical references provided by the section lines of the model. These lines lines are are visible visible in in the the three-dimensional three-dimensional model, model, its its geometric geometric development development and and on on the the images images These used for the collage. used for the collage. The methodology methodology proposed proposed for for modeling, modeling, texturing texturing and planar development The and planar development was was verified verified by by both calculating the theoretical error introduced by the single processing step and by comparing the both calculating the theoretical error introduced by the single processing step and by comparing the final products products with with aa reference reference survey survey and and then then directly directly with with the thesurveyed surveyedobject. object. final The theoretical theoretical development development accuracy accuracy is is 33 mm. mm. The the laser The The comparison comparison between between the laser scanner scanner model model textured with oriented images through SFM and the original laser scanning point cloud yielded a 3textured with oriented images through SFM and the original laser scanning point cloud yielded a 3-mm mm accuracy. Finally, the direct verification of the development of the model confirmed an accuracy accuracy. Finally, the direct verification of the development of the model confirmed an accuracy of 3 mm, which allowed to be obtained that usable are fully usable by for 3D fresco 3ofmm, which allowed drafts drafts to be obtained that are fully by restorers for restorers 3D fresco reconstruction reconstruction on a vaulted surface. on a vaulted surface. In particular, particular, these these are are most most useful useful for for faithful faithful reconstruction reconstruction of of the the geometry geometry in in damaged In damaged fresco fresco portions, for which a photographic documentation suitable for 3-D modeling is available. portions, for which a photographic documentation suitable for 3-D modeling is available. The same same methodology methodologycan canalso alsobebeapplied applied domes and vaults of different types. authors The toto domes and vaults of different types. TheThe authors are are currently planning further testing on barrel and pavilion on elliptical and spherical currently planning further testing on barrel and pavilion vaultsvaults and onand elliptical and spherical domes. domes. The present research will be prosecuted with the aim of automating the different processing steps, The present research will be of prosecuted withand the errors aim ofintroduced automating different processing particularly as regards monitoring deformations in the the final representation. steps,Further particularly monitoring differences of deformations errors introduced in extracting the final interest as alsoregards lies in investigating betweenand developments obtained by representation. contours by actual surfaces or by approximating them to ideal surfaces. Further interest also lies in investigating differences between developments obtained by Acknowledgments: work wasorfunded by the University of Pisa internal research funds and by PRIN extracting contoursThis by research actual surfaces by approximating them to ideal surfaces. (Programmi di Ricerca Scientifica di Rilevante Interesse Nazionale—Scientific Research Programs of Relevant National Interest) 2010–2011 funds (coordinator: Raffaele Santamaria—Pisa local research unit). Acknowledgments: This research work was funded by the University of Pisa internal research funds and by Author Contributions: This research wasdiperformed Marco Nazionale—Scientific Giorgio Bevilacqua, Gabriella Isabel PRIN (Programmi di Ricerca Scientifica Rilevante by Interesse ResearchCaroti, Programs of Martínez-Espejo Zaragoza and Andrea Piemonte. Each author contributed extensively and equally to prepare Relevant National Interest) 2010–2011 funds (coordinator: Raffaele Santamaria—Pisa local research unit). this paper. Author Contributions: research was by Marco Giorgio Bevilacqua, Gabriella Caroti, Isabel Conflicts of Interest: TheThis authors declare noperformed conflict of interest. Martínez-Espejo Zaragoza and Andrea Piemonte. Each author contributed extensively and equally to prepare this paper.
Conflicts of Interest: The authors declare no conflict of interest.
Remote Sens. 2016, 8, 239
15 of 16
References 1.
2.
3. 4. 5. 6. 7. 8. 9.
10.
11. 12.
13.
14.
15.
16.
17.
18.
19. 20.
Dequal, S. An unconventional application of analytical plotters to architectural photogrammetry: Projection, plotting and digitizing on non plane surfaces. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 1988, 27, 140–146. Karras, G.E.; Petsa, E.; Dimarogona, A.; Kouroupis, S. Photo-textured rendering of developable surfaces in architectural photogrammetry. In Virtual and Augmented Architecture (VAA’01); Springer: London, UK, 2001; pp. 147–158. Karras, G.E.; Patias, P.; Petsa, E. Digital monoplotting and photo-unwrapping of developable surfaces in architectural photogrammetry. Int. Arch. Photogramm. Remote Sens. 1996, 31, 290–294. Georgopoulos, A.; Makris, G.N.; Tournas, E.; Ioannidis, C.H. Digitally developing works of art. In Proceedings of the CIPA 2001 International Symposium, Potsdam, Germany, 18–21 September 2001. Valanis, A. Fitting, portrayal and mapping for the production of 2nd order surfaces photomosaics. Int. Arch. Photogramm. Remote Sens. 2002, 34, 463–467. Lerma, J.L.; Tortosa, R.V. Digital development of a small Valencian tower. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2004, 35, 451–454. Pera, L. Tecnica Dell’architettura. Tipologia Strutturale; Goliardica: Pisa, Italy, 1987. De la Plaza Escudero, L. Diccionario Visual de Términos Arquitectónicos; Grandes Temas Cátedra: Madrid, Spain, 2008. Lerma, J.L.; Pérez, C. 3D Photorealistic and interactive reconstruction of covered up frescoes. In Recording, Modeling and Visualization of Cultural Heritage; Baltasavias, E., Gruen, A., Van Gool, L., Pateraki, M., Eds.; Taylor & Francis Group: London, UK, 2006; pp. 485–491. Bevilacqua, M.G.; Caroti, G.; Piemonte, A. Rilievi integrati della Basilica romanica di San Gavino a Porto Torres. In Proceedings of the 16a Conferenza Nazionale ASITA, Vicenza, Italy, 6–9 November 2012; pp. 391–396. Caroti, G.; Piemonte, A. An integrated survey for knowledge and preservation of a cultural heritage: The Albanian fortified citadel of Elbasan. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2008, 37, 373–378. Caroti, G.; Piemonte, A. Studio relativo al rilevamento di dettaglio di una porzione del transetto nord della chiesa di San Paolo a Ripa d’Arno al fine di procedere al consolidamento strutturale. In Proceedings of the Un incontro informale per i 70 anni del Prof. Carlo Monti, Milano, Italy, 3 May 2012; pp. 1–6. Dore, C.; Murphy, M. Integration of Historic Building Information Modelling (HBIM) and 3D GIS for recording and managing cultural heritage sites. In Proceedings of the IEEE 18th International Conference on Virtual Systems and Multimedia (VSMM), Milan, Italia, 2–5 September 2012; pp. 369–376. Juan Vidal, F.; Martínez-Espejo Zaragoza, I. Hypotesisforthe virtual anastilosis of themainchapel of the Iglesia de Los Desamparados de Les Coves de Vinromá. In Proceedings of the 16th International Conference on Cultural Heritage and New Technologies, Vienna, Austria, 14–16 November 2011; pp. 201–212. Lerma, J.L.; Cabrelles, M.; Navarro, S.; Seguí, A.E. La documentación patrimonial mediante sensores de imagen o de barrido láser. In Documentación Gráfica del Patrimonio; Ministerio de Cultura: Madrid, Spain, 2011; pp. 108–117. Marambio, A.; Pucci, B.; Nuñez, M.A.; Buill, F. La aplicación del escáner láser terrestre en la catalogación del patrimonio arquitectónico del casco histórico de Castellfollit de la Roca. In Proceedings of the VIII Semana Geomática Internacional, Barcelona, Spain, 3–5 March 2009; pp. 1–8. Martínez-Espejo Zaragoza, I.; Juan Vidal, F. Gestión gráfica avanzada de edificaciones antiguas. In Proceedings of the XI Congreso Internacional de Expresión Gráfica Aplicada a la Edificación, Universidad Politécnica de Valencia, Valencia, Spain, 29 November–1 December 2012; pp. 710–714. Alsadik, B.; Remondino, F.; Menna, F.; Gerke, M.; Vosselman, G. Robust extraction of image correspondences exploiting the image scene geometry and approximate camera orientation. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2013, 40, 1–7. [CrossRef] Guidi, G.; Russo, M.; Beraldin, J.A. Acquisizione 3D e Modellazione Poligonale; McGraw-Hill: Milano, Italy, 2010. Guidi, G.; Remondino, F. 3D Modelling from real data. In Modelling and Simulation in Engineering; Catalin Alexandru, C., Ed.; InTech: Vienna, Austria, 2012; pp. 69–102.
Remote Sens. 2016, 8, 239
21.
22.
23. 24. 25. 26.
27. 28. 29. 30.
31. 32.
33.
34.
16 of 16
Koska, B.; Kˇremen, T. The combination of laser scanning and structure from motion technology for creation of accurate exterior and interior orthophotos of St. Nicholas Baroque church. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2013, 40, 133–138. [CrossRef] Lerma, J.L.; Navarro, S.; Cabrelles, M.; Villaverde, V. Terrestrial laser scanning and close range photogrammetry for 3D archaeological documentation: The upper Palaeolithic Cave of Parpalló as a case study. J. Archaeol. Sci. 2010, 37, 499–507. [CrossRef] Wenzel, K.; Rothermel, M.; Fritsch, D.; Haala, N. Image acquisition and model selection for multi-view stereo. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2013, 40, 251–258. [CrossRef] Remondino, F.; Spera, M.G.; Nocerino, E.; Menna, F.; Nex, F. State of the art in high density image matching. Photogramm. Rec. 2014, 29, 144–166. [CrossRef] Apollonio, F.I.; Ballabeni, A.; Gaiani, M.; Remondino, F. Evaluation of feature-based methods for automated network orientation. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2014, 40, 47–54. [CrossRef] Caroti, G.; Martínez-Espejo Zaragoza, I.; Piemonte, A. Accuracy assessment in structure from motion 3D reconstruction from UAV-born images: The influence of the data processing methods. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2015, 40, 103–109. [CrossRef] Apollonio, F.I.; Remondino, F. Modellazione 3D da sensori attivi Pipeline con laser scanner. In Modelli Digitali 3D in Archeologia: Il Caso di Pompei; Scuola Normale Superiore Pisa: Pisa, Italy, 2010; pp. 94–117. Carpiceci, M. Survey problems and representation of architectural painted surfaces. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2011, 38, 523–528. [CrossRef] Menna, F.; Rizzi, A.; Nocerino, E.; Remondino, F.; Gruen, A. High resolution 3D modeling of the behaim globe. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2012, 39, 115–120. [CrossRef] Cipriani, L.; Fantini, F.; Bertacchi, S. Survey and representation of vaults and cupolas: An overview on some relevant Italian UNESCO Sites. In Proceedings of the IEEE International Conference on Virtual Systems & Multimedia (VSMM), Hong Kong, China, 9–12 December 2014; pp. 50–57. Chiabrando, F.; Rinaudo, F. Recovering a collapsed medieval fresco by using 3D modeling techniques. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2014, 2, 105–112. Pancani, G. Lo svolgimento in vera grandezza delle volte affrescate delle sale dei quartieri al piano terreno di Palazzo Pitti a Firenze. In Proceedings of the Giornate di Studio Il Disegno delle trasformazioni, Napoli, Italy, 1–2 December 2011; pp. 1–11. Cannella, M. Sviluppo e Rappresentazione Digitale di Superfici Architettoniche Complesse per la Documentazione e il Restauro. Available online: http://hdl.handle.net/10447/130394 (accessed on 25 November 2015). Meyer, E.; Parisel, C.; Grussenmeyer, P.; Revez, J.; Tidafi, T. A computerized solution for the epigraphic survey in Egyptian Temples. J. Archaeol. Sci. 2006, 33, 1605–1616. [CrossRef] © 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons by Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).
