MINIMAL SURFACE DISTORTION FUNCTION FOR OPTIMIZING 3D

MINIMAL SURFACE DISTORTION FUNCTION FOR OPTIMIZING 3D WATERMARKING Adrian G. Bors

and

Ming Luo

Dept. of Computer Science, University of York, York YO10 5GH, UK E-mail: [email protected] ABSTRACT A blind 3D watermarking method based on optimizing the placement of displaced vertices on the surface of the 3D object is proposed in this paper. A new mesh distortion function is used as a cost function for the Levenberg-Marquardt optimization algorithm in order to ensure a minimal distortion to the 3D object surface after watermarking. The surface distortion function consists of the sums of Euclidean distances from the displaced vertex to the original surface, to the new surface and to the original vertex location. Experimental results assess the level of distortion produced to 3D surfaces by watermarking as well as the robustness to various attacks. 1. INTRODUCTION This paper provides a new approach for preserving 3D object surfaces after digital watermarking. A message is embedded in a 3D object by displacing vertices on its surface such that the resulting changes are not visible. The watermark message can be used for database organization, for copyright protection or for other 3D object specific applications. We should be able to detect the message even after the object would be attacked by adding noise, smoothing its surface, mesh simplification, etc. 3D watermarking methods can be categorized as deterministic [1, 2] or statistical [3, 4, 5]. Statistical watermarking alters, differently for a bit of 0 or 1, the shape of histograms representing object shape information and would distribute the distortions onto the surface of the object. From the point of view of 3D object data representations we have watermarking in spatial and in transform domains. Transform domain watermarking considers embedding a message in the mesh spectral domain, in the wavelet domain or the spheroidal harmonic coefficients. Spatial domain watermarking considers embedding changes directly in the geometry of the 3D object [1, 4]. Watermarking, invariably introduces distortions in the mesh surface [1]. 3D watermarking methods which minimize the surface distortion employs vertex displacements along the geodesic front propagation lines [4] and the local minimization of the Quadric Error Metric (QEM), using geometric projections [6]. In the proposed methodology we form normalized distributions of distances from vertices to the object center, where the 3D object is represented by its mesh surface. As in [3, 4], either the mean or the variance of normalized distributions of vertex norms are changed according to the given watermark code. Displacement of vertices from the object’ mesh is optimized using the Levenberg-Marquardt algorithm in spherical coordinates. A new surface distortion metric is proposed as the cost function for the optimization. This metric consists of the Euclidean distance from the vertex which is displaced by the watermark embedding, to the original surface, summed with the distance to the watermarked surface and to the original vertex location. Section 2 outlines the surface distortion metric used as a watermark embedding cost function. Section 3 describes the watermark embedding approach using the Levenberg-Marquardt algo-

rithm. Section 4 provides the experimental results, while Section 5 concludes the paper. 2. SURFACE DISTORTION METRIC CRITERION For a given vertex from the mesh surface of an object vj ∈ O, let us denote its distance to the reference point given by the center, located at o, representing the vertex norm, by ρj = kvj − ok and consider this as a statistical variable. After ranking the distances from the center to vertices, we evaluate ρmin = minvj (ρj ) and ρmax = maxvj (ρj ), where vj ∈ O. Consequently, the vertices are grouped into M sets, where the message length is of M bits, according to their distances to the object center: Bi = {vj ∈ O | ρmin + ε(ρmax − ρmin ) + (i − 1)ρb ≤ ρj , ρj < ρmax − ε(ρmax − ρmin ) + iρb } where i = 1, . . . , M and each of these sets contains a number of vertices located in a specific range of distances, where ε ∈ [0, εmax ] represents a trimming ratio, whose value is generated according to a secret key in order to ensure the security of the watermark. The vertex norms from each Bi are normalized and used to form histograms. These histograms are statistically modified, according to the bit embedded, as in [3, 4]. Two different approaches are considered by changing the mean or the variance of the histogram. Let us consider that we are provided with the statistical variable ρˆj , sampled from a distribution function characterizing the statistical watermark. A chosen vertex Vj is displaced to a new location Vˆj such that kOVˆj k = ρˆj , where O is the center of the 3D object. In the following we consider E(·) = f T f as the distortion between the watermarked and the original surface, the smoothness of the resulting watermarked surface well as √ the original vertex displacement √ as√ according to f = ( k1 f1 , k2 f2 , k3 f3 )T with each of the components f1 , f2 and f3 characterized by their weighting parameters k1 , k2 and k3 , such that k1 + k2 + k3 = 1. The first error metric component f1 defines the distortion of the watermarked vertex with respect to the original surface:   < (ˆ vj − vj ), n1 > n1   .. f1 =  (1)  . < (ˆ vj − vj ), nl > nl ˆ j is the location of the watermarked vertex Vˆj , < ·, · > is where v the dot product and nl , l = 1, . . . , NVj , is the normal vector of a triangle adjacent to the vertex Vj from the original surface. The vector < (ˆ vj − vj ), nl > nl represents the projection of the vertex displacement along the orthogonal direction from Vˆj to the polygon Fl , which is adjacent to Vˆj . Let us define D(ˆ vj , Fl ), the distance from vertex Vˆj to the polygon Fl . We have: NVj

f1T f1 =

X l=1

D2 (ˆ vj , Fl ).

(2)

The metric f1T f1 represents the Quadric Error Metric (QEM), i.e. the squared distance from the watermarked vertex to the original surface following the vertex displacement due to watermark embedding [6]. Nevertheless, if only this distance would be used as an error function, watermarking the 3D model could result in twisted polygons. The second vector function component f2 enforces surface smoothness and is defined as the distance of the watermarked vertex to the updated surface:   ˆ1 > n ˆ1 < (ˆ vj − vj ), n   .. f2 =  (3)  . ˆl > n ˆl < (ˆ vj − vj ), n ˆ l , l = 1, . . . , NVj is the normal vector of the polygon conwhere n tained in the modified surface, neighboring the watermarked vertex ˆj . Vˆj of location v The third error function component f3 represents the vertex displacement between the watermarked and the original vertices : ˆ j − vj . f3 = v (4) The third constraint is added in order to compensate for the surface error which is not accounted for by the previous two error function components f1 and f2 . For example, by displacing any point to a location somewhere else on the plane containing the original triangle, will produce no errors with respect to the first and second error function components if none of their neighbors’ locations are changed by watermarking. Nevertheless, such a vertex displacement can introduce a very large distortion to the object surface. When k2 = 0 and k3 = 0 we obtain the criterion used for watermarking in [6], while for k1 = 0 and k2 = 0 we have the criterion used in [3]. 3. OPTIMIZATION OF VERTEX PLACEMENT In this following we describe how we embed the watermark by displacing a given vertex while minimizing the surface distortion criterion function E(·). The constraint that we have to enforce is ||OVˆ || = ρˆj , i.e. the watermarked vertex should be on the sphere S(O, ρˆj ) centered in the object center O, of the radius given by the statistical variable ρˆj corresponding to the bit to be embedded. We have a case of embedding onto a sphere and the spherical coordinate system is the most appropriate for representing the vertex location : ˆ ρˆj sin φˆ sin θ, ˆ ρˆj cos θ) ˆT ˆ = (ρˆj cos φˆ sin θ, v (5) In the following we update the vector ψ = ( φˆ , θˆ )T , while considering the vertex norm ρˆj as constant. For the initialization we consider moving the vertex V to Vˆ −−→ along the direction of OV such that ||OVˆ || = ρˆj , as in [3]. Then, we use the iterative Levenberg-Marquardt optimization in order to find the optimal vertex location minimizing the surface error E(·). Levenberg-Marquardt method [7] represents an iterative gradientdescend minimization approach which solves nonlinear least square problems, subject to constraints. Levenberg-Marquardt firstly linearizes the given nonlinear problem by using a Taylor expansion around the vector ψ = ( φˆ , θˆ )T : f (ψ + h) = f (ψ) + Jh (6) where f (·) is the given surface distortion function, defined in the previous section and used here as a constraint criterion, and h = ( ∆φ , ∆θ )T is the step size. J is the Jacobian matrix of the vector function f and is calculatedas: T ∂f ∂f J= , (7) ∂ φˆ ∂ θˆ An optimal step hk is calculated at each iteration k, in order to update the vector ψk : ψk+1 = ψk + hk (8)

Levenberg-Marquardt uses iteratively the following equation in order to calculate hk : (JTk Jk + µk I)hk = JTk fk

(9)

where µk > 0 is the damping factor, fk and Jk represent the surface distortion metric and its Jacobian, calculated at iteration k with respect to ψk . The initial damping factor is chosen as µ0 = 10−6 max{ diag (JT0 J0 )} where diag represents the diagonal of the matrix JT0 J0 , which corresponds to the Jacobian matrix of f calculated for ψ0 . The algorithm updates µk as in [8]. When ψk is close to the optimal value, the convergence rate of the LevenbergMarquardt is almost quadratic. Levenberg-Marquardt was proved to converge and the optimization process is terminated when either the step size hk becomes too small, or the error E(·) is too small, or when the loop exceeds a pre-set number of iterations [7]. In the detection stage, the watermark message is extracted bit by bit using statistical detection as in [3, 4, 6]. 4. EXPERIMENTAL RESULTS In this following we consider six 3D graphical models: Bunny, Fish, Gear, Dragon, Buddha and Head, shown in Fig. 1. For each result we consider the embedding of one hundred different watermark codes into each graphical object and a message length of M = 64 bits, watermark strength parameter α = 0.1 and εmax = 0.15. We compare the results provided by the proposed watermarking methodology based on the Levenberg-Marquardt optimization method (denoted as L-M) with the methods proposed by Cho et al. [3], and that the using quadric error metric (QSP) from [6]. All these methods rely on using the object center as the reference and for each of them we consider modifying either the mean or the variance of the histogram of distances from vertices to the object center (identified by appending either “Mean” or “Var”).

(a) Bunny

(b) Fish

(c) Gear

(d) Dragon

(e) Buddha

(f) Head

Fig. 1. 3D Models used in the experiments. We analyze the effect when varying the weighting parameters for the cost function components, described in Section 2, used in the optimization process. The parameters k1 , k2 and k3 weigh the surface distortion function components f1 , f2 and f3 characterizing the Euclidean distances of the updated vertex with respect to the original surface, watermarked surface and that from the original vertex location, respectively. Various parameter combinations (k1 , k2 , k3 ), emphasizing one or another of the error components, are listed in the first column of Table 1 when applied for two graphical objects, Bunny and Gear. The results are evaluated according to the distortion produced to the 3D object surface, measured according to MRMS surface error from [9] which represents an approximation of

Table 1. Evaluating the MRMS error from [9], when changing k1 , k2 , k3 , where the results should be multiplied with 10−4 . Mean and Var embeddings are denoted by M and V. Parameters k1 k2 k3 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0 0.5 0.5 0.0 0.5 0.0 0.5 0.0 0.5 0.5 0.45 0.45 0.1 0.4 0.4 0.2 0.49 0.49 0.02

Bunny L-MM L-MV 0.44 0.17 0.46 0.18 1.18 0.62 0.41 0.18 1.10 0.59 1.17 0.62 0.69 0.36 0.88 0.46 0.40 0.21

Gear L-MM L-MV 949.09 428.09 1025.21 355.11 1861.15 1024.07 654.57 303.15 1546.87 862.52 1817.69 998.94 621.97 313.10 1007.14 537.90 410.57 148.12

the Hausdorff distance for 3D mesh surfaces. These results show the importance of minimizing the error with respect to the original object surface while enforcing the smoothness of the resulting watermarked object surface and without neglecting the vertex displacement with respect to the original vertex location. Throughout the rest of the experiments, we consider the configuration of (k1 , k2 , k3 )=(0.49, 0.49, 0.02). A very important requirement for watermarking graphical objects consists of achieving a minimal surface distortion such that it is not noticeable. Table 2 compares the distortions introduced by the two methods proposed in this paper and the Cho’s methods [3], under the same parameter settings. We use MRMS from [9] for evaluating the numerical distortion measure. As it can be observed from these results, L-MMean and L-MVar provide better surface preservation results than the other methods. Fig. 2 compares close details of the distortions produced by L-M and QSP. The surface distortion minimization methods proposed in this paper produce much smaller distortion than Cho’s approaches as it can be observed from these images. The proposed methodology is robust against attacks that do not distort the graphical object surface including the affine transformations, vertex reordering, etc. In the following, we assess the robustness against additive noise, Laplacian smoothing, mesh simplification, quantization and uniform resampling. We compare the results provided by the proposed methodology with those of Cho’s method [3] for two graphical objects, Dragon and Fish. Each test result represents the average of bit error ratios (BER) representing the ratio of bits lost after each attack from the total number of embedded bits when considering the embedding of one hundred different watermark codes into each object. Additive random noise is added to the mesh according to the following equation: → ˜ =v ˆ + kˆ v v k− p (10) i

i

max

˜ i represents the noisy watermarked vertex v ˆ i ,  ∈ [0, 1] where v is the percentage of kˆ vmax k, which corresponds to the largest Euclidean distance measured from the object center to all object ver→ tices, and − p is a unitary vector of random direction. The plots from Fig. 3 show the robustness against noise. From these plots we can observe that L-MMean is better in Fish object while ChoMean is slightly better for the Dragon object. For the smoothing attack test we use the Laplacian algorithm proposed in [10]. The robustness of the watermarking methods against the Laplacian smoothing when this is applied for 20 iterations with λ = 0.5 is illustrated in Fig. 4. It can be observed that L-MVar is better than ChoVar in Fish object. The quadric metric simplification from [11] was used for testing the

(a) Bunny

(b) Fish

(c) Gear

(d) Dragon

Fig. 2. Visual comparison of watermarked graphical object details. From the first row to the bottom, the following are represented on each row: original objects and watermarked by L-MMean, L-MVar, QSPMean and QSPVar.

(a) Dragon (b) Fish Fig. 3. Robustness against additive noise. robustness at mesh simplification. Fig. 5 provides the robustness to the simplification attack. The proposed methodology performs excellently in the case of the mesh simplification attack. According to the results from Fig. 5, L-MMean method provides the best results for both objects, while L-MVar is better than ChoVar. This is because the L-M methods represent the watermarked object surface as being consistent with the original 3D object surface and the mesh simplification algorithm tends to preserve features of object’ surface. In Fig. 6 we consider the robustness to quantization attacks and in Fig. 7 that to uniform resampling and remeshing when using the method proposed in [12] for this attack. 5. CONCLUSIONS This research study describes an optimization based approach to 3D watermarking aiming to preserve the surface of the graphical object. The proposed methodology considers a novel surface distortion function used as an optimization cost function for 3D watermarking. This function consists of three components measuring the distortion

Table 2. Watermarked object distortion measured by MRMS, where all the figures should be multiplied with 10−4 . Object Bunny Fish Gear Dragon Buddha Head

L-MMean 0.40 0.12 409.59 0.29 0.29 0.12

L-MVar 0.21 0.06 148.16 0.13 0.16 0.06

QSPMean 0.43 0.15 679.46 0.36 0.27 0.13

QSPVar 0.25 0.07 212.11 0.15 0.16 0.06

ChoMean 1.18 0.48 1860.67 1.09 0.91 0.32

ChoVar 0.62 0.24 1023.67 0.57 0.47 0.16

(a) Dragon (b) Fish Fig. 4. Robustness against Laplacian smoothing.

(a) Dragon (b) Fish Fig. 6. Robustness against bit quantization.

(a) Dragon (b) Fish Fig. 5. Robustness against mesh simplification.

(a) Dragon (b) Fish Fig. 7. Robustness against uniform resampling and remeshing where the horizontal axes show the percentages of remaining original vertices.

with respect to the original surface, the watermarked surface and the Euclidean distance from the original vertex location. The 3D object surface distortion is minimized by employing the LevenbergMarquardt algorithm in spherical coordinates. The robustness of the proposed 3D watermarking methodology to various attacks was shown experimentally. The security of the watermark code is guaranteed by a random parameter deciding the ranges of vertices considered for extracting each bit. Watermarks can be used for information hiding, copyright protection, embedding information relevant to database organization or other object-specific information. 6. REFERENCES [1] A. G. Bors, “Watermarking mesh-based representations of 3-D objects using local moments,” IEEE Trans. on Image Processing, vol. 15, no. 3, pp. 687–701, 2006. [2] F. Cayre and B. Macq, “Data hiding on 3-D triangle meshes,” IEEE Trans. Signal Processing, vol. 51, no. 4, pp. 939–949, 2003. [3] J. W. Cho, R. Prost, and H. Y. Jung, “An oblivious watermarking for 3-D polygonal meshes using distribution of vertex norms,” IEEE Trans. Signal Processing, vol. 55, no. 1, pp. 142–155, 2007. [4] M. Luo and A. G. Bors, “Surface-preserving watermarking of 3-D shapes,” IEEE Trans on Image Processing, vol. 20, no. 10, pp. 2813–2826, 2011. [5] S. Zafeiriou, A. Tefas, and I. Pitas, “Blind robust watermarking schemes for copyright protection of 3D mesh objects,” IEEE

Trans. on Visualization and Computer Graphics, vol. 11, no. 5, pp. 496–607, 2005. [6] M. Luo and A. G. Bors, “Shape watermarking based on minimizing the quadric error metric,” in Proc. IEEE Int. Conf. on Shape Modeling and Applications, Beijing, China, 2009, pp. 103–110. [7] K. Levenberg, “An algorithm for least-squares estimation of nonlinear parameters,” The Quarterly of Applied Mathematics, vol. 2, pp. 164–168, 1944. [8] Nielsen, “Damping parameter in marquardt’s method,” Tech. Rep., Technical University of Danemark, 1999. [9] P. Cignoni, C. Rocchini, and R. Scopigno, “Metro: Measuring error on simplified surfaces,” Computer Graphics Forum, vol. 17, no. 2, pp. 167–174, 1998. [10] G.Taubin, “Geometric signal processing on polygonal meshes,” in Eurographics state of the art report, 2000. [11] M. Garland and P. Heckbert, “Surface simplification using quadric error metrics,” in Proc. SIGGRAPH, Graphical Models, vol. 66, no. 6, 1997, pp. 370–397. [12] M. Attene and B. Falcidieno, “Remesh: An interactive environment to edit and repair triangle meshes,” in Proc. IEEE Int. Conf. on Shape Modeling and Applications, Matsushima, Japan, 2006, pp. 271–276.