As a first step to solve this problem, a new digital watermarking method for 3D polygonal ... model viewer, virtual reality, finite element analysis, rapid prototyping etc. ...... a signature to pictures under wavelet transformation. (in Japanese) ...
Digital Watermarking for 3D Polygons using Multiresolution Wavelet Decomposition Satoshi KANAI ,
Hiroaki DATE
and Takeshi KISHINAMI
Dept. of Systems Engg., Graduate School of Engg., Sapporo 060, JAPAN
Hokkaido University
Abstract: Recently, much interest is being taken in methods to protect the copyright of digital data and preventing illegal duplication of it. However, in the area of CAD/CAM and CG, there are no effective ways to protect the copyright of 3D geometric models. As a first step to solve this problem, a new digital watermarking method for 3D polygonal models is introduced in this paper. Watermarking is one of the copyright protection methods where an invisible watermark is secretly embedded into the original data. The proposed watermarking method is based on wavelet transform (WT) and multiresolution representation (MRR) of the polygonal model. The watermark can be embedded in the large wavelet coefficient vectors at various resolution levels of the MRR. This makes the embedded watermark imperceptible and invariant to the affine transformation. And also makes the control of the geometric error caused by the watermarking reliable. First the requirements and features of the proposed watermarking method are discussed. Second the mathematical formulations of WT and MRR of the polygonal model are shown. Third the algorithm of embedding and extracting the watermark is proposed. Finally, the effectiveness of the proposed watermarking method is shown through several simulation results. Key words: digital watermarking, data hiding, copyright protection, polygonal model, three-dimensional geometric model, information security, wavelet transformation, multiresolution representation, CG, CAD, CAM.
Introduction Recent remarkable growth of digital media such as the Internet enables us to easily access, copy, modify and distribute digital contents such as electronic documents, images, sounds and videos. In this situation, there is a strong need for developing techniques to protect the copyright of the original digital data and to prevent unauthorized duplication or tampering of it. Digital watermarking or data hiding is one of the solutions for the copyright protection of digital data. Until now, many researches have been intensively done on the watermarking for digital text, image, video and sound signals. Figure1 shows the concept of the general digital watermarking process. In the process, the digital watermark such as copyright or ownership data is secretly embedded into the original data first. The embedding method leaves the original data perceptually unchanged, but imposes small modifications on it. This process is called watermark embedding or watermarking. Then the author distributes this watermarked data instead of the original data. When the copyright of the distributing data is in question, the author can extract the embedded watermark again by using the appropriate algorithm. This process is called watermark extraction. On the contrary, so far, there is no effective way to protect the copyright of 3D geometric models. The 3D geometric model, such as surface model, solid model or polygonal model can be recognized as valuable products of intellectual activities in CAD, CAM, CAE and CG systems. The current situation of the distributed engineering environment increases opportunities to
Digital Watermark 0101
1.
Original Data
Watermarked Data
Embedding Algorithm
Distribution 0101...
(a) Watermark Embedding
Original Data Watermarked Data 0101... 0101...
Extraction Algorithm
Extracted Watermark 0101
(b) Watermark Extraction
Fig.1
General Digital Watermarking Process
globally exchange digital data of geometric models between various organizations through a computer network. This also increases the possibility of theft of the geometric model by illegal duplication. Therefore the copyright property of the geometric model must be strongly protected from theft. However, so far, almost all current digital watermarking techniques are only focused on the copyright protection of digital text, image, video and sound data. And very little research on digital watermarking for 3D geometric models has been done. The purpose of this paper is to propose a new digital watermarking method for 3D polygonal models, and to investigate the performance of the proposed method 1
Watermark
through several simulations. Currently, the polygonal model is widely used in many applications; CG, CAD model viewer, virtual reality, finite element analysis, rapid prototyping etc. Our proposed watermarking method is based on the wavelet transform (WT) and multiresolution representation (MRR) of the 3D polygonal model. The watermark can be embedded in the large wavelet coefficient vectors at various resolution levels of the MRR. This makes the embedded watermark imperceptible, and the control of the geometric error caused by the watermarking reliable. Moreover, the embedded watermark is invariant to the 3D affine transformation. In Section 2, the related research on the digital watermarking is surveyed. In Section 3, the requirements and the features of the proposed watermarking method for the polygonal model are discussed. In Section 4, the mathematical formulations of WT and MRR of the polygonal model are shown. In Section 5, the algorithm of embedding and extracting the watermark is discussed. Finally, in Section 6, the performance and effectiveness of the proposed watermarking method is investigated through several simulation results.
2.
Related Works on the Digital Watermarking
2.1 Digital Watermarking for Text, Images and Videos Most of the recent research work in digital watermarking aims for embedding the watermark into the digital data of texts[2,3], images[4-11], videos[11] and sounds. There is much work on watermarking for digital images. There are already some watermarking techniques which are commercially available[13,14]. Theoretically, the principle of watermarking for these images can be grouped into two categories; spatial domain methods[9,10,13,14], and frequency domain methods [4,5,6,7,8,11]. Shown in Figure2(a), the spatial domain method embeds the watermark by directly modifying the original image data. While, shown in Figure 2-(b), the frequency domain method embeds the watermark by modifying the frequency domain coefficients in a process between the transform and inverse transform. As for the transform method to the frequency domain, discrete cosine transform (DCT)[11] or wavelet transform (WT) [4-8] are popular. Generally, in the spatial domain watermarking method, the embedding process is simpler and faster than the frequency domain method, but the embedded watermark is weaker for the operations of image processing, and is more perceptible than the frequency domain method. While in the frequency domain watermarking method, the watermarking process is more complex and slower than the spatial one because of the need to transform and inverse transform. However, the embedded watermark is stronger to the operations. It is also easier to make the embedded watermark more in-perceivable than the spatial one, because the bands (resolution level) and quantities of embedding can be selected and controlled in the frequency domain. For this reason, more interest has been recently aroused in the frequency domain method in the area of
Original Data
Modification on the Spatial Data
Watermarked Data
(a) Spatial Domain Watermarking Method
Original Data
Watermark Transform to the Frequency Domain
Modification on the Frequency Data
Frequency Data
ModifiedFrequency Data Inverse Watermarked Transform Data to the Spatial Domain
FFT, DCT, Wavelet Transform (b) Frequency Domain Watermarking Method
Fig. 2
Category of Watermarking Methods
watermarking for the digital images. 2.2 Digital Watermarking for 3D Geometric Models In contrast to digital watermarking for images, there is very little research on digital watermarking methods for 3D geometric models. Until now, only the pioneer work by Dr. Ohbuchi et al has been reported[15,16]. He proposed the method for embedding the visible watermark of copyright data into 3D polygonal models. In this method, the watermark is embedded into the polygon data by modifying either the vertex coordinates, the vertex connectivity, or both. They also discussed the ordering schemes into a set of geometrical or topological elements for embedding and extracting the watermark. Several different embedding algorithm were proposed in ; triangle similarity quadruple algorithm, tetrahedral volume ratio embedding and triangle strip peeling symbol sequence algorithm. The embedded watermark is robust to the 3D affine transformation and cropping. According to the former classification, the Dr. Ohbuchi’s watermarking algorithm can be classified into the spatial domain method. On the contrary, in this paper, we propose the frequency domain method for the watermarking of 3D polygonal models. The advantages and disadvantages of our watermarking method will be discussed in the next section.
3. Fundamentals of the proposed Watermarking Method for the 3D Polygonal Model 3.1 Requirements on the Watermarking for 3D Polygonal Model For the purpose of the copyright protection of 3D polygonal models, the watermarking method must satisfy the following technical requirements; (1) Direct Embedding: The watermark should be directly embedded into the geometry data or topology data of the polygonal model. It should not be embedded in the header or comment section. 2
WCVs Original Polygon Data
W1 W2
V0
Wavelet Transform
W
WCV: Wavelet Coefficient Vector MRR: Multi-Resolution Representation
Lower Bound of WCVs for embedding Upper Bound of WCVs for embedding
d
MRR Watermark 1011....
Lazy Wavelet
Wˆ Wˆ
V V
d
Modified WCVs
2
Wˆ
Modification dof WCVs
1
d
Watermarked Polygon Data
Vˆ 0
Inverse Wavelet Transform
Embedding manipulation
Coarsest Approximation
1011....
Lazy Wavelet
(a) Watermark Embedding Process
Original Polygon Data
V0
WCVs
Wavelet Transform Lazy Wavelet Modified WCVs
Watermarked Polygon Data
Vˆ 0 1011....
Wavelet Transform Lazy Wavelet
W1 W2
Wˆ 1 Wˆ 2
Wd
Wˆ d Difference Calculation
Extracted Watermark 1011....
Extracting manipulation
Fig. 3
Outline
Process of (b) theWatermark proposed Extraction Wavelet-based Watermarking
(2) Perceptually Invisible: The embedded watermark must be perceptually invisible and unnoticeable for the user within the model. (3) Small geometric error: The geometric error of the polygon data caused by the embedding must be small enough not to disturb the application use. (4) Ready Extraction: The embedded watermark can be readily extracted by the appropriate algorithm. (5) Robustness and Invariance: The embedded watermark must be unchanged or difficult to be destroyed under the possible 3D geometric operations done on the 3D polygonal model; affine transformation, local modifications on the geometry, re-meshing, re-indexing, data compression and cropping. It must be also robust to the intentional removal operation by the third person. (6) Embedding Capacity: The amount of the watermark which can be embedded in the model is large enough to record the information needed for the application. Some of these requirements are in tradeoff relationships with each other. When the robustness and capacity increases, the visibility and geometric error increases and vice versa. The priority among these
Method
requirement should be determined depending on which application the watermark is used. At least, it is important that invariance to the affine transformation must be kept, and perceptual invisibility of the watermark and the geometric error from the original polygon shape due to the watermarking can be controlled in the proposed watermarking method. 3.2 Overview of the proposed Wavelet based Watermarking Method Figure3 illustrates the outline of the proposed watermarking method. The method can be classified into the frequency domain one and, is based on the wavelet transform (WT) and multiresolution representation (MRR) for 3D polygonal models. The method includes the watermark embedding and extraction processes. (1) Watermark Embedding Process The watermark embedding process consists of the following three steps; (1-1) Wavelet Transform of the Original Polygon Data The original polygon data V0 is decomposed into the MRR by applying the wavelet transform at several times. In the MRR, V0 is decomposed both into the set of wavelet coefficient vectors W1, W2, .... , Wd at every resolution level, and into the coarsest approximation Vd, where d means the coarsest resolution level. The wavelet transform for the 3D polygon data was first proposed by Lounsbery 3
and DeRose [17,18]. In our method, “lazy wavelet” [18] is selected for the type of wavelet. (1-2) Modification of the Wavelet Coefficient Vectors by the watermark. Depending on the watermark to be embedded, the norm of the wavelet coefficient vectors are modified, and the set of modified wavelet coefficient vectors Wˆ 1 , Wˆ 2 , , Wˆ d are obtained. The modification is followed by the rules explained in Section 5. The norm of a wavelet coefficient vector shows the degree of bumpiness in certain areas near the vector. It is usually true that the human eyes are not sensitive to the small geometric changes in bumpy areas[4]. So in order to make the embedded watermark perceptually invisible, only the wavelet coefficient vectors which are larger than the specified lower bound should be selected to be modified in the watermarking. Moreover the upper bound of the modification of wavelet coefficient vector is managed so that predefined geometric tolerance from the original polygon shape must be satisfied. (1-3) Inverse Wavelet Transform for the modified Wavelet Coefficient Vectors Inverse wavelet transform is taken for the set of modified wavelet coefficient vectors Wˆ 1 , Wˆ 2 , , Wˆ d , and the unchanged coarsest approximation Vd. The watermarked polygon data Vˆ 0 is obtained from the result of the inverse transform. The watermarked polygon Vˆ 0 will be distributed instead of the original data. (2) Watermark Extraction Process The watermark extraction process consists of following steps; (2-1) Wavelet Transform of the Watermarked Polygon Data and the Original Polygon Data Both the received (watermarked) polygon data Vˆ 0 will and the original polygonal data V0 are decomposed respectively into the MRR Wˆ 1 , Wˆ 2 , , Wˆ d and W1, W2, .... , Wd by applying the wavelet transform. (2-2) Difference calculations of the wavelet coefficient vectors of the received data and original data In every resolution level, we compare the corresponding wavelet coefficient vectors Wˆ j and W j of received polygon data and original polygon data. From the difference between them, the embedded watermark can be extracted. The detail algorithm is described in Section5. 3.3 Features of the proposed watermarking method The advantages of our proposed watermarking method for the polygonal model are summarized as follows; (1) We can easily determine the area to be embedded by selecting the large wavelet coefficient vectors so that the embedded data can be perceptually invisible. This is because the norm of the wavelet coefficient vector indicates whether certain areas in the original polygon is planer or bumpier, and generally the human eyes are not sensitive to small changes in the bumpy area. (2) The embedded watermark can be spread into various
resolution levels. The localization of the watermark at high resolutions provide the ability to identify the distinct region of the watermarked polygon data on which the local modification has been done. While the global spreading of the watermark at low resolutions makes the embedded watermark invariant for the local modification on the geometry. This makes the watermark more robust than the one in the spatial domain method. (3) From the property of the wavelet decomposition, we can embed the watermark not only into the original polygon data, but also into the approximated polygon data at several higher resolution levels only by executing the one embedding process. (4) The method based on the wavelet transform by using the lazy wavelet enables us to define a clear geometrical relation between the overall geometric tolerance and the upper bound of modification on each wavelet coefficient vectors. Therefore we can easily control the geometric error between the watermarked polygon data and the original one. These features were not discussed in the previous watermarking method [15, 16]. On the contrary, due to the frequency domain method, our watermarking method still has following disadvantages; (1) The wavelet transform only can be applied to the restricted topological class of the mesh. The permitted mesh topology of the watermarking must be the one fit to 4-to-1 subdivision connectivity schemes. (2) The embedding and extracting process needs more computational time than the one in the spatial domain method. (3) Unlike the topology embedding method proposed in [15, 16], our embedding method essentially generates a geometric error from the original polygon shape. The first disadvantage makes very strict limitations on the mesh topology, but will be able to be solved by the other remeshing algorithm which can convert the arbitrary mesh topology to 4-to-1 connectivity[19]. As for the second disadvantage, we do not need the real-time performance of the embedding process like watermarking for digital videos, so that this disadvantage is not necessarily serious problem for the watermarking of the polygonal model. The third disadvantage can be solved by the geometric error control algorithm in which the error caused by the watermarking can be suppressed within the specified tolerance. We explain this error control in the next section.
4.
Multiresolution Wavelet Decomposition for Polygonal Model Mathematical formulations of Wavelet Transform and inverse wavelet transform for 3D polygons are given in this section. This theory of the transform has been proposed by Lounbery and DeRose [17,18]. In this theory, it is assumed that all of the face in a polygon mesh must be triangle, and must have a mesh topology fitted to a 4
V1
V0
V2
1
A
A
…..
P2 B2
P1 B1
Q1
Q2
W1
Vd
Ad
2
Pd Bd
Qd
W2
V j+1 = A j+1 V j W j+1 = B j+1 V j Wd
Fig. 4 Wavelet based Decomposition and Multiresolution Representation of 3D Polygonal Model
V V v4j
v2j
t1
v1j t4
t2 j
j
v6j
v1j+1 (= v1j )
j+1
v2j+1 (= v2j )
t1j+1
j
A j+1
v5j
v3j+1 (= v3j )
t3j v3
B j+1
j
w1j+1
[
where V j = v1j
Wavelet Coefficient Vectors
j
coefficient vectors w1j +1 , w 2j +1 , w3j +1 . We can formulate this wavelet transform by using the Lazy wavelet[18] as;
Wj
Fig. 5
4-to-1 subdivision connectivity scheme. Figure4 illustrates the decomposition and multiresolution representation of the polygonal model by using lazy wavelet transform[18]. The high-resolution polygon are decomposed into a low-resolution part and a detail part. These detail parts are expressed as the wavelet coefficient vectors. From the geometrical view, the transform can be interpreted as shown in Figure 5. Four triangles j j j j t1 , t 2 , t3 , t 4 at high resolution level j are merged into one triangle t1j +1 at low-resolution level j+1. The positions of the three vertices v1j , v 2j , v 3j out of six v1j , , v 6j are unchanged in the approximated triangle t1j +1 even at the low-resolution. The detail parts which is lost in the t1j +1 are represented and stored as the three vectors w1j +1 , w 2j +1 , w3j +1 each of which are called wavelet coefficient vectors. Each of
w1j +1 ,
w 2j +1 ,
starts from a midpoint of the edge of
v 4j ,
v 5j ,
v 6j
one of vertices which resolution level. Therefore, multiresolution representation j j v1 , , v 6 can be represented as
w3j +1 the t1j +1
respectively and ends at
are lost at the lower in this case, the of the polygon the combination of an
approximated polygon v1j +1 , v 2j +1 , v 3j +1
and wavelet
m
]
T
denotes the matrix
[
n
]
matrix whose row corresponds to a wavelet coefficient vector at resolution level j+1. nj+1 denotes the number of wavelet coefficient vectors at resolution level j+1, and can be calculated from nj+1 = mj − mj+1. The matrices Aj+1 and Bj+1 are called as analysis filters at resolution level j+1. Aj+1 is the (mj+1 x mj ) nonsquare matrix which characterize the triangle reduction by merging four triangles into one. The example the triangles in Figure 5 (mj =6, mj+1=3), Aj+1 is defined as;
A
Geometrical Interpretation of the Wavelet Transform for 3D Polygons
vjj
whose row corresponds to vertex coordinates at resolution level j (high resolution), V j+1 is the one at low resolution level j+1, and mj is the number of vertices at resolution level j. W j +1 = w1j +1 w 2j +1 w j +j+11 T denotes the
w2j+1 w3j+1
v 2j
(1) (2)
j +1
1 0 0 0 0 0 = 0 1 0 0 0 0 0 0 1 0 0 0
(3)
Bj+1 is the (nj+1 x mj ) non square matrix, and produces the wavelet coefficient vectors each of which starts from the midpoint of the edge in the low-resolution triangle and ends at the vertices which are lost at lower resolution. For the example of the triangles in Figure5, B j+1 is defined as; −1 −1 0 2 0 0 1 j +1 B = − 1 0 − 1 0 2 0 (4) 2 0 −1 −1 0 0 2 The process defined in eqs(1)and (2) is recursively applied to the low resolution part until the coarsest representation of the polygon V d is obtained. After this process, the wavelet coefficient vectors W1 ...., Wd at each resolution level are also obtained. The processes can be formulated as; Vd
=
Ad Ad-1 Ad-1 .... A2 A1 V 0
(5)
Wd =
Bd Ad-1 Ad-2...... A2 A1 V 0
(6)
On the contrary, the inverse wavelet transform can be formulated as; V j = P j+1 V j+1 + Q j+1 W j+1
(7)
where P j+1 and Q j+1 are called as synthesis filters. The synthesis can be viewed more concretely as involving two steps: splitting of a triangle of the low-resolution polygon into four sub-triangles by introducing the new vertices at 5
edge midpoints by using P j+1, and correcting the position of these new vertices according to the wavelet coefficients vectors by using Q j+1. A mathematical relation between the matrices of analysis filters and synthesis filters is defined as;
[P
j +1
Q
j +1
]
A j +1 = j +1 B
−1
(8)
For the example of the Figure5, it can be defined as;
[P
5.
j +1
Q j +1
]
1 = 2
2 0 0 1 1 0
0 2 0 1 0 1
0 0 0 1 0 0
0 0 2 0 1 1
0 0 0 0 1 0
0 0 0 0 0 1
(9)
The Embedding and Extraction Algorithm of the Watermark
(1) Embedding Policy invariant to the affine transformation From the application view point, the embedded watermark should remain under the affine transformation done on the polygonal model. Therefore the watermark must be embedded in the geometric measure which is invariant to the affine transformation, and the value of the measure must be modified according to the watermark. In our embedding method, we modify the ratio of the norm of a wavelet coefficient vector to the length of the other edge in the polygonal model. Shown in Figure 6, the edge v aj v bj on which the start point of the wavelet coefficient vector wij lies is selected as a partner, and the ratio is defined in eqs(10)(11).
vad
wij lij ( j=d )
vi
vˆ aj
vbd
lij (j=d−1,..., 2,1)
(10)
vˆ aj , vˆ aj ∈ Vˆ
j
∈V
d
v ad , v bd
( j = 1,2,, d − 1) (11) (j = d )
For the reasons of the watermark extraction, note that, in eq(11), the reference length li j at resolution level j must be measured between the vertices on the intermediate modified polygon Vˆ j in which the watermark has been already embedded at the coarser resolution levels than j ( e.g. j+1, j+2,...., d ). The concrete modification method of the ratio α i j depending on the watermark is discussed below. (2) Selection Policy of the Wavelet Coefficient Vectors for Perceptually Invisible Watermarking The embedded watermark should be perceptually invisible. Usually the human eye is not sensitive to the small geometric changes in the bumpy areas. The norm of the wavelet coefficient vector wij indicates the degree
means more bumpiness. Therefore, we only select the wavelet coefficient vector wij for embedding whose norm satisfies w ij > δ 1 , where δ1 is the user-specified lower bound for the norm of the wavelet coefficient vector for embedding. (3) Control of the Geometric Tolerance in Embedding Considering the purpose of the 3D polygonal model, the geometric error caused by the watermark embedding should be restricted within specified tolerance. When the wavelet transform by the lazy wavelet is used, the geometric relation between the maximum error ∆max at the vertex and the upper bound δ2 of the norm change of each wavelet coefficient vector caused by the embedding, can be defined by eq(12).
∆max =
j-1
wij
vaj
vˆ j − vˆ j , b a li j = d d v a − v b ,
li j
of bumpiness of the small area near the wij . Larger wij
5.1 Fundamentals of the Watermark Embedding
vij-1
α i j = w ij
vbj vˆbj
Fig. 6 The Ratio of the Wavelet Coefficient Vector to the Partner Edge
d +1 δ2 2
(12)
where d is the coarsest resolution level of the wavelet transform. If we denote the geometric tolerance between the original polygon and the watermarked polygon as ε, ∆max must be satisfied with ∆max < ε . From this condition, in order to keep the tolerance ε , the allowable norm change δ2 for each wavelet coefficient vector wij caused by the embedding must be satisfied with
δ2 ≤
2ε d +1
(13)
6
watermark into the value of wij , range from O(10r) to The range for watermark embedding
δ2
µ
10r
10q
O(10p) in wij can be modified.
w ij
eq(10), if α i = O(10 ), j
wˆ ij
10p
10q
10q-p+s
Fig. 7
10r-p+s
αˆ ij
10s
Manipulation of the Decimal Digits for embedding the Watermark
αˆ i j = α i j + β i j
β i j = b1′ ×10 r − p + s + b2′ × 10 r − p + s −1 + + bk′ × 10 r − p + s − k (15) Therefore the wavelet coefficient vector wij is modified to wˆ ij as the eq(16).
(
V
WT until level d
0
5.2 Watermark Embedding Algorithm Figure8 shows the detail flow of a proposed watermark embedding algorithm. The algorithm consists of the processing steps described below;
ε
(Vˆ j )
µ k WT: Wavelet Transform WCV: Wavelet Coefficient Vector
eq(13)
Length Calculation
Vd
eq(11)
eqs(1)(2)
Selection of WCV wi ≥ δ1 j
Watermark
(16)
For the watermarking, the modified wavelet coefficient vectors wˆ ij calculated from eq(16) are used as the inverse wavelet transformation to obtain the watermarked polygon data.
Conversion of Tolerance
W1,…,Wd
)
wˆ ij = αˆ ij li j wij wij
10r , µ =10q (p>r>q), where y=O(10x) means y has the decimal order of 10x. In this case, in order to embed the
Original Polygon
(14)
where;
δ2 =
Shown in Figure7, assume that the wij = 10p,
δ1
above condition becomes
modified to αˆ ij by eqs(14) and (15).
(4) Embedding Method of the Watermark into the Wavelet Coefficient Vectors by Digit Manipulation In the common approaches to the watermarking on the image, the least significant bit in the integer variable is modified according to the watermark. However, in the case of the 3D polygonal model, the vertex coordinates can be usually stored in the floating point variable, and the least significant bit of the binary mantissa in the floating point variable can be easily destroyed by the round off error caused by the geometric operation. Therefore the method of only modifying the less significant bit is not adequate for our purpose. In our method, we modify several decimal digits in the certain range of less significant part of the mantissa of the variable α ij defined in eq(11), according to the watermark. We denote the lower bound of this range as µ. If we use the IEEE double precision floating point number, then µ approximately equals to the one of order of 10-15.
d
δ2
li j
w ij Raito Calculation
α ij Modification of Ratio by Watermark
eq(10)
B
αˆ ij
d
eqs(14)(15)
w ij
Vd
Modification of WCV
wˆ ij (Wˆ j ) One Level Inv. WT
eq(16)
(Vˆ j )
Fig. 8
From the relation of
equivalent that the value of ratio α ij can be modified in the range from O(10 r-p+s) to max{ O(10q-p+s), O(10q) }. If we denote the entire watermark data in the form of a binary sequence of length lb as B=[b1, b2,......, blb] ( { 0, 1}∋ bi ), and a subsequence of the length k in B as B ′ = [ b1′ , b2′ ,, bk′ ] (k ≤ r− q), the ratio α ij are
k decimal digits
α ij
s
eq(17)
Vˆ
j −1
Vˆ 0
Watermarked Polygon
Detail Flow of the Wavelet-based Watermark Embedding Algorithm 7
Step 1: The following data needed for the embedding are prepared; V 0 : original polygonal model data, B : watermark data of a binary vector of length lb Also the following control parameters for embedding are pre-specified by the user; ε : geometric tolerance, k : bit length in the watermark which can be embedded per one wavelet coefficient vector, δ1 : lower bound of the original wavelet coefficient vector norm for invisible embedding, µ : lower bound the embedding range, d : the coarsest resolution level of the wavelet transform.
vector wˆ ij by using eq(16). Step 5-5: If i < nj , then i ← i + 1 and go to Step 5-1. Otherwise prepare the set of modified wavelet coefficient vectors Wˆ j = wˆ 1j wˆ 2j wˆ j j T in this
[
m
]
resolution level, and go to Step 6. Step6: Execute the inverse wavelet transform at one resolution level to obtain polygon data Vˆ j −1 of one finer level by eq(17); j −1
Vˆ
= P jVˆ j + Q jWˆ
, Vˆ d = V d
j
(17)
Step 7: If j > 0 , then j ← j − 1 and repeat from Step 5. Otherwise stop.
Step 2: The wavelet transform of eq(1) and (2) are iteratively applied to V0 until the resolution level d. And set of wavelet coefficient vectors at every resolution W j = w1j w 2j w j j T ( j=1,2,....,d), and coarsest
[
n
]
From the embedding algorithm, we can obtain the watermarked polygon data as Vˆ o
d
approximation V are obtained.
5.3 Watermark Extraction Algorithm Figure 9 shows the detail flow of the proposed watermark extraction algorithm. Basically the extraction process is reversed to that of the embedding. The extraction algorithm consists of the processing steps described below;
Step 3: Upper bound of the norm change δ2 is calculated from ε by using eq(13). Step 4:
Set j=d, and execute the following steps.
Step 5:
Set i=1, and execute the following steps.
Step 1: The following data has been prepared; Vˆ 0 : watermarked polygon data, V 0 : original polygonal model data, and all control parameters used at the time of embedding.
Step 5-1: If w ij ≥ δ 1 , then this wij can be selected for embedding, and go to Step5-2. Otherwise go to Step 5-5. Step 5-2: The ratio α ij is calculated from eqs(10)(11). Step 5-3: By using eq(15), calculate β i j by cutting the appropriate subsequence B ′ = [ b1′ , b2′ ,, bk′ ] of
Step 2: The wavelet transforms of eqs(1)(2) are iteratively applied both to the Vˆ 0 and V 0 until the resolution level d.
[
And W j = w1j
length k from B. Then, calculate the modified ratio αˆ ij by using eq(14). Step5-4: Calculate the modified wavelet coefficient
Wˆ
[
= wˆ 1j
j
wˆ 2j
w 2j
wjj
wˆ j j m
]
m
T
]
T
,
(j=1,2,....,d)
are
both
obtained.
δ1
d Original Polygon
V
WT until level d
0
eqs(1)(2)
W1,…,Wd
V
V1,…,Vd
WT until level d eqs(1)(2)
µ k
Conversion of Tolerance eq(13)
Selection of WCV
w ij
w ij ≥ δ 1
d
Watermarked Polygon
ˆ0
ε
Vˆ 1 , , Vˆ d Wˆ 1 , , Wˆ d ( wˆ ij )
δ2
(li j ) Calculation of Ratio
α ij
eq(18)
(lˆi j ) Calculation of Ratio eq(18)
αˆ i j
Extracted Watermark Comparison j of Two Ratios
B
wi
β i j = αˆ ij − α ij
Fig. 9
Detail Flow of the Wavelet-based Watermark Extraction Algorithm 8
Step 3: Upper bound of the norm change δ2 is calculated from ε by using eq(13). Step 4: Set B =[ null ]. Step 5: Set j=d, and execute the following steps. Step 6: Set i=1, and execute the following steps. Step 6-1:
If
w ij ≥ δ 2 , then go to Step6-2.
Otherwise go to Step 6-4. Step 6-2: Calculate the original and modified ratio α ij and αˆ ij respectively by using eq(18).
α i j = w ij
li j , li j = v aj − v bj , ∀j , v aj , v bj ∈ V
j
αˆ ij = wˆ ij
lˆi j , lˆi j = vˆ aj − vˆbj , ∀j , vˆ aj , vˆbj ∈ Vˆ
j
(a) Original Polygon
(b) Watermarked Polygon
(-10< x,y,z 0 , then j ← j − 1 and repeat from Step 6. Otherwise stop. From this extraction algorithm, watermark B is extracted from Vˆ 0 . 6.
Simulation Results The software for watermark embedding and extraction was developed, and the performance of the watermarking method were evaluated. The perceptual invisibility, invariance to the affine transformation, robustness for the additive random noise, geometric error distribution caused by the embedding and processing time were investigated. The “human head” shown in Figure10-(a) was used as an example of original polygon data which had 1026 vertices and 2048 triangles. The polygon had the mesh
(a) Original Polygon Data (1026Fig. Vertices, Triangles)for 10 2048 An Example
Fig. 10
(b) Watermark
(70x70 Binary Image) the Watermarking An Example of the Watermarking
Fig. 11 The Change of Appearance of the Watermarked Polygon at different Tolerance Settings. topology fit to 4-to-1 subdivision schemes. The dimension of the original polygon is approximately 20x20x20. A binary image 70x70 (approx. 620 byte) shown in Figure 10-(b) was embedded as a watermark into this polygon. In all case of these simulations, δ 1 is set as δ 1 = δ 2 . (1) Perceptual Invisibility of the Watermark Figure11-(b,c,d) shows the change of appearance of the watermarked polygon at different settings of the The other parameters for geometric tolerance ε. embedding were set as k=5, d=4, µ=10-20 and δ1=δ2. The watermarked polygon is perceptually identical to the original polygon in case of ε = 10−1 and 10−2, but visible in case of ε = 1. It is known that the geometric tolerance ε must be appropriately set corresponding to the polygon dimension in order to make the embedded watermark perceptually invisible and to avoid quality degradation. (2) Invariance to the Affine Transformation The effect of the affine transformation on the embedded watermark was investigated. The watermark was embedded into the polygon under the same condition of (1) and ε = 10−2. Then rotation, translation, scaling and the concatenation of them shown in Figure12 were done for the watermarked polygon, and finally the watermark was extracted. The extracted watermark after the transformation is also shown in the figure. The result showed that the watermark remained complete without loss in all case. This shows that our watermarking is always invariant to the affine transformation. (3) Robustness for the Noise Imposition.
9
Translation
Rotation
Scaling
(Rotation +Scaling +Translation)
Concatenation
Fig. 12
Extracted Watermarks after several Affine Transformations
In order to investigate the robustness of the watermark, random noise was added to the vertex coordinates in the watermarked polygon, and the bit error rates (BER) of the watermark were evaluated. The BER is defined by the ratio of the number of destroyed bits to the total bit length in the extracted watermark. Figure13-(a) shows the relation of BER with the number of the vertices on which the noise of 10-5 order was imposed. While Figure13-(b) shows the relation of BER with the strength of the noise in the case that the noise was imposed on all vertices. From both results, it was shown that additive noise on the vertices of the watermarked polygon more or less causes the destruction of the embedded watermark in our method. Figure14 shows the corresponding degradation of the embedded watermark at different BER values. (4) Capacity of Embedding Usually there is a trade-off relation between the capacity of the watermark and the degradation of the original polygon geometry. In our method, if we take more bit length k which can be embedded per one wavelet coefficient vector, then the capacity theoretically can be
60
ε =10-2 d = 4, k = 5 No. of vertices = 1026 Amplitude of Noise = O(10-5)
50
BER [%]
40
BER=0%
30
BER= 6.8%
BER= 2.2%
BER= 10%
20 10
BER= 23.3% BER= 30.3% BER= 41.2%
BER= 51.7%
0
0
20
40
60
80
100
Rate of Noise-imposed Vertices [%]
Fig. 14
Degradation of the Watermark at different Noise Imposition Settings
(a) BER versus Rate of Noise-imposed Vertices
ε d = 4, k = 5 No. of vertices = 1026 Rate of Noise Impose =100%
50
BER [%]
40 30 20 10 0 1.E-15
1.E-13
1.E-11
1.E-09
1.E-07
1.E-05
1.E-03
Amplitude of Noise (b) BER versus Amplitude of imposed Noise
s
Fig. 13
Bit Error Rate (BER) of the Watermark at different Noise Imposition Settings
( x10-3)
Average Positional Error of the Vertex
60
=10 -2
5.0
4.5
4.0
ε =10-2 d=4
3.5
3.0 1
2
3
4
5
6
Embedded bit length of the Watermark per one Wavelet Coefficient Vector ( k )
Fig. 15 Average Positional Error of the Vertices caused by the Watermark Embedding 10
600
k=5,d=4 k=5,d=2 k=3,d=4 ε =10-2
Frequency
500 400 300 200 100 0 0
Fig.16
2 4 6 8 Geometric Error in each Vertex
10 (x10-3)
Error Distribution of the Vertices at different settings of the Embedding Condition
increased. Figure15 shows the relation between k and the average positional error of the vertices caused by the watermark embedding. The result shows that the degradation of polygon geometry are saturated even if k increases. This is because the embedding digits are taken from a more significant digit to a lesser one. (5) Geometric Error Control The distribution and boundary of the geometric error of the watermarked polygon was investigated at different settings of the embedding condition ( k, d ). At the same time, it was verified whether pre-specified geometric tolerance ε was completely satisfied in this embedding. Figure16 shows these results. It was shown that the geometric error of all vertices in the watermarked polygon distributes in the range less than the pre-specified tolerance value (ε =10-2). Therefore the geometric error control method of the proposed watermarking is effective. (6) Processing Time As for the polygon of 1024 vertices and k=5, d=4, ε=10-2, it takes about 35sec for embedding and 55sec for extraction by 266Mhz Pentium II computer. In case of a polygon of 258 vertices and k=5, d=3, ε=10-2. Most of the processing time was consumed for the wavelet transform. The processing time is approximately proportional to the square of the number of vertices. 7. Conclusions and Future Work In this paper, we proposed a new digital watermarking method for 3D polygonal model based on the wavelet transform and data fusion in the frequency domain for the purpose of copyright protection in the area of CAD,CAM,CAE and CG. The conclusions of the research are summarized as follows; (1) Technical requirements on the watermarking for 3D polygonal model were discussed. It was pointed out that the perceptual invisibility and robustness of the watermark, the geometric error control due to the embedding, and the invariance of the watermark to the
affine transformation must be considered in the watermarking. (2) The mathematical formulation of wavelet transform and inverse wavelet transform for 3D polygons were shown. And the basic concept of the watermarking in the frequency domain was proposed. The watermark was embedded by modulating the wavelet coefficient vectors with the watermark in the frequency domain. (3) Detail algorithm of watermark embedding, and the extraction process based on the wavelet transform was discussed. The embedding method considering the effective digits and invariance to the affine transformation were proposed. (4) The performance of the proposed watermarking method were evaluated through the simulation. The results demonstrated that the control of the invisibility of the watermark and geometric error control of the embedding were realized. Also the invariance of the watermark to the affine transformation was verified. Several possible uses of this watermarking on the polygon can be considered; hiding copyright information, tamper detection of the model, tracking illegal copying and distribution, inserting textual comment or small programs into the model. In future watermarking should be more robust to possible geometric operation, noise imposition and intentional attack. The embedding capacity should also be increased, and processing time be decreased. The method to extract the watermark without the original polygon should be proposed. Moreover, watermarking should be expanded to the free-form surface model or solid model which has to be more secret than the polygonal model in the CAD/CAM area.
Acknowledgements This research was financially supported by NK EXA Co. Ltd. The authors would like to thank Mr. S.Furushima for providing us the software of the polygonal modeling. References [1] Bender,W., Gruhl,D., Morimoto,N., Lu,A.: Techniques for data hiding, IBM Systems Journal, Vol.35, No.3-4, pp313-336 (1996). [2] Brassile,J., Low,S., Maxemchuk,N. and O’Gorman,L.: Electronic Marking and Identification Techniques to Discourage Document Copying, Proc. IEEE INFOCOM’94, Vol.3, pp1278-1287(1994). [3] Nakamura, H. and Matsui, K.: Digital Watermarking onto Japanese Documents by Seal Image (in Japanese), Trans. of I.P.S.J, Vol.38, No.11, pp2356-2361 (1997). [4] Kundur, D. and Hatzinakos,D.: Robust digital image watermarking method using wavelet-based fusion, Proc. of IEEE Int. Conf. on Image Processing, Vol.1, pp.544-547 (1997). [5] Xia,X., Boncelet,C.G. and Arce, G.R.: A multiresolution watermark for digital images, Proc. of IEEE Int. Conf. on Image Processing, Vol.1, pp.548551 (1997). 11
[6] Ohnishi, J. and Matsui, K.: Embedding a seal into a picture under orthogonal wavelet transform, Proc. of Int. Conf. on Multimedia Computing and Systems, pp.514-521 (1996). [7] Ohnishi, J. and Matsui, K.: Image coding for copyright protection by using wavelet (in Japanese), Trans. of I.P.S.J., Vol.38 No.3, pp.534-539 (1997). [8] Matsui, K., Ohnishi, J. and Nakamura, Y.: Embedding a signature to pictures under wavelet transformation (in Japanese), Trans. of IEICE, Vol. J-79-D-II, No.6, pp1017-1024 (1996). [9] Pitas, I.: A method for signature casting on digital images, Proc. IEEE Int. Conf. on Image Processing, Vol.3, pp.215-218 (1996). [10] Wolfgang, R.B. and Delp, E.J.: A watermark for digital images, A method for signature casting on digital images, Proc. IEEE Int. Conf. on Image Processing, Vol.3, pp.219-222 (1996). [11] Swanson, D.M., Zhu, B., Chau, B. and Tewfik, A.H. : Robust Data Hiding for Images, Proc. of IEEE Digital Signal Processing Workshop, pp.37-40 (1996). [12] Swanson, D.M., Zhu, B., Chau, B. and Tewfik, A.H. : Multiresolution Video Watermarking using Perceptual Models and Scene Segmentation, Proc. of IEEE Int. Conf. on Image Processing, Vol.2, pp.558561 (1997). [13] Digimarc Corporation : http://www.digimarc.com /applications/copyright/copyright_in.html. [14] SignumTechnologiesLtd.: http://www.signumtech.com/. [15] Ohbuchi,R., Masuda, H. and Aono, M.: Watermarking 3D polygonal models, Proc. ACM Multimedia ’97, (1997). [16] Ohbuchi,R., Masuda, H. and Aono, M.: Watermarking three dimensional polygonal models through geometric and topological modifications, IEEE Journal on Selected Areas in Communication, , Vol. 16, No. 4, pp. 551-560 (1998). [17] Lounsbery, J.M.: Multiresolution Analysis for Surfaces of arbitrary Topological Type, PhD Thesis, Univ. of Washington (1994). [18] Stollnitz, E.J., DeRose, T.D. and Salesin, D.H. : Wavelets for Computer Graphics: Theory and Applications, Morgan Kaufmann, (1996). [19] Eck,M., DeRose,T.D., Duchamp,T., Hoppe, H., Launsbery, M., Stuetsle,W.: Multiresolution Analysis of Arbitrary Meshes, Computer Graphics ( SIGGRAPH 95 Proc.), pp.173-182 (1995).
12
