Shape Deformation Using Skeleton ... - Semantic Scholar

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 11, NO. 2, APRIL 2014

409

Shape Deformation Using Skeleton Correspondences for Realistic Posed Fashion Flat Creation Xianmei Wan, P. Y. Mok, Member, IEEE, and Xiaogang Jin

Abstract—We propose a 2D shape deformation method to fit technical drawings of garments (“flats”) to body figure drawings with a diversity of fashion poses. We first dress a flat onto a body figure in a standard standing pose using Radial Basis Function (RBF) mapping. For different types of clothing, we suggest two levels of treatment to determine handles automatically. We deform the flats using the selected handles to create realistic fashion sketches that fit the garments onto fashion figures in different poses. Shape deformation is performed to minimize the distortion of all the triangles of the garment mesh and preserve garment properties in the deformation. Finally, the garment details, such as style lines and seams, are deformed accordingly for realistic deformation results. Experimental results have shown that our method can deform various garment flats to fit fashion figures in different poses. Note to Practitioners—Two-dimensional shape deformation can approximately imitate an object’s change and has been used widely in graphic applications. This paper was motivated by the problem that in the fashion industry a design must be repeatedly sketched for varied presentation purposes. We present an automatic and effective method to create realistic fashion sketches with full color and details, from technical drawings of the products (“flats”). The method helps designers efficiently present their ideas in a professional manner, detect any inappropriateness in the designs, provide instant feedbacks on design and presentation, and thus accelerate the design process. Preliminary experiments suggest that this approach is feasible. In future research, we will address the problem of rotation and incorporate other textile material properties to improve the sketch deformation results. Index Terms—As-rigid-as possible, fashion design, shape deformation, skeleton-based correspondence deformation.

Manuscript received November 12, 2012; revised June 09, 2013; accepted October 17, 2013. Date of publication January 17, 2014; date of current version April 03, 2014. This paper was recommended for publication by Associate Editor C.-H. Chu and Editor D. Tillbury upon evaluation of the reviewers’ comments. This work was supported in part by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. PolyU5269/09E), and the Hong Kong Polytechnic University (with project account: A-PL78). The work of X. Jin was supported by the National Natural Science Foundation of China under Grant: 61272298 and the China 863 Program under Grant: 2012AA011503. X. Wan is with the State Key Lab of CAD&CG, Zhejiang University, 310058, Hangzhou, Zhejiang, China, and also with the Institute of Textiles and Clothing, Hong Kong Polytechnic University, Kowloon, Hong Kong, and the Zhejiang University of Finance and Economics, 310018, Hangzhou, Zhejiang, China. P. Y. Mok is with the Institute of Textiles and Clothing, Hong Kong Polytechnic University, Kowloon, Hong Kong (e-mail: [email protected]). X. Jin is with the State Key Lab of CAD&CG, Zhejiang University, 310058, Hangzhou, Zhejiang, China. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TASE.2013.2296554

I. BACKGROUND OF THE RESEARCH

I

N FASHION design, the most natural way for a designer to create a new style is with a pencil and a sheet of paper. Accordingly, sketch-based modeling has attracted much interest in recent years. The real strength of a sketch-based system is that, as the name claims, designers are only expected to draw in order to create styles, which is something that they are used to doing. To develop a truly user-friendly design system, it is important to understand how a design is developed for production from the early stages of creating ideas, developing ideas by research, design sketching and selecting fabrics trimmings and colors for the collection [1]. A variety of sketches are used in the everyday design process. Every season, based on a determined design direction, designers select from the pool of quick sketches and turn them into real products for their target group of customers. In this process, two types of drawing are developed for production and presentation purposes [2]. If a design is selected, then technical sketch is developed for production purposes. Technical drawings, also called “flats,” are specialized drawings like blueprints for communicating the design ideas with the production team. They are drawn to scale and include sewing and construction information. Flat sketches look short and wide, just like a garment laid flat onto a table, containing no three-dimensional sense of the garment on a body figure. Flat drawings are specific, precise and literal; they are used to explain the design to the production team for pattern making and prototype development. Among the different types of sketches, the flat drawing is the most important sketch in the product development process, any mistake in the “flat” would cause problems in the final garment as it is part of the product specification. Because of the prevalence of globalization in apparel production, the accuracy of the flats is now more critical than it used to be. As a universally understood language, flat sketches are used to communicate the design ideas with other professionals in the offshore production team. Because of this, designers nowadays often bypass other sketches but create flat sketches directly. Computer software like SnapFashun [3], Verve Sketch [4] and sketch system [5] have been developed to help designers create flat drawings with just a few clicks. These systems can output flat sketches in vector graph format and can work together with Abode Illustrator to add color, prints and patterns. These systems dramatically shorten the time needed for creating a design and enable easy style editing. In addition to flats, fashion sketches, also called fashion illustrations, are needed for presentation purposes. Fashion sketches

1545-5955 © 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

410


II. RELATED WORK

Fig. 1. Human figure template croquis: (a) standard pose and (b)–(f) five other poses.

are figure drawings, which show merchandise on figures with accessories and jewelry, for presenting to target customers how merchandise can be arranged into groups and what seasonal themes are being followed. Fashion illustrations are often developed with the help of a set of master sketches with template figures in different poses called croquis, see Fig. 1. The croquis shows the figure in correct proportion and by sketching over the croquis on semitransparent paper, the drawing process is considerably speeded up [2]. Fashion sketches give customers a sense of attitude, showing how the customer might feel in the fashion outfit and into what target category the garment look fits. This is achieved by picking the right pose to match the specific clothing and fashion look. Fashion companies have their own set of posed template figures for presentation purposes [6]. In the fashion industry, to demonstrate the “feels” of the garment, designers need to create fashion sketches based on the flats with the croquis template as a guide. The same design needs to be redrawn a number of times for different colors and patterns. If the garment has a lot of details, then repeated sketching is tedious, dull and time consuming. In this paper, we present a method to generate fashion sketches from flat drawings by shape deformation. The flats can be any vector diagrams developed by designers using generic design software like Adobe Illustrator, CorelDraw or other specialized systems [3]–[5], which include different patterns, details and colorways. The proposed method is based on 2D shape deformation, which can create posed flats with full color and details in just a few seconds. A. Contributions We provide a novel tool to assist fashion designers to improve their efficiency in the early stages of fashion design. It helps the designers to assess how the garments are put on body figures, to observe the garments in different poses, to select suitable poses for creating the “feels” and presenting the products and also to evaluate and edit the designs. We deform the flat sketches according to different posed figure templates with respect to the deformation properties of the garment. The remainder of this paper is organized as follows. Section II reviews the recent developments of the related technologies. Section III first describes the overall methodology and then explains the deformation algorithm in detail. Experimental results are given in Section IV, which also discusses the advantages and the limitations of the method. Section V concludes the paper and outlines further work.

Research related to this paper includes sketch-based garment design, 2D shape deformation and skeleton-driven deformation. We review the recent developments in these areas one by one. Sketch-based garment design provides a natural and effective way to communicate design ideas by mimicking the traditional paper-and-pencil design style. Since some early work in interactive design systems [7]–[9], many new developments have been achieved recently, in the domain of sketch-based design. Igarashi et al. [10] described a sketch-based method to dress clothes on the body. The user paints freeform marks on the clothes and corresponding marks on the body and in a few seconds the system places the clothes on the body by matching the corresponding marks. Bourguignon et al. [11] provided a 3D sketching method that can be used to design garments over virtual actors. It used a 3D model to obtain the body proportions and used that to assist the design process. Turqin et al. [12], [13] developed a sketch-based interface, such that the user can sketch directly onto the 3D body of the character and quickly construct 3D virtual garments. They used the distance from the 2D garment silhouette to the character model to infer the variations of the distance between the remainder of the garment and the character in 3D. Based on [12], Decaudin et al. [14] presented an automatic approach for clothing design, which could create developable 2D patterns for the sewing of real replicas of the virtual garment. It can also generate folds for the 3D garment to improve the realism. The methods in [10]–[14] are all based on 3D model of the character, which becomes a major limitation for ordinary use. Ma et al. [15] developed a sketch pad for conceptual design of 2D garments. It can help designers to identify various design solutions before the product goes through the product development process. The designers draw outlines of different parts of the garment and the system automatically makes reasonable geometric inferences about the process-planning data of the garment using an existing design database. A survey on 3D garment design can be found in [33]. Recently, Liu et al. [36] proposed a method to retrieve 3-D CAD models based on 2-D pen-based sketch inputs, which can account for users’ drawing habits. To sum up, sketch-based garment design provides a simple and interactive way for designers to explore new directions and to communicate ideas with others. It must obtain further development in the garment design community in the future. 2D shape deformation has been widely studied in the community of computer graphics. It has been successfully applied in image-editing, computer-aided design, computer simulation and computer animation. Many deformation algorithms have been presented that generate satisfactory results in specific applications according to different circumstances. Among them, Igarashi et al. [16], [17] presented a two-step close-form algorithm, called the as-rigid-as-possible algorithm, to realize very efficient shape manipulation by solving a set of simultaneous linear equations and minimizing quadratic error metrics. The method avoids the traditional expensive computational approaches of physically-based simulation or nonlinear optimization and thus, can be used for real-time manipulation of 2D shapes – either in the form of vector graphics or bitmap images. Experimental results showed that the method can deform a 2D shape naturally, just like manipulating the object

WAN et al.: SHAPE DEFORMATION USING SKELETON CORRESPONDENCES FOR REALISTIC POSED FASHION FLAT CREATION

with two hands, by moving, rotating, stretching, squashing, and bending it interactively. Based on [16] and [17], Weng et al. [18] proposed to deform the 2D shape using nonlinear least square optimization. Yang et al. [19] presented an intuitive 2D/3D shape deformation method according to the rigidity and stiffness of the object’s material and the user can also tune the stiffness of the object during the manipulation. Afterwards, Yang et al. [20], [21] proposed to manipulate the 2D shape via a topology-aware rigid grid, automatic feature matching and hierarchical interpolation [31]. Yu and Zhang [31] presented a novel framework for 2D shape deformation and image magnifying by incorporating topology preservation constraints to the deformation. As-rigid-as-possible concept was also applied in 3D surface modeling [29], [32]. While revising this paper, two recent publications came to our attention in the area of 2D shape deformation [34], [35]. Sykora et al. [35] used rough image registration to texture hand-drawn cartoon, avoiding the calculation of 3D model correspondence. They produced convincing results on cartoon video sequences. Different from most positional constrained deformations, Solomon et al. [34] proposed velocity-based as-killing-as-possible algorithm to generate near isometric deformation. Inspired by Igarashi’s work [16], [17], we propose a method in this paper to generate design sketches in various fashion poses from standard flat drawings by 2D shape deformation. In contrast to Igarashi’s approach, in which handle points are defined interactively, our method defines handle points automatically along the skeleton of the human figure and based on the characteristics of the clothing products, achieving realistic fashion sketches in seconds in an automatic process. There is a large volume of literature in skeleton-driven shape deformation. The overall shape of the object deforms according to the skeleton configuration. Capell et al. [22] proposed to interactively simulate body deformation controlled by the underlying skeletons, which embedded the body into the coarse volumetric control lattice in order to meet the skeleton constraints rapidly. Lewis et al. [23] presented pose space deformation to unify shape interpolation and skeleton-driven surface deformation. Baran et al. [24] attached a skeleton to the mesh surface of a character, allowing skeleton motion data to animate the character automatically. Instead of controlling movement of vertices, Yan et al. [32] use skeleton to control the simplices defining the model. By controlling simplices, the model is deformed without defining vertex weights or fixing any arbitrary vertices. In this paper, we propose to use skeleton correspondence to control 2D shape deformation to generate posed fashion sketches, as clothing in the fashion sketch deforms in a consistent manner with changes in the pose of the figure in the sketch. III. METHODOLOGY AND DETAILED ALGORITHM IMPLEMENTATION A. Method Overview We present a method to generate realistic posed fashion flats in a three-stage approach. In the first stage, we generate a triangle mesh based on the silhouette outline of a selected garment flat drawing and we “dress” the garment by mapping the flat on a figure model in a standard standing pose (rest pose). In

411

Fig. 2. The framework of our garment deformation system.

the second stage, we select a new pose to present the garment and generate the fashion sketch from the flat drawing by 2D shape deformation. The deformation is done by first automatically selecting handle points with reference to the skeleton in the rest pose and then deforming the mesh model according to these handle points in the new pose. Considering the textile properties, we add constraints to preserve the total length of the silhouette during mesh deformation. Therefore, the proposed shape deformation method can easily generate many posed flats fitting the garments to a set of figure template croquis in different poses. In the third stage, we deform the style lines and pattern curves with respect to the deformed garment mesh in order to generate realistic fashion sketches. Fig. 2 describes the framework of our garment deformation system and the details of each stage procedure are described in later sections. B. Dressing the Garment Onto the Figure 1) Triangulation of the 2D Garment: As previously mentioned, 2D flat drawings can be designed manually using Adobe Illustrator or CorelDraw, or generated from other specified software systems [3]–[5]. Given the vector graph of any 2D flat drawing, we depict its boundary outline with a simple closed polygon using lines and splines. We generate a triangular mesh for the garment flat based on its boundary outline by Delaunay triangulation [16], [25]. It is noted that better deformation results can be achieved if the mesh model is composed of nearequilateral triangles of similar sizes. 2) Dressing the Garment Onto the Body: After triangulation, we dress the garment onto the body figure in rest pose [Fig. 1(a)]. It is the only interactivity in the whole process of fashion sketch generation. This can be done by anchoring the garment on top of the body figure by matching a few key feature points, for example, the high-point shoulder (HPS). Although garment flat drawings are usually developed on top of a template croquis, the croquis figure used for drawing flats is often different from the fashion figure for presentation (Fig. 1). In the industry, the croquis figure used exclusively for drawing flats looks “fat” compared with the fashion figure used for presentation [6], because a flat is a laid out garment. Moreover, there are different fit designs in fashion, some garments are loose-fit, some are just-fit, and some are tight-fit. For tight-fit garments,

412


Fig. 3. (a) A loose-fit four-layer dress flat; (b) the try-on result; (c) a tight-fit knitted dress with gathers flat; and (d) the try-on result.

which are usually made of elastic knitted fabrics, the flats would be developed directly based on garment measurements, and the actual sizes of these garments are often smaller than the body figure. Therefore, garment flats, especially tight-fit ones, may not “fit” the fashion figure model by simply overlaying and must be stretched in order to be dressed. We follow a two-step procedure to dress a garment. First, by matching a key anchor point, e.g., HPS, we trial fit the garment on top of the fashion figure model in a standard rest pose, on which some key landmarks are already defined. If the garment covers the body properly, then the dressing process is finished. Otherwise, a second step of garment mapping is carried out. To do this, we choose some key points in the garment mesh and match them with corresponding landmarks on the body figure. These correspondences are served as constraints for garmentbody mapping by RBF interpolation (1) where is the scalar basis function, is the weight of the RBF, and is a quadratic trivariate polynomial. To solve the RBFs that interpolate the above correspondences, we set (2) , and the centers for a set of constraint points , of the RBFs are chosen to coincide with the constraint points, i.e., . We use the basis function with , which yields a symmetric linear system. By solving the linear equations, the weights and the coefficients of the quadratic polynomial are obtained. Fig. 3 shows a loose-fit four-layer dress (a) and a tight-fit knitted dress with gathers (c), on which the selected matching points are shown in red. For the loose-fit four-layer dress, only two pairs of constraints at HPS and armpit are needed for a satisfactory dressing result [Fig. 3(b)], whereas for the tight-fit knitted dress with gathers, nine pairs of constraints are needed for dressing the garment, as shown in Fig. 3(d). C. Shape Deformation of the Garment Mesh 1) Skeleton Definition: With reference to the theories of human anatomy and the studies of skeleton-driven animation in

the computer graphics community, we use a skeleton of 17 line segments with 18 connecting joints to approximate a human figure skeleton, as shown in Fig. 4(a). By defining different layouts of this skeleton structure, most human body poses can be captured and represented. We define the skeleton layouts for each of the posed croquis figure templates [Fig. 1(g)–(l)] and these skeletons are used to define handle points for garment deformation. In computer graphics, a line segment structure, as shown in Fig. 4(a), is often analogue to the human skeletal structure for animation and deformation applications. In this paper, we define extra body features in addition to the 18-joint skeleton structure for our specific application. We define two horizontal levels of body feature, similar to the concepts for the line of action and action angle in fashion illustration literature [6], as shown in Fig. 4(b). These horizontal lines, called “feature lines”, can better illustrate the volume of the body torso in different poses. Along each of these feature lines, we define a pair of body feature points. As shown in Fig. 4(c), the resulting skeleton structure consists of 17 line segments plus two pairs of feature points. The positions of these body feature points in a new pose can be estimated based on their positions in the figure model in rest pose. For example, the position of the body feature point A, in the rest posed figure [Fig. 4(b)] is represented as the distances of and , and point A is bounded to the line segment connecting joints and . The distance is the projected distance from point A to the bounded skeleton, and the distance is from joint to the projection of point A on the bounded skeleton. The target position , in a new pose can be estimated by calculating the distance using formula , which is proportional to the length of in the rest pose, in a factor of the segment length connecting joints and in the new pose, to that of the corresponding segment length in the rest pose. The distance can be calculated similarly by , as shown in Fig. 4(d). 2) Handle Points for Garment Deformation: Based on the defined skeleton and body feature points (Section III-C–1), when a garment mesh is dressed on the croquis figure in rest pose (Section III-B), a set of handle points is then selected automatically. The handle points are critical to the quality of deformation. We suggest two levels of handle selection for different types of garment. In the first level of handle point selection, we traverse the garment mesh using the defined skeleton and body feature points. For skeletal mesh traversal, the number of handle points to be selected , for each line segment is defined. Among the garment mesh vertices, the vertices with the shortest distances to the skeleton segment concerned are selected as handles. As shown in Fig. 4(e), two red points are selected as handle points from the garment mesh vertices. The mesh structure is preserved in this procedure. Subsequently, handles corresponding to the two pairs of body feature points are selected. It is easy to determine in which triangle of the garment mesh a specific feature point falls. If the feature point is close to one of three vertices of the triangle, that vertex is selected as a handle. If the feature point is close to the center of the triangle, the triangle is re-meshed; the feature point is added to the vertices collection of the garment


413

Fig. 4. (a) Skeleton configuration in the rest pose. (b) Skeleton plus additional body feature points. (c) Body feature points (red and blue ones) in rest posed skeleton. (d) Body feature point position in a posed skeleton. (e) Traverse with vertices through skeleton. (f) Traverse with triangles for reference points. (g) Curve segments for traversing second level of handles.

Fig. 5. Layout of the handle points.

mesh. It divides the triangle into three triangles and the center point is selected as the handle, see Fig. 4(f). The mesh structure is changed in this remeshing procedure. The first-level handle point treatment is used to define handle points for loose-fit garments or some just-fit garments, in which the deformation need not follow too closely the body silhouette, so that fewer handle points are needed. For tight-fit and and some just-fit garments, the handle points defined in the first level are not sufficient to ensure the deformation closely follows the body contour. Therefore, to describe better the volume of the body, extra curve segments are used to traverse the garment mesh to obtain more handle points in the second level. For a uniform distribution of handle points, curve segments sitting in the mid position between the body contour and the skeleton, as shown in Fig. 4(g), are used to define extra handles. These curve segments together with the original skeleton and the body features can be viewed as another definition of skeleton structure. The curve segments traverse the garment mesh vertices with the method described earlier in Fig. 4(e). With the handle points defined in both the first level and the second levels, RBF mapping is needed to dress the garment onto the body figure for tight-fit or some just-fit garments. Fig. 5 shows the layout of the handle points of (a) loose-fit four-layer dress, (b) a pair of just-fit jeans, (c) a just-fit jacket, and (d) a tight-fit knitted dress with gathers. The green points are handles defined in the first level, and the purple points are extra handles defined in the second level.

Each of the selected handle points is associated with a line segment of the skeleton. Given the positions of these handle points in the rest pose, the target positions of the handle points in a new pose can be calculated based on the skeleton configuration of the new pose. The calculation is similar to the calculation for key body feature point , as described in Section III-C1. 3) Garment Mesh Deformation Algorithm: In this paper, we employ the main idea of 2D shape deformation [16] to generate fashion sketches from flat drawings. The inputs to the deformation algorithm include the garment mesh (from Section III-B1) and the -coordinates of the constrained mesh vertices (positions of the handle points, from Section III-C2), and the output is the resulting mesh in which the deformation distortion of all triangles in the garment mesh and the total length of the garment silhouette are minimized. The deformation involves two steps. In the first step, we calculate an intermediate result by rotation and uniform scaling, to preserve the local coordinates [16], [17], [26] of all the triangles in the mesh. In the second step, we adjust the triangles of the intermediate result by scaling to avoid shrinkage and inflation. In this paper, we improve the deformation algorithm [16] for our specific application by considering the garment properties in the deformation. In reality, the size of the garment would not change in different poses. Thus, in addition to minimizing the overall triangle distortion in the mesh deformation, we add a constraint to preserve the garment silhouette property (the length and direction of the mesh boundary) in the second step of the algorithm. The garment deformation algorithm is described in detail below. In step one, rotation and uniform scaling is performed to generate an intermediate result based on a set of constrained vertices. Suppose that and are the three vertices of triangle before and after the first step of deformation, respectively. We first compute the relative coordinates of in the local coordinate system defined by and with the following formula: (3) where denotes counterclockwise rotation by 90 , as shown in Fig. 6(a).

414


Fig. 6. Error metrics for the calculation of the target triangle. (a) Original triangle. (b) Target triangle.

We try to preserve the local coordinates for the mesh vertices in the deformation and thus, the desired location for can be represented as (4) The error associated with

is then represented as (5)

The desired locations and can be obtained similarly and the error associated with triangle is expressed as (6) Considering the entire mesh, we try to minimize the sum of errors for all triangles in the mesh. As the error metric is quadratic in all mesh vertices , we can express it in matrix form (7) We solve the error minimization problem by setting the parwith respect to the free tial derivatives of the function vertices in to zero [16]. By reordering to put the free vertices first, we can write , where represents the constrained vertices (i.e., handle constraints). This gives us (8) (9) We rewrite this as (10) and are fixed, and only represents all the handle where constraints, which changes during manipulation. Therefore, we can obtain by simple matrix multiplication by precomputing . Because the error function in step one is vertex based, which does not capture changes in scale, we adjust the scale of the triangles to prevent shrinkage and inflation in step two. The second step of the deformation includes two sequential processes: fitting and scaling. We first fit each triangle in the original mesh

Fig. 7. Scale adjustment of the triangle. (a) Fitting of the triangle. (b) Scaling of the triangle.

to a corresponding triangle in the intermediate result . Let be a new triangle that is congruent to , see Fig. 7(a). The fitted triangle can be obtained by minimizing the following function: (11) Again, using the local coordinates expression , can be minimized by setting the partial derivative of over fitted vertices coordinates of and to zero. The obtained triangle is then scaled by a factor of to make it congruent to its original shape. Each vertex in the original mesh appears in several triangles and similarly, each vertex corresponds to multiple fitted triangles in the above fitting process. The final position of the vertex is reconciled by averaging the positions desired by the triangles in which it lies. Therefore, we calculate the final -coordinates by minimizing the difference between the resulting triangle in the mesh and the fitted triangle, as shown in Fig. 7(b). Again, the minimization problem is solved by using the chosen handle points in Section III-C2 as constraints and they are prevented from moving in the deformation. At the same time, some garment silhouette properties such as length and direction should be preserved during the deformation. We add the constraint to the quadratic error function as follows:

(12) where is the weight of boundary edge constraint, and is the set of all the boundary edges of the garment mesh. In (12), the


415

Fig. 8. The deformation of a tight-fit knitted dress with gathers. (a) A garment flat. (b) Delaunay triangulation. (c) RBF mapping. (d) The garment in a new pose. (e) Deformed result with detail. (f) Deformed result with texture.

error is associated with edges of the triangle, not the vertices of the triangle. We only present the construction of the error functions , , and in (6), (11), and (12) in this paper. Equations (11) and (12) can be solved in a similar way as (6); readers can refer to [27] for details. We use the general library SuperLU for the calculation of matrix factorization. The above deformation is based on the garment silhouette outline. For realistic deformation results, design detail like style lines and seams on the garment would deform accordingly as well. The design detail are represented as spline curves and lines in a garment flat drawing. During the deformation, we preserve the relative position of the design details in the garment by keeping the control points of all the style lines and curves. It is easy to observe that each control point falls into one of the triangles in the garment mesh. For the affine property of barycentre coordinate, it can effectively depict simple proportional deformation. In this paper, we use it to describe the relative position of the control point to the corresponding triangle. IV. EXPERIMENTAL RESULTS AND DISCUSSION A. Experimental Results and Comparison To evaluate the proposed method, we experimented on deforming various garment flats to create posed sketches. All the garment flats were developed by Adobe Illustrator. Fig. 8 shows all the steps of our deformation system. Fig. 8(a) is the original flat drawing of a tight-fit knitted dress with gathers. Fig. 8(b) is the mesh structure of the garment generated by Delaunay triangulation. Fig. 8(c) shows the try-on effect of the garment on a figure in rest pose by RBF mapping. Fig. 8(d) is the deformation result of the garment on a figure with a new pose. Fig. 8(e) shows

Fig. 9. Garments deformation in different poses. (a) Loose-fit four-layer dress: flat and in four different poses. (b) Tight-fit knitted dress with gathers: in rest pose and four different poses. (c) Just-fit checked jacket and jeans: flat and in four different poses.

the deformation of the design detail (gathers) and Fig. 8(f) is the deformed garment with texture. It shows that our approach can accurately describe the deformation of both the garment silhouette and the detail. Fig. 9 shows more results generated with our system. Fig. 9(a) and (b) shows the loose-fit four-layer dress and the tight-fit knitted dress with gathers in different poses. Fig. 9(c) gives the deformation of a pair of jeans and a checked jacket in different poses. These experimental results show that our system effectively generates various posed fashion illustrations from different garment flats. Table I summarizes the performance of our system. We obtained the data by running the routines on a 3.00 GHz Intel Core 2 Quad Q9550 CPU with 4 GB main memory. As the calculation is highly dependent on the number of vertices and the number of handle points, Table I shows the approximate speed of our system. The main calculation focuses on the LU factorization [27] during the garment deformation. The system shows

416


TABLE I SAMPLE RUNNING TIMES (MILLISECONDS) FOR GARMENT DEFORMATION

Fig. 10. The results generated using direct RBF mapping for comparison with Fig. 9(b).

complete real-time performance and that the users experience no delay. Given a set of poses with the skeletons defined (Fig. 1), all the users need to do is to define several matching points for mapping the garment flat onto the body figure in rest pose. Our system can show the garment in different poses immediately. The calculation is automatic. A video is provided to supplement this paper. The system is easy to use, even for novices. To illustrate the validity of our system, we have compared the results generated by our proposed method with those from direct RBF mapping. For realistic fashion illustrations, the garment should follow the body contours in different poses, especially in tight-fit clothing. Fig. 10 shows the results obtained by direct RBF mapping, which were obtained by defining the relative positions of the matching points [Fig. 3(c)] in every new pose. As shown, RBF mapping can, to some extent, capture the garment (outline) shape in different poses. By comparing the results in Figs. 9(b) and 10, our method outperforms RBF mapping in two areas and generates more realistic deformations. First, the whole garment including both outline and detail should deform coherently in different poses. For example, the tight-fit dress in Fig. 9(b) has detail of gathers along the center front. The gathers should always be in the center front area in different poses, even though the body is slightly twisted [e.g., the second pose of Fig. 9(b)]. However, in the RBF mapping results of Fig. 10, garment detail are distorted and displaced, examples are the gathers in the second pose and the shoulder yoke in the fourth pose. Our method adopts local coordinates based deformation, which can better preserve the relative deformation of nearby triangle mesh; global interpolation based RBF mapping deforms the mesh by preserving the outline shape. We demonstrate this advantage of our method by calculating the average distortion ( ).

Fig. 11. Divided zones of tight-fit knitted dress mesh.

We first divide the garment mesh vertices into zones, and total number of zones is the number of skeleton feature points [Fig. 4(c)] being covered by the garment. The zoning is done between each mesh by calculating the distance being covered by vertex and each skeleton feature point belongs to a zone the garment in the rest pose. The vertex is the shortest among distances to all feature if the distance . points , Fig. 11 shows the divided zones of the tight-fit knitted dress mesh. After zoning, the vertex-to-feature distance is recorded . Similarly, the vertex-tofor each vertex as can be feature distance of deformed mesh in any new pose calculated. The average distortion (AD) is calculated by

(13)

is the number of vertices in zone , . where The average distortion indicates the association of the garment detail with the body surface; smaller the value higher the association. Table II compares the average distortion of the results in Figs. 9(b) and 10. It is shown that our method has smaller distortion than RBF mapping in all four poses. Second, in order to generate results by RBF mapping, the relative positions of all matching points in every pose must be defined. The process is tedious because the number of matching points needed vary between garments. The quality


417

TABLE II COMPARISON OF AVERAGE DISTORTION (AD) IN RESULTS IN FIGS. 9(B) AND 10

of the mapping results depends on the number of matching points defined. The garment may penetrate or the garment outline may have some sharp angles if not sufficient matching points are defined, see Fig. 10. Increasing the number of matching points create better results with smoother garment outlines. However, increasing the number of matching points would make the process more labour intensive. Furthermore, direct RBF mapping may not be suitable for loose-fit clothing deformation, because loose-fit garments drape over the body and the garment outline may be away from the body contours. Users must imagine how the garments behave in different body postures and define matching point positions subjectively. Comparatively, the deformation of our method is automatic. In our method, the only interaction is the selection of a small number of matching points for dressing the garment onto the figure in rest pose. The matching points would not affect the deformation results. In our method, the deformed results cover the body well without any penetration in diverse poses and have smooth garment outlines.

Fig. 12. The jeans deformation with different handle treatments. (a) Deformation result with two levels of handle points. (b) Deformation result with one level of handle points.

B. Discussion on Handle Point Selection We have concluded two levels of treatment for handle selection in Section III-C2, after some tests. For loose-fit garments, the garment deformation mainly shows the rhythm of the body movement in different poses, fewer handle points selected along the body skeleton and the four body features are sufficient to generate realistic and natural garment deformation. For tight-fit garments or garments with just-fit design, the garment must follow more closely the movement of the body and, therefore, more handles should be chosen in the second level in order to capture the volume of the body in different poses. Fig. 12 compares the jeans deformations with two levels of handle treatment. In Fig. 12(a), more handle points are selected in the inner and outer legs, so the result is natural. The jeans deform in an undesirable way, as shown with the red rectangle in Fig. 12(b), if only a small number of handle points are selected along the skeleton. C. Processing of Overlap Additional processing is necessary when different parts of the garment overlap. For the jeans, when two legs overlap, we assign different depths for the two legs. As for the jeans in Fig. 9(c), the third figure shows the left leg has a larger depth value than the right leg and in the four figure, the right leg has a larger depth value than the left leg. The results show that the two legs of the jeans overlap naturally. D. The Effect of Outline Length Constraint We have made comparison between the results with and without preserving the total length of the garment outline.

Fig. 13. Comparison of garment deformation with and without outline length constraint.

We adjusted the weight of the outline length constraint in (12), and the results are shown in Fig. 13. Fig. 13(b) is the result without preserving the total length of garment outline. Fig. 13(c)–(h) shows the results for the same pose with weights 1.0, 5.0, 10.0, 50.0, 100.0, and 1000.0, respectively. With the outline length constraint, the deformation looks more natural, and different weights can be used to simulate the different stiffness of fabrics for different drape effects. From our experiments, we conclude that the results shown in (d) with weight of 5.0 and (e) with weight of 10.0 are the most natural results. The results with weights of 50.0 and 100.0 show that the fabrics are heavier than the others and the weights of more than 1000.0 would generate deformation that is too rigid. Table III compares

418


TABLE III DIFFERENCES OF THE TOTAL LENGTH OF THE OUTLINE (MM)

the differences of outline length before and after deformation. The original length of the dress sketch outline is 362.9 mm. E. Presentation of Different Colorways and Prints by Texture Mapping In fashion design, once a flat is developed it is also important to add different textures, prints and colorways to the design. To generate fashion illustrations with attractive colors and dynamic figures, we present different textures and colors by texture mapping. We first define the texture coordinate by mapping a texture image to the garment mesh in the rest pose and such a texture image can be easily obtained using Adobe Illustrator or other design systems [3]–[5]. Next, we deform the garment flat to create different posed sketches in real time by our method. Then, we map different prints, textures and colorways onto the deformed garments by texture mapping [28] for realistic and attractive fashion illustrations. As shown in Fig. 14, our method imitates the garment deformation effectively and realistically. F. Limitation and Extension of Rotation Handling We generate fashion sketches by 2D deformation. One drawback of the current method is that it is not suitable for very dynamic poses where the body turns substantially, because the back information is missing. When a pose involves a large degree of rotation, it may not capture the deformation well, especially for garments where the design detail is close to the sides of the garments. It is interesting to note that the flat is a laid out garment, usually with both front view and back view. For poses with a large degree of rotation and garments with detail at the sides, a special rotation handling is proposed. Instead of mapping the original flat drawing to the figure template in rest pose, we suggest to do a 3D warping to dress the garment on the body figure, to approximate a 3D try-on effect. We warp both front view and back view of the flat drawing along its boundary around an elliptic cylinder to reconstruct a 3D garment, as shown in Fig. 15(a). The elliptic cylinder approximates the 3D body shape. As illustrated in the cross section in Fig. 16, point represents some design detail information and its depth information, namely, , that can be obtained by simple calculation. By warping an ellipse between each pair of points along the garment boundary, we approximately reconstruct a 3D garment corresponding to the original 2D flat. We then rotate the reconstructed 3D garment with a suitable angle according to the new pose. After rotation, we obtain a garment projection as its rest pose image for deformation. Fig. 15(a) shows the flat drawing of a sailor dress with patch pockets at two sides, (b) is the front view after

Fig. 14. Presentation of different textures and colorways based on deformed flat drawing.

Fig. 15. Sailor dress (a) original flat drawing, (b) “3D garment” by warping around an approximated 3D shape, and (c) projected view after rotation.

warping, and (c) is a projection after a certain degree of rotation. It can be seen that the right pocket is distorted after rotation. Fig. 17 compares the deformation results (a) and (d) with and (b) and (e) without rotation handling, and those (c) and (f) obtained by nonuniform distortion. For nonuniform distortion,


419

sults without rotation handling (b) and (e) did not handle the left and right asymmetric distortion well. V. CONCLUSION

Fig. 16. Approximating the torso cross section as ellipse.

Given the fashion flats in the rest pose, our method can mimic them in different new poses immediately. The method can be applied to various garments, including loose-fit, just-fit and tight-fit clothing, and it is capable of handling overlaps. The method generates realistic fashion sketches, namely posed flats with full color and detail using 2D shape deformation. It aims to minimize the deformation distortion of the mesh model of the garment and reflect the garment properties in the deformation process by preserving the total length of the garment outline. The method has been proven an important tool for designers to present their design ideas and accelerate the design process. There are still limitations in our implementation. First, the garment deformation is geometrical-based without consideration of different textile material properties in the deformation process. In addition, the method first generates a mesh based on the silhouette outline of the flat drawing and it is difficult to triangulate meshes with near-equilateral triangles for garment drawings with tiny sharp angles in the silhouette outline. In the future, we would further explore the influence of gravity factor on clothes deformation. On the other hand, we would also like to reconstruct 3D garments based on both 2D flat drawings and a 3D human model, to better imitate the spatial deformation of garments. At the same time, we propose to imitate garment deformation from one pose to another with computer animation techniques, producing realistic animation of garment try-on and deformation. ACKNOWLEDGMENT The authors would like thank all anonymous reviewers for thoughtful comments and constructive suggestions. They are grateful to M. Cheung for preparing some croquis and texture images used in this paper. REFERENCES

Fig. 17. Comparison of results with rotation handling (a) and (d), without rotation handling (b) and (e), and nonuniform distortion (c) and (f).

we shrink the garment, along the center of the garment, in -direction, while keep their length in -direction unchanged. The scale factor is bigger when the point is further away from the center. Therefore, the garment details near the edges are compressed as nonuniform distortion. It shows that the suggested rotation handling can give more realistic details after deformation, referring in particular to the collar and the pocket shape of the deformed garments, see Fig. 17(a) and (d). In nonuniform distortion results (c) and (f), although trapezoid pockets can be obtained, the collars are not deformed correctly. More specifically, the suggested rotation handling gives more realistic pocket shape than nonuniform distortion does, when the body is twisted (d versus f); the collar is displaced from the center front of the body when the body is rotated/turned in nonuniform distortion results (a versus c and d versus. f). The deformation re-

[1] K. McKelvery and J. Munslow, Fashion Design: Process, Innovation & Practice. Oxford, U.K.: Blackwell, 2003. [2] J. Lee and C. Steen, Technical Sourcesbook for Designers. New York, NY, USA: Fairchild, 2010. [3] Snapfashun, 2011. [Online]. Available: http://www.snapfashun.com/ [4] Verve Sketch Software, 2012. [Online]. Available: https://twitter.com/ VerveSketch [5] P. Y. Mok, X. X. Wang, J. Xu, K. L. Kwok, J. T. Fan, and J. Xin, “Customized skirt design by interactive evolutionary computing techniques,” Hong Kong Polytechnic Univ., Kowloon, Hong Kong, 2012, Working Paper. [6] B. Abling, Fashion Sketchbook, 5th ed. New York, NY, NY: Fairchild, 2007. [7] B. K. Hinds and J. McCartney, “Interactive garment design,” Visual Comput., vol. 22, no. 6, pp. 53–61, 1990. [8] H. M. Werner, N. M. Thalmann, and D. Thalmann, “User interface for fashion design,” in Proc. IFIP TC5/WG5.2/WG5.10 CSI Int. Conf. Comput. Graph., Amsterdam, The Netherlands, 1993, pp. 197–204. [9] T. Bonte, A. Galimberti, and C. Rizzi, A 3D Graphic Environment for Garments Design. Norwell, MA, USA: Kluwer, 2002, pp. 137–150. [10] T. Igarashi and J. F. Hughes, “Clothing manipulation,” in Proc. 15th Annu. ACM Symp. User Interface Softw. Technol., New York, NY, USA, 2002. [11] D. Bourguignon, M.-P. Cani, and G. Drettakis, “Drawing for illustration and annotation in 3D,” Comput. Graph. Forum, vol. 20, no. 3, pp. C114–C122, 2001.

420


[12] E. Turquin, M.-P. Cani, and J. F. Hughes, “Sketching garments for virtual characters,” in Eurograph. Workshop on Sketch-Based Interfaces and Modeling, Grenoble, France, 2004, pp. 175–182. [13] E. Turquin, J. Wither, L. Boissieux, M.-P. Cani, and J. F. Hughes, “A sketch-based interface for clothing virtual characters,” IEEE Comput. Graph. Applicat., vol. 27, no. 1, pp. 72–81, 2007. [14] P. Decaudin, D. Julius, J. Wither, L. Boissieux, A. Sheffer, and M.-P. Cani, “Virtual garments: A fully geometric approach for clothing design,” Comput. Graph. Forum, vol. 25, no. 3, pp. 625–634, 2006. [15] C. X. Ma, Y. J. Liu, H. Y. Yang, D. X. Teng, H. A. Wang, and G. Z. Dai, “KnitSketch: A sketch pad for conceptual design of 2D garment patterns,” IEEE Trans. Autom. Sci. Eng., vol. 8, no. 2, pp. 431–437, Apr. 2011. [16] T. Igarashi, T. Moscovich, and J. F. Hughes, “As-rigid-as-possible shape manipulation,” ACM Trans. Graph., vol. 24, no. 3, pp. 1134–1141, 2005. [17] T. Igarashi and Y. Igarashi, “Implementing as-rigid-as-possible shape manipulation and surface flattening,” J. Graph., GPU, Game Tools, vol. 14, no. 1, pp. 17–30, 2009. [18] Y. Weng, W. Xu, Y. Wu, K. Zhou, and B. Guo, “2D shape deformation using nonlinear least squares optimization,” Visual Comput., vol. 22, no. 9, pp. 653–660, 2006. [19] W. Yang, J. Feng, and X. Jin, “Shape deformation with tunable stiffness,” Visual Comput., vol. 24, no. 7–9, pp. 495–503, 2008. [20] W. Yang and J. Feng, “2D shape manipulation via topology-aware rigid grid,” Comput. Animat. Virtual Worlds, vol. 20, no. 2-3, pp. 175–184, 2009. [21] W. Yang and J. Feng, “2D shape morphing via automatic feature matching and hierarchical interpolation,” Comput. Graph., vol. 33, no. 3, pp. 414–423, 2009. [22] S. Capell, S. Green, B. Curless, T. Duchamp, and Z. Popovic, “Interactive skeleton-driven dynamic deformations,” ACM Trans. Graph., vol. 21, no. 3, pp. 586–593, 2002. [23] J. P. Lewis, M. Cordner, and N. Fong, “Pose space deformation: A unified approach to shape interpolation and skeleton-driven deformation,” in Proc. ACM SIGGRAPH, New York, NY, USA, 2000, pp. 165–172. [24] I. Baran and J. Popovic, “Automatic rigging and animation of 3D characters,” ACM Trans. Graph., vol. 26, no. 3, pp. 72–72, 2007. [25] L. Markosian, J. M. Cohen, T. Crulli, and J. Hughes, “Skin: A constructive approach to modeling free-form shapes,” in Proc. ACM SIGGRAPH, 1999, pp. 393–400. [26] O. Sorkine, D. Cohen-Or, Y. Lipman, M. Alexa, C. Rossl, and H.-P. Seidel, “Laplacian surface editing,” in Proc. ACM SIGGRAPH Symp. Geometry Process., 2004, pp. 175–184. [27] K. Madsen, H. B. Nielsen, and O. Tingleff, “Methods for non-linear least squares problems,” in Informatics and Mathematical Modeling, 2nd ed. Lyngby, Denmark: Tech. Univ. Denmark, 2004. [28] J. Wang, G. Lu, L. Chen, Y. Geng, and W. Deng, “Customer participating 3D garment design for mass personalization,” Textile Res. J., vol. 81, no. 2, pp. 187–204, 2011. [29] O. Sorkine and M. Alexa, “As-rigid-as-possible surface modeling,” in Proc. ACM Int. Conf. Ser., 2007, vol. 257, pp. 109–116. [30] H. B. Yan, S. M. Hu, R. R. Martin, and Y. L. Yang, “Shape deformation using a skeleton to drive simplex transformations,” IEEE Trans. Vis. Comput. Graph., vol. 14, no. 3, pp. 693–706, 2008. [31] H. Yu and J. J. Zhang, “Topology preserved shape deformation,” Visual Comput., vol. 28, no. 6–8, pp. 849–858, 2012.

[32] C. Xian, H. Lin, and S. Gao, “Automatic cage generation by improved OBBs for mesh deformation,” Visual Comput., vol. 28, no. 1, pp. 21–33, 2012. [33] Y. J. Liu, D. L. Zhang, and M. M. F. Yuen, “A survey on CAD methods in 3D garment design,” Comput. Ind., vol. 61, no. 6, pp. 576–593, 2010. [34] J. Solomon, M. Ben-Chen, A. Butscher, and L. Guibas, “As-killing-aspossible vector fields for planar deformation,” Comput. Graph. Forum, vol. 30, no. 5, pp. 1543–1552, 2011. [35] D. Sykora, M. Ben-Chen, M. Cadik, B. Whited, and M. Simmons, “TexToons: Practical texture mapping for hand-drawn cartoon animations,” in Proc. ACM SIGGRAPH/Eurograph. Symp. Non-Photorealistic Animat. Rendering, 2011, pp. 75–84. [36] Y. J. Liu, X. Luo, A. Joneja, C. X. Ma, X. L. Fu, and D. W. Song, “User-adaptive sketch-based 3-D CAD model retrieval,” IEEE Trans. Autom. Sci. Eng., vol. 10, no. 3, pp. 783–795, 2013.

Xianmei Wan received the Ph.D. degree in computer science from Zhejiang University, Zhejiang, China, in 2012. She is currently a Lecturer with the Institute of Dongfang, Zhejiang University of Finance and Economics, China. Her research interests include computer-aided design, facial animation, and computer animation. She conducted the work reported in this paper when she was a Visiting Researcher at the Hong Kong Polytechnic University.

P. Y. Mok (M’12) received the B.Eng. degree (first class honors) and the Ph.D. degree from the University of Hong Kong, Kowloon, Hong Kong, in 1998 and 2002, respectively. She is an Associate Professor with the Institute of Textiles and Clothing, Hong Kong Polytechnic University. Her current research interests are in garment pattern engineering, fashion CAD and cloth simulation, and artificial intelligence.

Xiaogang Jin received the B.S. degree in computer science and the M.S. and Ph.D. degrees in applied mathematics from Zhejiang University, Zhejiang, China, in 1989, 1992, and 1995, respectively. He is a Professor with the State Key Lab of CAD&CG, Zhejiang University, China. His current research interests include multimedia authoring, crowd animation, cloth animation, facial animation, video abstraction, implicit surface computing, special effects simulation, mesh fusion and texture synthesis.