(This is a sample cover image for this issue. The actual cover is not yet available at this time.)
This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues. Other uses, including reproduction and distribution, or selling or licensing copies, or posting to personal, institutional or third party websites are prohibited. In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier’s archiving and manuscript policies are encouraged to visit: http://www.elsevier.com/copyright
Author's personal copy
Computer-Aided Design 45 (2013) 591–604
Contents lists available at SciVerse ScienceDirect
Computer-Aided Design journal homepage: www.elsevier.com/locate/cad
Integrating shape grammars into a generative system for Zhuang ethnic embroidery design exploration Cui Jia ∗ , Tang Ming-Xi School of Design, The Hong Kong Polytechnic University, 852, Hong Kong, China
article
info
Article history: Received 10 September 2011 Accepted 5 August 2012 Keywords: Shape grammar Generative system Aesthetic evaluation Design exploration B-spline curves
abstract The integration of shape grammars within a generative system is thought to have good potential to support design applications in the art and cultural domains. Based on a shape decomposing method, two-level generative shape grammars are developed within the application of Zhuang ethnic embroidery design. In our approach, free B-spline curves are used to derive initial design concepts together with a new interpolation algorithm. In order to cope with the potentially huge solution space, a novel aesthetic evaluation model and a design exploration algorithm are developed for designers to find their favorite design solutions based on their subjective assessments. A system is introduced to demonstrate this research with an example of Zhuang ethnic embroidery design. © 2012 Elsevier Ltd. All rights reserved.
1. Introduction Through social communication, design (in particular, visual design) becomes a medium to represent pictorial descriptions of concrete or abstract concepts. The physical and psychological issues focusing on the image perception or mental conception are full of fuzziness and uncertainty, which may best be resolved by the designer’s intuitive feeling, experience and inspiration. Therefore, design is traditionally recognized as a manual process. Designers generate concepts based on functional and aesthetic requirements, then represent them by CAD (Computer Aided Design) technology. This will lead to substantial turnaround times because of the labor-intensive nature of the undertaking [1]. Most designers prefer choosing a pencil-and-paper way at the early design stage to depicting ideas by CAD tools, since these tools, which are not designed to manipulate the changeable, may become a burden rather than a relief when designers try to alter the design alternatives [2]. In recent decades, with the development of Computer Graphics, Automation Control and Artificial Intelligence, CAD technology can provide advice to designers on the progress towards meeting design requirements [3]. Making use of computational abilities and supporting intelligent design behavior have long been the pursuit of the automation design research community. Shape grammars, with their grammatical rule base and recursively operational mechanism, can interpret designs by formal descriptions, and
∗
Corresponding author. E-mail addresses:
[email protected] (J. Cui),
[email protected] (M.-X. Tang). 0010-4485/$ – see front matter © 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.cad.2012.08.002
motivate the development of CAD tools more generic, useful and sophisticated [4]. However, because of the difficulties in shape manipulation, especially with the curvy ones, plenty of applications and researchers used orthogonal models, which are to some extent, different from the practical design requirements. Furthermore, complex design processes and subjective design evaluation make knowledge representation by the rules of shape grammars in a traditionally narrative way more difficult. It is a challenging task for users or designers to find out their favorite design candidates from a potentially large design space, because the design works mainly rely on designers’ sensitively subjective assessment. How to find an efficient way to enhance the communication between the end users and the computers, with the aim of prompting creative ideas, especially in the art domain, is an unavoidable issue for computer-aided design research. This paper presents research on improving some of the above discussed issues through shape grammars. First, instead of using narrative shape rules, descriptive rules are used to represent the design process. The descriptive rules are formulated in a way similar to the syntactic structure of natural language with words corresponding to shapes. Second, in order to support the practical design process, we divide an application into two levels, i.e., a Coarse Level and a Refined Level. Each of them has internal key points that can be dealt with separately. Third, B-spline curves are used at the Refined Level to generate smooth and natural shapes following the Zhuang ethnic embroidery design examples. Fourth, employing a subjective evaluation method, a four-dimensional aesthetic evaluation model is introduced in this system to explore the design space in a heuristic way. A Zhuang Ethnic Embroidery Design System (EDS) is implemented based on these features, with an automatic generating ability and heuristic design exploration functions.
Author's personal copy
592
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
This paper is organized in five sections. The research issues, including reviews of shape grammars, the language of design and evolutionary applications, are presented in Section 2. The shape grammar for Zhuang ethnic embroidery design is introduced in Section 3. The design exploration with a novel four-dimensional aesthetic evaluation model is presented in Section 4. In the last section, the conclusions and future research directions are presented. 2. Research issues 2.1. Shape grammars Shape grammars, inspired by phrase structure grammar [5], were introduced by Stiny and Gips [6] in 1972, and the formal definition was presented by Stiny [7] in 1980. A shape grammar contains a vocabulary set S (Shape), a set of rules R (Rule), a set of symbols L (Label), and an initial shape I, to make up a formal language. Therefore, it defines a 4-tuple set {S, L, R, I}. By applying the shape rules recursively to the initial shape, a formal composition of design can be generated. Then, Stiny proposed the Kindergarten Method as a theory of shape grammars’ application with a preliminary attempt on shape decomposition. Knight studied the role of labels [8] and transformations [9] in shape grammars. The shape arithmetic and shape recognition were discussed in the literature [10–12]. Early applications of shape grammars were mainly in the field of architecture, such as the Palladian Grammar [13], the Prairie House Grammar [14], the Queen Anne House Grammar [15], the Row-House Grammar [16], and the Turkish House Grammar [17]. In that period, most of the work was performed in manual ways to create their own vocabularies and rules. Gero presented a method combining genetic engineering and a shape grammar approach to identify architectural styles [18]. An automation construction method was proposed by Howe [19] in 2000. The Discursive Grammar [20] in an interactive computer system for exploring customizing requirements was mentioned by Duarte in 2005. CGA (Computer Graphic Architecture) shape, a novel shape grammar for producing building shells with high quality visualization and geometric details, was presented in [21–24]. To provide computational support, more and more shape grammar systems have been developed in product design. A wellknown example is the Coffee-maker Grammar [25]. This research used a parametric shape grammar to auto-generate coffee maker designs by changing grammar rules for novel products. In 2004, the Buick Shape Grammar [26] (‘‘brand DNA’’) was proposed by McCormak to develop and maintain a consistent brand identity. Incorporating a genetic algorithm within shape grammars, the Coca-cola Grammar [27] was used to generate new brand product design shapes in accordance with designers’ preferences. There are many other studies on product design using shape grammars, such as motorcycle design [28], inner hood panel design [29], camera design [30], and Ming-style furniture design [31]. The original application of shape grammars was in art, painting and sculpture, involving the use of a primary structural component [6]. In this area, the research kept silent for a long time after the preliminary application was published. Then, Kirsch and Kirsch [32] used a generative method to build connections between shape grammars and fine arts. Tapia presented the Lattice Grammar [33] to trace stylistic changes, and defined a wide variety of design languages for some non-parametric applications in algebra. Zhang and Lin [34] used a theory of shape grammars to analyze Tibetan Tangka. Cui and Tang [35] were the first to apply shape grammars to Chinese ethnic culture and design. 2.2. Design language For its vague interior indications and discrete exterior expressions, design is difficult to be grasped by pure logic and mechanical
mechanisms which can easily run on computers. However, shape grammars, with their formal descriptions [36], intuitive visualization [37], and flexible and heuristic applications [38], can interpret a design in an explicit form. Furthermore, they can support designers for better understandings and perceptions in a creative and innovative way. 2.2.1. Formal approach to design Geometric design (Visual design) is central to many areas of engineering, computer graphics and architecture [39]. There are many grammatical methods concentrating on design formalism, in which to populate a spectrum where, at one end, representations correspond to geometries, and at the other end, representations are mapped to one or more realizations [40]. Shape grammars are intended to form a basis for visual computation. Stiny [36] illustrated that shape grammars can abstract concrete shapes into informative and constructive descriptions to facilitate computational implementation. This can extend shape grammars’ potential functions into the design area. In this paper, we use shape grammars for design representation, analysis and generation. Through the characters of shape grammar, we aim at making designs understandable and computable by computers, while simultaneously keeping their visual properties. 2.2.2. Design interpretation In design cases, the designers may not know what they feel or perceive until they start to see it. It is difficult to describe a design by verbal expressions, i.e., symbols. But it is relatively easy to represent it by visual expressions such as shapes. There is an assumption that the best way to evaluate a design is not by symbolic calculation, but by visual forms consisting of various shapes. This does not necessarily mean that informative symbols and shapes cannot be reconciled. The problem is how to reason a design not only relying on symbols, but also with visual computing. Using shape decomposition [37,41] to segment shapes into lowerlevel constituents, such as single patterns, lines, even points, a design can be manipulated computationally by seeing what the transformations are. With this viewpoint, shape grammars can be considered as a kind of visual mathematics. Given a language of design defined by a shape grammar, the design descriptions can be explicitly constructed using a recursive schema. As such, a shape grammar can specify a design through mapping the composition of spatial elements to relevant design issues, such as purposes, functions and intentions. Flasinski [42] presented a representation formalism defined on the basis of parsing a family of IE-graphs. The grammatical method can reason the design into a structural description. Wang and Duarte [43] discussed the feature of design languages and their computer applications on prototyping complex 3D forms. 2.3. Design evolution The shape vocabulary and the recursive rule usage constitute the knowledge base, which can create a great deal of new design alternatives. Evolutionary systems have been applied to design domains which involved design optimization and design learning. The genetic engineering approach [44] comes from an analogy with human intervention in biological evolution. Shape grammars can be used to search the best solutions in a design space, employing some computational methods such as Genetic Algorithm [30], Cellular Automata [45], Genetic Programming [46], and Markov Random Fields [47]. Although design has been categorized as a problem solving activity by many [48], there are still some uncertainties and queries
Author's personal copy
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
593
Fig. 1. Embroidery tradition. (a) Snapshot of local women. (b) Extract from design work.
Fig. 2. Categories of embroidery patterns. (a) Single stem. (b) Double stem. (c) Quadruple stem.
[49]. The prior issue is how to evaluate the evolutionary results, which are mostly dependent on visual judgments. In design, it is hard to confirm the destination before finishing the design process (obtaining satisfactory solutions). Sometimes, an evolutionary agent has to search the potential design solutions in a large space without clear assessing criteria. In this paper, we present a different view on the evolutionary process of design. We propose a design exploration algorithm with a four-dimensional aesthetic evaluation model, which will be introduced in Section 4.
3.1. Zhuang ethnic embroidery examples
3. Integration of shape grammars in a generative system
3.2. Two-level shape grammar
The difficulties in shape recognition, sub-shape matching and shape transformation control have become the barriers for shape grammar implementation in computer systems. Due to these difficulties and demanding design requirements, many researchers applied shape grammars to an orthogonal model in 2D or 3D form. However, McCormack and Cagan [50] introduced a sub-shape matching method for curve-based shapes in computers. Parts [41] described research on the style of design using different control points in curvilinear shapes. In this paper, we present a novel approach to apply shape grammar on design in order to overcome the problems commonly seen in 2D applications. A generative design system can create solutions with strong visual impact by controlling emergent shapes. In this section, a descriptive shape grammar is introduced. The Zhuang ethnic embroidery design system using shape grammars is described as an example to demonstrate how our approach works.
These embroideries have a long history and tradition in Zhuang and Yi autonomous prefectures in China’s Yunnan province. The embroideries from these regions have strong cultural characters in shape, color and pattern, pursuing a simple and decorative beauty. The main themes of their design include flowers, leaves, birds and other natural shapes. The embroidery products include shoes, slings, clothes, aprons and some accoutrements (Fig. 1).
3.2.1. Analysis of pattern features After analyzing the copra of the real embroidery design, we abstract the features as follows: 1. There are strong inherent spatial relationships; 2. The structures of the designs show balanced beauty; 3. The elements, named petals, are naturally smooth and irregular; 4. There are stylistic differences among different ethnic groups. These features indicate that it is feasible to use grammar rules to depict the embroidery design process. The balanced beauty permits the usage of spatial labels to create symmetrical effects. The petals, shaped in the form of random curves, can be manipulated at an single step. Therefore, it will not influence the rule implementations, especially the sub-shape detection and transformation. For
Author's personal copy
594
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
Table 1 Design languages of samples based on three basic styles. Category
Design languages
Single-stem
No
1
1
Double-stem
2
1
Quadruple-stem
2
different styles, we use the stem as a major character for identifying the embroidery categories. That is, different stem styles denote different embroidery styles. Fig. 2(a) shows the single-stem category, which has been demonstrated in Ref. [35]. Other categories include the double-stem category (b) and the multi-stem category (c). After applying shape grammars, more characters can be extracted from the three basic ones. 3.2.2. Design languages for basic embroidery style In a preliminary experiment, we develop several design languages pointing to the single-stem style, the double-stem style and the quadruple-stem style. In a narrative way, strictly following the original design corpora, some design languages set forth the design process how to generate them step by step, as shown in Table 1. Studying the languages, we find that although there are different visual features for different styles, i.e., the double-stem 2 (half-symmetric, balanced) and the quadruple-stem 1 (rotational feature, unbalanced and none symmetric), some common qualitative characters can still be obtained. For example, there exist symmetry characters in all design languages, including ‘none-symmetry’, ‘half-symmetry’, ‘quadruple-symmetry’ and ‘rotational-symmetry’. As a result, we extract these characters at a general level to prepare for introducing a descriptive shape grammar.
3.2.3. Shape grammars for Zhuang ethnic embroidery design A real design may involve several design processes, approaches, and interactions which are complex to formalize with a single formal approach. To segment a design process into low level constituents, concentrating on a single aspect, is helpful to deal with the problem by computer. Similar to the research in the literature [37,41], we divide the shape grammars into two levels, based on different decomposing degrees. The advantage of this is to make sure that each level has its own focus without disturbance from others. Generally, designers depict the sketches at an early stage to confirm the design framework. Therefore, in our shape grammar, the first level is the Coarse Level, which specifies general issues, such as categories, symmetries, balance relationships and petal orientations. The considerations at the Coarse Level only refer to a skeletal construction. At the second level, i.e., the Refined Level, embodiment issues, such as petal shapes and colors for every segment specified at the former level, are determined. 3.2.4. Coarse Level In many shape grammars, a grid of registration points facilitates formal transformations [51]. Sometimes this grid is visible, playing an active role in the design process [52]. However, we do not
Author's personal copy
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
595
Fig. 5. Terminal rule. (a) Rule for petal generation. (b) Rule for stem generation.
Fig. 3. Original rule.
of ‘a’ in the R+ direction. For overlap avoidance mentioned in 3.3.1, we need to add a new parameter, α , which represents the angle between a diagonal line and the x sideline. The value of α , which can be calculated through arctan (y/x), is a key rationale to classify the rule family into different groups. In this research, we divide the ten families of rules into two groups based on Eq. (1): y+b x+a
y
= . x
(1)
In Eq. (1), α does not change whatever the variation of ‘a’ or ‘b’ might be. In Rule Family 1, if b > a ∗ y/x, then α 2 > α 1. The same situation applies for the rules in Family 2 to Family 5. Consequently, we classify Families 1–5 as a category in which α will increase, and Families 6–10 as the ones in which α will decrease.
Fig. 4. Transitional rule. x, y, a, b ∈ R+ , x, y, a, b in different family, could make different values.
use the grid, but rectangles to locate the orientation of different parts in an embroidery design. At the conceptual stage, there are not many restrictions on the shape details. Therefore, we can use rectangles to represent the style of a pattern without worrying about the overlap avoidance, curvilinear complexity, and petals redundancy. In a computer implementation, this can reduce the degree of difficulties in dealing with sub-shape detection and emergent control. The rule base includes the Original Rule and the Transitional Rule, as shown in Figs. 3 and 4. The former one is used at the beginning of the design process to generate the stem and to ascertain the position of petal patterns. The latter one is used to generate a rough distribution of the petal patterns. In Fig. 3, a slim rectangle represents one stem, while a plump rectangle represents one petal. A dot in this figure signifies the label to mark the current operational shape. In our shape grammars, labels can indicate the shape manipulating sequences, and spatial relationship concurrently. Ten families of transitional rules can represent all the possibilities of shape transformations. In Fig. 4, ‘x’ and ‘y’ represent the original scale of a rectangle, while ‘a’ and ‘b’ represent the variable values of ‘x’ and ‘y’, respectively. For convenience in computation, we arrange all the four parameters in a domain of R+ . Therefore, ‘x + a’ means increasing the length of ‘x’ with a value of ‘a’ in the R+ direction, ‘x − a’ means decreasing the length of ‘x’ with a value
3.2.5. Refined Level At the Refined Level, we only need to focus on the transformations from the rectangles into curvy petals and stems. As mentioned above, the generation of petal and stem is a process involving both users and the automatic program. The color issue and overlap avoidance issues need to be considered as well. Following the feature analysis of our embroidery examples, we decomposed every single petal into a composition of two curvy lines for two reasons. The first is to avoid breaking the fluency and naturalness of one petal. The second is the inadequacy of the control ability to employ more advanced decomposition methods, such as the sets of control points. The system implementation will be demonstrated in Section 3.3.2. Here, the rule base is the Terminal Rule, as shown in Fig. 5. Terminal rules are mainly to create the final petal and stem shape. Once the replacements from all rectangle shapes to curvilinear shapes are computed, the process stops. The formal definition of our shape grammar can be defined below: SG (Shape grammar) = {S, L, R, I} S(Shape set) = ⟨Rectangle, Curvilinear Petal, Stem⟩ L(Label set) = ⟨•⟩ R(Rule set) = ⟨Original Rule, Transitional Rule, Terminal Rule⟩ I(Initial Shape) = ⟨⟩. In this two-level shape grammar, we do not treat the rules in a narrative way, since the purpose of this research is to utilize generative computation. For a generative system, the use of a descriptive rule base can achieve better computational efficiency. At the Coarse Level, the structure and skeleton framework are fixed. If the original rules and transitional rules are changed, we believe a new kind of design language for other product designs can be generated, such as chair structure, table structure and so on. At the Refined Level, particular elements can be helpful for generating specific styles of designs in a same category. 3.3. Computational implementation 3.3.1. Sub-shape detection and emergent shape First, we must solve the sub-shape detection problem. At the Coarse Level a shape is defined by a rectangle. Therefore, the detection algorithm introduced by Krishnamuti can be used [53].
Author's personal copy
596
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
Fig. 6. Emergent shape. (a) Uninteresting overlap. (b) 3-tuple set for petal.
Fig. 7. Images for overlap control. (a)–(c) The distribution of work areas; (d)–(f) the overcrowded situations only under the control of the 3-tuple set.
The emergent shapes are helpful to stimulate the creativity of a designer at the conceptual design stage [54]. A rational utilization is workable in our research to deal with the problems of emergent shapes. Using shape grammars, we can control the emergent shapes within an acceptable scale in order to create intuitive and interesting impacts. In our method, after application of the Coarse Level, the distribution of rectangles may generate lots of unintended overlaps by applying the terminal rule at the Refined Level (Fig. 6(a)). For reducing the overlap, we used a 3-tuple set {(Plefttop , Prightdown ), β, sub-direction} (Fig. 6(b)) to locate the petal at the Refined Level. A point pair (Plefttop , Prightdown ) is used to locate the proper rectangle, which is detected by a sub-shape detection algorithm. Here, β is the angle between diagonal line and x sideline, calculated from the rectangle parameters at the Coarse Level. Actually, a rectangle is divided into two working areas for petal generation: the upper part and the lower part, which can be valued by ‘sub-direction’ based on the calculations done by our algorithm. In Fig. 7, working areas marked by a dot will be replaced by the petals using terminal rules. In this case, this kind of distribution implemented by the 3-tuple set can reduce the possibilities of the unintended overlaps. It is noted that only by using the 3-tuple set, we cannot completely eliminate the overcrowded situation (Fig. 7(d)–(f)). Therefore, at the Refined Level, we have other parameters, the ‘petal number’ and ‘petal orientation’, which can be used to restrict the overcrowded situation further. Through experiments, we find that if the number of petals is less than 8, then we will minimize the overlaps to an acceptable degree. Some acceptable emergent shapes are shown in Fig. 8.
3.3.2. B-spline knot interpolation algorithm In the embroidery examples, petals are natural and smooth, which are difficult to find suitable mathematical equations to simulate. So, we used free B-spline curves to generate them. However, we find that the petal, composed by two B-spline curves, can bring some unintended changes after the transformations made by our grammar rules (Fig. 9). In Fig. 9, the small shapes are generated by B-spline curves; after we change the positions of these control points, transforming into different sizes, the distances between neighboring points were changed. This will lead to some unintended transformations which were aesthetically unacceptable. The reason is the control points in the original shapes are not required to be that many, while more are needed for keeping the shape appearance after scale transformation. So, if the number of control points is not increased accompanying the size changes, some unexpected transformations may occur. The use of isometric transformation can be one solution. However, this method generates the petals twice during one process, which will double the system cost in terms of both space and time. In our research, we introduce a new practical B-spline interpolation algorithm. Generally, the B-spline curves have a local feature. The changes of some knots in a local area will not influence the whole feature of the curve. Our method is to produce several parallel lines between two knots. The intersection points will be the interpolation knots of the curve. However, the parallel lines may intersect with other parts of the spline or have no intersection points, sometimes. As a result, this will cause undesirable exceptions during a computer simulation (Fig. 10). To overcome this problem, we define a sub-group, knots G = {G1 , G2 , . . . , Gn (n < m)} in a free spline with m knots. Normally,
Author's personal copy
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
597
Fig. 8. Some emergences of embroidery design. (a)–(c) The original parts; (d)–(f) emergent shapes through overlapping.
Fig. 9. Unsatisfactory shape transformations.
we can choose n = 4, but at the two ends we set n to 3. Some other parameters are also defined to calculate the tendency of the spline, as follows: Di(i+1) =
2
(Pxi − Px(i+1) )2 + (Pyi − Py(i+1) )2 ,
(2)
in which (Pxi , Pyi ), (Px(i+1) , Py(i+1) ) are two point positions of the spline. Di(i+1) is the distance of them;
µi = arccos((D2(i−1)i + D2i(i+1) − D(i−1)(i+1) )2 /2 ∗ D2(i−1)i ∗ D2i(i+1) ),
(3)
which is the angle of adjacent lines between three neighboring knots. After studying, we find that there is a rule to the tendency of spline curves. For example, four neighboring knots, Pi−1 , Pi , Pi+1 , Pi+2 , are in the group G with n = 4, in which Pi and Pi+1 are the objective points required to insert knots. D(i−1)i , Di(i+1) , D(i+1)(i+2) , µi , µ(i+1) are the lengths and the angles in the group, respectively. First, when the Pi−1 , Pi+2 are on the same side of the line (Pi Pi+1 ), the spline may protrude towards the opposite direction. Second, when Pi−1 , Pi+2 are on different sides of the line (Pi Pi+1 ), the segment of the spline between Pi and Pi+1 may protrude towards the opposite direction of Pi−1 at the part nearby Pi−1 , and vice versa. Third, the degree of the protrusion in the second situation depends on the proportion of D(i−1)i /Di(i+1) and Di(i+1) /D(i+1)(i+2) . Thus, we can calculate the protruding tendency based on the calculation, and produce the proper length and direction of the parallel lines to intersect with the spline between Pi and Pi+1 . The algorithm is as follows: B-spline interpolation computer algorithm. Object phase: Line Pi Pi+1 ; theGroup = {Pi−1 , Pi , Pi+1 , Pi+2 }, which n = 4; the free spline curve C ; the number of increasing knots is k per phase.
Our experiment shows the algorithm can deal with almost all the situations occurring in our application. However, there are two exceptions. First, it is not suitable for helical curves. Second, it is not suitable for pointed curves. If we can decompose the two cases into several smaller sub-segments, then the algorithm will be workable. We compared the results generated by our B-spline interpolation algorithm and isometric transformation; the conclusion is satisfactory. There is no obvious difference between the two examples generated. In Fig. 11, we can see clearly that after changing the size of the shapes, the one at the right side, generated by our algorithm, is the same as the middle one, generated by isometric transformation. The reason for this is we added 10 more control points between every pair of control points in the original shape. There are some famous B-spline knot insertion algorithms in the literature [55,56]. However, the algorithm mentioned here can
Author's personal copy
598
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
Fig. 10. Problem of interpolation.
Fig. 11. The result of B-spline interpolation computer algorithm. The left one is original model, the middle one is generated by isometric transformation, the right one is generated by the algorithm. Table 2 The parameters in Coarse Level. Character
Parameter value
Category Symmetry Number of parts Relationship Distribution
Single, double, multi None, half, quadruple, rotation 1, 2, 3, 4 Strictly, loosely, random The proportion between every length of two parts
Fig. 12. The petal of pattern. (a) Generated by algorithm. (b) Generated by user.
Table 3 The parameters in Refined Level. Character
Parameter value
Petal number Petal shape Petal color Petal orientation Stem category Stem color
1, 2, 3, 4, 5, 6 Petal database Color function Up, mid, down None, straight, lightly curve, moderate curve, exquisite curve Color function
resolve the B-spline interpolation problem in a computer for most cases of our embroidery examples automatically. 3.4. System implementation The Zhuang ethnic embroidery design system is built by Visual Studio C++ 2003 in a Windows XP system. The interface is implemented by XTreme ToolketPro 2008. Through a generative production mechanism, i.e., shape grammar, we can generate a large number of patterns with novel effects retaining the original features of embroidery examples. At the Coarse Level, several aspects of embroidery features are specified, such as the category style, the symmetry style, the number of parts, the relationship of parts and the distribution of parts, in which every single character is parameterized based on the corpora (Table 2). It should be noticed that the distribution is a proportion between two adjacent parts specified by the evaluating mechanism with some preconditions from the corpora. At the Refined Level, the petal number, petal shape, petal color, stem category, and stem color, which define the details of an embroidery design example, are parameterized, as shown in Table 3.
Fig. 13. Snapshot of XML database.
These characters are extracted from the examples of embroidery patterns. Among the parameters listed in Tables 2 and 3, the category, symmetry, number of parts, relationship, and petal number are finite variables, whilst distribution, petal shape, petal color, stem color and stem category are valued from a potentially large space based on a random possibility model. By considering only the finite parameters, the system can generate more than 4000 different styles of patterns using the generative mechanism. Including more parameters and the emergent shapes, there are too many new design patterns created by the system for designers/users. On the petal generation, we use two ways to build the petal database. The first is specified by the algorithm in our program. The second is specified by users or designers through a friendly interface. The algorithmic generation is following the style of the original corpora (Fig. 12(a)). The user generation can express different preferences of designers or users (Fig. 12(b)). After generation, a user can input a brief introduction, specify the petal orientation, and then save the petal in a database. We used the XML format to build the petal database to record the key numerous petal data (Figs. 13 and 14). In the system, a user/designer can control the generating process through input parameters. The system will then randomly select some petals from the ‘Petal Database’ and receive the necessary grammar rules. After data calculation and rule application, some of the results will be shown on the screen. Because of the random selection, our system can create quite different visual effects
Author's personal copy
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
Fig. 14. The system implementation process.
in different generations. At the same time, the style feather can be kept. In our experiment, we collected 14 corpora (Fig. 15) from the real embroidery designs of Zhuang ethnic minority people in southwest China. We choose some of the generation results shown in Fig. 16 with the key values of style parameters. Among the new generations, similar styles can be felt with different visual impacts. More embroidery designs created by our system are shown in the Appendix. 4. Design exploration Through a generative system, there are huge potential solutions created by the system. However, it is rather difficult for designers to find their favorite patterns from the system, because it is a timeand labor-consuming task. Therefore, a method to find the best solution(s) from numerous alternatives becomes necessary in our system. 4.1. Aesthetic evaluation model Many researchers in the field of CAD try to solve this problem by evolutionary algorithms. However, in design, especially visual art design, it is still difficult to define the search space. Besides, the powerful searching capabilities of evolutionary algorithms require a definite destination. In design, there is no clear description of the model in a designer’s mind before the final design has emerged. Therefore, the designers need to search for more materials or information, mostly images and patterns, in order to stimulate their inspiration. This process can be described as one in which the designers browse massive information many times, trying to find the matching requirements. Furthermore, during this process,
599
the requirements in a designer’s mind may not be stable, but will fluctuate in a range based on the received information. Sometimes, a designer may change his search direction to more information or alternatives following this fluctuating and subjective assessment. The weaker the fluctuation becomes, the more explicit the objectives may appear in a designer’s mind. At the end of this process, the most relevant information will be presented in front of designers, to stimulate their intuition and inspiration further. Comparing with the search process, exploration is a more proper process to model a real design situation. The main difference between search and exploration is whether the problem is well-defined [2]. If a problem is well-defined, the problem can be evaluated by definite criteria. In this case, the process can be regarded as a search. However, if the problem is ill-defined, the assessment standard may be ambiguous. In this case, the process can only be considered as an exploration. Many argue that design is an ill-defined problem. In our research, we take this view and introduce a novel aesthetic evaluation model for design exploration, based on the traditional aesthetic viewpoints [57]. There are four dimensions for the aesthetic evaluation: Structure, Element, Complexity and Subjective. Structure and Element are the traditional aesthetic standards [57]. Complexity mainly focuses on the element number of a pattern. The fourth dimension is subjective, which is different from the former three. One feature of the subjective dimension is data sensitive. For example, any tiny change in the former three dimensions may lead to a big fluctuation in the subjective dimension. Another feature is non-linear. It looks like a surface with many protrusions. There is no global maximum or minimum, but many extrema in a local region. We can consider that there is no absolute value, but lots of relative values in the subjective dimension. With the sensitive and non-linear characters, we can build a many-to-many mapping between the Structure, Element, Complexity and Subjective evaluation, in Fig. 17. 4.2. Design exploration Based on the viewpoint of romance of aesthetics [58], three aspects are chosen to define the values of subjective dimension, i.e., ‘balance’, ‘redundancy’ and ‘harmony’. It is not our objective to define the notions of the three, since different users with different design backgrounds may have different perceptions on them. Our aim is, however, to support the designers or users to explore their own favorite patterns in the large design space. For the structure dimension, ‘category’, ‘part proportion’, ‘symmetry’, and ‘relationship’ are extracted from the characters of embroidery examples to express the structure value. For the element dimension, ‘petal style’ ‘petal color’, ‘stem style’, and ‘stem color’ are extracted from the characters of embroidery examples to express the element value.
Fig. 15. The corpus of original embroidery designs.
Author's personal copy
600
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
Fig. 16. The generations from system.
For the complexity dimension, ‘petal number’ ‘pattern size (height and width)’, and ‘part number’ are extracted from the characters of embroidery examples to express the complexity value. Even if 12 characters are extracted from the three physical dimensions, this does not mean that each character only works in its source dimension. For example, the character of ‘petal number’ will have a strong relation with its source dimension, i.e., the complexity dimension, but at the same time with a weak relation with other dimensions, such as the structure dimension and element dimension. Therefore, a many-to-many mapping between subjective dimension and three physical dimensions can be built by Eq. (4) given in Box I. The subjective evaluation matrix is equal to the product of the parameter evaluation matrix and the characteristic matrix. From Eq. (4) we can know that when the Ebalance , Eredundancy and Eharmony are specified by a user, there are more than one group {a1 , a2 , . . . , a12 }, {b1 , b2 , . . . , b12 }, {c1 , c2 , . . . , c12 } that can match them, such as with the feature of subjective dimension, the instability, and multi-potential values. For convenient calculation by computer, we translate Eq. (4) into Eqs. (5)–(16): Ecategory = a1 ∗ Ebalance + b1 ∗ Eredundancy + c1 ∗ Eharmony ;
(5)
Epartproportion = a2 ∗ Ebalance + b2 ∗ Eredundancy + c2 ∗ Eharmony ;
(6)
Esymmetry = a3 ∗ Ebalance + b3 ∗ Eredundancy + c3 ∗ Eharmony ;
(7)
Erelationship = a4 ∗ Ebalance + b4 ∗ Eredundancy + c4 ∗ Eharmony ;
(8)
Epetalstyle = a5 ∗ Ebalance + b5 ∗ Eredundancy + c5 ∗ Eharmony ;
(9)
Epetalcolor = a6 ∗ Ebalance + b6 ∗ Eredundancy + c6 ∗ Eharmony ;
(10)
Estemstyle = a7 ∗ Ebalance + b7 ∗ Eredundancy + c7 ∗ Eharmony ;
(11)
Estemcolor = a8 ∗ Ebalance + b8 ∗ Eredundancy + c8 ∗ Eharmony ;
(12)
Epetalnum = a9 ∗ Ebalance + b9 ∗ Eredundancy + c9 ∗ Eharmony ;
(13)
Epatternheight = a10 ∗ Ebalance + b10 ∗ Eredundancy + c10 ∗ Eharmony ;
(14)
Epatternwidth = a11 ∗ Ebalance + b11 ∗ Eredundancy + c11 ∗ Eharmony ;
(15)
Epartnumber = a12 ∗ Ebalance + b12 ∗ Eredundancy + c12 ∗ Eharmony .
(16)
4.3. Experiment of design exploration Based on Eqs. (5)–(16), we can calculate the evaluation value of every parameter from the Structure, Element and Complexity dimensions. Using a heuristic method to change the pace of every generation following the users’ evaluation, the exploration of design can be developed towards a users’ desirable direction.
However, Eqs. (10) and (12), which deal with the colors of petal and stem, do not work in this experiment. This is because our current system implementation only determines the colors by a
Author's personal copy
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
Ebalance a1 Eredundancy = b1 E c harmony
1
a2 b2 c2
a3 b3 c3
a4 b4 c4
a5 b5 c5
a6 b6 c6
a7 b7 c7
a8 b8 c8
a9 b9 c9
a10 b10 c10
a11 b11 c11
E category Epartproportion Esymmetry Erelationship a12 Epetalstyle b12 ∗ Epetalcolor . c12 Estemcolor Epetalnum Epatternheight Epatternwidth
601
(4)
Epartnumber
Box I.
Fig. 17. Model of aesthetic evaluation.
Fig. 19. Exploration result 1. (a) Data analysis. (b) Generative result of exploration.
Fig. 18. Working environment of design exploration.
random model. Eq. (9) is also not used, which focuses on the petal shape. The petals generated by our algorithm or chosen by the users are saved in a petal database. Users choose the petal to form an embroidery design. After the design solutions are confirmed, the system cannot change the shapes of the ones chosen, but only by re-selection. Therefore, Epetalstyle is not calculated in our exploration. Here, the characteristic matrix is valued as follows, based on the real design sources calculated by the system designer:
0.9 0.1 0
0.7 0.1 0.2
0.5 0 0.5
0.2 0.1 0.7
0 0 0
0 0 0
0.1 0.3 0.6
0 0 0
0.2 0.6 0.2
0.1 0.7 0.2
0.1 0.7 0.2
0.2 0.5 . 0.3
To avoid disturbance of mass parameters for designers, we reduce the difficulties of operability and build a real time visual working environment. Designers need only drag the mouse to value their subjective value. The current parameter generated from
users’ response is shown in the control panel and the corresponding visual generation is shown on the computer screen. The work environment is shown in Fig. 18. We invited three designers with different design backgrounds to use the system, with the purpose of testing our evaluation method in the exploration process. Designer A is an interior designer, pursuing the shape of a powerful and free style. After the exploration process, his favorite parameters are: ‘double’, ‘none’, ‘1’, ‘strictly’, ‘2’, ‘none’, (sequenced by category, symmetry, partnum, relationship, petalnum, and stemstyle). The data and pattern are shown in Fig. 19. He said the selected patterns can let him feel fluent. The whole layout of the irregular patterns let him feel free without any limitation by some predefined structure. When he found this kind of pattern in the system, the final decision is made immediately. So the process is short with 13 generations only. Designer B is a city landscape designer, whose favorite is balanced and symmetric beauty. His choices of parameters are: ‘double’, ‘rotation’, ‘2’, ‘loosely’, ‘4’, ‘moderate curve’. The data and patterns are shown in Fig. 20. Most cities are built followed some regular rules. In our system, the attributions of ‘Symmetry’ and ‘Relationship’ have strong links to designer B’s preference. Actually, there are big amounts of patterns following the balanced and symmetric standard. Designer B selected Fig. 20(b) because the curvy stems give him a feeling outside expectation. Designer C is a conceptual designer on products. We gave her several patterns produced by the system, and let her choose the favorite ones. She likes the patterns with some special styles of Chinese culture (Fig. 21).
Author's personal copy
602
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
the four-dimensional evaluation model can translate the intangible design thinking into a tangible way (some data). In the design exploration process, we can record the data of every generation, i.e. Figs. 19(a) and 20(a). On the one hand, the data can be used to guide the next generation of design exploration. Based on the user’s subjective value, the new data can be calculated for another circle. The way how we can do this is implemented by the shape grammars. This process is the communication between designers (users) and computer. On the other hand, when we collect enough data sets, the data analysis methods and AI approaches can be used to analyze the data to improve the system more intelligently. We expect the analyzing result can help designers to find their design fixation and to inspire their creative thinking. This is the future final aim of introducing the four-dimensional evaluation model. In the near future, we hope the system can be improved more intelligently by the data we recorded. 5. Conclusions
Fig. 20. Exploration result 2. (a) Data analysis. (b) Generative result of exploration.
Fig. 21. Exploration result 3.
There are also some responses from other designers after using the system. Generally, they all considered that it is an interesting and novel experience in this form of design exploration, and confirmed that it is useful to help designers in their design processes. After analyzing their data, we found the conclusion is useful to help them to understand clearly about what their potential desirable design inclinations are. Although they all agreed that the system can help them to find their favorite styles, the generations from the system cannot be the final design without further modifications. There was some advice from our users. First, the color control is important for the visual design to present a stronger impact directly, which is the current deficiency of our system. Second, in most situations, they could find only part of the satisfactory patterns in one generation. As such, it is necessary to enhance the degree of control in order to represent designers’ feelings more accurately. Third, they suggested adding a call back operation, since sometimes they noticed that the patterns in previous generations are their truly favorite ones. But they could not call those solutions back because they had already been replaced by the new ones generated. CAD technology can help designers to show the solution visually. However, in design thinking creation, the effect is not as high as the expectation of CAD researchers. Towards this direction,
In this research, we study the application of shape grammars in art and design through a concrete sample, i.e., Zhuang ethnic embroidery design. We can see that using a descriptive way to represent shape grammar rules is easier for computer implementation, than the narrative way. Our two-level shape grammar structure can assure the system works in a quiet environment. By extracting appropriate features (parameters), the system can generate emergent shapes to support designers in a creative and intuitive manner. Integrating with a generative production system, a large number of design solutions with new styles can be generated. To enhance the communication between users and the system, a novel four-dimensional aesthetic evaluation model was developed for the exploration process of design, which was evaluated to be useful and feasible for professional designers. The preliminary results are exciting and promising. The CAD technology, embedding shape grammars as the formal approach, can indeed help designers in their creative design process, not just as a mechanical tool for mass production at the detailed design stage. This research can be further developed in several ways. First, one aim of using descriptive shape grammars is to generate a grammar rule base in a general way, not only suitable for embroidery designs, but also workable for other visual designs. We intend to explore this further. Second, the design exploration process needs to be improved continuously. At the current stage, the feature matrix is designated by experts or system designers. How to automatically adjust the feature matrix in the exploration process will be a challenging work in our further research. The current design generations from the system still need more changes in many details by designers before they can be considered as the final solutions. Therefore we need to focus more on shape aesthetic improvement and refinement, and for this part of the design process, only designers can still do the best job. Acknowledgments The authors are grateful to the assistance of designers Dr. H.Y. Song, Mr. Y.H. Huang and Miss Y.W. Zheng in Hong Kong, China, for their kindest participation in the research. This paper is sponsored by the Hong Kong UGC research grant, the general research fund (project number BQ-20L). Appendix. Parts of the embroidery design solution See Fig. 22.
Author's personal copy
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
Fig. 22. Parts of the embroidery design solution.
References [1] Agarwal M, Cagan J. On the use of shape grammars as expert systems for geometry-based engineering design. (AI EDAM) Artificial Intelligence for Engineering Design, Analysis and Manufacturing 2000;14:431–9. [2] Maher ML, et al. Formalising design exploration as co-evolution: a combined gene approach. In: Gero JS, editor. Advances in formal design methods for CAD. London (UK): Chapman & Hall; 1996. p. 3–30. [3] Brown KN, et al. Describing process plans as the formal semantics of a language of shape. Artificial Intelligence in Engineering 1996;10:153–9. [4] Trescak T. et al. General shape grammar interpreter for intelligent designs generations. In: International conference on computer graphics, imaging and visualization. vol. 6. 2009. p. 235–40. [5] Chomsky N, editor. Syntactic structures. Mouton: S-Gravenhage; 1957. [6] Stiny G, Gips J. Shape grammars and the generative specification of painting and sculpture. Information Processing 1972;71:1460–5. [7] Stiny G. Introduction to shape and shape grammars. Environment and Planning B: Planning and Design 1980;7:343–51. [8] Knight TW. Transformations of languages of designs: part 2. Environment and Planning B: Planning and Design 1983;10:129–54. [9] Knight TW. Transformations of languages of designs: part 1. Environment and Planning B: Planning and Design 1983;10:125–8. [10] Krishnamurti R. The arithmetic of shapes. Environment and Planning B: Planning and Design 1980;7:463–84. [11] Krishnamurti R, Earl CF. Shape recognition in three dimensions. Environment and Planning B: Planning and Design 1992;19:583–603. [12] Krishnamurti R. The maximal representation of shape. Environment and Planning B: Planning and Design 1992;19:267–88.
603
[13] Stiny G, Mitchell WJ. The palladian grammar. Environment and Planning B: Planning and Design 1978;5:5–18. [14] Koning H, Eizenberg J. The language of the prairie: Frank Lloyd wright’s prairie houses. Environment and Planning B: Planning and Design 1981;8:295–323. [15] Flemming U. More than the sum of parts: the grammar of Queen Anne houses. Environment and Planning B: Planning and Design 1987;14:323–50. [16] Cagdas G. A shape grammar model for designing row-houses. Design Studies 1996;17:35–51. [17] Cagdas G. A shape grammar: the language of traditional Turkish houses. Environment and Planning B: Planning and Design 1996;23:443–64. [18] Gero JS, Ding L. Exploring style emergence in architectural designs. Presented at the CAADRIA’97. Taipei (Taiwan). 1997. [19] Scott Howe A. Designing for automated construction. Automation in Construction 2000;9:259–76. [20] Duarte J. A discursive grammar for customizing mass housing: the case of Siza’s houses at Malagueira. Automation in Construction 2005;14:265–75. [21] Halatsch J. et al. Using shape grammars for master planning. Presented at the design computing and cognition’08. 2008. p. 637–55. [22] Müller P, et al. Procedural modeling of buildings. ACM Transactions on Graphics 2006;25:614–23. [23] Wonka P, et al. Instant architecture. ACM Transactions on Graphics 2003;22: 669–77. [24] Parish YIH, Muller P. Procedural modeling of cities. In: Proceedings of ACM SIGGRAPH 2001. 2001. p. 301–08. [25] Agarwal M, Cagan J. Shape grammar and their languages—a methodology for product design and product representation. In: Proceeding of DETC’97. 1997. [26] McCormack JP, et al. Speaking the Buick language: capturing, understanding, and exploring brand identity with shape grammars. Design Studies 2004;25: 1–29. [27] Alison Mckay M, Pennington A. Combining evolutionary algorithms and shape grammars to generate branded product design. In: Gero J, editor. Design computing and cognition’06. Netherlands: Springer; 2006. p. 521–39. [28] Pugliese MJ, Cagan J. Capturing a rebel: modeling the Harley–Davidson brand through a motorcycle shape grammar. Research in Engineering Design 2002; 13:139–56. [29] McCormack JP, Cagan J. Designing inner hood panels through a shape grammar based framework. (AI EDAM) Artificial Intelligence for Engineering Design, Analysis and Manufacturing 2002;16:273–90. [30] Lee HC, Tang MX. Generating stylistically consistent product form designs using interactive evolutionary parametric shape grammars. In: CAIDCD’06, 2006. [31] Yu W. Research of the style computational model for ming-style learning system. Presented at the international conference on networking and digital society. 2010. p. 484–7. [32] Kirsch JL, Kirsch RA. The structure of paintings: formal grammar and design. Environment and Planning B: Planning and Design 1986;13:163–76. [33] Tapia M. Chinese lattice designs and parametric shape grammars. The Visual Computer 1992;9:47–56. [34] Zhang WL, Lin SN. Research on tibet tangka based on shape grammar. Presented at the computer-aided industrial design and conceptual design, 2008. p. 373–6. [35] Cui J, Tang MX. Chinese pattern design using generative shape grammar. Presented at the GA2010. Milan (Italy) 2010. [36] Stiny G. A note on the description of designs. Environment and Planning B: Planning and Design 1981;8:257–67. [37] Stiny G, editor. SHAPE: talking about seeing and doing. Cambridge: The MIT Press; 2008. [38] Stiny G. Kindergarten grammars: designing with Froebel’s building gifts. Environment and Planning B: Planning and Design 1980;7:409–62. [39] Heisserman J, Woodbury R. Generating languages of solid models. In: SMA’93: proceedings on the second ACM symposium on solid modeling and applications, 1993. p. 103–12. [40] Heisserman J. Generative geometric design. Computer Graphics 1994;14: 37–45. [41] Parts M, et al. Shape exploration of designs in a style: toward generation of product designs. (AI EDAM) Artificial Intelligence for Engineering Design, Analysis and Manufacturing 2006;20:201–15. [42] Flasinski M. Use of graph grammars for the description of mechanical parts. Computer-Aided Design 1995;27:403–33. [43] Wang YF, Duarte JP. Automatic generation and fabrication of design. Automation in Construction 2002;11:291–302. [44] Gero JS. Creativity, emergence and evolution in design. Knowledge-Based Systems 1996;9:435–48. [45] Speller TH, et al. Using shape grammar to derive cellular automata rule patterns. Complex Systems 2007;17:79–102. [46] O’Neill M. et al. Shape grammars and grammatical evolution for evolutionary design. In: Proceedings of the 11th annual conference on genetic and evolutionary computation. 2009. p. 1035–42. [47] Koutsourakis P. et al. Single view reconstruction using shape grammars for urban environments. Presented at the 12th international conference on computer vision. 2009. p. 1795–02. [48] Simon HA, editor. The sciences of the artificial. Cambridge: MIT Press; 1996. [49] Come D. et al. Solving design problems by computational exploration. Presented at the formal design methods for computer-aided design. NorthHolland. Amsterdam. 1994.
Author's personal copy
604
J. Cui, M.-X. Tang / Computer-Aided Design 45 (2013) 591–604
[50] McCormack JP, Cagan J. Curve-based shape matching: supporting designers’ hierarchies through parametric shape recognition of arbitrary geometry. Environment and Planning B: Planning and Design 2006;33:523–40. [51] Talbott K. Context-sensitivity and visual engagement in generative systems. Automation in Construction 2007;16:54–60. [52] Woodbury R. et al. Tartan worlds: a generative symbol grammar system. In: Mission–method–madness: conference proceedings of association for computer aided design in architecture. 1992. p. 211–20. [53] Krishnamurti R. The construction of shapes. Environment and Planning B: Planning and Design 1981;8:5–40.
[54] Krish S. A practical generative design method. Computer-Aided Design 2011; 43:88–100. [55] Qin KH, Guan YJ. Two algorithms for inserting knots into B-spline curves. Chinese Journal of Computers 1997;20:556–61. [56] Pan RJ. An uniform approach to knot insertion and degree elevation of B-spline curves. Mini-Micro Systems 2002;23:335–8. [57] Parker DH, editor. The principles of aesthetics. New York: Appleton-CenturyCrofts; 1946. [58] Reich T. A model of aesthetic judgment in design. Artificial Intelligence in Engineering 1993;8:141–53.
