An Educational Tool for Basic Techniques in Beginner’s Pencil Drawing Saeko Takagi†∗ Noriyuki Matsuda† Masato Soga† Hirokazu Taki† ‡ Takashi Shima Fujiichi Yoshimoto† Faculty of Systems Engineering, Wakayama University† L.O.T Co., Ltd.‡
Abstract A picture is one of important research subjects in order to make our life spiritually rich. Most studies on pictures, however, only propose some substitute functions of actual drawing or painting materials. There is no system that evaluates pictures drawn by users and gives advice about them. We propose a learning support system for basic techniques in beginner’s pencil drawing. The proposed system receives a subject (motif) data set and an image of user’s sketch and returns advice to the user. The system is composed of the four subsystems: feature extraction of motifs, feature extraction of sketches, error identification, and generation and presentation of advice. We developed and experimented a prototype system limited to treat a basic motif and some principal advice. As a result, the validity of the proposed system was confirmed. Keywords: computer aided instruction in sketching, image processing of pencil drawings, evaluation of pictures
1. Introduction Our life is becoming materially rich owing to recent developments of various technologies. People are coming to want to making their life spiritually rich [13]. It is therefore important to support various activities for making our life spiritually rich. Drawing or painting is one of the activities that can provide us with mental satisfaction. There are many studies related to generating artistic computer graphics. In the field of non-photorealistic rendering, including a historic work by Haeberli [6], a lot of drawing or painting algorithms were proposed [4]. In particular, Kasao et al. extracted some features from several types of actual sketches and converted test images into these types of expressions [10]. However, it has not been considered that a system gives a user some evaluation or advice about pictures generated by him/her. In the design fields, some assistant systems were proposed. Although these systems aid users in making a certain design ∗ Postal
address: 930 Sakaedani, Wakayama city, Wakayama 640-8510, Japan, E-mail:
[email protected]
such as posters [8, 12], they do not teach us how to draw what we see. As applications of image analysis, there were some systems which gave users some appropriate examples in order to refine their art works [11, 14]. These systems evaluate user’s art works at color or composition based on some rules or comparison with several professional paintings. However, a user’s work is not evaluated in relation to what the user actually sees. For six months, we had observed a culture school of sketches for beginners. There are many errors in the sketches drawn by beginners, because they could not draw what they saw. In addition, although a teacher pointed out the errors to them, common students were not able to correct the errors by themselves. Most beginners need concrete advice based on their errors. In this paper, we propose an educational tool for basic techniques in beginner’s pencil drawing. The system aids beginners in learning a basic sketch using real pencils and drawing paper. The system receives a subject (motif) data set and an image of user’s sketch and returns advice to the user. We consider that beginner’s sketches can be evaluated with comparison of the sketch and the portrayed objects. Sketching is the essential basis of drawing and painting. To observe an object correctly is a one of fundamental skills in sketching and to point out errors in user’s drawings is helpful in order to correct misunderstanding of objects. The purpose of the system is not only the support of selfeducated beginners but also the auxiliary role of teachers in mass schooling. The rest of this paper is organized as follows. In Section 2, the overview of the proposed system is described. In Section 3, the details of four subsystems and the process flow are explained. In Section 4, the developed prototype system is mentioned and some experiments of it are described. Section 5 contains concluding remarks.
2. Overview of the proposed system The purpose of beginner’s sketch study is to raise the ability of portraying what he/she is seeing itself [3]. Then, if a sketch is drawn based on this purpose, it can be evaluated
Figure 1. Overview of the proposed system.
Figure 2. Example of a sketch.
with comparison of the sketch and the portrayed objects. The overview of the proposed system is shown in Figure 1. First, a learner inputs the motif composition data such as size of objects and distance between objects. A 3D model of the motif is reconstructed and some features of the motif are extracted. Next, the learner takes a picture of his/her sketch with a digital camera. The photographed image is taken into the system. Some features of the sketch are also extracted. Comparing the obtained features, the system identifies drawing errors based on a knowledge base of errors. Then, several kinds of advice are generated based on a knowledge base of advice, his/her learning history, and a curriculum. Finally, the advice is presented to the learner. By the way, we considered which is better for learners, direct drawing on a computer with some devices such as a tablet, or classic drawing with the actual pencil and drawing paper. These computer devices have not yet reproduced the high expression and good feeling of the actual drawing materials. The eraser techniques are used extensively in the actual sketch. In the use of such devices, however, it is difficult to draw with a eraser. There is a comparison report of drawing to paper and drawing on a computer [2]. In the report, using a computer for drawing, user’s attention was turned to how to use a tool, and there was bad influence on drawing activities. For these reasons, we adopt the classic and familiar drawing materials.
Figure 3. Drawn areas.
3. Data processing in the proposed system 3.1. Feature extraction of motifs A 3D model of objects portrayed by a user is built from the composition data of the shape and the arrangement. The motif composition data contains size of the objects, distance between objects, distance from the user to the objects, the user’s eye level, etc. From the composition data, a 3D model is generated and several images of the 3D model seen from the user’s view-
point are rendered. Some feature parameters of the motif are calculated from these images. The feature parameters are used as a correct model in identification of errors.
3.2. Feature extraction of sketches For the evaluation of sketches, it is necessary to extract some useful features from sketches. In the feature extraction subsystem for sketches, the following steps are performed. First, a rectangular region is cropped from the original photographed image that is regarded as the drawing paper. The pencil-drawn area is extracted from the cropped region using the texture analysis based on the co-occurrence matrix and the gradient-based edge detection [1]. The texture analysis is used to divide the cropped region into two parts, temporary drawn areas and paper areas. The edge detection is used to extract the outlines of the actual drawn area. Because a good result cannot be got by using only one method, we combine both of results from the two methods for the extraction of the actual drawn area. The temporary drawn areas based on the texture analysis are calculated a little extensively. In the temporary drawn areas, the areas surrounded by outlines the are regarded as the actual drawn
Figure 5. Example of presenting text advice. Figure 4. After classification. areas. An example of a cropped sketch is shown in Figure 2, and the actual drawn areas in the sketch is calculated as Figure 3. In order to obtain single-pixel-wide center lines, the thinning algorithm [1] is performed to the drawn areas. The obtained lines are utilized as the skeleton data of the sketch for specification of the each target part. The result of thinning is classified to branch points and segments which connects two branch point. Making use of the composition data of the motif and some geometric features, each specific part is described with these branch points and segments. Some noises are reduced in this stage. An example after classification is shown in Figure 4, which is the result of processing the image in Figure 3. After specifying the target parts, some feature parameters are calculated by several heuristic methods. The reason using such methods is that a user’s sketch is frequently quite different from the motif looked at from his/her viewpoint.
3.3. Error identification From the feature parameters of a motif and a sketch obtained by the previous stages, some parameters for judging errors are calculated. The judging parameters are checked with the rules in a knowledge base of errors, then, some errors are identified. Certain parameter groups about the degree of errors are also stored in the knowledge base. The error knowledge base and the parameter groups for judging error are built on advice data sets obtained from the advice in actual art classes for beginners. About each error, the kind of the error, the part containing the error, and the degree of the error are identified. The results are passed to following subsystem.
3.4. Generation and presentation of advice Generated advice is presented to a user through four media: text, voice, sample illustration, and 3D model.
First, checking with the rules in a knowledge base of advice, the advice by text (and voice) is generated based on the kind and degree of errors. The curriculum and each leaning history are also considered. The knowledge base of advice is built on advice data sets obtained from the advice in actual art classes for beginners and related books. The advice on errors is first generated and if necessary, supplementary advice is also generated. Since too much quantity of advice will spoil user’s will to sketch, the advice about all of the errors is not presented to the user. Advice is selected based on the importance in pencil drawing, the frequency of use in actual art classes, each user’s learning history, etc. In addition, the presentation order of advice is also controlled for making the advice more effective. The selection and control are executed by certain algorithms established on analyzing the advice data sets in actual art classes. The generated advice of text is displayed at an advice window and also read aloud with synthetic voice. Sample illustrations are used as assistance of text and voice advice. They contain auxiliary lines, technical illustrations about perspective, and correct sketches. Overlaying sample illustrations on a user’s sketch is considered to raise understanding of advice. Figure 5 shows an example of windows presenting text advice with overlaying auxiliary lines on a user’s sketch. Advice by displaying a 3D model is helpful for users to understand the existence of errors in their own sketches[9]. In the field of computer aided instruction, “error visualization” is used in order to support users’ awareness of the errors for learning from mistakes [7]. In this system, based on the motif composition data, a basic 3D model is generated. Incorrect 3D models are generated by adding the result of error identification to the basic 3D model. In usual means in art classrooms, such as oral explanation and 2D illustration, it is difficult to explain 3D contradiction in a sketch. On the other hand, the explanation becomes easy by error visualization with the incorrect 3D model, and users can also understand errors more intuitively. Figure 6 shows an
In the error identification subsystem, the degree of each error was judged in a five-step evaluation. In the generation of advice, the number of errors contained in one advice presentation is limited up to three. In addition, when the sketch has few errors, the system praises the sketch. The control of advice is based on the analysis of the coverage in the art class.
4.2. Experiment of feature extraction of sketches
Figure 6. Example of error visualization with 3D model.
example of windows presenting 3D models.
4. Experiment of a prototype system 4.1. About handled advice in the prototype system A prototype system was developed to verify validity of the proposed system. For the prototype system, we chose a circle plate and a beer mug as a target motif. Straight-lines and ellipses are basic factors in sketching. They are clearly contained in this motif. In addition, ellipses at different eye levels are good examples for training beginners to observe an object correctly. Generally, a sketch consists of two elements, which are a form and a torn [5]. Beginners start for catching a form. Therefore, as a first step, we treat a form. We establish several conditions of target sketches: a sketch is drawn with deep lines and without any assisting lines or shade. We selected the following eleven items of advice whose frequency were high in a certain art class for beginners. 1. 2. 3. 4. 5. 6. 7. 8.
The viewpoint for the plate is too high. The margins are not same. The plate is big/small. The mug is thick/thin. The roundness of the plate is not enough. The end of the plate is sharp. The relations of ellipses in the mug are wrong. The width of the mug is not constant from top to bottom. 9. The rim of the plate bottom is high. 10. The mug is slanted. 11. The plate is deep/shallow. For these items of advice, the system calculates about forty parameters of the motif, about fifty parameters of each sketch, and about twenty parameters for judging errors.
Unless the required parameters can be correctly extracted from sketches, any effective evaluation and advice about the sketches cannot be returned to the users. Then, we investigated whether the required parameters could be extracted from actual sketches. The experimental objects were the sketches which were drawn by three types of people: ten students in Wakayama University, sixteen people in a culture school held at a public hall, and twenty three fourth-grade schoolchildren. All of them had little knowledge and skill in drawing. Since university students had been told in advance that drawing with clear and deep lines was desirable, in sketches drawn by the students, the feature extraction had been carried out almost correctly. In the culture school, some auxiliary lines were left in many sketches and there were two sketches drawn with shade. After erasing these lines and shaded parts finely, the feature extraction of sketches was mostly able to be carried out. Enough parameters were not calculated in one sketch. The texture analysis was unsuccessful in this sketch, all of the drawn areas were not extracted. On the other hand, about the sketches drawn by schoolchildren, it was hard to extract the required parameters. The main reason is failure in the classification of the target part, because the drawing line was not continuous in the original sketch. In addition, the schoolchild’s sketch often protruded from the drawing paper. The current prototype system can not deal with this type of sketches.
4.3. Experiment and evaluation of the prototype system by beginners Ten students used the prototype system. After learning sketch with our system, they answered a questionnaire about the feeling of use. The questionnaire was composed of six items about the operation of the system and eight items about the advice given by the system. Figure 7 (a) shows an example of motifs and (b) shows a typical example of sketches drawn by the students before using the system. The prototype system gave the student the following comments about this sketch: (1) “The ellipses in the cross sections of the mug are drawn from incorrect viewpoints.” This advice was presented with an illustration as shown in Figure 8(a).
(a) Motif. (a) About the mug.
(b) Sketch.
(b) About the plate.
Figure 7. Each example of actual motifs and sketches.
Figure 8. Examples of advice windows with text, voice, and illustration.
(2) “The composition is almost good, but it tends to be in a little right.” (3) “Although the form of the plate is almost well drawn, the plate drawn by you looks as if your eye level is too high. It is seen more slender in your viewpoint.” (4) “In order to draw an ellipse, by drawing an appropriate rectangle in this way, you can draw an ellipse more easily. a and b, and c and d are horizontally symmetrical. a and c, and b and d are almost vertically symmetrical, too.” This advice was presented with an appropriate rectangle and an ellipse on the sketch as shown in Figure 8(b).
About operation of the system, most users answered that measurement of the motif data was troublesome. Since the input of the photographed images to the system took time, the complaint also came out mostly that the process of the system was not smooth. It is necessary to improve the input interface. On the advice given by the system, about half of the students answered that the advice was easy to understand. Moreover, almost all the students answered that his/her sketch became better after correcting it according to the advice. As mentioned above, it is considered that the advice given by the system was effective. However, there was also a complaint that it was hard to understand how a sketch should be corrected. It is found that the given advice was not enough for beginners.
Then, 3D models were displayed as shown in Figure 9 that are the correct model based on the motif (left) and the incorrect model based on the sketch (right). After the set of advice was given by the system, the student modified his sketch. As shown in Figure 10, several points were improved from the sketch before using the system, which were the size of the plate, the roundness of the plate, two ellipses and the bottom curve of the mug, etc. The sketch after the advice was resembling the actual motif more than the sketch before using the system.
4.4. Evaluation of advice from the viewpoint of an art teacher We showed an art teacher the experimental result of university students’ sketches before and after advice. Consequently, it turned out that advice given by our system was almost suitable from the viewpoint of the art teacher. More-
Furthermore, we should deal with a tone. After dealing with both of a form and a tone about a circle plate and a beer mug, we will study other motifs such as fruits on a plate.
Acknowledgment We thank Ryuzo Goda, Hideaki Kawanishi, Nobuyuki Kajimoto, and Yoriko Maruyama who contributed to develop the prototype system.
References Figure 9. Example of advice windows with 3D model.
Figure 10. Example of sketches after advice. over, comparing sketches before and after advice, it was estimated that most sketches were improved. Moreover, the system could judge a minute error which was not noticed easy by the art teacher. However, it was pointed out that the system gave a user unsuitable advice about one sketch. The reason is that the plate region and the mug region were recognized contrary and the feature extraction failed, because the plate was too big and the mug was too small in the sketch.
5. Conclusion We proposed a learning system which advise beginners on a form in pencil drawing. The system is composed of the four subsystems: feature extraction of motifs, feature extraction of sketches, error identification, and generation and presentation of advice. A prototype system was developed and experimented to about fifty sketches drawn by several kinds of beginners. The prototype system gave effective advice about the sketches to the beginners and they could modify their sketches based on the advice. The validity of the proposed system was confirmed. However, it is necessary to raise the accuracy of the feature extraction.
[1] K. R. Castleman. Digital Image Processing. Prentice Hall, 1995. [2] A. Chikawa, M. Iwata, and S. Tano. Effect on human creative work by computer-supported drawing and sketching on paper. In Proceedings of Human Interface 2000, pages 219– 222. Human Interface Society, 2000. [3] R. De Reyna. How to Draw What You See. Watson-Guptill Publications, 1996. [4] B. Gooch and A. Gooch. Non-Photorealistic Rendering. A K Peters Ltd., 2001. [5] A. L. Guptill. Rendering in Pencil. Watson-Guptill Publications, 1979. [6] P. E. Haeberli. Paint by numbers: Abstract image representations. Computer Graphics (Proceedings of ACM SIGGRAPH 90), 24(4):207–214, 1990. [7] T. Hirashima, T. Horiguchi, A. Kashihara, and J. Toyoda. Error-visualization by error-based simulation and its management. International Journal of Artificial Intelligence in Education, 9(1):17–31, 1998. [8] J. Ichino and S. Tano. User interface for artistic patterns creating support. The Transactions of the Institute of Electronics, Information and Communication Engineers, J82D-II(10):1693–1709, 1999. [9] N. Kajimoto, N. Matsuda, H. Taki, M. Soga, S. Takagi, F. Yoshimoto, and T. Shima. The proposal of the technique of error visualization for a learner’s pencil drawing. In Proceedings of International Conference on Computers in Education 2002, pages 153–157. APC/AACE, 2002. [10] A. Kasao and M. Nakajima. Characteristics extracted from sketches and a drawing simulation. The Journal of The Institute of Image Information and Television Engineers, 53(6):873–881, 1999. [11] K. Nakakoji, B. N. Reeves, A. Aoki, H. Suzuki, and K. Mizushima. emmac: knowledge-based color critiquing support for novice multimedia authors. In Proceedings of ACM Multimedia ’95, pages 467–476. ACM, ACM Press, 1995. [12] T. Obata and M. Hagiwara. A color poster creating support system on reflect kansei. Transactions of Information Processing Society of Japan, 41(3):701–710, 2000. [13] Public Relations Office, Minister’s Secretariat, Cabinet Office. Public Opinion Poll about the National Life (Investigation in June, 2002). Government of Japan, 2002. [14] S. Tanaka, J. Kurumizawa, S. Inokuchi, and Y. Iwadate. Composition analyzer: support tool for composition analysis on painting masterpieces. Knowledge-based Systems, 13(7-8):459–470, 2000.
