information features into 3D model retrieval, analyze (discuss) technologies of knowledge extraction in 3D model ..... Journal of Chongqing University of Posts.
Available online at www.sciencedirect.com
Physics Procedia 25 (2012) 1445 – 1450
2012 International Conference on Solid State Devices and Materials Science
Research on Key Technologies of 3D Model Retrieval Based on Semantic Siqing Luo, Li Li, Jing Wang Information and Computer Engineering College in Northeast Forestry University,Harbin, Heilongjiang, China
Abstract Content-based three-dimensional (3D) model retrieval technology mainly reflects the physical information of model but it can not properly express the semantic information. As to this problem this article will apply user's semantic information features into 3D model retrieval, analyze (discuss) technologies of knowledge extraction in 3D model retrieval based on semantic, semantic, active learning mechanism of semantic retrieval, semantic retrieval of 3D model and other key technologies. On the basis of that, this paper finally proposes research directions in future.
© 2012 2011 Published by Elsevier ofof [name © Elsevier Ltd. B.V.Selection Selectionand/or and/orpeer-review peer-reviewunder underresponsibility responsibility Garryorganizer] Lee Open access under CC BY-NC-ND license. Keywords: content-based retrieval; semantic features; ontology; semantic retrieval;
1.
Introduction
With the rapid development of multimedia, computer vision technology and three-dimensional graphics hardware, 3D model has got rapid growth in terms of number, and has an increasingly wide range of applications. Therefore studying on 3D model retrieval technology (Content-based 3D Retrieval), helping users quickly and accurately obtain three-dimensional models which meet the design intention, and reusing resources have become a hot issues in research [1]. Content-based retrieval is trying to achieve the retrieval goal by automatically creating a characteristic index which can reflect the content and information of shape features of three-dimensional model. Contentbased 3D model retrieval focuses on the feature extraction, feature description and feature comparison of 3D model. Shape features mainly include shape features based on geometric structure analysis, shape features based on topological relation, shape features based on function projection and shape features based on statistical characteristics. Any kind of shape feature has both its advantages and disadvantages. So far no representation form can be in full compliance with human’s recognition and ability and understanding about shape features. For research in this area, there are a large number of review articles for referring [24].
1875-3892 © 2012 Published by Elsevier B.V. Selection and/or peer-review under responsibility of Garry Lee Open access under CC BY-NC-ND license. doi:10.1016/j.phpro.2012.03.260
1446
Siqing Luo et al. / Physics Procedia 25 (2012) 1445 – 1450
Different from shape feature, semantic feature converts low-level geometric data of 3D model (point coordinates, sides, surface equation, etc.) into a high-level knowledge, and forms a unified framework of 3D model retrieval and reuse with semantic knowledge representation mechanism. With the help of 3D semantic retrieval framework, users can not only effectively express query intent and retrieve model fast and easily, but also can reflect products information of usage, functions, design principles, types and others. Therefore, research and application of 3D semantic retrieval technology has become a new research field. 2.
3D model semantic retrieval framework
3D Model retrieval framework mainly includes 3D model semantic extraction, semantic representation, semantic knowledge learning and matching mechanism. The whole framework can be represented by the Figure 1, including the definition of semantic rules, initialization of semantic learning mechanism, the definition and storage of index structure of 3D model semantic knowledge, and return retrieval results of user-submitted retrieval content by regular understanding and matching them with semantic knowledge matcher. At the same time, according to information users input, such as user tagging, user feedback information, system can update 3D model semantic knowledge base through semantic learning mechanism.
Figure.1 semantic-based 3D model retrieval process
3.
Research on technology of semantic-based 3D model retrieval
Technology of semantic-based 3D model retrieval is mainly about how to describe like shape features of 3D model with high-level semantic. This paper realizes such descriptions through establishing a corresponding user model library, including descriptions of users’ interests on a certain model, and features of design and application for special areas, as shown in Figure 2. Model library can help to realize tagging on 3D model, bridging the gap between low-level visual and high-level semantics, and realizing automatic semantic tagging according to users’ interests and intentions by means of learning and inference mechanism. For this purpose, this paper proposes theory and methods of semantic-based 3D model retrieval technology theoretically.
Siqing Luo et al. / Physics Procedia 25 (2012) 1445 – 1450
Figure.2 user semantic model
3D model semantic can be divided into three layers. From the bottom up they are user feature semantic, user object semantic, user scene semantic, user behavior semantic and user emotional semantic. User feature semantic gives description of some visual features like color, texture and shape that users interested in. User object semantic describes the objects and stuff users interested in. User scene semantic, user behavior semantic and user emotional semantic describe scene, behavior and emotion users interested in. User semantic model can be obtained by conducting implicit cumulative learning on information users have visited. (1) Level 1: user feature semantic describes some visual features like color, texture and shape and other features users interested in. It can be extracted directly and automatically through 3D models. (2) Level 2: User object semantic describes the objects and stuff users interested in. It cannot be extracted directly through 3D models, and it needs to first establish object semantic template for every type of 3D model and then get object semantic after identification. (3) Level 3: User scene semantic, user behavior semantic and user emotional semantic describe scene, behavior and emotion users interested in. This is the highest-level semantic which needs to establish inference rules and can obtained through inference. After creating semantic model, system has the function of advanced semantic query. For example, users can submit “Happy”, “sad” and other high-level semantic to retrieve relevant 3D models. Level2 and level3 are considered as high-level semantic, and the gap between Level1 and level2 semantic is the gap of semantic. 3.1 D model semantic Extraction Extraction of 3Dl model semantic needs to convert those low-level visual knowledge features and topological structure features in low level like shape feature of 3D into high-level semantic. In the establishing process of 3D model semantic, it is necessary to create semantic index first for all 3D models in 3D model library, and then extract feature semantic, object semantic, scene semantic, behavior semantic and emotional semantic for every 3D model. Through learning Information accessed by users, we can get 3D models users interested in, and then get user feature semantic, user object semantic, user scene semantic, user behavior semantic and user emotional semantic. In the model in user semantic of level 3, the low-level visual feature semantic such as color, texture and shape can be calculated directly through 3D model. Color feature extraction in this paper adopts nonuniform quantization algorithm of 20 colors based on HSV space. In addition, a new non-uniform quantization algorithm of 20 color based on HSV space has been proposed in color feature extraction.
1447
1448
Siqing Luo et al. / Physics Procedia 25 (2012) 1445 – 1450
As to quantization of component H, S, V in non-uniform quantization algorithm of 20 colors based on HSV space, please see formula (1) ° ° ° ° °H ° ° ° °° ® ° ° ° ° °S ° ° °V ° °¯
0 °1 ° °2 ° °3 ® °4 °5 ° °6 °7 ¯
if if
h ( 330 , 360 ] [ 0 , 25 ] h ( 25 , 41 ]
if if if if
h ( 41 , 75 ] h ( 75 ,156 ] h ( 156 , 201 ] h ( 201 , 272 ]
if if
h ( 272 , 285 ] h ( 285 , 330 ]
0 ® ¯1
if if
s ( 0 . 1 , 0 . 65 ) s [ 0 . 65 ,1 ]
0
if
v ( 0 . 15 ,1 ]
(1 )
In accordance with quantization level above, synthesize the three components H, S, V to onedimensional feature vector
l HQsQv SQv V
(2)
Where Qs and Qv are quantization results of components S and V. from equation (1) we can know tha Qs = 2, Qv = 1, so formula (2) can be converted into:
According to the formula (3), range of l is [0, 1… 19], calculate I and then get a one-dimensional histogram of 20 handle. Texture features extraction in this paper uses gray level co-occurrence matrix. Select four statistics including contrast, uniformity of texture, the correlation of pixels and gray, and entropy which represent texture features as feature vector. This paper integrates color, texture and shape features of 3D model as feature vector of 3D model, uses support vector machines to establish the semantic classifier for all semantic class, and then achieve the automatic extraction of semantic objects of 3D model. Calculation of scene semantic, behavior semantic and emotional semantic of 3D model is relative complex s. it needs to establish inference rules and be obtained by inference. 3.2 User interest model User interest model is a kind of learning mechanism which is combined with theories of computer vision and statistical learning based on all kinds of extraction knowledge to bridge the gap between lowlevel visual and high-level semantic. User intention model is to describe user’s next operation or other goals according to their current or former operation. User interest model describes the recent interests and hobby of users, which is mainly obtained through the statistics and information users visited, and it requires constant update. User interest model can provide users with personalized 3D model retrieval, and personalized recommendation service.
Siqing Luo et al. / Physics Procedia 25 (2012) 1445 – 1450
User interest model is automatically implicit. System automatically adjusts the user's interest model in accordance with their visit history. If a user conduct operations such as queries, browse, zoom in 3D model, click relevant links, save 3D model and so on, corresponding weight of this user’s semantic code will increase. When retrieving 3D model with user interest model, system will give a preferential return the semantic class with larger weight to users. According to the principle of nearest neighbor, it is more likely for users to visit the 3D model they have visited recently. Therefore, when the weights of semantic class are the same, semantics class user recently visited will be returned first. Table 1
Impact of user operations on the user interest user operations
Impact on user interest
Influence factor
Query
moderate
1
Browse
moderate
1
Zoom in 3D model
great
2
Click relevant links
great
2
Save 3D model
enormous
3
3.3 User intention model The user intention model is about semantic matching and inference based on ontology. Users’ intention can be described in multiple levels, as shown in Figure 3. Intentions in relative low level can be called as behavior intention which includes mouse clicks, keyboard input and other basic behaviors. Intentions in relative higher level are called as semantic intentions, which express users’ abstract ideas, for instance, user’s ultimate goal of browsing 3D models is to enjoy or get some relevant information. We can conclude users’ semantic Intention according to behavior intention, and then help users to complete tasks and provide them with personalized service.
Figure.3 User intention model
Using the user intention model can deduce the user intention and help users to complete tasks. Definition 1. In user intention model, suppose A and B as two arbitrary behaviors of users, and
P( AB) P( B) ! 0 .The Ratio p( B) is called the conditional probability of the behavior A happening under the behavior B
1449
1450
Siqing Luo et al. / Physics Procedia 25 (2012) 1445 – 1450
P( A B)
P( AB) p( B)
The example of deducing user intention: E.g. After modeling the intention of one user, the probability of him to selects 3D model and zoom in model is P( A) 0.8 , while the probability of saving 3D model is P( B ) probability of the saving 3D model is
P( AB) p( A) Due to B A , so AB B , then P( AB) P( B) 0.4 P( B A) p( A) p( A) 0.8
0.4 . And then, this user’s
P( B A)
0.5
It is known that users save the probability of this 3D model as 0.5. Therefore, when the user selects 3D model and enlarges and displays, it is direct to help users download 3D model automatically. 4.
Conclusions
In recent years, 3D model semantic retrieval has caused researchers’ extensive interests, and has obtained some achievements. This thesis studies the retrieval technology of 3D model, including the semantic level model, semantic knowledge extraction, and learning mechanism of semantic retrieval, etc.It mainly focuses on the automatic extraction of high-level semantic knowledge and semantic matching and inference based on ontology combining with statistics study, which is based on the previous ontology knowledge standards.
Acknowledgment This research was sponsored by Heilongjiang Natural Science Foundation project (F200922)
References [1] Pan Xiang㧚3D model shape analysis and retrieval[Ph㧚D㧚dissertation]㧚Zhejiang University㧘Hangzhouޕ2005(in Chi̅ nese) [2] PRUSINKIEWICZ P㧘LINDENMAYER A㧚The algorithmic beauty of plants[M]㧚New York㧦Springer̅Verlag㧘1990㧚 [3] Tangelder J㧘Veltkamp R㧚A survey of content based 3D shape retrieval methods㧛㧛Proceedings of the International Conference on Shape Modeling㧚Genova㧘Italy㧘2004㧦145-156 [4] Yang Yu-Bin㧘Lin HuiޕZhu Qing㧚Content̅based 3D model [5] retrieval㧦A survey㧚Chinese Journal of Computers㧘2004ޕ27 (10)㧦1297 ৻ 1310(in Chinese) [6] Zheng Bochuan.3D model retrieval technique using complementarity of features [J]. Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition). 2006㧘l8(1)㧦l19—122㧚 [7] Xiong Hua. 3D Model Retrieval Technique Based on Area Distribution [J]. Computer Engineering. 2008㧘34㧔12㧕㧦257㨪 261 [8] Osada Robert㧘Funkhouser Thomas㧘Chazelle Bernard㧚Shape distributions[J]㧚ACM Transactions on Graph̅ics㧘2002㧘 21(4)㧦807̅832㧚
