Using Handheld Devices as Alternatives to Inspire

Using Handheld Devices as Alternatives to Inspire Students Innovation in Designing New Information Technology Authors: Yo-Ping Huang, Tatung University, Department of Computer Science and Engineering, Taipei, Taiwan 10451 [email protected] Tsun-Wei Chang, Tatung University, Department of Computer Science and Engineering, Taipei, Taiwan 10451 [email protected]

Abstract — In this paper, we propose a new learning way under a wireless environment. A bird searching system for bird-watchers is proposed to help students’ creation. By utilizing the back-propagation model to train the neural network, we provide a graphical bird information searching system. While opening the feature selection interface, users can click or drag-and-drop the features according to the birds what they have observed outdoors. Based on the instinct feature-selection method, we can fulfill an intelligent bird information retrieval system. After users having finished feature selection, the system can proceed to a pattern comparison. The best-matched results are displayed on the PDA screen. Besides, students can study some frameworks including: the fundamental theory, system engineering, and creation of new learning way. Based on the experience from developing a bird searching system, we also discuss how we extend the ideas to inspire the students’ innovation to create more useful systems for handheld devices. Index Terms — Handheld device, information retrieval, information technology, PDA. INTRODUCTION The applications of handheld devices direct a new research field for information technology. The related technology is on the start-up stage. Along with the construction of national information infrastructure (NII) as backbone, the new handheld devices with the wireless network will be the best integrating solution to provide better services for people. In this century, innovation is the best method to promote a country. We are pursuing a new learning environment to stimulate our students’ originality by modern information technology. In the past years, we have concentrated on developing application systems for handheld devices. Based on our research, we have proposed a lot models including Web Pad-based meal-ordering system, content-based retrieval system, meta-search engine, and fuzzy data mining model. Due to both the entrance examination and low tuition fee policies every student prefers to enter into the national universities in Taiwan. Private university graduates, however, play more important roles in the country’s economical development [1]-[2]. Actually private universities have more flexibility to adjust their teaching and research directions and focus on some specific topics to develop in the quickly changing information technology world. Distance learning and e-learning have been successfully operated for many years. They overpass the distance problem to provide the students alternatives to acquire the new knowledge. Apart from broadening knowledge by the wired network, how can students learn more from the wireless world? With the advent of new technology in wireless network, handheld devices have become more and more popular. Thus, we summarize our experience from teaching students using the handheld devices and then creating new models during the past several years. In this paper, we investigate how to provide alternative learning paths for students in private universities to study new information technologies. Through using the handheld devices, how to inspire students innovation to design more useful systems will also be discussed. For example, we teach students to design a plant retrieval system based on the plant’s features as shown in Fig 1. Students get hand-on experience from designing such a system. Due to the system is very useful and convenient to help people recognize a plant’s name and basic characteristics, students later find that they can design some other similar systems, such as bird, fish, butterfly, and McDonalds meal-ordering system as shown in Fig. 2. Through the designing process, students learn how to simplify the feature extraction, how to process the image, and how to design a user-friendly interface to meet most users’ need. As a result, we can design a series of retrieval systems, such as plant, bird, butterfly, fish, and national flag, for PDA in one year. With their portable and communication ability, handheld devices offer supplementary ways for students to learn or review the contents of some specified courses anytime and anywhere. According to a report in IThome magazine, more than 13 million PDA (personal digital assistant) were sold in 2002. It is estimated that about 18 million PDA will be sold this year. How to sufficiently use the PDA portability becomes the main concern of our study. To break off people’s habit in using computers, Tablet PC was released in November 2002. Although both PDA and Tablet PC are very convenient to use, the International Conference on Engineering Education

July 21–25, 2003, Valencia, Spain. 1

prices are still an obstacle for most users. However, the wireless environment gradually becomes more mature. Using a handheld device to access the resource in the Internet is a new trend in the information age [3]-[7]. How to construct a wireless learning environment, utilize the Web resource effectively, and then teach students to design a user-friendly interface to meet users’ need become a challenge for engineering education in information technology. To ensure the effectiveness of e-learning, the system must be accessible by any new handheld device. This is the way to reach the goals of lifelong learning and break the boundaries of retrieving knowledge. Besides, the contents of e-learning materials must be updated constantly to attract the students’ attention. Multimedia materials play an important role as supplement for engineering education nowadays. The researchers concentrate on developing new strategies to enhance the effectiveness of e-learning. For example, Tablet PC with the embedded wireless card allows users to connect to Internet in wireless environment and provides an alternative to download or view the e-learning contents. Take our bird retrieval system, which is operated by content-based information retrieval mechanism, for example. We work with the Wild Bird Federation Taiwan. The Federation provides us the information contents about various types of birds. We first identify what technologies should be integrated in our design of mobile learning system. For example, bird classification, feature extraction, and evolutionary computation issues should be carefully studied. We then summarize a constructive methodology and propose the pedagogy goal of learning in the system. A scenario is created to simulate learning bird’s knowledge outdoor. Based on simulation environment, we introduce frameworks and guidelines of instruction strategy as follows: (1) There is a user-friendly interface for users to choose the features on the handheld device screen. After the user clicks one of the major features, the possible sub-features will be popped up with images on the screen. Then, the user can click the most possible pattern as input features. The system allows users to select one or more features as inputs to retrieve the desired patterns. (2) Any bird that meets the query features will automatically show on the screen. The users can click any one of the birds’ names or small images to retrieve full image and brief introduction. The users also can listen to the bird’s sound if they like to. In advance, the users can select to play a short video in our Tablet PC version. It helps users better know the flying style about the bird. (3) Users can have options to input the bird’s partial or full name directly to search the bird. Our system can infer all possible names according to user’s preference from the user profile. (4) The visual effect of the retrieval system is also carefully studied. A system must be designed to attract user’s interest at first glance. The location of each icon should be also carefully allocated. We demonstrated the system during the 2003 Taipei Wild Bird Festival in Taiwan. We guided people how to use our system and showed the query results by short videos. After the users having completed the bird search system, we asked them to answer the questionnaires. We have recall about 40 questionnaires. We summarize the results from questionnaire as follows: 1) Our system provides another learning way to meet users’ requirements by using new technology. 2) It is better to add more contents in the system such that common bird-watchers and experts can find their own need. 3) We port our system on different platforms such as PDA and Tablet PC. The users can choose the devices flexibly according to what they have with them now. 4) The query mechanism that selects bird’s features as inputs is close to the user’s intuition. Users can select the instinct features based on their first glance at a bird. 5) The friendly user interface interacting between the mobile devices and the users is very attractive. 6) The users hope to record more contents related to the bird they have seen. For example, the users want to know the bird’s originality or life behavior. And they can make some notes of that bird. After the demonstration in the bird-watching festival, we received many positive suggestions which in turn helped us implement the life-notes function in our system.

RESEARCH MOTIVATION When we walk in the park, go on a picnic, or even walk down the street, a flying bird passing by us will suddenly attract our attention. The graceful bodies and the harmonic sounds of birds always impact our deep heart. But not all people can recognize the bird immediately and we are puzzled about the bird’s name and information. Most people will refer to a bird sketch or search the bird from Internet to make sure what the bird is. Is there any convenient tool that can help us retrieve the correct bird’s information? When we watch various behaviors of the same bird at different times, is it the only way to mark a footnote on the paper or bird sketch? Can we find another easy way to note the bird’s information? To solve these problems, we utilize the convenient characteristics of PDA and Tablet PC based on the artificial intelligence theory and modern computer technology to fulfill an unsophisticated bird information search system. The user can search a bird by clicking the features according to the bird’s characteristics what they have seen. This instinct selecting mechanism can achieve the efficient retrieval goal. Sometimes, people cannot input exact keywords because they do not know the bird’s real name. The design of our system can solve this problem. Our system uses clicking buttons to select features as inputs. By this method, we International Conference on Engineering Education

July 21–25, 2003, Valencia, Spain. 2

can search the information that we need easily. In this paper, we introduce the bird search system with some advantages such as simple and clear feature selection icons, useful contents, and interactive design modules. The proposed system is very suitable for a teacher or a narrator to do team learning at outdoors. It is also convenient for people to do bird-watching and record birds’ behavior as personal notes.

SYSTEM DESIGN PRINCIPLE A practical system has to take user’s perception into account. It means when we design the bird search system, we pretend to stand on the user’s side. We consider how users refer to a bird sketch and how they record what they have observed. According to the analysis, our system can be more powerful and provide more friendly functions. We ponder some reality situations: 1) Information is insufficient from what the users have seen. 2) The user’s remembrance is very temporary. 3) People will have different key observations about the same bird. We hope to construct a robust system under these impact factors. Hence, we classify the bird’s features as body size, peck shape, flying way, walking way, color, and foot length. The user can select one or some of the features according to what he/she has seen. After the user selecting the features, our system can efficiently search the candidate birds. Except for clicking the features, our system allows users to query by inputting bird’s whole or partial name. For example, if the user knows or guesses a bird named Black-browed Barbet, he/she can use “Black-browed”, “Barbet”, or the combination of bird’s name as input. All the features a user has selected will be displayed on the screen. Our system will pop up all matched results. Besides, users can choose text mode or picture mode to browse the query results. The text mode will display all the names of matched birds. The picture mode displays all the matched birds’ pictures. These two modes can provide users a second chance to verify what they have seen. Users can click on the text or picture that they want to view the bird in details. The system will show a larger image, a short article, build-in scientific name, vulgar name, and the behavior of the query bird. The information can help users understand the birds in more detail. The most inspiration of a bird to human is its sound. Users can record the bird’s sound or download some free sound wave files related to the bird from the websites. Then they can store these sound wave files in specified directories and the system can play the sounds. We achieve the interactive and educative goal. If users are interested in some birds just passing by them, or they want to research that species in long term, our system also provides bird-life notes to allow users memorizing the shortcuts. Users can record what they have observed by handwriting at different times in various places. Users can review, modify, and insert all the information in PDA memory card that they have recorded before. Hence, users do not need to write down information on bird sketch and they can search the contents easily.

SYSTEM DESIGN APPROACH System Design Basis The keyword-based information retrieval method shows its insufficiency in describing multimedia data. The originality of our system is to help users search information in an instinct graphical selection. We provide a graphical selection interface to fulfill an information retrieval system. Users can click the features according to the birds what they have observed outdoors. After they have done the selection, the system will transform the selection into a feature vector which in turn maps to our back-propagation neural network to search the candidate patterns in the database. Our system can achieve the following objectives: (1) Users probably give short of features to the system, because they may remember whole or only partial of the bird’s features they have seen. Our system even allows the “don’t care” conditions for some features. (2) Users can add or delete bird’s information including images and sound files to the system without changing the system model. (3) Sometimes, users may select erroneous features by mistake. Our system still can proceed to find the most possible matched answers to achieve fault tolerance. Feature Encoding Our system extracts the features of birds and classifies those features into some classes. Following, we will classify each feature into detailed items. The encoding items are shown in Table 1. One kind of encoding schemes used in our system is tabulated in Table 2. After encoding those items, we perform back-propagation neural network algorithm to train the given set. When we classify each feature and its detailed items, we can generate a unique feature vector for each bird like Figure 3. And we store this feature vector into database for query.

International Conference on Engineering Education

July 21–25, 2003, Valencia, Spain. 3

Information Retrieval When users have finished selecting features, the system will encode these selected features and either use the trained neural network or execute an intersection operation as (1) to search the desired outcome. If using the intersection operation, our system can proceed to the similarity computation according to the feature vector. T

R = I fi .

(1)

i =1

R : A set of candidates from the selected features. f i : The ith feature clicked by users. T: The number of selected features. Development Environment Our system platforms include Pocket PC, Web Pad, and Tablet PC. And the system is developed in EVB 3.0, EVC 4.0, and Pocket PC 2002 SDK under the IEEE 802.11b environment. The back-end database is established by Microsoft Access 2002. Based on the software used, students can better understand how to integrate them into the most popular handheld devices.

SYSTEM CHARACTERISTICS •

• • •

Handy and Portable: The average weight of PDA is between 120g to 200g. Its size is one fifth of an A4 paper. Compared with most bird sketches, the PDA not only shows its superiority in mobility and functionality, but also is better in keeping minutes than that of most books. Besides, the same designing concepts can be easily applied to Tablet PC and other handheld devices. e-Book Manipulation: The system itself is an e-book. Users can browse all the build-in information by query. After users click features or input keywords, system can search the database efficiently. Compared with traditional sequential search, we provide a more precision and efficient searching method to retrieve specific bird’s information. Multimedia Embedded: With supporting multimedia resources, our system can be easily diversified by integrating such functions. We can utilize this characteristic to improve the interactivity between system and users. Besides, this feature can enrich users’ interests in enhancing their own systems. Highly Extendable: The PDA platform can modify data stored in CF card (Compact Flash) or SD card (Secure Digital). PDA can plug in multimedia devices through CF card interface. For example, we can plug in the CCD camera to take a bird picture and then upload to server through wireless network. When users get somewhere, the system can detect the users’ location via GPS [6]. Then system will automatically transmit related files around the users nearby position. Users can update bird image data or extend information contents by peripheral devices. After updating system contents, users can establish a personal bird search system.

THE SYSTEM OPERATIONS We describe the operations of our bird retrieval system on PDA as follows: (1) Whenever the users click the execution file, our system will randomly pick up a bird image as the initial screen as in Figure 4(a). (2) Clicking the “bird world” icon on the initial screen, users can enter the features selection screen. (3) Users can select one or some features at the right hand side of the screen. Our system allows users to select any possible combination of features. For example, if users want to search birds according to peck shape, there will be four possible selections on the screen as shown in Figure 4(b). (4) Users can proceed retrieving based on one or some of the six basic characteristics. While finish selecting features, users can click the “start searching” icon at the down right corner. The system will pop up the best-matched birds as shown in Figure 4(c). (5) Users can also change to “picture mode” by clicking “picture mode” icon. Each time, our system will show six bird images with their names in this mode as shown in Figure 4(d). And users can browse all the images page-by-page to select possible results. (6) When users recognize the most likely bird image, they can click that bird’s name to view a larger picture. (7) If the bird’s sound file is available, users can click the “sound” icon at the upper left corner. Then the system will play the sound. If the bird’s sound file does not exist, the “sound” icon will be disabled and appear more darker. When users store the bird’s sound file later, the system will enable the icon again. (9) Users can click the “Bird-life Notes” icon to open a bird-life notes window as shown in Figure 4(e). Our system will show the related information about the bird such as the bird’s name and memo time. Users can write down the memo place and what they have observed for future retrieval. After writing the notes into notes window, users can save them. Users can recall the key points by referring to the records. It is very convenient for International Conference on Engineering Education

July 21–25, 2003, Valencia, Spain. 4

users to observe, to analyze valuable information, and to record some species in long tern.

SYSTEM DESIGN GOAL We design this system to help students both in familiar with new information technology and fundamental theory. From the process of developing such a system, we encourage students to discover different approaches to implement the system. During the designing stage, the participant students are more and more familiar with the following topics: (1) The embedded system and simulation programming: We ask our students to try different approaches to all the problems they have encountered. Well-designed serial courses will play an important role in training students on discovering, analyzing, solving problems, and validating the results [1]. (2) Network related technology: The system frameworks include TCP/IP protocol, web server, IEEE 802.11b, and remote control. Students can practice network programming, remote accessing via wireless network, and network devices. When constructing the system, students will study the network integration. It is the basis to learn advanced study of network. (3) Handheld device interface: Our system can be implemented on PDA and Tablet PC. Students can learn the programming skills on heterogeneous platforms. It is helpful for them to quickly find jobs after graduation. (4) Communication between handheld devices and wireless network: The practical system development experience can increase the impression of theory. Students know more about programming of connecting handheld devices with server. (5) Familiar with feature extraction: The search mechanism is based on features comparison. Hence, students will know how to select important features for some objects and the different impact degrees of features. (6) Fundamental theories: The system utilizes back-propagation neural network and other intelligent models to train the given set. Students have to study artificial intelligence related courses. By taking those courses and implementing this system, students can know detailed application fields. (7) Vertical technology integration: We integrate some technologies such as the information retrieval, data mining, and personalized service into our system. After designing the system, students can exactly know the detailed data flow and relationship between different technologies. (8) Cooperation experience: We expect students to learn more practical designing and cooperation experience. Students can not only learn programming skill, but also get hand-on experience from the system integration. Besides, we trained our students to do independent projects, meanwhile they have to cooperate with other team members. (9) Information exchange: By utilizing the new IT technology, students can catch up the trend of new technology and renew their knowledge any time from any place. The system is not only for students to use, all people can carry the easily portable tool with them to retrieve information. Our model can be transferred into different knowledge management system. People can communicate with their director, colleague, subordinate, and family through handheld devices. (10) Personalized database: People can view this system as a personal database. People can access and exchange data via wireless network, browse the information, and query image information. They can synchronize the data with home or office server to achieve data consistency.

CONCLUSION From the practical experience, we proved that our system can work properly both in the virtual scenario and the real world. Our research in the alternative learning path accomplishes in two ways. One is from the system designer’s point of view. We help our students be familiar with heterogeneous platforms of handheld devices, design a system that can access the Internet under a wireless environment, notice the need from user requirements, know how to extract features from an image, and realize how to adjust a system to meet different requirements. The other is from the user’s point of view. We teach people how to utilize the modern information technology to filter out a lot of uninteresting information. By interactively choosing features, the users can learn the knowledge of retrieval systems anytime and anywhere. This learning spirit can encourage them to pursuit lifelong learning through every kind of learning paths. Today, we stand on a quickly changing information age. The traditional teaching and learning methods are not sufficient to support the challenges in the twenty-first century. The new information keeps on impacting us. When we face this situation, how can we overcome this problem? The alternative and flexible learning path is one of the solutions. The new information technology (IT) not only stimulates the innovation of system designers, but also creates a new channel to learn knowledge. Taking the advantages of IT will allow us to teach our private university engineering students having more competence in the changing world.

International Conference on Engineering Education

July 21–25, 2003, Valencia, Spain. 5

FUTURE WORK The network and computer technologies break the bottleneck of spatial limitation. They not only influence our daily life, but also impact our learning ways. In the future, we live in a wireless network environment. The teachers and students have to adjust their attitude toward teaching and learning. We should help students acquire new knowledge and technology from any place at any time. Besides, we have to create a learning scenario by modern information technology. In order to attract students’ interest, the teaching materials have to be vivid, novel, suitable, and easy to carry. The handheld devices combined with wireless network will be the major development trend for industry. Just as the mobile phones, most students will carry handheld devices with them in the near future. Learning motivation can easily arise from using handheld devices as learning tools. It is our goal to develop vivid teaching materials on handheld devices under wireless network. Well-designed courses with the prescribed order can foster students’ interest in learning. We utilize the system to train our students to have more competence in the information age. According to the statistics from IDC and IThome magazine, the growth rate of PDA sales will increase more than 20% from last year. In the information service, the handheld devices can integrate all the functions of mobile phone communication, web service, and remote control. The handheld devices will connect work, home, and entertainments together and provide more convenient services to people. There will be a burst growth in the IA industry. Our goal is to establish a learning environment under wireless network. The core concept of our system is to connect the living, working, learning, and relaxations together. When facing the non-stop changing information technology, we would embed fundamental artificial intelligence technology in our system. This model can be employed into various fields such as information retrieval, data mining, graphics or images searching, and e-commence. We expect our students can catch the development trend of information technology. E-learning is one of our country’s development directions. How can we provide an easily tutorial system? How can we build up a platform for various students to learn new information or to acquire knowledge? How can we inspire students’ innovation? These are the goals we are striving toward.

ACKNOWLEDGEMENT This project is supported by National Science Council, Taiwan, R.O.C. under Grant NSC91-2516-S-036-001 and by Tatung University under Grant B91-I01-024.

REFERENCES [1]

Huang, Y.-P., Ouyang, C.-P., and Chang, Tsun-Wei “Information technology education for private university students: from summer practice to courses design”, Proc. Int. Conf. on Engineering Education, Manchester, UK, Aug. 2002, pp.1-6.

[2]

Huang, Y.-P. and Ouyang, C.-P., “The interaction between private university students and industry: competence building from summer practice”, Proc. Int. Conf. on Engineering Education, Oslo, Norway, August 2001, pp.6B6-8-6B6-13.

[3]

Siegmann, B., “Practical evolution of IT in the college classroom”, 28th Annual Frontiers in Education Conference, Vol. 1, 1998, pp.27.

[4]

Buyukkokten, O., Garci-Molina, H., Paepcke, A., and Winograd, T., “Power browser: efficient web browsing for PDAs”, Proc. Human-Computer Interaction Conference, 2000, pp.430-437.

[5]

L.Yaeger, “Neural networks provide robust character recognition for Newton PDAs”, IEEE Expert, Vol. 11, No. 4, August 1996, pp.10-11.

[6]

Y.-W. Chen; Z.-J. Yan; J.-C. Huang; I.-H. Peng, and J.-W. Zhan “Implementation of a PDA/GPS based development platform and its applications in native education”, IEEE Int. Conf. on Communications, Circuits and Systems and West Sino Expositions, Vol. 2, 2002, pp.1556–1560.

[7]

Yang, J., Chen, X., and Kunz, W., “A PDA-based face recognition system”, IEEE 6th Workshop on Applications of Computer Vision, 2002, pp.19-23.

International Conference on Engineering Education

July 21–25, 2003, Valencia, Spain. 6

FIGURES AND TABLES FIGURE. 1 THE PLANT SEARCH SYSTEM DEVELOPED ON PDA.

(a)

(b)

(c)

(e)

(d)

FIGURE. 2 THE MCDONALDS MEAL-ORDERING SYSTEM DEVELOPED ON WEB PAD.

FIGURE. 3 THE ENCODING SCHEME USED IN OUR SYSTEM.

1

0

1

Body Size

0

0

0

Peck Shape

1

0 Flying Way

0

1

0

1

Walk Way

0

0 Color

0

1

0 Foot Length

FIGURE. 4 THE RETRIEVAL PROCESSES OF THE DESIGNED BIRD RETRIEVAL SYSTEM.

(a)

(b)

(c)

(d)

International Conference on Engineering Education

(e)

July 21–25, 2003, Valencia, Spain. 7

TABLE 1 THE MAJOR FEATURES AND DETAIL ITEMS OF BIRD INFORMATION SEARCHING SYSTEM. Detail Item Major Characteristic Body Size

Item 1

Item 2

Item 3

Item 4

Similar to or Smaller Than a Sparrow

Bigger Than a Sparrow

--

--

Peck Shape

Duck Shape

Long Shape

Hook

Short

Flying Way

Wave

Straight Line

Spiral

--

Walk Way

Jump

Walk

--

--

Color

Darker

Lighter

Red

Green

Foot Length

Long

Short

--

--

TABLE 2 THE FEATURE ENCODING SYSTEM. Body Size

Peck Shape

Flying Way

Walk Way

Color

Foot Length

Larger (1,0)

Duck Shape (1,0,0,0)

Wave (1,0,0)

Jump (1,0)

Darker (1,0,0,0)

Long (1,0)

Smaller (0,1)

Long Shape (0,1,0,0)

Straight Line (0,1,0)

Walk (0,1)

Lighter (0,1,0,0)

Short (0,1)

--

Hook Shape (0,0,1,0)

Spiral (0,0,1)

--

Red (0,0,1,0)

--

--

Short (0,0,0,1)

--

--

Green (0,0,0,1)

--

International Conference on Engineering Education

July 21–25, 2003, Valencia, Spain. 8