Designing a Streamlined Viewport Strategy System to ... - IEEE Xplore

2012 IIAI International Conference on Advanced Applied Informatics

Designing a Streamlined Viewport Strategy System to Enhance Performance in Context Awareness in Mobile Learning Environments Jia Zhang

Huei-Tse Hou

Graduate Institute of Information and Computer Education National Taiwan Normal University Taipei, Taiwan [email protected]

Graduate Institute of Applied Science and Technology National Taiwan University of Science and Technology Taipei, Taiwan [email protected]

mobile learning, situational learning theory is often matched with role-playing teaching strategies. In specified physical environments, learners explore different roles in the expectation that this can assist in the cultivation of knowledge and the ability to resolve problems. These educational activities and programs require context-aware technology to quickly and accurately detect information on the precise position and orientation of learners in physical learning environments, along with the objects on which they are focusing. This information is used to perform adaptive or interactive teaching design. Numerous studies have indicated that in both outdoor and indoor mobile learning activities, the majority of ultimate learning outcomes are influenced by the performance of context awareness systems. When using context awareness systems that are unsuitable or have poor performance in education, the advantages that this technology can bring are not realized; instead, the technology has a negative impact on learning. The accuracy of context awareness systems and their fluency and ease of operation influences the emotions of learners. In addition, the technology can place an additional cognitive burden on learners, thereby decreasing their learning motivation and effectiveness. Some studies have indicated that context awareness systems require the addition of extra operations by learners for positioning, including sensor or context tagging. These systems often mistakenly sense target information due to insufficient sensitivity, complex operations, or slow reactions, frustrating users [4]. Because current context-aware technology has been developed based on desktop computer technology, its operation in mobile devices is often limited because of the relatively poor performance of its hardware. The processors and memory of mobile devices cannot bear the necessary massive calculation loads, leading to system lag, inaccuracy, or errors. Thus, streamlining the complexity of context-aware technology can allow it to be applied smoothly and accurately to education activities. This is a substantial aid to the effectiveness of mobile learning. Context awareness must rely on different positioning technologies to obtain the position and orientation of operators. In the past, positioning technology has mostly relied on technology such as the Global Positioning System (GPS, for long distances outdoors), radio-frequency identification (RFID, for short distances), Quick Response

Abstract—Context-aware technology has been widely applied to every field of mobile learning. However, in the process of context-aware detection, mobile devices commonly produce extra barriers of risk or complexity. In the majority of educational activities, overly complex technology or procedures can have negative effects, preventing numerous technologies that use context awareness from improving learning results. On the contrary, they increase the burden of learners in recognition and operation, or are unable to be precisely operated in the course of ordinary activity because of the high requirements for resources of their complex operating systems. This study presents an innovative Streamlined ViewPort Strategy System (SVSS) architecture aimed at the problems caused by the use of complex technology in the majority of mobile learning activities in the hopes of improving upon the complexity and instability of past technology operation. Through empirical research and testing aimed at performance and usability, this study discovers that the architecture has the potential to effectively increase the satisfaction of learners at every age level with the smoothness and stability of context awareness systems, providing effective location and orientation methods. Keywords-image recognition; mobile learning; context awareness

I.

INTRODUCTION

Context-aware technology has been widely applied to every field of mobile learning. Related technologies, such as GPS, RFID, and QR codes are also becoming increasingly mature. Numerous studies have indicated that the application of context-aware technologies to mobile learning can result in numerous advantages; including strengthening the learning motivations of students, increasing the focus of learners [1] and expanding the vividness of learning materials, thereby increasing the effectiveness of education. Context awareness plays an even more important role in the field of outdoor mobile learning. It is the key to realizing the ubiquitous learning architectures of outdoor education. The majority of studies have confirmed that the application of this technology can improve outdoor learning experiences, assist students in absorbing new knowledge and resolving problems, strengthen their capabilities for creativity and exploration, and increase the stability of their interactions with teachers [2] [3]. Particularly in the field of 978-0-7695-4826-5/12 $26.00 © 2012 IEEE DOI 10.1109/IIAI-AAI.2012.23

Kuo-En Chang Graduate Institute of Information and Computer Education National Taiwan Normal University Taipei, Taiwan [email protected]

72

II.

(QR) codes (for short distances), electronic compasses, and G-sensors. The majority of these technologies are restricted in two aspects: one is that they can only obtain single messages (for example, only position or only orientation), and the other is a restriction in the fields of usage. For example, the accuracy of GPS positioning is affected by possible impediments in weather conditions and location [5], and cannot be used in indoor environments. A tag must first be installed to make use of RFID, leading to higher costs. A corresponding identification label must also be installed for QR codes, which is not conducive to the development and application of authoring tools for mobile learning. In addition, the locations of numerous physical learning environments (such as museums and art galleries) either do not permit extra labels or only allow them temporarily. Electronic compasses and G-sensors are special device accessories that have yet to become widespread among the mobile devices popular in the current market. They are also only able to provide operators with one-sided orientation information, and cannot detect position. Currently, typical methods do not use a single positioning technology as the sole reference for context awareness. Instead, compound architectures are used, such as matching GPS with electronic spaces. Because compound architectures commonly use more awareness hardware, they must be matched with more complex algorithms that integrate several aspects of information, leading to the expenditure of substantial amounts of electricity and resulting in relatively poor operating performance. The majority of currently available mobile devices may be unable to deal with this type of operation smoothly. In recent years, nearly all mobile devices include cameras, and hardware integration has become more mature than in other awareness devices. With superior performance and lower failure rates and power consumption, these devices provide simple solutions for context awareness systems. Therefore, this study employs the image capturing of the cameras in mobile devices to design a Streamlined ViewPort Strategy System (SVSS) architecture specifically aimed at the requirements of mobile learning devices. This architecture uses four simple programming algorithms— grayscale, fuzzy, sampling, and related— along with a streamlined image acquisition process to rapidly compare samples of the authoring tool within the database, and to determine the positions and orientations of learners in real time. This method can quickly acquire the relationship of the positions and orientations between learner and target. In addition, it can more accurately obtain the visual focus of the operator. In contrast with conventional context-aware technology, which can only know that learners are near their targets, SVSS is better able to detect whether learners are facing or focusing on their targets in real time. By using simple correlation coefficient algorithms, the burden on the system is greatly reduced, which allows even low-level mobile devices to operate smoothly, thereby improving the feasibility and universality of educational practices.

MATERIALS AND METHODS

A. Streamlined Viewport Strategy System (SVSS) architecture and workflow The SVSS is used to decrease the load produced by algorithms on mobile devices, reducing the interference that the context-aware technology factors described above have on education. Its primary strategy is to streamline unnecessary external information and remove the additional operations produced by the context-aware technology. The SVSS is composed of three parts: the learner’s client (the mobile device), the authoring tool (Web 2.0), and the database (MySQL) (Fig. 1). Not all of the user data acquired by all of the contextaware components increase positioning accuracy; and even if accuracy is increased, it does not necessarily assist in education. Therefore, the SVSS uses statistical principles to obtain a number of samples from only the pixels of the screen. Comparison of samples within the database can then be made to learn the position and visual angle of the picture, thereby inferring the position and orientation of the operator. In addition to removing superfluous information, simplifying redundant operating procedures avoids additional operations produced by technological problems. It can also effectively decrease the cognitive load of learners, allowing them to focus on the learning target without being restricted by the operational complexity of the

machine. Figure 1. The SVSS system architecture

B. Authoring Tool In this system, educators do not have to go to the actual sites to install tags or perform software and hardware construction. They only need to upload images of targets or of target views on the web interface of the authoring tool, and define the position and orientation information represented when the camera acquires similar images. The SVSS can then conduct sampling and establish a comparison file in the SQL database. C. Student Client The operation of the student client makes use of intuitive response. The camera on the handheld mobile device is aligned toward the front. The SVSS obtains the forward field of view, and determines in real time whether it contains an image similar to the learning target, thereby judging whether the learner has found the correct target. Learners do not have to take photographs, record video, or

73

0.15, it indicates that the currently acquired image has a higher degree of similarity with Targets D and E (noted with *). The R values of Targets D and E are then compared. The higher value is the final correct target. In general, unless the operating visual field of the learner entirely accords with the angle during sampling, R values seldom exceed 0.50.

touch or sense any tags, substantially eliminating operations that are not concerned with the goal of learning from the mobile device. D. Streamlined Wireless Feature After SVSS processing of the images, the files for comparison of each target have a length of approximately 862 bytes. After logging into the system on the student client, all of the target comparison files of the educational activity are downloaded. Thus, when conducting future teaching activities, students do not have to remain connected to the Internet. The SVSS design can greatly decrease the system’s reliance on wireless connections, while reducing the barriers of risk they produce (Fig. 2).

TABLE I. R(CORRELATION) VALUE FROM SVSS Object R(0.15)

(B,C) level = ड़ॡ१ ቀ

ቁ

ቁ

Figure 3. fuzzy procedure

In the third step, calculation of the correlation coefficient is conducted on the sample value arrays and the comparison file arrays within the database (Equation 2).  Luminance=0.299×R+0.587×G+0.114×B  R(Correlation)=

σ൫x-x൯(y-y) ටσ (x-x)2 ට(y-y)2

Target D 0.48*

Target ˘ 0.29*

H. Experiment Protocol This study performed testing and verification on performance and usability to verify whether the SVSS could operate effectively. 1) Performance: To verify the identification ability of the SVSS and its range of limitations, this experiment randomly constructed 42 sampling targets (21 indoor targets and 21 outdoor targets) and performed reconstitution error and limit tests. The reconstitution error test requested that subjects use the SVSS in activities at the 42 sampling targets to verify whether it could correctly identify location and orientation. The subjects were prompted on the features of each location and were then free to seek it themselves. Each time they successfully found a target location (SVSS identification success), the system then prompted them on the characteristics of the next target location. Because the goal of this test was to determine whether the SVSS could successfully perform identification under the operations of different subjects, the prompts on characteristics were relatively clear and simple. The limit test recorded the correlation between the R value of the camera and the deviation angle in the SVSS when facing a fixed interior target. The results of the test revealed the permitted error value and the greatest

ሺ୅ൈଵ଺ሻାሺ୆ൈ଼ሻ

ଽ

Target C 0.11

G. Participants A total of 19 people were sampled via purposive sampling for participation in this study. Within the sample, 4 participants (2 male, 2 female) were 20 years old or older. There were 15 5th grade elementary school students between the ages of 12 and 13, 4 of whom were female. The four adults all had two or more years of experience with smart phones, and had used or were currently using tablet computers. One of the adults was a graduate student in a field of information technology, whereas the remaining three were elementary school teachers. All of the participating elementary school students had experience using smart phones or tablet computers.

E. Algorithm After the SVSS acquires images, four procedures are performed: grayscale, fuzzy, sampling, and related. In the first step, grayscale processing is performing on the acquired image, obtaining luminance (Equation 1). In the second step, the geometric mean of the pixels in the neighboring 5*5 rectangular range is obtained, acquiring fuzzy values to act as samples (Fig. 3).

ଶସ ሺ୆ൈ଼ሻାሺେൈଵሻ

Target B -0.12

F. System Requirements This system was implemented on the Android 2.1 operating system. It can run on any mobile device or embedded platform that uses this operating system. A physical camera is necessary in the hardware. CPU performance should be at least 528 MHz (ARM), with 512 MB or more of memory space.

Figure 2. Streamlined wireless feature

(A,B) level = ड़ॡ१ ቀ

Target A 0.05

(1) (2)

For every image acquired in real time, the SVSS calculates the R value with each comparison file in the database. It then takes the greatest R value and those values that exceed the effective recognition threshold. As shown by the examples in Table 1, when the effective value is set to

74

restrictions that can be borne under the individual operating factors of the learners when identifying targets with the SVSS. The limit test experiment set a fixed table 140 cm in height and a directional ruler at a position 100 cm in front of the target location (Fig. 4). The tablet computer was then turned horizontally and vertically from 0° to 15° while changes in R value were recorded.

excessively large error in the acquired image. In addition, after performing a non-parametric test (Mann-Whitney U test), no significant discrepancies existed between adults and children in identification results (p = 0.077 > 0.05), indicating that the SVSS also has a certain degree of applicability across age levels. TABLE II. RECONSTITUTION ERROR TEST RESULT

Indoors (age 20+) Outdoors (age 20+) Indoors (age 12-13) Outdoors (age 12-13) Indoors (all) Outdoor (all)

Figure 4. Reconstitution error and limit tests

2) Usability: This experiment gave procedural tasks to the 19 volunteers during the reconstitution error test. After the experiment concluded, open interviews were held to allow the subjects to express their opinions and experiences with each task throughout the process. The experimental process and the questions asked are shown in the figure below:

Number of People

Identification Errors(Avg.)

Correct Ratio

Standard Deviation

Ͷ

ʹǤʹͷ

ͺͻǤʹͻΨ

͵Ǥͺ͸

Ͷ

ͲǤͷͲ

ͻ͹Ǥ͸ʹΨ

ͲǤͷͺ

ͳͷ

ͷǤʹ͹

͹ʹǤͻʹΨ

ͷǤ͵ͷ

ͳͷ

ʹǤ͸͹

ͺ͹Ǥ͵ͲΨ

ͶǤͳʹ

ͳͻ ͳͻ

ͶǤ͸͵ ʹǤʹͳ

͹͹ǤͻͶΨ ͺͻǤͶ͹Ψ

ͷǤͳ͵ ͵Ǥ͹ͷ

B. Limit Test The SVSS identification technology calculates a relationship value R for the real-time acquired images and each comparison file in the database. This R values represents the degree of similarity between the current image and the target in the database. In general, because only a minority of the features of different images are identical, their R values are distributed at approximately 0.15 and below. Thus, pictures with R values greater than this value represent the effective identification of the target’s features. The limit test reveals that the range of effective identification is within a horizontal angle of 10° and a vertical angle of 5°. When exceeding this range, though some of the image of the target is retained, the correlation coefficient is unable to make significant distinctions. TABLE III. Horizontal Degree 15 10 5 0 Mean R Value

Figure 5. Usability testing process

III.

RESULT

A. Reconstitution Error Test The results of the reconstitution error test indicate that in ordinary conditions, the accuracy of SVSS identification exceeds 70 %. The results of using a non-parameter test (Wilcoxon two-sample test) on all of the participants indicate that indoor identification results were significantly superior to outdoor identification results (p = 0.007 < 0.05). The reason for this could be the comparatively lower variability of indoor environments, and that the camera is typically closer to the target. Thus, when there are slight discrepancies in the positions and angles of different operators facing the target, changes in R values drop quickly, leading to a loss of identification accuracy. Outdoors, there is frequently a substantial distance from the target. Therefore, slight artificial factors do not lead to an

LIMIT TEST RESULT 

Mean R Value

0.04 0.21 0.40 0.94

-0.04 0.05 0.18 0.32

-0.24 -0.15 0.01 0.09

-0.17 0.01 -0.06 0.04

0

5

10

15

Vertical Degree

C. Usability Opinions on each stage were collected from the open feedback of the subjects. Positive feelings emerged regarding satisfaction with speed, and personal curiosity toward the new technological product. Negative opinions existed toward the operation of the touch screen, design flaws in the system UI, and weight problems in the hardware device itself. Overall, the participants held positive opinions a total of 55 times during the interviews, the majority of which came from satisfaction with the feeling of smoothness in the system. Negative opinions were presented a total of 24 times. These primarily concerned obstacles created by the non-context-aware technology. These data indicate that the application of the

75

awareness using RFID and symbology tags,” Journal of Educational Multimedia and Hypermedia, 2007, Vol.16, No.4, pp.411-428.

SVSS in context-aware technology for educational fields can effectively improve the smoothness and performance of systems. None of the subjects experienced crashes or system lag during the testing process, and they largely gave extremely positive assessments of the smoothness of the system. Of the 19 subjects, only 3 suffered from poor identification rates due to excessively larger differences between the target field and the sampling angle caused by height factors. This problem could be improved in future studies by using algorithms. IV.

DISCUSSION & CONCLUSION

The SVSS strategy developed in this study can provide mobile learning plans which demand context-aware technology with a rapid and stable alternative. In comparison to other context-aware technologies, the SVSS produces higher satisfaction with performance and system smoothness among users, and has lower system requirements (only requiring a camera) and more accurate positioning technology that is unrestricted by location (including both orientation and position). In education fields that apply context-aware technology, we anticipate that the SVSS can significantly improve negative impacts caused by technological factors, and reduce the additional cognitive burden created on learners because of equipment failures. Future studies can improve the identification success rate of the algorithm, thereby eliminating the inaccuracy that can result from height factors, and use the evidence in this study to compare the gap in additional cognitive burden on learners created by this technology versus other contextaware technologies. ACKNOWLEDGEMENTS This research was supported by the projects from the National Science Council, Republic of China, under contract number NSC-100-2628-S-011-001-MY4, NSC-100-3113-S011-001, NSC-100-2631-S-011-002 and NSC -99-2511-S011-007-MY3. REFERENCES [1]

[2]

[3]

[4]

[5]

L. Mifsud, “Alternative learning arenas-pedagogical challenges to mobile learning technology in education,” in Proc. IEEE Int. Workshop Wireless Mobile Technologies in Education, Aug. 2002, pp. 112–116. T. Y. Liu, T. H. Tan, and Y. L. Chu. “Outdoor natural science learning with an RFID-Supported immersive ubiquitous learning environment,” Journal of Educational Technology & Society,2009, Vol.12, No. 4, pp.161-175. J. Zhang, Y.T. Sung, and K.E. Chang.” Using a Mobile Digital Armillary Sphere (MDAS) in astronomical observation for primary school students,” in Proc. World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, Oct. 2011, pp. 2632-2641. Chesapeake, VA: AACE. Available: http://www.editlib.org/p/39128. J. Zhang. “Using a Mobile Digital Armillary Sphere (MDAS) in astronomical observation for primary school students,” M.S. thesis, Dept. Graduate Inst. of Infor. and Comput. Educ., National Taiwan Normal Univ., Taipei, Taiwan, 1992. N. Osawa, K. Noda, S. Tsukagoshi, Y. Noma , A. Ando, T. Shibuya, K. Kondo, “Outdoor education support system with location

76