From Multimedia Micro-University to Macro University

From Multimedia Micro-University to Macro University and Beyond Timothy Arndt and Angela Guercioa Department of Electrical Engineering and Computer Science and Department of Information Systems, Cleveland State University, Cleveland, Ohio 44115, USA a

Department of Computer Science, Kent State University-Stark, North Canton, Ohio 44720, USA

Abstract For this special issue of the Journal of Visual Languages and Computing in honor of Prof. Shi-Kuo Chang we review Prof. Chang’s research contributions to the field of education. Always visionary and inspirational, these works reflect Prof. Chang’s studentcentered approach and his interest in building relationships among international educators in order to bring the benefits of such collaborations among educators to the students while keeping in mind the motto “small is beautiful”. Keywords: distance transformations

education;

multimedia

learning;

educational

materials

1. Introduction Prof. Shi-Kuo Chang is best known to the research community for his contributions to the fields of visual language and image database systems – including starting the first scientific conference on visual languages and founding the Journal of Visual Languages and Computing. He has made major contributions to many other areas however, and, in particular, as part of his long career as an educator – more than 35 years as a professor of computer science in U.S. universities including terms of service as Chair of department at both Illinois Institute of Technology and the University of Pittsburgh – education has long been an area of interest. He has performed research with education as the central topic as well as applying his research results from other areas in the education domain. Those of us who have been his students and collaborators know Prof. Chang’s passion for education and we might point to his founding of the Knowledge Systems Institute (KSI) as an institution for teaching graduate level computer science using the principles he holds dear – the personal touch and the introduction of students to cuttingedge research, encouraging them to get involved in the same. In looking at the corpus of Prof. Chang’s research in the field of education, a couple of themes shine through – first enabling the means to build international collaborations in teaching and research which will benefit students by exposing them to multiple points of view and emphases from international experts in their chosen field of

1

specialization. This theme, which was evident more than 30 years ago when Prof. Chang brought together a group of researchers from many countries to collaborate in research in visual languages at the early visual languages conferences, is increasingly important in todays diverse and globalized education environment. Second, while wanting to leverage the possibilities brought about by the rise of the Internet and distance education, Prof. Chang has always been concerned not to lose sight of the individual student in this online environment, thus providing a customized and personalized learning experience is necessary in order that the student not be overwhelmed and in order to stick to Prof. Chang’s education motto “Small is beautiful”. Of course, as one of the pioneers of research in the area of visual languages, the themes of visual interfaces, multimedia, and visual languages and grammars are all interwoven in the research in learning carried out by Prof. Chang. In the rest of this paper we will provide an overview (not exhaustive) of Prof. Chang’s research work in the education area, at times indicating how subsequent developments have vindicated the previous research, while at other times technology has driven practice in somewhat different directions, we will show how the parts fit together, and we will conclude with suggestions for further research in this area inspired by Prof. Chang’s work. 2. Multimedia Micro-University From his earliest research, Prof. Chang has shown in an interest in how computer technology can improve education (two of these early efforts are [1] and [2]). The first of his education research efforts in the modern Internet age concerned what he termed a Multimedia Micro-University (MMU) [3]. A micro-university is a teaching or learning environment for an individual teacher/student, designed to optimally serve the needs of that teacher or student. This contrasts with a virtual university which consists of one or more MMUs and embodies a collection of personalized learning environments for a single student interacting with a (virtual) teacher. In the extreme case a virtual university is the same as a single microuniversity, but usually a virtual university contains many micro-universities and performs many administrative functions supported by a virtual administration office. The administrative tools include software tools for managing users, curriculum, facilities, resources, plans, etc. Each of the above tools can be used by authorized users. MMUs can be small, medium or large ranging from a single software module to a complex configuration of software modules. Example MMUs include: Virtual Courseroom, Virtual Collaboration Room, Virtual Laboratory, Virtual Library and Virtual Private Office, which operate on a dynamically changing collection of tele-action objects (TAOs) [11]. The multimedia micro-university was developed at the Knowledge Systems Institute (KSI) which was founded by Prof. Chang in 1978 to support research and professional education. KSI maintains a limited enrollment and offers courses on evenings and weekends and through distance learning. These characteristics influenced the design of the micro-university to support KSI’s education program and management operations. The components of the multimedia micro-university consisted of a virtual library, an intelligent distance learning system and visualization and planning tools for

2

the school’s operations. The virtual library was implemented with a WYSIWYG paradigm and supported both keyword-based retrieval and visualization of the titles in the (physical) KSI library. The intelligent distance learning system is a complex finite-state machine which uses a knowledge base to model the educational process. It serves as an intelligent communication channel between the author of distance learning materials and the cooperative recipient. Different user roles have different interpretations of the system states, and the system states correspond to four application programs – the Tutor, the Advisor, the Publisher, and the Administrator. The Tutor program provides the course material to the student and then assesses mastery by means of online quizzes. The Advisor facilitates communication between student and teacher. The Administrator mediates and facilitates the logistics of the education process for administrators. The Publisher moves learning materials to a server, updating configuration tables along the way. The KSI distance learning system was supplemented with multimedia e-books incorporating multimedia elements such as audio and video as well as interactivity. The administration functions for KSI were facilitated by multimedia microuniversities as well, in the form of visualization tools which give an overview of the school’s current status. The whole system is developed using HTML, Java applets and CGI programs. Planning support is provided through a structured collection of knowledge tables [4] representing goals, plans, expected outcomes, status, and related plans. The knowledge tables are continuously created, updated, evaluated, and revised by key administrators. A tool called the Knowledge Table Editor supports knowledge-tablebased planning and evaluation. The micro-university was designed, implemented and evaluated by careful measurement of measurement of graduate student academic achievement. This achievement was measured quantitatively as part of a preliminary assessment which demonstrated that the micro-university components already implemented contributed to the learning process at KSI. It can be seen that much of the functionality of the multimedia micro-university has been implemented in Learning Management Systems (LMSs) such as Blackboard Learn and Moodle which have been developed since the research described in this section was carried out. However, it can also be seen that several features of the multimedia micro-university concept are either lacking or implemented in different manner in existing LMSs. First, the administrative functions of the the micro-university are often incorporated in separate software tools, while in the research described they are developed on the same micro-university paradigm. The research also incorporates knowledge through the TAO and knowledge table paradigms which allows for personalized learning experiences which supports the “Small is Beautiful” paradigm which is often missing in today’s gigantic online distance learning classes. The incorporated knowledge structures also support planning in a unified and organic manner. 3. Macro University Following the successful introduction of the multimedia micro-university, the vision was extended from a single institution to a federation of universities in order to

3

provide the best opportunities for learning and research for students by exposing them to an international array of experts. Technical and financial considerations in the late 1990s led many academic institutions to set up a web site and start a distance learning program on the Internet. Coupled with a digital library, many an academic institutions experimented with the concept of a virtual university. In some countries the local government takes the initiative to form a consortium of universities offering online courses from each institution. These consortia are sometimes called a virtual university or virtual campus. Thus the term “virtual university” sometimes means the distance learning courses of a single academic program, at other times to the distance learning program of an academic institution, and still others to a consortium of distance learning programs. Establishment of such programs can give administrators of academic institutions unrealistic expectation. Some administrators believe once they have a virtual university, they can greatly reduce the number of instructors and thus become more cost effective. However, they fail to realize the importance of providing an effective learning environment on the Internet, which is not an easy task. Just hosting some courses on the Internet does not provide a learning environment. The students can easily get confused and disoriented, if left alone on the Internet. Beneath the virtual university, there needs to be another layer, offering a student a personalized and personable learning environment. This is where the previously developed concept of the multimedia micro-university becomes important. At the same time, many visionaries have hinted at the possibility of combining the educational resources of a large number of academic institutions, thus creating a super rich learning environment. Without losing sight of the individual student’s microuniverse, it is hoped that the coupling and coalition of numerous academic institutions will constitute a macro-universe for the students. Based on these concepts, Chang and his colleagues introduced a new framework for distance learning consisting of three layers: the micro-university, the virtual university, and lastly the macro university. A micro-university, as described in the previous section, is a self-contained learning environment, usually on a single PC (or notebook, tablet, smartphone, etc.), for a single student interacting with a (virtual) teacher. A micro-university is designed to serve a single student, but many instances of the same micro-university can be instantiated to serve many students. A virtual university is a collection of personalized learning environments so that students can engage in learning activities from home, remote locations, etc. A virtual university is usually owned and operated by an academic institution and therefore has more administrative functions than a micro-university. The Macro University was developed to be a framework such that multiple virtual universities could pool their resources together, thus creating a very rich learning environment for a large number of students worldwide [6]. Macro University was an international consortium of Virtual Universities. As such there was a wide variety of computational platforms used by students of Macro University. In order to minimize the problems associated with exchange of data in a heterogeneous computing environment, an open standard for information exchange in the Macro University framework was required. An emerging standard (which has since become well established) for Electronic Data Interchange (EDI) — XML, eXtensible Markup Language was chosen. Virtual

4

Universities are structured as a collection of MMUs where each MMU is a collection of TAOs. TAOs are described by TAOML [5e], and TAOML is itself an XML application. In Macro University the TAOs represent the salient objects for a distance learning environment: class slides; recorded and live lectures; exams; syllabi; recorded and live demos; etc. Each of these objects has its own characteristics which need to be described in order to present the TAO at the student’s computer. The formal specification of objects is based upon tele-action objects (TAOs). Each tele-action object is a dual representation (G, K) consisting of a hypergraph structure G and a knowledge structure K [11]. The hypergraph G is used to describe the connections and relations between the sub-TAOs, and the knowledge structure K the actions and how to synchronize the actions. A formal model called the index cell that combines the desirable features of finite state diagram and Petri net is used to specify the knowledge structure. The presentation of the multimedia document, the hypergraph component and the knowledge component are specified using a Tele-action Object Markup Language (TAOML) [12]. A multimedia development environment called MICE (Multimedia IC developer’s Environment) supports the prototyping of multimedia applications using formal specification based upon TAOs. The front-end to this environment is a visual specification tool that allows the user to specify a TAO in a userfriendly way and that automatically converts the specification to TAOML. TAOML was defined as an XML application. A DTD (Document Type Definition) contains the elements and the attributes necessary for the specification of a multimedia system described via TAOs. At least eleven universities and corporations participated in the Macro-Univerisity experiment in distance education. It is interesting to note that since the Macro-University was described around the year 2000, a standard for LMS materials based on XML has been developed – SCORM (Sharable Content Object Reference Model) – as an initiative of the Office of the United States Secretary of Defense [13]. This validates the choice of Macro-University to use an XML for content interchange, although the TAOML described above is in many ways more advanced than the SCORM standard currently in widespread use. It would be interesting to see if a mapping of (at least a portion) of TAOML to and from SCORM could be defined. The Experience API [14] is a more recent standard which is a possible successor to SCORM and which offers greater functionality. The Experience API is a web service that uses JavaScript Object Notation (JSON) for its data format. 3.1 Sentient Map for Macro University One of the problems raised by distributed e-learning environments such as Macro University is that the user may be unaware of the location and/or existence of instructional resources and information items which may be stored in heterogeneous databases and data sources. This makes it difficult for a novice user to formulate spatial/temporal queries in order to locate the required information items. To overcome those difficulties, the use of the sentient map is proposed [7]. The sentient map provides a uniform presentation of real space/time or virtual space/time which allows the user to interact via the same set of gestures in either context [8]. The search requests communicated by those gestures are then transformed

5

automatically into queries targeted at a specific information space. The retrieved data is then also automatically transformed according to the user’s current interest. A sentient map has a type (geographical map, directory page, web page, etc.), a profile consisting of a set of creator-specified attributes, a visual appearance defined by a visual sentence defined by a visual grammar [9], and a collection of index cells [10]. The combination of index cell and visual appearance defines a tele-action object (TAO) [11]. A set of gestures is defined to create a gesture-based interface for interaction. The gestures are then transformed into an intermediate representation known as a σ-query for spatial/temporal information retrieval. The exact σ-query generated by a particular gesture query depends on the data model or models being used. A prototype sentient map system for a virtual conference room was developed at Carnegie Mellon University. Users are represented by avatars in the virtual conference room which has a sentient map on the wall. There are three different possible maps that can be chosen. The avatars formulate gesture queries against the sentient map which are transformed to the intermediate and then final query results. The prototype system was also applied to the Macro University project. The avatars in this context represent students and instructors and gesture queries can be used to query e-learning courses offered anywhere in the world by a member of the Macro University project. A preliminary user study was carried out with a small group of users who confirmed the ease of use of the sentient map paradigm but requested more expressive gesture queries as well as better feedback in the form of SQL queries. This study was carried out before hardware for motion/gesture recognition such as Microsoft Kinect was widely available, so even better results might be found with a present-day implementation. 4. The Growing Book The Macro University project was a model for combining the educational resources from a large number of academic institutions, thus creating a rich learning environment. Without losing sight of the individual student's needs, it was hoped that the coupling and coalition of academic institutions would constitute an ideal learning environment for the students. However, to provide an effective distributed learning environment through a consortium of institutions raises many technical problems. Just offering a set of courses on the Internet does not provide an effective distributed learning environment. The students can easily get confused and disoriented, if left alone on the Internet (some ideas on using the concept of design patterns from object-oriented software engineering in order to provide a standard way for students to interact with a distance learning course in order to avoid this disorientation are discussed in the study reported in [5]). Beneath the virtual university, there needs to be another layer, offering the students a personalized and personable learning environment. These considerations led to the Growing Book project [15][19]. A Growing Book is an electronic book co-developed by a group of international instructors who collaborate in teaching and research through the Macro University. Since the learning materials they use are constantly evolving, the Growing Book must be frequently updated and expanded, leading to its name. The Growing Book is used by each teacher both locally and in the distance learning environment. The various chapters of the Growing Book are owned by different instructors who may utilize and/or provide different tools for distance

6

learning, self learning and assessment (note the difference between this ‘owner’ concept and the related Wiki concept in which any user may edit and/or modify contenet). The Growing Book was to be accessed in the distributed learning environment by learners with diverse linguistic skills, cultural backgrounds and perceptual preferences, and also can be used for teaching and research. The structure of the Macro University incorporating the Growing Book is shown in figure 1. Macro University consists of a federation of virtual universities where each virtual university contains one or more micro-universities. The micro-university can be a Growing Book. A virtual university can also perform many complex administrative functions supported by a virtual administrative office.

Figure 1 – Architecture of the Macro University The objectives of developing the Growing Book were to: 1) share resources in developing distance learning materials, 2) share experiences through the teaching of a common distance learning course, 3) test and evaluate the distance learning and/or administrative tools, and 4) discover problems and possible solutions in distance learning. The Growing Book supported multi-level, multi-lingual and multi-modal usage of common learning materials. (a) Multi-Level Usage: The same learning materials can be used for courses at various levels (introductory, advanced undergraduate, graduate, etc.). Therefore, multi-level usage is necessary for the Growing Book. Multi-level usage of the Growing Book, is supported by a formal specification of the type of objects to be managed thrtough the tele-action object formalism [5d] – the same formalism used in developing the multimedia micro-university. This also facilitates multilingual and multi-modal usage of the Growing Book. (b) Multi-Lingual Usage: The same learning materials can be transformed into different languages so that the presentation of the Growing Book is multi-lingual. Language translation functions implemented by commercial language translators associated with the tele-action objects are used to transform the tele-action objects into different languages.

7

(c) Multi-Modal Usage: The same learning materials can be used by physically challenged people or people with different perceptual preferences, so that the presentation of the Growing Book is multi-modal. Perceptual translation functions transform the tele-action objects into different media. The focus is on the visualization of tele-action objects and the development of gesture-oriented interface because it can be used by people with hearing disabilities and shared by people with different languages and cultures. The prototype Growing Book was intended as a textbook for an undergraduate course on data structures and algorithms. The prototype Growing Book had the following characteristics: 1) learning materials accessible with a browser or customized interface with common look-and-feel and common buttons, 2) common assessment tools, 3) individual tools downloadable as plug-ins, 4) common programming examples, 5) adaptive learning with embedded audio/video clips. In the Growing Book project, the student feels himself to be actively driving his course by requesting explanations, special documents, homework corrections, etc. In this sense interactivity is a basic issue in the Growing Book model. Interactivity is emphasized in managing all the different type of documents (images, text, video clips, audio, etc.), reflecting a teaching/learning communication model. Students utilize an interactive language, or a series of communication tools (triggered as events on specific parts of the multimedia documents) allowing them to interact and communicate in a variety of ways. There is a richer learning experience with respect to the standard series of documents as, for example, when a new view is obtained by the processing of a query in a relational database system. In order to support multi-level, multi-lingual and multi-modal usage of the Growing Book and to facilitate the exchange of information in a heterogeneous computing environment, a formal specification of objects and an open standard for information exchange are required. Students, teachers and authors all need an interactive language built upon some basic operations (triggered as events on specific and specified parts of the multimedia documents) that simplify their response with respect to novel concepts in a given course, retrieving extra documents from virtual libraries, or simply communicating with other students and/or other teachers. In this way there may be an enrichment with respect to a standard series of documents as when a new view is obtained with respect to the processing of a query in a given database management system. The basic operations defined on the Growing Book are divided into several groups. a) Operations for Multi-level, Multimedia Customization: The first set of operations supports matching, abstraction, weaving and customization of multimedia documents. These are called MAWC operations [16]. b) Operations for Increasing/Updating Awareness: The user can specify an awareness vector, so that he/she can be informed about certain events. The awareness vector is a binary vector where each entry indicates the absence/presence of an awareness attribute. A user can also set privacy, so that he is not included in any awareness information activity. c) Operations for Communication: Communication operations are for sending messages to authors, instructors and students. A user may not know their exact names and/or e-mail addresses, but he can still send messages to the group of people he wants to communicate with. d) Operations for

8

Watermarking: Watermarks can be added to or displayed for a multimedia document. e) Operations for Managing the Growing Book: There are many operations for gathering statistics about and managing the Growing Book. The Growing Book operations can be seen as commands for the customized IC (Index Cell) Manager (described in [10]) of the Growing Book. When the user submits a command consistsing of a name and its parameters to the Growing Book, the customized IC Manager processes this command. The name is treated as a message type by the IC Manager to be passed on with the parameters to the appropriate IC for processing. The Growing Book operations are implemented as "actions" (implemented as C programs) of the ICs managed by the IC Manager. The Growing Book project resulted in a collaboratively developed textbook on data structures and algorithms which was used by several members of the Macro University consortium. A hard copy of the resulting text was also published [17]. Given modern day technologies, it is easy to imagine incorporating Wiki technology [18] into the Growing Book project. 5. Transformation of Distance Learning Materials The previous section described a Growing Book which is created by an international team or teachers and which has the goal of being multi-level, multi-lingual, and multi-modal – in other words, the Growing Book should support rich personalization for the needs of particular students. The MAWC operations on the Growing Book are the means to achieve this customization and can be implemented as commands for the customized IC manager. In this section we look at how those operations by the IC manager on the Tele-Action Objects (TAOs) which make up the Growing Book can be implemented [20]. Since micro-universities are just a generalization of the Growing Book, all of the transformation techniques described in this section apply to MMUs as well. Interoperability of diverse hardware and software platforms is one of the most pressing needs in computing today. This problem is of particular relevance in academic environments where the software systems and materials used for teaching are extremely varied. Almost every university or school nowadays has a distance learning platform and course materials developed by individual instructors. These may be regarded as “legacy” materials since, for the most part, they are unusable outside the context in which they were developed due to hardware and software dependencies. Federations such as Macro University using techniques such as the Growing Book require exchanged documents to be reformatted, different databases to be queried and the results of the queries sent over the network in a format independent of that of the databases queried, among other local differences. The instructor needs to be able to construct some virtual containers by composing resources that were not necessarily created by him but possibly by other colleagues and which are available on the network and reusable. The use of browsers and the Internet only appears to have furnished interoperability for legacy and newly created didactic software systems since on the one hand the limited flexibility of the HTML language (and in particular its tags) doesn’t

9

allow for the creation of easily manipulated structured documents and on the other hand there still exist various hardware platforms and operating systems. The student in the Macro University, for his part, often asks very simple questions of the type: “Who can offer me the most up-to-date C++ course?”, or “Where can I find an exhaustive explanation of Laplace transforms?”, or “Who has solutions for exercises on the subject of cinematography?”. It is true that sophisticated techniques to identify keywords in web documents have been developed for search engines, but none of these are able to satisfy the student’s needs. For this reason, a new generation of markup languages with tags able to describe the contents of the document itself have been developed. The concept of information that describes some other information is known as metadata. In the rest of this section we will describe a methodology for distance learning based on the transformation of TAOs. The Growing Book is intended to support multi-level, multi-lingual and multimodal usage of shared learning material. This goal is fully supported by the decision to use XML to represent the TAOs comprising the Growing Book since the XML approach separates content from presentation. The Growing Book can be customized using a Teleaction Object Transformer based on XML technology by applying a ComponentBased Multimedia Software Engineering Approach resulting in a complete process of dataflow transformation that presents the Growing Book in different ways and at different levels of abstraction. The Teleaction Objects comprising the Growing Book can also be transformed to conform to summary and awareness information collected during a distance learning session. The Teleaction Object Transformer was implemented using the XSLT transformation language, XSL style sheets and the SAX API Java parser. In the context of this research, the output of the Teleaction Object Transformer is a Sentient Map (previously described in section 3.1). The main idea of the sentient map [8] is to present all kinds of objects visually in a virtual map that senses the user’s input gestures and reacts by retrieving and presenting the appropriate information. We use the term “map” here in the general sense. Geographical maps, directory pages, list of 3D models, web pages, documents, slides, images, video clips, etc. are all considered maps, as they all may serve as indexes and lead the user to more information. In practice, a sentient map is a gesture-enhanced interface for an information system. In advanced distance learning applications, students and instructors can use the sentient map environment in a virtual course-room. When the user, or the user’s surrogate (the user’s avatar), points at the sentient map on the wall, more information becomes available and is also visible to all the participants in the virtual course-room. Two scenarios have been identified corresponding to the usage mode of the dataflow transformation: stand-alone or distributed. In the stand-alone scenario, the main dataflow transformation process is local. The stand-alone compute platform loads the Teleaction Object Transformer and delivers a combination of the style sheet and the Growing Book information to be transformed on the same platform. The results are various media formatted as requested by the user (e.g. video format, audio format, text format, etc.) and transformed in accordance with the collected summary and awareness information. In the distributed scenario the dataflow transformation process is performed on a server which provides a transformation process to the clients that require it. The client forwards the TAOML document to the Teleaction Object Transformer or an equivalent generic dataflow; it also requests a data transformation and specifies the

10

desired output data format. The server loads the Teleaction Object Transformer and delivers a combination of the style sheet and the source data to be transformed to the recipient's platform. In the distributed scenario it is possible to use a style sheet document and Teleaction Object Transformer transformation not only for receiving a desired presentation format, but also possibly to receive a data flow properly formatted for some other mobile device such as a tablet or smartphone. The dataflow transformation process is illustrated in figure 2. Note that the main components of the Teleaction Object Transformer are shown inside of the Transformer itself. Another advantage of the use of XML is also illustrated in the figure - we have adopted the IEEE LOM (Learning Object Metadata) [21] which is a standard for metadata for learning objects such as the metadata format for TAOs. This makes the approach interoperable with other distance learning systems and allows the reuse of learning resources. The process for generic TAOML-based data stream manipulation is principally composed of two main transformations: • Transformation from a generic data format (not necessarily hierarchically organized) to a well-formed XML/TAOML format. • Transformation from XML/TAOML format to a document format (e.g. text, Microsoft Word, PDF, html etc.) or a media format (e.g. audio, video).

Awareness TAOs

TAOML Distance Learning Materials

Summary Information

SESSION SUMMARIZER

TELEACTION OBJECT TRANSFORMER TAOML DTD

Distance Learning Materials w. IEEE

Import/Export via IEEE LOM Metadata SAX Java Parser

Tablet XSLT Processor Web XSLT Processor Smartphone XSLT Processor

Tablet Stylesheet Web Stylesheet Smartphone Stylesheet

MAWC Operations

Sentient Map for Web/smartphone/tablet etc.

Figure 2 – Dataflow Transformation for the Growing Book 11

The prototype developed reflects this organization. The modular approach adopted allows one to obtain a standard document format in a modular prototype that can be reused or extended in a very easy way in other similar Java-based applications. The main driver of “Transformation from generic data format to well-formed XML/TAOML format” is the SAX API validating Java parser. It parses an input data stream and prints it out in XML format; it also generates events that correspond to different features found in the parsed XML document. In this context the SAX API Java parser is superior to the DOM API Java parser in many aspects of runtime performance. The SAX API parser used in this prototype was the Java-based open-source tool called Xerces. The XSLT processor reads in both the XML document and the XSLT style sheet. The XSLT style sheet describes a set of patterns to match within the XML document and the transformations to apply when a match is found. Pattern matches are described in terms of the tags and attributes for elements found within an XML document. Transformations extract information from the XML document and format it into a desired format. Each match-transformation pair is called an XSLT template. The XSLT transformation process works in a way very analogous to the way scripting languages such as Python or Perl operate - applying regular expressions to an input stream and then transforming the elements that were found to an output stream. The XSLT processor used in this prototype was an excellent Java-based, open-source tool called Xalan. In a modern day implementation of the prototype system described in this section, the distributed option for dataflow transformation of the Growing Book could be performed in the Cloud. 6. Active Index for e-Learning The active index has already been encountered in research described in the previous sections, both through the index cells which make up part of the TAOs and as a standalone component. The research described in this section takes a more in depth look at how an active index can be integrated with an e-learning system incorporating mobile communications, sensor networks and search engines in order to form a distributed intelligence system, one of whose applications is in e-learning [24]. The active index consists of a number of index cells (ICs). An IC is a kind of Finite State Machine which accepts input messages, executes operations and sends one or more output messages to one or more ICS or to the external environment. The Index Cell is an intelligent agent which perceives its environment through the messages received and executes actions in order to reach a desired goal. An e-learning system can be described as a distributed intelligence system with the instructors and students being intelligent beings. As part of such a system, a kind of intelligent behavior can be identified and synthesized into index cells. In the research described, two intelligent behaviors are discussed – proficiency and deficiency. These descriptions apply to students’ mastery of learning concepts described in multimedia documents which are each associated with index cells. Depending on the students’ understanding of the material, it will need to be adapted (made more difficult or easier) in order to adapt to the students.

12

A proficiency-level index cell is associated with a particular multimedia document. When the level of proficiency of a group of students who are using that document reaches a certain predefined threshold, the IC will send a message to the instructor of the course informing him that a critical number of students have reached a high level of proficiency in the material covered in the document. The IC may also send messages to a set of multimedia documents, to increase the level of difficulty. Likewise, a deficiency-level index cell fires messages to the instructor and other multimedia documents with the aim of decreasing the level of difficulty. There is a self-adjustment index cell associated with the multimedia documents which are in charge of transforming the document in order to increase or decrease its difficulty, depending on the message received. A prototype Multimedia Knowledge Environment for Index Cells, which is an Eclipse environment for modelling TAOML private knowledge was produced as part of this research. The tool captures both static and knowledge structure and produces an XML file containing that information. It can be seen that this approach can be used to achieve the multi-level usage goal for the Growing Books which are part of the Macro University, just as the TAOML transformations described in the previous section can be used to achieve the multi-lingual and multi-modal characteristics of the Growing Book. This all of the pieces fit together into a harmonious whole. 7. Chronobot The last of Prof. Chang’s research projects to be surveyed – the Chronobot/Virtual Classroom (CVC) [23] – takes a somewhat different approach than the previous ones, though the techniques can be incorporated into the Macro University/Growing Book framework. The chronobot is a device for storing and borrowing time [22]. The underlying premise of the chronobot is that there is a device for exchanging knowledge and time. For example, one can spend some time now to acquire knowledge and later use that knowledge to save some time. The chronobot can also facilitate the exchange of time and knowledge among a group of people. E-learning is a natural application for the chronobot. As an example, a student who encounters a problem he us unable to solve on his own interacts with other students in the virtual classroom who may help him resolve the problem. The student who was helped must in turn lend time/efforts either in the past or future in order to reimburse those who helped him. The chronobot serves as the platform which facilitates time and knowledge exchanges. The virtual classroom is a communication tool which combines the functionality of a web browser, chat room, white board and multimedia display. The CVC integrates chronobot and virtual classroom into a single system. The CVC implements a client/server architecture. The clients include chronobots, teachers, and students which implement user interfaces for different classes of users. A number of servers are part of the system: name server; communication server; storage server; KEM/TM server. The first three of these are fairly self-explanatory while the KEM/TM server manages time and knowledge exchange for the chronobot clients. Communication between the various components of the system are facilitated by the communication servers which are implemented using Java Messaging Service (JNS). The messages are formatted using

13

XML and may be archived and retrieved on the basis of a time range. A customized testing tool was also implemented. The Chronobot/Virtual Classroom implements technology for student interaction, as well as interaction between students and teachers, which could be usefully integrated into the Macro University/Growing Book project. Prof. Chang was granted a US Patent Office patent on the chronobot in 2010. 8. Conclusions and Future Directions The following table documents the use of several technologies developed by Prof. Chang and his colleagues and their usage in the major projects described in this paper. Name Knowledge Table

Description Use Reference Unified knowledge MI [4] representation. Active Index Indexing structure which can GB [10] initiate action. Tele-action Object Multimedia object with MI, MA, GB [11] attached private knowledge. Sentient Map Visual display manipulated MA [8] via gestures. Chronobot Time management tool for CVC [22] storing and borrowing time. MI=Micro-university, MA=Macro University, GB=Growing Book, CVC=Chronobot/Virtual Classroom Table 1 – Prof. Chang’s research technologies used in his learning projects As is evident in this paper, Prof. Chang has been a pioneer in the area of education, distance learning, and virtual universities. His research has inspired many of his colleagues as well as researchers around the world to continue his path in advancing the research in this particular field. The approach of the micro- and Macro University can be considered the basis for the creation of an evolutionary model of smarter universities that adapts to the need of the student, the environment, and the community as described in [28]. A smarter university strives to offer rich, interactive and ever-changing learning environments by exploiting the suite of technologies and services available through the Internet, by empowering individuals’ abilities and attitudes, and by encouraging them to interact and collaborate in a framework in which people are co-responsible for raising and appraising the attitudes of everyone. Opinion mining and collection of needs guide the creation of a strategic vision, which thanks to a prioritization of selected goals and objectives guide the learning environment to a selection of proper contents, both domain specific and common shared. The selection of competencies in accordance to the standards and the policy of the educational environment must be identified and properly matched in order to achieve the planned objectives. The process must be iteratively monitored in an evolutionary approach [28].

14

Modern and advanced models today still explore and expand the original Growing Book idea of stimulating collaboration by supporting multi-level, multi-lingual and multimodal usage of shared learning material as analyzed in [29][30] where newer tools such as modern platforms of collaboration, the PaaS model, and cloud computing are integrated into the learning process. Ubiquitous eLearning via mobile computing platforms and incorporating social networks also require transformation of learning materials for personalized user experiences [25][26][27]. While such research advancements are to be expected, they serve to confirm the visionary ideas of Prof. Chang. Those of us who have had the pleasure of collaborating Prof. Chang in research and teaching are grateful for his leadership in these areas. References [1] S.-K. Chang, "Computers and Education - Toward Man-Machine Symbiosis", Proceedings of International Conference on Computers and Education, Taipei, Taiwan, Republic of China, July 5-7, 1978, 5-15. [2] S.-K. Chang, D. Zhang and J. Stigler, "A Computerized Abacus for Office Automation and Computer-Aided Instruction", International Journal of Computer Processing of Chinese & Oriental Languages, published by Chinese Language Computer Society 1 (1984), 205-210. [3] S.-K. Chang, E. Hassanein, and C. Y. Hsieh "A Multimedia Micro-University", IEEE Multimedia Magazine 5 (1998), 60-68. [4] S.-K. Chang and C. Y. Hsieh, “Knowledge Table as a Unified Knowledge Representation for Office Information System Design”, IEEE TC on Office Automation Newsletter 3 (1989), 12-25. [5] T. K. Shih, S.-K. Chang and T. Arndt, “Database Support of Web Course Development with Design Patterns”, in Proceedings of the Workshop on Web-based Education & Learning (WEL 2000), Hong Kong, June 18 - 20, 2000. [6] S.-K. Chang, T. Arndt, F.R. Guo, S. Levialdi, A.C. Liu, J.H. Ma, T. Shih, and G. Tortora “The Macro University - A Framework for a Consortium of Virtual Universities”, International Journal of Computer Processing of Oriental Languages 13 (2000), 205-222. [7] S.-K. Chang, “The Sentient Map and Its Application to the Macro University ELearning Environment”, Proceedings of VISUAL 2000, LNCS 1929 (2000), 1-13. [8] S.-K. Chang, “The Sentient Map”, Journal of Visual Languages and Computing 11, (2000), 455-474.

15

[9] T. Arndt, S.-K. Chang, and A. Guercio “Formal Specification and Prototyping of Multimedia Applications”, International Journal of Software Engineering and Knowledge Engineering, 10, (2000), 377-409. [10] S.K. Chang, “Towards a Theory of Active Index”, Journal of Visual Languages and Computing, 6, (1995), 101-118. [11] H.J. Chang, T.Y. Hou, A. Hsu, S.K. Chang, “The Management and Application of Tele-Action Objects”, ACM Multimedia Systems Journal 3 (1995), 204-216. [12] S.-K. Chang, “Extending visual languages for multimedia”, IEEE Multimedia Magazine 3 (1996), 18-26. [13] SCORM website: http://adlnet.gov/adl-research/scorm/ [14] Experience API website: http://experienceapi.com [15] S.-K. Chang, “A Growing Book for Distance Learning”, Proceedings of Advances in Web-Based Learning, First International Conference, ICWL 2002, LNCS 2436, 3-16. [16] S.-K. Chang, M. J. Chen, L. Y. Deng and Y. J. Liu, "MAWC Operations for the Growing Book", Proc. of Seventh Int'l Conf. on Distributed Multimedia Systems, Sept 26-28, 2001, Taipei, Taiwan, 1-10. [17] S.-K. Chang, Data Structures and Algorithms, World Scientific Publishing, 2003. [18] M. Raman, “Wiki Technology as a ‘Free’ Collaborative Tool Within an Organizational Setting”, Information Systems Management 23, 2006, 59-66. [19] S.-K. Chang and S. Y. Shao, "Management of the Growing Book as Generalized Objects", Proc. of 15th Int'l Conf. on Software Engineering and Knowledge Engineering, July 1-3, 2003, San Francisco, 599-606. [20] T. Arndt, S.-K. Chang, A. Guercio and P. Maresca, "An XML-based Approach to Multimedia Software Engineering for Distance Learning", Journal of Distance Education Technologies, 1 (2003), 21-45. [21] IEEE Standard for Learning Object Metadata: https://standards.ieee.org/findstds/standard/1484.12.1-2002.html [22] S.-K. Chang and G. Santhanakrishnan, "Chronobot: A Time and Knowledge Exchange System for E-Learning and Distance Education", Proceedings of 2004 Distributed Multimedia Conference, September 8-10, 2004, 443-450. [23] S.-K. Chang, X. Li, R. Villamarin, D. Lyker and C. Bryant, "The Design and Implementation of the Chronobot/Virtual Classroom (CVC) System", Proceedings of

16

2006 International Conference on Distributed Multimedia Systems (DMS06), Grand Canyon, USA, Aug 30-Sept 1, 2006. [24] P. Maresca, S.-K. Chang and M. Pesce, "Application of Active Index to the Management of E-Learning", Journal of e-Learning and Knowledge Society (2006) 331341. [25] T. Arndt and A. Guercio, “XML-Based Course Material Transformation for Ubiquitous eLearning Applications”, in Proceedings of KMIS, Madeira, Portugal, October 6-8, 2009. [26] T. Arndt and A. Guercio, “Social Network-Based Course Material Transformations For A Personalized And Shared Ubiquitous E-Learning Experience”, in Proceedings of UBICOMM 2011, Lisbon, Portugal, November 20-25, 2011. [27] T. Arndt, A. Guercio, “Course Personalization for Ubiquitous E-Learning”, ISAST Transactions on Computers and Intelligent Systems 2 (2010), 1-11. [28] M. Coccoli, A. Guercio, P. Maresca L. Stanganelli, “Smarter Universities - A Vision for the Fast Changing Digital Era”, Journal of Visual Languages and Computing 25 (2014), 1003–1011. [29] P. Maresca, A. Guercio, L. Stanganelli, T. Arndt, “Experiences in Collaborative Learning”, Journal of e-Learning and Knowledge Society 10 (2014), 121-145. [30] M. Coccoli, P. Maresca, L. Stanganelli, A. Guercio, “Experience of Collaboration in PaaS for the Smarter University Model”, Journal of Visual Languages and Computing 31 (2015), 275-282.

17