Evaluation of Using Software Infrastructure and ...

2014 International Conference on Intelligent Networking and Collaborative Systems

Evaluation of Using Software Infrastructure and Multimedia Technologies for the Creation of Complex Learning Resources David Gañán1, Santi Caballé1, Jordi Conesa1, Tetsuya Oda2, Leonard Barolli2 1

Open University of Catalonia, Department of Computer Science, Multimedia, and Telecommunication Barcelona, Spain {dganan, scaballe, jconesac}@uoc.edu 2 Fukuoka Institute of Technology, Department of Information and Communication Engineering Fukuoka, Japan [email protected] | [email protected]

Abstract— This paper proposes the provision of software infrastructure to support new types of pedagogically and technologically augmented collaborative learning resources by means of an innovative application framework called Virtualized Collaborative Sessions Framework (VCSF). The VCSF approach helps meet challenging pedagogical requirements in online collaborative learning, such as increasing student's engagement and learning performance during the collaboration. In turn, the systematic application of the VCSF enriched with semantic and multimedia technologies enables e-learning developers to embed successful collaborative learning experiences from external learning tools in a software reuse fashion. Our application framework was prototyped and successfully tested in real environments, thus showing the software reuse capability and the pedagogical benefits of the VCSF approach. In this paper, we report on the evaluation stage of our research.

In order to overcome these limitations, in previous research, we reported on a new collaborative learning methodology called Collaborative Complex Learning Object (CC-LO) [5] through the development of a system prototype called Virtualized Collaborative Session (VCS) that enables the embedding and execution of the CC-LO approach. The VCS application creates CC-LOs from the virtualization and registration of live collaborative sessions, which can be augmented by alternative learning paths, cognitive and emotional features, additional content, etc., during an authoring phase (i.e. an expert managing the CC-LO). The registered CCLOs are eventually packed and stored in the form of Collaborative Complex Learning Resource (CC-LR) for further reuse, enriching live sessions of collaborative learning with balanced levels of interaction, challenge and empowerment [7]. However, current Learning Management Systems (LMS) do not support a straightforward integration of modern pedagogical models and methodologies nor the corresponding complex learning resources yielded [8], instead great development efforts must be made to integrate these resources into LMSs most of times in an ad hoc fashion, with very inefficient results in terms of productivity, quality and cost [9]. Further, LMS developers have no opportunities to leverage computational experiences of on-line collaborative learning designed to meet modern and changing pedagogical requirements, hence the software reuse capacity in this field is rather low [10][11]. In previous research we proposed the provision of software Infrastructure to support new types of pedagogically augmented collaborative learning methodologies and resources by means of an innovative application framework [12] called Virtualized Collaborative Sessions Framework (VCSF) [6][14]. The VCSF supports the entire process of creation, management and execution of the mentioned SLO and CC-LR resources from any LMS and collaborative learning tool. As part of the process, the VCSF exploits the VCS application and leverages Web semantic technologies [13] to enable e-learning software developers to embed many types of successful collaborative learning experience into different LMSs in a software reuse and systematic fashion.

Keywords—software infrastructure; application framework, VCSF, software reuse; multimedia technologies; collaborative learning; virtualized collaborative sessions

I.

INTRODUCTION AND BACKGROUND

Computer-Supported Collaborative Learning (CSCL) is a mature research field in the educational domain dedicated to improving teaching and learning through the introduction of modern ICT [1]. Collaborative learning activities vary widely, though most of them are centered upon student’s exploration or application of the course material, not simply the teacher’s presentation or explication of it [2]. However, many researchers ([2][3][4]) argue that students must be meaningfully engaged in the CSCL resources for effective collaborative learning to occur. Such a lack of engagement is especially evident in on-line collaborative learning content, and can be attributed to the lack of (i) real interactivity (in many cases the only interaction available is to click on the next button to obtain the next message in a discussion forum); (ii) challenging collaborative tools, instead of tools which fail to stimulate learners, making the collaborative experience unattractive and discouraging progression; and (iii) empowerment, as learner expects to be in control of their own collaborative learning. 978-1-4799-6387-4/14 $31.00 © 2014 IEEE DOI 10.1109/INCoS.2014.29

457

(Storyboard Learning Object) and stored into a repository. The rest of components in this layer interact with this repository in order to play and edit such SLOs.

In this paper, we present a new evaluation of the VCSF approach from both final-user and technical perspectives of the application of our framework into different LMS and elearning tools to support the virtualization process of live collaborative learning sessions. The ultimate aim is to show the software reuse capability from the systematical application of the VCSF approach as well as the resulting pedagogical benefits, such as increasing students’ engagement and learning performance. The paper is structured as follows: Section 2 presents introduces the main concepts of the VCSF approach for supporting the development of complex collaborative learning resources. Section 3 is the core of the paper, which extensively evaluates our application framework through steps of technical and pedagogical experimentation by actual users and developers. Finally section 4 concludes the paper and outlines ongoing and future work. II.

B. Conversion layer There are many data sources providing collaborative sessions data, principally forums and chats tools from different LMS. An abstraction layer was designed in the VCSF to implement the conversion between the collaborative data into the corresponding data sources, and the resulting SLO. This way the SLO approach can be reusable and adapted to new data sources from many live collaborative sessions. The conversion layer of the VCSF defines a generic converter component that can be particularly implemented for different data sources. Each data source should implement its own converter which will map data from the corresponding data source into CS2 entities, following a set of predefined mapping rules.

VCSF APPROACH

C. VCS layer The VCS layer and its components are the core of the VCSF architecture. They provide the CC-LO/SLO model and the tools and services required to create, edit and reproduce such type of complex learning objects. The architecture of the VCS layer is composed by three types of components:

In this section we briefly present the main concepts of the VCSF approach, which is evaluated in next section (see a complete description of the VCSF in [6][14]). The main purpose of the VCSF system is to be able to extract knowledge from a collaborative session and virtualize this knowledge into a sequence of animated characters that reproduce the dialog between the participants of that session [5][13]. The result of such conversion is packaged for reuse in the form of CC-LOs (Collaborative Complex Learning Objects). CC-LOs can also be extended with additional content such as assessment or emotional features into a CC-LR (Collaborative Complex Learning Resource) [7].

x

SLO Repository: This is a core component of the system, which takes care of managing the storage of previously created SLO for reutilization. x Tools: They enable the user to interact with the system and manage SLOs. SLOPlayer enables to reproduce an SLO while SLOEditor lets modify its content. The VCSCreator helps in the process of creating new SLOs. x Services: They offer different generic functionalities to the system, which are used within the tools and other components. Next we show the main functionalities: a. Speech service. Provides text-to-speech (TTS) capabilities to the framework. b. Multimedia repository service. Manages a repository for the storage of multimedia resources. c. Conversion service. Communication with the conversion layer is addressed by this service, to avoid coupling with many modules. d. Activity log service. Enables logging capabilities. e. Keywords and Classification services. Provides access to a dictionary of predefined keywords for tagging. Following the reuse capability of the framework, the VCS layer can be easily extended by adding new services and tools for meeting new requirements and functionalities.

Figure 1. VCSF architecture and components

D. CC-LR construction process The process of creation of CC-LRs with the VCSF starts when an editor user (e.g. a teacher) decides to convert any collaborative session into a reusable learning object. The editor user starts the VCS Creator tool to do so, which allows for selecting a collaborative data source, and then it connects to the corresponding data source and shows a list of available collaborative sessions to choose. Then the user selects a session from the list, and the tool uses the corresponding converter in order to translate the collaborative session into CS2 format.

A. The architecture of the application The presented framework is based in a model for representing Storyboard Learning Objects (SLOs), which are specific instantiations of the CC-LO approach [5]. The VCSF architecture describes two main layers (see Figure 1): (i) a conversion layer which converts data from different data sources of collaborative sessions into a common ontology specification known as CS2 (Collaborative Session Conceptual Schema) [13], and (ii) the VCS Layer where the information stored in CS2 format is converted into an SLO

458

The test group at Site A had three members: A researcher familiar (Expert) with the CC-LR approach and its objectives, but not directly involved in the VCSF framework; a doctoral student (Technician) working in a technical discipline but unfamiliar with the VCSF approach; a doctoral student (Novice) unfamiliar with the VCSF approach and working in a non-technical discipline. Users were given 10-15 minutes to interact with the online demonstration then were asked to supply Likert-based assessments and qualitative feedback. They primarily interacted with the CC-LR “Discussion on Software Engineering”. This CC-LR is planned as supplementary material for understanding the concepts of software engineering, and it contains dialog scenes coming from a live collaborative session on the subject and extended with both assessment and emotional feedback scenes to guide the student through the apprenticeship. The test group at site B was composed of two members: an analyst programmer and researcher working in e-learning projects (Expert) and a Computer Science university student (Novice). The whole experiment took about 40 minutes, with the following steps: (i) discussion about the aim of experiment; (ii) user manual reading; (iii) playing of the CC-LR “Discussion on software Engineering”; (iv) elicitation of comments and results. The testing was run in each pilot site and by each tester by using different data input and running several executions of the prototype. To evaluate each test question the 5 point Likert scale was used to state their level of agreement or disagreement. The rating scale ranged from “I strongly disagree” (1), “I disagree” (2), “neither/nor” (3) to “I agree” (4), “I strongly agree” (5). The aim was to validate the concept of CC-LR and the VCS platform, which in turn validate the VCSF capabilities by asking the next three indicators of interest to the testers at the end of each test: 1. Create, edit, manage, store and playback the generated storyboard. Score on scale 0-5 and open comments. 2. The CC-LR allows users to observe how knowledge is constructed. Score on scale 0-5 and open comments. 3. The cognitive information provided in the CC-LR becomes an enjoyable and complete collaborative learning experience. Score on scale 0-5 and open comments.

Once the collaborative session data is represented using the CS2 data model, the creator tool passes this representation to the CS2toSLO component, which converts it into a CC-LO using an SLO representation for that purpose. Then, the created SLO from the original source collaborative session is playable, though it should be refined before using it in order to enhance the knowledge acquisition and students’ performance. The SLO Editor tool is used for the refine purpose, which also allows teachers or content editors to modify the recently created SLO by creating new scenes, or modifying and deleting existing ones. Characters and speeches can also been modified. Finally, among basic editing features, this tool also allows for introducing extended capabilities as assessment or emotional awareness to the CC-LO, converting it into a CC-LR. The process explained in this subsection was the method used to create the CC-LR used in the experimentation detailed in the next section [7]. III.

EXPERIMENTATION AND VALIDATION

For the validation of the VCSF we will follow a bottom-up process. In a first place we evaluate and validate that the CCLRs generated by the VCSF are valuable resources, then we induce that the VCSF which constructed them is also valuable. Two empirical experiments were conducted to evaluate the VCSF prototypes: (i) subjective user assessment and (ii) technical test. These studies were performed on a proof of concept of a CC-LR coming from a SLO built and supported by the VCSF framework. Therefore, the results and interpretation of these studies help validate the underlying VCSF framework used to develop the proofs of concept in different collaborative learning tools. In the next sub-sections, we first describe the set-up and procedure of each study and then show the experimental results along with an analytical data discussion on these results. A. User assessment The proof of concept of our framework for the development of a CC-LR was tested. Firstly, following the VCSF process of modelling and representing forum data, the data source of a live collaborative learning session was derived from the IWT web forum. Then, a specific converter was built to turn the data model of the IWT forums into common representation CS2, which was the input of the VCS Creator that generated an animated SLO showing how people discussed and collaborated, how discussion threads grew and how knowledge was constructed, refined and consolidated. This SLO was eventually stored for further reuse in the form of a CC-LR augmented with cognitive and emotional information.

2) Results and analytical data discussion Table 1 shows basic statistics of the quantitative marks on the scale 0-5 scored by all testers for each of the three indicators of interest considered. Table 4 shows an extract of qualitative results from the open comments of each indicator provided by the testers of the two sites after running the tests. Testers

1) Assessment set-up and procedure The design of this study consisted of two user assessments of the VCSF framework used to develop a CC-LR embedded in the VCS platform with the aim to evaluate the proof of concept. The first user assessment was at Coventry University Serious Games Institute in United Kingdom (Site A), and the second at the company Modelli Matematici e Applicazioni in Italy (Site B). In both sites, the object of the experiment was only the CC-LR play, thus comments and results are only relating to this aspect.

Site A

Site B

Questions / indicators of interest #1

#2

#3

Total

# Expert

5

5

4

4.7

# Tech

4

4

4

4.0

# Novice

2

4

3

3.0

# Expert

3

4

1

2.7

# Novice

4

2

3

3.0

3.6(1.1)

3.8(1.0)

3.0(1.2)

3.4(0.8)

Total M(SD)

Table 1. Results of both sites. 0-5 scale for Mean (M) and Standard Deviation (SD) statistics.

459

complex learning resources with the framework’s components and services. Additionally, combined feedback is provided by the lecturer as a user of the VCSF tools to manage the CC-LR generated by the developer. Therefore, in this phase of the experiments we were primarily interested in the functionality of the VCSF.

At Site A, the subjective assessment of the Expert user ranked highly with 5/5 for the first two criteria, and 4/5 for the remaining one. Qualitative evidence suggested the slightly lower marks for the last indicator was due to the difficulty in conclusively assessing impact on empowerment. Similarly, the Expert noted that enjoyment of the experience may be strongly influenced by the overriding perceptions of the learners towards technology-enhanced learning, though acknowledged the system was more engaging and entertaining than the static web forums used as a source of learning content. The Technician user again ranked the system well with a mean score of 4/5. They commented on the robustness of the interface, although also noted that the prototype they worked with for the evaluation did not fully equip them with an understanding of how collaborative sessions were stored and might be reused. With respect to knowledge construction, they observed that for a typical learner this would be an incidental learning outcome rather than a formal learning objective, which would itself by in, for example, software engineering. This may hence require scaffolding if it is to be an integral learning requirement of a course which uses the technology. Finally, The Novice user provided the lowest score though on average. Much of their criticism stemmed from usability concerns: “People talk very fast”, though they did describe the overall experience as “Enjoyable”. In part this was their perception as they participated primarily in the playback of existing scenes rather than their creation. However, this may not be so much a limitation as a potential advantage of the system: it can be used to support collaborative learning, but generated content can also be used in different pedagogical applications. At site B, all the scores were over average, except for the third indicator. On the other hand, the results provided by the Novice user were better than that coming from the Expert. Focusing on the first indicator, the common feeling is having a good appeal to the user, only with little annotations about sound quality by the Expert. About the second indicator, there is a quite strong difference in evaluation. Indeed, the Expert user recognized a well-organized built of knowledge while the novice user had some difficulties to discern the unitary knowledge defined by the flow of the scenes. Finally, as for third indicator, the management of cognitive information was appreciated by Novice users who liked very much the chance for self-evaluation while Expert users found the tests potentially problematic if abused or not designed in line with course objectives. In summary, all the indicators scored over the average 0 – 5 (see Table 1). In particular, indicators \#1 and \#2 were well scored by all testers who acknowledged that the new learning material (CC-LR) was engaging and entertaining though the sense of empowerment may depend on individual learners.

Indicators of interest

Testers’ open comments (type of the tester: E: Expert; T: Technician; N: Novice)

Create, edit, manage, store and playback the generated CC-LR.

"The stream of the scenes is fluid and the contents are quite well explained. The quality of the speech should be improved" (E, Site B)

The VCS platform allows users to observe how knowledge is constructed.

"[the] relationship to forum data is obvious, and this looks like a useful tool" (E, Site A) "Robust interface, though methods for creation and editing are not as transparent as those for playback" (T, Site A) "The prototype represents the building of knowledge in a quite consistent way. Some graphic aspects should be improved" (E, Site B)

The CC-LR becomes an enjoyable and complete collaborative learning experience.

"Knowledge construction observation seems a secondary goal... learning content on this would be required to support the learner if this is the core aim" (T, Site A) "...a very useful technical starting point for course development which features significant collaborative and peer-driven aspects" (T, Site A) "it’s interactive, though the sense of empowerment may depend on individual learners. I can see how avatars might allow for social identity to be expressed" (E, Site B)

Table 2. Excerpt of the open comments at both pilot sites.

1) Test set-up and procedure The sample of this experiment consisted of 2 people: a professional software developer with large experience in the development of e-learning systems and tools and a lecturer of the Computer Science Degree at the UOC. The experimentation procedure was as follows. The developer had to use the VCSF’s components and services to generate the CS2 data models corresponding with two different web forums: UOC phpBB and IWT, considering the different data sources of each forum. Once the output data models were ready, the lecturer created the corresponding SLOs for each forum using the VCS Creator tool. Then he edited the SLOs using the SLO Editor, in order to create CC-LRs enriched with complex pedagogical aspects (e.g., assessment and emotional aspects). Both Creator and Editor tools were included in the VCS suite of tools within the VCSF framework. The work sessions were as follows: x Work session 1: The developer is asked to integrate the VCSF with two data sources: UOC phpBB forums and IWT forums. The developer collects requirements for the converters and prepares the technology for the later development. Count the time invested. The lecturer chooses two representative collaborative sessions (one from each data source) in order to create a SLO from them later. Count the time invested.

B. Technical test The aim of this scenario was to test the whole building process of a CC-LR by means of the VCSF framework. This experimentation, therefore, focused on testing the technical and functional aspects of the VCSF components and tools to generate and manage CC-LR. To this end, the test was conducted at the UOC site from the developer’s and lecturer’s viewpoint. The results of this study provide relevant feedback of how the VCSF supports developers in order to create

460

x

b) VCSF as a valuable resource In order to evaluate the framework as a valuable resource for both education and software development, we asked the developer (D) and lecturer (L) to evaluate the experience, especially concerning the functionality of the VCSF by answering five open questions.

Work session 2: The developer creates a SLO from the UOC and IWT Forums by using the VCSF Framework. Three-step procedure: 1. Install and configure VCSF for each data source. Count the time invested. 2. Develop the converter for each data source. Count the time invested. 3. Add the converters to the list of available data sources converters. Count the invested time. x Work session 3: The lecturer creates the selected collaborative sessions into SLOs using the VCS Creator tool. Count the overall time invested. Procedure: 1. Open the VCS Creation tool and select the data source 2. Select the chosen collaborative session from the list and press the button to create de corresponding SLO. x Work session 4: The lecturer works on the newly created SLOs to produce a teaching material in the form of CC-LR by the SLO Editor (see [7]).

1. Please describe what you liked regarding VCSF. L: The lecturer found IWT a very complete and useful suite of tools and functionalities for helping teachers and students in e-learning activities. In particular, the lecturer reported that he liked “the ability of editing and personalizing each SLO in order to meet the specific requirements of the course”. D: The architecture was found well designed. The developer mentioned “it enables the integration of different data sources with the system.” This is in line with the development time counted with this framework, which was considered as not so high as developing the same tools from scratch. The saving in time and effort was thus considered evident by the developer and also the gain in confidence and quality of the product when reusing the experience provided by the VCSF components and technical documentation.

The developer and the lecturer were instructed to use the manual of the VCSF. No training sessions on the VCSF were programmed given the strong background of both of them in developing and using e-learning systems. After the task was finished, both participants were asked to fill out a questionnaire about their experiences with the system, especially concerning the functionality of the VCSF.

2. Please describe what you did not like regarding VCSF L: The tools found in VCSF are not very intuitive (especially the SLO Editor) and, for this reason, it is hard to use them the first time. He says that, perhaps, practice can help in this sense. The lecturer highlighted the great efforts to edit and customize an SLO. He reported that “I had to make a great effort to customize each element of the SLO. It is inherent to the task itself, but the SLO Editor tools should be improved in order to facilitate more the work.” However, the lecturer added that “…considering the result is a real learning material if we compare it to the workload to create a regular learning module or activity, the effort to create a CC-LR may be reasonable”.

2) Results and analytical data discussion In this section, we evaluate 2 aspects of this scenario: time to run the experience and VCSF as a valuable resource. a) Time to run the experience Next the time invested by the participants in each work session described above is shown below (note that some sessions were run by one participant only): x

Work session 1: 4 hours (developer). Time spent: 30 minutes (lecturer) x Work session 2: 1. 2 hours each converter 2. 6 hours each converter 3. 1 hours each converter Time spent: 18 hours considering the development of 2 converters (developer). x Work session 3: 30 minutes (lecturer) x Work session 4: 2 hours and 30 minutes (lecturer)

D: The components and services of the framework were found with lack of interoperability. In this line, the developer mentioned “tools are too tight to Microsoft technologies (like Silverlight), so they produced some compatibility problems when testing with students and teachers”. 3. Do you have any suggestions for improvements? L: Lecturer’ recommendations were in line with the comments provided in the previous questions. In particular, even if usability was not considered an issue, they suggested improving it in order to speed up their work. In particular, the lecturer suggested “add functionalities to enable ‘mass modifications’”. He also suggested improving many small functional details, such as getting the scene/part/character selected in the list when going back from working on this element, instead of always getting the first element of the list…this can avoid wasting a lot of time with large SLOs”.

The time invested in the whole experiment was quite high for the developer (22 h), and slightly higher time than expected for the lecturer (3 h and 30 min.). As for the developer, the high time spent was in line with the high learning curve needed to exploit the VCSF potential efficiently, as pointed out by the developer. The developer also mentioned that after learning how to use the framework the development time will surely decrease in next developments. As for the lecturer, most of the time was spent in preparing the dialog scenes of the CC-LR, and creating the tests (assessment scenes). The lecturer was also asked to estimate the time required to create more CCLRs. The answer was that time would increase linearly.

D: Following the previous comments, the developer keep insisting on the interoperable issues, such as “…tools could be implemented using a more portable technology like html5”. On the other hand, the developer liked the idea of the abstraction layer to extract knowledge from different collaborative sessions, which can be applied to create other types of LOs”.

461

and tools experimentation in further prototype implementations of the framework and of feedback gained during implementation and experimentation in further requirements and design improvement.

4. Concerning the user manual you have got, how clear was the description of the VCSF for you? Did the user manual support you in following the individual steps? L: The lecturer stated that the manual helped him in some parts but, for other parts, he required external assistance to configure the desire course structure.

ACKNOWLEDGMENT This work has been supported by the European Commission under the Collaborative Project ALICE “Adaptive Learning via Intuitive/Interactive, Collaborative and Emotional System”, VII Framework Programme, Theme ICT-2009.4.2 (Technology-Enhanced Learning), Grant Agreement n. 257639 (www.aliceproject.eu).

D: The developer found the user manual brought enough technical information though he would have appreciated the inclusion of a detailed case study to follow as an example. 5. From your point of view, do you think that developers would find easy to integrate the VCSF suite of tools, and lecturers would like to use them in their online courses? What are the pros and cons?

REFERENCES [1]

L: The lecturer recognized that VCSF is a good platform, with a lot of useful functionalities. However, he pointed out that the learning curve to efficiently exploit its potential is rather high.

[2]

D: The developer thought that the integration itself is easy “if you have clearly in mind the data source model and CS2, and then you find a mapping between them”. He mentioned that the problems may come because of security or authorization problems for example, “but the experience told us that these problems can be solved by designing a solution that fits the specific requirements”. IV.

[3]

[4]

[5]

CONCLUSIONS AND ONGOING WORK

This research proposes the provision of software infrastructure in the domain of CSCL to support the development of modern and pedagogically augmented collaborative learning resources. As a result, in this paper, an innovative application framework called VCSF has been presented and evaluated. From the user evaluation results, we can conclude that, the products created by the VCSF are correct both at functional and pedagogical level, so we deduce by induction that the framework which construct them is also correct. On the other hand, from the technical evaluation shows us that the systematic application of the VCSF provides eLearning domain developers the opportunity to leverage successful collaborative learning experiences in a software reuse fashion. However, the results achieved are not conclusive due to the exploratory and technical nature of the empirical study, in part because of the difficulty of finding developers with the required expertise in e-learning tools construction and willing to participate in the experiment. More experiences are expected to come and validate the VCSF as a general application framework to support the development of demanding types of complex and advanced learning resources. Therefore, further directions of research will go to subsequent iterations of the VCSF development of technological and experimentation activities, aimed at the improvement and refining of VCSF components from feedback of previous iterations. This iterative approach will allow the integration of feedback gained through the VCSF components

[6]

[7]

[8]

[9] [10] [11]

[12]

[13]

[14]

462

Dillenbourg, P. (1999) Introduction; What do you mean by “Collaborative Learning”? In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches, Elsevier Science. Goodsell, A., Maher, M., Tinto, V., Leigh Snith, B. and MacGregor, J. Collaborative Learning: A Sourcebook for Higher Education. Pennsylvania State University: National Center on Postsecondary Teaching, Learning, and Assessment, 1992. Dillenbourg, P. (1999). Collaborative Learning: Cognitive and Computational Approaches. Advances in Learning and Instruction Series., Elsevier Science Ltd. Stahl, G. (2006). Group Cognition: Computer Support for Building Collaborative Knowledge. Acting with Technology Series, Cambridge, MA: MIT Press. ISBN: 978-0262195393. Caballé, S., Gañan, D. Dunwell, I., Pierri, A., Daradoumis, T. CC-LO: Embedding Interactivity, Challenge and Empowerment into Collaborative Learning Sessions. Journal of Universal Computer Science, 18(1), 25-43, 2012. Gañán, D., Caballé, S. and Conesa, J. Towards software infrastructure for the systematic virtualization of collaborative learning sessions, Intelligent Networking and Collaborative Systems (INCoS), 2013 5th Unternational Conference on, IEEE, pp. 422-439, 2013. Caballé, S., Mora, N., Feidakis, M., Gañán, D. Conesa, J., Daradoumis, Th. Prieto, J. CC-LR: Providing Interactive, Challenging and Attractive Collaborative Complex Learning Resources. Journal of Computer Assisted Learning. Wiley, 30(1), 51–67, 2013. Abad, C.L. Learning through creating learning objects: experiences with a class project in a distributed systems course. 13th annual conference on Innovation and technology in computer science education. Madrid, Spain, ACM: 255-259, 2008. Gamma, E., Helm, R., Johnson, R. and Vlissides, J. Design Patterns: Elements of Reusable Software Architecture. Addison-Wesley, 1995. Czarnecki, K. and Eisenecker, UW. Generative Programming: Methods, Techniques, and Applications. Addison-Wesley, 2000. Caballé, S., Xhafa, F. CLPL: Providing Software Infrastructure for the Systematic and Effective Construction of Complex Collaborative Learning Systems. Journal of Systems and Software, 83(11), 2083-2097. Elsevier, 2010. Schmidt, D. C. (1997). Applying design patterns and frameworks to develop object-oriented communication software, in Handbook of Programming Languages, MacMillan Computer Publishing. Conesa, J., Caballé, S., Gañán, D. and Prieto, J. Exploiting the semantic web to represent information from on-line collaborative learning, International Journal of Computational Intelligence Systems 5(4): 653667, 2012. Gañan, Caballé, S., Conesa, J., Xhafa, F. VCSF: An Application Framework to Systematically Develop and Reuse Collaborative Complex Learning Resources. International Journal of Applied Mathematics and Computer Science, 2014. Submitted.