AbstractâSCEPPSys is a distributed pair programming system especially designed to ... a part of the Eclipse IDE, and a web-based administration environment.
2014 IEEE 14th International Conference on Advanced Learning Technologies
A web-based Authoring Tool for Scripting Distributed Pair Programming
Despina Tsompanoudi, Maya Satratzemi Dept. of Applied Informatics University of Macedonia Thessaloniki, Greece e-mail: {despinats, maya}@uom.edu.gr
Abstract—SCEPPSys is a distributed pair programming system especially designed to support the application of pair programming in computer science courses. Compared to other implementations, SCEPPSys enables students to solve programming assignments guided by collaboration scripts. This approach exploits the benefits of collaborative learning in pair programming, and aims to maximize learning outcomes by structuring the problem-solving process. The system consists of a shared programming environment, which runs as a part of the Eclipse IDE, and a web-based administration environment. The main focus of this paper is to present an overview of the system and a detailed description of the features of the administration section, which contains an authoring tool for collaboration scripts and instructional support tools. The incorporation of various group formation strategies and the analysis of students’ interactions make this software particularly useful for educators as well as researchers. Keywords-distributed pair programming; script; collaborative programming
I.
collaboration scripts, and to apply various group formation strategies. The main idea behind this implementation was to develop an educational system for DPP with embedded support for students as well as teachers. Since PP involves collaborative learning, students’ built-in support was based on a scaffolding technique which fosters collaborative activities. Scripted collaboration aims to improve the effectiveness of collaborative learning by structuring and coordinating students’ interactions, and its main advantage is its applicability in a wide area of disciplines [8, 9, 10]. In a similar way, we embedded collaboration scripts in a DPP system in order to improve the effectiveness of pair programming. A collaboration script contains a sequence of activities to be performed by a group of students. It also describes how students should collaborate, including guidelines on group composition and role distribution. The definition of collaboration scripts requires an authoring tool used by the instructor, and a workspace for the students, which supports scripted collaboration. For this purpose, SCEPPSys has two subsystems. A web-based administration environment for the teacher, and an Eclipse extension for the students. The administration panel is used as an authoring tool, containing functionalities related to the definition of collaboration scripts, and also support for monitoring the outcomes of students’ collaboration. It also enables the application of various group formation strategies which is an area still under ongoing research. A plugin for Eclipse provides means for the application of DPP, communication among group members and guided solving of programming assignments. A detailed description of how collaboration scripts are applied in PP will follow in the next sections. Section II provides an overview of SCEPPSys, including design and implementation issues. Section III focuses in the administration section of the system, describing the authoring process of collaboration scripts and the group formation policies supported by the system. In the last section we discuss conclusions and future directions (Section IV).
collaboration
INTRODUCTION
One way to gain the advantages of collaborative learning in computer programming is to adopt the pair programming (PP) technique. Proper application of PP has shown to be more effective and enjoyable than solo programming, and enhances students’ teamwork skills [1], [2], [3], [4]. In PP two programmers share one computer to develop software. In Distributed Pair Programming (DPP) they share a virtual workspace and may collaborate from distant locations using their own computers. Nowadays a variety of solutions exist in order to apply DPP. Some DPP systems cover the needs of professional teams, while a considerable number of systems derived from research studies [5], [6], [7]. The majority of them were built as a part of Eclipse because it’s an open source IDE and also very popular among Java developers. SCEPPSys (stands for Scripted Collaboration in an Educational Pair Programming System) was developed using an existing Eclipse plugin for DPP. The main difference between SCEPPSys and other implementations is the integration of programming assignments inside the development environment. Furthermore, the system incorporates a powerful administration panel which allows instructors to manage programming classes in the context of 978-1-4799-4038-7/14 $31.00 © 2014 IEEE DOI 10.1109/ICALT.2014.81
II.
SYSTEM ARCHITECTURE
SCEPPSys is based on a client-server architecture and consists of a server, a database, a plugin for Eclipse and the web-based administration environment. The plugin allows 259
III.
students to pair program adopting the roles of the “driver” and “navigator”, as the PP methodology suggests. While the “driver” is writing the code in the shared editor, the “navigator” inspects the program and intervenes whenever necessary via the embedded chat area. Role changes occur whenever users request a role switch or they are automatically initiated by the system. In this case role switches are performed based on the task distribution policy defined by the teacher. The system also guides students through a sequence of programming tasks which contribute to the solution of a major programming assignment. The activities of each task are described within the development environment and a hint mechanism displays useful information and examples when students seek help. The server is responsible to dispatch driver’s interactions, like opening, saving and executing a class file, to the navigator. These actions, in addition to text editing and highlighting are replicated in the remote workspace in order to provide appropriate conditions for an effective collaboration. An underlying database is used to store students’ profiles, assignments, projects, grades and statistical information. This means that one part of the database is maintained by the instructor and the remaining information is created and saved during pair programming sessions. Collaboration scripts are also created through the administration of the database and a detailed presentation follows in the next section. In summary, as depicted in Fig. 1, students require the Eclipse IDE and the plugin in order to run the application, while the instructor uses the server, the database and the administration environment in order to set up the system and to manage his courses. The system implementation was based on an open-source Eclipse plugin called XecliP which is referred in [6]. The initial plugin supported the basic requirements of DPP, and we made the appropriate changes in order to incorporate the definition and the application of collaboration scripts, as described in our previous work [11].
AUTHORING COLLABORATION SCRIPTS FOR DISTRIBUTED PAIR PROGRAMMING
A. Related Work There are many Eclipse plugins which were built to support remote collaboration and DPP. But none of them was built to support collaboration scripts. Thus, existing plugins provide limited capabilities for administration and tutoring. For example, XecliP and XPairtise [6] support user and group management, which are maintained inside of Eclipse. In both systems user accounts serve the authentication process and allow online users to view the profiles of other users. In XPairtise users may create their own accounts providing username, password and personal information. In XecliP users can edit their profiles which contain personal data and programming skills. Groups are supported only in XecliP and have a different scope than groups in our system. They are used in order to organize online users in topics and not as a tool to create pairs. Sangam [12] and RIPPLE [3] support both basic user management for authentication purposes. RIPPLE, which is an extension of Sangam, provides a logging capability of users’ interactions but only for research purposes. The log files are saved in a database but the system does not contain a tool for their analysis or representation. Saros [12] is a plugin for distributed collaborative programming, still under ongoing research, but not oriented as an instructional support tool. In conclusion, DPP plugins usually contain basic management features, like user management and server setup. On the other hand, authoring tools for computersupported collaboration scripts are general purpose editors, because they facilitate the creation of pedagogical scenarios in various areas that involve collaborative learning. Such tools are RELOAD [14] and Collage [15]. These tools help teachers to create a sequence of collaborative learning activities by defining learning objectives, participants, roles, tasks, resources etc. A runtime environment is then needed in order to interpret and execute the script. SCEPPSys incorporates the design of collaboration scripts, tailored to the needs of DPP, within the administration environment (Fig. 2). The “execution” of a script is initiated when students use the Eclipse plugin for pair programming. The authoring tool is not compliant with any standard and works only as a part of SCEPPSys. However, the whole system is an open-source software and its architecture allows to perform changes or extensions in each part of the system independently. B. SCEPPSys’ Authoring Tools As proposed in [16] a collaboration script can be described using a number of components and mechanisms. Script components include the definition of participants, groups, activities, roles and resources. Script mechanisms define group formation, task distribution and sequencing. The authoring process of these attributes was adapted to the requirements of DPP. Some of them were fixed, like the number of members per group which is always two. The only attribute which was not included in the authoring
Figure 1. System architecture of SCEPPSys.
260
Figure 2. Screenshot of SCEPPSys’ Administration Panel.
process is resources. This component usually contains a description of available learning materials, tools, resources etc. In the case of DPP this involves the shared development environment (Eclipse IDE & Plugin) and the built-in assignment description with the hints. The remaining attributes of the collaboration script have to be defined by the teacher using the respective tools of the administration panel: 1) Participants. The participants of a collaboration script are the students that take part in the collaborative learning activity. We designed a unit where the teacher can create user profiles with basic user information, e.g. personal data and login information. Each unit in the administration panel contains all basic functions of database manipulation which allow the teacher to browse and edit the contents of the database. Moreover, students’ data may be imported from an external file in order to facilitate processing of multiple students. The system also handles multiple classes, meaning that one teacher may supervise more than one class. A class contains the students that will participate in a particular collaboration script, but as a feature it is not a part of the typical components of collaboration scripts. 2) Roles. The roles in PP are predefined because a user may act as a “driver” or as a “navigator”. Hence, the system does not include the definition of roles. How roles are distributed among group members is a feature of the task distribution mechanism which is described below. 3) Groups. As previously mentioned, the number of members per group is fixed, thus all participants are grouped in pairs. The teacher defines a group by selecting two students from the database. Groups may be copied
between courses or consecutive programming assignments in order to reduce the workload of the teacher. 4) Group Formation. The group composition is a key factor which determines the outcomes of the collaboration. For this reason several group formation strategies have been studied in the literature [17]. SCEPPSys supports the most common practices, which are: a) Free Selection: A common way to form groups is to take into account students’ preferences or let the teacher to decide the members of each group. Therefore, the default group formation tool of SCEPPSys is based on free selection of group members. b) Random groups: his group formation policy assures that students are randomly distributed in pairs, allowing thus the creation of heterogeneous groups since other common characteristics are note taken into account. c) Comparable skill levels: Research suggests that pairing students with similar skill levels has positive results on motivation and participation [17], [18]. For this reason we included the creation of groups with comparable skill levels. This policy considers students’ performance in past programming assignments, and creates pairs with members having approximately the same average grade. The created groups are then automatically saved in the database. The average grade is calculated based on submitted grades of previous assignments, which were solved using the system. d) Comparable contribution levels: This policy also depends on data of past programming assignments. When an assignment is completed and submitted by the students, the system saves in the database the contribution level of each pair member. Although the system saves two types of contribution, this group formation policy considers students’
261
contribution in the program code in order to form the groups. The contribution level in this case is calculated by means of the amount of the inserted program code. This policy aims to pair two non-participating students or two dominating students in order to achieve participation rates within satisfactory limits. 5) Task distribution. This feature describes how tasks and roles should be distributed among group members. Assigning a task to a student means that he takes the role of the “driver” and is responsible to type the solution in the shared workspace. The teacher may select one of the following task distribution policies, or may apply different policies within one class. a) Free collaboration: This policy allows students to distribute tasks based on their own decisions. Since the main drawback of free collaboration is disengagement on the part of one student, the system contains another two policies where task distribution is arranged by the system.
the sequence of tasks within one programming assignment, which is defined while creating the assignment. The other type contains a schedule of programming assignments. The teacher may specify the order in which programming assignments are given to students and the deadline of each assignment. C. Assessment and Interaction Analysis Besides aforementioned authoring tools, SCEPPSys’ administration environment contains two additional tools for assessing submitted programming assignments and monitoring the outcomes of students’ collaboration. An assessment form provides for each assignment task the description, students’ submitted solution and a sample solution in order to facilitate the assessment process. The grade of each task is entered by the teacher and then saved in the database. The embedded assessment feature not only provides an overview of students’ performance, it also serves the formation of groups with comparable skill levels. During DPP sessions the system monitors students’ interactions in order to keep track of collaboration related factors and to assess their level of participation. This information is available in the “Statistics” section of the administration panel, and may be exported as a spreadsheet for further research. In order to assess each student’s contribution to the submitted programming assignment, the system saves the following values: • Driving time of each student, which is the time he spent for typing the program code • Non-driving time of each student, which is the time he spent for communication, coordination and other interactions (e.g. requesting a role switch) • Amount of inserted program code per student. This value contains the number of characters typed by each student. Collaboration is indicated by means of the following values: • Number of exchanged messages (per student level) using the embedded chat function • Number of program executions using the customized run command which triggers a program execution on the remote workspace • Number of role switches, which is calculated as the total number of system-driven role switches plus the number of individual role switches. Related to this value is the number of tasks solved according to the role distribution mechanism, which is saved by the system in order to assess if students followed the recommended role assignments. • Number of retrieved hints per group level • Distribution of tasks per student. This section shows the number of tasks solved by each student and also the distribution of learning goals. • Productivity is the driving rate of each group. It’s calculated using the driving time of each user and the total time they spent in solving the assignment. • Grade, which is calculated as the average grade of all subtasks of a programming assignment
b) Rotating Roles: In order to achieve frequent role switches and an equal number of tasks per student this policy performs successive role alternations. All role switches between assignment tasks are performed by the system, however students are allowed to request role switches during the solving process of one task. c) Balanced knowledge acquisition. When students work in groups they tend to distribute tasks based on their abilities. This policy aims to achieve a symmetry in skill acquisition. The system assigns a task to a student based on the learning objective of the task. In this case the goal is that pair members solve an equal number of tasks of each learning objective. Again, students are allowed to perform individual role switches during the solving process of one task. 6) Activities. In PP all activities that participants engage in are associated with the solving process of programming assignments in Java. The definition of activities involves the creation of a logical sequence of programming tasks which lead to the solution of a greater Java programming assignment. For each individual assignment step the teacher must provide a description of the problem and a hint to the solution. The description is displayed in students’ workspace while they are in a DPP session, and the hint is a feature aimed to help students who get stuck and to encourage their participation. To each task is assigned a learning objective which is then used by the task distribution mechanism. Learning objectives are defined by the teacher in a separate section, prior to the creation of programming assignments. Some sample learning goals are: understanding and using conditional statements, searching and sorting an array, using logical operators etc. 7) Sequencing. This attribute refers to the order in which tasks and roles are distributed. The teacher may define two types of sequencing in SCEPPSys. The first one is related to
262
The calculation of such parameters aims to assist to the evaluation of group projects, where the contribution of each individual member is hard to assess. Additionally, teachers may examine various group formation strategies or task distribution policies and monitor the impact on participation and collaboration. A future research direction in this area is to examine which group composition is more effective in DPP, something that is still missing in the literature. So far we have used the system in two studies [11]. The first study was conducted during one laboratory session and the goal was to test the consistency of the system. This pilot study followed an evaluation study which lasted 8 weeks. Over that period, two groups of twelve pairs were assigned five Java projects, and were asked to solve them using the DPP plugin. We applied different task distribution policies for each group. In the first group, task distribution was based on “free collaboration”. In the second group tasks were distributed by the system in order to achieve “balanced knowledge acquisition”. Evaluation results revealed that the second group achieved symmetry in knowledge acquisition, performed more role switches and solved the assignments in a shorter time. In both groups students achieved sufficient contribution levels. These findings suggest that collaboration scripts have a positive impact on DPP and may provide the basis for further research. IV.
REFERENCES [1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
CONCLUSION
[9]
In our work we propose an instructional framework which introduces collaboration scripts in the pair programming pedagogy. To this purpose we have developed a novel authoring tool, which is part of a DPP system, and therefore oriented to the requirements of pair programming. In this paper we have presented the design of the DPP system and a detailed description of the embedded authoring tools. Compared to the components and mechanisms that form a collaboration script, we demonstrated how these attributes were adapted in order to cover the needs of DPP. Furthermore, two additional tools were presented which serve the assessment process and the analysis of collaboration. As an alternative approach to teach computer programming, the proposed methodology not only provides a theoretical framework, but also the corresponding infrastructure. SCEPPSys is an integrated system mainly designed to enable DPP over the Eclipse IDE, and enhanced with added support for organizing programming assignments and fostering the problem-solving process. Nevertheless, further research is needed in order to study the effectiveness of this new approach. For instance, does scripted pair programming improve students’ performance compared to traditional assignment methods? Which strategies for task distribution and group formation are more effective? In our future work we intend to investigate these issues using the features of SCEPPSys. The main focus of the next evaluation phase will be the study of various group formation policies, and how they affect the collaboration and contribution related factors. At the same time, we aim to gain feedback on system usability and the embedded features from instructors participating in the evaluation process.
[10]
[11]
[12]
[13]
[14] [15]
[16]
[17]
[18]
263
C. McDowell, B. Hanks, and L. Werner, “Experimenting with pair programming in the classroom,” ACM SIGCSE Bulletin, vol. 35, no. 3, p. 60, Sep. 2003. L. Williams, D. S. McCrickard, L. Layman, and K. Hussein, “Eleven Guidelines for Implementing Pair Programming in the Classroom,” Agile 2008 Conference, pp. 445–452, 2008. K. E. Boyer, A. a. Dwight, R. T. Fondren, M. a. Vouk, and J. C. Lester, “A development environment for distributed synchronous collaborative programming,” ACM SIGCSE Bulletin, vol. 40, no. 3, p. 158, Aug. 2008. E. Mendes, L. B. Al-fakhri, and A. Luxton-reilly, “A Replicated Experiment of Pair-Programming in a 2nd - year Software Development and Design Computer Science Course,” ACM SIGCSE Bulletin , vol. 38, no. 3, p. 108-112. 2006. C. Bravo, R. Duque, and J. Gallardo, “A groupware system to support collaborative programming: Design and experiences,” Journal of Systems and Software, vol. 86, no. 7, pp. 1759–1771, Jul. 2013. T. Schümmer and S. Lukosch, “Understanding Tools and Practices for Distributed Pair Programming,” Journal of Universal Computer Science, 15 (16), 3101-3125, 2009. K. Granville and T.J. Hickey, “The design, implementation, and application of the grewpEdit tool.” Proceedings of the 2005 conference on Diversity in computing, 14 - 16, 2005. S. Atmatzidou and S. N. Demetriadis, “Evaluating the Role of Collaboration Scripts as Group Guiding Tools in Activities of Educational Robotics: Conclusions from Three Case Studies,” IEEE 12th International Conference on Advanced Learning Technologies, pp. 298–302, Jul. 2012. D. Tsovaltzi, et al., "Extending a virtual chemistry laboratory with a collaboration script to promote conceptual learning," International Journal of Technology Enhanced Learning 2.1, 91-110, 2010. D. Diziol, N. Rummel, H. Spada and B. McLaren, "Promoting learning in mathematics: Script support for collaborative problem solving with the Cognitive Tutor Algebra," In The Proceedings of the Conference on Computer-Supported Collaborative Learning, 2007. D. Tsompanoudi, M. Satratzemi and S. Xinogalos. "Exploring the effects of collaboration scripts embedded in a distributed pair programming system." Proceedings of the 18th ACM conference on Innovation and technology in computer science education, 225-230, 2013. C. Ho, S. Raha, E. Gehringer, and L. Williams, “Sangam – A Distributed Pair Programming Plug-in for Eclipse,” Proceedings of the 2004 OOPSLA workshop on eclipse technology eXchange, 73–77, 2004. S. Salinger, C. Oezbek, K. Beecher, and J. Schenk, “Saros : An Eclipse Plug-in for Distributed Party Programming Categories and Subject Descriptors,” Human Factors, pp. 48–55, 2010. C. D. Milligan and P. Beauvoir, “The Reload Learning Design Tools,” Journal of Interactive Media in Education, 2005 (6). D. Hernández-Leo, et al., “COLLAGE : A collaborative Learning Design editor based on patterns,” Educational Technology & Society, vol. 9, pp. 58–71, 2006. L. Kobbe, A. Weinberger, P. Dillenbourg, A. Harrer, R. Hämäläinen, P. Häkkinen, and F. Fischer, “Specifying computer-supported collaboration scripts,” International Journal of Computer-Supported Collaborative Learning, vol. 2, no. 2–3, pp. 211–224, Sep. 2007. L. Williams, L. Layman, J. Osborne, N. Katira, and N. Carolina, “Examining the Compatibility of Student Pair Programmers,” Agile Conference, 2006. N. Zacharis, “Evaluating the Effects of Virtual Pair Programming on Students’ Achievement and Satisfaction,” International Journal of Emerging Technologies in Learning (iJET), vol. 4, no. 3, pp. 34–39, Sep. 2009.
