Supporting Team Members Evaluation in Software ...

CDM 2008

Supporting Team Members Evaluation in Software Project Environments Sergio F. Ochoa, Osvaldo Osorio, José A. Pino Computer Science Department Universidad de Chile

Outline Motivation Application Scenario Related Work

Team Evaluation Method Supporting Tool Preliminary Results Conclusions and Future Work Computer Science Department, Universidad de Chile

CDM 2008 | 2

Motivation Software companies are increasingly encouraging employees to: work cooperatively, coordinate their activities in order to reduce costs, increase the productivity improve quality of its services augment the robustness of the organization.

Available time frames for most of the software projects are quite tight [Reifer 2000, Ochoa, 2006]. Computer Science Department, Universidad de Chile

CDM 2008 | 3

Motivation Team members performance evaluation is the main tool to diagnose how well the group is working [London, 2003; North, 2004]. Many software companies do not evaluate their team performance because the available methods are complex, expensive, slow or error-prone [Stefanov, 2004; Sherestha, 2007]. Current measurement approaches lack of adequate metrics [List, 2005].

Those that evaluate team performance have also to deal with team members feeling about the fairness of such evaluations [Villena, 2008]. Computer Science Department, Universidad de Chile

CDM 2008 | 4

Outline Motivation

Application Scenario Related Work


CDM 2008 | 5

Application Scenario Software development is a collaborative, highly dynamic and stressing activity. Many Web software development projects are characterized by [Ochoa, 2006]: Small/medium size (2-8 months). Time pressure. Goals and scope are poorly defined. Limited budget.

Computer Science Department, Universidad de Chile

CDM 2008 | 6

Application Scenario In such projects, development teams are characterized by: Small development teams (4-7 persons). High number of novice developers. Several team members work in a distributed and asynchronous way. Roles and work protocols are poorly defined.

The project success it is not easy to reach. This paper proposes a Team Evaluation Method. Computer Science Department, Universidad de Chile

CDM 2008 | 7

Outline Motivation



CDM 2008 | 8

Related Work Most available measurement methods in software engineering are oriented to improve the software process or product. Applicable methods come from disciplines such as management or psychology. Employee Performance Appraisal Method [Sherestha, 2007]. Psychological indicators for co-located collaboration [Mandryk, 2004].

Most of them are complex, slow, expensive and/or error-prone [Sherestha, 2007]. Computer Science Department, Universidad de Chile

CDM 2008 | 9

Outline Motivation

Application Scenario Related Work Team Evaluation Method Supporting Tool Preliminary Results Conclusions and Future Work Computer Science Department, Universidad de Chile

CDM 2008 | 10

Team Evaluation Method (TEM)


CDM 2008 | 11

TEM: Evaluation Instances Questionnaire Redefinition

Questionnaire Definition

Evaluation Process

Feedback

Repository of Evaluations


CDM 2008 | 12

TEM: Evaluation Instances Questionnaire Redefinition

Evaluation Process

Questionnaire Definition

Feedback

Repository of Evaluations

D

E

F

…..

D

E

F

…..

D

E

F

Repository of Evaluations Computer Science Department, Universidad de Chile

CDM 2008 | 13

TEM: Application Constraints Small development teams (4-7 persons). The team should be responsible for the final result.

Role of each team member is specified. Hierarchies within the team should not be strong or restrictive. Most tasks to be performed are suitable for group collaboration rather than individual work.

There is trust among team members concerning their teammates’ shared goals and work. Computer Science Department, Universidad de Chile

CDM 2008 | 14

TEM: Internal Evaluations Self-evaluation

Statement

Co-Evaluation

1. The evaluated person considers the project as a team work, offering support for the project tasks. 2. The evaluated person is able to ask for help when having problems. 3. The evaluated person completes the assigned tasks in a good way, making clear the work done and trying to generate as much value as possible during each workday. 4. The evaluated person shows dedication and creativity to achieve project success. 5. The evaluated person shows interest to investigate new solutions and to acquire new skills to be able to complete the assigned tasks. 6. The evaluated person is open to interact with other persons, easing team work. 7. The evaluated person looks for a trusting relationship with the client/user through a continuous interaction (intending to clarify requirements, make changes, contribute to make clear the project progress and validate the work done). 8. The evaluated person is able to accept mistakes made and is open to receive criticism. 9. The evaluated person is objective and accepts changes when needed. 10. The evaluated person prevents knowledge fragmentation within the team, by sharing information and offering timely support. 11. Describe the evaluated person’s strengths. 12. Describe the evaluated person’s weaknesses.


Response Type Typed Typed Typed

Typed Typed Typed Typed

Typed Typed Typed Free Text Free Text CDM 2008 | 15

External Evaluations Managers

Clients

Team Members

Other Development Teams


Users

CDM 2008 | 16

Outline Motivation



CDM 2008 | 17

Supporting Tool: Software Architecture Supported User Types: Users Presentation Layer (Users interface)

Application Layer (Background Processes)

1.

Collaborator

2.

Co-worker

3.

Supervisor

4.

Administrator

5.

External

Data Layer

Data

Services


CDM 2008 | 18

Supporting Tool: Services


CDM 2008 | 19

Supporting Tool: Services


CDM 2008 | 20

Supporting Tool: Internal Evaluation


CDM 2008 | 21

Outline Motivation



CDM 2008 | 22

Preliminary Results TEM was applied in three different scenarios: Two computer science courses. One software company. Case 1

Case 2

Case 3

Course CC61A: Software Course CC62V: Agile Software Company X. Project. Development Workshop. - 1 team. - 3 teams. - 2 teams. - 5 team members - 5-6 members per team. - 6-7 members per team. - 2 clients (external - 2 clients (external - 2 clients and 2 users (external evaluators). evaluators). evaluators).


CDM 2008 | 23

Preliminary Results A total of 406 assessments were gathered by the tool. No evaluations were in the “undelivered” category. Internal evaluations score increased between 1030% for every team between the first and second applications. The external evaluations indicate most of the stakeholders have observed an improvement of the team behavior after the first internal evaluation. Computer Science Department, Universidad de Chile

CDM 2008 | 24

Preliminary Results The feedback provided by evaluators was rich and informative for the team members.

Team members feel comfortable with the evaluation method.


CDM 2008 | 25

Outline Motivation



CDM 2008 | 26

Conclusions TEM was designed to evaluate the software team and their members’ performance. The method is simple, fast and involves low application cost. Team members and stakeholders had positive opinions. TEM could be useful not only to evaluate team members’ performance, but also to help the team members to improve their individual and collective behavior. Computer Science Department, Universidad de Chile

CDM 2008 | 27

Future Work Formal experiments are planned.

We need formally identify the advantages and disadvantages of the method. We want to study the trade-off between time invested on evaluations and value obtained with them.


CDM 2008 | 28

Thank you …


CDM 2008 | 29

References [List, 2005] List, B., Bruckner, R., Kapaun, J. (2005) Holistic Software Process Performance Measurement from the Stakeholders' Perspective. First Int. Workshop on Business Process Monitoring & Performance Management (BPMPM’05), Copenhagen, Denmark, IEEE CS Press. [London, 2003]. London, M. (2003). Job Feedback: Giving, Seeking and Using Feedback for Performance Improvement (2 ed.), New Jersey, Lawrence Erlbaum. [Mandryk, 2004] Mandryk, R.L., Inkpen, K. (2004). Physiological Indicators for the Evaluation of Co-located Collaborative Play. Computer Supported Cooperative Work (CSCW). Chicago, IL. [North, 2004]. North, A. (2004), Introduction to Performance Appraisal. URL: http://www.performanceappraisal.com/intro.htm. [Ochoa, 2006] Ochoa, S., Pino, J., Guerrero, L., Collazos, C. (2006). SSP: A Simple Software Process for Small-Size Software Development Projects. First IFIP Int. Workshop on Advanced Software Engineering, Santiago, Chile. Springer Science + Business Media. Vol. 219. pp. 94-107. [Reifer, 2000] Reifer, D.J. Web Developement: Estimating Quick–to-Market Software, IEEE Software Nov./Dec. 2000. [Sherestha, 2007] Sherestha, S., Chalidabhongse, J. (2007). Improving Employee Performance Appraisal Method through Web-Based Appraisal Support System: System Development from the Study on Thai Companies, IEICE Trans. on Information and Systems, E90-D(10), 1621-1629. [Villena, 2008] Villena, A. (2008) An Empirical Model to Teach Agile Software Process. Master’s Thesis. University of Chile. Computer Science Department, Universidad de Chile

CDM 2008 | 30