Using the Iterated Prisoner's Dilemma for Explaining the ... - CiteSeerX

Using the Iterated Prisoner's Dilemma for Explaining the Evolution of Cooperation in Open Source Communities Daniel Eckert

Stefan Koch

Institute of Public Economics, Karl-Franzens University Graz, Austria [email protected]

Johann Mitlöhner

Department of Information Business, University of Economics and BA Vienna, Austria [email protected]

Abstract – Software development, and especially open source projects, typically involve repeated interactions between participants and groups of participants. We propose to analyse this situation by means of the standard model for the evolution of cooperation, the iterated prisoner's dilemma. The prisoner's dilemma is a well-known model for a two-person game, in which each side can choose to either cooperate or defect, and in which the payoffs are arranged in a defined hierarchy (e.g. the highest payoff is achieved by defecting while the other player cooperates). As a first step, the prisoner's dilemma needs to be formulated for the open source development model, i.e. what constitutes cooperation, playing defect and payoffs. Then, computer simulations using a population of stochastic reactive strategies can be applied, using a strategy's payoff as fitness measure for determining its frequency in the next generation. As a further extension, the effects of misinterpretation of other's behaviour can be included into the model. We will discuss which insights into open source software development can be gained by applying this model.

I. INTRODUCTION Software projects typically consist of multiple steps with several partners cooperating to achieve a certain goal. The dynamics of these interactions are an important factor influencing software development risk, especially because of misunderstandings possible due to differences in organizational culture or group selective behaviour. We propose to analyse this situation by means of the standard model for the evolution of cooperation, the iterated prisoner's dilemma, and will discuss whether and in which form this model can be applied to open source software development. This form of software development very much relies on the participants' cooperation, and it is not yet clear how this cooperation forms and evolves. It has been assumed that common beliefs and values play an important part in this process [1], which are increasingly reinforced during participation [2]. II. GAME THEORY AND COMMERCIAL SOFTWARE DEVELOPMENT A. Prisoner's Dilemma Limiting the analysis to projects with two participating individuals or teams we model all interactions as a 2person game with the strategies cooperate (C) and defect (D) available to both players. The payoff matrix defines the utilities for the row and column players depending on their respective strategy choices (see Table I.).

[email protected]

TABLE I. PAYOFF MATRIX OF GENERIC PRISONER'S DILEMMA

C

D

C

(R,R)

(S,T)

D

(T,S)

(P,P)

Given this situation, the so-called prisoner's dilemma is realized with T>R>P>S: both partners opting for Defect results in the low payoff (P,P), while both partners chosing Cooperate returns the higher payoff (R,R). However, the highest payoff T, termed temptation, can be realized by defecting on a cooperating partner, who in turn receives the lowest payoff S. An example in terms of negative utilities can be formulated using costs associated with information exchange: by chosing to cooperate and expending documentation costs d player A can document their part of the system sufficiently so that player B can readily use player A's modules; if player A plays Defect then this documentation is lacking, and player B has to expend higher costs e to derive the necessary information. The payoffs in terms of negative utility are R = -d, P = -e, T = 0, S = -d-e. With e>d the payoffs of the prisoner's dilemma are again realized. B. Iterated Prisoner's Dilemma For a fixed number of iterations which is known in advance to the players the dominant strategy is Defect since it maximizes minimal payoff for each player. However, in an iterated prisoner's dilemma where the number of iterations is not initially known to the players cooperative outcomes are possible [3]. Software development projects typically involve repeated interactions of the participants during their lifecycle. Due to changes of schedule and other factors the number of interactions is not known initially. Therefore, the iterated prisoner's dilemma is a plausible model of the development process. In an iterated game cooperative and non-cooperative behaviour will typically be reciprocated to a certain extend. The players are influenced in their choice of strategy by the opponent's previous move, if they are able to perceive it. Such reactive strategies in the iterated prisoner's dilemma can be modelled stochastically. A stochastic reactive strategy specifies the conditional probabilities of C and D as a reaction to all possible histories of the game that fall into the memory of the player. In the simplest case, where the initial probability to play C in the first round can be ignored due to the infinite iteration of the game, and where the memory of the players

Proceedings of the First International Conference on Open Source Systems Genova, 11th-15th July 2005 Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191

is minimal, taking account only of the opponent's move in the previous round, a reactive strategy E can be determined by the pair (p,q), where p is the conditional probability to play C after an opponent's C in the previous round, and q is the probability to play C after an opponent's D. Some wellknown strategies have been termed Tit for Tat, which can be represented by the pair (1,0), generous TFT by (1, 0.3), the random strategy by (0.5, 0.5) and AlwaysD by (0,0). Nowak and Sigmund [4] have shown that the payoff A for strategy Ei against strategy Ej the infinitely iterated prisoner's dilemma is A(Ei,Ej) = [(R-T)+(P-S)] c c' + (S-P) c + (T-P) c' + P

(1)

with c = (qi+(pi-qi)qj) / (1-(pi-qi)(pj-qj))

(2)

population is necessary to bring about a cooperative outcome highlights the importance of a minimal social structure that is required for the evolution of cooperation. Even if Tit for Tat can be proven to be the strategy that can invade a population of universal defectors in the smallest cluster [6], its role shows that the prestructuration of the population determines the evolution of the patterns of interaction that constitute the final social structure. The precise impact of the prestructuration of the population depends on the degree of its incorporation into the strategies: As Frank [7] has shown, if cooperators can recognize each other with the help of some label they can increase their payoff by interacting selectively with one another. This mechanism can even explain the spontaneous emergence of label-selective behaviour, as Rick Riolo quoted by Holland [8] has shown with the help of computer simulations. C. Group Selective Behaviour and Misinterpretation

and c' = (qj+(pj-qj)qi) / (1-(pj-qj)(pi-qi))

(3)

for 0I, because A would have been more familiar with the problem, or for similar reasons. If both players are cooperating they invest their own efforts, and both get the resulting software. If both players were to defect, no effort is invested and no software is produced. This results in the payoff matrix depicted in Table II. F. Group Selective Behaviour and Misunderstandings The presence of group selective behaviour might be of interest in the context of open source software development, especially when considering the whole user group as players: If a player knows that the other player is sharing the same norms and beliefs, e.g. on freedom of software, he can be more sure of the other player's cooperation, and might himself cooperate to a larger degree. In this case, group selective behaviour might indeed be present in open source software development.

Proceedings of the First International Conference on Open Source Systems Genova, 11th-15th July 2005 Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191

TABLE II. PAYOFF MATRIX IN OPEN SOURCE SOFTWARE DEVELOPMENT

C

C

D

(F-I,F-I)

(F-I-I',F)

f i x=r n f in xr a f ia x .

D (F,F-I-I') (0,0) The sometimes elaborate joining rituals and scripts [20,1,2] might then be interpreted as a mechanism implemented to reduce the possibility for misunderstandings: Anyone having joined through these rituals and scripts shares these norms and beliefs, and can easily be identified as such a player, who therefore is less likely to defect. Therefore noise is reduced, which can lead to a more cooperative outcome even if discrimatory strategies are present. We construct group selective strategies EJ (for Mr. Jekyll) and EH (for Mr. Hyde) such that for each nondiscriminatory Jekyll strategy with basic p and q values pi and qi both in inter- and intra-group interaction EiJ=((pi,qi), (pi,qi)) there is a corresponding Hyde strategy EiH=((pi*,qi*), (pi*,qi*)) with pi*=pi and qi*=qi in case of intra-group interaction, and pi*=qi*=θ=0.001 in case of inter-group interaction: i.e. EiH=((pi,qi),(θ,θ)). Each of these two subpopulations of strategies is characterized by a frequency vector consisting of the frequencies of the corresponding strategies and zeros otherwise such that xJ + xH = x. The fitness fi(x) of a given strategy i within a population of m strategies consists of the contributions of fitness it achieves when played by members of the own (native) group (fin(x)) and the alien group (fia(x)) weighted by their respective population shares rn and ra . We denote by πij the proportion of group is contacts that are with js (under the idealizing assumption characterizing our model πnn = rn and πna = ra: m

f in x =∑ nn x j A E i , p j ,q j 

(7)

j=1

na[ x Jj A E i , p j ,q j x Hj A E i ,,] n

f ia x =∑ aa x j A E i , p j ,q j 

(8)

j=1

an[ x Jj A E i , p j ,q j x Hj A E i ,,]

The total fitness fi is simply (9)

If we denote by xin and xia the frequencies of strategy i in native and alien use, resp., the frequencies in time step t+1 are given by x int1=x int  f in x/ fn

(10)

a a a a x i t1=x i t  f i  x / f

(11)

and

with the average native and alien fitness values n

fn=∑ x in f in x

(12)

i=1

and n

fa=∑ x ia f ia x .

(13)

i=1

The total frequency of strategy i is x it1=r n x int1r a x iat1.

(14)

On the basis of this model we investigated the dynamic behaviour of the strategy population. Following Nowak and Sigmund [21] we initialized the population with 100 pairs of p- and q-values distributed uniformly in the unit square; for each pair we added a duplicate strategy: a Mr. Hyde lurking behind his Dr. Jekyll sibling and ready to defect in inter-group interaction. All 200 strategies have the same initial frequency of 0.005, allowing an equal proportion of Jekylls and Hydes. We ran 1000 time steps since preliminary tests showed in accordance with the results of Nowak and Sigmund that interesting dynamics occur only after the first several hundred time steps. We set the proportion of aliens to 0.1. Figure I shows the result of a typical run, where typical is defined as resulting in a total fitness value with the smallest deviation from the average of 20 runs with different random numbers. Shortly after t=200 the Tit-for-Tat Jekyll strategies take over the population, drastically improving native and alien fitness; after t=400 there is a shift towards Generous TFT which further increases fitness.

Proceedings of the First International Conference on Open Source Systems Genova, 11th-15th July 2005 Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191

FIGURE I. NON-DISCRIMINATORY OUTCOME IN SPITE OF DISCRIMINATORY STRATEGIES

IV. CONCLUSION The main question remaining is what is to be gained by applying the iterated prisoner's dilemma in the context of open source software development. If the model can indeed be used, it might allow for several insights into this phenomenon. These insights can be roughly grouped into two different levels: The open source idea in general and its proliferation, and the situation in specific projects. On the general level, this model gives an explanation why open source software development has arisen and is prospering. Evolving cooperation and thus formation of open source development communities can be seen as a natural choice. Therefore it strengthens other explanatory approaches [22,23] in the reasoning that this idea is not strictly motivated by altruism or similar motivations, but a sound decision. Given a certain software functionality, the possible user base can be estimated, and starting with a certain rate of cooperators, the evolution of cooperation within the total population can be simulated. This gives a positive outlook for the open source movement, as prior results indicate that even a small rate of cooperators might be sufficient to establish cooperation in the long run. On the level of specific projects, the environment, especially any steps taken to reduce noise and thus possibilities for misunderstandings seem to gain prominence. Indeed, the joining scripts and even rituals in some projects might in this way find a grounding in game theory and can thus be explained. Their presence and success might be found to be necessary for achieving a cooperative situation and thus form a critical success factor for open source projects. Group-selective behaviour is interesting to explore in hybrid or gated open source communities [24], composed of a mixture from a leading company's employees and voluntary developers. Between those groups selective behaviour might arise and endanger a cooperative, i.e. successful, outcome. Data currently available on open source projects and communities could be used to document the situation in these environments, i.e. the population of strategies in place. This could be used as a starting point for simulations of future behaviour, in order to get an understanding of whether and when a cooperative outcome will result.

V. REFERENCES [1]

M.S. Elliott and W. Scacchi, “Free software development: Cooperation and conflict in a virtual organizational culture”, in S. Koch (ed.), Free/Open Source Software Development, Hershey, PA: Idea Group Publishing, 2004.

[2]

E.G: Coleman and B. Hill, “The social production of ethics in debian and free software communities: Anthropological lessons for vocational ethic”, in S. Koch (ed.), Free/Open Source Software Development, Hershey, PA: Idea Group Publishing, 2004.

[3]

R. Axelrod, The Evolution of Cooperation, London: Penguin Books, 1984.

[4]

M. Nowak and K. Sigmund, “Game dynamical aspects of the prisoner's dilemma”, J. Appl. Math. Comp., vol. 30, 1989, pp. 191-213.

[5]

J. Hofbauer and K. Sigmund, The Theory of Evolution and Dynamical Systems, Cambridge: Cambridge University Press, 1988.

[6]

M. Nowak and K. Sigmund, “The evolution of stochastic strategies in the prisoner's dilemma”, Acta Applicandae Mathematicae, vol. 20, 1990, pp. 247265.

[7]

R. Frank, Passions within Reason. The strategic role of the emotions, New York: W.W. Norton & Co., 1988.

[8]

J.H. Holland, Hidden Order - How adaptation builds complexity, Reading, Mass.: Addison-Wesley, 1995.

[9]

R. Axelrod and J. Wu, “How to cope with noise in the iterated prisoner's dilemma”, Journal of Conflict Resolution, vol. 39, 1995, pp. 183-189.

[10] E.S. Raymond, The Cathedral and the Bazaar, Cambridge, Massachusetts: O’Reilly & Associates, 1999.

Proceedings of the First International Conference on Open Source Systems Genova, 11th-15th July 2005 Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191

[11] G.M. Weinberg, The Psychology of Computer Programming, Silver Anniversary Edition, New York: Dorset House Publishing, 1998.

framework for creating hybrid-OSS communities”, Information Systems Journal, vol. 12, no. 1, 2002, pp. 7-25.

[12] B. Perens, “The Open Source Definition”, in DiBona, C. et al. (eds.), Open Sources: Voices from the Open Source Revolution, Cambridge, Massachusetts: O’Reilly & Associates, 1999 [13] R.M. Stallman, Free Software, Free Society: Selected Essays of Richard M. Stallman, Boston, Massachusetts: GNU Press, 2002. [14] W. Janko and J. Mitlöhner, “Ein spieltheoretisches Modell der Berater-Kunden-Interaktion in ITProjekten”, in W. Gaul and M. Schader (eds.) Mathematische Methoden der Wirtschaftswissenschaften, Heidelberg: Physica-Verlag, 1999. [15] A. Mockus, R. Fielding and J: Herbsleb, “Two case studies of open source software development: Apache and Mozilla”, ACM Transactions on Software Engineering and Methodology, vol. 11, no. 3, 2002, pp. 309-346. [16] S. Koch, “Profiling an Open Source Project Ecology and Its Programmers”, Electronic Markets, vol. 10, no. 2, 2004, pp. 77-88. [17] A. Hars and S. Ou, “Working for free? - Motivations for participating in Open Source projects”, in Proceedings of the 34th Hawaii International Conference on System Sciences, Hawaii, 2001. [18] G. Hertel, S. Niedner and S. Hermann, “Motivation of software developers in open source projects: An internet-based survey of contributors to the Linux kernel”, Research Policy, vol. 32, no. 7, 2003, pp. 1159-1177. [19] A.J. Albrecht and J.E. Gaffney, “Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation”, IEEE Transactions on Software Engineering, vol. 9, no. 6, 1983, pp. 639-648. [20] G. von Krogh, S. Spaeth, and K.R. Lakhani, “Community, joining and specialization in open source software innovation: A case study”, Research Policy, vol. 32, no. 7, 2003, pp. 1217-1241, 2003. [21] M. Nowak and K. Sigmund, “Tit for tat in heterogenous populations”, Nature, vol. 355, January 1992. [22] R.A. Ghosh, “Cooking pot markets: an economic model for the trade in free goods and services on the Internet”, First Monday, vol. 3, no. 3, March 1998. [23] J. Lerner and J. Tirole, “The simple economics of Open Source”, Working Paper 7600, National Bureau of Economic Research, 2000. [24] S. Sharma, V. Sugumaran and B. Rajagopalan, “A Proceedings of the First International Conference on Open Source Systems Genova, 11th-15th July 2005 Marco Scotto and Giancarlo Succi (Eds.), pp. 186-191