A Non-Cooperative File Caching for Delay Tolerant ...

A Non-Cooperative File Caching for Delay Tolerant Networks: A Reward-based Incentive Mechanism Sidi Ahmed Ezzahidi* , Essaid Sabir† , Mohamed El Kamili‡ and El-Houssine Bouyakhf* * LIMIARF, University of Mohammed V-Agdal, B.P. 1014 RP, Rabat, Morocco † UBICOM Research Group, ENSEM, Hassan II University of Casablanca, Morocco ‡ LIMS Lab., FSDM, Sidi Mohamed Ben Abdellah University of Fez, Morocco [email protected], [email protected], [email protected], [email protected] Abstract—This paper introduces a reward-based incentive mechanism for file caching in Delay Tolerant Networks (DTNs). In DTNs, nodes use relay’s store-carry and forward paradigm to transmit data till final destinations under intermittent connectivity. However, the relays are not always available to assist data transmission due to limited energy or low storage capacity. Our proposal is based on a reward mechanism to sustain cooperation among relays. We model this distributed network problem as a non-cooperative game. On one hand, the source offers to the relays a positive reward if they accept to cache and to forward a given file successfully to a target destination. On the other hand, the relays can either accept or reject the source offer, depending on the reward value and the expected energy consumption of the caching-forwarding operation. Next, a full characterization of the equilibria of this game is provided. Then, we propose two fully distributed algorithms to discover the game Nash equilibria, both for pure/mixed equilibria and discrete/continuous strategy sets. We validate our proposal using extensive numerical examples and numerous learning simulations, and draw some conclusions and insightful remarks.

I. I NTRODUCTION Delay Tolerant Networks (DTNs) are complex distributed systems where a permanent path between pair nodes sourcedestination is hard to be completed due to the lack of the network infrastructures. Nevertheless, a huge delay may be tolerated in such a network and the storing-carrying-forwarding paradigm of relays is designed to maintain a complete endto-end transmissions. Then, the relays can store and carry a given file and wait until finding a contact opportunity and forward it to target destination. These relays are generally mobile devices with limited battery and storage capability, hence the fact that the relays are not always available to assist the file transmission is plausible. Therefore, building a scheme inciting cooperation by accepting to cache and forward files is promising and could improve significantly the DTNs’ performance. A. Literature review In literature, there are many works propose the solutions to optimize DTNs performance. Indeed, there are some focus on the routing, others analyze the buffer management, however all these proposed works have almost the same objectives. They aim to reduce the rate of loss, delay from end to end, the number of transmissions, energy consumption, and to maximize the rate of average message delivery. However, most of This work was supported by the Mobicity Project funded by The Moroccan Ministry of Higher Education and scientific research.

their results are difficult to generalize due to the assumptions proposed to simplify their models. In [1], the authors present a new policy to optimize DTNs routing algorithms that maintain the network relays operational for a longer period of time and select the best relay to forwarding a file during its life time, but they did not consider the energy constraint, however in practice the relay battery energy is limited. Neglia et. al. [2] propose a activation forwarding control at relay, but their optimization suppose that relay is always available to cooperate with the source. Contrariwise, the relay can not always be active due to limited energy or buffer space. Yue Lu et. al.[3] study how energy harvesting can be exploited to improve the performance of opportunistic forwarding in mobile DTNs. In [4] the authors study under energy and buffer space constraint how the relay act and form coalitions to help each other to forwarding a given file. Furthermore, several studies give a very high importance to the buffer management, e.g., Amir kerifa et. al.[5] propose an optimal buffer management policy under assumption that there is unlimited bandwidth, which is far from reality. Wang and al.[6] consider the popularity of the files is the basic parameter to distribute its on relays buffer, but the model is simplified with the assumptions that the files have the same size and the density of relays is sufficient for file transmission to the destination. B. Our Contribution In this work, we introduce a mechanism for file caching in DTNs by offering some reward to relays nodes willing to cooperate. The mechanism consists of asking the relays to cache a file according to an incentive rewarding offer. Yet, the source is offering a reward for each relay accepting to cache the file and succeeding to forward it to the final destination. However, the relays can either accept or decline the offer depending on the reward value and its battery status. We use the two-hop routing [7] to route the files to their destinations, the source attempts to transmit the file to any relay that it encounters, and the relays are allowed to transmit the file only to the destination node. The choice of the two-hop routing is due to its efficient management of resources such as bandwidth, buffer space and relay energy. We construct a non-cooperative game to study the competition between the relay and the source. We define the pay-off of each player as a linear function of the contact probability, the delivery probability and the energy

consumption by caching-forwarding operation. Each relay has two pure strategies: “accept” and “reject”. However, a relay is allowed to mix between her two actions and thereby can choose her strategy according to some probability distribution (mixed strategy). Furthermore, each relay accepting to cache a file should forward it to the final destination within its lifetime, likewise its not beneficial for her if another relay forwarded this file earlier. More precisely, the relay has a decision-making problem which consists to play under its available energy constraint (battery powered), the reward value and the probability that another relay had already delivered transmitted file. The rest of this paper is organized as follows. In section II, we describe the problem, interaction details and utilities formulation. We analyze the game equilibrium section III and describe some learning schemes in section IV. Numerical results and conclusion are presented in section V and section VI respectively. II. SYSTEM MODEL Consider a DTN network with a single source, a single destination, and n relays which are involved in the transmission of files. The files are generated at the source and the relays should reach them at their destination. Each file has a utility time h (finite horizon) during which the destination is interested for its content. We assume that the relays can store one file and should transmit it only to the destination (i.e., two-hop routing). Moreover, the contact time is quite sufficient to successful the file-carrying transmission when the link between nodes is up, and the inter-contact times between any pair of nodes are independent identically distributed (i.i.d.) random variables. Furthermore, the source rewards the relay for agreeing to caching the file, each relay accepts a new file and succeed to deliver it to a target destination receives a reward (each file has its own reward) and each transmission attempt incurs a cost which is the energy consumed. Moreover, Several previous work (e.g., [8]) claimed that the inter-contact time between a pair of relays follows an exponential distribution with rate λ. In this paper we assume, without loss of generality and clearness, that every node has the same contact probability. A. Source-Relay Interaction and Reward Mechanism As mentioned above, at each encounter the relay chooses either accept or reject a given file, hence the source adopts a payment strategy to encourage the relay to accept, it proposes for it a positive reward1 α ≥ 0 if it accepts to caching and relaying successfully this file to a target destination. The figure thereafter explain this interaction. Source problem: Whenever the source generates a new file, it associates to it a value α ∈ [0, αmax ]. This value may translate an additional cost to be experienced by the source in exchange for successfully forwarding the file. Thus, the source objective is related to minimize this value while ensuring that the relay accepts the file. When the contact takes place, if the relay accepts and delivers the file to destination 1 The reward could be a certain virtual credit or an amount of bit-coins that the source offers to send its own files over the DTN network.

I have to pay less to maximize my income...

I am offering $α to the first node to forward this packet to destination.

?

Should I accept this reward offer? What about Delay, Energy, Buffer, ...?

?

Should I accept this reward offer? What about Delay, Energy, Buffer, ...?

Relay Relay

Destination Source

?

Should I accept this reward offer? What about Delay, Energy, Buffer, ...?

Relay

Fig. 1.

Interaction between source and relay

within time t ≤ h, the source gives (pays) to it the reward α which is the associated value of the file. The source is assumed to yield αmax − α. Otherwise, if the relay fails to deliver the file to the destination, the source loses all of the value, i.e., αmax . In case the relay rejects the caching offer, the source receives a penalty (αmax − α). Relay problem: When a contact with the source and a caching offer take place, the relay sets a control admission on the file, i.e., the relay has two options: either to accept ’a’ or to reject ’r’ the file. The decision making depends on the utility function which is function of the value α and the energy consumed while relaying the file. Yet, when a success transmission occurs (i.e. the relay delivers the file within the file lifetime h), the relay receives a positive reward α (the file value). Moreover, the relay receives a negative regret β if it accepts the file but fails to forward it to the destination, or γ if it rejects this file and −α if it rejects and it encounters the destination within utility timeh. Furthermore, each transmission attempt has a cost, which is the energy provided to relaying. The source could provide only the energy σt consumed while transmitting a file in time slot and the relay consumes σh the average energy needed when forwarding a file during h. Z h (1 − (1 + λh)e−λh ) σh = E(σ) = σλte−λt dt = σ[ ]. λ 0 (1) According to [11], the probability that a single relay delivers a file to destination within time h is given by 1 − Qh , where Qh = (1 + λh)e−λh ,

(2)

is the probability that a relay is not succeeding in file relaying to destination. B. The Utility Functions i According to [11], we define Psucc (z) the probability that a given relay i among z, played strategy ’a’ and succeed to deliver a given file ( i.e., the first to deliver the file to the destination) as following  z  X z − 1 (1 − Qh )j−1 (Qh )z−j i Psucc (z) = (1 − Qh ) j−1 j j=1

1 − Qzh . (3) z Let us denote by na the number of relays accepting to cache the file among k relays having encountered the source. Relay’s Utility: The utility function of relay i is defined to be the difference between the reward received from the source and the energy consumed during caching-forwarding operation. We =

denote Ui (0 a0 , α) the utility function of the relay when it plays its pure strategy accept ’a’, and Ui (0 r0 , α) if it plays its pure strategy reject ’r’.  0 0 i i  Ui ( a , α) = αPsucc (na ) − β(1 − Psucc (na )) − σh 

(4)

i Ui (0 r0 , α) = −αPsucc (na ) − γ

Source’s Utility: The utility function of source is denoted to be the difference between the unit reward that it is willing to pay and the energy consumed. i i Us (0 a0 , α) = na (αmax − α)Psucc (na ) − na αmax (1 − Psucc (na ))

−kσt − (k − na )(αmax − α), i i Us (0 a0 , α) = (k − na (Psucc (na ) + 1))α − αmax (k − 2na Psucc (na ))

−kσt .

In this section, we study the full conditions that drive the system to a equilibrium point with reward-based incentive mechanism where both the source and the relays know all system parameters mentioned above. Before we begin to study the stable equilibrium strategy, we briefly recall the noncooperative game in one shot (static game) with complete information and the corresponding results. The existence of a Nash equilibrium [9] in pure strategies for the game one shot is ensuring according to the Theorem Debreu-Fan-Glicksberg [10], when The continuity of each utility function Ui in p = (p1 , ......, pn ) and the quasiconcavity of Ui in pi on compact convex sets. The uniqueness of equilibrium will be achieved with the correspondence of the best response.

We are in the case where the relays adopt a pure strategy, they will either accept or reject. Let the game (Ana , Rk−na , α) with Ana =(’a’,’a’,...’a’) is the profile that na relays play strategy ’a’ and Rk−na =(’r’,’r’,...’r’) is the profile that k − na play strategy ’r’. Lemma 1: The profile (Ak , R0 , 0), (resp. (A0 , Rk , αmax )) is a pure Nash equilibrium iff: i σh ≤ γ − β(1 − Psucc (k)), i σh ≥ γ − β + (2αmax + β)Psucc (k).

(6)

Proof: In the case where, all relays k play their strategy ’a’ that means, for i ∈ k, Ui (0 a0 , α) ≥ Ui (0 r0 , α),

i i i αPsucc (k) − β(1 − Psucc (k)) − σh ≥ −αPsucc (k) − γ.

(7)

Since na = k, The source’s utility becomes, Us ( a , α) =

− kαmax (1 −

i 2Psucc (k))

− kσt ,

which is a linear function of which the slope is negative, then it reaches its maximum for α = 0. By substituting α = 0 into (7) we obtain −β(1 −

i Psucc (k))

Since na = 0, The source’s utility becomes, Us (0 a0 , α) = kα − kαmax − kσt ,

which is a linear function of which the slope is positive, then it reaches its maximum for α = αmax . By substituting α = αmax into (8) we obtain i ⇒ σh ≥ γ − β + (2αmax + β)Psucc (k).

Lemma 2: The profile (Ana , Rk−na , 0), (Ana , Rk−na , αmax )) is a pure Nash equilibrium iff:

(resp.

λβk λ(2β + γ) + σ(1 − Qh ) (resp. λ(2αmax + β)k na ≤ ) λ(2β + 2αmax − γ) + σ(1 − Qh ) na ≥

(9)

Proof: Consider that na relays play strategy ’a’ and k−na play strategy ’r’. For i ∈ na , Ui (0 a0 , α) ≥ Ui (0 r0 , α).

For i ∈ k − na , Ui (0 a0 , α) ≤ Ui (0 r0 , α).

Then the equilibrium is given by the indifference property Ui (0 a0 , α) = Ui (0 r0 , α),

A. Pure Nash Equilibrium

i −kPsucc (k)α

i i i −αPsucc (k) − γ ≥ αPsucc (k) − β(1 − Psucc (k)) − σh . (8)

(5)

III. NASH E QUILIBRIUM A NALYSIS

0 0

Ui (0 r0 , α) ≥ Ui (0 a0 , α),

i i i −αmax Psucc (k) − γ ≥ αmax Psucc (k) − β(1 − Psucc (k)) − σh

which can be arranged as



In the case where, all relays k play their strategy ’r’ that means, for i ∈ k,

− σh ≥ −γ ⇒ σh ≤ γ − β(1 −

i Psucc (k)).

i i i αPsucc (na ) − β(1 − Psucc (na )) − σh = −αPsucc (na ) − γ. (10)

The source’s utility, i i Us (0 a0 , α) = (k − na (Psucc (na ) + 1))α − αmax (k − 2na Psucc (na ))

−kσt .

which is a linear function, hence it can reach its maximum, (1) for α = 0 if its slope is negative, i.e., i k − na (Psucc (na ) + 1) ≤ 0 ⇒ na ≥

k . i (Psucc (na ) + 1)

By substituting α = 0 into (10) we obtain i −β(1 − Psucc (na )) − σh = −γ

i ⇒ Psucc (na ) =

λ(β − γ) + σ(1 − Qh ) λβ

which yields the first condition. (2) For α = αmax if its slope is positive, i.e., i k − na (Psucc (na ) + 1) ≥ 0 ⇒ na ≤

k i (Psucc (na ) + 1)

By substituting α = αmax into (10) we obtain i i i αmax Psucc (na ) − β(1 − Psucc (na )) − σh = −αmax Psucc (na ) − γ

Thus i Psucc (na ) =

λ(β − γ) + σ(1 − Qh ) λ(2αmax + β)

Summary: the mixed Nash equilibrium is

which completes the proof of the second condition. In the next section, we use another concept of equilibrium, named mixed Nash equilibrium, in which the relay will accept to cache the file with some probability. B. Mixed Nash Equilibrium When mixed strategy is allowed, the relays randomize between accepting and rejecting the source offer according to common probability distribution, such they can play a strategy ’a’ with probability pi ∈ [0, 1] and play ’r’ with probability 1 − pi . We consider the symmetric case, that means, all relays accept with the same probability, i.e, ∀i ∈ n pi = p. For this, the excepted utility of a given relay becomes, ! n − 1 na Ui (p, α) = p p (1 − p)n−na −1 Ui (0 a0 , α) + n a na =0 ! n−1 X n−1 n p a (1 − p)n−na −1 Ui (0 r0 , α). (11) (1 − p) n a n =0 n−1 X

a

When a mixed Nash equilibrium is achieved, each relay becomes indifferent about which strategy to choose. Namely Ui (0 a0 , α∗ ) = Ui (0 r0 , α∗ ), i i i α∗ Psucc (p∗ , n)−β(1−Psucc (p∗ , n))−σh = −α∗ Psucc (p∗ , n)−γ, (12)

where i Psucc (p, n)

! n X n − 1 (Z)n−1 (1 − Z)n−j j n−1 j=1

=

Z

=

1 − (1 − Z)n , n

with Z = p(1−Qh ). Next, we solve equation (12) and obtain λβ(n − 1 + (1 − (1 − Qh )p∗ )n ) − n(λγ − σ(1 − Qh )) . α∗ = 2λ(1 − (1 − (1 − Qh )p∗ )n ) (13)

 ∗   α =

Us (p, α) =

na =0

=

n X na =0

IV. L EARNING ALGORITHMS In this section we introduce a distributed reinforcement learning algorithm for the relays and the source to allow them to take the best decision locally and independently over time, i.e., they use the local observations to estimate their payoffs. The source chooses a real value α at each instant k depending on the current strategy probability distribution of relays, it has a problem to provide the optimal value of α that permits it to maximize its play-off, it takes this value continually and independently in interval [0, αmax ] based on local observations, in this case, it estimates its own pay-off at each interaction. For solving this source optimization problem we use the stochastic approximation scheme, precisely the kiefer-Wolfowitz scheme [12]. The relays problem consists to choosing the best strategy that allows them to maximize their pay-offs depending on the value of α provided by source, they play their pure strategies according to probability distribution, because they take the decision locally and independently based their probability distribution, we use Linear Reward-Inaction (LRI) algorithm to seek the pure Nash equilibria, and Gibbs Distribution algorithm to converge to mixed Nash equilibrium. A. Kiefer-Wolfowitz stochastic approximation algorithm In section III, we proved that the source utility U s has amaximum at α∗ , but in this case this utility is unknown, however, only certain observations Uˆs (α ˆ ), such as Us = E[Uˆs (ˆ α)|ˆ α]. Then the Kiefer-Wolfowitz stochastic approximation algorithm is : α ˆ k+1 = α ˆ k + µk ∇Uˆs (α ˆ k ),

where the estimated gradient is given by ∇Uˆs (α ˆk ) =

X

!

n i pna (1 − p)n−na ((n − na (Psucc (na ) + 1))α na

(14)

At Nash equilibrium, the first order derivative of the source utility vanishes. Namely ∂Us (p∗ , α) = 0, ∂α n − 1 + (1 − (1 − Qh )p∗ )n − np∗ = 0,

(17)

1 ˆ (Us (αˆk + ck ) − Uˆs (α ˆ k − ck )) ck

(18)

If the step sizes µk and ck satisfy

! n pna (1 − p)n−na Us (0 a0 , α) na

i −αmax (n − 2na Psucc (na )) − nσt ) = (n − 1 + (1 − (1 − Qh )p)n − np)α − αmax (n − 2 + 2(1 − (1 − Qh )p)n ) − nσt .

(16)

  p∗ is the unique solution of the equation (15)

Now, the excepted utility of the source becomes n X

λβ(n−1+(1−(1−Qh )p∗ )n )−n(λγ−σ(1−Qh )) . 2λ(1−(1−(1−Qh )p∗ )n )

(15)

The left side of previous equation is continuous, strictly ∗ decreasing monotone function on [0, 1] with ∂Us (0,α ) ∂Us (1,α∗ ) n = n > 0 and = Q ∗ ∗ h − 1 < 0, then ∂α ∂α ∂Us (p∗ ,α∗ ) ∗ = 0 has an unique solution p . ∂α∗

k

µk = ∞;

X k

µk ck < ∞;

X µk 2 ( ) < ∞; µk < ρc2k , ck

(19)

k

where 0 < ρ < ∞, hence the algorithm converges to local maximum of Us under some conditions on the function Us (·) ˆ (ˆ and the variance of the observations U α). The usual choices µ1 c1 for step sizes is ck = kγ , µk = k , 0 < γ < 12 and µ1 , c1 > 0. B. Linear Reward-Inaction (LRI) algorithm We propose Linear Reward-Inaction (LRI) algorithm that permits the relays to learn the optimal strategy among reject or accept (pure Nash equilibria) according to the reward estimate α ˆ provided by source. they update the strategy probability distribution independently over time with a view to take the best strategy for them. Initially, they choose strategy based on the strategy probability distribution, after each time instant it increases the strategy probability of whose the reward α is maximal and decreases other probability, according to the following rule: pk+1 = pk + ηk Ui (ek , α)(e ˆ k − pk ),

(20)

−9

where ek is the unit vector with the ith component unity corresponding to the action selected at k. And 0 < ηk < 1 is the learning parameter at the instant k. pk = [pk , 1 − pk ]T is the strategy probability vector at instant k.

−10

Source Utility

−11 −12 λ=1 λ=0.3 λ=0.1 λ=0.05

−13 −14

C. Gibbs distribution algorithm We propose exponential-family (Gibbs distribution) algorithm to drive the relays to optimal acceptance probability p∗ (mixed Nash equilibrium) according to reward value α ˆ estimated by source using kiefer-Wolfowitz algorithm. Indeed, the relays update their strategy probability independently over time according to following rule: pk+1

−16 0

5

10

25

30

20

25

30

λ=1 λ=0.3 λ=0.1 λ=0.05

−1

At each iteration, the relays take a strategy locally and independently over time, and update the strategy probability vector at instant k till achieving the p∗ ∈ argmax Ur (α∗ , p).

−1.5 0

5

10

15 Horizon h

(b)

p

Fig. 3.

Utility of source and relay as a function of the file lifetime. 1

Acceptance Probability p*

This section evaluates the performance of our scheme by some numerical results. We used following setting αmax = 3, β = 0.01, σ = 0.25, σt = 0.02, γ = 0.05. Next, we depict the behaviour of the source and the relays while varying the horizon h (file lifetime) and the parameter λ quantifying the contact rate. We depict in Fig.2 (a) and Fig.2 (b) the acceptance

20

−0.5

(21)

V. N UMERICAL RESULTS

15 Horizon h

(a)

0

Relay Utility

pk Ui (ek , α) ˆ . = pk Ui (ek , α) ˆ + (1 − pk )Ui (1 − ek , α) ˆ

−15

0.9

0.8 h=5 h=10 h=30 h=50

0.7

0.6 3

0.5

Value of α*

2

10

15 20 25 Number of relays n

30

35

40

35

40

(a)

1.5

3

1

2.5 2

5

10

15 Horizon h

20

25

30

Value of α*

0.5

0 0

5

λ=1 λ=0.3 λ=0.1 λ=0.05

2.5

h=5 h=10 h=30 h=50

1.5 1

(a)

0.5

1 0.98

*

Acceptance Probability p

0

λ=1 λ=0.3 λ=0.1 λ=0.05

0.96 0.94 0.92

5

10

15 20 25 Number of relays n

30

(b)

0.9 0.88

Fig. 4. Equilibrium file value and equilibrium acceptance probability as a function of the number of relays and the file lifetime.

0.86 0.84 0.82 0.8 0

5

10

15 Horizon h

20

25

30

(b) Fig. 2. Equilibrium file value and equilibrium acceptance probability as a function of the file lifetime, for several values of λ.

probability p∗ and the file value α∗ at NE as function of the file lifetime. We note that the file value (reward given to the relay node) is increasing as the horizon increases. Meanwhile, the relay tends to decrease its willingness to cache the file as the value of α increases. Moreover, we note that the source has incentive to offer less reward for the file caching as the contact rate increases, which is quite intuitive. Whereas, the relays seem to behave a bit in a counter-intuitive way. Indeed, the relays have incentive to accept file caching as the probability to contact the destination is low. This can be easily explained by the reward offered by the source node. Fig.3 (a) and Fig.3 (b) show the variation of the equilibrium payoff functions over

file lifetime h. For low values of h, the source utility increases till some given threshold hth = h(λ), and converges to the value Us∗ = limh→∞ Us (α∗ , p∗ ) = −3.8. We remark that the threshold horizon hth increases as the λ decreases. As for the relays, we note that its utility is strictly decreasing as h goes to infinity. A special feature is that for both nodes, it is more interesting to consider long enough file lifetime with some reasonable contact probability. Fig.4 (a) and Fig.4 (b) show the impact of the number of relays in the acceptance probability and the file value at NE, we remark that p∗ and α∗ are increasing as the number of relays in interaction which is explained by the behaviour of the relays, when the number of opponents is high, each relay require to the source more reward because its succeed probability becomes low.. Next, we depict in Fig.5 (a) and Fig.5 (b) the influence of the rate contact (λ) and the number of relays on

3 2.5

0.9

2

Value of α*

Acceptance Probability p*

1

0.8 λ=0.05 λ=0.1 λ=1 λ=2

0.7

1

0.6

0.5

1.5

0.5

5

10

15 20 25 Number of relays n

30

35

0

40

5

10

15

20 25 Times Instants

30

35

40

30

35

40

(a)

(a) 3

1 0.9

2.5

0.8

Value of p*

Value of α*

2 1.5 λ=0.05 λ=0.1 λ=1 λ=2

1 0.5 0

5

10

15 20 25 Number of relays n

30

35

0.7 0.6 0.5 0.4 0.3 0.2

40

5

10

15

(b)

(b) Fig. 5. Eq. file value and acceptance probability as a function of the number of relays and the contact rate.

20 25 Times Instants

Fig. 6.

Seeking the pure Nash equilibrium (Ana , R0 , 0) 3 2.5

Value of α*

2 1.5 1 0.5 0

[1] Maia, S.L.F.; Silva, E.R.; Guardieiro, P.R., ”A New Optimization Strategy Proposal for Multi-Copy Forwarding in Energy Constrained DTNs,” Communications Letters, IEEE , vol.18, no.9, pp.1623,1626, Sept. 2014 [2] G. Neglia and X. Zhang. Optimal delay-power trade-off in sparse delay tolerant networks: a preliminary study. in Proc. of ACM SIGCOMM CHANTS 2006, pp. 237244, 2006. [3] Yue Lu; Wei Wang; Lin Chen; Zhaoyang Zhang; Aiping Huang, ”Opportunistic forwarding in energy harvesting mobile delay tolerant networks,” Communications (ICC), 2014 IEEE International Conference on , vol., no., pp.526,531, 10-14 June 2014 [4] Niyato, D.; Wang, P.; Tan, H.; Saad, W.; Kim, D., ”Cooperation in Delay Tolerant Networks with Wireless Energy Transfer: Performance Analysis and Optimization,” Vehicular Technology, IEEE Transactions on , vol.PP, no.99, pp.1,1, 2014

30 Times Instants

40

50

60

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

R EFERENCES

20

(a)

VI. C ONCLUSION In this work, we propose a reward-based incentive mechanism for file caching in Delay Tolerant Networks under constraint of the energy consumed, the file lifetime and the rate of contact.The source-relays interaction is modeled using a noncooperative game. Next, we give some necessary conditions for existence of pure Nash equilibria. Then, we provide the unique mixed equilibrium of the game which seems to have good fairness properties. Our findings are validated/illustrated using extensive numerical results and simulation runs.

10

0.9

Value of p*

the reward value and acceptance probability. Indeed, For low value of λ and n the source offer more reward to induce the low number of relays to accept to cache the file. Moreover, the value of reward increases till some given threshold nth = n(λ) and converges to maximum value (αmax = 3), this threshold increases as the λ decreases. Fig.6 and Fig.7 illustrate the behaviour of our proposed learning algorithms discussed in section IV. Fig.6 (a) and Fig.6 (b) show a case where the pure equilibrium (Ana , R0 , 0) is achieved. Next, Fig.7 (a) and Fig.7 (b) indicate convergence to the pure equilibrium (A0 , Rk , αmax ), we note that the acceptance probability of 5 relays converge to the same probability either 0 ou 1 according to the case of the equilibrium.

10

20

30 40 50 Times Instants

60

70

80

(b) Fig. 7.

Seeking the pure Nash equilibrium (A0 , Rk , αmax )

[5] Krifa, A., Barakat, C., Spyropoulos, T. (2012). Message drop and scheduling in DTNs: Theory and practice. Mobile Computing, IEEE Transactions on, 11(9), 1470-1483. [6] Wang, T., Hui, P., Kulkarni, S., Cuff, P. (2014). Cooperative caching based on file popularity ranking in delay tolerant networks. arXiv preprint arXiv:1409.7047. [7] Al Hanbali, A., Nain, P., Altman, E. (2008). Performance of ad hoc networks with two-hop relay routing and limited packet lifetime (extended version). Performance Evaluation, 65(6), 463-483. [8] Gao, W., Li, Q., Zhao, B., Cao, G. (2009, May). Multicasting in delay tolerant networks: a social network perspective. In Proceedings of the tenth ACM international symposium on Mobile ad hoc networking and computing (pp. 299-308). ACM. [9] Nash, J. F. (1950). Equilibrium points in n-person games. Proc. Nat. Acad. Sci. USA, 36(1), 48-49. [10] Fudenberg, D., Tirole, J. (1991). Game theory. [11] Eitan Altman. Competition and cooperation between nodes in delay tolerant networks with two hop routing. In Network Control and Optimization, pages 264278. Springer, 2009. [12] P. S. Sastry Mandayam and A. L. Thathachar, ”Networks of Learning Automata: Techniques for Online Stochastic Optimization”, Business and Economics, (2004).