Random Matrix Games in Wireless Networks

Random Matrix Games in Wireless Networks Manzoor A. Khan and Tembine Hamidou DAI-Labor TU Berlin, Germany; Supelec, France

IEEE Global High Tech Congress on Electronics

November 18-20, 2012

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Plan

Point of Interest of paper Random Access - A bried discussion Introduction to user-centric network selection Strategic decision making view of network selection QoE based network selection (interference)

Rand Matrix Games: A 3 users - 2 channels view of network selection problem Learning in Random Matrix Games Cognitive MAC Games - An example



/ 21

Multiple access control (MAC) problem Shannon 1961: Two-way Communication Channels Since then multiple access using shared common medium have extensively studied Data Link Layer Techniques: TDMA, FDMA, (TDS)CDMA, etc. Physical Layer Techniques: FDD, TDD, etc.

but still far from complete.

One user+queue size n = 1. Action: Transmit (T) or Wait (W). State: queue empty or not. Payo↵: Gain on success (G) - Cost of energy consumption (C) : G C, How to decide? If G > C , Transmit when queue is non-empty. Else do nothing.



/ 21

Random medium access as strategic decision-making User 2 T

W

T

0, 0⇧

1, 0?

W

0, 1?

0, 0

User 1

HH Access Game HW H pr p p p p p p p p p 2p p p p p p p p H pH p r @ W @ W T @ @ @r @r r

T

r

T

0, 0

1b

1, 0

0, 1

0, 0

Continuum of equilibria. Weakly dominance, Best Response Function



/ 21

User-centric network selection as strategic decision-making Service Provider

Operator Level

Technology Level T2

SP1

OP1

T1 T1

OP2

SP2

Upstream Market LEGEND:

Users

T2

Downstream Market

Reference Market Setting S1

Setting S3

Setting S2

Figure : Envisioned Telecommunication Paradigm



/ 21

User-centric network selection as strategic decision-making User 2 LTE

WLAN

LTE

b0 b0 2, 2

b0 , b1?

WLAN

b1 , b0?

b1 b1 2, 2

User 1

HH HWLAN H pr p p p p p p p p p 2p p p p p p p p H pH p r @ WLAN @ WLAN LTE @ @ @r @r r

LTE

LTE r

b0 b0 2, 2

Network Selection Game

1b

b0 , b1

b1 , b0

b1 b1 2, 2

2b1 > b0 > b1



/ 21

QoE based Network Selection: the frequency interference User 2

User 1

r

0, 0

f2

f1

0, 0

1, 1?

f2

1, 1?

0, 0

Frequency Selection Game

1b

HH f H2 H 2 pr p p p p p p p p p p p p p p p p p H pH p r @ f2 @ f2 f1 @ @ @r @r r

f1

f1

f1

1, 1

1, 1

0, 0



/ 21

QoE based Network Selection: the frequency interference

Figure : An overview of a potential QoE based network selection IEEE Globalapproach High Tech Co


/ 21

3 users, 2 channels: f1

f2

f1

0, 0, 0

0, 1, 0?

f2

1, 0, 0?

0, 0, 1?

f1

f2

f1

0, 0, 1?

1, 0, 0?

f2

0, 1, 0?

0, 0, 0

f1

f2

Figure : A three user strategic channel selection game.

This is a minority game, El Farol bar problem1 .

1

W. Brian Arthur, Inductive Reasoning and Bounded Rationality, American Economic Review, 84,406411, 1994



/ 21

Random Matrix Games (RMGs) 3 users, 2 channels: (A more realistic view) In practice, the outcome is influenced by a random variable: channel state, weather, mobility etc this leads to a matrix game where the entries are random =) random matrix game (RMG) 1\ 2

c1

c2

r1 r2

1 , n2 , n3 ) (0, 0, 0) + (n111 111 111 1 , n2 , n3 ) (↵, 0, 0) + (n211 211 211

1 , n2 , n3 ) (0, ↵, 0) + (n111 121 121 1 , n2 , n3 ) (0, 0, ↵) + (n221 221 221

1\ 2

c1

c2

r1 r2

(0, 0, ↵) + (0, ↵, 0) +

1 , n2 , n3 ) (n112 112 112 1 , n2 , n3 ) (n212 212 212

1 , n2 , n3 ) (↵, 0, 0) + (n122 122 122 1 , n2 , n3 ) (0, 0, 0) + (n22 222 222

Table : Strategic form representation for 3 players - 2 actions IEEE Global High Tech Co


/ 21

Learning in Random Matrix Games 8 Initialization > > > > > m ˆ = ( m ˆ 1,0 (1), . . . , m ˆ 1,0 (l1 )) 1,0 > > > > x = (x (1), . . . , x 1,0 1,0 1,0 (l1 )) > > > > m ˆ 2,0 = (m ˆ 2,0 (1), . . . , m ˆ 2,0 (l2 )) > > > > x = (x (1), . . . , x 2,0 2,0 2,0 (l2 )) > > > > Define the sequences up to T : j,t , µj,t > > > > For t 2 {1, 2, . . . , T } > > < Learning pattern of the row player > x (r )(1+ )mˆ 1,t (r ) > > x1,t+1 (r ) = P 1,t 0 1,t mˆ 1,t (r 0 ) > > > r 0 x1,t (r )(1+ 1,t ) > > > m ˆ (r ) = m ˆ (r ) + µ1,t 1l{a1,t =r } (m1,t m ˆ 1,t (r )) 1,t+1 1,t > > > > > > > > > Learning pattern of the column player > > > > x (c)(1+ )mˆ 2,t (c) > x2,t+1 (c) = P 2,t 0 2,t mˆ 2,t (c 0 ) > > > c 0 x2,t (c )(1+ 2,t ) > : m ˆ 2,t+1 (c) = m ˆ 2,t (c) + µ2,t 1l{a2,t =c} (m2,t m ˆ 2,t (c))



/ 21

RMG: Frequency selection in noisy environment: two users

1 x algorithm1 yalgorithm1 xalgorithm2 yalgorithm2

,y

0.8 0.7

algorithms:

0.7

1 0.9

0.6 0.4

0.4

0.3

0.3

Two

0.8

x

0.9

0.2

0.2

0.1

0.1

1

yf (t)

0.6

0.5

0 0

0.5

0

500

1000

1500 time

2000

2500

3000

0

0.1

0.2

0.3

0.4

xf (t) 0.5

0.6

0.7

0.8

0.9

1

1



/ 21

Probability of playing the first action

Convergence to one of the Global Optima - three users case

1 x1,t (r1 ) x2,t (c1 ) x3,t (a1 )

0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0

1000

2000 3000 Number of iterations

4000

Figure : Convergence of the strategies to a global optimum of RMG



/ 21

Convergence to one of the Global Optima

Average of perceived payoffs

1

avg.m1,t avg.m2,t avg.m3,t

0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0

1000 2000 3000 Number of iterations

4000

Figure : Evolution of the average payo↵ estimations of RMG



/ 21

Convergence to one of the Global Optima



/ 21

Two-step Cognitive MAC game sensing cost cs and transmission ct 0 < cs + ct < p

ST SW WW

ST ( cs ct , cs ct ) ( cs , p cs ct ) (0, p cs ct )

(p

SW cs ct , cs ) ( cs , cs ) (0, cs )

(p

WW cs ct , 0) ( cs , 0) (0, 0)



/ 21

Estimated payo↵s 1

Row user Column user

rST

0.5 0

rSW

−0.5 0 0.5

50

100


0

Estimated rWW

−0.5 0 0.4

50

100

150


0.2 0 0

150

50

Elapsed time

100

150

Convergence to global optimum payo↵



/ 21

Strategies

xST

1

xSW

0 0 0.4

50

100

50

100

150


0.5 0 0

150


0.2 0 0 1

xWW


0.5

50

Elapsed time

Convergence to a global optimum

100

150



/ 21

Main results

Result expectation approach: existence of equilibria in RMG for arbitrary number of users. These solutions are noise-independent. observation: it is very important to include the variance in the performance criterion. appropriate solution concept for the mean-variance approach. learning scheme for the mean, the variance and the optimal strategies.



/ 21

Combined learning in Random Matrix Games CODIPAS: COmbined fully DIstributed PAyo↵ and Strategy learning.

Imitative CODIPAS 8 > > >
> > : rˆ j,t+1 (sj ) = rˆj,t (sj )

sj :pure strategy of user j xj,t (sj ) : probability to choose sj rˆj,t (sj ) : estimated payo↵. µj,t : payo↵-learning rate t : strategy-learning rate The idea of CODIPAS is simple: The consequences influence behaviors. The behaviors influence the outcomes.

Foreword: Preface:

Tamer Ba¸sar Eitan Altman



/ 21

Summary Random matrix games are important in wireless networks expectation approach mean-variance approach based on numerical measurement of payo↵s.

Future works Learning how make coalitions in RMG How heterogeneous mobile devices can learn the optimal variance in RMGs? Thank you Questions, Comments?



/ 21