Random Matrix Games in Wireless Networks

4 downloads 0 Views 2MB Size Report
Random medium access as strategic decision-making. User 1 ... User-centric network selection as strategic decision-making. User 1. User 2. LTE. WLAN. LTE.
Random Matrix Games in Wireless Networks Manzoor A. Khan and Tembine Hamidou DAI-Labor TU Berlin, Germany; Supelec, France

IEEE Global High Tech Congress on Electronics

November 18-20, 2012

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Plan

Point of Interest of paper Random Access - A bried discussion Introduction to user-centric network selection Strategic decision making view of network selection QoE based network selection (interference)

Rand Matrix Games: A 3 users - 2 channels view of network selection problem Learning in Random Matrix Games Cognitive MAC Games - An example

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Multiple access control (MAC) problem Shannon 1961: Two-way Communication Channels Since then multiple access using shared common medium have extensively studied Data Link Layer Techniques: TDMA, FDMA, (TDS)CDMA, etc. Physical Layer Techniques: FDD, TDD, etc.

but still far from complete.

One user+queue size n = 1. Action: Transmit (T) or Wait (W). State: queue empty or not. Payo↵: Gain on success (G) - Cost of energy consumption (C) : G C, How to decide? If G > C , Transmit when queue is non-empty. Else do nothing.

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Random medium access as strategic decision-making User 2 T

W

T

0, 0⇧

1, 0?

W

0, 1?

0, 0

User 1

HH Access Game HW H pr p p p p p p p p p 2p p p p p p p p H pH p r @ W @ W T @ @ @r @r r

T

r

T

0, 0

1b

1, 0

0, 1

0, 0

Continuum of equilibria. Weakly dominance, Best Response Function

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

User-centric network selection as strategic decision-making Service Provider

Operator Level

Technology Level T2

SP1

OP1

T1 T1

OP2

SP2

Upstream Market LEGEND:

Users

T2

Downstream Market

Reference Market Setting S1

Setting S3

Setting S2

Figure : Envisioned Telecommunication Paradigm

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

User-centric network selection as strategic decision-making User 2 LTE

WLAN

LTE

b0 b0 2, 2

b0 , b1?

WLAN

b1 , b0?

b1 b1 2, 2

User 1

HH HWLAN H pr p p p p p p p p p 2p p p p p p p p H pH p r @ WLAN @ WLAN LTE @ @ @r @r r

LTE

LTE r

b0 b0 2, 2

Network Selection Game

1b

b0 , b1

b1 , b0

b1 b1 2, 2

2b1 > b0 > b1

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

QoE based Network Selection: the frequency interference User 2

User 1

r

0, 0

f2

f1

0, 0

1, 1?

f2

1, 1?

0, 0

Frequency Selection Game

1b

HH f H2 H 2 pr p p p p p p p p p p p p p p p p p H pH p r @ f2 @ f2 f1 @ @ @r @r r

f1

f1

f1

1, 1

1, 1

0, 0

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

QoE based Network Selection: the frequency interference

Figure : An overview of a potential QoE based network selection IEEE Globalapproach High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

3 users, 2 channels: f1

f2

f1

0, 0, 0

0, 1, 0?

f2

1, 0, 0?

0, 0, 1?

f1

f2

f1

0, 0, 1?

1, 0, 0?

f2

0, 1, 0?

0, 0, 0

f1

f2

Figure : A three user strategic channel selection game.

This is a minority game, El Farol bar problem1 .

1

W. Brian Arthur, Inductive Reasoning and Bounded Rationality, American Economic Review, 84,406411, 1994

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Random Matrix Games (RMGs) 3 users, 2 channels: (A more realistic view) In practice, the outcome is influenced by a random variable: channel state, weather, mobility etc this leads to a matrix game where the entries are random =) random matrix game (RMG) 1\ 2

c1

c2

r1 r2

1 , n2 , n3 ) (0, 0, 0) + (n111 111 111 1 , n2 , n3 ) (↵, 0, 0) + (n211 211 211

1 , n2 , n3 ) (0, ↵, 0) + (n111 121 121 1 , n2 , n3 ) (0, 0, ↵) + (n221 221 221

1\ 2

c1

c2

r1 r2

(0, 0, ↵) + (0, ↵, 0) +

1 , n2 , n3 ) (n112 112 112 1 , n2 , n3 ) (n212 212 212

1 , n2 , n3 ) (↵, 0, 0) + (n122 122 122 1 , n2 , n3 ) (0, 0, 0) + (n22 222 222

Table : Strategic form representation for 3 players - 2 actions IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Learning in Random Matrix Games 8 Initialization > > > > > m ˆ = ( m ˆ 1,0 (1), . . . , m ˆ 1,0 (l1 )) 1,0 > > > > x = (x (1), . . . , x 1,0 1,0 1,0 (l1 )) > > > > m ˆ 2,0 = (m ˆ 2,0 (1), . . . , m ˆ 2,0 (l2 )) > > > > x = (x (1), . . . , x 2,0 2,0 2,0 (l2 )) > > > > Define the sequences up to T : j,t , µj,t > > > > For t 2 {1, 2, . . . , T } > > < Learning pattern of the row player > x (r )(1+ )mˆ 1,t (r ) > > x1,t+1 (r ) = P 1,t 0 1,t mˆ 1,t (r 0 ) > > > r 0 x1,t (r )(1+ 1,t ) > > > m ˆ (r ) = m ˆ (r ) + µ1,t 1l{a1,t =r } (m1,t m ˆ 1,t (r )) 1,t+1 1,t > > > > > > > > > Learning pattern of the column player > > > > x (c)(1+ )mˆ 2,t (c) > x2,t+1 (c) = P 2,t 0 2,t mˆ 2,t (c 0 ) > > > c 0 x2,t (c )(1+ 2,t ) > : m ˆ 2,t+1 (c) = m ˆ 2,t (c) + µ2,t 1l{a2,t =c} (m2,t m ˆ 2,t (c))

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

RMG: Frequency selection in noisy environment: two users

1 x algorithm1 yalgorithm1 xalgorithm2 yalgorithm2

,y

0.8 0.7

algorithms:

0.7

1 0.9

0.6 0.4

0.4

0.3

0.3

Two

0.8

x

0.9

0.2

0.2

0.1

0.1

1

yf (t)

0.6

0.5

0 0

0.5

0

500

1000

1500 time

2000

2500

3000

0

0.1

0.2

0.3

0.4

xf (t) 0.5

0.6

0.7

0.8

0.9

1

1

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Probability of playing the first action

Convergence to one of the Global Optima - three users case

1 x1,t (r1 ) x2,t (c1 ) x3,t (a1 )

0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0

1000

2000 3000 Number of iterations

4000

Figure : Convergence of the strategies to a global optimum of RMG

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Convergence to one of the Global Optima

Average of perceived payoffs

1

avg.m1,t avg.m2,t avg.m3,t

0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0

1000 2000 3000 Number of iterations

4000

Figure : Evolution of the average payo↵ estimations of RMG

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Convergence to one of the Global Optima

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Two-step Cognitive MAC game sensing cost cs and transmission ct 0 < cs + ct < p

ST SW WW

ST ( cs ct , cs ct ) ( cs , p cs ct ) (0, p cs ct )

(p

SW cs ct , cs ) ( cs , cs ) (0, cs )

(p

WW cs ct , 0) ( cs , 0) (0, 0)

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Estimated payo↵s 1

Row user Column user

rST

0.5 0

rSW

−0.5 0 0.5

50

100

Row user Column user

0

Estimated rWW

−0.5 0 0.4

50

100

150

Row user Column user

0.2 0 0

150

50

Elapsed time

100

150

Convergence to global optimum payo↵

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Strategies

xST

1

xSW

0 0 0.4

50

100

50

100

150

Row user Column user

0.5 0 0

150

Row user Column user

0.2 0 0 1

xWW

Row user Column user

0.5

50

Elapsed time

Convergence to a global optimum

100

150

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Main results

Result expectation approach: existence of equilibria in RMG for arbitrary number of users. These solutions are noise-independent. observation: it is very important to include the variance in the performance criterion. appropriate solution concept for the mean-variance approach. learning scheme for the mean, the variance and the optimal strategies.

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Combined learning in Random Matrix Games CODIPAS: COmbined fully DIstributed PAyo↵ and Strategy learning.

Imitative CODIPAS 8 > > >
> > : rˆ j,t+1 (sj ) = rˆj,t (sj )

sj :pure strategy of user j xj,t (sj ) : probability to choose sj rˆj,t (sj ) : estimated payo↵. µj,t : payo↵-learning rate t : strategy-learning rate The idea of CODIPAS is simple: The consequences influence behaviors. The behaviors influence the outcomes.

Foreword: Preface:

Tamer Ba¸sar Eitan Altman

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21

Summary Random matrix games are important in wireless networks expectation approach mean-variance approach based on numerical measurement of payo↵s.

Future works Learning how make coalitions in RMG How heterogeneous mobile devices can learn the optimal variance in RMGs? Thank you Questions, Comments?

IEEE Global High Tech Co

Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)

/ 21