Random medium access as strategic decision-making. User 1 ... User-centric network selection as strategic decision-making. User 1. User 2. LTE. WLAN. LTE.
Random Matrix Games in Wireless Networks Manzoor A. Khan and Tembine Hamidou DAI-Labor TU Berlin, Germany; Supelec, France
IEEE Global High Tech Congress on Electronics
November 18-20, 2012
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Plan
Point of Interest of paper Random Access - A bried discussion Introduction to user-centric network selection Strategic decision making view of network selection QoE based network selection (interference)
Rand Matrix Games: A 3 users - 2 channels view of network selection problem Learning in Random Matrix Games Cognitive MAC Games - An example
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Multiple access control (MAC) problem Shannon 1961: Two-way Communication Channels Since then multiple access using shared common medium have extensively studied Data Link Layer Techniques: TDMA, FDMA, (TDS)CDMA, etc. Physical Layer Techniques: FDD, TDD, etc.
but still far from complete.
One user+queue size n = 1. Action: Transmit (T) or Wait (W). State: queue empty or not. Payo↵: Gain on success (G) - Cost of energy consumption (C) : G C, How to decide? If G > C , Transmit when queue is non-empty. Else do nothing.
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Random medium access as strategic decision-making User 2 T
W
T
0, 0⇧
1, 0?
W
0, 1?
0, 0
User 1
HH Access Game HW H pr p p p p p p p p p 2p p p p p p p p H pH p r @ W @ W T @ @ @r @r r
T
r
T
0, 0
1b
1, 0
0, 1
0, 0
Continuum of equilibria. Weakly dominance, Best Response Function
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
User-centric network selection as strategic decision-making Service Provider
Operator Level
Technology Level T2
SP1
OP1
T1 T1
OP2
SP2
Upstream Market LEGEND:
Users
T2
Downstream Market
Reference Market Setting S1
Setting S3
Setting S2
Figure : Envisioned Telecommunication Paradigm
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
User-centric network selection as strategic decision-making User 2 LTE
WLAN
LTE
b0 b0 2, 2
b0 , b1?
WLAN
b1 , b0?
b1 b1 2, 2
User 1
HH HWLAN H pr p p p p p p p p p 2p p p p p p p p H pH p r @ WLAN @ WLAN LTE @ @ @r @r r
LTE
LTE r
b0 b0 2, 2
Network Selection Game
1b
b0 , b1
b1 , b0
b1 b1 2, 2
2b1 > b0 > b1
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
QoE based Network Selection: the frequency interference User 2
User 1
r
0, 0
f2
f1
0, 0
1, 1?
f2
1, 1?
0, 0
Frequency Selection Game
1b
HH f H2 H 2 pr p p p p p p p p p p p p p p p p p H pH p r @ f2 @ f2 f1 @ @ @r @r r
f1
f1
f1
1, 1
1, 1
0, 0
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
QoE based Network Selection: the frequency interference
Figure : An overview of a potential QoE based network selection IEEE Globalapproach High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
3 users, 2 channels: f1
f2
f1
0, 0, 0
0, 1, 0?
f2
1, 0, 0?
0, 0, 1?
f1
f2
f1
0, 0, 1?
1, 0, 0?
f2
0, 1, 0?
0, 0, 0
f1
f2
Figure : A three user strategic channel selection game.
This is a minority game, El Farol bar problem1 .
1
W. Brian Arthur, Inductive Reasoning and Bounded Rationality, American Economic Review, 84,406411, 1994
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Random Matrix Games (RMGs) 3 users, 2 channels: (A more realistic view) In practice, the outcome is influenced by a random variable: channel state, weather, mobility etc this leads to a matrix game where the entries are random =) random matrix game (RMG) 1\ 2
c1
c2
r1 r2
1 , n2 , n3 ) (0, 0, 0) + (n111 111 111 1 , n2 , n3 ) (↵, 0, 0) + (n211 211 211
1 , n2 , n3 ) (0, ↵, 0) + (n111 121 121 1 , n2 , n3 ) (0, 0, ↵) + (n221 221 221
1\ 2
c1
c2
r1 r2
(0, 0, ↵) + (0, ↵, 0) +
1 , n2 , n3 ) (n112 112 112 1 , n2 , n3 ) (n212 212 212
1 , n2 , n3 ) (↵, 0, 0) + (n122 122 122 1 , n2 , n3 ) (0, 0, 0) + (n22 222 222
Table : Strategic form representation for 3 players - 2 actions IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Learning in Random Matrix Games 8 Initialization > > > > > m ˆ = ( m ˆ 1,0 (1), . . . , m ˆ 1,0 (l1 )) 1,0 > > > > x = (x (1), . . . , x 1,0 1,0 1,0 (l1 )) > > > > m ˆ 2,0 = (m ˆ 2,0 (1), . . . , m ˆ 2,0 (l2 )) > > > > x = (x (1), . . . , x 2,0 2,0 2,0 (l2 )) > > > > Define the sequences up to T : j,t , µj,t > > > > For t 2 {1, 2, . . . , T } > > < Learning pattern of the row player > x (r )(1+ )mˆ 1,t (r ) > > x1,t+1 (r ) = P 1,t 0 1,t mˆ 1,t (r 0 ) > > > r 0 x1,t (r )(1+ 1,t ) > > > m ˆ (r ) = m ˆ (r ) + µ1,t 1l{a1,t =r } (m1,t m ˆ 1,t (r )) 1,t+1 1,t > > > > > > > > > Learning pattern of the column player > > > > x (c)(1+ )mˆ 2,t (c) > x2,t+1 (c) = P 2,t 0 2,t mˆ 2,t (c 0 ) > > > c 0 x2,t (c )(1+ 2,t ) > : m ˆ 2,t+1 (c) = m ˆ 2,t (c) + µ2,t 1l{a2,t =c} (m2,t m ˆ 2,t (c))
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
RMG: Frequency selection in noisy environment: two users
1 x algorithm1 yalgorithm1 xalgorithm2 yalgorithm2
,y
0.8 0.7
algorithms:
0.7
1 0.9
0.6 0.4
0.4
0.3
0.3
Two
0.8
x
0.9
0.2
0.2
0.1
0.1
1
yf (t)
0.6
0.5
0 0
0.5
0
500
1000
1500 time
2000
2500
3000
0
0.1
0.2
0.3
0.4
xf (t) 0.5
0.6
0.7
0.8
0.9
1
1
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Probability of playing the first action
Convergence to one of the Global Optima - three users case
1 x1,t (r1 ) x2,t (c1 ) x3,t (a1 )
0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0
1000
2000 3000 Number of iterations
4000
Figure : Convergence of the strategies to a global optimum of RMG
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Convergence to one of the Global Optima
Average of perceived payoffs
1
avg.m1,t avg.m2,t avg.m3,t
0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0
1000 2000 3000 Number of iterations
4000
Figure : Evolution of the average payo↵ estimations of RMG
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Convergence to one of the Global Optima
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Two-step Cognitive MAC game sensing cost cs and transmission ct 0 < cs + ct < p
ST SW WW
ST ( cs ct , cs ct ) ( cs , p cs ct ) (0, p cs ct )
(p
SW cs ct , cs ) ( cs , cs ) (0, cs )
(p
WW cs ct , 0) ( cs , 0) (0, 0)
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Estimated payo↵s 1
Row user Column user
rST
0.5 0
rSW
−0.5 0 0.5
50
100
Row user Column user
0
Estimated rWW
−0.5 0 0.4
50
100
150
Row user Column user
0.2 0 0
150
50
Elapsed time
100
150
Convergence to global optimum payo↵
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Strategies
xST
1
xSW
0 0 0.4
50
100
50
100
150
Row user Column user
0.5 0 0
150
Row user Column user
0.2 0 0 1
xWW
Row user Column user
0.5
50
Elapsed time
Convergence to a global optimum
100
150
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Main results
Result expectation approach: existence of equilibria in RMG for arbitrary number of users. These solutions are noise-independent. observation: it is very important to include the variance in the performance criterion. appropriate solution concept for the mean-variance approach. learning scheme for the mean, the variance and the optimal strategies.
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Combined learning in Random Matrix Games CODIPAS: COmbined fully DIstributed PAyo↵ and Strategy learning.
Imitative CODIPAS 8 > > >
> > : rˆ j,t+1 (sj ) = rˆj,t (sj )
sj :pure strategy of user j xj,t (sj ) : probability to choose sj rˆj,t (sj ) : estimated payo↵. µj,t : payo↵-learning rate t : strategy-learning rate The idea of CODIPAS is simple: The consequences influence behaviors. The behaviors influence the outcomes.
Foreword: Preface:
Tamer Ba¸sar Eitan Altman
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21
Summary Random matrix games are important in wireless networks expectation approach mean-variance approach based on numerical measurement of payo↵s.
Future works Learning how make coalitions in RMG How heterogeneous mobile devices can learn the optimal variance in RMGs? Thank you Questions, Comments?
IEEE Global High Tech Co
Manzoor A. Khan and Tembine Hamidou (DAI-Labor Random Matrix TU Berlin, GamesGermany; in Wireless Supelec, Networks France)
/ 21