1. QoE model driven for network services. Hai Anh Tran, Abdelhamid Mellouk. LiSSi Laboratory, University of Paris-Est Creteil (UPEC), France. WWIC, June 1-3, ...
QoE model driven for network services Hai Anh Tran, Abdelhamid Mellouk
LiSSi Laboratory, University of Paris-Est Creteil (UPEC), France
WWIC, June 1-3, 2010
1
Outline E2E QoS: context and challenges QoE paradigm Integration of QoE measurement in a routing system
2
History of Quality of Service
Early definitions:
“collective effect of service performance which determines the degree of satisfaction of a user of the service” [ITU-T Rec. E.800, 1994] “a set of qualities related to the collective behavior of one or more objects” [ISO/IEC 13236, 1998] “used to define the network‘s capability to meet the requirements of users and applications” [Kilkki, 1999]
Then:
“ability of the network to provide a service at an assured service level” [Soldani, 2006] “capability of a network to provide better service to selected network traffic … described by the following parameters: delay and jitter, loss probability, reliability, throughput and delivery time” [Markaki, 2007]
3
End-to-end Quality of Service
E2E QoS:
QoS perceived by the users Desires and requirements of users:
Capacity of the system Stability and synchronization of video and audio Accuracy of information, etc.
Cooperation of all the system components E2E QoS: the “summation” of the local QoSs
Challenges:
Dynamic changes in the resources of communication networks provide E2E QoS for individual flows Complexity and stability:
NP-complete [*] Heterogeneity Complexity and cost: limiting factors in the future evolution of networks 4
[*]: Wang Z. and Crowcroft J. “Quality of Service Routing for Supporting Multimedia Applications”
User to User Quality of Experience
Buzzword extension: “QoE has been defined as an extension of the traditional QoS in the sense that QoE provides information regarding the delivered services from an end-user point of view” [Lopez et al. 2006]
Usability metric: “QoE is how a user perceives the usability of a service when in use – how satisfied he/she is with a service in terms of, e.g., usability, accessibility, retainability and integrity” [Soldani 2006]
Hedonistic concept: “QoE describes the degree of delight of the user of a service, influenced by content, network, device, application, user expectations and goals, and context of use” [Dagstuhl Seminar May 2009]
5
Quality of Experience QoE does not replace QoS, but improves E2E QoS Important measure of the E2E performance an important metric for the design of systems and engineering processes. Contribute to Real-Adaptive Control Network’s Components
6
Quality chain for User to User QoE (U2U-QoE)
QoE: Quality of Experience QoS: Quality of Service QoD: Quality of Design
QoE = QoD QoS 7
QoE aspect in Autonomous Network
Functions: Self-optimization Self-healing
Attributes: Environment-
awareness Self-adjusting
8
QoE measurement
Subjective methods: in Mean Opinion Score (MOS), QoE is measured subjectively by a scale of score (1-5). Problems: time consuming, tedious, expensive and not applicable in a production environment. Objective methods: Full Reference, No Reference, Reduced Reference PEVQ measurement (Perceptual Evaluation of Video Quality):
9
Impact of QoS/QoE User to User QoE QoS
Functions: • Admission control • Resource management • Routing • Traffic control
10
Integration of QoE measurement in a routing system
11
Reinforcement Learning
- Agent’s goal: maximize the total amount of reward it receives. π - State-Value function V ( s ) : expected return when starting in s and following π thereafter.
V π (s) = Eπ {Rt | st = s} π
-Action-Value function Q ( s, a ) : expected return starting from s, taking the action a, and thereafter following policy π
Qπ ( s, a ) = Eπ { Rt | st = s, at = a}
12
Approach based on Reinforcement Learning
π t +1 ( st ) = arg max Qπ ( st , a) t
a
E
Policy iteration:
π0
I
E
I
E
I
E
π 0 →V → π 1 →V → π 2 → ... → π →V * π1
*
13
Function approximation • •
Combining reinforcement learning methods with existing generalization methods It takes examples from a desired function (e.g., a value function) and attempts to generalize from them to construct an approximation of the entire function
Generalization methods
+ Reinforcement learning methods
Function approximation
Linear methods (Least Squares Policy Iteration (LSPI))
14
Least Squares Policy Iteration (LSPI) Based on a Linear method: LSPI [*]
Q * ( s, a ) = r ( s, a ) + γ ∑ P ( s ' | s, a ) maxQ * ( s ', a ') a'
s'
k
Qˆ ( s, a) = ∑ φ i ( s, a )ωi =φ ( s, a)T ω π
i =1
Φ(s,a)={b(s,a),t(s,a),f(s,a),fb(s,a)} π
Φω ≈ R + γ P Φω
⇔ Φ T (Φ − γ Pπ Φ )ωπ = Φ T R
⇒ ω = A−1b
with
b: bandwidth of link ss’ t: estimated delay of delivery f: length of the queue of s’ fb: reliability of link ss’
A = Φ T (Φ − γ Pπ Φ ) b = ΦT R 15
[*]: M. Lagoudakis et R. Parr, « Model-free Least Squares Policy Iteration »
Calculating φ ( s1 , a1 )T ˆ Φ = ... φ ( s , a )T L L φ ( s '1 , π ( s '1 ))T π P Φ = ... φ ( s ' , π ( s ' ))T L L
ω
⇒ω
r1 Rˆ = ... r L 16
Preliminary simulation results Simulation parameters Simulator
Opnet modeler 14.0
Topology
Irregular network 3 areas 38 routers
Duration time
24 hours
QoE evaluation method
Reference table represents humain estimated MOS function of loss packet rate based on real experiment
17
MOS score
Preliminary simulation results
Duration time(hour)
SPF RIP RL-QoE
18
Conclusions End-to-end
QoS problems QoE paradigm Integration of QoE measurement in a routing system Improving the QoE in the system
Future works
Integrate the discrimination concept Implement protocol on IPTV flows Study Scalability 19
20
QoE evaluation model
Video Server Real Video
Proxy Packet lost model
LAN
Client Video Packet lost measurement
Packet lost ratio
MOS
Reference table
21
Topology
