Y. C. Chang et al.: A Low Complexity Hierarchical QAM Symbol Bits Allocation Algorithm for Unequal Error Protection of Wireless Video Transmission 1089
A Low Complexity Hierarchical QAM Symbol Bits Allocation Algorithm for Unequal Error Protection of Wireless Video Transmission Yoong Choon Chang, Sze Wei Lee and Ryoichi Komiya, Members, IEEE Abstract — A low complexity hierarchical quadrature amplitude modulation (QAM) symbol bits allocation algorithm for unequal error protection of video transmission over wireless channels is proposed in this paper. An unequal error protection (UEP) scheme using hierarchical QAM, which takes into consideration the non-uniformly distributed importance of intracoded frame (I-frame) and predictive coded frame (P-frame) in a group of pictures (GOP), is first proposed. In order to optimally allocate the hierarchical QAM’s high priority (HP), medium priority (MP) and low priority (LP) symbol bits to the H.264/AVC video, a generic solution for optimal allocation of hierarchical QAM’s high priority (HP), medium priority (MP) and low priority (LP) symbol bits to the H.264/AVC video is then proposed. Finally, a low complexity symbol bits allocation algorithm, namely ranking search algorithm, which reduces the computational complexity of the proposed optimal symbol bits allocation algorithm, is proposed. Simulation results show that our proposed UEP scheme outperforms the classical equal error protection (EEP) scheme and also the previous UEP scheme. Index Terms — Unequal error protection, H.264/AVC, hierarchical QAM, wireless video transmission.
I. INTRODUCTION Transmission of compressed video over wireless channels is a challenging task due to the inherent limited bandwidth and the error-prone nature of the wireless channels. A single transmission error might cause the compressed video to be undecodable due to the extensive use of variable length coding in modern video coding standards. In addition, due to the extensive use of predictive coding technique in video compression, it is very likely that the effects of bit error in a compressed video bit-stream will propagate to neighboring spatial blocks and frames [1], [2]. In order to minimize the effects of transmission errors on reconstructed video quality, error resilient video coding techniques [3]-[5] have been proposed. Among the error resilient video coding techniques, unequal error protection (UEP) [6], [7], which is based on the
Yoong Choon CHANG is with the Faculty of Engineering, Multimedia University, Malaysia (e-mail:
[email protected]). Sze Wei LEE is with the Institute of Postgraduate Studies & Research, University Tunku Abdul Rahman, Malaysia (email:
[email protected]). Roiychi KOMIYA is with the Faculty of Information Technology, Multimedia University, Malaysia (e-mail:
[email protected]). Contributed Paper Manuscript received June 15, 2009
idea that the binary bits in a compressed video bit-stream are not equally important, has gained attention in recent years. The reconstructed video quality will be severely degraded if transmission errors fall on these important bits and thus, these important bits should be allocated with a higher protection order compared to the rest of the video bit-stream. One of the UEP schemes is to leverage on the inherent property of the hierarchical modulation [8]-[13], in which the compressed video’s highly sensitive high priority (HP) data bits are mapped to the most significant bits (MSB) of the modulation constellation points with lower bit error rate (BER) while the low priority (LP) data bits are mapped to the least significant bits (LSB) of the modulation constellation points with higher BER. The overall received video quality will be improved, compared with the classical equal error protection (EEP) using non-hierarchical modulation, especially at low channel signalto-noise (SNR) conditions, since the highly sensitive coded video’s HP data bits are mapped to the MSBs of the hierarchical modulation with low BER. This paper proposes a robust wireless video transmission scheme using UEP in conjunction with hierarchical QAM. An UEP scheme using hierarchical QAM, which takes into consideration the non-uniformly distributed importance of intracoded frame (I-frame) and predictive coded frame (P-frame) in a group of pictures (GOP), is first proposed. In order to maximize the expected video quality at the receiver, a generic solution for optimal allocation of the hierarchical QAM’s HP, MP and LP symbol bits to the H.264/AVC video is then proposed. The proposed optimal symbol bits allocation can be obtained by using an exhaustive search process but this is not practical in real-time applications because of the high computational power requirement of the exhaustive search process. In view of this, a low complexity symbol bits allocation algorithm, namely ranking search algorithm, is also proposed in order to reduce the computational complexity of the optimal symbol bits allocation problem. Compared with the classical equal error protection (EEP) and the previous UEP schemes, our proposed UEP scheme results in better reconstructed video quality at the receiver. This paper is organized as follows. In Section II, a brief overview of hierarchical 16-QAM and 64-QAM is presented. Section III presents the proposed hierarchical QAM symbol bits optimal allocation algorithm and also the proposed ranking search low complexity symbol bits allocation algorithm. Results of the proposed UEP scheme are presented in Section IV, followed by conclusions and future work recommendation in Section V.
0098 3063/09/$20.00 © 2009 IEEE
Authorized licensed use limited to: MULTIMEDIA UNIVERSITY. Downloaded on October 10, 2009 at 22:18 from IEEE Xplore. Restrictions apply.
IEEE Transactions on Consumer Electronics, Vol. 55, No. 3, AUGUST 2009
1090
II. HIERARCHICAL QAM Hierarchical modulations were initially proposed to provide different classes of data to users in different wireless signal reception conditions [8], [11]. Users within the coverage range can reliably receive basic information while users under more favorable conditions can receive additional refined information. A. Hierarchical 16-QAM Fig. 1 depicts the constellation diagram of hierarchical 16QAM modulation, where d1 and d2 are the minimum distance between quarters and the minimum distance between points inside each quarter. To achieve UEP for HP data and LP data, d1 and d2 are adjusted such that d1 > d2. By referring to Fig. 1, let alpha value, α, be given by
α=
d1 d2
(1)
When α = 1, the signal constellation becomes a conventional rectangular 16-QAM with each layer having the same reliability. On the other hand, if α > 1, the signal constellation becomes a hierarchical 16-QAM. As a result of the Gray bit distribution of the constellation points, the two MSBs have the same value in each quadrant and these two bits are more robust against channel noise and thus have lower bit error rates than the remaining two LSB. By controlling the value of α, we can control the bit error rate (BER) of the HP data and LP data respectively. Increasing α will increase the robustness of HP data against channel noise at the expense of the LP data robustness against channel noise. 1011
0010
1010
0011
Two values are introduced here, namely the alpha value, α, and the beta value, β. Alpha has been defined in (1). By referring to Fig. 2, let beta value, β, be given by
β=
d3 d2
(2)
To achieve UEP for HP, MP and LP data using hierarchical constellation, α and β values need to be adjusted. When the α and β values are equal to one, the signal constellation becomes a conventional 64-QAM. When α or β or both of the α and β values are larger than one, the signal constellation becomes a hierarchical 64-QAM, as shown in Table 1. Table 1 also shows that there are three priority modes in hierarchical 64-QAM, namely mode 2, 3 and 4. The number of HP bits in each priority mode is different, ranging from 4 bits (66% of the capacity) to 2 bits (33% of the capacity). The optimum priority mode, which results in the highest expected reconstructed video quality in the receiver, has to be determined. 101111 101101
100101 100111
000111 000101
001101 001111
101110 101100
100100 100110
000110 000100
001100 001110
101010 101000
100000 100010
000010 000000
001000 001010
101011 101001
100001 100011
000011 000001
001001 001011
HP bits
001000
d2
d1 MSB = 10 MSB = 11
LP bits MP bits
d3 MSB = 00 MSB = 01
111011 111001
110001 110011
010011 010001
011001 011011
111010 111000
110000 110010
010010 010000
011000 011010
111110 111100
110100 110110
010110 010100
011100 011110
111111 111101
110101 110111
010111 010101
011101 011111
HP bits 0001 LP bits 1001
1101
1000
d1
0000
d2
MSB = 10
MSB = 00
MSB = 11
MSB = 01 1100
0100
0001
0101
Fig. 2. Hierarchical 64-QAM constellation diagram.
Priority Mode
1 1111
1110
0110
0111
Fig. 1. Hierarchical 16-QAM constellation diagram. The bits in bold face are mapped to high priority bits and the rest are mapped to low priority bits.
B. Hierarchical 64-QAM Fig. 2 depicts the constellation diagram of hierarchical 64QAM modulation, where d1 is the minimum distance between quarters, d2 is the minimum distance between points inside each sub-quarter inside a quarter and d3 is the minimum distance between sub-quarters inside each a quarter.
2 3 4
TABLE 1 HIERARCHICAL 64-QAM PRIORITY MODES [10] High Medium Low Priority α value β value priority priority priority bits bits bits (HP) (MP) (LP) Non-hierarchical (EEP) Hierarchical 2 priorities (UEP) Hierarchical 2 priorities (UEP) Hierarchical 3 priorities (UEP)
1
1
1
>1
>1
1
>1
>1
100% 66% (4 bits) 33% (2 bits) 33% (2 bits)
33% (2 bits)
33% (2 bits) 66% (4 bits) 33% (2 bits)
III. PROPOSED UEP SCHEME USING HIERARCHICAL QAM The concept of non-uniformly distributed importance of I- and P-frames in a GOP, which is used in developing the UEP schemes in [6], is shown in Fig. 5. and briefly described here. The extent of error propagation caused by transmission errors depends on the
Authorized licensed use limited to: MULTIMEDIA UNIVERSITY. Downloaded on October 10, 2009 at 22:18 from IEEE Xplore. Restrictions apply.
Y. C. Chang et al.: A Low Complexity Hierarchical QAM Symbol Bits Allocation Algorithm for Unequal Error Protection of Wireless Video Transmission 1091
position of the error in the coded video sequence. The earlier an error occurs in a GOP, the more frames are affected. For example, transmission errors in a P-frame immediately right after an I-frame will affect all the following frames in the GOP whereas an error in the last P-frame in the GOP does not affect any other frames. In other words, the frames in the GOP have descending importance and thus, the first frame in the GOP should be allocated with a higher protection order, compared to the last frame in the GOP. It is therefore beneficial to partition the coded video bit-stream in such a way that the earlier frames in a GOP are better protected than the later frames in a GOP. Unlike the UEP scheme in [6], which only takes into consideration the unequal importance of frames in a GOP, our proposed UEP scheme takes into consideration the unequal importance of frames in a GOP and also the unequal importance of macroblocks in a video frame, as shown in Figs. 6 & 7 respectively. When transmission errors occur at the macroblocks at the beginning of a video frame, the decoder will lose synchronization due to the variable length code used in video compression and this causes the decoder to stop the decoding process of the current video frame and restart the decoding process at the next video frame. On the other hand, if transmission errors occur on the macroblocks located toward the end of the video frame, the affected macroblocks are fewer. By taking this into consideration, macroblocks at the beginning of every video frame should be allocated with a higher protection order, compared to the macroblocks at the end of the video frame since macroblocks at the beginning of a video frame will cause more error propagation if errors occur at them. The compressed video bit-stream in our proposed UEP scheme with hierarchical QAM is classified into two or three priorities, depending on the modulation order (16- or 64-QAM) used. For UEP using hierarchical 16-QAM, the compressed video bit-stream is classified into two priorities, namely HP and LP data bits, as shown in Fig. 3. For UEP using hierarchical 64-QAM, the compressed video bit-stream is classified into three priorities, namely HP, MP and LP data bits, as shown in Fig. 4. The compressed video data is classified into HP, MP and LP data by taking into consideration the non-uniformly distributed importance frames in a GOP and macroblocks in a video frame. HP data is the compressed video bit-stream which is most sensitive to transmission errors and the reconstructed video image quality will be severely degraded if transmission errors fall on this data. Therefore, higher amount of protection is allocated to protect the HP data. Compared with the HP data, errors that occur to LP data will not cause significant distortions on the reconstructed video image and therefore lower amount of protection can be applied. As can be seen in Fig. 6, all the macroblocks in the first frame of a group of picture (GOP), together with macroblocks at the beginning of video frames are classified as HP data while the rest of the macroblocks are classified as LP data. The following sub-sections address the optimal assignment problem of the hierarchical 16- and 64-QAM HP, MP and LP symbol bits to the H.264/AVC video, in which the aim of our
optimally assignment algorithm is to maximize the expected received video quality at the receiving end. Source distortion, which is the video quality distortion caused by lossy quantization process during compression, is not considered in this paper. Raw Video
Proposed Ranking Search Low Complexity Symbol Bits Allocation Algorithm
H.264/AVC Encoder
HP Data Bits
HP Symbol Bits
LP Data Bits
LP Symbol Bits Hierarchical 16-QAM Modulator
HP Data Bits
Decoded Video
H.264/AVC Decoder
AWGN Channel
HP Symbol Bits
LP Data Bits
LP Symbol Bits Hierarchical 16-QAM Demodulator
Fig. 3. Overall block diagram of our proposed UEP with hierarchical 16QAM.
Raw Video
Proposed Ranking Search Low Complexity Symbol Bits Allocation Algorithm
H.264/AVC Encoder
HP Data Bits
HP Symbol Bits
MP Data Bits
MP Symbol Bits
LP Data Bits
LP Symbol Bits Hierarchical 64-QAM Modulator AWGN Channel
HP Data Bits Decoded Video
H.264/AVC Decoder
HP Symbol Bits
MP Data Bits
MP Symbol Bits
LP Data Bits
LP Symbol Bits Hierarchical 64-QAM Demodulator
Fig. 4. Overall block diagram of our proposed UEP with hierarchical 64QAM.
…
Frame 1 (I)
Frame 2 (P)
Frame 3 (P)
Frame 4 (P)
Frame 15 (P)
GOP High protection order
Medium protection order
Low protection order
Fig. 5. Illustration of the UEP model in [6], which takes into consideration the unequal importance of frames in a GOP.
...
Frame 1 (I)
Frame 2 (P)
Frame 3 (P)
Frame 15 (P)
GOP = HP data
= LP data
Fig. 6. Illustration our proposed UEP with hierarchical 16-QAM by taking into consideration the unequal importance of different frames in a GOP and also the unequal importance of macroblocks in a video frame.
Authorized licensed use limited to: MULTIMEDIA UNIVERSITY. Downloaded on October 10, 2009 at 22:18 from IEEE Xplore. Restrictions apply.
IEEE Transactions on Consumer Electronics, Vol. 55, No. 3, AUGUST 2009
1092
symbol bits allocated to frame i in a GOP is subject to the following constraints: ...
N
j =1
Frame 1 (I)
Frame 2 (P)
Frame 3 (P)
Frame 15 (P)
= MP data
= LP data
Fig. 7. Illustration our proposed UEP with hierarchical 64-QAM by taking into consideration the unequal importance of different frames in a GOP.
A. Problem Formulation 1: Optimal Allocation of Hierarchical 16-QAM HP and LP Symbol Bits to H.264/AVC Video. Let F 16 HQAM denotes the hierarchical 16-QAM HP and LP symbol bits allocated to the H.264/AVC coded video and is given by
F 16 HQAM = ( RHP1,1 , RLP1,1 , RHP2, 2 , RLP2, 2 ,..., RHPi , j , RLPi , j ,..., RHPL , N , RLPL , N ) where
N
∑ RLP j =1
GOP = HP data
N
N
N
j =1
j =1
N
N
N
j =1
j =1
j =1
∑ RHP1, j ≥ ∑ RHP2, j ≥ ... ≥ ∑ RHPi, j ... ≥ ∑ RHPL, j (6)
(3)
1, j
j =1
≤ ∑ RLP2, j ≤ ... ≤ ∑ RLPi , j ... ≤ ∑ RLPL , j
(7)
Let Q(i,j) denotes the amount of peak signal-to-noise ratio (PSNR) increment for successfully decoding macroblock j in frame i, denoted as macroblock(i,j), and is given by
⎞ ⎛ 255 ⎟⎟ Q(i, j ) = 20 log10 ⎜⎜ ⎝ RMSE (i, j ) ⎠
(8)
where RMSE is the root mean square error between the successfully decoded macroblock pixel values and original raw video macroblock pixel values. Let PE (i, j ) denotes the probability of a single binary bit error in macroblock(i,j), Ki,j denotes the number of binary bits in macroblock(i,j), the probability of successfully decoding macroblock(1,1), denoted as PSD (1,1) , is then given by
PSD(1,1) = (1 − PE (1,1))
K 1,1
(9)
RHPi , j and RLPi , j are the hierarchical 16-QAM HP
and LP symbol bits allocated to macroblock j in frame i in a GOP respectively. Given an overall coded video bits, Rbit , the optimal hierarchical 16-QAM symbol bits allocation problem is therefore to assign RHPi , j and RLPi , j to the H.264/AVC coded video such that the expected received video quality,
Q _ Total ( F 16HQAM ) , is maximized, which corresponds to minimizing the expected video quality distortion at the receiver due to the inherent noisy environment of wireless channel. The optimization problem, therefore, is to find the hierarchical QAM HP and LP symbol bits allocation, F 16 HQAM , that maximizes Q _ Total ( F 16HQAM ) . In view of the descending priority of macroblock within a frame, the number of hierarchical QAM HP and LP symbol bits allocated to macroblock j of frame i, is subject to the following constraints:
RHPi ,1 ≥ RHPi , 2 ≥ ... ≥ RHPi , j ≥ ... ≥ RHPi , N (4) RLPi ,1 ≤ RLPi , 2 ≤ ... ≤ RLPi , j ≤ ... ≤ RLPi , N (5) Taking into consideration the descending frame priority in a GOP, the total number of hierarchical QAM HP and LP
Since successfully decoding macroblock(1,2) requires that the current and previous macroblocks are received without a single bit error, the probability of successfully decoding macroblock(1,2) is then given by
PSD(1,2) = (1 − PE (1,2))
K 1, 2
× (1 − PE (1,1))
K 1 ,1
(10)
In general, the probability of successfully decoding macroblock(1,j), where 2 ≤ j ≤ N , is then given by
PSD(1, j ) = (1 − PE (1, j ))
K 1, j
× PSD(1, j − 1) (11)
Since successfully decoding macroblock(2,1) requires that the current macroblock and the macroblock from the previous frame are received without a single bit error, the probability of successfully decoding macroblock(2,1) is then given by
PSD (2,1) = (1 − PE (2,1))
K 2 ,1
× (1 − PE (1,1))
K 1,1
(12)
In general, the probability of successfully decoding macroblock(i,1), where 2 ≤ i ≤ L , is then given by
PSD (i ,1) = (1 − PE (i ,1))
Authorized licensed use limited to: MULTIMEDIA UNIVERSITY. Downloaded on October 10, 2009 at 22:18 from IEEE Xplore. Restrictions apply.
K i ,1
× PSD (i − 1,1)
(13)
Y. C. Chang et al.: A Low Complexity Hierarchical QAM Symbol Bits Allocation Algorithm for Unequal Error Protection of Wireless Video Transmission 1093
Since successfully decoding macroblock(2,2) requires that the current macroblock, the previous macroblock in the current frame and the macroblock from the previous frame are received without a single bit error, the probability of successfully decoding macroblock(2,2) is then given by
PSD (2, 2 ) = (1 − PE (2, 2 ))
In view of the descending priority of macroblock within a frame, the number of hierarchical 64-QAM HP and LP symbol bits allocated to macroblock j in frame i, is subject to the following constraints:
RHPi ,1 ≥ RHPi , 2 ≥ ... ≥ RHPi , j ... ≥ RHPi , N (20)
× (1 − PE (2,1))
K2,2
K 2 ,1
× (1 − PE (1,2 ))
K1 , 2
× (1 − PE (1,1))
K1 ,1
RLPi ,1 ≤ RLPi , 2 ≤ ... ≤ RLPi , j ... ≤ RLPi , N (14)
In general, the probability of successfully decoding macroblock(i,j), where 2 ≤ j ≤ N and 2 ≤ i ≤ L is then given by
PSD(i, j ) = (1 − PE (i, j ))
Ki , j
× PSD (i, j − 1)
× (1 − PE (i − 1, j ))
K i −1, j
where j=1,2,…,N and N is the number of macroblocks in a frame. The number of hierarchical 64-QAM MP symbol bits allocated to macroblock j in frame i, is then given by RMPi ,1 ≤ RMPi , 2 ≤ ... ≤ RMPi , j ≤ ... ≤ RMPi , P (22)
RMPi , P ≥ RMPi , P +1 ≥ ... ≥ RMPi , N (15)
The expected total PSNR increment, denoted as Q_Total, is then given by N
Q _ Total = ∑∑ PSD(i, j ) × Q(i, j )
(16)
j =1
In this paper, it is assumed that there is a channel estimator that is able to indicate the error probability of HP and LP data. The optimal allocation problem is then solved by allocating the hierarchical QAM HP and LP symbol bits to the macroblocks so that the probability of successfully decoding all the macroblocks in a GOP is maximized. The optimal allocation problem can then be written as
N
subject to Rbit =
∑ ∑ (RHP j =1
L
i =1
i, j
+ RLPi , j )
(17) (18)
B. Problem Formulation 2: Optimal Allocation of Hierarchical 64-QAM HP, MP and LP Symbol Bits to H.264/AVC Video. Let F 64 HQAM denotes the hierarchical 64-QAM HP, MP and LP symbol bits allocation to the H.264/AVC coded video and is given by
1, j
j =1
j =1
N
N
N
..., RHPi , j , RMPi , j , RLPi , j ,..., RHPL , N , RMPL , N , RLPL , N ) (19)
where RHPi , j , RMPi , j and RLPi , j are the hierarchical 64QAM HP, MP and LP symbol bits allocated to macroblock j in frame i in a GOP respectively.
j =1
j =1
j =1
j =1
≤ ∑ RLP2, j ≤ ... ≤ ∑ RLPi , j ... ≤ ∑ RLPL , j
(25)
The total number of hierarchical 64-QAM MP symbol bits allocated to frame i in a GOP is then given by N
∑ RMP
1, j
N
j =1
N
∑ RMP j =1
N
N
≤ ∑ RMP2, j ≤ ... ≤ ∑ RMPi , j ... ≤ ∑ RMPQ , j (26) Q, j
j =1
j =1
N
N
j =1
j =1
≥ ∑ RMPQ +1, j ≥ ... ≥ ∑ RMPL , j (27)
The optimal hierarchical 64-QAM allocation problem is then solved by allocating HP, MP and LP symbol bits to the macroblocks so that the probability of successfully decoding all the macroblocks in a GOP is maximized. The optimal allocation problem can then be written as Maximize Q _ Total ( F 64HQAM ) , N
Subject to Rbit = ∑ j =1
F 64 HQAM = ( RHP1,1 , RMP1,1 , RLP1,1 , RHP2, 2 , RMP2, 2 , RLP2, 2 ,
N
where i= 1,2,…,L
j =1
N
N
≥ ∑ RHP2, j ≥ ... ≥ ∑ RHPi , j ... ≥ ∑ RHPL , j (24)
1, j
j =1
Maximize Q _ Total ( F 16HQAM ) ,
N
∑ RHP
∑ RLP
i =1 j =1
(23)
Taking into consideration the descending frame priority in a GOP, the total number of hierarchical 64-QAM HP and LP symbol bits allocated to frame i in a GOP is given by N
L
(21)
∑ (RHP L
i =1
i, j
(28)
+ RMPi , j + RLPi , j ) (29)
C. Solution: A New Low Complexity Hierarchical QAM HP, MP and LP Symbol Bits Allocation Algorithm, Ranking Search Algorithm. The optimal allocation problems in (17) and (28) can be solved by using computational extensive exhaustive search method. However, the use of exhaustive search method is unrealistic for real-time applications because of the excessive
Authorized licensed use limited to: MULTIMEDIA UNIVERSITY. Downloaded on October 10, 2009 at 22:18 from IEEE Xplore. Restrictions apply.
Step 1. Calculate the PSD for all the rows in all the frames in a GOP. Step 2. Allocate hierarchical QAM HP symbol bits to the row with the highest PSD. Step 3. Allocate hierarchical QAM HP symbol bits to the rows with the next highest PSD until the total allocated HP symbol bits is equal to the HP bits of the hierarchical 16/64-QAM. Continue to step 4 for hierarchical 64-QAM. Otherwise, continue to step 5. Step 4. Allocate hierarchical QAM MP symbol bits to the rows with the next highest PSD until the total allocated MP symbol bits is equal to the MP bits of the hierarchical 64-QAM. Step 5. Allocate hierarchical QAM LP symbol bits to the rest of the rows. TABLE 2 COMPUTATIONAL COMPLEXITY ANALYSIS OF OUR PROPOSED LOW COMPLEXITY ALLOCATION ALGORITHM
Video Sequence
Carphone Suzie
Number of Trials Required (Proposed Number of Trials Ranking Search Low Required (Exhaustive Complexity Symbol Search Method) Bits Allocation Algorithm) 302 1 280 1
RHP2,1 = K2,1 RLP2,1 = 0
1
RHP3,1 = K3,1 RLP3,1 = 0
1
RHP15,1 = 0 RLP15,1 = K15,1
2
RHP1,2 = K1,2 RLP1,2 = 0
2
RHP2,2 = K2,2 RLP2,2 = 0
2
RHP3,2 = K3,2 RLP3,2 = 0
2
RHP15,2 = 0 RLP15,2 = K15,2
3
RHP3,3 = K3,3 RLP3,3 = 0
3
RHP15,3 = 0 RLP15,3 = K15,3
3
M
RHP1,3 = K1,3 RLP1,3 = 0
3 . . .
RHP1,M = K1,M RLP1,M = 0
M
Frame 1 (I)
RHP2,3 = K2,3 RLP2,3 = 0
. . .
RHP2,M = 0 RLP2,M = K2,M
M
Frame 2 (P)
...
Row
1
Row
RHP1,1 = K1,1 RLP1,1 = 0
Row
1
. . .
. . .
RHP3,M = 0 RLP3,M = K3,M
M
Frame 3 (P)
RHP15,M = 0 RLP15,M = K15,M Frame 15 (P)
GOP
RHPi,j = Ki,j RLPi,j = 0
RHPi,j = 0 RLPi,i = Ki,j
= HP data
= LP data
Fig. 8. Illustration of our proposed ranking search low complexity hierarchical 16-QAM HP and LP symbol bits allocation algorithm.
RHP1,2 = K1,2 RMP1,2 = 0 RLP1,2 = 0
3
RHP1,3 = K1,3 RMP1,3 = 0 RLP1,3 = 0
. . . M
RHP2,1 = K2,1 RMP2,1 = 0 RLP2,1 = 0
2
RHP2,2 = K2,2 RMP2,2 = 0 RLP2,2 = 0
3
RHP2,3 = K2,3 RMP2,3 = 0 RLP2,3 = 0
1
. . .
RHP1,M = K1,M RMP1,M = 0 RLP1,M = 0 Frame 1 (I)
RHP3,1 = K3,1 RMP3,1 = 0 RLP3,1 = 0
2
RHP3,2 = K3,2 RMP3,2 = 0 RLP3,2 = 0
3
RHP3,3 = K3,3 RMP3,3 = 0 RLP3,3 = 0
1
...
. . .
M
RHP2,M = K2,M RMP2,M = 0 RLP2,M = 0
M
Row
2
1
Row
RHP1,1 = K1,1 RMP1,1 = 0 RLP1,1 = 0
Row
1
Row
amount of computation required. In order to reduce the number of computations required in (17) and (28), a low complexity allocation algorithm, namely ranking search algorithm, which can substantially reduce the computational complexity for the optimal UEP allocation problems in (17) and (28), is proposed and described in the following. As shown in Fig. 8, our proposed low complexity allocation algorithm divides one frame into M rows during the UEP allocation process, instead of dividing one frame into N macroblocks. As N, which is the number of macroblocks in a frame in (18) and (29) has been reduced to M, which is the number of rows in a frame, and also the fact that M is much smaller than N, computational complexity in (16) will be significantly reduced. Our proposed ranking search low complexity symbol bits allocation algorithm makes use of the PSD in (9), (11), (13) and (15) and is based on the fact that the higher the PSD, the more important the macroblock is since macroblock with a higher PSD will cause more number of macroblocks error propagation if transmission errors occur on the macroblock. Because of this, our proposed low complexity symbol bits allocation algorithm allocates high priority symbol bits with lower BER to the macroblock with the highest PSD. The macroblocks with the next highest PSD are also allocated with high priority symbol bits until all the high priority symbol bits are allocated. Once all the high priority symbol bits have been allocated, the macroblocks with the next highest PSD are then allocated with medium priority symbol bits until all the medium priority symbol bis are allocated. Finally, the remaining macroblocks are allocated with low priority symbol bits. The following steps summarize our proposed low complexity symbol bits allocation algorithm.
Row
IEEE Transactions on Consumer Electronics, Vol. 55, No. 3, AUGUST 2009
1094
2
RHP15,2 = K15,2 RMP15,2 = 0 RLP15,2 = 0
3
RHP15,3 = K15,3 RMP15,3 = 0 RLP15,3 = 0
. . .
RHP3,M = K3,M RMP3,M = 0 RLP3,M = 0
M
Frame 3 (P)
Frame 2 (P)
RHP15,1 = K15,1 RMP15,1 = 0 RLP15,1 = 0
RHP15,M = K15,M RMP15,M = 0 RLP15,M = 0 Frame 15 (P)
GOP
RHPi,j = Ki,j RMPi,j = 0 RLPi,j = 0
= HP data
RHPi,j = 0 RMPi,j = Ki,j RLPi,j = 0
= MP data
RHPi,j = 0 RMPi,j = 0 RLPi,j = Ki,j
= LP data
Fig. 9. Illustration of our proposed ranking search low complexity hierarchical 64-QAM HP, MP and LP symbol bits allocation algorithm
Table 2 shows the number of iterations or trials required for our proposed low complexity allocation algorithm, in comparison with the exhaustive search method in (28). Since our proposed ranking search low complexity symbol bits allocation algorithm does not require exhaustive trial and error process and the allocation process can be completed in 1 trial process with the calculation of PSD, it is able to reduce significantly the number of trials required. IV. SIMULATION RESULTS A software simulation was carried out for our proposed UEP of H.264/AVC coded video transmission using hierarchical 16/64-QAM. The overall block diagrams of our proposed UEP using hierarchical QAM are shown in Figs. 3 and 4 respectively. H.264/AVC official reference software was used and the hierarchical 16/64-QAM and AWGN channel model were designed in MATLAB. Two 30-frame video sequences, namely Carphone and Suzie in Quarter Common Intermediate Format (QCIF) of spatial resolution 176 x 144 pixels compressed to 40 Kbit/s were used in the simulation work. The coded group of picture (GOP) is 15. In other words, Iframe is inserted periodically every fifteen frames in order to prohibit the temporal error propagation when errors occur during transmission. It is anticipated that other combinations of I-frames and P-frames will result in similar results for our proposed UEP scheme. The transmitted signal is subject to additive white Gaussian noise (AWGN). Our proposed UEP scheme is benchmarked against the classical EEP scheme and
Authorized licensed use limited to: MULTIMEDIA UNIVERSITY. Downloaded on October 10, 2009 at 22:18 from IEEE Xplore. Restrictions apply.
Y. C. Chang et al.: A Low Complexity Hierarchical QAM Symbol Bits Allocation Algorithm for Unequal Error Protection of Wireless Video Transmission 1095
the UEP scheme in [6]. In order to make sure that our proposed UEP scheme and the benchmark UEP scheme in [6] result the same transmission bit-rate, our UEP scheme is protected by a convolutional encoder of rate 1/2 while a convolutional code rates of 1/3, 1/2 and 2/3 are used to protect the HP, MP and LP data respectively in [6]. PSNR between the reconstructed video and the original video is used to measure the reconstructed video quality. The higher the PSNR value of the reconstructed video quality, the better the video quality. Results of thirty simulations, performed with different AWGN seeds, were averaged in order to obtain more reliable results. The average PSNR, is thus given by
Average _ PSNR =
1 30 ∑ PSNR(s) 30 s=1
degradation in video quality in [6]. Our proposed UEP scheme is also found to outperform the UEP scheme in [6] in subjective video quality test as it can be clearly seen in Figs. 17 to 20 that our UEP scheme results in a much better subjective visual quality.
(30)
No error concealment is applied to the decoded macroblocks with transmission errors and thus, our PSNR values for video sequences corrupted by transmission errors are generally lower than the PSNR values reported in other literatures. Figs. 10 to 12 show the average PSNR performance of our proposed UEP using hierarchical 16- and 64-QAM for a range of alpha values, in comparison with the classical EEP scheme. It can be seen that in terms of average PSNR, our proposed UEP scheme using hierarchical QAM outperforms the classical EEP scheme by up to 4dB at low SNR condition. In other words, our proposed UEP scheme has much better visual quality when the SNR of the channel is low. This can be attributed to the fact that when the channel SNR is low, the probability of errors occurring to the I-frames and earlier P-frames are much lower, since the Iframes and earlier P-frames are better protected against channel noise than late P-frames. However, when the channel quality is good (SNR = 18.5 dB and 19.0 dB), our proposed UEP scheme may result in lower visual quality due to the inherent higher BER of hierarchical 16-QAM. Based on the results shown in Figs. 10 and 11, the optimum alpha value for hierarchical 16-QAM, which results in the highest PSNR values at most of the channel conditions is 1.10. The optimum alpha and beta values for hierarchical 64-QAM is thus 1.10 and 1.10 respectively. The performance of our proposed UEP scheme, in comparison with the UEP scheme in [6], is shown in Figs. 13 and 14. It can be seen that our UEP scheme outperforms the UEP scheme in [6] in all the channel conditions. The frame by frame PSNR values in Figs. 15 and 16 show that the PSNR values of our UEP scheme drop slowly, as compared to the sudden drop of PSNR values after frame 7 and 22 for UEP scheme in [6]. The reason why PSNR values for the UEP scheme in [6] experience a drop down from frames 7 to 15, and frames 22 to 30, is because they are protected by low protection order convolutional code, with a code rate of 2/3. The low protection order convolutional code causes a higher bit error rate for these frames and this causes a sudden drop down for the PSNR values after frame 7 and 22 for the UEP scheme in [6]. This shows that our UEP scheme offers a gradual degradation in video quality, as compared to the sharp
Fig. 10. Average PSNR performance of our proposed UEP scheme using hierarchical 16-QAM in comparison with the classical EEP scheme for Carphone video sequence. No channel coding protection is applied for this simulation.
Fig. 11. Average PSNR performance of our proposed UEP scheme using hierarchical 16-QAM in comparison with the classical EEP scheme for Suzie video sequence. No channel coding protection is applied for this simulation.
Fig 12. Average PSNR performance of our proposed UEP scheme using hierarchical 64-QAM in comparison with the classical EEP scheme for Suzie video sequence. No channel coding protection is applied for this simulation.
Authorized licensed use limited to: MULTIMEDIA UNIVERSITY. Downloaded on October 10, 2009 at 22:18 from IEEE Xplore. Restrictions apply.
1096
Fig. 13. Average PSNR performance of our proposed UEP scheme using hierarchical 16-QAM, in comparison with the UEP scheme in [6] for Carphone video sequence.
IEEE Transactions on Consumer Electronics, Vol. 55, No. 3, AUGUST 2009
Fig. 16. Average PSNR versus frame number of our proposed UEP scheme using Hierarchical 16-QAM, in comparison with the UEP scheme in [6] for Suzie video sequence at SNR=18.0dB, alpha=1.20.
(a) (b) Fig. 17. Comparison of subjective reconstructed video quality for Carphone video sequence at SNR=14.0dB; (a) UEP scheme in [6]; (b) Our proposed UEP scheme.
Fig. 14. Average PSNR performance comparison of our proposed UEP scheme using hierarchical 16-QAM, in comparison with the UEP scheme in [6] for Suzie video sequence.
Fig. 18. Comparison of subjective reconstructed video quality for Carphone video sequence at SNR=15.0dB; (a) UEP scheme in [6]; (b) Our proposed UEP scheme.
Fig. 15. Average PSNR versus frame number of our proposed UEP scheme using hierarchical 16-QAM, in comparison with the UEP scheme in [6] for Carphone video sequence at SNR=18.0dB, alpha=1.20.
Fig. 19. Comparison of subjective reconstructed video quality for Suzie video sequence at SNR=14.0dB; (a) UEP scheme in [6]; (b) Our proposed UEP scheme.
Authorized licensed use limited to: MULTIMEDIA UNIVERSITY. Downloaded on October 10, 2009 at 22:18 from IEEE Xplore. Restrictions apply.
Y. C. Chang et al.: A Low Complexity Hierarchical QAM Symbol Bits Allocation Algorithm for Unequal Error Protection of Wireless Video Transmission 1097
Fig. 20. Comparison of subjective reconstructed video quality for Suzie video sequence at SNR=15.0dB; (a) UEP scheme in [6]; (b) Our proposed UEP scheme.
V. CONCLUSIONS AND FUTURE WORK UEP of H.264/AVC coded ideo transmission using hierarchical QAM, which takes into consideration the nonuniformly distributed importance of I- and P-frames, is first proposed in this paper. In order to optimally allocate the hierarchical QAM HP, MP and LP symbol bits to the H.264/AVC coded video, a generic solution that is able to optimally allocate hierarchical QAM symbol bits to H.264/AVC coded video is then proposed. Finally, a low complexity optimal allocation algorithm, which can substantially reduce the computational complexity of our proposed generic optimal allocation algorithm for UEP of wireless video transmission, is proposed. Simulation results show that our proposed UEP scheme outperforms the classical EEP scheme and also UEP scheme in [6]. The inclusion of Rayleigh fading channel model in the simulation model shall be implemented in the future. REFERENCES [1]
[2] [3] [4] [5] [6]
[7] [8] [9]
Yoong Choon Chang, Sze Wei Lee, Ryoichi Komiya, "A Fast Forward Error Correction Allocation Algorithm for Unequal Error Protection of Video Transmission over Wireless Channels," IEEE Transactions on Consumer Electronics, vol. 54, no. 3, pp. 1066-1073, August 2008. Yao Wang, Jorn Ostermann and Ya-Qin Zhang, Video Processing and Communications, Prentice Hall, 2002, pp. 472-509. Abdul H. Sadka, Compressed Video Communications, John Wiley & Sons, 2002, pp. 121-172. Yao Wang and Q.-F. Zhu, “Error control and concealment for video communication: A review,” Proceedings of the IEEE, vol. 86, pp. 974997, May 1998. Mohammed Ghanbari, Standard Codes: Image Compression to Advanced Video Coding, IEE, United Kingdom, 2003, pp. 256-272. Francois Marx and Joumana Farah, “A novel approach to achieve unequal error protection for video transmission over 3G wireless networks,” Signal Processing: Image Communication, vol. 19, pp. 313323, April 2004. Lee-Fang Wei, “Coded modulation with unequal error protection,” IEEE Transactions on Communications, vol. 41, no. 10, pp. 1439-1449, October 1993. Seamus O’Leary, “Hierarchical transmission and COFDM systems,” IEEE Transactions on Broadcasting, vol. 43, no. 2, pp. 166-174, June 1997. Chee-Siong Lee, Thoandmas Keller and Lajos Hanzo, “OFDM-Based turbo-coded hierarchical and non-hierarchical terrestrial mobile digital video broadcasting,” IEEE Transactions on Broadcasting, vol. 46, no. 1, pp. 1-22, March 2000.
[10] B. Barmada, M. M. Ghandi, E. V. Jones, M. Ghanbari, “Prioritized transmission of data partitioned H.264 video with hierarchical QAM,” IEEE Signal Processing Letters, vol. 12. no. 8, pp. 577-580, August 2005. [11] Alexander Schertz and Chris Weck, “Hierarchical modulation - the transmission of two independent DVB-T multiplexes on a single frequency,” EBU Technical Review. April 2003. [12] Hamid Gharavi, “Pilot-Assisted 16-Level QAM for wireless video,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 12, no. 2, pp. 77-89, Feb. 2002. [13] Vasileios Theodorakopoulos and Michael E. Woodward, “Comparative analysis of a twin-class M-QAM transmission system for wireless video application,” Multimedia Tools and Aplications. Springer, vol. 28, pp.125-139, February 2006.
Yoong Choon CHANG obtained his B.Eng (Hons) in Electrical & Electronic Engineering from University of Northumbria at Newcastle, UK, MEngSc and PhD from Multimedia University, Malaysia in 1998, 2003 and 2008. He was an Electrical Engineer in an Electrical & Mechanical contractor company at Malaysia from 1999 to 2001 before he joined Multimedia University, Malaysia in 2001 as a tutor. He is a lecturer at the Faculty of Engineering, Multimedia University, Malaysia since May 2003. His research interests include wireless multimedia communications and video coding. He is a member of IEEE. Sze Wei LEE obtained his BEng in Electronics, MPhil and PhD in digital communications from University of Manchester Institute of Science and Technology, UK in 1995, 1996 and 1998 respectively. He joined Multimedia University as a lecturer in 1999 and later became a senior lecturer and associate professor in 2001 and 2004 respectively. He is a professor at University Tunku Abdul Rahman, Malaysia since 2008. His research interests include wireless networking and communications. He has supervised more than 20 master and PhD postgraduates and published more than 10 international journal papers and 30 international conference papers thus far. He has successfully completed numerous research projects funded by the Malaysian Government and collaborative research projects with the industry. Ryoichi KOMIYA was born in Tokyo, Japan on March 16, 1945. He received the B.E. and Ph.D. degrees from Waseda University, Tokyo, Japan, in 1967 and 1986, respectively. Since he joined the Electrical Communication Labs of NTT in 1967, he has been engaged in the development of the PCM repeatered line, digital data terminal equipment, video coder/decoders, stuff multiplexers, ISDN subscriber loop transmission systems and fiber optic remote multiplexer systems. After quitting from NTT in 1992, he was in Siemens, Nippon telecommunication consulting co.ltd., NTT Advanced Technology and Distribution and Economics University in Japan. He is currently at Faculty of Information Technology of Multimedia University, Malaysia, where he is responsible for research and development of next generation telecommunication systems, services, terminals, IP network, virtual education environment, e-commerce terminal, Intelligent Transportation System, Fiber to The Home and smart home. Professor Komiya is a member of IEEE and IEICE.
Authorized licensed use limited to: MULTIMEDIA UNIVERSITY. Downloaded on October 10, 2009 at 22:18 from IEEE Xplore. Restrictions apply.
