Future Generation Computer Systems (
)
–
Contents lists available at ScienceDirect
Future Generation Computer Systems journal homepage: www.elsevier.com/locate/fgcs
Power-efficient video encoding on resource-limited systems: A game-theoretic approach Wen Ji a , Jiangchuan Liu b , Min Chen c,∗ , Yiqiang Chen a a
Institute of Computing Technology, Chinese Academy of Sciences, China
b
School of Computing Science, Simon Fraser University, Canada
c
School of Computer Science and Engineering, Seoul National University, South Korea
article
info
Article history: Received 18 October 2010 Received in revised form 27 March 2011 Accepted 7 April 2011 Available online xxxx Keywords: Video encoding system Power control Game theory
abstract Most mobile video applications are often deployed on battery-operated devices. With limited power supply, it is a challenging issue to support video applications with high resolution due to the complex functionality and high resource requirements. Thus, power-efficient design is important in computation intensive applications especially for mobile video terminals. Previous works on power-efficient control in video encoding systems focus on the low complexity design while typically ignoring the impact of scalable design by considering various power consumption involved in the encoding process. This paper is dedicated to developing a power-scalable video encoding (PSVE) strategy for energy-limited mobile terminals. PSVE can help the video encoding terminal to adjust its power consumption budget efficiently so as to enhance the power-scalable capability in mobile video terminals. This paper first establishes game theoretical analysis to model the power consumption problem as a bargaining problem. Then, the tradeoff between encoding effect and power consumption achieved by the use of game theory. The scalable and low power video encoding system based on Nash equilibrium solution is derived through the analysis on the power consumption and encoding effect. Experimental results demonstrate the efficiency of the proposed approach. © 2011 Published by Elsevier B.V.
1. Introduction 1.1. Motivation Video encoding applications have become very popular and prevalent in recent years. Compressed video data cover more and more application domains, including digital movies, internet videos, home entertainment, digital libraries, news programs, and multimedia searching. Furthermore, emerging applications pose unique challenges for resource-limited video, such as wireless video camera systems, wireless low power surveillance networks, wireless video sensor networks, mobile video phones, etc., where low power encoders are important because computational power, memory and battery energy are scarce. In most cases, video applications often operate on these mobile devices or terminals with limited battery resources. It is well known that using power control mechanisms can increase battery efficiency and also extend the battery life. Since in many scenarios, low power control
∗
Corresponding author. E-mail addresses:
[email protected],
[email protected] (W. Ji),
[email protected] (J. Liu),
[email protected] (M. Chen),
[email protected] (Y. Chen). 0167-739X/$ – see front matter © 2011 Published by Elsevier B.V. doi:10.1016/j.future.2011.04.002
schemes are required in order to prolong the battery life. Thus, energy efficiency becomes one of the most critical issues in the design of battery-powered portable devices. Consequently, the immense growth of the amount of video encoding systems requires appropriate methods and strategies to control and manage the video encoder efficiently and powerfully. However, there are still many open problems in terms of how to efficiently provision power adaptability video encoding for energylimited systems. The main technical difficulties are as follows: (1) Efficient video encoding significantly reduces the amount of the video data to be storied or transmitted, which saves much energy consumption correspondingly. However, more efficient video encoding leads to higher computational complexity and larger power consumption. In order to lower complexity and power consumption, video encoder need to provide power-efficient control especially under energy constraint applications. (2) Video encoding is a hybrid architecture based on the amount of coding tools, such as motion estimation, prediction mode decision, discrete cosine transform, and entropy coding [1,2]. The implementation complexity/power consumption heavily relies on the characteristics of the hardware platform (e.g., DSP
2
W. Ji et al. / Future Generation Computer Systems (
processor, FPGA, ASIC). Without loss of generality, platform independent analysis is employed to express the module complexity. However, it still has close relation with video content, profile defined, etc. Thus, in complexity/power control video encoder, the rate-distortion-complexity behavior analysis among coding modules should be taken into consideration. (3) Under resource-limited systems, all the coding modules compete for a fair share of resources, such as power consumption budget, bit rate target. In energy-limited applications, video encoding works as an entity such that the composed coding modules compete for available power consumption constraint to optimize it own quality. However, most of previous work on available computational resource allocation in video encoding does not directly apply competition, and a new optimization solution is needed. Traditionally video encoding and power control in energy resources limited are designed separately. Intense research focus on: (1) to minimize the energy consumption or to maximize the lifetime achieved while meeting required video encoding quality; (2) to maximize the video encoding quality while meeting required resource constraints. However, in order to support video streaming over variable and limited energy support circumstance, video encoding need to be adapted through a variety of schemes. This paper aims at this topic: power-aware and scalable design so as to adapt the variable resource, while meeting the tradeoff between video encoding quality and energy resource constraints. 1.2. Background and related work Intense research has been conducted in the field of designing low power video encoding systems. Research worked on power control video encoding techniques have been conducted based on different platforms, these are system level design [3], algorithm level [4,5], circuit levels, and architecture [6], etc. To achieve low power consumption video encoding, Mochizuki et al. [7] developed a low power H.264/MPEG-4 video codec for mobile applications. Their codec is capable of encoding/decoding HD (high definition) video sized moving pictures in real time while employing only 64 mW power for encoding HD with high picture quality. Prasad and Korada [8] implement the MPEG-4 video encoder on RISC (Reduced Instruction Set Computing) core with low power consumption. Satoshi Kumaki et al. [9] develop a MPEG-2 video encoder with scalable architecture. The power consumption of this encoder is suppressed to 0.7 W by adopting a low power DRAM core. In general video encoders, motion estimation occupies most computational complexity and memory access, so that it becomes the most critical component for low power applications. Stewart et al. [10] introduced the results of the intraprediction stage into motion estimation process, and thus further enhancing the power reduction achievable through bit width reduction. Miyama et al. [11] featured a gradient descent search algorithm into the motion estimation processor core so as to reduce computational complexity. From integrated circuit aspect, Chen et al. [12] proposed a hardware-oriented fast algorithm in integrate motion estimation. They use a parallel architecture with efficient data reuse techniques. More than half computational complexity is saved by this fast algorithm, and most memory bandwidth is further saved with data reuse techniques. For the rest other components in video encoder, August and Dong Sam Ha [13] studied low power design techniques for discrete cosine transform (DCT) and inverse discrete cosine transform (IDCT) modules. They use skipping DCT computation of low energy macroblocks, skipping IDCT computation of blocks with all coefficients equal to zero, lower precision constant multipliers, and reducing transitions in the data path. These techniques reduce
)
–
power dissipation above 90% in DCT and IDCT circuits. Parlak and Hamzaoglu [14] presented efficient low power deblocking filter hardware implementations. These implementations include starting filtering the available edges by using a novel edge filtering order to overlap the execution of deblocking filter module with other modules in the H.264 encoder. Their work is rudimental to ours. We combine these thoughts in scalable encoder and methods in low power design so as to achieve integration scalability and efficiency. The problem of power adaptive video encoding has attracted attention from the research community only recently. Related work in this area can be divided into two categories: framework (e.g., [4,15]) and module-based (e.g., [16]). For example, in [4], Chung-Jr et al. proposed a power adjusting architecture to achieve optimal visual quality under the rate and power constraints. In [15], Tan et al. designed a parameterized complexity scalable scheme so as to meet power constraints requirements. From the view of module-based methods, Deepak et al. [16] present a comprehensive analysis of the encoder complexity in terms of computations required during motion estimation (ME) module. They derived expressions for the ME complexity under different coding parameters and combined with rate-distortion model to determine the encoding structure and parameters for optimal ratedistortion-complexity tradeoffs. One the other hand, since there is close relation among power consumption, energy consumption and computational cost, specially for video signals, which belong to period sources, it is well accepted that the results on power consumption are equivalent to energy consumption. Furthermore, recent work bridge power/energy consumption in video encoding process to computational cost measurement, such as [16–20]. To achieve better encoding quality in variable and limited energy support system, a more intelligent approach to video adaptation is needed. In this case, all the coding modules in video encoder compete for limited resources. However, to our best knowledge, there has been no analytic framework for solving the energy/power resource competition problem in video encoding. Recently, game theory has been introduced to mobile applications and becomes an effective tool for solving power control problem in resource constraint circumstance. Many researches have focused on using non-cooperative game theory to model power allocation problem in wireless networks. Non-cooperative power control game maximizes the individual utility while keeping global optimality, which shows advantages for solving the tradeoff in resource allocation problem. In [21], game theory has been recognized as an emerging solution for rate control in video coding systems. Game theory has been used to study multi-player decision problems [22]. A well known property from game theory is the equilibrium optimization based on notions of fairness, efficiency, and mathematical characteristics [23]. These show great potentials in providing flexible power control method in video encoding systems. 1.3. Summary of contributions In this paper, we proposed a power-scalable video encoding architecture for energy constraint systems and investigated the Nash equilibrium in game theory for video encoding scalable problem. Our main contributions include three aspects: (1) presenting a integrated framework for power-efficient control video encoding, which perceives and allocates the available energy resource to composed coding modules; (2) deriving a scalable and low power control video encoding system based on Nash equilibrium solution, so as to maintain low power consumption at each scalable level; (3) investigating how to apply optimal criteria, such as sub-game perfection and fairness, to obtain Nash equilibrium solution.
W. Ji et al. / Future Generation Computer Systems (
)
–
3
Fig. 1. Illustration of the power-scalable control video encoding system.
This paper is organized as follows. After briefly reviewing, we first describe the general solution framework in Section 2. In Section 3, we discuss in details of how the framework use game theory to solve the tradeoff between encoding effect and power consumption so as to obtain better performance in each energy levels. Section 4 evaluate them; and Section 5 concludes. 2. Framework of power-efficient video encoding Since power control problem is an obstacle in the way of making mobile multimedia more feasible and popular, how to economize the use of energy becomes very important specially in resource-limited applications. There are three major issues: (1) smart energy resource allocation method; (2) maximum the video encoding effect; (3) minimum the energy consumption of video encoding. Thus, typical power control video encoding design includes two parts, one is to provide a scalable power control method so as to adapt the available resource; the other is to find a good configuration that has optimal visual quality and optimal bit rate under power constraint. In practice, the encoding effect varies with both composed modules and video contents. The encoding complexity and corresponding power consumption show obvious regulation in unit of modules [24,1]. Motivated by [5,17], through analysis the rate-distortion-complexity relation among composed encoding modules, we can translate the power control in video encoding process to composed modules complexity adjusting problem. Then, in this section, we describe a solution framework that answers how to provide power-scalable control in video encoding system, as shown in Fig. 1. This framework involves the following steps. 2.1. Power budget mapping and computing First, compute the power budget of video encoding process. This module includes two parts. Part 1, user can specify working mode of video encoder. These states include ‘Maximum battery time mode’, ‘Battery optimized mode’, ‘Maximum performance mode’ and so on. Each state corresponds to a battery working mode of the device. These states are widely used in mobile devices and terminals. This step ensure the implementation of power-aware operation. Part 2, automatic adjust working mode of video encoder according to remaining battery capacity aware. For instance, ‘Maximum battery time mode’ can be used automatically when
the residual capacity is under 30% while ‘Maximum performance mode’ is adopted automatically in case of available battery capacity beyond 90%. Specifically, when the results of part 1 and part 2 conflict, i.e. user configures his device as ‘Maximum performance mode’ but the residual capacity is under 30% at that time, the final available power consumption profile is computed based on remaining battery capacity aware. That is to say part 2 has high priority than part 1, and user can manually specify working mode only when device resource is sufficient. Thus, this module implements a mapping and computing bridge between the power profile and available energy resource. 2.2. Module scalable based on R–D–C analysis For a typical video codec, it has been commonly recognized that the most consuming computation comes from the major modules, including motion estimation (ME), motion vector resolution (MVR), interprediction mode decision (InterMD) with variable block sizes, intraprediction mode decision (IntraMD), discrete cosine transform (DCT), deblocking filter (DF), and entropy coding (EC) [25,2]. For the trust, we analyze the general advanced video codec based on the AVS standard [26–28], the recent video coding standard developed by the Audio and Video Coding Standard Workgroup of China, which promises similar performance but lower complexity compared with H.264. And the conclusion from this discussion is easily applied or transferred to the other video standards such as MPEG-4/2, H.264, etc. [24,1]. Four modules, including ME, MVR, InterMD and IntraMD, cover nearly the main features of computation and memory complexity. Since each module has different effect on distortion, and it is difficult to compute this effect accurately, we estimate the relationship between encoding effect and corresponding complexity from empirical method. We build a database to collect each subconfiguration and its results in unit of module, these include the collection of the bit rate, distortion, and computational cost. For example, variable search range in motion estimation forms a behavior in ME module, through adjust the configuration of this behavior, we can further control encoding effect and corresponding power consumption of the whole video encoder. Consider a typical encoding system. Let ci denote the computational complexity measured in process cycle, qi and ri are respective the PSNR and coded bit rates, related to the encoding effect. i indexes the composed modules. We use a utility function Ψi (ri , qi , ci ) to demonstrate how
4
W. Ji et al. / Future Generation Computer Systems (
50
45 40 Y
35
PSNR
PSNRY
40
30 25
35 30 25
20
20
15 4000
15 8000
6.5
3800 3600 3200
Bit Rate
3000 5.5
6000
x 105
6
3400
4000 2000
Bit Rate
Computational Cost
0
(a) SR.
3
3.5
4
4.5
5
5.5
6.5
6 x 10
5
Computational Cost
(b) MVR.
50
48
45
46
40
44 Y
35
PSNR
Y
–
50
45
PSNR
)
30 25
42 40 38 36
20
34
15 4000
32 8000 6.5
3800 5.5
3400 Bit Rate
7000
6
3600 5
3200
6000
5
5000
x 10
4.5 3000 4
Bit Rate
Computational Cost
(c) IntraMD.
4000 3000 0
1
2
3
4
5
7
6
5
x 10
Computational Cost
(d) InterMD. Fig. 2. The rate-distortion-complexity behavior analysis on coding modules.
these each modules affect video encoding. Thus, we need to further investigate the rate-distortion-complexity behavior of each module so as to get the expression of Ψi (·). The method in [5] is introduced: ① From the view of reducing the overhead of power control, power constraints are translated to the encoding computation costs, which are measured by the number of scaled SAD operations [29]. That is, the complexity can be measured in video processing unit (VPU) and in term of SAD number. More precisely, it is measured in cycles in this paper. ② The relation among ri , qi , ci is estimated by the statistics of typical video sequences. Through the collection of the cycleconsumed, bit rate and distortion, the statistic results of these composed modules can be obtained.
• Motion estimation: It has been well recognized that motion estimation is the most complex and with highest computation consumption module. According to previous analysis in [1,25, 2], ME module can be summarized into two behaviors: (1) Search range (SR): as we know, better ME search quality need more computation and corresponding power consumption. To search for the best matched MB, full search ME, which search for all possible candidates in a search range, can guarantee the smallest Sum of Absolute Differences (SAD) value, but the complexity of exhausted search is the highest. Different ME algorithms have different search patterns and flows, but they can share the same processing unit of parallel tree to accumulate the SAD [4]. As mentioned in [1], increasing both reference frame numbers and search size leads to higher access frequency, up to approximately 60 times, while it has a minimal impact on PSNR and bit rate performances. Thus, SR reflects the computation
consumption in video codec under the rule of SAD. Fig. 2(a) shows the curving surface of RDC analysis, where the distortion is reflected by PSNR, and the complexity is mapping by the number of processing units consumed. (2) Motion vector resolution (MVR): the accuracy of motion vectors depend on search granularity, such as: integerpixel, fractional pixel in 1/2, and fractional pixel in 1/4. For the encoder, 1/2 pixel search results in a serious increase of access frequency processing time, and 1/4 pixel accuracy increases the processing time about 10% while reduces the bit rates up to 30% [1]. MVR is regarded as an important behavior feature for all ME algorithms. Its RDC analysis result is shown in Fig. 2(b). • Mode decision: It is quite common that new techniques adopted in video encoding, such as the spatial prediction in intramode coding, leads to mass increase in computational complexity. Mode decision is classified into two categories: inter and intraprediction decisions. For interprediction, most of computations consumed in this stage depend on variable block size ME employed. In H.264, seven different block sizes (16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8 and 4 × 4) are supported in intermode decision. In addition, the SKIP mode, direct mode and two intramodes (INTRA4 and INTRA16) are also supported in H.264. In AVS, there are four different block sizes (16 × 16, 16 × 8, 8 × 16, 8 × 8) in intermode decision, besides, the SKIP mode and five intramodes, including vertical (v) mode, horizontal (h) mode, DC (dc) mode, down-left (dl) mode, and down-right (dr) mode. Thus, to achieve best encoding efficiency, the encoder usually tries all these possible modes and select the best. Many researches focus on low complexity mode decision, they often use zero blocks detection and early termination techniques [30,31] or direction detection method [32,33] to decrease the
W. Ji et al. / Future Generation Computer Systems (
number of mode decisions. As we know, the objective of MD module is how to adaptively select candidate modes before an MB is actually coded. The other fact is that the complexity increases in proportion to the number of candidate modes. Consider the case that mode decision has n individualcandin i dates, then the mode decision module can produce i=1 Cn output. Therefore, it is possible to reduce the computational complexity by early termination after going through only a few modes in MD module, if the actual best matched mode is in the candidates [29]. Then the solution of MD module comes up to two points: (1) providing a scalable MD output through control the number of candidates; (2) trying to make actual best matched mode in the candidates. On the other hand, it is noticed that for mode decision module, the best matched mode does not comply with equal probability distribution. Then, acquire some statistics of the mode distribution beforehand can help to solve point (2). To solve these two points, we use two steps: (1) orderly arrange the elements in each set according to the statistics of the mode distribution; (2) orderly select the modes in each set, under consumed computation budget. There are: (1) Intramode decision. The combination of possible intrapredictions: let k be the number of candidate intramode, (i.e. k = 5 in AVS and k = 2 in H.264). The number of possible k outputs are i=1 Cki . In AVS, we build a simulation platform for intraprediction analysis. All the results of each mode are in worst case that only one mode is accepted in intracode, while best mode represents that the encoder are using best match intraprediction through fully computing all five modes. Experiments on video sequences yield approximately coincident results. Fig. 2(c) provides its RDC analysis result. (2) Intermode decision. Following the definition of intramode decision, this case is the combination of possible interpredictions. Given k candidate intermode, (i.e. k = 4 in AVS). 4 i The number of possible outputs are i=1 C4 . The possible value is the combination of these four different block sizes (16 × 16, 16 × 8, 8 × 16, 8 × 8). Fig. 2(d) gives the corresponding RDC analysis result. 2.3. Power allocation control The proposed power-scalable control includes two stages. In the first stage, target power consumption are allocated at the frame level. In the second stage, the fine-scalable control for each coding module is determined by game theoretical approach, which is discussed in the following section. At the frame level, power allocation is to calculate the power profile that the frame belongs to according to the result of power budget mapping and computing module and the result of the encoding effect and power re-computing module. It is as follows level Pprofile
(j) =
level Pbudget
(j) + εθ (j − 1)
(1)
() () frame j. θ (j − 1) is the revised information and can update the available power budget so as to obtain more fine control. θ (j − level level 1) = Pbudget (j − 1) − Pframe (j − 1). ε ∈ [0, 1] denotes the scale
level where Pbudget j is the power consumption budget level of frame level j Pprofile j represents the final available power consumption of
·
of perturbation. Fig. 1 illustrate the main process of this powerscalable control encoding system. At the module level, the objective is to find the best parameter configurations, then decide the best modes in its composed modules so as to keep video encoder work under best states in variable resource circumstance. Game-theoretic approach is introduced to solve the module competition under power constraint.
)
–
5
3. Using game theory to optimize power-efficient control in video coding Since the encoding process can be regarded as the aggregation of modules, each module is also under common resourceconstrained and is a competitor of limited energy for the others. It is widely accepted that, PSNR and bit rates are the measurement of encoding effect of all modules. However, a module unit will achieve high encoding quality while costs amount of energy. There is a tradeoff between obtaining high encoding effect and obtaining low energy consumption. Hence, finding a good balance between the two conflicting objectives is the primary focus of the power control and management. It is known that there are various configurations that can adjust power consumption in each module. Then power management is equivalent to do decision under certain conditions in these modules, and finally receive some benefit or tradeoff under energy resource constraint. On the other hand, the modules in encoder compete in limited resource. Each module expects enough resource to complete better performance. Then, games are characterized by the modules. Game theory is appropriate framework for modeling such a strategic situation. Thus, each module is regarded as a player in the game. These players compete for the use of a fixed power resource, which is the target power consumption budget. Thus, we propose a non-cooperative power allocation game, in which each module seeks to choose its encoding power over whole budget to maximize its overall utility. The concept of utility is commonly used in economics and refers to the level of satisfaction the decision-taker receives as a result of its actions. A utility function for each player reflects its preference [34]. Next, we will investigate how to build power control encoding game in resource constraint environments.
• Players: is the finite number of modules in the video encoding process, denoted by N. For the sake of clarity, we use i, i ∈ N to represent the SW, MVR, interMD, intraMD module in this paper, separately. The game model is easy extended to the case of N > 4 when other modules are introduced such as transform (T), entropy encoding (ENC), inverse transform (IT), etc. • Strategy space: The modules in video encoder are most complexity configurable. For every modules, each configuration represents a strategy and consequently leads to an encoding result. Thus, the strategy sets of each module constitute the strategy space. These are consistent with the modules player behavior, which had been discussed in Section 2 detailedly. • Cost: For any player i ∈ N, encoding a video frame, either for itself or for the others, incurs cost ci measured in terms of CPU cycles, which can easily mapping to the corresponding energy/power consumption according to Section 2.2. 3.1. Utility function in power control game Consider the block-based video encoding system. The modules compete limited resource. Game theory is the appropriate framework for modeling such a strategic situation. To pose the power control problem as a non-cooperative game, we first need to give more basic representation for video encoding systems. In this power allocation game, each module seeks to choose is encoding power over whole frame budget to maximize its overall utility. We use a non-cooperative power control game in which each module is allocated power budget so that it can reasonably select suitable mode during encoding modules (Fig. 3). For the sake of low power design, the expression of utility function is anticipated with efficiency but clearness. Since our focus here is on coding efficiency under consumed computation, we define the utility function of a player from the result of
6
W. Ji et al. / Future Generation Computer Systems (
)
–
Utility
Utility
Fixed Encoding Effect
Fixed Power
Encoding Effect
(a) Fixed power.
Power Consumption
(b) Fixed encoding effect.
Fig. 3. The utilities as a function of encoding effect and power consumption.
module behavior analysis, which reflect the relation among power consumption, quality and bit rates for each encoding adjusting. That is ui (·) = Ψi (·), where i represents module SW, MVR, interMD, intraMD, or more other modules separately. From the curve fitting of the RDC analysis, we can get the expression of the ui components. For the sake of simplicity, we use weight linear form to build a uniform utility of the power control game. That is, u = uSW + uMVR + uinterMD + uintraMD
(2)
boundaries according to the definition of pi . That is 0 ≤ pi ≤ P. Since ui is strictly monotonically increasing in closed interval, Nash equilibrium point is an exact solution for game G = [Pi , ui ]. • Fairness: Define p′i as a point with high available power consumption comparing with the Nash equilibrium point p∗i . That is p′i = p∗i + 1p∗i . Then, fairness is achieved when n ui (p′ ) − ui (p∗ ) i
3.2. Non-cooperative power control game in video encoding system Using game theory, we can get the non-cooperative power control game G = [Pi , ui ], where i ∈ [1, N ] and N represents the max number of module in the game. In this game, N = 4. Thus we have the modules’ power allocation strategy set {Pi } and the payoff function ui for ith module. Each module selects a power level pi and pi ∈ Pi . Let the power vector p = (p1 , . . . , pN ) ∈ P denote the outcome of the game in terms of selected power levels of all modules, where the P is the set of all power vectors. The resulting utility level for ith module is ui (p). Use this notation p−i , alternative, is the combination of the power vectors except player i. This notation emphasizes that ith module controls only its own power pi . The strategy space of all the module excluding the ith module is denoted by p−i . Thus the utility ith module obtained by consumption pi can be re-expressed as ui (pi , p−i ). With the definition of best response correspondence in each composed module, the Nash equilibrium definition can be described as follows: only if pi satisfies ui (p−i ) for all i ∈ N MBs, the power vector p = (p1 , . . . , pN ) = p∗ is a Nash Equilibrium of the non-cooperative power control game G = [{Pi }, {ui }]. 3.3. Nash equilibrium in non-cooperative modules power control game
• Existence: For game G = [Pi , ui ], ui represents the utility function of each module. pi is the corresponding power consumption. As mentioned above, the process of utility function constructing implies two steps so as to keep monotonicity property, ① Ordering: Order the pi in each module, s.t. p1 < · · · < pi < · · · < pm . This step is the sufficient condition of keeping monotonicity property. ② Singularity point elimination: Definition: Given point (pj , uj ), when (cj+1 > cj ), j ∈ [1, m], there is (uj+1 < uj ). Then this point is a singularity point. Since the utility function is built from empirical statistic method, singularity points usually emerge. These unreasonable points are eliminated so as to keep the consistency of utility function. After these two steps, the utility function keeps monotonically increasing. Besides, each pi has both upper and low
i=1
ui (p∗i )
i
≥ 0.
(3)
Proof. Consider ui is monotonically increasing, from the condition p′i = p∗i + 1p∗i , there is ui (p′i ) − ui (p∗i ) ≥ 0.
(4)
Furthermore, since ui (p′i ) > 0, formula (3) always holds. Therefore, the allocation from Nash equilibrium solution is fairness. 3.4. Solution of power allocation among the modules The utility function ui for each module reflects its preference and influence to the whole module encoding unit. We use the bit rates, visual quality and power consumption as a measure of utility. The power allocation scheme satisfies the following marginal player principle: PSVE system allocates power levels so that the utility of the module is maximized. Efficient encoding reflected through maximum utility is then educed. From Section 3.1, we obtain the appropriate power management framework for blockbased video encoding system. Game theory model is established and characterized by a number of modules. Furthermore, the optimal power allocation scheme can be obtained by finding a solution to maximize the modules’ utility in non-cooperative power control game, which is discussed in Section 3.2. This solution proves the power chosen can be transferred to find the best response to the powers chosen by the other players. As mentioned above, we use curve fitting from RDC analysis to re-analysis the encoding process, consequently, guarantee the strategy space Pi of each module is a compact set with minimum and maximum power consumption constraints denoted by Pi ∈ [Pmin , Pmax ]. Then we can obtain the Nash Equilibrium point, which is proved in Section 3.3. Then for power-scalable control, the non-cooperative power control encoding game can be developed by solving the utility function in each power consumption level. The problem to obtain the optimal power allocation scheme in each power level, can be transferred into finding a solution to maximize the modules’ utility in a distributed fashion in non-cooperative power control game. It can be expressed as max ui (pi , p−i ), pi ∈Pi
i ∈ [1, N ].
(5)
Here, the best playoff result is in the best response from the other players. Thus, the optimized solution can be obtained by
W. Ji et al. / Future Generation Computer Systems (
)
–
7
using Cournot approach. As is known, Cournot solution [35] has been widely discussed and accepted in equilibrium solution. Hence we use its main feature to solve the power equilibrium in modulelevel power control. We get
∂ ui ∂ pi = 0 i = 1, N N N level(j) (ui ) s.t. pi ≤ Pprofile . max i =1
Here
level(j) Pprofile
(6)
i =1
represents the power consumption budget in level j.
3.5. Algorithm summary The proposed PSVE model is built on Game Theory architecture. It is embedded in block-based video encoder. PSVE system provides two controls: scaling the encoding states according to power profiles; optimize the power consumption and maximum encoding performance in each power profile. We summarize the architecture in the following steps,
Fig. 4. Power scalability and bit rate of three CIF sequences.
(1) Power aware will be performed prior to the power scheduling and management in video encoder. This step complies with the beginning of Section 2.1 and implements a mapping bridge between the power profile and the available resource of the level device. Pbudget (j) is the initial estimation of power budget.
level (2) Scaling the encoder into corresponding profile Pprofile (j), which is calculated with (1). (3) Build a static encoding state table to index the configuration and influence of major composed modules, through module behavior analysis. (4) The power allocation among encoding modules is established based on the derived game theoretical model. Encoding strategy is mapped from control information set to module actions. (5) The power allocation of every modules is computed based on the derived game theoretical model. pi is computed with (6). (6) Finding the configurations closed to the power pi in static encoding state table, the optimal encoding depends on the configuration with maximum utility. (7) Update the power consumptions θ (j − 1) in (1), go to step 2 to begin a new frame encoding.
4. Experiments and results In this section, we give numerical results for these analysis presented in the previous sections. To evaluate the performance of PSVE strategy, we implement PSVE model and proposed power control scheme in the AVS encoder. Similar performance is expected for the other coding systems and the conclusion from this platform is easy applied or transferred to the other video standards such as H.264, MPEG-4/2, etc. 4.1. Power-scalable performance In this subsection, we investigate the scalability of encoder in PSVE. PSVE provides an efficient scalable video encoding and is highly adaptable to the power consumption budget by combined adjusting working modes of main modules. The video sequences are foreman, news and mother–daughter. Specific encoding parameters are as follows: CIF resolution, GOP size 30, frame rate 30, IPBPB types and two reference frames. We use computation cost to map the budget, and test different budget cases such as 5%, 9%, 11%, . . . , 100% percentages of maximum performance computation cost. Fig. 4 shows the variable situation in bit rates
Fig. 5. Power scalability and PSNR of three CIF sequences.
when the encoder works under different power consumption budget. Fig. 5 shows the corresponding encoding quality measured in PSNR. Both of the two figures show the following facts: (1) PSVE can help video encoder to work under different scalable power consumption budget, range from 5% to 100%; (2) when the encoder work under resource constraint circumstance, PSVE can help the encoder have excellent encoding effect because the PSNR in Fig. 5 does not drop abruptly; (3) efficient available resource leads to better coding efficiency because bit rate in Fig. 4 becomes lower when the power consumption budget increases; (4) finally, PSVE helps video encoder have power saved function. 4.2. Power-scalable performance In this subsection, video encoder generates power-scalable video (Section 3) to meet different power budget. 4.2.1. Comparing with RPC We further compare the proposed game theoretical approach rapid power control algorithm (RPC). The RPC allocates the available resources to each module rapidly, and do fast decision among adjacent encoding states in each module. This means, all encoding modules share the resource fairly, and if the available energy resource declines, the coding modules lower the working state fairly so as to adapt limited resource. 4.2.2. Numerical results and discussion For the sake of objectivity, both PSVE based on game theoretical approach and RPC work under the same power consumption
8
W. Ji et al. / Future Generation Computer Systems (
Fig. 6. PSVE performance on PSNR of sequence foreman CIF.
)
–
Fig. 9. PSVE performance on bit rate of sequence news CIF.
Fig. 7. PSVE performance on bit rate of sequence foreman CIF. Fig. 10. PSVE performance on PSNR of sequence mother–daughter CIF.
Fig. 8. PSVE performance on PSNR of sequence news CIF. Fig. 11. PSVE performance on bit rate of sequence mother–daughter CIF.
budget so as to obtain fair evaluation. We give the results when the two encoders work in the situations of 0%–20%, 20%–40%, 40%–60%, 60%–80% and 80%–100% power budget. We test the performances of PSVE and PRC in these four power situations. The video sequences are also foreman, news and mother–daughter, and specific encoding parameters are the same with Section 4.1. Figs. 6, 8 and 10 show the encoding quality of three sequences, separately; while Figs. 7, 9 and 11 show the bit rate results. For the sake of
uniform comparison, the blue lines and points represent the results of PSVE, while the red lines and points represent the results of RPC. These six figures demonstrate that PSVE has better performances than RPC in terms of encoding quality and bit rate. We also see that using PSVE, the bit rate can be reduced to a great extent than using RPC. Therefore, PSVE can make the encoder work with lower computational consumption while have better encoding
W. Ji et al. / Future Generation Computer Systems (
effects, and the amount of saved computational cost is much better than RPC used situations. In these results, PSVE can also scale the consumption at different computational cost budget levels. Under different cost budget, PSVE with game theoretical approach helps the encoder to choose best parameter configurations and decide best modes in its composed modules so as to keep video encoder work under best states in variable resource circumstance. On the other hand, since video quality and low bit rate are both important considerations when measure the encoding states in PSVE, sometimes different situation level may have the same optimization decision results. This shows the scalable accuracy of PSVE still remains to improve. Generally, Figs. 6–11 show that: (1) The encoding effect from PSVE is better than that from RPC; (2) PSVE can make the encoder have power consumption scalable function, at the same time maintain best encoding effect in each scalable mode. 5. Conclusion This paper presents the PSVE strategy in power control video encoding systems. There are two major contributions in this work. First, video encoder can work under variable energy resource constraint marked with different power consumption budgets; Second, game theory is introduced to solve the tradeoff between encoding effect and power consumption, so as to obtain better performance in each power/energy levels. The experiment results are based on four composed modules game. In fact, PSVE will be fine scalable when introduce more composed modules. On the other hand, the degree of lower power consumption depends on not only PSVE but also the algorithm of each modules. But PSVE can help video encoder provide many low power working modes while keep better performance, whatever the algorithm of modules will be designed. Thus, the proposed scheme provides an efficient architecture in power control video encoding system design. In future work, we will extend the PSVE model to joint source and channel encoding systems in wireless communication.
Acknowledgment This work was supported by the Natural Science Foundation of China (No. 61001194).
References [1] J. Ostermann, J. Bormans, P. List, D. Marpe, M. Narroschke, F. Pereira, T. Stockhammer, T. Wedi, Video coding with H.264-AVC_tools, performance, and complexity, IEEE Circuits and Systems Magazine 14 (1) (2004) 7–28. First Quarter. [2] Thomas Wiegand, Gary J. Sullivan, et al., Overview of the H.264/AVC video coding standard, IEEE Transactions on Circuits and Systems for Video Technology 13 (7) (2003) 560–576. [3] Sergio Saponara, Kristof Denolf, Gauthier Lafruit, Carolina Blanch, Jan Bormans, Performance and complexity co-evaluation of the advanced video coding standard for cost-effective multimedia communications, EURASIP Journal on Applied Signal Processing, 2004 (2), pp. 220–235. [4] Chung-Jr Lian, Shao-Yi Chien, Chia-Ping Lin, Po-Chin Tseng, Liang-Gee Chen, Power-aware multimedia: concepts and design perspectives, IEEE Circuits and Systems Magazine 7 (2) (2007) 26–34. [5] Zhihai He, Yongfang Liang, Lulin Chen, Ishfaq Ahmad, Dapeng Wu, Powerrate-distortion analysis for wireless video communication under energy constraints, IEEE Transactions on Circuits and Systems for Video Technology (2004). [6] S. Saponara, L. Fanucci, Data-adaptive motion estimation algorithm and VLSI architecture design for low-power video systems, IEE Proceedings Computers & Digital Techniques 151 (1) (2004).
)
–
9
[7] Seiji Mochizuki, Tetsuya Shibayama, Masaru Hase, Fumitaka Izuhara, Kazushi Akie, Masaki Nobori, Ren Imaoka, Hiroshi Ueda, Kazuyuki Ishikawa, Hiromi Watanabe, A low power and high picture quality H.264-MPEG-4 video codec IP for HD mobile applications, IEEE Asian Solid-State Circuits Conference (2007). [8] R.S.V. Prasad, Ramkishor Korada, Efficient implementation of MPEG-4 video encoder on RISC core, IEEE Transactions on Consumer Electronics 49 (1) (2003). [9] Satoshi Kumaki, et al. A 99-mm2 0.7-W single-chip MPEG-2 422P@ML video, audio, and system encoder with a 64-Mb embedded DRAM for portable 422P@HL encoder system, IEEE 2001 Custom Integrated Circuits Conference, 2001. [10] Graeme Stewart, David Renshaw, Martyn Riley, A novel motion estimation power reduction technique, International Conference on Field Programmable Logic and Applications (2007) 546–549. [11] Masayuki Miyama, et al., A sub-mW MPEG-4 motion estimation processor core for mobile video application, IEEE Journal of Solid-State Circuits 39 (9) (2004). [12] Tung-Chien Chen, Yu-Han Chen, Sung-Fang Tsai, Shao-Yi Chien, Liang-Gee Chen, Fast algorithm and architecture design of low-power integer motion estimation for h.264-avc, IEEE Transactions on Circuits and Systems for Video Technology 17 (5) (2007). [13] N.J. August, Dong Sam Ha, Low power design of DCT and IDCT for low bit rate video codecs, IEEE Transactions on Multimedia 6 (3) (2004) 414–422. [14] Mustafa Parlak, Ilker Hamzaoglu, Low power H.264 deblocking filter hardware implementations, IEEE Transactions on Consumer Electronics 54 (2) (2008). [15] Yih Han Tan, Wei Siong Lee, et al., Complexity Scalable H.264/AVC encoding, IEEE Transactions on Circuits and Systems for Video Technology 20 (9) (2010). [16] Deepak S. Turaga, Mihaela van der Schaar, Beatrice Pesquet-Popescu, Complexity scalable motion compensated wavelet video encoding, IEEE Transactions on Circuits and Systems for Video Technology 15 (8) (2005) 982–993. [17] Ji Wen, Min Chen, Ge Xiaohu, Li Peng, Yiqiang. Chen, A perceptual macroblock layer power control for energy scalable video encoder based on just noticeable distortion principle, Journal of Network and Computer Applications, doi:10.1016/j.jnca.2010.06.011, available online at 2010. [18] Yih Han Tan, Wei Siong Lee, Jo Yew Tham, S. Rahardja, Kin Mun Lye, Complexity scalable H.264/AVC encoding, IEEE Transactions on Circuits and Systems for Video Technology 20 (9) (2010) 1271–1275. [19] Tung-Chien Chen, Yu-Han Chen, Sung-Fang Tsai, Shao-Yi Chien, Liang-Gee Chen, Fast algorithm and architecture design of low-power integer motion estimation for H.264/AVC, IEEE Transactions on Circuits and Systems for Video Technology 17 (5) (2007) 568–577. [20] Chenglin Li, Junni Zou, Hongkai Xiong, Chang Wen Chen, Joint coding/routing optimization for distributed video sources in wireless visual sensor networks, IEEE Transactions on Circuits and Systems for Video Technology 21 (2) (2011) 141–155. [21] I. Ahmad, Jiancong Luo, On using game theory to optimize the rate control in video coding, IEEE Transactions on Circuits and Systems for Video Technology 16 (2) (2006) 209–219. [22] http://www.statslab.cam.ac.uk/~rrw1/mor/. [23] Eric Rasmusen, Games and Information: An Introduction to Game Theory, 3rd ed., Blackwell Publishers, 2001. [24] Dellev Marpe, Thomas Wiegand, The H.264/MPEG4 advanced video coding standard and its applications, IEEE Communications Magazine (2006). [25] Iain E.G. Richardson, H.264 and MPEG-4 Video Compression, Wiley, 2003. [26] Information technology—advanced coding of audio and video—part 2:video, audio video coding standard workgroup of China (AVS), GB/T 200090.2-2006. [27] Liang Fan, Siwei Ma, Feng Wu, Overview of AVS video standard, IEEE Conf. Multimedia and Expo, vol. 1, 27–30 June 2004. [28] Tiejun Huang, The comparison between AVS and MPEG. http://www.avs.org. cn/reference.asp. [29] Li Su, Yan Lu, Feng Wu, et al., Complexity-constrained H.264 video encoding, IEEE Transactions on Circuits and Systems for Video Technology 19 (4) (2009) 477–490. [30] Hanli Wang, Sam Kwong, Chi-Wah Kok, An efficient mode decision algorithm for H.264-AVC encoding optimization, IEEE Transactions on Multimedia 9 (4) (2007) 882–887. [31] F. Gerardo, K. Hari, et al., A fast MB mode decision algorithm for MPEG-2 to H.264 P-frame transcoding, IEEE Transactions on Circuits and Systems for Video Technology 18 (2) (2008) 172–185. [32] Feng Pan, Xiao Lin, S. Rahardja, K.P. Lim, Z.G. Li, Dajun Wu, Si Wu, Fast mode decision algorithm for intraprediction in H.264/AVC video coding, IEEE Transactions on Circuits and Systems for Video Technology 15 (7) (2005) 813–822. [33] An-Chao Tsai, Jhing-Fa Wang, Jar-Ferr Yang, Wei-Guang Lin, Effective subblock-based and pixel-based fast direction detections for H.264 intra prediction, IEEE Transactions on Circuits and Systems for Video Technology 15 (7) (2008) 975–982. [34] Thomas S. Ferguson, Game theory, Class Text for University of California at Los Angeles, 2000. [35] J.P. Mayberry, J.F. Nash, M. Shubik, A comparison of treatments of a duopoly situation, Econometrica (1953) 141–154.
10
W. Ji et al. / Future Generation Computer Systems ( Wen Ji received the Ph.D. degree from the Department of Communication and Information Systems from Northwestern Polytechnical University, China, in 2006. She is now an Assistant Professor in the Institute of Computing Technology, Chinese Academy of Sciences, Bei Jing, China. Her research interests include video coding and communication, channel coding, and video networking.
Jiangchuan Liu received the BEng degree (cum laude) from Tsinghua University, Beijing, China, in 1999, and the Ph.D. degree from The Hong Kong University of Science and Technology in 2003, both in computer science. He is a recipient of Microsoft Research Fellowship (2000), Hong Kong Young Scientist Award (2003), and Canada NSERC DAS Award (2009). He is a co-recipient of the Best Student Paper Award of IWQoS’2008, the Best Paper Award (2009) of IEEE ComSoc Multimedia Communications Technical Committee, and Canada BCNet Broadband Challenge Winner Award 2009. He is currently an Associate Professor in the School of Computing Science, Simon Fraser University, British Columbia, Canada, and was an Assistant Professor in the Department of Computer Science and Engineering at The Chinese University of Hong Kong from 2003 to 2004. His research interests include multimedia systems and networks, wireless ad hoc and sensor networks, and peer-to-peer and overlay networks. He is a Senior Member of IEEE and a member of Sigma Xi. He is an Associate Editor of IEEE Transactions on Multimedia, an Editor of IEEE Communications Surveys and Tutorials, and an Area
)
–
Editor of Computer Communications. He is TPC Vice Chair for Information Systems of IEEE INFOCOM’2011. Min Chen was an assistant professor in School of Computer Science and Engineering at Seoul National University (SNU). He was a research associate in the Department of Computer Science at University of British Columbia (UBC) for half year. He has worked as a PostDoctoral Fellow in the Department of Electrical and Computer Engineering at UBC for three years since March 2009. Before joining UBC, he was a Post-Doctoral Fellow at SNU for one and half years. He has published more than 120 technical papers. He is the sole author of a textbook OPNET Network Simulation (Tsinghua Univ. Press, 2004). Dr. Chen received the Best Paper Runner-up Award from The Fifth International Conference on Heterogeneous Networking for Quality, Reliability, Security and Robustness (QShine) 2008. He serves as editor or associate editor for Wiley I. J. of Wireless Communication and Mobile Computing, Wiley I. J. of Security and Communication Networks, Journal of Internet Technology, KSII Transactions on Internet and Information Systems, International Journal of Sensor Networks (IJSNet). He is a managing editor for International Journal of Autonomous and Adaptive Communications Systems. He has worked as session chairs in several conferences, such as VTC’08, QShine’08, ICACT’09, Trientcom’09, and ICC’09. He is a TPC co-chair of BodyNets 2010. He is a symposia co-chair and workshop chair of CHINACOM 2010. He is the co-chair of MMASN-09 and UBSN-10. He was the TPC chair of ASIT-09, ASIT-10, TPC co-chair of PCSI-09 and PCSI-10, publicity co-chair of PICom-09. He serves as the corresponding guest editors for several international journals, such as ACM/Springer Mobile Networks and Applications (MONET), International Journal of Communications System (IJCS). He is an IEEE senior member.
