772
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 4, APRIL 2011
Minimum Data Rate for Mean Square Stabilizability of Linear Systems With Markovian Packet Losses Keyou You, Student Member, IEEE, and Lihua Xie, Fellow, IEEE
Abstract—This paper investigates the minimum data rate for mean square stabilizability of linear systems over a lossy digital channel. The packet dropout process of the channel is modeled as a time-homogeneous Markov process. To overcome the difficulties induced by the temporal correlations of the packet dropout process and stochastically time-varying data rate due to packet dropouts, a randomly sampled system approach is developed to study the minimum data rate for mean square stabilizability. It is shown that the minimum data rate for scalar systems can be explicitly given in terms of the magnitude of the unstable mode and the transition probabilities of the Markov chain. The number of additional bits required to counter the effect of Markovian packet losses on stabilizability is exactly quantified. Our result contains existing results on data rate and packet dropout rate for stabilizability of linear scalar systems as special cases and provides a means for better bandwidth utilization by jointly considering bits per sample and an effective sampling. Necessary and sufficient conditions on the minimum data rate problem for mean square stabilizability of vector systems are provided respectively and shown to be optimal under some special cases. Index Terms—Data rate, Markovian packet losses, networked system, randomly sampled system, stabilizability.
I. INTRODUCTION ETWORKED control systems (NCSs) with their feedback loop closed via a limited bandwidth communication network have attracted increasing attention in the past decade. The advantages of NCSs over conventional control systems include low cost of installation, flexibility in system implementation, and the ease of maintenance. Due to limited network resources and uncertainties such as packet losses, limited transmission data rate, transmission errors and latency, the incorporation of communication networks in the feedback loop also induces new challenges in control system analysis and design. This has motivated the research on networked control to better understand the interplay between control and communication and develop new tools for analysis and synthesis of networked control systems; see the special issue [1] and the references therein. One of the main challenges in NCSs is the analysis and synthesis of control over feedback channels with limited commu-
N
Manuscript received November 26, 2009; revised April 26, 2010; accepted August 09, 2010. Date of publication August 23, 2010; date of current version April 06, 2011. This work was supported by the National Natural Science Foundation of China under Grant NSFC 60828006. Recommended by Associate Editor Y. Hong. The authors are with the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798 (e-mails:
[email protected];
[email protected]). Digital Object Identifier 10.1109/TAC.2010.2068590
nication data rate. There is a vast amount of research on the stabilizability of linear systems under limited communication data rate feedback [4]. The fundamental problem of finding the minimum data rate for stabilization was solved in [3], [15], [16], [21], [22] under various settings, leading to the celebrated data rate theorem. The result reveals a striking relationship between the minimum data rate for stabilizability of a linear system and the unstable eigenvalues of the open loop system. More precisely, the minimum data rate is completely determined by the so-called intrinsic entropy rate of the system which poses a fundamental constraint on data rate to guarantee the existence of a quantization and control scheme for stabilizing an unstable linear system. However, all of the aforementioned results were established under the crucial assumption that the limited communication data rate feedback channel supports a constant data rate. This may easily be violated in a wireless channel where the quality of the communication link may stochastically vary over time due to random fading. On the other hand, because of congestion and fading, communication channels may experience packet losses during data transmissions as well. Stabilizability of a system over a communication network of infinite bandwidth subject to packet losses has been extensively studied in the literature, see [7], [18] and the references therein. The packet dropout process is commonly modeled as an independent and identically distributed (i.i.d.) process [19] or a Markov chain [9]. With such a characterization of the packet losses process, the stability of the Kalman filter is analyzed in [9], [19], [23], [24]. Sinopoli et al. [19] investigated the effect of packet dropout rate on the stability of the state estimation error dynamics and proved that there exists a critical packet dropout rate above which the mean state estimation error covariance will diverge. As for Markovian packet losses, Huang and Dey [9] introduced the notion of peak covariance stability and provided sufficient conditions on the uniformly bounded peak covariance in the mean sense for general vector systems which are also necessary for the scalar case. An alternative sufficient condition for the peak mean covariance stability is given in [23] which is less conservative than that of [9] in some cases. Observe that the above works studied the stability of the Kalman filter with packet losses under a single sensor framework, the case of multiple sensors was investigated in [6] for both the i.i.d. and Markovian packet losses. In a practical networked system, especially in a resource limited wireless sensor network, the issues of packet losses and limited bandwidth generally co-exist. Therefore, it is of theoretical and practical significance to study the problem of minimum data rate for stabilization over lossy networks. Recently, much effort has been devoted to examining how the limited data rate of the
0018-9286/$26.00 © 2010 IEEE
YOU AND XIE: MINIMUM DATA RATE FOR MEAN SQUARE STABILIZABILITY OF LINEAR SYSTEMS
communication channel and the randomness of channel variation affect the stabilizability of unstable linear systems. Intuitively, due to possible packet losses, additional bits are required to stabilize the system. Thus, one of the fundamental issues is to quantify these additional bits required for the stabilization of the system. The problem is further complicated by the fact that different data rate may be required under different notions of stabilizability. For instance, the necessary and sufficient condition on the almost sure stabilizability over an erasure channel for a certain class of linear systems turns out to be that the Shannon capacity of the channel should be strictly greater than the intrinsic entropy rate of the system [11], [12], [20], which unfortunately fails for the moment stabilizability [17]. The focus of this paper is to explicitly quantify the minimum date rate required for the mean square stabilizability of a linear system with Markovian packet losses. We consider a feedback configuration where the sensor and controller are linked by a lossy network over which the transmitted packet carrying information of a finite number of bits may be stochastically dropped/ received. The packet dropout process of the channel is modeled as a time-homogeneous Markov process. To our knowledge, existing works on minimum data rate for mean square stabilizability of unstable systems over lossy digital channels mostly look at i.i.d. packet losses, see, e.g., [10], [13], [26]. It is known that the Markovian packet loss model exploits the temporal correlations of channel conditions and contains the i.i.d. packet loss model as a special case. However, due to the temporal correlations of random packet dropouts, the minimum data rate problem becomes much more challenging than the case of i.i.d. packet losses and the techniques employed in the aforementioned works cannot be directly applied. In particular, under the i.i.d. packet losses and scalar systems, the rate of reducing the mean square estimation error at the decoder for each transmission can be explicitly obtained, which is the key to establishing their results. Obviously, this can no longer hold for the case of Markovian packet losses. To remove this obstacle, we develop a new framework by adopting the notion of sojourn time which represents the stochastic time duration between two successive successful packet deliveries. It is interesting that the introduced sojourn times happen to form an i.i.d. process [24]. Associated with a realization of sojourn times, down sampling is carried out over the original discrete time system. The fact that future realizations of the dropout process are unknown in advance is not a problem since the system can be successively down sampled by adapting to the packet dropout process. Thus, the original problem is partially converted to the stabilizability of a randomly down sampled system. Because of the unboundedness and randomness of the sojourn time, new tools are developed to fill up the gap between the stabilizability of the randomly sampled system and that of the original discrete time system. Note that a random sampling approach has been used in [27], [28] to examine the stochastic stability of adaptive quantizers for Markov sources where sampling is taken with respect to a zoom in/out process of the quantizer. However, they do not consider the Markovian packet losses. We emphasize that our approach in this paper is not limited to the Markovian packet loss model and can be directly applied to networked control systems with transmission times driven by
773
an i.i.d. stochastic process. This remarkable feature should not be underestimated. Indeed, the limited bandwidth of the communication network in a networked control system is a major concern. As pointed out in [14], the bit rate is determined by two factors: the transmission times or how often a measured signal is sampled and transmitted through the network; and the number of bits per measurement or how accurately the transmitted value is represented in digital form. Our approach can be utilized to address the bit rate issue by jointly considering bits per sample and effective sampling, leading to better bandwidth utilization. Note that a method of minimizing the number of data transmissions between the sensor and the controller/actuator without the consideration of reducing the size of packet at each transmission is studied in [14]. The contribution of this paper shows that, for unstable scalar systems, if the probability of the channel recovering from dropping packet is larger than the threshold derived in [9], the minimum data rate can be explicitly given by the intrinsic entropy rate of the system, which is the exact minimum data rate without packet losses, plus a nonnegative item expressed in terms of the magnitude of the unstable mode and the transition probabilities of the Markov chain. Hence, the additional number of bits induced by Markovian packet losses for mean square stabilizability is exactly quantified. The result can recover existing results on data rate and packet dropout rate for stabilization of a linear scalar system. Necessary and sufficient conditions are respectively provided for vector systems, which are optimal for some special cases. The implication and relationships with the existing results are discussed as well. The rest of the paper is organized as follows. The problem formulation is described in Section II. Some important preliminary results are provided in Section III. The study of the minimum data rate for scalar systems is carried out in Section IV, where we start from an unstable noise free system with bounded initial state and then proceed to the more general case with unbounded initial state and unbounded process noise. Necessary and sufficient conditions for vector systems are respectively studied in Section V. Conclusion remarks are drawn in Section VI. Notation: represents the probability density function of a continuous random variable . Let be the Euler’s constant number. The differential entropy [5] of a continuous random is written as , where variable is the logarithm function with the base . Mathematical expectation taken with respect to (w.r.t.) the random variable is . denotes the norm for vectors or denoted by represents the the induced matrix norm for matrices while norm for vectors. , and respectively denote the set of integers, nonnegative integers and complex numbers. II. PROBLEM FORMULATION Consider the following stochastic linear time-invariant system: (1) where is the measurable state, is the input, is the stochastic disturbance of zero mean. The and and have uniformly bounded random variables
774
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 4, APRIL 2011
Fig. 1. Network configuration.
th absolute moment, i.e., and , such that and . Also, and are mutually independent and have probaexists and such that bility densities. Moreover, . It is worth mentioning that the above conditions have been considered in [16] and contain the general additive white Gaussian noise assumption as a special case. To make the problem interesting, we focus on unstable systems is a stabilizable pair. and that Suppose that the state measurement and the controller are connected by a lossy forward digital channel. See Fig. 1 for the networked control configuration. At each time slot, the encoder measures the state, quantizes it with a packet size of bits and transmits the quantizer output to the decoder via the forward channel. Due to random fading of the channel, the packet may be lost while in transit through the network. It is assumed that there is an additional perfect (without packet losses and transmission errors) feedback channel to send a reception/dropout acknowledgement to the encoder. Neglecting transmission errors for both forward and feedback channels, we further assume that the transmission of the quantized symbol and acknowledgement can be completed within one sampling interval. As in [9] and [24], the packet dropout process in the forward channel is mod, which is eled as a time-homogenous Markov process more general and realistic than the i.i.d. case studied in [19] due to the existence of temporal correlations of channel conditions. does not contain any information of the Furthermore, system state, suggesting that it is independent of the channel indicate that the packet has been successinput. Let corresponds to the fully delivered to the decoder while dropout of the packet. Moreover, the Markov chain has a transition probability matrix defined by
The objective is to find necessary and sufficient conditions in relation to the failure rate on the minimum data rate and recovery rate such that there exists a control strategy and a coding-decoding scheme to achieve the MS-Stabilization of the system (1). It should be noted that the goal is to address the issue of limited communication capacity rather than that of limited computation and storage. The present quantized signal at time is generated by allowing the encoder to access , the past quantized all the past and present states and packet reception status . symbols and similarly and That is, by defining , then , where is the encoder mapping at time . Likewise, at time , the decoder generates by , where denotes the the control input collection of all received quantized data up to time , e.g., and is the decoder mapping at time . Actually, the encoder and decoder to be designed later require only a finite memory. Before closing this section, it is worth pointing out that there is no loss of generality to focus on the quantized state feedback case. If the encoder can only access an output measurement rather than the state, under the mild condition of detectability, a linear minimum variance estimator can be constructed to estimate the state with a bounded mean square estimation error [20], [25]. III. PRELIMINARIES Denote the common probability space for all random variables in the paper and let be an increasing sequence of -fields (filtration) generated by . In the sequel, the terminology random variables of almost everywhere (abbreviated as a.e.) is always with respect to the probability measure . Associated with the , the stochastic time sequence Markov chain is introduced to denote the time at which the encoder receives a packet reception acknowledgement from the decoder. Without 2 and .1 Then, , is loss of generality, let precisely defined by
.. .
(2)
.. . (3)
where is the state space of the Markov chain. To avoid any trivial case, the failure rate and recovery rate of the channel are assumed to be strictly positive and less than 1, , , so that the Markov chain is ergodic. i.e., Obviously, a smaller value of and a larger value of indicate a more reliable channel. Definition 1: The system (1) with network configuration of Fig. 1 is said to be mean square stabilizable (MS-Stabilizable) and , there is a control policy reif for any initial state lying on the quantized data such that the state of the closed-loop system is uniformly bounded in the mean square sense, i.e., , where the mathematical expectation is taken w.r.t. the packet dropout process , operator and the initial state . the noise sequence
By the ergodic property of the Markov chain , is finite a.e. [9]. Thus, the integer valued sojourn to denote the time duration between two suctimes cessive packet received times are well-defined a.e., where (4)
=0
=1
1If , there exists a finite k 2 a.e. such that due to the ergodic property of the Markov chain. Then, we can set time k as our initial time. 2The encoder can only know whether the current packet is dropped or received at the next sampling time. For example, if the packet generated at time k is successfully delivered, the decoder will receive the packet and send a reception acknowledge to the encoder. Since this process is assumed to be completed within one sampling interval, the encoder will know the packet . reception status at time k
( = 1) +1
YOU AND XIE: MINIMUM DATA RATE FOR MEAN SQUARE STABILIZABILITY OF LINEAR SYSTEMS
With regard to the probability distribution of sojourn times , we recall the following interesting result. are independent Lemma 1: [24] The sojourn times and identically distributed. Furthermore, the distribution of is explicitly expressed as (5) We shall exploit this fact to establish our results. Unlike the technique employed in [10], [13], [26], we develop a new framework by down sampling the system (1) with the . Here can be interpreted sampling interval equal to as a “communication logic” to trigger the transmission of the packet. IV. SCALAR SYSTEMS To better convey our idea, we first focus on a scalar system of the form (6) where
and
.
A. Noise Free Systems With Bounded Initial Support Consider a noise free system as follows: (7) is a random variable with a known where the initial state such that , bounded support, i.e., there exists an . and a probability density Definition 2: The system in (7) is said to be asymptotically MS-Stabilizable via quantized feedback if for any initial state and , there is a control policy relying on the quantized data such that the state of the closed-loop system is asymptotically driven to zero in the mean square sense, namely, , where the mathematical expectation is taken w.r.t. the packet dropout process operator and the initial random variable . We are now in the position to present the first main result. Theorem 1: Consider the system (7) and the network configuration in Fig. 1 where the packet dropout process of the forward channel is a time-homogeneous Markov process with the transition probability matrix (2). The networked system is asymptotically MS-Stabilizable if and only if the following conditions hold: a) The probability of the channel recovering from packet dropping is large enough, namely (8) b) The data rate
satisfies the following strict inequality:
775
Remark1: 1) The data rate condition (9) has an intuitive interpretation. , the square of state estimaAt the time interval tion error at the decoder grows by . By the definition of , only one packet is successfully sent to the decoder during the cycle, which can reduce the square of . Thus, if the growth of the estimation error at most by the mean square estimation error equals or exceeds , i.e., , it is impossible to asymptotically stabilize the system in the mean square sense. , the in2) Neglecting quantization effects, i.e., equality (10) is automatically satisfied. It is interesting to note that our condition recovers the result in [6], [9]. 3) Due to stochastic packet dropouts, additional bits are required to asymptotically stabilize the system in (7). When the packet dropout process is specialized to an i.i.d. in the transition process, corresponding to probability matrix, the necessary and sufficient condition reduces to that of [10], [13], [26]. 4) In light of (10), the larger the magnitude of the unstable mode, the more bits are needed to compensate the effect of packet dropouts. As mentioned in Section II, a smaller value of and a larger value of correspond to a more reliable channel. Thus, fewer bits are required to counter the dropout effect on the MS-Stabilizability, which is confirmed in (10). For the special case that there is no packet and dropout, corresponding to the limiting case , the minimum data rate in (7) converges to the well-known data rate theorem for stabilization of a linear system [15], [21], [25]. The following technical lemma is used to establish the result of Theorem 1. Lemma 2: [15] Let the distribution of a real valued random be absolutely continuous w.r.t. Lebesgue meavariable sure with density and has the second absolute moment . Denote the Borel measurable quantizer with the number of quantization levels not greater than . Then, (11) where
is a parameter determined only by and is the Rényi differential entropy power of order [5], i.e., . Proof of Theorem 1: Necessity: Since an acknowledgement from the decoder to indicate the packet reception/dropout status will be sent to the encoder through a perfect feedback channel, the encoder can access the full knowledge of the decoder and recover the control produced by the decoder. Thus, in view of [15], the asymptotic MS-Stabilization of the system (7) is equivalent to to finding a sequence of admissible quantizers satisfy that (12)
(9) (10)
Here the admissible quantizer means that the number of quantization level of the quantizer is adapted to the information available at time for the decoder.
776
Given any data rate
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 4, APRIL 2011
bits/transmission for the forward
as channel, denote the random variable the accumulative number of quantization levels that has been , received by the decoder at time . Then, for any it follows that:
Sufficiency: In control strategies to be developed in this subsection, a uniform quantizer will be utilized. Precisely, if is ,a a real-valued number between 1 and 1, i.e., mid-rise uniform quantizer that uses bits of precision to represent each quantizer output is expressed as if if
(16)
is the round floor function given by . If for some , the quantization error induced by the above -bit uniform quantizer is computed as . Given any data rate satisfying (10), a sequence of -bit uniform quantizers to recursively acquire initial state informa, the encoder and decoder tion are to be designed. At time share a state estimator based on the symbols sent via the forward channel and packet acknowledgement and update the estimator as follows: where
(17)
(13)
(18)
where the inequality is due to Lemma 2 and the change of inis performed in the second last tegration variable equality. Let , it leads to that
(19)
(14) To achieve (12), (13) implies that since . Consider the randomly samit holds that and the definition pled system of (14) at stopping times in (4), we obtain that of
is chosen to satisfy that where the stabilizing control gain . The input signal is produced by . The scaling factor is simultaneously updated on both sides of the channel via
(20) (15) By Theorem 4 of [24], the asymptotic MS-Stability of the system (14) in discrete time is equivalent to the asymptotic MS-Stability of the system (15) in the stop, i.e., ping times . Thus, it is sufficient to focus on the randomly sampled system (15). Since in view of are i.i.d., it is easy to Lemma 1, the sojourn times derive that . Consequently, a necessary condiis that tion to make by the equivalence property. Applying the distribution informain Lemma 1, the proof of the necessity is completed tion of by the following arguments:
if if
(21) Before proceeding further, the implementation of the above algorithm in the encoder and decoder is described. At the random time , both the encoder and decoder have a state and the corresponding scaling . The enestimator coder quantizes the “normalized innovation”, denoted as , by a -bit uniform quantizer. The quantizer output is transmitted to the is dropped at time ’s decoder via the forward channel. If transmission, the decoder sends a packet dropout acknowledgeto the encoder and updates its estimator and ment scaling respectively according to (18) and (20) in the next time . Since we assume there is a perfect feedback instant channel to send the packet acknowledgement, the encoder can update its estimator and scaling in the same manner as the at time decoder. The encoder retransmits the same signal until it receives the packet reception acknowledgement at time . See Fig. 2 from the decoder , the for illustrations. By the definition of random time packet will be successfully delivered at time ’s
YOU AND XIE: MINIMUM DATA RATE FOR MEAN SQUARE STABILIZABILITY OF LINEAR SYSTEMS
777
in (8), it is easy to see that . Here we have utilized the fact that the is independent of identically distributed random variable and is adapted to in (23). , it immediately Together with the fact that . follows from (24) that By iteration of (7) and substituting the control signal into (7), it yields that . Jointly with the inequality , we finally obtain that by the Toeplitz lemma [2, Sec 6.1.2] and the preceding result that .
since for
Fig. 2. Communication protocol.
transmission. Then, both the encoder and decoder update their . estimator and scaling according to (19) and (21) at time Thus, the synchronization between the encoder and decoder is guaranteed and this process can be continued. Now, we shall prove that the estimation error asymptotically converges to zero in the mean square sense. Denote the estima, the dynamical equation tion error by governing the error evolution is given by
B. General Stochastic Scalar Systems Theorem 2: Consider the system (6) and the network configuration in Fig. 1 where the packet dropout process of the forward channel is a time-homogeneous Markov process with the transition probability matrix (2). The networked system is MS-Stabilizable if and only if the following conditions hold: a) The probability of the channel recovering from dropping packet is large enough, namely
(22)
(25) b) The data rate satisfies the following strict inequality:
It can be shown that the quantizer does not overflow at all times, i.e., . In fact, it obviously holds for , 1 since and . , , then Assume that . Further, for any
(26)
(23)
Remark 2: Although the above necessary and sufficient condition remains the same as that for the case of noise free systems with a finite initial state support, the proof is much more challenging. Due to the unbounded noise, uncertainties about the system state at the decoder arise from both the initial state and the noise, making the above asymptotic quantization approach inapplicable. Thus, a completely different method is developed to establish the result. As in [13], we will find a lower bound for the second moment of the state to establish the necessity. To this end, let be the conditional entropy power of conditioned on the event , . Here is a particular reaveraged over all possible . It is clear [5] that alization of the random vector is a lower bound of the second moment of the state , i.e., . The following lemma is needed to prove the necessity. Lemma 3: Given any data rate bits/transmission, the following inequality holds:
(24)
Proof: In view of Lemma 4.2 in [13], the assertion is straightforward.
, . Thus, by induction,
. we have that On the other hand, the mean square of the scaling at will exponentially converge to zero. Let random times by (10), we have , are a where the third equality is due to that sojourn times sequence of i.i.d random variables by Lemma 1. Next, one can further derive that
778
Proof of Theorem 2: Necessity: Let the data rate be transmission, the following result can be established:
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 4, APRIL 2011
bits/
(30) where (29) is inductively derived as in (28) and (30) is due to that the initial state is independent of the packet dropout process . To study the lower bound (30), let and consider an auxiliary system as follows: (31) By
(27)
the
iteration
of
(31),
is
written
as
. Associated with the system (31), we introduce the following notation: (32)
In the above, the first equality is due to that the control input is measurable w.r.t. and the translation invariance property of entropy [5]. The first inequality follows from the enis independent tropy power inequality [5] and the fact that of . The second inequality is due to the assumption that , whereas the last inequality is the result of forms a Markov Lemma 3. Since chain, we can similarly derive that
where the indicator function zero. Thus, the recursive equation for
if , otherwise is written as follows:
(33)
where the distribution of . Let
is defined by with
and
(28) Inserting (28) into (27) leads to that
Rewriting (33) in a compact form leads to the following recursion: (34) Note that , where is the standard norm in . Assume the networked system in (6) is MS-Stabilizable, it follows that:
(29)
which, together with (30), implies that . is ergodic, Moreover, due to that Markov process will converge to a unique stationary distribution, that is, as . Then, ,
YOU AND XIE: MINIMUM DATA RATE FOR MEAN SQUARE STABILIZABILITY OF LINEAR SYSTEMS
as follows that the spectral radius of since otherwise, (34), we obtain that
. In view of (34), it is strictly less than one . Thus, letting in . Again by (30), it yields
(35) As in the proof of the necessity of Theorem 1, a necessary con. The rest of dition for (35) to hold is that proof directly follows from that of Theorem 1. Sufficiency: We adopt the adaptive quantizer developed in [16] to capture the unbounded noise so that the upper bound of the second moment of the -bit quantization error decays if the random quantizer input variable has a at a rate of bounded th moment for some . In particular, given a , the -bit quantizer generates quantization parameter . intervals labeled from left to right by Let , and . , the quantization intervals are generated by If intervals of equal • partitioning the set [ 1, 1] into length; and re• partitioning the sets , intervals of spectively into equal length. and The two infinite length intervals are respectively the leftmost and rightmost intervals of the quantizer. Let be the half-length of interval for • , be equal to if and if . be the midpoint of interval if , • , be equal to if and equal to if . to The above quantizer allows the quantization intervals . For example, be generated recursively by starting from , are produced by partiquantizer intervals for tioning each bounded interval of into two uniform subintervals and the unbounded interval into to , . A similar partition is applied to the other infinite subinterval . More details can be found in [16]. and constant real number , For any random variables define the functional
It can be verified that [16]. A fundamental property of the above quantizer is given below. be random Lemma 4: [16, Lemma 5.2] Let and for some . Given a bit variables with adaptive quantizer as above and , then the quantization satisfies error
779
where is the index of the levels of the and is a constant greater than 2 determined quantizer only by and . , , then Lemma 5: ( -inequality) Given if if Proof: It is a standard result in elementary inequality theory and can be easily verified. such that . Let By (25), there exists a and satisfy the adaptive quantizer parameters be (26). Now, we use the above quantizer to approach the lower into cycles bound of (26). To this aim, divide the integers with length , which is determined by the available data rate and is to be specified later. The encoder and decoder simultaneously construct an estimator of the state based on the quantized symbol and packet acknowledgement as follows:
(36)
(37) where the stabilizing control gain is chosen to make and the input is formed by . The quantizer works as follows. At random times , the encoder quantizes the normalized innova, by the above described tion, denoted as -bit adaptive quantizer and sends the quantizer output to the decoder via the forward channel. Based on the definition of , the decoder will receive the packet during random times and send a packet recepthe time interval tion acknowledgement to the encoder. By the assumption that the transmission of the packet and acknowledgement can be finished within one sampling interval, the encoder and decoder at time . Then, agree on that into subinterthe encoder and decoder further divide vals in the manner described above. The quantizer output related to the subinterval will be sent to the decoder to further reduce the uncertainty of the normalized innovation for the decoder. By receiving the second packet in the time interval , the encoder and decoder agree on that . Continuing the same steps and after receiving the th packet reception acknowledgement in the time interval , the encoder and debelongs to one coder agree on the fact that of the subintervals . Noting that the Markovian packet is ergodic, the above procedure can be realloss process ized with probability one.
780
Since
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 4, APRIL 2011
, select an such that . Then, it follows from Lemma 1 that . Using -inequality in Lemma 5, we
estimation error by by
, the error dynamics is governed
obtain that
(44) and let . In Define view of the error dynamics in (44), one can easily derive the following result: (38) Now, define make that:
(45)
and choose to . By using the Hölder inequality it follows
which is bounded by (25) and Lemma 1. Denote Then, we have the following results:
(39) Let absolute moment of
. Then, the is uniformly bounded. Precisely
th
(40) (41) (42) In the above, we applied the inequality of Lemma 5 in (40) and (41) was obtained in view of the assumption that the channel variation is independent of the noise process while (42) is due has the to that by same distribution as that of Lemma 1. By using the Hölder inequality [2], it can be shown . The scaling that with is updated as follows: coefficient
(46) where the first equality follows from that independent of the sigma field is independent of 1 and the fact that from (43) that
is by Lemma . Also, it follows (47)
By summing the above two inequalities we obtain that
(43)
(48)
The proof of [16] is extended to our case for the proof of the . Define the stability of the error dynamics in random times
(49)
YOU AND XIE: MINIMUM DATA RATE FOR MEAN SQUARE STABILIZABILITY OF LINEAR SYSTEMS
where (48) follows from the property of the quantizer given in Lemma 4. By (26), it is clear that . This implies that there exists an such that . In addition, it immediately follows from (49) . For any given , we get that that
781
case of i.i.d. packet losses, there is no explicit characterization of the minimum data rate for the mean square stabilizability of a general vector system. For example, [13] and [10] derived the minimum data rate for scalar systems and [26] generalized it to vector state but single input systems. For brevity, we consider noise free vector systems with bounded initial support in this section. The study of the general stochastic system (1) can be similarly proceeded as the case of scalar systems. A necessary condition for mean square stabilizability will be given in terms of a group of inequalities that are related to unstable eigenvalues of the open-loop system and the transition probabilities of the Markov chain. A sub-optimal bit-allocation scheme, which is optimal for some special cases is provided to ensure the mean square stabilizability. When specialized to the case of i.i.d. packet losses, our work naturally recovers the results in [13].
(50) A. Real Jordan Form Observing that we have designed a stabilizing controller for , the estimator, it is straightforward that which further implies the stabilizability of the system since . Remark 3: It should be noted that in [13], an i.i.d. dropout process is considered. In this case, (27) will directly lead to that , from which the . necessary condition follows by letting However, due to temporal correlations of the Markov process , the above arguments are no longer applicable. To overcome this difficulty, the properties of the Markov process are further exploited in the proof of the necessity. Remark 4: In comparison with [13], [16], a similar adaptive quantizer is adopted to establish the sufficiency. What makes the current problem more challenging is that the down sampling in, is stochastic and unbounded. terval, denoted by While in [13], [16], it is a finite constant. This implies that for their cases, the stabilizability of the periodically sampled system immediately results in the stabilizability of the original system, which does not trivially hold for the randomly down sampled system. Further, Lemma 1 plays an indispensable role in proving the MS-Stabilizability of randomly down sampled systems. Remark 5: We have developed a tool to down sample the . It is not difficult system with a random sampling interval to verify that this approach is applicable for any i.i.d. satisfying . That is, it can be directly applied to networked control systems with transmission times that are driven by an i.i.d. stochastic process. Hence, our approach can jointly address the issues of minimizing the number of transmissions between the sensor and the controller/actuator as well as reducing the size of packet at each transmission, leading to a better utilization of the bandwidth of a communication network. V. VECTOR SYSTEMS The main challenge in stabilizing a vector system with Markovian packet losses consists of optimally allocating bits to each unstable state variable. It is worth mentioning that even for the
As in [13], [16], we adopt a real Jordan form for the system under investigation which is briefly reviewed below. Assume that all the eigenvalues of lie outside or on the unit circle. Otherwise, the matrix can be transformed to a block by a coordinate transformation, diagonal form and respectively correspond to the stable and where unstable(including marginally unstable) subspaces. The mean square of the state variables associated with the stable block will be uniformly bounded for any finite control sequence. be the distinct unstable eigenvalues of Let (if is not a real number, we exclude its conjugate from the be the corresponding algebraic multiplicity of list) and let . Then, there exists a real transformation matrix such that . The real Jordan canonical form [8] has the block diagonal structure with and where if otherwise. In summary, consider the vector system as follows: (51) where the state vector is partitioned in conformity with the block diagonal structure of and the pair is controllable. Moreover, the initial state has a known bounded support, i.e., for some and has a probability density . We are ready to deliver our main results on vector systems. B. Necessity Theorem 3: Consider the system (51) and the network configuration in Fig. 1 where the packet dropout process of the forward channel is a time-homogeneous Markov process with the transition probability matrix (2). Let , denote the dimension of an invariant real subspace of , . Then, a necessary condition for the asymptotic MS-Stabilizability of the networked system is that
782
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 4, APRIL 2011
and
, the following
conditions hold: a) The probability of the channel recovering from packet dropping is large enough (52)
b) The data rate satisfies the following strict inequality: (53) Remark 6: 1) Note that in view of Lemma 1, the data rate condition given in (53) can be easily evaluated. and , it 2) For a lossless digital channel, i.e., is obvious that the sojourn time is always equal to one, (52) is automatically enforced, and the inequality (53) reby selecting duces to . This is the well-known minimum data rate condition for stabilizing an unstable linear system [16], [21]. 3) When it is specialized to unstable scalar systems, (53) be, which is the same as comes the rate condition in (10). 4) When the packet loss is i.i.d., our result recovers the one derived in [13]. Proof of Theorem 3: Together with the results in Section IV, the proof of [13] is extended here to establish the stability of . By the definition of , error dynamics in random time , the block has an invariant real subspace, for any denoted by , of dimension . Denote the indices of the , e.g., and the nonempty subspaces by corresponding state variables w.r.t. by . Consider formed by taking the Cartesian product of all the subspace the nonempty invariant real subspaces, i.e., , the dimension of is computed as . Stack the unstable state variables to get a new vector state , where is some transformation matrix. Thus, the new vector state evolves as follows: (54) where the mean square of
. Similarly, a lower bound of is chosen as
Following the proof of the necessity of Theorem 2, we can obtain that: (55) As in Theorem 1, a necessary condition for (55) is that
into the above By substituting equality and after some simple manipulations, the necessity is established. C. Sufficiency To achieve the asymptotic MS-Stabilizability, we propose a sub-optimal bit allocation to each state variable. The number of bits assigned to each state variable is proportional to the magnitude of its corresponding unstable mode. Theorem 4: Consider the system (51) and the network configuration in Fig. 1 where the packet dropout process of the forward channel is a time-homogeneous Markov process with the transition probability matrix (2). The networked system is asymptotically MS-Stabilizable if the following conditions hold: a) The probability of the channel recovering from packet dropping is large enough (57) b) The unstable eigenvalues are inside the convex hull determined by the following constraints: (58) where
the
rate
allocation satisfies
vector
(59)
Remark 7: 1) For a Markovian lossy channel with infinite bandwidth, the data rate condition of (58) is automatically satisfied. The probability condition of the channel recovering from packet dropping in (57) reduces to that of [6]. That is, the sufficient condition is also necessary in this case. 2) The sufficient condition is optimal when the magnitude of strictly unstable eigenvalues is the same. For example, such that and assume for the vector system of (51), the transition probability and rate condition in (57) and and (58) are respectively written as which are the same as the necessary conditions in (52) and (53) by choosing and . In particular, if all the unstable eigenvalues have the same , the sufficient condition magnitude, e.g., is necessary as well. The following lemma is essential to the proof of the sufficient condition. and Lemma 6: [25] There is a positive such that
(60) (56)
Proof of Theorem 4: For any data rate satisfying (58) and (59), the uniform quantizer of (16) will be adopted. Similar to
YOU AND XIE: MINIMUM DATA RATE FOR MEAN SQUARE STABILIZABILITY OF LINEAR SYSTEMS
the proof of Theorem 2, divide the integers into cycles with length , which is determined by and is to be specified later. The communication protocol is the same as in Theorem 2, except that the , the uniform quantizer is used here. Thus, at each time based on the encoder and decoder share a state estimator quantized messages and packet acknowledgement, and update the estimator as follows:
783
the update recursion for can be established that
in (61) and Lemma 6, it .
In addition, (57) implies that . That is, there exists a such that (63) Then, it is straightforward that 1. Moreover, given any sequence , it is easy to show that
by Lemma such that
By the dominated convergence theorem [2], it follows that:
where a stabilizing control gain is chosen to satisfy that is strictly less than one. Dethe spectra radius of note the th component of the state vector corresponding to the th unstable mode by , where and . The vector symbol is composed by , with while the vector symbol is given by , with , where is a uniform quantizer of (16) using bits of precision to represent quantizer output and similarly for . Note that by (59), we have . Denote the identity matrix. Then, the scaling is given by . Morematrix is simultaneously updated on both sides of the channel over, via
By setting the control to be error, defined by
Thus, there exists a holds:
such that the following inequality
(64) Select a
such that
, we obtain that
(61)
(65)
, the estimation , is recursively computed by
(66) (67)
(62) By Lemma 6, it can be shown that the quantizer will not overflow, i.e., . For ex. Assume , ample, it obviously holds for , then . By the error dynamics in (62),
Here the first equality is obtained from (61) and the fact that is independent of by Lemma 1. The Hölder inequality was applied in (65). The constant in the second inequality is finite by (38). In light of (64), there exists an such that
(68)
784
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 4, APRIL 2011
Using the above inequality and Lemma 6, we can further derive that
Fig. 3. Stabilizable and unstabilizable regions for the example in Section V-D.
(69) (70) In the above, the second inequality follows from Lemma 6. The inequality in (69) is due to that , and that is of . the same distribution as such that and using the Choosing Hölder inequality, we obtain that
(71) Thus,
in
view
of
(68) and (70), we obtain that . This immediately implies that , which further concludes that . . Consequently, we have shown that With the designed stabilizing controller, it is trivial that . The rest of the proof follows from a similar line of arguments as in [13] and the detail is omitted.
VI. CONCLUSION Packet losses and data rate constraint are two important issues of networked control systems. In this paper, we have investigated the necessary and sufficient data rate conditions for the mean square stabilizability of networked unstable linear systems where the communication channel is subject to Markovian packet dropouts. The temporal correlations of the packet dropout process posed significant challenges for the investigation of the minimum data rate which were overcome by converting the networked system with random packet dropouts into a randomly sampled system. The minimum data rate for the scalar case was then derived and is explicitly given in terms of the magnitude of the unstable mode and the transition probabilities of the Markov chain. The result completely quantifies the joint effect of Markovian packet losses and finite communication data rate on the mean square stabilizability of linear scalar systems and contains existing results on packet drop probability and data rate for stabilizability as special cases. We have also studied the mean square stabilizability problem for vector systems. Necessary and sufficient conditions were respectively derived and shown to be optimal for some special cases. Our approach can also be directly used to networked control systems where transmission times are driven by an i.i.d. stochastic process with the consideration of reducing the size of the data transferred at each transmission. Future research includes closing the gap between the necessary and sufficient conditions for general vector systems.
D. An Example In this subsection, an example is included to examine the gap between the necessary and sufficient conditions. Let the transi, tion probabilities of the Markov process be and the data rate be . Consider an unstable system with , and . The distinct eigenvalues stabilizable and unstabilizable regions respectively determined by Theorem 3 and Theorem 4 are plotted in Fig. 3. It is clear from the figure that they are optimal for the three cases respec, and . tively corresponding to that It also shows that the necessary condition is almost sufficient.
REFERENCES [1] P. Antsaklis and J. Baillieul, “Special issue on technology of networked control systems,” Proc. IEEE, vol. 95, no. 1, pp. 5–8, Jan. 2007. [2] R. Ash and C. Doléans-Dade, Probability and Measure Theory. New York: Academic Press, 2000. [3] J. Baillieul, “Feedback designs in information-based control,” Lecture Notes in Control and Information Sciences, pp. 35–58, 2002. [4] J. Baillieul and P. Antsaklis, “Control and communication challenges in networked real-time systems,” Proc. IEEE, vol. 95, no. 1, pp. 9–28, Jan. 2007. [5] T. Cover and J. Thomas, Elements of Information Theory. New York: Wiley-Interscience, 2006.
YOU AND XIE: MINIMUM DATA RATE FOR MEAN SQUARE STABILIZABILITY OF LINEAR SYSTEMS
[6] V. Gupta, N. Martins, and J. Baras, “Optimal output feedback control using two remote sensors over erasure channels,” IEEE Trans. Autom. Control, vol. 54, no. 7, pp. 1463–1476, Jul. 2009. [7] J. Hespanha, P. Naghshtabrizi, and Y. Xu, “A survey of recent results in networked control systems,” Proc. IEEE, vol. 95, no. 1, pp. 138–162, Jan. 2007. [8] R. Horn and C. Johnson, Matrix Analysis. Cambridge, U.K.: Cambridge Univ. Press, 1985. [9] M. Huang and S. Dey, “Stability of Kalman filtering with Markovian packet losses,” Automatica, vol. 43, no. 4, pp. 598–607, 2007. [10] N. Martins, M. Dahleh, and N. Elia, “Feedback stabilization of uncertain systems in the presence of a direct link,” IEEE Trans. Autom. Control, vol. 51, no. 3, pp. 438–447, Mar. 2006. [11] A. Matveev and A. Savkin, “Comments on “Control over noisy channel” and relevant negative results,” IEEE Trans. Autom. Control, vol. 50, no. 12, pp. 2105–2110, Dec. 2005. [12] A. Matveev and A. Savkin, “An analogue of Shannon information theory for detection and stabilization via noisy discrete communication channels,” SIAM J. Control Optim., vol. 46, pp. 1323–1367, 2007. [13] P. Minero, M. Franceschetti, S. Dey, and G. Nair, “Data rate theorem for stabilization over time-varying feedback channels,” IEEE Trans. Autom. Control, vol. 54, no. 2, pp. 243–255, Feb. 2009. [14] L. Montestruque and P. Antsaklis, “Stability of model-based networked control systems with time-varying transmission times,” IEEE Trans. Autom. Control, vol. 49, no. 9, pp. 1562–1572, Sep. 2004. [15] G. Nair and R. Evans, “Exponential stabilisability of finite-dimensional linear systems with limited data rates,” Automatica, vol. 39, no. 4, pp. 585–593, 2003. [16] G. Nair and R. Evans, “Stabilizability of stochastic linear systems with finite feedback data rates,” SIAM J. Control Optim., vol. 43, no. 2, pp. 413–436, 2004. [17] A. Sahai and S. Mitter, “The necessity and sufficiency of anytime capacity for control over a noisy communication link: Part I: Scalar systems,” IEEE Trans. Inform. Theory, vol. 52, no. 8, pp. 3369–3395, Aug. 2006. [18] L. Schenato, B. Sinopoli, M. Franceschetti, K. Poolla, and S. Sastry, “Foundations of control and estimation over lossy networks,” Proc. IEEE, vol. 95, no. 1, pp. 163–187, Jan. 2007. [19] B. Sinopoli, L. Schenato, M. Franceschetti, K. Poolla, M. Jordan, and S. Sastry, “Kalman filtering with intermittent observations,” IEEE Trans. Autom. Control, vol. 49, no. 9, pp. 1453–1464, Sep. 2004. [20] S. Tatikonda and S. Mitter, “Control over noisy channels,” IEEE Trans. Autom. Control, vol. 49, no. 7, pp. 1196–1201, Jul. 2004. [21] S. Tatikonda and S. Mitter, “Control under communication constraints,” IEEE Trans. Autom. Control, vol. 49, no. 7, pp. 1056–1068, Jul. 2004. [22] W. Wong and R. Brockett, “Systems with finite communication bandwidth constraints. II. Stabilization with limited information feedback,” IEEE Trans. Autom. Control, vol. 44, no. 5, pp. 1049–1053, May 1999. [23] L. Xie and L. Xie, “Stability of a random Riccati equation with Markovian binary switching,” IEEE Trans. Autom. Control, vol. 53, no. 7, pp. 1759–1764, Jul. 2008. [24] L. Xie and L. Xie, “Stability analysis of networked sampled-data linear systems with markovian packet losses,” IEEE Trans. Autom. Control, vol. 54, no. 6, pp. 1368–1374, Jun. 2009. [25] K. You, W. Su, M. Fu, and L. Xie, “Optimality of the logarithmic quantizer for stabilization of linear systems: Achieving the minimum data rate,” in Proc. 48th IEEE Conf. Decision Control 28th Chinese Control Conf., Shanghai, China, Dec. 2009, pp. 4075–4080.
785
[26] K. You and L. Xie, “Minimum data rate for mean square stabilization of discrete LTI systems over lossy channels,” IEEE Trans. Autom. Control, vol. 55, no. 10, pp. 2373–2378, Oct. 2010. [27] S. Yüksel, “A random time stochastic drift result and application to stochastic stabilization over noisy channels,” in Proc. 47th Annu. Allerton Conf., 2009, pp. 628–635. [28] S. Yüksel, “Stochastic stability of adaptive quantizers for Markov sources,” in Proc. IEEE Int. Conf. Symp. Inform. Theory, 2009, vol. 1, pp. 527–531.
Keyou You (S’10) was born in Jiangxi, China, in 1985. He received the B.S. degree in statistical science from the Sun Yat-sen (Zhongshan) University, Guangzhou, China, in 2007 and is currently pursuing the Ph.D. degree in electrical and electronic engineering at Nanyang Technological University, Singapore. From May 2010 to July 2010, he was with the ARC Center of Excellence for Complex Dynamic Systems and Control, the University of Newcastle, Australia, as a Visiting Scholar. His current research interests include quantized estimation and control, networked control, system identification and distributed control and estimation. Mr. You received the Guan Zhaozhi Best Paper Award at the 29th Chinese Control Conference, Beijing, China, 2010.
Lihua Xie (F’07) received the B.E. and M.E. degrees in electrical engineering from Nanjing University of Science and Technology, Nanjing, China, in 1983 and 1986, respectively, and the Ph.D. degree in electrical engineering from the University of Newcastle, Australia, in 1992. He was with the Department of Automatic Control, Nanjing University of Science and Technology from 1986 to 1989. Since 1992, he has been with the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, where he is currently a Professor, Deputy Head of Control and Instrumentation Division, and the Director of Centre for Intelligent Machines. He is also a Changjiang Visiting Professor with South China University of Technology. He is an Editor of IET Book Series on Control and an Editor-at-Large of the Journal of Control Theory and Applications. He served as an Associate Editor of Automatica. His current research interests include robust control and estimation, networked control, sensor networks, time delay systems, and control of disk drive systems. In these areas, he has published 4 books, 2 patents and many journal papers. Dr. Xie is an Associate Editor of the IEEE TRANSACTIONS ON CONTROL SYSTEM TECHNOLOGY. He served as an associate editor of a number of archival journals including the IEEE TRANSACTIONS ON AUTOMATIC CONTROL and the IEEE TRANSACTIONS ON CIRCUIT AND SYSTEMS II. He was the General Chairman of the 9th International Conference on Control, Automation, Robotics and Vision and the 7th IEEE International Conference on Control and Automation.
