System Identification: New Paradigms, Challenges, and Opportunities

System Identification: New Paradigms, Challenges, and Opportunities WANG Le-Yi1

ZHAO Wen-Xiao2

Abstract The traditional paradigm of system identification employs prior information on system structures and environments and input/output observation data to derive system models. Extensive research and development on its methodologies, theoretical foundation, algorithms, verifications, and applications over the past half century have established a mature field with a rich literature and substantial benchmark applications. However, rapid advancement in science, technology, engineering, and social medias has ushered in a new era of systems science and control in which challenges and opportunities are abundant for system identification. In this sense, system identification remains an exciting, young, viable, and critical field that mandates new paradigms to meet such challenges. This article points out some potentially important aspects of system identification in these new paradigms, suggests some worthy areas of research focus, and most importantly opens the forum for further discussions. Key words System identification, uncertainty, information, complexity, networked system, large data processing, integration of identification and decision Citation Wang Le-Yi, Zhao Wen-Xiao. System identification: new paradigms, challenges, and opportunities. Acta Automatica Sinica, 2013, 39(7): 933−942

1

Introduction

Introduced in the 1950s as a method to model a dynamic system for control design[1] , system identification in the classical sense has grown into a well-defined and mature field[2−9] . The traditional system identification comprehensively covers many aspects of the modeling process, including data acquisition (from a lab, a testing facility, or during system operation), model structure selection, model parameter estimation, and model validation. System identification was inherently control oriented, often treated as an integral part of feedback control design procedures and adaptive control algorithms. Its typical formulation models a process as a parameterized system with observations corrupted by random noises. Its early development in this stochastic framework employed many results in stochastic analysis, statistics, time series analysis, convergence types and properties, etc. However, its inherent connection with feedback control introduced many distinct and profound features. Many algorithms have been introduced and their properties rigorously established, including the major milestones and fundamental results on least-squares algorithms[8, 10] , prediction error methods[8] , stochastic approximations[11−14] , ordinary differential equation (ODE) approaches[11] , Akaike s information criteria[15−16] , Rissanen s minimum length data description[17] , and many others. Unique system features in input/output observation noises and system feedback structures led to errors-in-variables identification scenarios[18−21] , closed-loop identification[22] , model order reduction[23] , set-valued observations[24−25] , frequencydomain feature extractions[26] , and adaptation[27−29] , as well as many successful application benchmark cases. To accommodate real time data acquisition and integration with feedback control, many recursive algorithms were inManuscript received July 4, 2012; accepted November 7, 2012 Supported by National Natural Science Foundation of China (61134013, 61104052, 61273193) Recommended by Academician HUANG Lin 1. Department of Electrical and Computer Engineering, Wayne State University, Detroit, Michigan 48202, USA 2. Institute of Systems Science, Chinese Academy of Sciences, Beijing 100190, China

troduced to reduce online computational and memory complexities. Despite diversification in approaches, the traditional paradigms of system identification mostly employ the typical steps that cover both off-line modeling and real-time parameter estimation: 1) Physical system specifications: specify the plant that is to be identified together with controllable and uncontrollable inputs, measured and unmeasured outputs, signal ranges, system operating conditions, and targeted applications of the model. 2) Model class selection: select model structures, parameterizations, system orders, and other complexity entities. 3) Input design and data collection: design input signals, scenarios for data collection, sampling schemes and rates, quantization, and time horizons. Typically, input signals must be sufficiently rich to meet persistent excitation conditions that ensure its capability to extract parameter information. Based on the designed and implemented inputs, one performs experiments, collects data, either online or off line, and characterizes data errors either in statistical terms or bounds. 4) Parameter or function estimation: design estimation algorithms, either off-line bulk algorithms or online recursive ones, to extract model functions or parameters by using as much information as possible from the structural information and data. 5) Model validation: evaluate the model s reliability by applying it to a variety of operating conditions, comparing model predictions and actual measurement data, and evaluating model reliability, accuracy, robustness, convergence, etc. The above steps are often iterated until a satisfactory model is obtained. In this line of study, a vast literature has been generated, and most aspects have been carefully and rigorously studied. In this sense, the classical system identification is a highly mature field and provides a rich treatise of understanding and schemes to support related usages of the modeling process, including monitoring, control, diagnosis, decision, and system improvement. Naturally, when a field is so heavily studied, new but related problems will emerge, motivated by applications and also mathematical extension. Most notably the school of non-stochastic system identification, often called “worst-

934

ACTA AUTOMATICA SINICA

case identification”, that regarded noises as “unknownbut-bounded” was active in the 1990s[30−32] . In seeking a set-membership-type characterization of system models in relation to this worst-case formulation, this school linked system identification to its utility with robust control, especially H∞ , L1 , and structured uncertainty. Thanks to this connection with the robust control community, it often used explicitly the term “control-oriented” identification to emphasize its connection to robust control and the needs of a set-type model characterization[33−35] . This development borrowed several important concepts and findings from approximation theory and informationbased complexity[36−38] , including model approximation under the H∞ and L1 metrics[39−45] , worst case identification under time-domain data and unknown-butbounded noise characterization, frequency-domain H∞ identification[46−52] , and related model validation[53−55] . These methodologies characterize model structures and modeling error bounds to be compatible with the robust control frameworks, especially the H∞ and L1 theory developed in the 1980s. Emphasis of complexity analysis in this school[52, 56−58] has introduced new tools into the field of complexity analysis and enhanced our understanding of fundamental limitations of identification methods and data structures. On the other hand, certain emerging concerns with the methodologies, such as conservativeness of the bounds, time complexity in reducing model uncertainty, computational complexity in finding optimal models, and simple and practical algorithms for adaptation, provided much-needed food for thought for a holistic viewpoint of modeling processes, and as such energized the field of system identification towards developing broader frameworks and accommodating new application advancement. Technology advancement during the past two decades in many frontier fields, especially in information technology, has fundamentally changed the landscape for control system science and engineering. Classical industrial process control systems were mostly single-loop and individuallydesigned controllers, and only sporadically considered system interactions and coordinations. With penetration of networking concepts, communication systems, and complex systems into many application areas, systems have experienced profound structural expansion and become increasingly interconnected. For example, automotive systems are linked by communication networks both intra-vehicle and in regional V2V (vehicle to vehicle) and V2I (vehicle to infrastructure) highway systems; airplanes and space shuttles experience thrust into more extreme conditions in their speeds, power, temperature, and system complexity; systems biology departs from traditional reductionism and focuses on biological interconnections of cellular networks; medical devices are increasingly employing micro-electromechanical systems (MEMS) and NANO technologies; distributed renewable generators and smart grids have created many microgrids with communication networks; advanced sensor networks are inherently team operated; internet search and information processing engines have entered cloud computing age; and the list goes on. These new technology progresses have intrinsic needs for system identification. For instance, a key aspect of systems biology is mathematical modeling of cellular networks and their interactions, such as gene regulatory networks, to understand mechanism of inheritance, mutation, and living

Vol. 39

organism s feature development. Recently introduced concepts of cyber-physical systems emphasize coordinations of computer, control, and communications (3C) to study systems holistically. These technology growths have motivated the system identification community to reevaluate its traditional frameworks, look more broadly for new application areas, formulate new problems, recognize new issues, understand new constraints, and explore new solutions. This trend has provided exciting research opportunities, introduced daunting challenges in seeking new techniques, and prompted an urgent need for new tools with practical utility. In light of new system structures and diversified application domains, system identification must adapt itself into broader paradigms. In this broad and holistic doctrine, system identification aims to reduce uncertainties on signals, systems, and environment, to extract information from data and knowledge databases, to compress information into models, in order to support related information processing procedures such as control, monitoring, diagnosis, decision, and coordination[59] . Information processing, storage, transportation, and sharing take valuable resources from data, communications, computation, and measurements; and as such optimal resource utility and complexity analysis become mandatory. While system identification enjoys distinct features, it shares similar goals to feedback mechanism, robust control frameworks, and adaptation procedures, which employ different means to process information and counteract uncertainty. As a result, studies of system identification can benefit from an integrated and interdisciplinary approach in which coordination of information processing and resource distributions can be optimized. Mathematically, system identification is an inverse mapping problem: seeking an inverse of the suitable static, dynamic, or functional mapping from unknown internal parameters or states to measurable variables. Data mining, pattern recognition, machine learning, and statistics, among others, are pursuing similar objectives. Consequently, system identification can absorb ideas, methods, and algorithms from these fields to enrich its own toolboxes. In the following sections, we describe some areas and issues in which system identification can be studied and expanded to, hopefully, assist and impact technology development. As expected, the descriptions of future directions are always influenced by the writers limited knowledge and bias. Many predictions may turn out to be irrelevant in their parts or entirety. The purpose of this article is to point out some promising and important aspects of system identification in some new paradigms, suggest some focus areas, identify key technical obstacles and potential approaches, and most importantly stimulate innovative ideas from others and open the forum for further discussions. The remaining part of the paper is organized into the following sections. Section 2 discusses broader types of uncertainties that system identification should consider. Communication systems play a pivotal role in networked systems. Section 3 highlights some new issues that are unique to identification of systems that contain communication channels. When systems become interconnected and more complex, they naturally introduce nonlinearity and random factors. Although these have been extensively

No. 7

WANG Le Yi et al.: System Identification: New Paradigms, Challenges, and Opportunities

studied in traditional system identification, numerous new issues arise that present new challenges. These are summarized in Section 4. Section 5 is devoted to emergent phenomena for system identification caused by new data types and patterns from the information age. Section 6 elaborates reasons why complexity analysis should draw more attention in relation to smart usage of resources. Embedding system identification within intended applications, objective-oriented system identification is delineated in Section 7 such that more diversified performance indices and criteria can be germinated. Section 8 encourages software tool development that is user-friendly and efficient. Finally, Section 9 summarizes the key viewpoints of this article and suggests several pathways to nurture new paradigms of system identification. This article is based on its earlier Chinese version[60] , with some alterations.

2

Accommodation of broader types of uncertainties

Traditional system identification was mostly focused on random additive observation noises and actuator noises, as well as some randomly jumping processes affecting system parameters. Observation noises typically arise from sensor thermal noise, measurement errors, and lower-level communication errors. A typical paradigm of system identification states that when an input is applied to a dynamic system and the system output is measured with additive random noise corruptions, system identification aims to acquire as much information on the system s characteristic mappings or parameters as possible from such noiseinfested observations. In this framework, stochastic analysis methods[61−63] have played important roles. It is well perceived that they will retain their prominent positions in new paradigms of system identification. However, practical systems encounter much broader types of uncertainties. Incorporating more diversified uncertainty types in system identification will lead to new directions and methodologies of system identification beyond classical stochastic analysis. 1) Expanded stochastic noise types. While additive random observation noises form the central scenario in classical system identification, new applications will much broaden the varieties of noises. In new applications, stochastic uncertainties will stem from more diversified scenarios, such as simplified sensors like binary-valued sensors, data coding and decoding, compression, random transmission delays in communication systems, or even artificially added randomness for facilitating information treatment. Correlated noises with time- and space-dependent statistic characterizations, network structure-dependent noise types, random sampling schemes, asynchronous and uncoordinated random streaming of data in different parts of a networked system, and multiplicative noises are some examples of new types of stochastic noises that need to be accommodated. 2) Deterministic worst-case types. Much of a system s structural uncertainties arise from model simplification. For instance, representing an infinite dimensional system by a finite dimensional system or a higher-order system by a lower-order one may introduce unmodeled dynamics; using a linear model to approximate locally a nonlinear function or using simpler nonlinear functions to rep-

935

resent mappings of unknown structures can produce model mismatching; and clustering high-complexity system components will inevitably leave structural uncertainties. Such uncertainties are not of random nature since they do not change randomly with each observation point, and hence do not demonstrate “averaging” effects of random phenomena. A fundamental consequence is that their effects cannot be attenuated by applying the laws of large numbers or central limit theorems, nor appropriately described by statistical analysis. These uncertainties directly impact estimation accuracy and characterization, and hence they need to be included in system identification. Such uncertainties are more suitably represented by deterministic worst-case types[31, 64−68] . Other than some limited explorations[66] , incorporation of both worst-case and stochastic uncertainties remains an open field. For example, a battery system involves complicated chemical and electrical processes. Nano- and micro-level models exist, but are too complicated for real-time system identification. Typically, focusing on their macro behavior means that the model structure we choose involves errors from clustering system components, model mismatching due to unknown model structures, and unmodeled dynamics when a low-order differential equation model is used. 3) Uncertainties from lack of data and information. There are uncertainties that may be of random nature, but their statistical properties cannot be obtained due to lack of data or related information. For example, when we model a communication network for a mobile system (such as autonomous highway vehicles in platoons, or unmanned aerial, ground, and underwater vehicles), the effects of “mobility” and “terrain conditions” are extremely difficult to model. Unless the terrains are repeatedly used, the data we receive cannot be accumulated to suffice a statistical analysis of the network model. Similarly, reliable dynamic models for cellular networks require real-time and large-throughput data in next-generation DNA sequencing, which are still unavailable at present due to limitations in instrumentation. Vague and imprecise descriptions and partial information must be used to facilitate system identification and information processing under these circumstances. 4) Uncertainties due to lack of computational capability. Even for systems that have well established model structures, computational capability limitations may not allow us to build such a model for system identification. For example, a weather forecasting system involves many factors and their histories. While historical data may be available, factoring them into the model will render an enormous computational burden that makes “forecasting” irrelevant. Eliminating many factors which are deemed “minor” so that the model building process becomes feasible must be compensated with a description of “truncated” subsystems and their impact, which is usually not random. Studies of complexity issues in such cases will become increasingly important for quantitatively understanding how to use computational resources smartly. 5) Uncertainties due to structural switching. Complex systems are characterized by interactions of their subsystems. Such interactions are exemplified by their network topologies, which often change with time. Communication channels are assigned according to data traffic and priorities, and hence are always dynamically changing

936


their routing topologies. In internet searching and computing engines such as Google search, parallel processing, and cloud computing, data are streamed from multiple and ever-changing pathways and reconfigured back to their original structures at the receiving site. When such changes cannot be directly observed, they must be viewed as uncertainties. This is a vastly open field in which the questions of topology dynamics, their impact on data accuracy and reliability, and information processing robustness under such dynamic networks remain largely unknown at present. What is needed in this direction is to include multiple types of uncertainty descriptions in designated identification problems. Coexistence of multi-uncertainty types is easily understood in practical systems. For instance, while unmodeled dynamics (from order reduction) and model mismatch (from function simplification) should be described in an uncertainty set, modeling noises in stochastic frameworks is more reasonable and less conservative than the worst-case formulation[66−67] . In addition, partial information on the system will add another piece of structural information. Potential approaches include: 1) within the framework of the ODE approaches for convergence analysis of stochastic approximation algorithms[9, 12] , one may use differential inclusions to accommodate the impact of worst-case-type uncertainties; 2) in convergence analysis using martingale convergence theorems[69] , one may consider the worst-case scenarios when the underlying systems have deterministic uncertainties; 3) in estimation error characterization, one may use worst-case probabilistic errors when applying standard bounds and large deviations principles; 4) in regime-switching systems, one may deal with integrated models of hybrid systems and formulate joint identification problems. In this pursuit, it is necessary to study network topology uncertainties so that networked system identification can be properly studied. At present, research effort in integrating different types of uncertainties in system identification is increasing but remains sporadic. More persistent and organized team efforts will be needed to advance this research frontier.

3

System identification under networked systems and communications

System identification has become a mature field in standard system settings, mostly in small-scale and single-loop structures. On the other hand, one of the dominant trends in physical system developments is that systems become increasingly interconnected. More often than not, system interconnections are provided by communication networks, which include typical wireless communication systems and signaling systems of living organisms such as cellular signaling. Unlike traditional process control systems in which one concentrates on the control of one process at a time, nowadays system interaction and coordination are of substantial proportions that demand a new look at system identification frameworks. Examples are numerous. In smart grids, renewable and distributed energy generators, controllable loads, smart meters, phaser measurements, and the concepts of microgrids have broken the legacy notion of the standard power grid structures and mandated new methods for information gathering and characterization of subsystems. Large-scale battery systems consist of thousands of battery cells and

Vol. 39

modules, whose real-time characterization requires a new tool set of networked identification. Biological systems and human-patient modeling must include interactions of many subsystems, from blood circulation to metabolism, from immune systems to neural signal transmissions. Highway vehicle control, as well as unmanned aerial, ground, and underwater vehicles often form mobile teams to accomplish a coordinated mission. Distributed computing and information processing, such as cloud computing, will take over dedicated computer systems to become the new norm. Systems biology, material science, patient management, weather forecasts, and finance frequently employ the multiscale modeling structures that seek to seamlessly move from molecular or even atom level subsystems to congregated higher level models for targeted information processing. These problems demand new identification methods[70] and have already started to attract attention[71−72] . The main new challenges derived from the networks include communication constraints and characterizations, dynamic switching network topologies, data flow constraints, and network complexity issues. To accommodate such new developments, system identification must consider networkrelated issues including, but certainly not limited to, the following aspects[73−74] . 1) Local information. Networked systems can be of enormous scales. Information traffic through the network creates overhead costs, uses precious network resources, and causes congestions. It is desirable that data acquisition be confined to neighborhood data flows. How to use the network structures to propagate processed information, rather than centralized raw data storage and processing, for networked system identification will be of great utility in such systems. 2) Communication constraints and uncertainties. Unlike dedicated wired grids, wireless communications are far more dynamic and uncertain. Communications are constrained by power limits and available bandwidths. Typical communication systems involve sampling, quantization, data compression, and source coding to reduce data sizes and increase transmission reliability. Wireless communications are usually shared by many users. Packet loss, transmission delays, errors in transmission are common. The impact of such uncertainties on system identification must be carefully studied. These uncertainties are influenced by data flow throughput and power level, and can be managed by transmission protocols and coding/decoding schemes. Such new elements demand a fresh look at system identification. The recent development of quantized identification and sampling schemes for state estimation is an effort in this direction[24−25, 75] . The inclusion of communication design as part of system identification will be of great interest. 3) Reliability of system identification under network topology variations. Unique to internet and wireless communications, communication networks that link subsystems are changing all the time. Especially in mobile systems or networks of high traffic volume, channels can be lost and connected randomly due to signal path changes, terrain conditions, and competition for channels by users of different priorities. Burst-type traffic, such as conversations, uses network resources for a short time interval with large demands and then releases them. On the other hand, system identification is usually a steady task,

No. 7


requiring persistent data to maintain its functionality. Consequently, networked system identification under randomly switching network topologies and other uncertainties will be a worthy area for exploration. 4) Identification of network structures. Unlike traditional system identification in which model structures can be selected from a few typical choices, complex and interconnected systems must identify system structures[76] . This is especially true in gene regulation networks[77−79] . Understanding cell-cell interactions and signaling mechanism from molecular and cell biology will form a foundation for developing potential networked model structures to explain emergent properties of proteins, but present a major challenge for network structure identification. To deal with the above challenges, some recent efforts appear to be promising and worth further investigation. We list a few of them here. 1) We need a theory of system identification under irregular and asynchronous sampling times[75] . Communication networks introduce packet losses and delays. Consequently, data arrive at unpredictable times. Input excitation signals can be commanded but will be executed at some unknown times due to the same reason. 2) We need a theory of system identification for delay systems. Delays introduce infinite dimensional systems and force consideration of functional stochastic or regular differential-integral equations. To facilitate control design and adaptation, the delay time needs also to be estimated. 3) We need a theory of system identification using localized or neighborhood information. This information-structure constraint may be formulated within the identification algorithms (adaptive filtering, recursive least-squares, stochastic approximation, etc.) as a structural matrix that defines the information flow topology. Convergence analysis with such constraints will lead to a meaningful new area of networked system identification. 4) We need a comprehensive theory of joint system identification and state estimation of hybrid systems that involve system uncertainty and unknown events. Such a theory will include communication networks as part of identification problems.

4

Identification of stochastic and nonlinear systems

The identification of stochastic systems has a long history and now is in a new development stage, for instance: from linear systems to nonlinear systems; from a single process to networked systems; from control-oriented focus to interdisciplinary research, such as systems biology, aviation and aerospace technology, quantum and nano science, finance, etc. After decades of extensive research, stochastic identification frameworks have shown clear signs of maturity in theoretical foundation, algorithm development and refinement, accuracy and convergence properties, and applications. Typical tools include Chebyshev-Markov-Stieltjes inequalities, Chernoff bound, martingale convergence theorems, laws of large numbers, central limit theorems, large deviations principles, Markov chains, stochastic differential equations, etc. Nonlinear system identification has been pursued by many researchers[80−81] . Generally speaking, the nonlinear phenomenon is the rule rather than the exception. Consequently, nonlinear system identification is inherently di-

937

versified. Current research on nonlinear system identification is highly active, fruitful, and exciting on the one hand, but lacks inter-connections and collaborations on the other. Relatively extensive results exist for nonlinear systems of special structures, such as Wiener systems[11, 82−83] , Hammerstein systems[84−88] , and their extensions[89−90] , nonlinear autoregressive model with exogenous input (ARX) systems[91] , systems with quantized observations[92−93] , kernel and subspace methods[94−95] , and regime switching systems[96−97] , among others. System identification will encounter more and more time varying, stochastic, and nonlinear components. First, if system parameters vary with time or operating conditions, or experience sudden changes, their behavior resembles internal states. Joint estimation of system parameters and states inevitably introduces nonlinear problems. Essential properties of such systems, such as joint observability, identifiability, and signal persistent excitation conditions are largely unknown at present. When identified models are used in control, decision, and optimization in real time, random noises will enter the system through nonlinear pathways, leading to stochastic nonlinear systems. Typical environment information such as noise stochastic characterizations is difficult to obtain a priori, and hence must be extracted from observation data also. Combined identification of system parameters and environmental conditions is a nonlinear problem. Finally, time variation will result in irreducible errors, similar to the famous “Uncertainty Principle” in physics[98] . Since integration of system identification and control in real-time implementation often introduces time varying components, irreducible errors and time/space complexity in stochastic nonlinear identification problems are of essential importance. Along with the tremendous scientific progress in the last few decades, boundaries among different research domains become increasingly blurred, and correspondingly new identification and estimation problems for stochastic nonlinear systems emerge from diverse areas. These are exemplified by system modeling and identification of smartmaterial actuators, financial data modeling and prediction, the structural and functional inference of genetic regulatory networks, and the key node detection of world wide web and social networks, among others. These topics are challenging, require collaboration among experts with different backgrounds, and also open great opportunities. New schemes that can potentially promote expanded solutions to stochastic nonlinear identification may encompass cascaded and hierarchical recursive algorithms, setvalued system identification, stochastic subspace methods, kernel identification, etc.

5

System identification in the era of data explosion

Historically, acquiring information is expensive since it involves sensors, measurement devices, and wiring and packaging. On the other hand, due to the standard wired structure and readily available on-board computers, information transformation is easy and information processing in a computer is relatively cheap. Under such scenarios, data acquisition is well-specified, -designed, and controlled, and the collected data are all useful. Consequently, in typical system identification experiments, we

938


focus on choosing correct input forms to make them “persistently exciting” and the corresponding system outputs are well defined. In the process, almost no unnecessary information is generated. The new scenarios are emerging: systems are increasingly inter-connected by wireless networks; advanced data acquisition devices are placed in many ports to serve multiple purposes. For example, smart meters are now installed at millions of end users to support smart grid developments. Traffic conditions are monitored using V2V and V2I communication systems that collect data on traffic flows, weather conditions, and accidents, among others. Such devices indiscriminately collect data in fast sampling schemes and in large volume before storing them in designated databases. For system identification, such data flow schemes represent a new scenario in which data acquisition is cheap, but data transformation, transportation, and processing become very expensive. This trend has generated a hot buzz around “data mining” in the computing community[99] . System identification development has not adapted itself to be comfortable with such new information structures yet. In such scenarios, useful data must be intelligently extracted from the databases, compressed, quantized, and coded for reduced complexity to facilitate transformation through networks, then reconstructed and processed for system identification. There is a vast opportunity to develop new paradigms of system identification in this domain. Thanks to their data specificity, massive data carry information in vastly different structures and contents. As such the generic study of massive data processing and algorithms may not yield efficient tools. From this viewpoint, it is necessary to take an interdisciplinary approach, by which one identifies information structure, designates data formats, describes “useful” information for targeted applications, derives causal data relationships to model structures and parameters, devises efficient algorithms, and even develops new embedded or distributed systems for system identification with massive and sparse-information data. It is always desirable to treat (or pre-treat) data at local datagenerating sites to avoid unnecessary data shuffling. Consequently, parallel, asynchronous, distributed data processing will naturally be characterizing features in large-data system identification paradigms. In this respect, system identification can borrow ideas from many branches of data mining, machine learning, distributed computing, and pattern recognition communities, which started to deal with massive data issues decades ago[99−103] .

6

Complexity and smart utility of resources

Studies of complexity intend to understand fundamental relationships between resources and goals. To achieve uncertainty reduction via system identification, one encounters issues of resource consumptions and limitations. Depending on applications, resources can take different forms. For example, using a cheap binary-valued sensor can reduce cost, but increases data uncertainty. In networked system identification, the most commonly cited network resources are power and bandwidth. To reduce power consumption, system identification needs to tolerate more delays and er-

Vol. 39

rors that are typically associated with low-power transmission. To reduce bandwidth usage, one needs to reduce sampling rate, use low-resolution quantization, apply data compression, decrease data redundancy, employ partial information, etc. All of these considerations are relatively new to system identification and form promising grounds for new research directions. In traditional system settings, resources are commonly related to data acquisition costs, including sensors, sensor types, sensor locations, numbers of scenarios to be covered in identification experiments, data duration, sampling rates, and data storage needs, among others. In this sense, if one can eliminate a sensor, use a cheaper sensor, or rely on smaller data sizes, resource utility is reduced. Such reduction will have impact on identification quality (such as the accuracy or sizes of uncertainty sets) and speed (such as convergence rates, Cramer-Rao (CR) bounds, information criteria). Rigorous studies of such impact are of fundamental nature and become more and more important. Historically, worst-case identification employed information-based complexity and complexity notions in approximation theory to study model complexity, identification speed, and irreducible errors[36−38] . Similarly, pursuit of irreducible identification errors by using CR lower bounds and information criteria in model complexity reduction has been well accepted as essential to system identification in stochastic frameworks[8, 12, 93] . There are tremendous opportunities in exploring complexity issues in system identification and its connection with control[52, 104] . Due to technical difficulties, this has not been a common pursuit in the past. In new paradigms of system identification, one should look into building new complexity results from the profound theoretical foundations in approximation theory, statistics, information theory, and computational complexity. The following four pillars can be integrated to develop a new complexity theory for system identification in a cross-disciplinary platform. 1) Approximation theory. Identification time and sampling complexity[52, 56, 68, 98] , model complexity[57] , feedback complexity[58] can be viewed as different types of complexity results in approximation theory. In fact, in a broader view, a feedback system s capability in providing robustness can be expressed in approximation theory as well. Consequently, when we describe unmodeled dynamics, model mismatch, model uncertainty set, reduction of uncertainty by feedback, or uncertainty reduction by acquired data, our complexity analysis can benefit greatly from the complexity results in approximation theory[36−37, 103] . 2) Statistics. The celebrated Cramer-Rao lower bound[105] and Fisher information[106] define precisely the information contents contained in noise-corrupted data about system parameters. Fisher information matrices have found diversified applications[106−107] . However, it remains a puzzle how to combine this characterization with communications and approximation theory to derive a complexity theory of broader appeals. 3) Information theory. Shannon s information theory is a fundamental foundation for information. Its impact on source coding, channel coding, data distortion, and data compression are well recognized[108−111] . In addition, Shannon s sampling theory characterizes sampling

No. 7


complexity in signal processing. In system identification of networked systems, the information theory should be a foundation as well. However, it must be expanded to incorporate other complexity measures. 4) Computational complexity. Computational complexity was traditionally studied in computer logic, automata, and computer languages[112] . It focused on classification of computational procedures in terms of P-class vs. NP-class. Generally, a P-type problem is viewed as “computable” and an NP-type problem is “intractable”. However, for system identification and control applications, this fundamental characterization of computational complexity is not sufficiently detailed to provide a viable constraint on system identification algorithms. At present, it is not clear how this can be resolved. On the other hand, the concepts that have been used in physics, quantum computing, molecular modeling, may provide some guidance, including algorithmic complexity and information, entropy, etc.[113] .

7

Objective-oriented and system identification

integrated

System identification is focused on modeling a system using real-time operational data. The terms of “controloriented identification” and “adaptive control” represent the most pronounced effort in arguing that system identification should be integrated with control[114] . Integrated system identification aims to coordinate its functions and complexity with its goals in control, decision, monitors, and diagnosis, so that resources can be saved. 1) First, system identification should be formulated within its usage so that it will achieve the goals without excessive usage of its resources. Otherwise, valuable resources will be wasted. For instance, for most fault detection and diagnosis problems, the objectives are to distinguish if the system is in the “normal” range or a “fault” has occurred. In this aspect, it is unnecessary to pursue convergence of parameter estimates or even high precision. Another case is vital sign monitoring for humans, including heart rates, respiratory rates, blood pressures, etc. For patients with normal heart and lung functions but high blood pressures, more resources should be directed toward blood pressure estimation for closer monitoring on these higher priority variables. In other words, critical parameters and non-essential parameters can be assigned different accuracy and priority scores in goal-oriented identification. 2) Both system identification and feedback mechanisms are to enhance a system s ability in dealing with uncertainty, and as a result their functions can assist each other in a coherent platform[115−116] . The reason for uncertainty reduction is related to its targeted applications. Reduction of uncertainty on the system may imply more accurate control, better monitoring reliability, and better system protection through diagnosis. On the other hand, there are other mechanisms that can help to achieve these objectives. For example, feedback mechanism can, when used properly, achieve accurate control under large system uncertainties (robust and adaptive control). Partial information from interconnected systems can help in diagnosis (medical diagnosis and deductive learning). Consequently, the task of system identification, when viewed as part of a larger system structure, will vary, depending on its usage. It may not need to be so accurate if other means can help. The

939

smart design of control structures and functions in assisting identification is highly interesting and promising. 3) Networked systems interact, and subsystems influence and possess information about their neighbors. Also, subsystems have different objectives and specifications. Using networked information to help identification of subsystems will yield better solutions than centralized processing or uniform commitment of resources to all subsystems. At present, this direction has not been actively researched. Goal-oriented information processing takes more prominent roles in some related fields, such as machine learning, data mining, congregation of Markov chains, and Petri nets, perhaps due to their encounters with resource limitations, curse of dimensions, and computational complexity. Concepts and ideas from these fields can be of help for system identification.

8

Customer services: user-friendly and efficient tools

Tools for system identification have been pursued, mostly in code development, such as System Identification Toolbox in Matlab/Simulink[117−120] . At present, applying system identification methods in practical systems requires advanced training in this field. Although it is easily observed that system identification should be part of many practical systems, its online applications for real-time updating and controller adaptation are still limited. Currently, most modeling efforts are off-line. In particular, for more complicated systems, even traditional systems such as chemical processes and automotive systems of multi-input-multioutput configurations, the current tools for system identification remain inadequate. Learning from consumer product development, especially consumer electronics, one observes that while most people do not understand high technology concepts and details in iPhones, Facebook, or cloud computing, people use them routinely and happily. This is due to the user-friendly tools that are added to the core technologies in these gadgets. To make system identification part of routine utility in control systems, control strategy adaptations, and reliable monitoring and diagnosis, tool packages need to be developed so that engineers of very limited background in system identification can learn to use the tool packages easily and quickly. Image a hand-held box of “Smart SYSID” or a special-purpose embedded system: signal connections from the physical system prompt a window of a few manual items; and a few button pushes will generate system characteristics and models.

9

Conclusions and suggestions

This article summarizes promising and important directions for system identification, within the doctrine of uncertainty, information, and complexity, and its perceivable impact in several emerging and critical application areas. We hope that this preliminary exploration of strategic directions can stimulate further thinking in the field, and we wholeheartedly welcome comments and opinions. In a broad sense, system identification aims to reduce uncertainties for assisting decisions. Along with new technology advancement, the field must reposition itself to create new frameworks to meet emergent challenges in terms of problem formulation, methodology development, theoreti-

940


cal foundation, software tools and hardware modules, and user application packages. This will require coordinated effort, team collaboration, and persistent and designated funding. To accommodate such developments, we offer a few suggestions. 1) Critical areas. In consideration of the vast spectrum of new application areas and limited resources, it is advisable to coordinate research activity towards several thrust areas that have been identified as the high-impact areas by many countries, which have experienced rapid infrastructure development and vast R&D investment. In particular, the following three areas are worth special attention: 1) Life science advancement. These are exemplified by systems biology, medical devices, sensing and control technologies for drug delivery and human monitoring, and modeling and control for human systems. 2) Advanced transportation systems. These include electric and hybrid vehicles, batteries, automated highway vehicle management, unmanned aerial, ground, and underwater vehicles, mobile robots, distributed sensing systems, etc. 3) Renewable energy and green economy. Distributed generation, wind, photovoltaic, thermal generators, smart grids, and distributed optimization for power dispatching are typical examples. 2) Open access databases on system identification. Establishing openly accessible data centers for methods, algorithms, theoretical findings, application benchmarks, and software packages will facilitate research collaboration for researchers, engineers, and students. To promote technology transfer and enhance impact, the field s development must be interdisciplinary with broad participation. Collective development of the above databases is one of the means to stimulate team coordination and collective wisdom. 3) Advanced experimental centers. New paradigms of system identification will be characterized by large data and information processing, extensive networked systems, and interdisciplinary research activity. For example, experimental data acquisition and system validation on highspeed trains, space shuttles, smart grids, autonomous flying machines, and grid-scale battery systems require expansive testing facilities, fast computing machines, and vast communication network infrastructures. These are typically beyond most individual research groups or institutions. By providing such national facilities to all research groups, many more teams can pursue new strategic directions, creating a healthy and stimulating competitive environment to advance new system identification technologies. 4) Talent development. Paradigm shifting from traditional small and local systems to networked systems of various technology areas must be supported by new interdisciplinary talents who have both strong theoretical training and in-depth domain knowledge. As compared to traditional system identification, the new strategic directions are far more uncertain and require new ways of thinking in terms of problem definition, model structures, constraints, tools for solving new problems, software packages for simulation, data acquisition, optimization, etc. Interdisciplinary talents are integrators who will learn material from new fields, formulate meaningful problems, consolidate teams of diversified expertise, and seek usable solutions in sync with technology advancement so that the domain experts of specific areas can actually benefit from the findings.

Vol. 39 References

1 Zadeh L A. On the identification problem. IEEE Transactions on Circuits and Systems, 1956, 3(4): 277−281 2 Cai Ji-Bin. System Identification. Beijing: Beijing Institute of Technology Press, 1989 (in Chinese) 3 Guo Lei, Cheng Dai-Zhan, Feng De-Xing. Introduction to Control Theory: from Basic Concepts to Research Frontiers, Beijing: Science Press, 2005 (in Chinese) 4 Ljung L, Vicino A. Guest editorial: special issue on system identification. IEEE Transactions on Automatic Control, 2005, 50(10): 1473 5 Gevers M. A personal view of the development of system identification: a 30-year journey through an exciting field. IEEE Control Systems Magazine, 2006, 26(6): 93−105 6 Ljung L. Perspectives on system identification. In: Proceedings of the 17th IFAC World Congress. Seoul, South Korea: IFAC, 2008. 7172−7184 7 Ljung L, Hjalmarsson H, Ohlsson H. Four encounters with system identification. European Journal of Control, 2011, 17(5−6): 449−471 8 Ljung L. System Identification: Theory for the User. Englewood Cliffs, NJ: Prentice-Hall, 1987 9 Chen H F, Guo L. Identification and Stochastic Adaptive Control. Boston, MA: Birkh¨ auser, 1991 10 Ljung L. Analysis of recursive stochastic algorithms. IEEE Transactions on Automatic Control, 1977, 22(4): 551−575 11 Chen H F. Recursive identification for Wiener model with discontinuous piece-wise linear function. IEEE Transactions on Automatic Control, 2006, 51(3): 390−400 12 Kushner H J, Yin G. Stochastic Approximation and Recursive Algorithms and Applications (2nd edition). New York: Springer-Verlag, 2003 13 Chen H F, Zhu Y M. Stochastic approximation procedures with randomly varying truncations. Science in China, Series A, 1986, 29(9): 914−926 14 Chen H F. Stochastic Approximation and Its Applications. Dordrecht, The Netherlands: Kluwer Academic, 2002 15 Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control, 1974, 19(6): 716 −723 16 Burnham K P, Anderson D R. Multimodel inference: understanding AIC and BIC in model selection. Sociological Methods and Research, 2004, 33(2): 261−304 17 Rissanen J. Modeling by shortest data description. Automatica, 1978, 14(5): 465−471 18 Zheng W X, Feng C B. Identification of stochastic time lag systems in the presence of colored noise. Automatica, 1990, 26(4): 769−779 19 Zheng W X, Feng C B. A bias-correction method for indirect identification of closed-loop systems. Automatica, 1995, 31(7): 1019−1024 20 Soderstrom T. Accuracy analysis of the Frisch scheme for identifying errors-in-variables systems. IEEE Transactions on Automatic Control, 2007, 52(6): 985−997 21 S¨ oderstr¨ om T, Soverini U, Mahata K. Perspectives on errorsin-variables estimation for dynamic systems. Signal Processing, 2002, 82(8): 1139−1154 22 Forssel U, Ljung L. Closed-loop identification revisited. Automatica, 1999, 35(7): 1215−1241 23 Obinata G, Brian D O A. Model Reduction for Control System Design. London: Springer, 2001 24 Kalman R E. Design of a self-optimising control system. Transactions on ASME, 1958, 80: 468−478 25 Astr¨ om K J, Wittenmark B. On self tuning regulators. Automatica, 1973, 9(2): 185−199 26 Guo L. Convergence and logarithm laws of self-tuning regulators. Automatica, 1995, 31(3): 435−450 27 Durrett R. Probability: Theory and Examples (Cambridge Series in Statistical and Probabilistic Mathematics) (4th edition). Cambridge: Cambridge University Press, 2010

No. 7


28 Zhou Tong. Introduction to Control-Oriented System Identification. Beijing: Tsinghua University Press, 2004 (in Chinese) 29 Milanese M, Belforte G. Estimation theory and uncertainty intervals evaluation in presence of unknown but bounded errors: linear families of models and estimators. IEEE Transactions on Automatic Control, 1982, 27(2): 408−414 30 Milanese M, Vicino A. Optimal estimation theory for dynamic systems with set membership uncertainty: an overview. Automatica, 1991, 27(6): 997−1009 31 M¨ akil¨ a P M, Partington J R, Gustafsson T K. Worst-case control-relevant identification. Automatica, 1995, 31(12): 1799−1819 32 Chen J, Gu G X. Control-Oriented System Identification: An H∞ Approach. New York: Wiley-Interscience, 2000 33 Smith R S, Dahleh M A [Editor]. The modeling of uncertainty in control systems. In: Proceedings of the 1992 Santa Barbara Workshop. Berlin, Germany: Springer-Verlag, 1994 34 Helmicki A J, Jacobson C A, Nett C N. Control oriented system identification: a worst-case/deterministic approach in H∞ . IEEE Transactions on Automatic Control, 1991, 36(10): 1163−1176 35 Gu G X, Khargonekar P P. Linear and nonlinear algorithms for identification in H∞ . IEEE Transactions on Automatic Control, 1992, 37(7): 953−963 36 Gu G X, Khargonekar P P. A class of algorithms for identification in H∞ . Automatica, 1992, 28(2): 299−312 37 Chen J, Nett C N. The Caratheodory-Fejer problem and H∞ /l1 identification: a time domain approach. IEEE Transactions on Automatic Control, 1995, 40(4): 729−735 38 Chen J, Nett C N, Fan M K H. Worst case system identification in H∞ : validation of a priori information, essentially optimal algorithms, and error bounds. IEEE Transactions on Automatic Control, 1995, 40(7): 1260−1265 39 Zhou T, Kimura H. Time domain identification for robust control. Systems and Control Letters, 1993, 20(3): 167−178 40 Chen J. Frequency-domain tests for validation of linear fractional uncertain models. IEEE Transactions on Automatic Control, 1997, 42(6): 748−760 41 Poolla K, Khargonekar P, Tikku A, Krause J, Nagpal K. A time-domain approach to model validation. IEEE Transactions on Automatic Control, 1994, 39(5): 951−959 42 Smith R S, Doyle J C. Model validation: a connection between robust control and identification. IEEE Transactions on Automatic Control, 1992, 37(7): 942−952 43 Anderson T W. The Statistical Analysis of Time Series. New York: Wiley, 1971 44 Serfling R J. Approximation Theorems of Mathematical Statistics. New York: Wiley, 1980 45 Bohlin T. On the maximum likelihood method of identification. IBM Journal of Research and Development, 1970, 14(1): 41−51 46 Wang L Y. Uncertainty, information and complexity in identification and control. International Journal of Robust and Nonlinear Control, 2000, 10(11−12): 857−874 47 Casini M, Garulli A, Vicino A. Time complexity and input design in worst-case identification using binary sensors. In: Proceedings of the 46th IEEE Conference on Decision and Control. New Orleans, LA, USA: IEEE, 2007. 5528−5533 48 Milanese M, Vicino A. Information-based complexity and nonparametric worst-case system identification. Journal of Complexity, 1993, 9(4): 427−446 49 Wang L Y, Yin G G. Towards a harmonic blending of deterministic and stochastic frameworks in information processing. Robustness in Identification and Control, Lecture Notes in Control and Information Sciences Volume 245. London: Springer-Verlag, 1999. 102−116 50 Wang L Y, Yin G G. Persistent identification of systems with unmodeled dynamics and exogenous disturbances. IEEE Transactions on Automatic Control, 2000, 45(7): 1246−1256

941

51 Dahleh M A, Theodosopoulos T V, Tsitsiklis J N. The sample complexity of worst-case identification of FIR linear systems. Systems and Control Letters, 1993, 20: 157−166 52 Chong C Y, Kumar S P. Sensor networks: evolution, opportunities, and challenges. Proceedings of the IEEE, 2003, 91(8): 1247−1256 53 Wang J D, Zheng W X, Chen T W. Identification of linear dynamic systems operating in a networked environment. Automatica, 2009, 45(12): 2763−2772 54 Zhang Q, Zhang J F. Distributed parameter estimation over unreliable networks with Markovian switching topologies. IEEE Transactions on Automatic Control, 2012, 57(10): 2545−2560 55 Liu X H, Goldsmith A. Wireless communication tradeoffs in distributed control. In: Proceedings of the 42nd IEEE Conference on Decision and Control. Maui, HI: IEEE, 2003. 688−694 56 Yuksel S, Basar T. Optimal signaling policies for decentralized multicontroller stabilizability over communication channels. IEEE Transactions on Automatic Control, 2007, 52(10): 1969−1974 57 Wang L Y, Zhang J F, Yin G. System identification using binary sensors. IEEE Transactions on Automatic Control, 2003, 48(11): 1892−1907 58 Wang L Y, Yin G, Zhang J F, Zhao Y L. System Identification with Quantized Observations. Boston, MA: Birkh¨ auser, 2010 59 Wang L Y, Li C Y, Yin G, Guo L, Xu C Z. State observability and observers of linear-time-invariant systems under irregular sampling and sensor limitations. IEEE Transactions on Automatic Control, 2011, 56(11): 2639−2654 60 He T, Lu X L, Wu X Q, Lu J A, Zheng W X. Optimizationbased structure identification of dynamical networks. Physica A: Statistical Mechanics and Its Applications, 2013, 392(4): 1038−1049 61 Zhou T, Wang Y L. Causal relationship inference for a largescale cellular network. Bioinformatics, 2010, 26(16): 2020− 2028 62 Wang Y L, Zhou T. A relative variation-based method to unraveling gene regulatory networks. PLoS ONE, 2012, 7(2): e31194 63 Bolouri H, Bower J M. Computational Modeling of Genetic and Biochemical Networks. Cambridge, Mass: MIT Press, 2001 64 Billings S A. Identification of nonlinear systems — a survey. IEE Proceedings D: Control Theory and Applications, 1980, 127(6): 272−285 oberg J, Zhang Q H, Ljung L, Benveniste A, Delyon 65 Sj¨ B, Glorennec P Y, Hjalmarsson H, Juditskys A. Nonlinear black-box modeling in system identification: a unified overview. Automatica, 1995, 31(12): 1691−1724 66 Verhaegen M, Westwick D. Identifying MIMO Wiener systems using subspace model identification methods. Signal Processing, 1996, 52(2): 235−258 67 Bai E W, Reyland J Jr. Towards identification of Wiener systems with the least amount of a priori information on the nonlinearity. Automatica, 2008, 44(4): 910−919 68 Bai E W. An optimal two-stage identification algorithm for Hammerstein-Wiener nonlinear system. Automatica, 1998, 34(3): 333−338 69 Ninness B, Gibson S. Quantifying the accuracy of Hammerstein model estimation. Automatica, 2002, 38(12): 2037− 2051 70 Zhu Y. Estimation of an N-L-N Hammerstein-Wiener model. Automatica, 2002, 38: 1607−1614 71 Bai E W, Li K. Convergence of the iterative algorithm for a general Hammerstein system identification. Automatica, 2010, 46(11): 1891−1896 72 Zhao W X, Chen H F. Adaptive tracking and recursive identification for Hammerstein systems. Automatica, 2009, 45: 2773−2783

942


73 Zhao W X, Chen H F. Identification of Wiener, Hammerstein, and NARX systems as Markov chains with improved estimates for their nonlinearities. Systems and Control Letters, 2012, 61(12): 1175−1186 74 Mu B Q, Chen H F. Recursive identification of WienerHammerstein systems. SIAM Journal on Control and Optimization, 2012, 50(5): 2621−2658 75 Roll J, Nazin A, Ljung L. Nonlinear system identification via direct weight optimization. Automatica, 2005, 41(3): 475− 490 76 Zhao Y L, Wang L Y, Yin G G, Zhang J F. Identification of Wiener systems with binary-valued output observations. Automatica, 2007, 43(10): 1752−1765 77 Wang L Y, Yin G G. Asymptotically efficient parameter estimation using quantized output observations. Automatica, 2007, 43(7): 1178−1191 78 Katayama T. Subspace Methods for System Identification. London: Springer, 2005 79 Van Overschee P, de Moor B. A unifying theorem for three subspace system identification algorithms. Automatica, 1995, 31(12): 1853−1864 80 Vidal R. Recursive identification of switched ARX systems. Automatica, 2008, 44(9): 2274−2287 81 Wang J, Chen T. Parameter estimation of periodically switched linear systems. IET Control Theory and Applications, 2012, 6(6): 768−775 82 Zames G, Lin L, Wang L Y. Fast identification n-widths and uncertainty principles for LTI and slowly varying systems. IEEE Transactions on Automatic Control, 1994, 39(9): 1827−1838 83 Tan P N, Steinbach M, Kumar V. Introduction to Data Mining. Boston, MA: Addison-Wesley, 2006 84 Sayeed A M. A signal modeling framework for integrated design of sensor networks. In: Proceedings of the 2003 IEEE Workshop Statistical Signal Processing. St. Louis, MO, USA: IEEE, 2003. 7 85 Vapnik V N. Statistical Learning Theory. New York: WileyInterscience, 1998 86 Vapnik V N. The Nature of Statistical Learning Theory (Information Science and Statistics for Engineering and Information Science) (2nd edition). New York: Springer Verlag, 1999 87 Vidyasagar M. Learning and Generalization: With Applications to Neural Networks (2nd edition). London: Springer, 2003 88 Kolmogorov A N. On some asymptotic characteristics of completely bounded spaces. Doklady Akademii Nauk SSSR, 1956, 108: 385−389 (in Russian) 89 Pinkus A. N-Widths in Approximation Theory. New York: Springer-Verlag, 1985 90 Traub J F, Wasilkowski G W, Wozniakowski H. Informationbased Complexity. New York: Academic Press, 1988 91 Wang L Y, Yin G G, Zhang J F, Zhao Y L. Space and time complexities and sensor threshold selection in quantized identification. Automatica, 2008, 44(12): 3014−3024 92 Lin L, Wang L Y, Zames G. Time complexity and model complexity of fast identification of continuous-time LTI systems. IEEE Transactions on Automatic Control, 1999, 44(10): 1814−1828 93 Poolla K, Tikku A. On the time complexity of worst-case system identification. IEEE Transactions on Automatic Control, 1994, 39(5): 944−950 94 Tse D C N, Dahleh M A, Tsitsiklis J N. Optimal asymptotic identification under bounded disturbances. IEEE Transactions on Automatic Control, 1993, 38(8): 1176−1190 95 Zames G. On the metric complexity of causal linear systems: ε-entropy and ε-dimension for continuous time. IEEE Transactions on Automatic Control, 1979, 24(2): 222−230

Vol. 39

96 Cramer H. Mathematical Methods of Statistics. Princeton NJ: Princeton University Press, 1946 97 Frieden B R. Science from Fisher Information: A Unification. Cambridge: Cambridge University Press, 2004 98 Hannan E J, Deistler M. The Statistical Theory of Linear Systems. New York: John Wiley and Sons, 1988 99 Cover T M, Thomas J A. Elements of Information Theory. New York: Wiley-Interscience, 1991 100 Gallager R G. Information Theory and Reliable Communication. New York: John Wiley and Sons, 1968 101 Gersho A, Gray R M. Vector Quantization and Signal Compression. Norwell, MA: Kluwer Academic Publishers, 1991 102 Sayood K. Introduction to Data Compression (2nd edition). San Francisco: Morgan Kaufmann, 2000 103 Davis M D, Sigal R, Weyuker E J. Computability, Complexity, and Languages (2nd edition). San Diego: Academic Press, 1994 104 Zurek W H. Complexity, Entropy, and the Physics of Information. Reading, MA: Addison-Wesley, 1990 105 Gevers M. Towards a joint design of identification and control? Essays on Control: Perspectives in the Theory and Its Applications. Boston, MA: Birkh¨ auser, 1993. 111−151 106 Ljung L. System Identification Toolbox — for Use with Matlab, User s Guide (5th edition). Natick, MA: The MathWorks, Inc. Sherborn, Mass, 2000 1−350 107 Garnier H, Gilson M, Laurain V. The CONTSID toolbox for Matlab: Extensions and latest developments. In: Proceedings of the 15th IFAC Symposium on System Identification. Saint-Malo, France, 2009. 735−740 108 Ljung L. Educational aspects of identification software user interfaces. In: Proceedings of the 13th IFAC Symposium on System Identification. Rotterdam, The Netherlands: IFAC, 2003. 1590−1594 109 Rivera D E, Lee H, Braun M W, Mittelmann H D. Plantfriendly system identification: a challenge for the process industries. In: Proceedings of the 13th IFAC Symposium on System Identification. Rotterdam, Netherlands: IFAC, 2003. 917−922

WANG Le-Yi Received his Ph. D. degree in electrical engineering from McGill University, Montreal, Canada, in 1990. Since 1990, he has been with Wayne State University, Detroit, Michigan, where he is currently a professor in the Department of Electrical and Computer Engineering. He is a fellow of IEEE. His research interest covers complexity and information, system identification, robust control, H∞ optimization, time-varying systems, adaptive systems, hybrid and nonlinear systems. Corresponding author of this paper. E-mail: [email protected] ZHAO Wen-Xiao Received the bachelor degree from Shandong University, China in 2003 and the Ph. D. degree in operation research and cybernetics from the Institute of Systems Science (ISS), Academy of Mathematics and Systems Science (AMSS), Chinese Academy of Sciences (CAS), China in 2008. After graduation, he worked as a postdoctoral fellow in the Department of Automation, Tsinghua University, and a visiting scholar at the School of Mathematics, University of Western Sydney, Australia. He is currently an associate professor with ISS, AMSS, CAS. His research interest covers identification, adaptive control, and systems biology. E-mail: [email protected]