Computational Intelligence in Urban Traffic Signal Control - IEEE Xplore

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 42, NO. 4, JULY 2012

485

Computational Intelligence in Urban Traffic Signal Control: A Survey Dongbin Zhao, Senior Member, IEEE, Yujie Dai, and Zhen Zhang

Abstract—Urban transportation system is a large complex nonlinear system. It consists of surface-way networks, freeway networks, and ramps with a mixed traffic flow of vehicles, bicycles, and pedestrians. Traffic congestions occur frequently, which affect daily life and pose all kinds of problems and challenges. Alleviation of traffic congestions not only improves travel safety and efficiencies but also reduces environmental pollution. Among all the solutions, traffic signal control (TSC) is commonly thought as the most important and effective method. TSC algorithms have evolved quickly, especially over the past several decades. As a result, several TSC systems have been widely implemented in the world, making TSC a major component of intelligent transportation system (ITS). In TSC and ITS, many new technologies can be adopted. Computational intelligence (CI), which mainly includes artificial neural networks, fuzzy systems, and evolutionary computation algorithms, brings flexibility, autonomy, and robustness to overcome nonlinearity and randomness of traffic systems. This paper surveys some commonly used CI paradigms, analyzes their applications in TSC systems for urban surface-way and freeway networks, and introduces current and potential issues of control and management of recurrent and nonrecurrent congestions in traffic networks, in order to provide valuable references for further research and development. Index Terms—Computational intelligence (CI), freeway network, surface-way network, traffic congestions, traffic signal control (TSC).

I. INTRODUCTION ODAY, it is estimated that there are over 600 million passenger cars in the world with the number increases roughly 50 million per year, even though it slows down because of the world financial crisis. We are enjoying conveniences of automobiles, while “enjoying” their byproducts: traffic congestions, environmental pollution and so on. There are many factors that are responsible for the traffic problems, such as unreasonable traffic infrastructures and planning, weak public awareness of traffic, etc. However, the major factor is that existing urban traffic signal control (TSC) systems do not adequately fulfill an optimal traffic control and management role. According to

T

Manuscript received December 22, 2010; revised March 25, 2011 and June 20, 2011; accepted June 30, 2011. Date of publication August 8, 2011; date of current version June 13, 2012. This work is supported in part by National Natural Science Foundation of China under Grant 60874043, Grant 60921061, and Grant 61034002. The paper was recommended by Associate Editor F. Chu. The authors are with the State Key Laboratory of Intelligent Control and Management of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Haidian, Beijing 100190, China (e-mail: [email protected]; [email protected]; [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TSMCC.2011.2161577

statistics, at Phoenix, AZ, the implementation of an advanced TSC system on a road reduced traffic collisions by 6.7%, vehicle travel time by 11.4%, delay by 24.9%, parking by 27%, and energy consumption significantly, highlighting the importance of an optimal TSC.

A. Traffic Signal Control TSC is commonly thought as the most important and effective traffic-control method for safe and expeditious travel. Since the introduction of simple automatic traffic signal controllers in US at the beginning of the last century, TSC systems have undergone continuous improvements. The theoretical research work of TSC algorithms could date back to the mid-20th century. From then on, TSC methods have gone through three stages: pretimed control, traffic-responsive control, and intelligent control. Because of advanced sensing and communication technologies, real-time traffic measurements have become commonly available. Both research work and advanced technologies build the basis for modern real-time TSC. Fixed-time control method, using a predetermined cycle and split (green duration as a portion of the cycle time) time plan, is suitable for relatively stable and regular traffic flows. Webster [1] and Miller [2] established a traffic signal timing model and calculation method to get minimal average delay of vehicles, building the foundation for modern fixed-time TSC. In order to optimize the early pretimed signal control, some offline software were developed to calculate optimal signal settings for a single intersection or a network. Traffic network study tool (TRANSYT) [3] is probably the best known example. It can compile a series of fixed-time signal plans for different hours of a day or for special recurring traffic conditions. Because the traffic system is a dynamic system, any predefined traffic signal plan cannot fit real traffic conditions well. Real-time traffic-responsive control came into practice with the help of sensing technologies. The idea is that any trafficcontrol action is made under a certain control strategy according to real-time traffic data. Actuated control is a well-developed and most widely used traffic-responsive method, regulating traffic signal timings according to real-time sensing of traffic flows by detectors that are located in the network. It selects phases and extends phase durations based on traffic demands and predefined logic rules. This method is suitable for the situations with traffic saturation less than 80% and traffic randomness relatively high. The extension of current phase only depends on vehicles of the current phase, without taking into account queues of other phases. Therefore, it cannot achieve the optimal usage of resources: time and space of the whole intersection.

1094-6977/$26.00 © 2011 IEEE

486

Fig. 1.


Hierarchical system structure of the PtMS.

Based on fixed-time traffic-responsive control methods, many TSC systems have been developed and widely applied in the world. Sydney coordinated adaptive traffic system (SCATS) [4] is a plan-selection control system, in which time plan for each intersection is chosen by the overall need of subsystems. Split cycle offset optimization technique (SCOOT) [5] is an adaptive real-time plan-generating control system, which makes realtime adjustments of splits, cycles, and time offset parameters with a small-step incremental optimization approach. It is often considered to be the traffic-responsive version of TRANSYT. Since the system adopts a centralized control structure, it is difficult to control traffic in a large region [6]. Italy’s urban traffic optimization by integrated automation (UTOPIA)/system for priority and optimization of traffic (SPOT) [7] is a distributed real-time traffic-control system, especially suited for countries with advanced public transportation systems, such as Italy, Norway, Netherlands, Sweden, Finland, and Denmark. Japan’s universal traffic management system (UTMS) [8] is based on the existing traffic-control system with more intelligence. Through two-way communications with equipments on vehicles, the system acquires and analyzes traffic information to control traffic signals. USA’s real-time hierarchical optimized distributed effective system (RHODES) [9] is a real-time hierarchical optimal distributed system. Field tests that are performed at Arizona cities showed that the system was more efficient under semicongested transportation situations. A newly emerged traffic management system is China’s parallel transportation management systems (PtMS) [10] with its architecture that is shown in Fig. 1. This system employs artificial system, computational experiment, parallel execution (ACP) mechanism for traffic control and management, and incorporates emerging technologies in cloud computing, social computing, and cyber–physical–social systems [11], [12]. There are three main targets: agent-based distributed and adaptive platforms for transportation systems

(aDAPTS), dynamic traffic assignment based on complex adaptive systems (DynaCAS), and operator training systems for transportation (OTSt). There is also an artificial system that is parallel to the real system. Analysis and evaluations can be easily conducted on the artificial system. Control and management actions can be taken through simultaneous executions of the real and the artificial system. Traffic-control systems are affected by many factors: traffic infrastructures, vehicles, travelers, weather, etc. Each factor has its own characteristics, which make the whole traffic systems large complex nonlinear stochastic systems and pose many problems and challenges for researchers and engineers. Even though traditional control theories have been applied for decades, they are still not satisfactory. Besides, traffic-control systems involve human behaviors. Therefore, human reactions to the systems and feelings of the systems should also be taken into account. As a background of the proposition of intelligent transportation system (ITS), TSC systems should behave in a “smart” way. Where does the intelligence come from? One major attempt is the introduction of computational intelligence (CI). B. Reasons of Computational Intelligence for Traffic Signal Control Since CI emerged several decades ago, it has become a hot research field very quickly. It is claimed as the successor of artificial intelligence (AI) and a way for future computing [14]. CI methodologies facilitate problem solving that was previously difficult or impossible. The common idea of CI is to simulate the intelligence of nature to some extent by the usage of certain computational methods, which include artificial neural networks (ANNs), fuzzy systems, and evolutionary computation (EC) algorithms. Each method has been well developed with many branches. In addition, CI also employs techniques, such as swarm intelligence (SI), artificial immune systems, reinforcement learning (RL), etc. There are several reasons why it is easy to apply CI methodologies in TSC systems. First, as mentioned earlier, TSC systems are large complex nonlinear stochastic systems. Therefore, it is hard to find optimal traffic signal settings. CI provides a feasible way to obtain optimal or suboptimal solutions. Second, most traffic signal optimal methods are based on precise mathematical traffic models. However, it is very hard to model dynamic traffic precisely. Most CI methodologies do not require precise models. Sometimes, no model is even needed. Third, as a broader definition [15], CI is a study of adaptive mechanisms to enable or facilitate intelligent behaviors in complicated, uncertain, and changing environments. CI methodologies can be adapted to dynamic traffic systems. TSC actions can be taken based on real-time traffic conditions and historical reasoning. Researchers have conducted a lot of work for applications of CI in the field of TSC. In this paper, we will survey applications of some commonly used CI paradigms, discuss their implementations in TSC systems, analyze research and development of TSC methods in surface-way and freeway networks, and introduce current and potential issues of nonrecurrent traffic-congestion management.

ZHAO et al.: COMPUTATIONAL INTELLIGENCE IN URBAN TRAFFIC SIGNAL CONTROL: A SURVEY

Fig. 2.

Illustration of surface-way TSC.

The rest of this paper is organized as follows. Sections II and III introduce applications of CI to TSC to solve recurrent traffic congestion in surface-way and freeway networks, respectively. Section IV briefly surveys research work in nonrecurrent trafficcongestion management. Section V concludes this paper. II. COMPUTATIONAL INTELLIGENCE FOR TRAFFIC SIGNAL CONTROL IN SURFACE NETWORK In recent years, researchers have conducted both theoretical and practical research work on TSC methods. A lot of work is expected to become the core methods in next generation of traffic-control systems. Generally speaking, traditional signalcontrol strategies based on mathematical traffic-flow models provide many useful ideas and new methods for traffic-control applications, but their calculations are often very complicate, hard to meet real-time requirements. The assumptions of mathematical models of traffic flow are strict, losing the generality of TSC algorithms. To solve the problem, some intelligent control methods have been introduced into TSC. Technologies in CI, such as ANNs, fuzzy systems, and EC algorithms, have been widely used. Some researchers also used CI technologies, such as SI, RL, etc. We will discuss applications of these technologies in detail. In this section, we mainly focus on their applications in surface networks, as shown in Fig. 2. At the beginning of each subsection, a brief summary of the corresponding technology will lay a common background for discussions. As intelligent technologies are easy to work together, sometimes, several intelligent technologies are integrated together to tackle complicate traffic problems. A. Fuzzy System Fuzzy describes imprecision or uncertainty. In contrast with “crisp logic” in which there are only two possible values, i.e., true or false, “fuzzy logic” reasons approximately or in a certain degree of true or false.Fuzzy theory emerged from the development of fuzzy sets that are first proposed by Zadeh [16]. A fuzzy system is a classical CI technology using fuzzy theory to solve problems in many fields. It has been applied successfully to control systems, vehicles, traffic signal, lifts, home appliances, etc. [15]. The fuzzy system might be the first successful attempt to implement CI in TSC. Pappis et al. [17] designed a fuzzy con-

487

troller for a single intersection of two phases. Trabia et al. [18] proposed a two-stage fuzzy controller. Lee et al. [19] extended this approach and designed a more complicate fuzzy control method to adjust phase sequence and splits into coordinates between intersections. The proposed method showed good performances in all simulation cases. Compared with the actuated method, improvements were from 3.5% to 8.4% under steady traffic conditions and from 4.3% to 13.5% under time-varying conditions [19]. While considering features of lanes and junctions, Chou and Teng [20] designed a fuzzy controller with only nine fuzzy rules for easy implementations in multiple junctions or lanes. Murat and Ergun [21] built a multiphased vehicle-actuated signal controller containing one fuzzy logic signal-time controller and one phase sequencer for isolated junctions. Simulations demonstrated that a fuzzy logic controller for a three-phased intersection improved performance by nearly 25% compared with a traffic-actuated controller under variable and high traffic volumes. Qiao et al. [22] proposed a two-stage fuzzy logic control method for an isolated signalized intersection, where both traffic efficiency and fairness were considered simultaneously. Simulation demonstrated that it outperformed the vehicle-actuated control in all performance indices. Gokulan and Srinivasan [23] adopted type-2 fuzzy set and designed a distributed multiagent traffic-responsive signal-control system. The system was tested on virtual road networks with several scenarios. Results showed superior performance of the approach in handling unplanned and planned incidents and obstructions. The fuzzy system is a vague reasoning system, so it is easier to be applied in many fields. A priori expert knowledge of objects can be easily reflected in well-designed fuzzy rules, so it does not need mathematical models of the objects making it simpler and more intuitive to construct a fuzzy controller. In addition, the implementation cost of fuzzy controllers is also low, so fuzzy systems have attracted increased attention from control engineers and traffic engineers. With the development of the CI technologies, it is often used in cooperation with other CI technologies because of its handy design and integration of expert knowledge. B. Artificial Neural Network The ANN imitates the function of biological neurons of brain and connections between them. The ANN simulates the way in which brain processes data. By the update of the weights of neurons, the ANN learns and memorizes training data, discovers patterns or features. The ANN is a prosperous CI technology. Different kinds of networks have been developed, such as singlelayer networks, multilayer networks, self-organizing networks, and recurrent networks. The ANN has also been successfully applied in many fields: pattern classifications, robotics, predictions, etc. ANN-based control is one of the most effective methods for uncertain, nonlinear, and time-varying systems [24]. Applications of the ANN in TSC are also fruitful. Spall and Chin [25] used an ANN controller in system-wide traffic-adaptive control (S-TRAC) to produce optimal instantaneous (minute-to-minute) signal timings, while automatically adapting to long-term (month-to-month) system changes. As a

488


learning feature of the ANN, it is often used with fuzzy logic, which represents human knowledge, to form a new type CI called neuro-fuzzy or fuzzy NNs. Shen and Kong [26] proposed a traffic coordination control technique with bus priorities in which a back propagation NN was adopted to implement fuzzy control rules. Simulations showed that the method performed better in all test cases. Even in rush-hour conditions, improvements were still from 14.79% to 18.11% in average delay time (ADT) and from 11.79% to 14.21% in average travel time (ATT), compared with the isolated fixed-time control method. Buses showed improvements from 30.76% to 45.01% in the ADT and from 16.40% to 17.17% in the ATT [26]. A similar technique was also adopted by Choy and Srinivasan [27], [28] in their local real-time signal controller that was capable of continuous online learning. Bingham [29] designed a two-phased single intersection fuzzy controller that is formulated by a NN and constructed an additional critic NN to optimize the controller, providing an early basic idea of RL in TSC. C. Evolutionary Computation and Swarm Intelligence EC and SI are inspired by nature. EC algorithms mainly include genetic algorithms (GA), genetic programming, evolutionary programming, differential evolution, cultural evolution and so on. They imitate natural processes, such as natural evolution under the principle of survival of the fittest. The origin of SI technology is from the collective behaviors of biological species, such as flocks of birds, ants, bees, etc. [15]. Both EC and SI use the heuristic searching mechanism to find optimal or near-optimal solutions, simplifying nonlinear programming problems. They are applied successfully in the optimization for many kinds of problems. Ceylan et al. [30] used the GA to optimize time parameters for junctions. Jan-Dirk et al. [33] converted the membership function of a fuzzy logic controller into chromosomes and used the GA to find optimal parameters. With the help of the GA, Ghassan and Rahim [34] presented a procedure to optimally design and evaluate different traffic-queue management and TSC strategies in oversaturated conditions. Srinivasan and Choy [28] also used the GA to find optimal parameters for their hybrid fuzzy NN-based multi-agent system. Wei et al. [31] used the particle swarm optimization (PSO) algorithm to optimize fuzzy rules for signal controllers. Zhang et al. [32] proposed a real-time online intelligent urban TSC approach, which employed a multiobjective discrete differential evolution (MDDE) technology to optimize configurations of traffic signal phases and cycles. Simulation experiments showed that this approach had better real-time overall performances, effectively alleviating urban traffic pressures and reducing waiting time of vehicles. D. Reinforcement Learning and Adaptive Dynamic Programming Calculation of EC or SI sometimes is time consuming. Therefore, optimal results only apply to offline plan-selection systems. For online optimization of TSC, RL is more often adopted. For instance, RL was involved in the learning process of a hybrid

Fig. 3.

Scheme of ADHDP.

fuzzy NN-based multiagent system in [28]. RL is a machinelearning method that tries to figure out better actions through interactions with environment [35]. It usually consists of following elements: state, action, and reward. An RL agent takes an action according to different states; then, the environment gives a reward with which the agent learns to improve its policies. States and actions can be described in discrete or continuous forms. Temporal difference (TD) learning and Q-learning methods are extensively explored discrete methods. The Q-learning method was used in optimal signal control for a single intersection [36], [37] or a region [38]. However, when used in optimal control of multiphase variable traffic-flow problems, the state– action pair value matrix of discrete Q-learning requires huge storage space. This is the so-called “dimension disaster,” restricting the application of this method to problems with large state or action spaces. Adaptive dynamic programming (ADP) is considered as a continuous form of RL. As shown in Fig. 3, ADP uses a critic module and an action module, which are usually represented by NNs, to mimic traditional value approximation functions and controllers, respectively. ADP has solved a key issue “curse of dimensionality,” in dynamic programming by function approximation. As a result, it has received increased attention from researchers in control and traffic fields [40]. According to evaluations of critic module’s output, ADP can be divided into three categories [41]: heuristic dynamic programming (HDP), dual heuristic programming (DHP), and globalized DHP (GDHP). If the output of the action module is also the input to the critic module, action-dependent forms are created, which are called ADHDP, ADDHP, and ADGDHP. Fig. 3 is a diagram of ADHDP. The critic module and the action module are not restricted to be represented by the ANN, but by any other linear or nonlinear approximator’s forms. Cai et al. [42] developed an ADP- and RL-based controller, which was capable of reducing vehicle delays that are substantially compared with the best fixed-time control, while being computationally efficient. Li et al. constructed an ADP traffic-signal controller for intersections, in which a neuro-fuzzy system served as the action network and a three-layer ANN acted as the critic network [43].


Fig. 4.

489

Network architecture of the aDAPTS [10].

E. Agent and Game Agent paradigm is rapidly emerging as one of the powerful technologies for the development of large-scale distributed systems. Many agent approaches have been applied to traffic systems [44]. Agent is more a technology assembly than a technology. It is a design and application architecture, with almost any technology usable under this architecture. Agent-based approaches can solve large-scaled traffic coordination control problems that traditional traffic control can hardly handle. Wiering et al. [38], [39] used a model-based RL method in a multiagent-based TSC. To save computing resources, each vehicle agent calculated its own value function, and the intersection agent only managed its local state–action pairs. Ferreira et al. [45] constructed a distributed multiagent control strategy for surface-way signal control. Each intersection was managed by an agent, exchanging traffic information, coordinating with other agents, and making its own decisions. Choy et al. [46] constructed a hierarchical and distributed multiagent traffic-control system by the use of a complex hybrid evolutionary fuzzy NN. Each agent makes its own real-time decisions based on reasoning abilities of the fuzzy system and learning abilities of the NN. In addition, the perception process of agents can be adjusted by online RL. Choy [27], [28] used a similar multiagent technology to achieve an overall better performance in simulations of real data in Singapore for three scenarios compared with the existing control algorithm. Roozemond et al. [47] designed a traffic prediction and control agent for intersections. Kosonen [48] proposed a multiagent-based fuzzy control method. Bazzan et al. [49] divided a large group of agents in a network into small groups. Each agent evolved individually and another “tutor” agent of every group supervised actions of its members. This is a potential way to reduce computational demands that are caused by viewing these agents as a combination, as well as a new approach to apply RL in multiagent systems. The agent idea was fully adopted and extended by the aDAPTS of PtMS [10], [13]. Agents hosting traditional traffic facilities, such as traffic centers, roadside controllers, sensing devices, and in-

formation systems, were used to fulfill their corresponding tasks or functions. It provides supporting and operating environments to design, construct, manage, and maintain these autonomous agents. To integrate and coordinate objectives and activities of agents in the network, it was built in a hierarchical architecture as shown in Fig. 4. Game theory has been widely recognized as an important tool in many fields. It has contributed essentially to multiagent RL. It is a study of multiple interacting agents trying to maximize their rewards and a theory of learning in games [50]. Alvarez et al. [51] proposed a noncooperative game approach to solve congestions at intersections. Two one-way streets of an intersection were taken as two players in a noncooperative game, where each player tried to minimize its queue, so as to find —Nash’s equilibrium. Cheng et al. [52] developed a parallel coordinated approach that is called CoSIGN, which stands for coordinated signals. An extended iterative process sampled fictitious play (SFP) was used to compute the Nash equilibria for the game, where each decision maker was viewed as a player who controlled a time period for a signal. A simulation was conducted for the City of Troy, MI, where there were 75 signalized intersections, and results demonstrated that the algorithm was robustly scalable to realistic-size networks. Bazzan [53] used an evolutionary game theory to develop a distributed approach for the coordination of TSC agents. Each agent played a two-person game against each member of its neighborhood, searching the Nash equilibrium under the evolutionary stable strategy. Agents did not need to know the strategy of their opponents, so the communication burden in the network was rather low, making this approach more reliable. From the introduction of CI technologies, we can see that each technology has its own advantages and disadvantages. It is hard to solve traffic problems just by one of them. One good feature of CI technologies is that they can easily be combined together or combined with traditional approaches in many ways. Hybrid of the CI technologies is promising to accomplish the optimal traffic control and management tasks. Based on the trend, the multiagent system, fuzzy control, and RL technologies are

490


Fig. 5. Urban freeway stretch with multiple on-ramps, off-ramps, and corridors.

mature enough to be integrated for intelligent optimal controller implementation in real urban TSC systems. III. COMPUTATIONAL INTELLIGENCE FOR TRAFFIC SIGNAL CONTROL IN FREEWAY NETWORK Modern urban traffic-control system is not only just signal control of intersections, but also signal control of a large traffic networks, which includes surface ways, freeways, and ramps. Urban freeway systems fulfill medium and large transportation tasks in big cities and provide rapid transit service. Like surfaceway networks, traffic congestions may happen occasionally in the freeway network mainlines, on-ramps, or off-ramps. A typical freeway with multiple on-ramps and off-ramps can be seen in Fig. 5. In this section, we will introduce the application of CI technologies to urban freeway signal control. Here, we would like to classify freeway traffic-control problems according to road structures, such as ramp metering and integrated corridor control, etc. [54], to introduce CI applications, respectively. A. Ramp Metering The basic ramp metering mechanism is to keep freeway mainline flow smoothly by the regulation of traffic signal at on-ramps or off-ramps. According to statistics [55], more than 20 cities across USA implemented the ramp metering algorithm. In Los Angeles, CA, there are 1382 ramps under control. Ramp metering applications showed that this method significantly reduced travel time, accidents, and fuel consumption. In a singleramp metering mode, each on-ramp is controlled independently. Commonly used ramp metering methods include traffic demandcapacity control and occupancy-rate control. ALINEA [56], i.e., a feedback control method based on the PID control scheme of linearized on-ramp model, shows good performances. However, traffic conditions always deviate from the stable linear equilibrium point, where the ALINEA method is fitted. The influence of on-ramp metering is inherently a nonlinear process, where the following CI technologies can be used instead: nonlinear gain integral-based NN controller or proportional-integralbased NN controller [57], fuzzy controller [58], genetic-fuzzy controller [59], or ADP controller [60]. Adjustments of freeway traffic flows by ramp metering have correlated effects, making it a coupling system. Therefore, to make use of resources more efficiently, it is necessary to coordinate related ramps. In linear control, Wattleworth [61] presented a classical static timing control strategy. Papageorgiou [62] improved the method with a dynamic traffic-flow model. Yang [63] further extended this method by a successive optimization model so that it could adapt to dynamic traffic conditions. Papageorgiou [64] extended the single-ramp metering ALINEA algo-

rithm to a multiramp metering METALINE strategy. Kotsialos et al. [65] proposed a classical nonlinear optimal coordination ramp metering strategy. It was based on a dynamic freeway traffic model, searching optimal values in a feasible descent direction. The optimized performance index was the combination of important factors, such as total time spent, queue length at on-ramps, and smooth variations of controlled variables. Test simulations carried out on A10 ring road of Amsterdam, the Netherlands, showed that the total time that vehicles spent was reduced by 43.5% compared with the condition without control. Recently, Jacob [66] proposed a coordinated ramp metering algorithm based on RL and verified its effectiveness via simulations. However, this method was constrained by defects of RL to some extent, such as slow learning rate and limited discrete points. Chen et al. [67] designed an algorithm by the use of a radial basis function-support vector machine (RBF-SVM) to optimize the coordination of mainlines and ramps. Simulations showed that the RBF-SVM controller performed more steadily compared with three other methodologies including ALINEA. Xu et al. [60] proposed an ADP-based coordinated multiramp metering strategy to overcome shortcomings of the discrete RL. Furthermore, Bai et al. [68], [69] proposed a multiramp metering strategy based on a distributed structure, promoting applications of this method to freeway on-ramps. B. Corridor Control Integrated freeway corridor control refers to comprehensive utilizations and coordination of ramp metering, guidance, and surrounding surface-way network signal control to achieve integrated signal control of freeway mainlines, side roads, and urban surface ways as seen in Fig. 5. Papageorgiou [70] extended the idea to store and forward to simplify complex issues to linear optimization of multiple control measures, which include ramp metering, TSC at intersection, urban freeway mainline control, variable message signs (VMS), route guidance, and so on. Then, Diakaki [71] proposed an IN-TUC strategy that considers both surface ways and freeways in urban traffic-control systems. The algorithm of ramp metering was ALINEA, and the algorithm of guidance was based on a user optimal dynamic feedback algorithm (assuming all drivers obey guidance), and the TSC was an improved pretimed control. Simulations showed that when only ramp metering was used, the travel time was reduced by 15%; when ramp metering and guidance were used, the travel time was reduced by 19%; and when ramp metering, guidance, and improved TSC were applied, simultaneously, the travel time was reduced by 26%.Field applications of this strategy in Glasgow, Scotland, also achieved good results. Wu et al. [72] divided urban freeways, ramps, surface ways into subsections, assuming that each section had a uniform density of vehicles. Then, the evolution of traffic could be expressed by subsections. Tian et al. [73] studied typical diamond joint areas of surface ways and ramps with a microsimulation software, i.e., VisSim, and demonstrated that integrated adaptive control improved traffic situations not only at the surface-way intersections but also on urban freeways. Kwon [74] proposed an adaptive control strategy that integrates ramp metering and intersection control.


The ramp metering module adjusted its control strategy according to congestions at adjacent surface intersections, and surface intersection module that determined signal phases according to traffic conditions around ramps. Berg [75] established control model for on-ramp, off-ramp, and surface traffic signals by adopting a micro traffic-flow Metanet model of freeways and an improved Kashani model of surface ways. Traditional corridor control often uses more than one agent, which will lead to less coordination when there is not enough or effective communication between them. Jacob et al. [76] constructed a corridor control agent that simultaneously controls freeway on-ramps and artery VMS. By the usage of the Q-learning algorithm of RL, a corridor control agent learned how to react to states of the traffic network built in microscopic simulation software that simulated an actual corridor in Toronto, Canada. Actions of the agent would divert traffic by VMS or ramp metering. Various traffic incident cases were conducted and analyzed, and the results were promising. Till now, there are very few studies on the integrated freeway corridor control. Moreover, corridor structures are still under development and vary with the rapid growth of the traffic flow. An example is the emergence of traffic signal controllers on the side road before intersecting with freeway off-ramp. Therefore, integrated optimal control strategies for freeways and surrounding surface-way networks still needs more study. IV. MANAGEMENT OF NONRECURRENT TRAFFIC CONGESTION Traffic congestions occur normally when road capacities are less than traffic demand and are usually classified as recurrent and nonrecurrent. Recurrent congestion have some regularity in time and place, e.g., congestions occur during rush hours or at bottlenecks of roads. Nonrecurrent congestions are usually uncertain in time and place, which are caused by traffic incidents, such as vehicle breakdown, bad weather (rain, snow, ice, and fog), collapses of bridges or roads, goods scattering, and so on. Management of nonrecurrent traffic congestions usually means to transfer traffic flow into diversion routes via guidance information, to increase the capacity of diversion routes by the adjustment of their signal time plan, and to inform drivers how to drive on diversion routes and where and how to return to original routes. It can be imagined that a lot of work and trivial management are needed. At present, nonrecurrent congestions are mainly managed by people. Therefore, efficiencies vary from person to person and performances are not quite satisfactory. Thus, it is urgent to develop management and decision-making theories and support methods for nonrecurrent congestions. When an incident occurred, DynaMIT [77] collected relevant information and offered guidance for drivers based on a user optimal strategy to reduce congestion time and economic losses to the maximum extent. For incidents in arteries, Sheu [78] proposed a stochastic optimal method, established a real-time dynamic discrete nonlinear model of the road states to control variables under traffic congestions, and developed a real-time dynamic prediction control algorithm. To alleviate highway nonrecurrent congestion, Sheu et al. [79] developed a real-time stochastic optimal control method based on local slope control.

491

There are also some attempts with CI technologies in nonrecurrent traffic congestion management. Xie et al. [80] developed a traffic-incident-detection algorithm based on fuzzy technologies. Simulations showed that the algorithm had lower false alarm rate, higher detection rate, and less mean time compared with other commonly used algorithms. Hu et al. [81] introduced fuzzy goals and fuzzy constraints into a multiobjective function based on the satisfactory control theory to obtain an optimal traffic diversion plan. PSO technology was used to solve this complex multiobjective problem. Hamza-Lup et al. [82] proposed a new smart traffic evacuation management system (STEMS) for the ITS. This system was used for real-time evacuations when natural disasters or other catastrophic incidents happened. When human-caused threats and disasters happened, it responded rapidly by automatically creating dynamic evacuation plans and controlling a set of traffic evacuation signals dynamically to reroute traffic effectively. Zhou [83] used two computer simulation packages, i.e., SYNCHRO and CORSIM, to model several incidents on I-75 and alternative routes in Sarasota County, FL. He also presented a methodology to estimate effects of incident management and signal-timing modifications. Even though there is little research on nonrecurrent congestions, it has gained much attention from relevant researchers recently. Causes and influences of nonrecurrent congestions should be detected rapidly, for prompt and smart decision making to coordinate TSC, route guidance, etc., to alleviate their influences to the maximum extent. In all aspects, it can be expected that CI technologies will play an important role. V. DISCUSSIONS AND CONCLUSION In this paper, we have comprehensively analyzed applications of CI methodologies in TSC for urban surface-way networks and freeway networks, introduced current and potential issues of recurrent and nonrecurrent traffic congestion management, summarized development status, and pointed out trends. Although TSC is not a new problem, traffic congestions have become increasingly serious as the population and vehicles increase continuously, posing many new challenges. In future, the development and applications of more effective TSC methods for urban surface-way networks and integrated corridor freeway networks would be a challenge to traffic managers, scientists, and engineers. As seen in this paper, CI methodologies have been widely adopted by researchers in TSC to solve various problems in this field. Since they show some good features in their applications compared with traditional methods, it becomes apparent that CI technologies are effective solutions for TSC problems. On the other hand, there are also many questions left (even raised by itself). There are few standards for this newly rising field in many aspects. For specified formulations of CI methodologies, there is no criterion to determine which technology is more suitable or how to apply these methodologies in the field of TSC. These problems are usually left for researchers, which means that people will make choices depending on their own fields, views, and experiences. Some practical problems, such as tedious parameter settings, may keep researchers away from

492


these fields. Besides, effects and results of CI applications still need more implementation studies to further support their methods. Furthermore, combinations of urban TSC, advanced public transportation system (APTS), and advanced vehicle control system (AVCS) are more in-depth topics. Even though the ITS gives a promising perspective for traffic science and engineering, it still needs a large amounts of research and technologies to pave its way. New technologies, such as wireless communications, lead to a prosperity of several projects under the ITS framework. V2X, which is a collective name for vehicle to vehicle (V2V) and vehicle to infrastructure (V2I), is a new technology that is designed for vehicles and infrastructures to communicate with each other [84]. A new market is gradually forming worldwide. By design, vehicles and infrastructures can “talk” to each other. Boundaries between traditional TSC concepts become vague due to these technologies. For example, urban TSC seems more like a combination and no longer need to be separated into surface and freeway networks since they are originally the same by design.Some specific problems, such as “blue corridors” for emergency vehicles, also find new ways to be solved. Meanwhile, more research work is needed to build the basis for these topics. It can be safely said that CI technology will play an active role in future ITS development. ACKNOWLEDGMENT The authors would like to thank the associate editor and anonymous reviewers for their comments and suggestions and would also like to thank G. Liu for help with a draft of the paper. REFERENCES [1] F. V. Webster, “Traffic signal setting,” Road Res. Lab., HMSO, London, U.K., Tech. Paper 39, pp. 1–44, 1958. [2] A. J. Miller, “Settings for fixed-cycle traffic signals,” Oper. Res. Q., vol. 14, no. 4, pp. 373–386, 1963. [3] (Dec. 2010). [Online]. Available: http://www.trlsoftware.co.uk/ [4] (Dec. 2010). [Online]. Available: http://www.scoot.co.uk/ [5] (Dec. 2010). [Online]. Available: http://www.scats.com.au/ [6] H. P. Lu, R. M. Li, and Y. Zhu, An Introduction to Intelligent Transportation Systems. Beijing, China: China Railway Publishing House, 2004. [7] (Dec. 2010). [Online]. Available: http://www.peektraffic.nl/page/484 [8] (Dec. 2010). [Online]. Available: http://www.utms.or.jp/english/index. html [9] P. Mirchandani and F.-Y Wang, “RHODES to intelligent transportation systems,” IEEE Intell. Syst., vol. 20, no. 1, pp. 10–15, Jan./Feb. 2005. [10] F.-Y. Wang, “Parallel control and management for intelligent transportation systems: Concepts, architectures and applications,” IEEE Trans. Intell. Transp. Syst., vol. 11, no. 3, pp. 630–638, Sep. 2010. [11] F.-Y. Wang, “Toward a paradigm shift in social computing: The ACP approach,” IEEE Intell. Syst., vol. 22, no. 5, pp. 65–67, Sep./Oct. 2007. [12] F.-Y. Wang, “Toward a revolution in transportation operations: AI for complex systems,” IEEE Intell. Syst., vol. 23, no. 6, pp. 8–13, Nov./Dec. 2008. [13] F.-Y. Wang, “Agent-based control for networked traffic management systems,” IEEE Intell. Syst., vol. 20, no. 5, pp. 92–96, Sep./Oct. 2005. [14] G. K. Venayagamoorthy, “A successful interdisciplinary course on computational intelligence,” IEEE Comput. Intell. Mag., vol. 4, no. 1, pp. 14–23, Feb. 2009. [15] A. Engelbrecht, Computational Intelligence: An Introduction, 2nd ed. New York: Wiley, 2007. [16] L. A. Zadeh, “Fuzzy sets,” Inf. Control, vol. 8, no. 3, pp. 338–353, 1965. [17] C. Pappis and E. Mamdani, “A fuzzy logic controller for a traffic junction,” IEEE Trans. Syst., Man Cybern., vol. 7, no. 10, pp. 707–717, Oct. 1977.

[18] M. Trabia, M. Kaseko, and A. Murali, “A two-stage fuzzy logic controller for traffic signals,” Transp. Res. C, Emerg. Technol., vol. 7, no. 6, pp. 353– 367, 1999. [19] J. H. Lee and K. H. Lee, “Distributed and cooperative fuzzy controllers for traffic intersection group,” IEEE Trans. Syst., Man Cybern. C, Appl. Rev., vol. 29, no. 2, pp. 263–271, May 1999. [20] C. H. Chou and J. C. Teng, ”A fuzzy logic controller for traffic junction signals,” Inf. Sci., vol. 143, nos. 1–4, pp. 73–97, 2002. [21] Y. S. Murat and E. Gedizlioglu, “A fuzzy logic multi-phased signal control model for isolated junctions,” Transp. Res. C, Emerg. Technol., vol. 13, no. 1, pp. 19–36, 2005. [22] J. Qiao, N. D. Yang, and J. Gao, “Two-stage fuzzy logic controller for signalized intersection,” IEEE Trans. Syst., Man Cybern. A, Syst. Humans, vol. 41, no. 1, pp. 178–184, Jan. 2011. [23] B. P. Gokulan and D. Srinivasan, “Distributed geometric fuzzy multiagent urban traffic signal control,” IEEE Trans. Intell. Transp. Syst., vol. 11, no. 3, pp. 714–727, Sep. 2010. [24] F. N. Chowdhury, P. Wahi, R. Raina, and S. Kaminedi, “A survey of neural networks applications in automatic control,” in Proc. 33rd Southeastern Symp. Syst. Theory, 2001, pp. 349–353. [25] J. C. Spall and D. C. Chin, “Traffic-responsive signal timing for systemwide traffic control,” Amer. Control Conf., vol. 4, pp. 2462–2463, 1997. [26] G. J. Shen and X. J. Kong, “Study on road network traffic coordination control technique with bus priority,” IEEE Trans. Syst., Man Cybern. C, Appl. Rev., vol. 39, no. 3, pp. 343–351, May 2009. [27] M. C. Choy, D. Srinivasan, and R. L. Cheu, “Neural networks for continuous online learning and control,”IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1511–1531, Nov. 2006. [28] D. Srinivasan, M. C. Choy and R. L. Cheu, “Neural networks for real-time traffic control system,” IEEE Trans. Intell. Transp. Syst., vol. 7, no. 3, pp. 261–272, Sep. 2006. [29] E. Bingham, “Reinforcement learning in neurofuzzy traffic control system,” Eur. J. Oper. Res., vol. 131, pp. 232–241, 2001. [30] H. Ceylan and M. G. H. Bell, “Traffic signal timing optimisation based on genetic algorithm approach, including drivers’ routing,” Transp. Res. B, Methodol., vol. 38, no. 4, pp. 329–342, 2004. [31] Y. Wei, Q. Shao, Y. Hang, and B. Q. Fan, “Intersection signal control approach based on pso and simulation,” in Proc. 2nd Int. Conf. Genet. Evol. Comput., 2008, pp. 277–280. [32] M. M. Zhang, S. G. Zhao, J. Lv, and Y. D. Qian, “Multi-phase urban traffic signal real-time control with multi-objective discrete differential evolution,” in Proc. Int. Conf. Electron. Comput. Technol., 2009, pp. 296– 300. [33] J. D. Schmocker, S. Ahuja, and M. G. H. Bell, “Multi-objective signal control of urban junctions: Framework and a London case study,” Transp. Res. C, Emerg. Technol., vol. 16, no. 4, pp. 454–470, 2008. [34] G. Abu-Lebdeh and R. F. Benekohal, “Design and evaluation of dynamic traffic management strategies for congested conditions,” Transp. Res. A, Policy Practice, vol. 37, no. 2, pp. 109–127, 2003. [35] R. Sutton and A. Barto, Reinforcement Learning: An Introductio. (Bradford Book Series). Cambridge MA: MIT Press, 1998. [36] X. J. Cheng, Q. S. Chang, and Z. X. Yang, “A traffic control system method based on Q-Learning,” Syst. Eng. Theory Practice, no. 8, pp. 136–140, 2006. [37] Y. P. Yang and H. T. Ou, “Self-organized control of traffic signals based on reinforcement learning and genetic algorithm,” Acta Autom. Sin., vol. 28, no. 4, pp. 564–568, 2002. [38] M. Wiering, J. Vreeken, J. Veenen, and A. Koopman, “Simulation and optimization of traffic in a city,” in Proc. IEEE Intell. Veh. Symp., 2004, pp. 453–458. [39] M. Wiering, “Multi-agent reinforcement learning for traffic light control,” in Proc. 17th Int. Conf. Mach. Learning, 2000, pp. 1151–1158. [40] T. Li, D. B. Zhao, and J. Q. Yi, “Adaptive dynamic programming for multi-intersections traffic signal intelligent control,” in Proc. 11th IEEE Int. Conf. Intell. Transp. Syst., Beijing, China, Oct. 12–15, 2008, pp. 286– 291. [41] J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, “Adaptive dynamic programming,” IEEE Trans. Syst., Man Cybern. C, Appl. Rev., vol. 32, no. 2, pp. 140–152, May 2002. [42] C. Cai, C. K. Wong, and B. G. Heydecker, “Adaptive traffic control system using approximate dynamic programming,” Transp. Res. C, Emerg. Technol., vol. 17, no. 5, pp. 456–474, 2009. [43] T. Li, D. B. Zhao, and J. Q. Yi, “Adaptive dynamic neurofuzzy for traffic control system,” in Proc. IEEE Int. Joint Conf. Neural Netw., 2008, pp. 1841–1847.


[44] B. Chen and H. H. Cheng, “A review of the applications of agent technology in traffic and transportation systems,” IEEE Trans. Intell. Transp. Syst., vol. 11, no. 2, pp. 485–497, Jun. 2010. [45] E. D. Ferreira and P. K. Khosla, “Multi-agent cooperation using distributed value functions,” in Proc. IEEE Intell. Veh. Symp., 2000, pp. 404–409. [46] M. C. Choy and D. Srinivasan, R. L. Cheu, “Cooperative, hybrid agent architecture for real-time traffic control system,” IEEE Trans. Syst. Man Cybern. A, Syst. Humans, vol. 33, no. 5, pp. 597–607, Sep. 2003. [47] D. A. Roozemond, “Using intelligent agents for pro-active real-time urban intersection control,” Eur. J. Oper. Res., vol. 131, pp. 293–301, 2001. [48] I. Kosonen, “Multi-agent fuzzy signal control based on real-time simulation,” Transp. Res. C, Emerg. Technol., vol. 11, no. 5, pp. 389–403, 2003. [49] A. L. C. Bazzan, D. de Oliveira, and B. C. da Silva, “Learning in groups of traffic signals,” Eng. Appl. Artif. Intell., vol. 23, no. 4, pp. 560–568, 2010. [50] L. Busoniu, R. Babuka, and B. De Schutter, “A comprehensive survey of multiagent reinforcement learning,” IEEE Trans. Syst. Man Cybern. C, Appl. Rev., vol. 38, no. 2, pp. 156–172, Feb. 2008. [51] I. Alvarez, A. Poznyak, and A. Malo, “Urban traffic control problem via a game theory application,” in Proc. 46th IEEE Conf. Decision Control, 2007, vol. 1–14, pp. 4979–4983. [52] S. F. Cheng, M. A. Epelman, and R. L. Smith, “CoSIGN: A parallel algorithm for coordinated traffic control system,” IEEE Trans. Intell. Transp. Syst., vol. 7, no. 4, pp. 551–564, Dec. 2006. [53] A. L. C. Bazzan, “A distributed approach for coordination of traffic signal agents,” Auton. Agents Multi-Agent Syst., vol. 10, no. 2, pp. 131–164, 2005. [54] D. W. Chen, L. X. Li, X. M. Liu, and F. -Y. Wan, “An overview on urban freeway traffic control and its prospect,” Inf. Control, vol. 31, no. 4, pp. 341–345, 2002. [55] P. Gary, and J. Robinson, “Ramp metering status in North America (1995 update),” US Dept. Transp., Washington, DC, Rep. DOT-T-95-17, 1995. [56] M. Papageorgiou, H. Hadj-salem, and J. M. Blosseville, “ALINEA: A local feedback control law for on-ramp metering,” Transp. Res. Record, pp. 58–64, 1999. [57] H. M. Zhang and S .G. Ritchie, “Freeway ramp metering using artificial neural network,” Transp. Res. C, Emerg. Technol., vol. 5, no. 5, pp. 273– 286, 1997. [58] L. Chen and A. May, “Freeway ramp control using fuzzy set theory for inexact reasoning,” Transp. Res. A, Policy Practice, vol. 24, no. 1, pp. 15– 25, 1990. [59] A. H. Ghods, A. R. Kian, and M. Tabibi, “Adaptive Freeway Rapm Metering and Variable Speed Limit Control: A Genetic-Fuzzy Approach,” IEEE Intell. Transp. Syst. Mag., vol. 1, no. 1, pp. 27–36, Sep. 2009. [60] J. Xu, W. S. Yu, and F.-Y. Wang, “Ramp metering based on adaptive critic designs,” in Proc. IEEE Int. Conf. Intell. Transp. Syst., Toronto, Canada, 2006, pp. 1531–1536. [61] J. A. Wattleworth, “Peak-period analysis and control of a freeway system,” Highway Res. Record, vol. 157, pp. 1–21, 1965. [62] M. Papageorgiou, “A new approach to time-of-day control based on a dynamic freeway traffic model,” Transp. Res. B, Method., vol. 14, pp. 349– 360, 1980. [63] X. G. Yang, “An Approach to dynamic traffic control system for urban expressway with constraint of queue length of on-ramp off-ramp,” J. Xi’an Highway Univ., vol. 19, no. 2, pp. 20–26, 1999. [64] M. Papageorgiou, J. M. Blosseville, and H. S. Habib, “Modeling and realtime control of traffic flow on the southern part of Boulevard Peripherique in Paris—Part II: Coordinated on-ramp metering,” Transp. Res. A, Policy Practice, vol. 24, pp. 361–370, 1990. [65] A. Kotsialos, M. Papageorgiou, M. Mangeas, and H. Salem, “Coordinated and integrated control of motorway networks via non-linear optimal control,” Transp. Res. C, Emerg. Technol., vol. 10, pp. 65–84, 2002. [66] C. Jacob and B. Abdulhai, “Automated adaptive traffic corridor control using reinforcement learning: Approach and case studies,” in Proc. 85th Annu. Meeting Transp. Res. Board, 2006. [67] F. Chen, Y. Jia, J. Li, and G. Ao, “SVM based intelligent traffic control and simulation for urban expressway and periphery freeway,” in Proc. Int. Conf. Artif. Intell. Educ., 2010, pp. 573–578. [68] X. R. Bai, D. B. Zhao, and J. Q. Yi, “The application of ADHDP(λ) method to coordinated multiple ramps metering,” Int. J. Innovat. Comput. Inf. Control, vol. 5, no. 10B, pp. 3471–3481, 2009. [69] D. B. Zhao, X. R. Bai, Fei-yue Wang, J. Xu, W. S. Yu. “DHP for coordinated freeway ramp metering,” IEEE Trans. Intell. Transp. Syst., to be published.

493

[70] M. Papageorgiou, “An integrated control approach for traffic corridors,” Transp. Res. C, Emerg. Technol., vol. 3, pp. 19–30, 1995. [71] C. Diakaki, “Integrated control of traffic flow in corridor networks,” Ph.D. Dissertation, the Dept. Prod. Eng. Manage., Technical University of Crete, Chania, Greece, 1999. [72] J. F. Wu and G. L. Chang, “An integrated optimal control and algorithm for commuting corridors,” Int. Trans. Oper. Res., vol. 6, pp. 39–55, 1999. [73] Z. Z. Tian, K. Balke, and R. Engelbrecht, “Integrated control strategies for surface streets and freeway systems,” in Proc. 81th Annu. Meeting Transp. Res. Board, 2002, pp. 92–99. [74] E. Kwon, R. P. Ambadipudi, and J. Bieniek, “Adaptive coordination of ramp meter and intersection signal for optimal management of freeway corridor,” in Proc. 82th Annu. Meeting Transp. Res. Board, 2003. [75] M. V. D. Berg, B. D. Schutter, and A. Hegyi, “Model predictive control for mixed urban and freeway networks,” in Proc. 83th Annu. Meeting Transp. Res. Board, 2004. [76] C. Jacob and B. Abdulhai, “Machine learning for multi-jurisdictional optimal traffic corridor control,” Transp. Res. A, Pol. Practice, vol. 44, no. 2, pp. 53–64, 2010. [77] H. Lieu, “A roadmap for the research, development and deployment of traffic estimation and prediction systems for real-time and off-line applications (TrEPS, TrEPS-P),” Travel Manage. Team Office Operations RD&T Federal Highway Administration, Report, 2003. [78] J. B. Sheu, “Stochastic system modeling and optimal control of incidentinduced traffic congestion,” Math. Comput. Model., vol. 38, no. 5, pp. 533–549, 2003. [79] J. B. Sheu, Y. H. Chou, and C. J. Yang, “A stochastic optimal control based approach to real-time incident-responsive local ramp control,” in Proc. Int. IEEE Conf. Intell. Transp. Syst., 2000, pp. 232–237. [80] B. L. Xie, Z. Hu, and H. W. Ma, “Fuzzy-logic-based traffic incident detection algorithm for freeway,” in Proc. Int. Conf. Mach. Learn. Cybern., 2008, pp. 1254–1259. [81] X. J. Hu, W. Wang, and J. Lu, “A method of traffic dispersion for accident management,” in Proc. Int. Conf. Meas. Technol. Mechatron. Autom., 2009, pp. 427–430. [82] G. L. Hamza-Lup, K. A. Hua, M. Le, and R. Peng, “Dynamic plan generation and real-time management techniques for traffic evacuation,” IEEE Trans. Intell. Transp. Syst., vol. 9, no. 4, pp. 615–624, Dec. 2008. [83] H. Zhou, “Evaluation of route diversion strategies using computer simulation,” J. Transp. Syst. Eng. Inf. Technol., vol. 8, no. 1, pp. 61–67, 2008. [84] (Mar. 2011). [Online]. Available: http://www.isuppli.com/Abstract/ P13176_20101021151042.pdf

Dongbin Zhao (M’06–SM’10) received the B.S., M.S., and Ph.D. degrees in material processing engineering all from Harbin Institute of Technology, Harbin, China, in August 1994, August 1996, and April 2000, respectively. He was a Postdoctoral Fellow with Tsinghua University, Beijing, China, from May 2000 to January 2002. He is currently an Associate Professor with the State Key Laboratory of Intelligent Control and Management of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China. He has published one book and more than 30 international journal papers. His current research interests includes the area of computational intelligence, adaptive dynamic programming, robotics, intelligent transportation systems, and process simulation. Dr. Zhao has been a Senior Member of Chinese Mechanical Engineering Society since 2006. He served as a member in many international program committees, and the Finance Chair of 2011 International Symposium on Neural Network. He worked as a Guest Editor for several international journals. He received the Second Award for Scientific Progress of National Defense from the Commission of Science technology and industry for National Defense of China in 1999, the First Award for Scientific Progress of Chinese Universities, Ministry of Education of China in 2001, the Third Award for Scientific and Technology Progress from China Petroleum and Chemical Industry Association in 2009, and the First Award for Scientific and Technology Progress from China Petroleum and Chemical Industry Association in 2010.

494


Yujie Dai received the B.S. degree in Tianjin University, Tianjin, China in 2006. Currently, he is working toward the Ph.D. degree at the State Key Laboratory of Intelligent Control and Management of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China. His main research interests include computational intelligence, adaptive dynamic programming, reinforcement learning, traffic signal control, and intelligent transportation systems.

Zhen Zhang received the B.S. degree from China University of Petroleum, Dongying, China, in 2006, and M.S. degree in Control Theory and Control Engineering in Dalian University of Technology, Dalian, China, in 2009. Since 2010 he has been working toward the Ph.D. degree at the State Key Laboratory of Intelligent Control and Management of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China. His main research interests include multiagent systems, reinforcement learning, neural networks, and traffic signal control.