Multi-Objective Traffic Light Control System based on Bayesian Probability Interpretation Mohamed A. Khamis*, Walid Gomaa*, and Hisham El-Shishiny‡ * Department of Computer Science and Engineering, Egypt-Japan University of Science and Technology (E-JUST),
[email protected];
[email protected] ‡ IBM Center for Advanced Studies in Cairo,
[email protected] Presented to: ITSC 2012 Anchorage, AK, USA; 16 Sep - 19 Sep 2012 Presented by: Prof. Hesham Rakha, Professor, CEE Dept. Director, Center for Sustainable Mobility Virginia Tech Transportation Institute ITSC 2012, Anchorage, AK, USA; 16 Sep - 19 Sep 2012
[email protected]
1
Motivation • Traditional traffic light control systems cause congestions and considerable time losses. • In addition, there is a need to a multi-objective traffic light control system that is adaptive to the high dynamics in traffic networks.
ITSC 2012, Anchorage, AK, USA; 16 Sep - 19 Sep 2012
[email protected]
2
Contributions 1. Applying the Intelligent Driver Model (IDM) (M. Treiber et al. 2002) time-continuous acceleration model on a reinforcement learning traffic light control system. 2. Extending a single-objective traffic light controller (M. Wiering 2000) to a multi-objective traffic light controller adaptive to the different road conditions. 3. Using the Bayesian probability interpretation that allows the multi-objective traffic light controller to respond effectively to the non-stationarity in the road network. 4. Checking the traffic light controller against varying driver behavior and traffic demand caused by adverse weather conditions. 5. Adding a set of new performance indices based on collaborative learning to the Green Light District (GLD) traffic simulator (M. Wiering et al. 2004) for better performance evaluation. ITSC 2012, Anchorage, AK, USA; 16 Sep - 19 Sep 2012
[email protected]
3
RL for Traffic Light Control • We adopt a vehicle-based RL traffic light control model (M. Wiering 2000). • A predictor is used to estimate the waiting time of each vehicle until it reaches its destination when the traffic light is red or green. • All vehicles predictions are then combined for the traffic light controller agent decision. • Using a trial-and-error process, the agent will learn a policy that optimizes the cumulative reward gained over time (i.e., minimize the total estimated waiting time until the vehicle reaches its destination). ITSC 2012, Anchorage, AK, USA; 16 Sep - 19 Sep 2012
[email protected]
4
Multi-Objective vs. Single-Objective Traffic Light Control • In (M. Wieing 2000), the sole reward function equals one if a vehicle waits at its position, otherwise equals zero, i.e., – minimize the expected trip waiting time.
• The proposed multi-objective function includes weighted summed reward functions that: – maximize the flow rate, – minimize the expected trip waiting time, – minimize the expected trip time, – minimize the expected junction waiting time, and – maximize the vehicles safety by taking decisions that minimize the vehicles speed. ITSC 2012, Anchorage, AK, USA; 16 Sep - 19 Sep 2012
[email protected]
5
Impact of Adverse Weather on Traffic Conditions
• We added in GLD the weather impacts on the IDM acceleration model parameters that comply with (R. Hoogendoorn et al. 2010). • The added weather conditions include; – light rain, normal rain, heavy rain, light fog, heavy fog, and sandstorm with 5%, 10%, 12%, 25%, 30%, 36% speed reduction than dry weather, respectively.
• Adverse weather condition decreases traffic intensity while high temperatures increase traffic intensity (Cools et al. 2010). • We simulate these two impacts by low and high vehicle generation frequencies, respectively. ITSC 2012, Anchorage, AK, USA; 16 Sep - 19 Sep 2012
[email protected]
6
Experimental Work
Traffic network with 2 edge nodes, 10 traffic light nodes, and 2 nodes without traffic lights. ITSC 2012, Anchorage, AK, USA; 16 Sep - 19 Sep 2012
[email protected]
7
Performance Evaluation
ITSC 2012, Anchorage, AK, USA; 16 Sep - 19 Sep 2012
[email protected]
8
Conclusion and Future Work • The multi-objective controller significantly outperforms the single-objective controller under congested periods and adverse weather conditions. • Future work: – Make the weights of the multi-objective controller be function in the labeled roads (e.g., residential areas, schools, main streets). – Study the proposed traffic light controller role in minimizing gas emissions produced by vehicles.
ITSC 2012, Anchorage, AK, USA; 16 Sep - 19 Sep 2012
[email protected]
9
Thanks! • Questions: –
[email protected]
• EJUST website: – http://www.ejust.edu.eg
• Personal website: – https://sites.google.com/site/mohamedabdelazizcv/ ITSC 2012, Anchorage, AK, USA; 16 Sep - 19 Sep 2012
[email protected]
10
