Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations
Dr. Erik Nygren Research and Innovation Lab Swiss Federal Railway
Swiss Railway Network. A Complex Dynamical System. Facts
Influencing Factors
KM
1
3,230 km
10,000
Weather
People
Events
1t
1,210,000
210,000 t
Infrastructure
12,997
31,266
Energy
33,000
Most dense network © SBB • Solution Center Infrastructure • Research & Innovation • October 2017
2
Train Dispatching and Scheduling. Challenges in the Worlds Densest Train Network.
Timetable
Production
RCS
Train runs
© SBB • Solution Center Infrastructure • Research & Innovation • October 2017
3
Evolution of Dispatching. Towards Full Automation. Past
Today
Future
© SBB • Solution Center Infrastructure • Research & Innovation • October 2017
4
Automated Train Dispatching. Current Challenges.
Measurements
Action
Automated Dispatching
Big Data Learning
Big Data: Not enough relevant information
© SBB • Solution Center Infrastructure • Research & Innovation • October 2017
5
Reinforcement Learning for Railway Dispatching. Overcoming Data Sparseness through Simulations. Measurements
Action
Big Data
Learning
Learning
Validation
Artificial Data
Data generation
High Performance Simulation
Action
WIP © SBB • Solution Center Infrastructure • Research & Innovation • October 2017
6
High Performance Simulations. Unleashing the Power of Parallel Computing. DGX-1 High Performance Simulations
Time speedup
Influencing factor analysis
Scenario variations © SBB • Solution Center Infrastructure • Research & Innovation • October 2017
7
Preliminary Results. Visualization of Simulation Results.
2h realtime
500x
Visualization speed
5000x Simulation speed
© SBB • Solution Center Infrastructure • Research & Innovation • October 2017
8
Reinforcement Learning. Playing the Dispatcher Game.
DGX-1 Automated Dispatcher Reward Artificial Data
DGX-1 High Performance Simulations
Action
© SBB • Solution Center Infrastructure • Research & Innovation • October 2017
9
Machine Learning on Artificial Data. Generating, Evaluating and Optimizing Train Dispatching. Automated Dispatcher
Building Blocks
Variable Topologies
Reinforcement Learning
Tree Search
1
Genetic Algorithm
2 Mixed Integer Linear Programming
Evolutionary Strategies
3
© SBB • Solution Center Infrastructure • Research & Innovation • October 2017
10
Current State... And Future Expected Reward. DGX-1
Fully Automated Process
High Performance Simulations Timetable
Production
DGX-1 Automated Dispatcher Train runs
© SBB • Solution Center Infrastructure • Research & Innovation • October 2017
11
Take Home.
Big Data
Big Information
Take Home.
Big Data
Big Information AI
Model
Research Team Dr. Erik Nygren
[email protected] AI Researcher © SBB • Solution Center Infrastructure • Research & Innovation • October 2017
13
Reward Function. How to Reward an Artificial Dispatcher.
Reward
© SBB • Solution Center Infrastructure • Research & Innovation • October 2017
14
