Reinforcement Learning for Railway Scheduling

5 downloads 0 Views 857KB Size Report
SBB • Solution Center Infrastructure • Research & Innovation • October 2017. 2. Swiss Railway Network. A Complex Dynamical System. Influencing Factors.
Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations

Dr. Erik Nygren Research and Innovation Lab Swiss Federal Railway

Swiss Railway Network. A Complex Dynamical System. Facts

Influencing Factors

KM

1

3,230 km

10,000

Weather

People

Events

1t

1,210,000

210,000 t

Infrastructure

12,997

31,266

Energy

33,000

Most dense network © SBB • Solution Center Infrastructure • Research & Innovation • October 2017

2

Train Dispatching and Scheduling. Challenges in the Worlds Densest Train Network.

Timetable

Production

RCS

Train runs

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017

3

Evolution of Dispatching. Towards Full Automation. Past

Today

Future

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017

4

Automated Train Dispatching. Current Challenges.

Measurements

Action

Automated Dispatching

Big Data Learning

Big Data: Not enough relevant information

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017

5

Reinforcement Learning for Railway Dispatching. Overcoming Data Sparseness through Simulations. Measurements

Action

Big Data

Learning

Learning

Validation

Artificial Data

Data generation

High Performance Simulation

Action

WIP © SBB • Solution Center Infrastructure • Research & Innovation • October 2017

6

High Performance Simulations. Unleashing the Power of Parallel Computing. DGX-1 High Performance Simulations

Time speedup

Influencing factor analysis

Scenario variations © SBB • Solution Center Infrastructure • Research & Innovation • October 2017

7

Preliminary Results. Visualization of Simulation Results.

2h realtime

500x

Visualization speed

5000x Simulation speed

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017

8

Reinforcement Learning. Playing the Dispatcher Game.

DGX-1 Automated Dispatcher Reward Artificial Data

DGX-1 High Performance Simulations

Action

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017

9

Machine Learning on Artificial Data. Generating, Evaluating and Optimizing Train Dispatching. Automated Dispatcher

Building Blocks

Variable Topologies

Reinforcement Learning

Tree Search

1

Genetic Algorithm

2 Mixed Integer Linear Programming

Evolutionary Strategies

3

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017

10

Current State... And Future Expected Reward. DGX-1

Fully Automated Process

High Performance Simulations Timetable

Production

DGX-1 Automated Dispatcher Train runs

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017

11

Take Home.

Big Data

Big Information

Take Home.

Big Data

Big Information AI

Model

Research Team Dr. Erik Nygren [email protected] AI Researcher © SBB • Solution Center Infrastructure • Research & Innovation • October 2017

13

Reward Function. How to Reward an Artificial Dispatcher.

Reward

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017

14