Way…nding under Uncertainty in Continuous Time and Space by Dynamic Programming Dr. Serge P. Hoogendoorn (
[email protected]) Transportation and Tra¢c Engineering Section Faculty of Civil Engineering and Geosciences - Delft University of Technology PO Box 5046 - 2600 GA Delft - The Netherlands
1
Introduction
Route choice models generally describe how individuals choose between a discrete number of routes between their origin and destination. This also when the hypothesis of a …nite number of available routes is not justi…ed, e.g. pedestrians choosing their path through a walking facility, or optimal way…nding of Automated Guided Vehicles (AGV’s) on non-dedicated infrastructure (non-infrastructure bound). In these cases, the subjects can choose between an in…nite number of alternatives. Nevertheless, in most cases the route choice problem is solved by choosing a …nite number of routes through the facility, and considering the resulting discrete problem ([1], [3], [4]). Clearly, such approximations may not be best practise. This paper discusses subjective individual way…nding modelling in continuous time and space under uncertain conditions. Here, routes are described by continuous paths through the facility. The uncertainty generally pertains to the tra¢c conditions expected by the subject. To describe the choice behaviour, it is assumed that the paths are chosen that entail minimal subjective disutility. This disutility (or cost) can re‡ect among other things route travel time, cost of getting too close to obstacles and walls, number of sharp turns or rapid directional changes, expected number of ‘interactions’ with other subjects, stimulation of the environment, etc. It turns out that the resulting problem can be solved by application of the dynamic programming principle of Bellman [2], implying that to solve the route choice problem a second-order partial di¤erential equation needs to be solved. The paper describes both numerical solution approaches and applications of the approach to pedestrian way…nding.
2
Problem formulation
Let A denote the walking area, and let x 2 A denote the location of a subject (e.g. a pedestrian, or an AGV). To describe the route choice behaviour of the subject, we hypothesise that he uses an internal model to estimate and predict the experienced route costs. To this end, the subject will estimate the current position at instant t, re‡ected by x ^. Moreover, the participant will predict his future positions x(t) for t > t0 by using an internal prediction model dx = vdt + ¾dw subject to x(t) = x ^ (1) where v = v(¿ ) denotes velocity (i.e. speed and direction) of the subject for ¿ > t. In this formulation, w denotes a standard Wiener process, meaning that for very small time periods [t; t + h), the increase w(t + h) ¡ w(t) is a N(0; hIm )-distributed random variate, where Im denotes the m £ m identity matrix; ¾ = ¾(x; v) is a 2 £ m matrix, re‡ecting the way in which the white noise vector w a¤ects the location. Note that ¾ (w(t + h) ¡ w(t)) is N(0; h¾¾0 )distributed. The stochasticity re‡ects the uncertainty in the expected tra¢c conditions and
71
the resulting e¤ects on the subject’s kinematics. This uncertainty stems from among other things the lack of experience, and observability of prevailing tra¢c conditions. Equation (1) describes the predicted path x[t;T ) resulting from applying the velocities v(¿ ) during the period [t; T ). The terminal position x(T ) is a random variate, whose distribution depends on the applied velocities. Given each velocity path v[t;T ) and (estimated) initial position x(t) = x ^, we can determine the expected costs of applying the control as follows ·Z T ¸ J(t; x ^; v[t;T ) ) = E L(¿ ; x(¿ ); v(¿ ))d¿ + Á(T; x(T )) (2) t
where L denotes the so-called running costs and Á denotes the so-called terminal costs. The running cost L(¿ ; x(¿ ); v(¿ ))d¿ re‡ects the costs that are incurred during a very small time period [¿ ; ¿ + d¿ ), given that the subject is located at x(¿ ) and is applying velocity v(¿ ) to change the position. Typical examples are travel time, costs incurred by moving too close to obstacles, and costs due moving at a certain speed. The terminal costs Á(T; x(T )) re‡ect the cost due to ending up at position x(T ) at the terminal time T . These costs typically re‡ect the penalty that may be incurred when the subject does not arrive at the destination areas Di ½ A in time. Note that since the route costs are described by means of an integral (2), the theory assumes that the route costs are additive (i.e. the total route cost can be described by the sum of the costs of its constituent subroutes). We hypothesise that the subject aims to minimise the subjective expected route cost, i.e. ¤ we aim to determine the optimal velocity v[t;T ) satisfying ¤ v[t;T )
3
= arg min E
·Z
t
T
¸ L(¿ ; x(¿ ); v(¿ ))d¿ + Á(T; x(T ))
(3)
Solution approach
To solve the route choice problem, let us de…ne the so-called value function W (t; x ^) by the ¤ expected value of the costs upon applying the optimal velocity v[t;T ) W (t; x ^) := E
·Z
T
¤
¤
¤
L(¿ ; x (¿ ); v (¿ ))d¿ + Á(T; x (T ))
t
¸
(4)
subject to (1). Now consider a very short period [t; t + h) of length h. We have E
·Z
t+h
¸
L(¿ ; x(¿ ); v(¿ ))d¿ = L(t; x ^; v(t))h + O(h2 )
t
(5)
and thus W (t; x ^) = min
v[t;t+h)
©
ª L(t; x ^; v(t))h + E [W (t + h; x(t + h))] + O(h2 )
(6)
The random variable x(t + h) can approximate by
p x(t + h) = x ^ + hv(t) + ¾ hw + O(h3=2 )
(7)
where w is a N(0; Im ) distributed random variable; W (t + h; x(t + h)) is also a random variable. Some straightforward computations show that E [W (t + h; x(t + h))] = W (t + h; x ^ + hv(t)) +
hX @ 2 W (t; x ^) £ij (x; v) + O(h3=2 ) 2 @xi @xj ij
72 1
(8)
where £ = £(x; v) = ¾(x; v)¾(x; v)0 . Substitution (8) into (4), and subsequently taking the limit h ! 0 yields the following partial di¤erential equation 9 8 < X @W 1X @2W = @ vi + £ij (x; v) (9) ¡ W (t; x) = min L(t; x; v) + v : ; @t x 2 @x @x i i j i ij with terminal conditions
W (T; x(T )) =
½
Ái x(T ) 2 Di J0 elsewhere
(10)
Ái denotes the terminal cost of arriving at destination area Di , and J0 re‡ects the penalty of not arriving at either of the destination areas Di in time. The optimal velocity v¤ satis…es. 8 9 < 2 X @W X 1 @ W = vi (11) v¤ = arg min L(t; x; v) + + £ij (x; v) : xi 2 @xi @xj ; i
ij
Equation (9) is the dynamic programming equation for the continuous stochastic dynamic user-optimal (CSDUO) path choice problem.
4
Numerical solution approach
We can solve the dynamic programming equation (9) by discretising the area A into small ± £ ±-cells, and considering approximate solutions on this lattice - at …xed time instants tk = ht (i.e. ± is the spatial step size, and h is the temporal step size). We can show that the resulting problem is a Markov di¤usion process in two dimensions with nearest-neighbor transitions that are determined by the stochastic di¤erential equations (1) (see [2]). Solving this (discrete) stochastic dynamic programming problem is related to solving equation (9) by replacing the partial derivatives with the appropriate …nite di¤erence quotients. Let ei denote the unit vector in the i-th dimension (i = 1; 2). Then, we denote the forward …nite ¡ di¤erence (¢+ xi ) and backward …nite di¤erences (¢xi ) by ¡1 ¢§ [W (t; x § ±ei ) ¡ W (t; x)] for i = 1; 2 xi W := ±
(12)
For the second-order terms, we then use the following approximations ¢2xi W := ± ¡1 [W (t; x + ±ei ) ¡ 2W (t; x) + W (t; x ¡ ±ei )] f ori = 1; 2 ¢§ xi xj W
(13)
1 : = § ± ¡2 [W (t; x + (±ei § ±ej )) + 2W (t; x) + W (t; x ¡ (±ei § ±ej ))] (14) 2 1 ¨ ± ¡2 [W (t; x + ±ei ) + W (t; x + ±ej ) + W (t; x ¡ ±ei ) + W (t; x ¡ ±ej )] (15) 2
In numerically approximating (9), the following numerical solution approach is proposed [2]: 2 § W (t ¡ h; x) = W (t; x) ¡ hH(x; ¢§ xi W; ¢xi W; ¢xi xj )
(16)
where 2 § H(x; ¢§ xi W; ¢xi W; ¢xi xj )
(
:= min L(t; x; v) + v
X¡ ¢ ¡ ¡ vi+ ¢+ xi W ¡ vi ¢xi W
i 9 = ³ ´ X X 1 1 ¡ + ¡ + £ii (x; v)¢2xi W + £+ (x; v)¢ W ¡ £ (x; v)¢ W (17) xi xj xi xj ij ij ; 2 i 2 i6=j
with b+ := maxfb; 0g and b¡ := minfb; 0g.
73
5
Modelling pedestrian behaviour
The previous sections discussed the CSDUO problem formulation and numerical solution approaches. This section discusses application of the approach to modelling pedestrian choice behaviour. To this end, we will assume m = 2, ¾ = ´(x)I2 , where ´(x) is a scalar function of the position x. An important factor determining pedestrian route choice are the present obstacles. In this example, obstacle l is described by an area Ol ½ A. Hoogendoorn and Bovy [5] hypothesise that the running cost L can be expressed by P a weighted sum, i.e. L(x; v) = ci Li (x; v). For illustration purposes, we consider the components Li : 1. Travel time L1 (x; v) = 1: P 2. Discomfort due to walking too close to obstacles L2 = l Al e¡kx¡Ol k=Bl , where parameters Al and Bl determine the way in which walking too close to obstacle l is valued (e.g. depending on the surface of the obstacle); kx ¡ Ol k denotes the minimal Euclidean distance from x to the area Ol . 3. Energy consumption L3 =
1 2
hv; vi.
Using equation (11), it can be shown easily that the optimal velocity v¤ equals v¤ = ¡
1 Dx W (t; x) c3
(18)
The partial derivative Dx W (t; x) can be interpreted as the marginal cost of x. Equation (18) shows that the optimal velocity v¤ is pointed into the direction e¤ = v¤ = kv ¤ k in which the optimal costs W reduce most rapidly. The optimal speed kv ¤ k depends on the rate kDx W (t; x)k at which the value function decreases in this direction. When this rate is high, the optimal speed will be equally high, and vice-versa. The dynamic programming equation (9) for the pedestrian path…nding problem equals: µ ¶ X X @ 2W @ 1 X @W 2 1 2 ¡kx¡Ol k=Bl ¡ W (t; x) = c1 + c2 Al e ¡ + ´ (x) @t 2c3 i @xi 2 @x2i i
(19)
l
Let us now illustrate the workings of the approach by an example. The case considered pertains to Schiphol Plaza, which is a multi-purpose multi-modal transfer station. Figure 1 depicts a snapshot of the microscopic simulation model NOMAD developed at Delft University of Technology. Let us consider a pedestrian that is located anywhere in Schiphol Plaza and aims to walk to either of the escalators E6 or E7. To describe the route choice behaviour of the pedestrian, the infrastructure was divided into square cells of 0:25m2 , and the numerical solution approach described in section 4 was applied. By doing so, we were able to study the e¤ect of uncertainty on the path choice behaviour of the pedestrian. For this particular example, we have assumed that the level-of-uncertainty is constant for all x, ´(x) = ´0 . Figures 2-a and 2-b respectively show the value function approximation for ´0 = 0:01 and ´ 0 = 0:25. The …gures clearly show the di¤erences in W due to the varying uncertainty levels. Generally, uncertainty causes smoothing of the value function. As a result, the value function has a large value nearby obstacles and walls. This is caused by the fact that a pedestrian in not sure whether he will end up very near an obstacle in the near future, and thus incur a very high cost. As a result, we can observe that the value function in very narrow passageways tends to increase very quickly.
74
V1 E1
E6
E2
E7
E3 E4 V2
E5
Figure 1: Snapshot from the Schiphol Plaza case using the NOMAD microsimulation model. Exits E1-E5 indicate exits from Schiphol Plaza; escalators E6 and E7 indicate exits to train platform. V1 and V2 depict the locations of the newspaper vendors. The colors re‡ect pedestrians having distinct activity schedules.
56
48
44
36
56
4448 56
8
4
40
36
32
20
40 x1-axis (m)
60
52
32
24 28
444 582
36 40
52
0 0
20
40 44
a) η = 0.01
283 2
36 40
28
20
8 12
16
8
4
20
24
48
20
12
20
16
8
4
12
656 0
36 40 48
28
458 2
44
40
32 4 2 28 24
16 12
12
8
56 60
20
16
12
28
28
16 20
56
36
32
20
40
48 52
6864
28
24
44
60
48
44
52
40 32
60
x2-axis (m)
56
36
44
40
b) η = 0.25 80
0
20
56
40 x1-axis (m)
60
80
Figure 2: Value functions W (t; x) for pedestrians leaving Schiphol Plaza via escalators for a) ´ 0 = 0:01 and b) ´0 = 0:25. Optimal paths are perpendicular to iso-value function curves.
75
Furthermore, we have sketched an optimal path for each situation to illustrate how path choice can be e¤ected by high levels of uncertainty. These paths show that when the uncertainty is high, the pedestrian will be less inclined to use paths that traverse through narrow passageways, due to the higher probability of having to walk very near or even collide with an obstacle (or otherwise experiencing high delays). Let us …nally remark that in [5], the combined path choice, destination / activity area choice, and activity scheduling problem is discussed; Hoogendoorn and Bovy [5] also show how changing tra¢c conditions can be included in the modelling approach.
6
Conclusions and future research
This paper describes an approach to model individual route choice behaviour in continuous time and space under uncertainty. Numerical solution approaches were proposed to approximate the dynamic programming equation. The approach is applicable to predict pathchoice behaviour in infrastructure facilities of realistic size, as was shown by application of the approach to pedestrian behaviour modelling in Schiphol Plaza. This example was used to illustrate the e¤ect of uncertainty on pedestrian choice behaviour. One of the main observations is that when uncertainty increases, pedestrians are tend to prefer routes given them wider berth. This can be explained by observing that when the conditions become uncertain, the likelihood of experiencing high costs on a narrow (i.e. risky) passageway increases. The paper focusses on route choice behaviour and uncertainty pertaining to tra¢c conditions expected by the pedestrian. Other types of uncertainty, for instance in the terminal cost Ái of arriving at a certain destination area (re‡ecting for instance service time, etc.) have not been considered. Future research will be directed towards extending the modelling approach to include other types of uncertainty. Acknowledgement 1 This research is funded by the Social Science Research Council (MaGW) of the Netherlands Organization for Scienti…c Research (NWO).
References [1] DAAMEN, W., P.H.L. BOVY, AND S.P. HOOGENDOORN (2001). Modelling Pedestrians in Transfer Stations. In: Pedestrian and Evacutation Dynamics, Springer, 59-74. [2] FLEMING, W.H., AND H.M. SONER (1993). Controlled Markov Processes and Viscosity Solutions. Applications of Mathematics 25. Spriner-Verlag. [3] GIPPS, P.G. (1986) Simulation of Pedestrian Tra¢c in Buildings. Schriftenreihe des Instituts fuer Verkehrswesen 35, University of Karlsruhe. [4] HAMACHER, H.W., AND S.A. TJANDRA (2001). Mathematical Modelling of Evacuation Problems: A State of the Art. In: Pedestrian and Evacutation Dynamics, Springer, 59-74. [5] HOOGENDOORN, S.P., AND P.H.L. BOVY (2002). Pedestrian Travel Behavior in Walking Areas by Subjective Utility Optimization. Transportation Research Board Annual Meeting 2002, Washington; paper nr. 02-2667
76
