Pedestrian recognition using automotive radar sensors - Advances in ...

Adv. Radio Sci., 10, 45–55, 2012 www.adv-radio-sci.net/10/45/2012/ Manuscript prepared for Adv. Radio Sci. doi:10.5194/ars-10-45-2012 with version 4.2 of the LATEX class copernicus.cls. © Author(s) 2012.2012 CC Attribution 3.0 License. Date: 9 May

Advances in Radio Science

Pedestrian recognition using automotive radar sensors Pedestrian recognition using automotive radar sensors

A. Bartsch, F. Fitzek, and R. H. Rasshofer

BMW Group Research and Technology, Munich, Germany A. Bartsch, F. Fitzek, and R. H. Rasshofer Correspondence A. Bartsch ([email protected]) BMW Groupto: Research and Technology, Munich, Germany Abstract. The The application of modern series production au-auAbstract. application of modern series production tomotive radar sensors to pedestrian recognition is an importomotive radar sensors to pedestrian recognition is an important topic in research on future driver assistance systems. The tant topic in research on future driver assistance systems. The aim of paper is toisunderstand thethe potential andand limits aimthis of this paper to understand potential limits of5 such sensors in pedestrian recognition. This knowledge of such sensors in pedestrian recognition. This knowledge be used to develop generation radar sensors with couldcould be used to develop nextnext generation radar sensors with improved pedestrian recognition capabilities.A A newraw raw improved pedestrian recognition capabilities. new signal processing algorithm is proposed that radarradar data data signal processing algorithm is proposed that al-alinsights the object classification process. The lows lows deepdeep insights intointo the object classification process. The 10 impact of raw radar data properties can be directly observed impact of raw radar data properties can be directly observed in every of classification the classification system avoiding main every layerlayer of the system by by avoiding machine learning and tracking. This gives information on chine learning and tracking. This gives information on thethe limiting factors of raw radar in terms classification limiting factors of raw radar datadata in terms of of classification decision making. To accomplish the very challenging disdecision making. To accomplish the very challenging dis15 tinction between pedestrians and static objects, five signifitinction between pedestrians and static objects, five significant and stable object features from the spatial distribution cant and stable object features from the spatial distribution and Doppler information are found. Experimental results and Doppler areautomotive found. Experimental results with data information from a 77 GHz radar sensor show that with over data 95 from a 77 GHz automotive radar sensor show thatop% of pedestrians can be classified correctly under over 95 % of pedestrians can isbecompareable classified correctly under op20 timal conditions, which to modern machine timallearning conditions, which is compareable to modern machine systems. The impact of the pedestrian’s direction learning systems. occlusion, The impact of thebeam pedestrian’s of movement, antenna elevationdirection angle, linof movement, beam elevation angle, linear vehicleocclusion, movement,antenna and other factors are investigated and ear vehicle movement, and show other that factors arereal investigated and discussed. The results under life conditions, discussed. Thebased results show that under realis life conditions, 25 radar only pedestrian recognition limited due to inradarsufficient only based pedestrian recognition is limited dueastowell in-as Doppler frequency and spatial resolution antenna side lobe effects. and spatial resolution as well as sufficient Doppler frequency antenna side lobe effects.

Fig. 1. A common recognition system with no backtracking possiFig. 1. is A shown common recognition system no recognition backtracking possibility on the left. On the right, with the new method bility is shown on the left. On the right, the new recognition method with possible backtracking of classification errors is shown. with possible backtracking of classification errors is shown.

other systems like video cameras (Wenger, 2007). Moreover, If high ACCresolution systems radar and pedestrian could sensors are recognition available in systems many modern the same sensor, hardware for (ACC) producing these 35 use vehicles as aradar part of Adaptive Cruisecosts Control systems. systems kept to minimum. recognition In this paper, the potenIf ACCare systems anda pedestrian systems could tial a same modern series production radar sensor, useofthe radar sensor, hardware automotive costs for producing these designed for pedestrian recognition is exsystems for areACC kept systems, to a minimum. In this paper, the potential of In a modern series automotive radarregarding sensor, plored. particular, theproduction limits of the radar sensor 40 decision designedmaking for ACC forclassification pedestrian recognition is exin systems, pedestrian are investigated plored. In particular, the limits of the radar sensor regarding to see what future developments of automotive radar sensors decision making in pedestrian classification are investigated are necessary to improve pedestrian recognition systems. 1 Introduction to see what future developments of automotive radar sensors A radar based pedestrian recognition system consists of 1 Introduction are necessary to improve pedestrian recognition systems. 30 The use of radar sensors in automotive pedestrian recognition two main components, a radar sensor and a signal processing radar based recognition system with consists of systems is ofsensors specialininterest since pedestrian radar sensors are less in- 45 unit,Ai.e. The use of radar automotive recognition radar raw pedestrian data preprocessing combined a clastwo main components, a radar sensor and a signal processing fluenced environmental conditions fog,are rain, etc.) systems is of by special interest since radar (e.g. sensors less in-as sification algorithm. In this paper, a radar signal processing unit, i.e. radar raw data preprocessing combined with a clasfluenced by environmental conditions (e.g. fog, rain, etc.) as unit is developed that allows the investigation of the potential Correspondence to: A. Bartsch sification algorithm. In this paper, a radar signal processing other systems like video cameras (Wenger, 2007). Moreover, and limits of the radar sensor in an online system. Thus, in ([email protected]) unit is developed that allows the investigation of the potential high resolution radar sensors are available in many modern case of a classification error, it can be examined if the sensor vehicles as a part of Adaptive Cruise Control (ACC) systems. data was not sufficient to be able to choose the correct object

Published by Copernicus Publications on behalf of the URSI Landesausschuss in der Bundesrepublik Deutschland e.V.

46

A. Bartsch et al.: Pedestrian recognition using automotive radar sensors

class or if the classification algorithm could not squeeze the necessary information out of the sensor’s data because of a non-optimal classification algorithm. Misleading sensor raw data of an object is statistically not distinguishable from data of an object belonging to a different class. This could be data from a slowly moving pedestrian for example, that is almost identical to raw data from a small static object like a traffic sign because of quantization errors. Previous approaches to pedestrian recognition with radar sensors (e.g. Benitez, 2011; Rohling, 2007; Freund, 2007; Büchele, 2008; Kouemou, 2008; He, 2010) mostly used complex signal features, machine learning for classification and often human gait models for interpreting micro Doppler signatures. But classification decisions made by machine learning algorithms are non-transparent. Thus, it is not possible to examine why exactly the algorithm decided for the classification result in every case. The significance of the different object features, extracted from the sensor’s data, was learned while processing a training data set. This curcial information is buried deep in the algorithm and cannot be easily obtained. Hence, machine learning does not allow for consistent backtracking from classification decisions to the critical information in the radar raw data that caused the decision (see Fig. 1). Human gait models (e.g., Nanzer, 2009, Kim, 2008, Ritter, 2007, Hornsteiner, 2008), which explain micro Doppler signatures, are limited to sensors with very high Doppler frequency resolution and need radar data over a longer, continous period of time instead of single frames. That is why a different approach in pedestrian recognition without using machine learning and complex object features is needed. In this paper, a knowledge based pedestrian recognition system, consisting of simplified components, is shown that allows transparent classification decisions. The selection, significance testing, weighting of object features and classification algorithm details were found empirically. A focus will be on boundary and surrounding conditions and their impact on radar raw data. The pedestrian’s direction of movement, occlusion, radar elevation angle and other factors will be investigated to see how real life scenarios affect the recognition results in comparison to recognition under optimal conditions.

2

The radar sensor’s raw data

In this paper, a 77 GHz band automotive scanning radar sensor is used. The sensor is designed for series production ACC systems. Table 1 shows some more detailed technical characteristics. Every 66 ms the sensor measures the received signal strength values and Doppler frequency shifts for every spatial resolution cell. A resolution cell is 1◦ by 0.25 m. The Doppler frequency shift is proportional to the radial component of the relative velocity between the sensor and the reflecting object and, therefore, the sensor can easily conAdv. Radio Sci., 10, 45–55, 2012

Table 1. Technical details of the radar sensor used in this paper. Frequency range

76–77

Resolution of Doppler frequency/ radial relative velocity

390/0.77

Hz/m s−1

1

◦ (degree)

Spatial discretization in azimuth Radial spatial discretization

GHz

0.25

m

Max. distance of objects

50

m

Beam steering range (azimuth)

17

◦ (degree)

Beam width (3 dB, azimuth)

2.5

◦ (degree)

Cycle time

66

ms

Modulation technique

chirp sequence

–

vert the Doppler shift information to radial velocity information. It is important to keep in mind that an object moving equidistant to the sensor does not cause any Doppler shift in the reflected signal. The sensor output for the Doppler information is not limited by frequency measurement resolution because it is internally interpolated to minimize quantization errors. In every measurement cycle, one data frame consisting of two matrices is created. The so called “intensity image” contains the received signal strength values for all spatial resolution cells and the “frequency image” contains the radial relative velocities of reflection centers in every resolution cell. Figure 2 shows some example radar raw data of a pedestrian. These data matrices constitute the low-level radar raw data used for pedestrian recognition.

3

The pedestrian recognition algorithm

In this chapter, a novel method for pedestrian recognition avoiding machine learning and tracking is developed. The main concern in this step is to keep the signal processing from radar raw data to the classification result as simple and transparent as possible. This guarantees easy backtracking from classification decisions to the radar raw data. An even more powerful advantage of simple signal processing is that object features can be tested for their significance in classification by hand with low effort. Tracking objects in radar raw data for better classification was discarded to benefit from single frame processing. For driver assistance systems, it is most essential to achieve minimum latency between appearance of a pedestrian in the raw radar data and the output of the correct classification decision. Tracking would introduce such latency and since radar based pedestrian recognition will be used together with other sensor systems in practice, tracking is more benefitial on a higher application level. www.adv-radio-sci.net/10/45/2012/

A. Bartsch et al.: Pedestrian recognition using automotive radar sensors A. Bartsch et al.: Pedestrian recognition using automotive radar sensors

165

170

175

Fig. 2. Example of a pedestrian’s radar raw data. The principle, Fig. 2. Example of a pedestrian’s radar raw data. The principle, how the sensor divides its spatial measurement range into resoluhow sensor dividesonitsthespatial measurement rangethe intoreflected resolutionthe cells, is depicted left. The sensor measures tion cells, is depicted on the left. The sensor measures the reflected power and Doppler shift in the reflected signal for every resolution power and Doppler in thethe reflected signal forand every resolution180 cell, giving two datashift matrices, intensity image the frequency cell, giving two data matrices, the intensity image and the frequency image (middle). The grey value of pixels in the intensity image image (middle). The grey value of pixels in the intensity image refers to the reflected power while the grey value of pixels in the refers to theimage reflected power while themeasured grey value of pixels in the frequency correspondes to the Doppler shift. A frequency to the shift measured Doppler shift. A grey valueimage of 128correspondes indicates a Doppler of 0 Hz. grey value of 128 indicates a Doppler shift of 0 Hz. 185

140

145

150

155

160

based pedestrian recognition will be used together with other Furthermore, paper highlights differentiation sensor systems this in practice, tracking isthe more benefitial onbea tween static objects and pedestrians. This is challenging behigher application level. cause walking pedestrians very low comparedbeto190 Furthermore, this paperhave highlights the speed differentiation −1 . the sensor’s velocity measurement resolution of 0.77 m s tween static objects and pedestrians. This is challenging beOther typical traffic e.g. compared cars, exhibit causeobjects walkinginpedestrians have scenarios, very low speed to significant higher velocities and canresolution be distinguished from the sensor’s velocity measurement of 0.77 m/s. pedestrians withinless effort. Other objects typical traffic scenarios, e.g. cars, exhibit significant higher velocities and can be distinguished from 195 3.1 Raw data pedestrians withpreprocessing less effort.

The chain starts with noise reduction and 3.1 signal Rawprocessing data preprocessing segmentation in the intensity image. Because simple lowpass would blurchain the critical border between objects Thefiltering signal processing starts with noise reduction and 200 and surrounding, a combined bit-depth reduction and segmentation in the intensity image. Because simplethreshlowolding approach was blur used.the The segmentation algorithm was pass filtering would critical border between objects developed in (Freund, 2007) and creates objects, i.e. sets of and surrounding, a combined bit-depth reduction and threshspatial neighboured sensor data that is likely from one physolding approach was used. The segmentation algorithm was ical object. in This segmentation step has big influence on the developed (Freund, 2007) and creates objects, i.e. sets of 205 classification result because it decides wherefrom the one border spatial neighboured sensor data that is likely phys-is ical object. Thisobjects segmentation hassurrounding. big influence on drawn between and thestep noisy In the the classification because it decides image where contains the border is border area of result objects, the frequency artedrawn objects and surrounding. In theIf facts thatbetween can extensively biasthe the noisy classification algorithm. border area of objects, theand frequency image contains the border between objects the surrounding is drawnartetoo210 factstothat extensively bias classification algorithm. If close thecan reflection center of the objects, not all related resoluthecells border and theand surrounding drawn too tion getbetween attachedobjects to the object importantisinformation closethe to the reflection centerThe of objects, notinallthis related resolu-is about object gets lost. optimum trade-off tion cells get attached to the object and important information highly dependent on background noise in the intensity image. about object gets lost.the The optimumofinthe thismeasurement trade-off is 215 In thethe frequency image, ego-speed highly vdependent on background noise in the intensity image. vehicle ego has to be compensated. If the vehicle moves with vego , a static object in front of the vehicle has a relative velocwww.adv-radio-sci.net/10/45/2012/

3

47

ityInofthe−v is measured by theofsensor. Experiments ego , which frequency image, the ego-speed the measurement showed that the vehicle’s ego-speed signal, generated by the vehicle vego has to be compensated. If the vehicle moves vehicle’s dynamic stability control system, is too noisy with vego , a static object in front of the vehicle has a relativefor satisfactory compensation. To eliminate this error, an object velocity of −v ego , which is measured by the sensor. Experiwhich is known to be static is ego-speed needed in signal, the radar raw data, ments showed that the vehicle’s generated e.g. tree or adynamic road barrier that was identified by noisy another by theavehicle’s stability control system, is too sensor system compensation. like a camera. To Using this information, for satisfactory eliminate this error, anthe ob-frequency image can betorecompensated with sufficient accuracy ject which is known be static is needed in the radar raw for the process. Inidentified this paper, data, e.g.following a tree or recognition a road barrier that was bythe an-information which in the radarthis datainformation, are static was other sensorabout system likeobjects a camera. Using provided by hand to can allowbeexperiments withwith vego sufficient 6= 0. the frequency image recompensated accuracy for the following recognition process. In this paper, the information about which objects in the radar data 3.2static Feature extraction are was provided by hand to allow experiments with vego 6= 0. In this section we propose a set of five object features that 3.2 Feature extraction change reliably and significantly when the object, represented by the segmented raw radar data, is a pedestrian or In this section we propose a set of five object features that a static object. Since pedestrians are radar point targets (Yachange reliably and significantly when the object, repremada, by 2005), their shaperaw in the intensity is the same sented the segmented radar data, isimage a pedestrian or as the one of any small reflective object. Yamada also(Yashows a static object. Since pedestrians are radar point targets that the strength a radar signal, reflected byisa the pedestrian, mada, 2005), theirofshape in the intensity image same and thus the intensity in the intensity image, is highly flucas the one of any small reflective object. Yamada also shows tuative (about 20 dB). Hence, with information from the that the strength of a radar signal, reflected by a pedestrian,intensity only, itinisthe impossible to distinguish and thusimage the intensity intensity image, is highlybetween flucpedestrians and small arbitrary objects. Nevertheless, tuative (about 20 dB). Hence, with information from theinin-this paper two object from thetointensity image, a size tensity image only,features it is impossible distinguish between and a shape factor, are calculated to exclude big or elongate pedestrians and small arbitrary objects. Nevertheless, in this objects from potential pedestrians. paper two object features from the intensity image, a size andFigure a shape factor, are to exclude bigreflected or elongate 3 reveals thatcalculated in a resolution cell the power objects from the potential pedestrians. value from intensity image and the radial relative velocreveals in a resolution reflected power ityFigure value3from thethat frequency image cell are the statistically uncorrevalue from the intensity image and the radial relative veloclated. The graphs depict the frequency distributions of resity value cells from with the frequency image areintensity statistically olution certain reflection anduncorreradial rellated. The graphs depict the frequency distributions of resative velocity for different types of objects. The statistical olution cells with certain reflection intensity and radial reluncorrelation means that the mean and variance of a resoluative velocity for different types of objects. The statistical tion cell’s radial velocity value are not correlated to its intenuncorrelation means that the mean and variance of a resolusity value. Therefore, principal component analysis (PCA) tion cell’s radial velocity value are not correlated to its intenor similar methods to transform possibly correlated data sets sity value. Therefore, principal component analysis (PCA) into uncorrelated sets are possibly obsolete.correlated Another data interesting or similar methods data to transform sets point is that the relative velocity information is not correlated into uncorrelated data sets are obsolete. Another interesting with is thethat distance fromvelocity the object center since resolution cells point the relative information is not correlated with high intensity are located at the center of an object. with the distance from the object center since resolution cells withWhile high intensity are located at the center of an the variance of the intensity values is object. almost equal the variance of the intensity values is the almost equal of forWhile moving pedestrians and static objects, variance for pedestrians and staticsignificantly. objects, the The variance of the moving radial velocity values varies classificathe velocity values variespedestrians significantly. Thestatic classificationradial algorithm distinguishes from objects tion algorithm distinguishes pedestrians static objects mainly using the higher variance of the from relative radial velocmainly using the higher of image the relative velocity components in the variance frequency data radial of pedestrians ity components in the frequency image data of the pedestrians Three object features from frequency (see Sect. 3.2.3). (see Sect. 3.2.3). Three object features fromgiving the frequency image data are calculated for every object, in combiimage calculated for every object, in combinationdata withare two object features from the giving intensity image an nation withdimensional two object features intensity image anfor only five feature from spacethe which is sufficient proper classification. Adv. Radio Sci., 10, 45–55, 2012

4


3

1 0

radially moving pedestrian 2 1 0

3 A. Bartsch et al.: Pedestrian recognition using automotive radar sensors laterally moving pedestrian static objects

2 3 2

0

1

2

3

laterally moving pedestrian

1

relative radial velocity / m/s

2


radially moving pedestrian



3 relative radial velocity / m/s

3 4

0


48

static objects

1 2 1

0

0

!1 !1 !1 !80 !70 !60 !50 !40 !30 !80 !70 !60 !50 !40 !30 !80 !70 !60 !50 !40 !30 !1 intensity / dBFS !1 !1 / dBFS reflection reflection intensity reflection intensity / dBFS !80 !70 !60 !50 !40 !30 !80 !70 !60 !50 !40 !30 !80 !70 !60 !50 !40 !30 reflection intensity / dBFS

reflection intensity / dBFS

reflection intensity / dBFS

Fig. 3. Relative distributions of resolution cells reflection intensity and relative radial velocity forobject different object Fig. 3. frequency Relative frequency distributions of resolution cellswith with certain certain reflection intensity and relative radial velocity for different Fig. 3. Relative frequency distributions of resolution cells with certain reflection intensity and relative radial velocity for different object The frequency distributions were retrieved from approximately 150 data frames giving aboutabout 2000 data points per graph.per Since meanSince mean types. Thetypes. frequency distributions were retrieved from approximately 150 data frames giving 2000 data points graph. types. The frequency distributions were retrieved from approximately 150 data frames giving about 2000 data points per graph. Since mean andof variance of the relative radial velocity values independent of of the reflection intensity, information from the intensity image andimage from and from and variance thevariance relative velocity values areare independent the intensity, information from the intensity and of radial the relative radial velocity values are independent of thereflection reflection intensity, information from the intensity image and from the frequency image is statistically uncorrelated. the frequency image is statistically uncorrelated. the frequency image is statistically uncorrelated. 3.2.1only The size of an object five dimensional feature space which is sufficient for

only five dimensional feature space which is sufficient for proper classification. The size of an object is a measure that is proportional to proper classification.

an object’s radius simply the square root of the 3.2.1 mean The size of anand object distance-compensated number of resolution cells assigned to 3.2.1 The size anofobject 220 object. Theof size an compensation object is a measure that is because proportional the Distance is necessary with to an object’s mean radius simply squarepower root of increasing distance r from theand sensor, the the received PDthe The size decreases of an object is 4a and measure that is to to distance-compensated number ofof resolution cellsresolution assigned with 1/r the width the proportional spatial ◦ the object. Distance compensation is necessary because with an object’s mean radius and simply the 1square root With of the cells get larger since they are always in azimuth. n0 increasing distance r from the sensor, the received power as the distance-compensated number of cells resolution cells to ofPD distance-compensated number of resolution assigned √ 1/r4 and 225 decreases with the width of the spatial resolution 0 object, m1compensation = n is the firstisobject feature. the object.theDistance necessary cells get larger since they are always 1◦ inbecause azimuth. with With n0 increasing3.2.2 distance r from the sensor, the received power P as The the distance-compensated shape of √ an object number of resolution cellsD of 4 0 the1/r object,and m1 = is the first object feature.resolution decreases with the nwidth of the spatial Fig.4. 4.An Anillustration illustrationofofthe thesensor sensor coordinate coordinate system, ◦radar raw data is deter-0 Fig. system, an an object object The shape of they a point target in the cells get larger since are always 1 in azimuth. With n with center at (rc , φc ) and a local coordinate system for calculating 3.2.2 The shape of an object with center at (r , φ ) and a local coordinate system for calculating c c 0 0 mined by the sensor’s antenna pattern. In the intensity image, the object’s shape factor using (x0 , y0 ) coordinates. The maximum as the distance-compensated number of resolution cells of √ has an elliptical the object’s shape factor using (x , y ) coordinates. The maximum shape theraw longer in beam steering angle φˆ is the border of the azimuthal measurement shape a point in feature. thewith radar dataaxis is deterthe object,a230small m1The =object n0 isofthe firsttarget object beam steering angle φˆ is the border of the azimuthal measurement azimuthal and an axis ratio of In about D/d = image, 1.8. range. mineddirection by the sensor’s antenna pattern. the intensity Fig. 4. An illustration of the sensor coordinate system, an object range. To obtain an object’s shape feature, the difference of the oba small object has an elliptical shape with the longer axis in with center at (rc , φc ) and a local coordinate system for calculating 3.2.2 The shape of an object ject’sazimuthal shape compared an an ideal is calculated. directiontoand axisellipse ratio of about D/d =An 1.8. the object’s shape factor using (x0 , y 0 ) coordinates. The maximum ideal To ellipse areashape and half-axes b is placed overob- 245 3.2.3 Doppler spectrum obtainofanequal object’s feature, thea,difference of the Doppler angle spectrum beam steering φˆ is the border of the azimuthal measurement The shapethe ofject’s a point target inc )the rawIn data is deter-An 3.2.3 center (rc ,φ (see Fig. this coordinate 235 object’s shape compared to anradar ideal4). ellipse is calculated. range. mined by system, the sensor’s In the intensity image, The Doppler spectrum (DS) is the histogram of an object’s ideal of equalpattern. area andan half-axes a, bniscells placed theellipse i-thantenna resolution cell of object with hasover a The Doppler spectrum (DS) is the histogram of an object’s 0 0 Doppler shift information, extracted from all resolution cells object’s center (r ,φ ) (see Fig. 4). In this coordinate local the coordinate (x , y ). A distance measure for a cell to the c c a small object has an elliptical shape with the longer axis in i i Doppler shift information, extracted from all resolution cells assigned to the object. It is a density function that tells system, the ith resolution cell of an object with n cells has a ellipse’s border is given by azimuthal direction and an axis ratio of about D/d = 1.8. assigned to the object. is a density tells how prominent different It relative velocitiesfunction are in thethat object. local coordinate (x0i , yi0 ). A distance measure for a cell to the r 245 how 3.2.3 Doppler spectrum 02 02 prominent different relative velocities are in the obTo obtain 240 an object’s shape feature, the difference of the obyellipse’s xi border is given D n·d 250 The relative radial velocity axis v is divided into small inby i δ = · b , b = . (1) + − 1 , a = ject. The relative radial velocity axis v is divided into small i r· D ject’s shape compared an ideald ellipse is calculated. An tervals. Resolution cells with radial relative velocity vrr inπ a 2 y 02b2 xto 02 D n·d intervals. cells with relative velocity i i crease theResolution counter for the corresponding intervals [v i ,vj [ with The Doppler spectrum (DS)radial is the histogram ofvrran object’s ideal ellipse of equal area and half-axes a, b is placed over δi = 2 + 2 − 1 , a = · b , b = . (1) The size-normalized cumulativedshape error gives a b π · the D second increase the counter for the corresponding intervals [v ,vj [ inorvi ≤ |vrr | shift < vj . information, To get a densitiy DDoppler Rfunctionfrom Doppler extracted all ,resolution cells the object’s center (rmc ,φ c ) (see Fig. 4). In this coordinate object feature : 2 with v ≤ |v | < v . To get a densitiy function D , nori rris necessary j The size-normalized cumulative shape error gives the second malization such that = 1 holds. Doppler dvDoppler R is D assigned to the object. It a density function that tells system, the ithobject resolution of an object with n cells has a 255 malization is necessary that spectra, DDoppler dv = 1 holds. feature mcell Figure 5 depicts samplesuch Doppler averaged over sev2: 1 X 2 0 0 m2 = √ (xi1, yδiX (2) how prominent different relative velocities areover in the object. local coordinate ). A distance measure for a cell to the 5 depicts sample data Doppler spectra, averaged Figure eral hundred independent frames. The wavy structure n=i:δ√>0 i δ 2 m (2) i 2 of the DS, especially of walking pedestrians, is not caused i several hundred independent data frames. The wavy 250 The relative radial velocity axis v is divided into small inellipse’s border is given n by i:δi >0 by movements of the object itself. The maxima of the wavy r tervals. Resolution cells with radial relative velocity v in-

D y 02 x02 n·d δi = i2 +Adv.i2 Radio − 1 , Sci.,a10, = 45–55, · b ,2012b = . (1) a b d π ·D The size-normalized cumulative shape error gives the second

rr

crease the counter for the corresponding intervals [vi ,vj [ with www.adv-radio-sci.net/10/45/2012/ vi ≤ |vrr | < vj . To get a densitiy Rfunction DDoppler , normalization is necessary such that DDoppler dv = 1 holds.


5


Table 2. Relative radial velocity interval borders for Doppler spec-

6 5

radial moving pedestrians static objects

Table 2. Relative velocity interval borders for Doppler spectrum feature radial extraction. trum feature extraction. v0

DDoppler / s/m

49

4

0 m s−1

v0

v1

v1

v2

v2

v3

v3

0 m/s 0.38 m/s 1.15 m/s 3 m/s 0.38 m s−1 1.15 m s−1 3 m s−1

3 2

Classification 3.3 3.3 Classification

The The challenge in theinclassification step is step to decide challenge the classification is to whether decide whether an object is a pedestrian or a static object by using cor-the correan object is a pedestrian or a static object by the using T only.TThe deciresponding feature vector m = [m ,...,m ] 5 285 sponding feature vector m 1= [m1 ,...,m 5 ] only. The decisionsion process is based on theon knowledge that some 0 process is based the knowledge thatvalues someofvalues of 0 0.5 1 1.5 2 2.5 3 thesethese features are more probable if the object is of a features are more probable if the objectcertain is of a certain relative radial velocity / m/s class C. For example, a static object is very unlikely to have class C. For example, a static object is very unlikely to have a significant fraction of Doppler information in the [v2 ,v3 ] a significant fraction of Doppler information in the [v2 ,v3 ] interval. Hence, a value of m3 close to one is very unlikely Fig. 5. Averaged Doppler spectra of radial moving pedestrians and 290 interval. Hence, a value of m3 close to one is very unlikely Fig. 5. Averaged Doppler spectra of radial moving pedestrians and for the class “Static Object”. staticobjects. objects. The The Doppler Doppler spectrum is plotted static spectrumfunction functionDDoppler DDoppler is plotted for the class “Static Both valid classes, P Object”. for pedestrians and S for static overthe therelative relative radial radial velocity compensation over velocityaxis axisv.v.Ego-speed Ego-speed compensation objects,Both valid classes, P for pedestrians andfuncS for static have a vector valued fuzzy membership not necessary because the data was recorded with a non-moving isisnot necessary because the data was recorded with a non-moving tion objects, 5 → 5 ,vector have a valued fuzzy membership funcf : R [0,1] C ∈ {P ; S} that maps the feature C radar sensor. radar sensor. tionmf C R5 object → [0,1]to5 , C {P ; S} that maps the feature vector of : an a ∈ membership value vector T uC = [µC,1 ,...,µ the corresponding class. 295 vector m of object to a membership value vector C,5 ]anfor T structure of the DS, especially of walking pedestrians, is not uC = [µC,1 ,...,µC,5 ] for the corresponding class. (6) caused by of theatobject itself. Thefrequency maxima ofbins uC = f C (m), µC,i = fC,i (mi ) structure aremovements always located the theoretical the wavy structure are always located at the theoretical freµC,i = fC,i (mi ) (6) C = f C (m), (if interpolation would not be used), and the local minima Auvalue of µC,i close to one states a high probability of quency bins (if interpolation would not be used), and the loright inbetween, independent from the object’s speed. Thus, mi for the corresponding class while a small value of µC,i of m A value of µC,i close to Cone states a high probability cal minima right inbetween, independent from the object’s i near zero indicates that the value of mi is unlikely for that the wavy structure is an artefact caused by the interpolation speed. Thus, the wavy structure is an artefact caused by the for the corresponding class C while a small value of µC,i In this way, the membership values µC,i of the five algorithm in the Dopplerin frequency measurement inside the 300class. interpolation algorithm the Doppler frequency measurenear zero of indicates that the value of m is unlikely for that object features every object are calculated fori both classes radar mentsensor. inside the radar sensor. class. In this way, the membership values µC,i of the five P and S. Graphs of the membership functions fC,i are plotcanbe beclearly clearly seen moving pedesobject features of every object are calculated for both ItItcan seen in inFig. Fig.55that thata aradial radial moving pedes- ted in Fig. 6. The functions fS,1 and fS,2 are not defined classes −1 . In contrast, DS of trianhas has aa broad broad DS m sm/s. P and S. objects Graphscan of the membership functions fC,i are plottrian DS from from0–2.5 0 – 2.5 In contrast, DS of because static be of any shape and size. Hence, static objects are very narrow since they only cover velocited in of Fig. fS,1 likelihood and fS,2 are static objects are very narrow since they only cover veloci- all values m16.andThe m2 functions have the same for not the defined ties from 0–0.4 m s−1 . 305 static can bevalues of any ties from 0 – 0.4 m/s. classbecause S and thus theirobjects membership forshape class Sand are size. sup- Hence, To boil down the shape of a DS to key figures, three interpressed in further givingthe them zerolikelihood weight in for the all values ofcalculations m1 and mby same 2 have To [v boil the shape DS to key(see figures, vals area considered Tablethree 2 for in0 ,vdown 1 ],[v1 ,v 2 ],[v 2 ,v3 ]of the weighting step. class S and thus their membership values for class S are suptervals (see Tab. 2 for actual [v values of v10,v -v22],[v ). The spectrum density func0 ,v1 ],[v 2 ,vDoppler 3 ] are considered Topressed decide in whether class P or S is more appropriate forweight in calculations by giving them zero tion Dvalues is vintegrated over these intervals, three actual Doppler spectrumgiving density func- an object, givenfurther Doppler of 0 -v2 ). The fuzzy membership vectors u and u , the P S the weighting step. object featuresismintegrated theintervals, proportional distri3 –m5 , representing tion DDoppler over these giving three membership vectors are mapped values 310 To decide whether classtoPreal or membership S is more appropriate for butionfeatures of radialm relative velocities in the object over the three object 3 -m5 , representing the proportional distri- µP , µS ∈ [0,1]. This is done by a weighted average of the an object, given fuzzy membership vectors u and u , the intervals. P S bution of radial relative velocities in the object over the three membership vectors’ components membership vectors are mapped to real membership values intervals. Rv1 µPw,TCµ·SuC∈ [0,1]. This is done by a weighted average of the m3 = DDoppler dv (3) P µC = , vectors’ components (7) v0 membership wC,i v v 1 RR2 i mm3 4== D dv DDoppler (4) (3) Doppler dv w T · uC vv01 µCw=C = C 315 , C,5 ]T is the weighting vector for a (7) where [w ,...,w T ·C,1 w 1 v v 5 3 2 C R R class C ∈ {P ;S}. DDoppler (5) (4) mm4 5== D dv Doppler dv T Experiments that constant weighting vectors vv12 where wC have = [wshown C,1 ,...,wC,5 ] is the weighting vector for a w are not sufficient for proper classification. Instead, thevector of C class C ∈ {P ;S} and 1 is a p-dimensional Rv3 column p −1 are neglected since combination of the membership values µ must be closer Radial velocities greater than 3 m s m5 = DDoppler dv (5) C,i ones. 2 unlikely to be in pedestrians’ DS and thus not the to a logical conjunction. This is achieved by allowing the they vare that on constant weighting weightExperiments vector w C (uChave ) to beshown dependent uC , meaning that vectors matter in this paper. 1

260

265

270

275

320

280

Radial velocities greater than 3 m/s are neglected since they www.adv-radio-sci.net/10/45/2012/ are unlikely for pedestrians and thus not the matter in this paper.

wC are not sufficient for proper classification. Instead, the combination of the membership values µC,i must be closer Adv. Radio Sci., 10, 45–55, to a logical conjunction. This is achieved by 2012 allowing the weight vector wC (uC ) to be dependent on uC , meaning


50


A. Bartsch et al.: Pedestrian recognition using automotive radar sensors 1

1 P,5

1

0.5 0.5

µµP,5

0.5

1

1

µP,3

P,3

0.5

µ

0.5

µP,2

µP,2

P,1

0.5

1

1

µP,4

1

1

µ

µP,1

1

µP,4

6

0.50.5

0.8 0.8 0.6 0.6

0

10 10 0 20 10 200 5 0 105 feature m2 feature m feature m1 feature m 1

0

0

0

2

3

µS,3

S,3

µ

µS,1 and µS,2 are not defined.

00.5 0.5 1 1 feature feature m m3

1

0.5

0.5

0

0

0

0

0.5 1 feature m3

0.5 1 feature m3

00 0.5 0.5 1 1 feature featurem5m

4

1

1 µS,1 and µS,2 are not defined.

0 0 0.5 11 0 0 0.5 featuremm4 feature 1

1

0.5

0.5 0

0

5

1

µ µS,5 S,5

0

0

µS,4

0

µS,4

0

0

0

0.5 1 feature m4

0.5 1 feature m4

0.5

0.5 0

0

0.5

1

0 feature m 0 0.5 5 1 feature m5

Fig. 6. Plots of membership functions fP,i and fS,i that map values of object features mi to membership values µP,i and µS,i for the classes P for6.pedestrians and S for static objects. Fig. Plots of membership functions fP ,i and fS,i that map values of object features mi to membership values µP ,i and µS,i for the

DDoppler / s/m

DDoppler / s/m

g. 6. Plots of membership functions and fobjects. classes P for pedestrians and Sffor P,istatic S,i that map values of object features mi to membership values µP,i and µS,i for the classes for pedestrians and S for static objects. wP,4 will rise exponentially, and the overall probability µP velocity in the [v1 ,vto2 ]be interval. Hence,will the object m4 12 for this object a pedestrian be low.feature This conclusion will would be zerobe and since there are some cells in the [v ,v ] 2 3 inwrong because the characteristic Doppler informa- µ w rise exponentially, and the overall probability v1 v2 v3 P,4 will P terval, m > 0 will hold. Because m equals zero, its weight 5 4 10 340 tion for a pedestrian is just concentrated in the [v2 ,v3 ] inter12 for to be a pedestrian will be low. This wP ,4this willobject rise exponentially, and the overall probability µPconclusion val rather than evenly distributed over the [v1 ,v3 ] interval. In for this object to be abecause pedestrian will be low. This conclusion would be wrong the characteristic Doppler informav1 8 v2 v3 this case, where m4 has a very low and m5 a high value, ξP,4 would be wrong because the characteristic Doppler informa10 340 tionisfor a pedestrian is just concentrated in the [v 2 ,v3 ] interchosen small is(ξjust 1/γ) to reverse the value dependent P,4 ≈ tion for a pedestrian concentrated in the [v ,v 2 3 ] inter6 val rather thanAnalogously, evenly distributed over them[v51is,vvery 3 ]Ininterval. weighting. ξP,5 chosen low in In val rather than evenly distributed overisthe [v1 ,v3if] interval. 8 this case, where m has a very low and m a high value, 345 value while a significant share of Doppler information 4 a very low and m5 a high value, 5 this case, where m4 has ξP ,4 is inξP,4 4 the [v1small ,v Since value dependent is chosen small ≈)1/γ) tocross-bonded reverse value dependent 2 ] interval. is chosen (ξP(ξ ≈ 1/γ to this reverse the valuethe dependent ,4 P,4 6 weighting affects only w and w , all ξ except P,4 P,5 C,i P,4 in weighting. Analogously, ξ is chosen if m is very low in P ,5 ξP,5 is chosen 5 if m5 is veryξlow weighting. Analogously, 2 and ξ have a constant value of one. value while a significant share of Doppler information is in P,5 345 value while a significant share of Doppler information is in an object is classified a pedestrian the [v1The ,v2 ] decision interval. whether Since this cross-bonded value as depen4 the [v ,v2a] static interval. Since this cross-bonded value dependent 0 1as dent weighting affects only w and w , all ξ except 350 or object and, therefore, is assigned to the class C,i P ,4 P ,5 0 0.5 1 1.5 2 2.5 3 weighting affects only by wvalue and wP,5 all ξC,i except ξP,4 P,4 ξP ,4Pand of one. orξclass S, aisconstant made comparing the, united membership P ,5 have relative radial velocity / m/s 2 andThe ξ have a constant value of one. decision whether an object is classified as a pedestrian P,5 µP and µS . If µP > µS , the object is assigned to the values or The asclass a static object and, therefore, is assigned to the as class decision whether object classified a pedestrian P and otherwise toanclass S. Ifisboth membership values 0 P or class S, is made by comparing the united membership Fig. 7. Example of a single frame Doppler spectrum of a pedestrian. Fig. 7. Example of a single frame Doppler spectrum of a pedestrian. µ and µ are small and hence the object is unlikely to be 350 or as a static object and, therefore, is assigned to the class P S 0 0.5 1 1.5 2 2.5 3 values µP and µS as . Ifwell µP >asµSa, static the object is assigned to the as an 355 a pedestrian object, it is declared P or Pclass S, is made by S. comparing the united membership relative radial velocity / m/s class and otherwise to class If both membership values unknown object. This happens almost exclusively in case to of the andsmall µS .and If µ µSobject , the object is assigned low membership valuesvalues µC,i are stronger by inby values µP and µ µP are hence is unlikely to be P > the that low membership µC,iweighted are weighted stronger S segmentation errors. creasing wC,i exponentionally: a pedestrian well as a static object, it is declared as an class P andasotherwise to class S. If both membership values 325 increasing wC,i exponentionally: unknown object. This happens almost exclusively in case of µC,ispectrum g. 7. Example of a single0frame of a pedestrian. Doppler µP and µS are small and hence the object is unlikely to be µ wC,i = ξC,i · w 0 · · 1 1++ γ γ· e·−e−τ C,i (8) (8) segmentation errors. τ . w . 4 Experimental C,i = ξC,i · wC,i C,i 355 a pedestrian as wellresults as a static object, it is declared as an

The exponential function for increasing the base weight 0 unknown object. This happens almost exclusively in case of The function increasing thestronger base weight w Because a binary classifier is used and the number of un0 exponential at low membership values µC,i for are weighted by wC,i was chosen empirically. While γ and τ parametrize this C,i segmentation 4 Experimental results errors. was chosen empirically. While γ and τ parametrize this 360 known declared objects is negligible, the quality of the creasing wvalue dependent weighting, ξC,i is used for compensating a C,i exponentionally: value dependent weighting, ξ is used for compensating a pedestrian recognition system can be only two certain effect has an unusual Doppler freBecause a binary classifier is used and the measured number ofbyun when a µpedestrian C,i C,i 330 certain effect when a pedestrian has an unusual Doppler frekey figures, r and r . The true positive rate r TP 0 distribution.−In τFig. 7, a DS of a walking pedestrian is quency declared objects is FP negligible, the quality of theTP tells wC,i · distribution. 1 + γ · e In Fig. .7, a DS of a walking pedestrian (8) is 4known Experimental results C,i = ξC,i ·quency what fraction of real pedestrians is correctly classified as depicted, obtained from a single data frame. In this case the

pedestrian recognition system can be measured by only two

depicted, from a single data frame. In thisradial case the key such. the rother hand, the false positive rate r or object doesobtained not contain a resolution cell with a relative figures, rOn and FP . The true positive rate rTP tells what FP 0 he exponential for increasing the cell basewith weight wC,i Because a TP binary classifier is usedof and number of unobjectfunction does not contain a resolution a relative radial 365 false alarm rate tells what fraction staticthe objects are clasas chosenvelocity empirically. γ andHence, τ parametrize this 360m4 known objects is negligible, the negative quality rate of the in the [v1While ,v2 ] interval. the object feature sifieddeclared incorrectly as pedestrians. The true Adv. Radio Sci., 10, 45–55, 2012 www.adv-radio-sci.net/10/45/2012/ alue dependent ξC,ithere is used for compensating can berate measured byrTP only 335 will beweighting, zero and since are some cells in the [v2 ,va3 ] in- pedestrian rTN = 1 recognition − rFP and thesystem false negative rFN = 1 − aretwo terval, m > 0 will hold. Because m equals zero, its weight implied under the above-mentioned assuptions. ertain effect when 5a pedestrian has an unusual Doppler frekey figures, rTP and rFP . The true positive rate rTP tells 4

uency distribution. In Fig. 7, a DS of a walking pedestrian is

what fraction of real pedestrians is correctly classified as

A. etal.: al.:Pedestrian Pedestrianrecognition recognition using automotive radar sensors A. Bartsch Bartsch et using automotive radar sensors


7

7 51 7

66 pedestrian, slow pace (0.58 m/s)m/s) pedestrian, slow pace (0.58 pedestrian, normal (1.2(1.2 m/s) pedestrian, slow pace (0.58 m/s) pedestrian, normal pace m/s) pedestrian, fast pace (1.7 m/s) pedestrian, normal pace (1.2 m/s) pedestrian, fast pace (1.7 m/s) pedestrian, fast pace (1.7 m/s)

6

55

/ s/m

4

4

3

Doppler

3 3

2

D

DDoppler DDoppler/ s/m / s/m

5

4

2 2

1

1 1 0

00 0 0 0

0.5

1

1.5

2

2.5

1.5 2 2.5 / m/s 0.50.5relative 1 1radial velocity 1.5 2 2.5 relative radial velocity / m/s relative radial velocity / m/s

3

3

3

Fig. 8. Averaged Doppler spectra of a walking pedestrian for difFig. 8. Averaged Doppler spectra of a walking pedestrian◦ for different paces. The angle of movement was ϕ = 45 for infor all difFig. 8. Averaged Doppler spectra awalking walking pedestrian Fig.ferent 8.walking Averaged Doppler spectra pedestrian difwalking paces. The angleof ofofamovement was ϕ = 45◦ in all cases. In the investigated range of walking paces from 0.58 m/s ◦ to ◦ ferent walking The movement was ϕ= 45 in all ferent walking paces. Theangle angleofofwalking movement was ϕ0.58 = 45 into all cases. In the paces. investigated range paces from m/s 1.7 m/s,Inthe DS is independent fromwalking the pedestrian’s walking pace. −1 cases. thethe investigated range paces 0.58 m sm/s 1.7 In m/s, DS is independent from the pedestrian’s walking pace. cases. the investigated rangeofof walking pacesfrom from 0.58 to −1 , the DS is independent from the pedestrian’s walking to 1.7 m s 1.7 m/s, the DS is independent from the pedestrian’s walking pace. pace.

375

380

380

385

385

390

390

395

395

°

nt /

me

ove

ang

le o

fm

/ s/m

Doppler

aanng gllee o off m moov veemm eent nt/ /° °

375

D

Doppler

370

TheThe testtest sample setset consisted in in every experiment sample consisted every experimentofofabout about 150 to 600 samples if not mentioned otherwise. 370 150 to 600 samples if not mentioned otherwise. The test sample set consisted in every experiment of about fraction of real pedestrians is correctly classified as such. On 44 90 90 150 toInfluences 600hand, samples if not mentioned otherwise. 4.1 onon the Doppler spectrum the4.1 other the false positive rate rFP or false alarm rate Influences the Doppler spectrum 80 80 90 tells what fraction of static objects are classified incorrectly 22 4 4.1 Influences on the Doppler spectrum A walking pedestrian’s DS turned out to be very stable in A walking pedestrian’s DS turned out to be very stable in 70 as pedestrians. The true negative rate rTN = 1 − rFP and the 70 80 terms of outside compared totoother tested object terms of outside influences compared other tested object false negative rateinfluences rFN = 1−r are implied under the above00 2 TP 60 60 A walking pedestrian’s DS turned out to be very stable in features from thethe frequency image. is isnot features from frequency image.It It notaffected affectedbybythe the 70 00 mentioned assuptions. 11 terms of outside influences compared to other tested object height in which the radar beam hits the person, due to differ375 height in which the radar beam hits the person, due to differre re la The test sample set consisted in every experiment of about latitiv0 veeraraddiaial lv 22 60 45 45 veelo loccitityy // m ent angles ofsamples elevation the antenna beam long asasenough ofthe elevation the antenna beam as long enough features from frequency image. It isasnot affected by the 3 m/s /s 0 150ent toangles 600 if of notof mentioned otherwise. 1 power is reflected proper segmentation ininthe intensity power is reflected for proper segmentation the intensity height in which theforradar beam hits the person, due to differrelative ra 2 dial veloc 45 Furthermore, the DS is fairly independent from the image. Furthermore, the DS isantenna fairly independent from the ity / m/s ent angles of elevation of the beam as long as enough 4.1image. Influences on the Doppler spectrum 3 walking pedestrian’s pace (see Fig. 8). Walking paces from walkingispedestrian’s pace (see Fig. 8). Walkinginpaces from Fig.10. 10. Averaged Averaged Doppler Doppler spectra spectra of a walking power reflected for proper segmentation the intensity Fig. walking pedestrian pedestrian over over Fig. 10. Averaged Doppler spectra of a walking pedestrian over 380 0.58 m/s to 1.7 m/s were tested and did not affect the pedes0.58 m/s to 1.7 m/s were tested and did not affect the pedesvarious angles of movement ϕ. A walking pedestrian’s DS turned out to be very stable in various angles of movement ϕ. image. Furthermore, the DS is fairly independent from the various angles of movement ϕ. trian’s DS because many different velocities within the segtrian’s DSpedestrian’s because many different velocities within the segterms of outside influences compared to other tested object walking pace (see Fig. 8). Walking paces from Fig. 10. Averaged Doppler spectra of a walking pedestrian over mented object occured due to arm and leg swing. Neverthemented object occured due toimage. arm andIt leg swing. Neverthefeatures from the frequency is not affected by pedesthe 0.58 m/s to 1.7 m/s were tested and did not affect the various angles of movement ϕ. ◦ less, the DS obviously is dependent on the pedestrian’s angle less, theinDS obviously is dependent pedestrian’s angle height the radar beam hits on thethe person, due to differlaterallymoving moving pedestrians pedestrians (ϕ (ϕ = = 90 90◦ ). 11 exhibits ). Figure Figurenear 11 ϕ exhibits trian’s DSwhich because many different velocities within the seg- laterally and itthe approaches zeroresult for angles of movement =75 90◦◦◦. . of movement. A radar sensor can only measure the radial of A radarofsensor can only measure radial entmovement. angles of elevation the antenna beam as long the as enough that classification is better than 88 % for |ϕ| ≤ that the classification result is better than 88 % for |ϕ| ≤ 75 mented object occured due to relative arm and leg swing. Neverthe385 component offor an object’s velocity . Agood good ap- 400 Hence, lateral moving pedestrians have DS very similar to. rr an rA component vrr vof object’s velocity vrv.the appower is reflected properrelative segmentation in intensity less, the Furthermore, DS obviously is the dependent the pedestrian’s angle proximation is assume to assume the centered front of400 static objects.moving In Fig. 10, averaged DS=of90a◦walking pedesproximation is to pedestrian centered inin front of image. the DS is pedestrian fairlyon independent from the laterally pedestrians Figure 11 exhibits Backtracking from classification classification(ϕ decisions).to raw Backtracking from decisions to radar radar raw the sensor, i.e. φ = 0 (see Fig. 9). By using this approximaof movement. A radar sensor can only measure the radial trian are plotted for different angles of movement ϕ. The the sensor, i.e. φ = 0 (see this approximawalking pedestrian’s paceFig. (see9). Fig.By8).using Walking paces from that the classification result is better than 88 % for |ϕ| ≤ 75◦ . data, to see whether the sensor data or the classification al◦ ,sensor data, to see whether the data or the classification altion, the radial velocity component can be simplified to closer ϕ approaches 90 the DS shows increasing similarity −1 −1 component v of an object’s relative velocity v . A good aption, velocity canand be simplified to the 0.58 the m s radial torr1.7 m s component were tested did notr affect 400gorithm is responsible for misclassified data frames, was exis responsible for misclassified was exto the one of static objects (compare todata Fig.frames, 5). Hence, the proximation is to assume the different pedestrian centered in front of gorithm pedestrian’s DS because many velocities within the tensively usedin in empirically empirically optimizing the classifier. As aa v = v · cosϕ (9) Backtracking from classification decisions to radar raw rr r tensively used optimizing the classifier. As true positive rate r after classification reaches 95.3 % for v = v · cosϕ (9) TP rr r the sensor, object i.e. φ = 0 (seedue Fig.to9). this approximasegmented occured armBy andusing leg swing. 405 result, almost all misclassifications◦ of pedestrians as static data, to see whether the sensor data or the classification al405 result, almost all misclassifications of pedestrians as static radially (ϕ = 0to )sensor and drops % for tion, theit radial velocity component can be simplified Nevertheless, the DS on theϕpedes390 and approaches zeroobviously for anglesisofdependent movement near =to90◦ . objects moving can nowpedestrians be tracked down data to that39.5 is indis◦ and it approaches zero for angles of movement near ϕ = 90◦ . objects can now be tracked down to sensor data that is indisgorithm iseven responsible for data was exlaterally moving pedestrians (ϕ misclassified = 90 ).object’s Figure 11 frames, exhibits Hence, lateral moving pedestrians have DS tinguishable, by hand, from a static raw data. In◦ trian’s angle of movement. A radar sensor can very only similar measureto Hence, lateral moving pedestrians have DS very similar to tinguishable, even by hand, from a static object’s raw data. In tensively used in empirically optimizing the classifier. As a |ϕ| that the classification result is better than 88 % for ≤ 75 . vthe =radial vr ·objects. cosϕ In Fig. of a walking rr static component vrr 10, of anaveraged object’sDS relative velocity pedesvr . A (9) Fig. 12, the averaged DS of correct and misclassified pedesstatic objects. In Fig. 10, averaged DS of a walking pedesFig. result, 12, the almost averagedallDSmisclassifications of correct and misclassified pedes-as static 405 of pedestrians trian are plotted for of movement ϕ. The trians are plotted as well as the averaged DS of static objects good approximation is different to assumeangles the pedestrian centered in ◦ trians are plotted as well as the averaged DS of static objects trianitare plotted for different angles of movement ϕ. ϕThe ◦ angles and approaches for of movement near objects can now tracked down to sensor from classification decisions to data radarthat raw closer ϕ approaches DS Fig. shows similarity comparison. The be DS of the misclassified pedestrians is is indisfront of the sensor,zero i.e.◦90 φ =, the 0 (see 9).increasing By using this = ap-90410. forBacktracking closer ϕ approaches 90 , the DS shows increasing similarity 410 for comparison. The DS of the misclassified pedestrians data, to see whether the sensor data or the classification alHence, lateral moving pedestrians have DS very similar to tinguishable, even by hand, from a static object’s raw 395proximation, to the one of (compare to Fig. Hence, the almost the same as the one of static objects. Thus, in theis data. In thestatic radialobjects velocity component can5). be simplified to the one of static objects (compare to Fig. 5). Hence, the almost same as the one of of static objects. Thus, in the gorithm is responsible for misclassified data frames, ex- pedespositive In rateFig. rTP10, after classification reaches 95.3 %pedesfor caseFig. ofthe misclassified pedestrians, the sensor does not was deliver static averaged DS of a walking 12, the averaged DS correct and misclassified to trueobjects. trueradially positive rate r after classification reaches 95.3 % for case of misclassified pedestrians, the sensor does not deliver ◦ TP moving pedestrians (ϕ = 0 ) and drops to 39.5 % for statistically different data from static objects and hence the tensively used in empirically optimizing the classifier. As trian are plotted for different angles of movement ϕ. The trians are plotted as well as the averaged DS of statica objects radially moving pedestrians (ϕ = 0◦ ) and drops to 39.5 % for statistically different data from staticofobjects and hence the ◦ v = v · cosϕ (9) result, all misclassifications pedestrians aspedestrians static rr r approaches 90 , the DS shows increasing similarity 410 for almost closer ϕ comparison. The DS of the misclassified is D / s/m D Doppler/ s/m

370

Fig. 9. A walking pedestrian with velocity vr relative to the sensor Fig.angle 9. A walking pedestrian with velocity vr relative the sensor with ofAmovement ϕ haswith a radial relative velocity walking pedestrian with velocity vr to relative to the sensor Fig. Fig. 9. A9. walking pedestrian velocity vr relative tocomponent the sensor with angle of movement ϕ has a radial relative velocity component vwith is angle measurable by ϕ thehas radar sensor. rr that angle of movement a radial velocityvelocity component with of movement ϕ has a relative radial relative component vrr that is measurable by the radar sensor. vrr that measurable by theby radar vrr isthat is measurable thesensor. radar sensor.

to the one of static objects (compare to Fig. 5). Hence, the true positive rate rTP after classification reaches 95.3 % for www.adv-radio-sci.net/10/45/2012/ radially moving pedestrians (ϕ = 0◦ ) and drops to 39.5 % for

almost the same as the one of static objects. Thus, in the case of misclassified Adv. pedestrians, the sensor does2012 not deliver Radio Sci., 10, 45–55, statistically different data from static objects and hence the

angle of movement / °

Fig. 13. single pa Fig.et 11.al.: Classification rTP for pedestrians walking with dif- sensors A. Bartsch Pedestrianresult recognition using automotive radar a gap bet A. Bartsch et al.:ferent Pedestrian recognition using automotive radar sensors angles of movement ϕ. due to mu

8

52

6

100

correct classified pedestrians wrong classified pedestrians static objects

the pede lateral m 80 425 the stree In the 4 hidden b 60 radar wa 3 signal re 40 430 sor is ve 2 blocking to sepera 20 1 positive ter than 0 0 If the 0 0.5 1 1.5 2 2.5 3 435 0 20 40 60 80 90 relative radial velocity / m/s parked c angle of movement / ° wave ca tions. Fig. 13. Top: A lateral moving pedestrian is occluded behind a T Fig.single 12. Averaged Averaged Doppler spectra of classified and incorFig. 12. spectra of correct correct classified andpedestrian incorparked Doppler car. Bottom: Now the lateral moving isradar in sen Fig. forfor pedestrians walking withwith dif- dif- rect classified pedestrians walking with various angles of moveFig. 11. 11. Classification Classificationresult resultrTP rTP pedestrians walking rect aclassified pedestrians walking various angles of move440 data if th gap between two parked cars,with where the radar wave can propagate ferent ferentangles anglesofofmovement movementϕ.ϕ. ment. The The similarity similarity of of Doppler Doppler spectra ment. spectra of of incorrect incorrect classified classified tion resu due to multiple reflections. pedestrians and and static static objects objects is is aa limitation pedestrians limitation of of the the radar radar sensor sensorinin ues up to pedestrian recognition, since the sensor does not deliver statistical pedestrian recognition, since the sensor does not deliver statistical recognit 6 different in this case. objects can now be tracked down to sensor data that is indisdifferent in this case. the case correct classified pedestrians 8 A. Bartsch et al.: Pedestrian recognition using automotive radar sensors the pedestrian is not in the line of sight to the sensor. The tinguishable, even by hand, from a static object’s raw data. In wrong classified pedestrians 445 Fig. 11) lateral movement is typical for pedestrians who try to cross 5 the averaged DS of correct and misclassified pedesFig. 12, static objects developed classification algorithm is optimal within the regap are s 425 the street without looking. trians are plotted as well as the averaged DS of static objects 415 strictions of the chosen method for pedestrian recognition. significa 100 In the case where the pedestrian is in the shadowed area for comparison. The DS of the misclassified pedestrians is 4 to segme hidden behind one parked vehicle at the side of the road, the almost the same as the one of static objects. Thus, in the case of misclassified pedestrians, the sensor does not deliver wavespedestrians cannot hit the pedestrian directly. Therefore, 4.3 the Fa 80 3 4.2 radar Occluded statistically different data from static objects and hence the signal reflected by the pedestrian and received by the sendeveloped classification algorithm is optimal within the reIn this typesAs of occlusions were investigated. 450 False 430 sor paper, is verytwo weak. a result, the pedestrian blursInwith the ala 260 strictions of the chosen method for pedestrian recognition. sified as the blocking first case,car a pedestrian hidden behind a single parked and oftenisthe segmentation algorithm is not able Firstly, p 420 car to and in the second the pedestrian is in the gapstep be- the true seperate the twocase objects. In the classification 140 icant fra tween two parked cars (see Fig. 13). In both cases the positive rate drops to rTP = 6.6%, which is only slightly bet4.2 Occluded pedestrians a higher pedestrian moves laterally while being occluded such that ter than the false alarm rate of r = 1.4%. DDoppler true positive rate / % / s/m

DDoppler / s/m

true positive rate / %

5

020

DDoppler / s/m

415

420

FP

If the pedestrian is moving laterally in a gap between two 0.5 types of 1 occlusions 1.5 were 2investigated. 2.5 In 3 435 In this 0paper, two relative radial velocity parked cars at the side of the road (see Fig. 13), the radar the first case, a pedestrian is hidden behind/ m/s a single parked wave can propagate in this gap because of multiple refleccar and0 0in the second case the pedestrian is in the gap be20 40 60 80 90 tions. Thus, the pedestrian reflects more power back to the tween two parked cars (see Fig. 13). In both cases the angle of movement / ° Fig. 12. Averaged of correct classified andthat incorradar sensor and can be segmented properly in the radar raw pedestrian moves Doppler laterally spectra while being occluded such rect classified pedestrians walking with various angles of move- Fig. 13. Top: A lateral moving pedestrian is occluded behind a 440 if theAgap is not smaller than 50 cm. Also, the recognithe pedestrian is not in the line of sight to the sensor. The Fig. data 13. Top: lateral moving pedestrian is occluded behind a ment. The similarity of Doppler spectra of incorrect classified single parked car. Bottom: Now the lateral moving pedestrian is in lateral is typical forfor pedestrians try to cross Fig. 11.movement Classification result rTP pedestrianswho walking with difsingle parked car.isBottom: Now the lateral moving pedestrian is in to valtion result much better, as the true poistive rate rises pedestrians and static objects is a limitation of the radar sensor in a gap between two parked cars, where the radar wave can propagate ferent angles of movement ϕ. a gapues between parked cars, where wave canbecause propagatenow the the street without looking. up totwo 29.4 %. This resultthe is radar remarkable due to multiple reflections. pedestrian recognition, since the sensor does not deliver statistical due to multiple reflections. In the case where the pedestrian is in the shadowed area recognition rate is only ten percent less than rTP = 39.5% in different in this case. hidden the case of a lateral moving pedestrian in line of sight (see 6 behind one parked vehicle at the side of the road, the radar waves cannot hit the pedestrian directly. Therefore, correct classified pedestriansthe the pedestrian is not in the line results of sightfor to different the sensor.sizes Thel of the 445 Fig. 11). The recognition signal by the pedestrian and is received by the sensor the pedestrian reflects more power the r is wrong classified pedestrians lateral movement isintypical forNote pedestrians who try totocross 5 reflected developed classification algorithm optimal within the re- tions. gapThus, are shown Tab. 3. that the falseback alarm rate FP static objects is very weak. As a result, the pedestrian blurs with the blockradar sensor and can be segmented properly in the radar raw 425 the street without higher looking.in case of narrow gaps up to 50 cm due strictions of the chosen method for pedestrian recognition. significantly ing car and often the segmentation algorithm is not able to data thecase gapwhere is notthe smaller than 50 cm. the recogIntoifthe pedestrian is blurred in theAlso, shadowed area 4 segmentation problems in the occluded areas. seperate the two objects. In the classification step the true nition result is much better, as the true poistive rate risesthe to hidden behind one parked vehicle at the side of the road, positive rate drops to rTP = 6.6%, which is only slightly betvalues up to 29.4 %. This result is remarkable because now radar4.3 waves cannot hit the pedestrian directly. Therefore, the 4.2 3Occluded pedestrians False alarms ter than the false alarm rate of rFP = 1.4%. the recognition ten percent than rTP 39.5% signal reflectedrate by is theonly pedestrian andless received by=the senIf the pedestrian is moving laterally in a gap between two in the case of a lateral moving pedestrian in line of sight (see 430 sor is very weak. As a result, the pedestrian blurs with the clasIn this 2 paper, two types of occlusions were investigated. In 450 False alarms, meaning that static objects are incorrectly parked cars at the side of the road (see Fig. 13), the radar 11). The recognition results for different sizes l of the Fig. blocking car and often the segmentation algorithm is not able the first case, a pedestrian is hidden behind a single parked sified as pedestrians, are consequences of three root causes. wave can propagate in this gap because of multiple reflecgap are shown two in Table 3. Note that the false alarm rate true rFP to seperate objects. In the Firstly, the parasitic antenna sideclassification lobe effectsstep canthe cause signifcar and 1 in the second case the pedestrian is in the gap bedrops toinrTP 6.6%, whichobjects is only slightly bet-to have icantrate fractions the=DS of static that seem tween two parked cars (see Fig. 13). In both cases the positive the false alarmthan rate of rFP objects = 1.4%.should have. Secondly, Adv. Sci., 10,laterally 45–55, 2012 www.adv-radio-sci.net/10/45/2012/ a higher velocity static pedestrian while being occluded such that ter than 0Radio moves If the pedestrian is moving laterally in a gap between two 0 0.5 1 1.5 2 2.5 3 435 relative radial velocity / m/s parked cars at the side of the road (see Fig. 13), the radar wave can propagate in this gap because of multiple reflec-

A. etal.: al.:Pedestrian Pedestrian recognition using automotive A.Bartsch Bartsch et recognition using automotive radarradar sensorssensors Table result for for aa lateral lateral moving movingpedestrian pedestrianinina a Table 3. 3. Classification Classification result gap cars. gapbetween between two two parked parked cars. 480

Size Sizeof of gap gap ll

30 cm

50 cm

70cm cm 70

90cm cm 90

110cm cm 110

rTP TP

20.9 %

% 29.4 %

28.5% % 28.5

20.1% % 20.1

25.2%% 25.2

rFP FP

17.6 %

9.5 %

2.4% % 2.4

5.4% % 5.4

1.4%% 1.4 485

490

495

Fig. 14. A real urban traffic scene used for measuring the false 500 Fig. 14. A real urban traffic scene used for measuring the false alarm rate. On the left a photography of the scene is depicted and alarm rate. On the left a photography of the scene is depicted and on the right the corresponding intensity image from the radar raw on the right the corresponding intensity image from the radar raw data. data. 505 455

460

465

470

475

in the ego velocity compensation canup offset iserrors significantly higher in case of narrow gaps to 50the cmDS due and finally, in case of errors in the segmentation step, resoto segmentation problems in the blurred occluded areas. lution cells from the noisy surroundings of an object can get assigned to alarms the object, resulting in DS fractions with higher 4.3 False velocity than possible for static objects. To see how good the 510 specifity of the pedestrian system is in practice, False alarms, meaning that recognition static objects are incorrectly clasa real urban traffic scene was chosen (see Fig. vesified as pedestrians, are consequences of three14). rootThe causes. hicle drives along a road with lots of highly reflective static Firstly, parasitic antenna side lobe effects can cause signifobjects, such asincars, whichtois have the icant fractions the in DStheofmeasurement static objectsrange, that seem worst case scenario in terms of false alarms, since highly re- 515 a higher velocity than static objects should have. Secondly, flective objects cause high amplitudes of antenna side lobe errors in the ego velocity compensation can offset the DS effects. As in a result, the described traffic scenario, a false and finally, case ofin errors in the segmentation step, resoalarm rate of r = 4.9% was obtained. FP lution cells from the noisy surroundings of an object can get assigned to the object, resulting in DS fractions with higher 4.4 Driver assistance scenarioTo see how good the520 velocity than possible forsystem static objects. specifity of the pedestrian recognition system is in practice, practical of the introduced pedestrian recognition aAreal urbanuse traffic scene was chosen (see Fig. 14). Thesysvetem would be collision avoidance by warning the driver static of a hicle drives along a road with lots of highly reflective pedestrian which moves towards the road, range, occluded in aisgap objects, such as cars, in the measurement which the in a row of roadside parked cars. This scenario consists worst case scenario in terms of false alarms, since highly of re-525 parked cars as highly reflective static objects, lateral moveflective objects cause high amplitudes of antenna side lobe ment of the pedestrian, occlusion and linear movement of the effects. As a result, in the described traffic scenario, a false measurement vehicle with about 8.5 m/s. With the combinaalarm rate of rFP = 4.9% was obtained. tion of all those influences in one real scenario, a recognition rate of rTP = 19.3% was achieved. The false alarm rate in 530 this particular measurement was rFP = 3.2%.

www.adv-radio-sci.net/10/45/2012/

9 53

Driver assistance system scenario 5 4.4Discussion A practical of the introduced pedestrian recognition The gathered use results reveal three main development areas forsysfuture radar sensors used for pedestrian tem would be collision avoidance byrecognition: warning theDoppler driver of a frequency antenna side lobes and occluded spatial resolupedestrianresolution, which moves towards the road, in a gap tion. in a row of roadside parked cars. This scenario consists of Doppler frequency resolution: object at the position parked cars as highly reflective An static objects, lateral move(r, φ) with relative velocity v to the radar sensor (see Fig. of 9) the r ment of the pedestrian, occlusion and linear movement −1 has a relative radial velocity component given by measurement vehicle with about 8.5 m s . With the combination all those vrr = vr ·of cos(ϕ − φ) .influences in one real scenario, a recogni(10) tion rate of rTP = 19.3% was achieved. The false alarm rate (Equation (9) is themeasurement approximation this= equation in this particular wasofrFP 3.2%. assuming φ = 0). If the object moves on a straight lateral path (ϕ = 90◦ ), it still causes a small Doppler shift given by vrr vr ·sinφ with absolute maxima at the azimuthal borders 5 =Discussion ˆ The described experimenof the measurement range ±φ. talThe results for recognizing pedestriansareas (see for gathered results reveallateral three walking main development Sect. 4.1) are based on a walking pace of about v = 1.2m/s. r future radar sensors used for pedestrian recognition: Doppler The theoretical maximum of the radial relative velocity comfrequency resolution, antenna side lobes and resoluˆspatial ponent in this case is vrr = 0.17m/s at |φ| = φ, which is tion. only about a fifth of the smallest measurable radial velocity Doppler frequency resolution: An object at the position of 0.77 m/s, given by the radar sensor’s Doppler frequency (r, φ) with relative velocity vr to the radar sensor (see Fig. 9) measurement resolution. Nevertheless, in the experiment the has a relative component giventhan by the exrecognition rateradial of rTPvelocity = 39.5% is much higher pected rate of about rTP ≈ 4%, caused by false alarms, even vr · cos(ϕ is −central φ) . in front of the sensor (φ = 0) where(10) rr = ifvthe pedestrian Eq. (10) gives vrr = 0. This leads to the conclusion that miEquation 9 is the approximation of this equation assuming cro Doppler effects (see (He, 2010)), e.g. swinging arms and ◦ φ =cause 0. If the objectinmoves onwhich a straight path (ϕ = 90 ), legs, artefacts the DS, leadlateral to these classificait still causes a small Doppler shift given by v = vr · sinφ rr tion results. with absolute maxima at the azimuthal borders of the meaFor a particular driver assistance system, a use case could ˆ The described experimental results for surement range ± φ. be to require direct measurement (without the help of micro recognizing lateral walking pedestrians (see Sect. Doppler effects) of a lateral walking pedestrian’s radial4.1) ve- are −1 . The theo◦ based on a walking pace of about v = 1.2m s r 3 , given a walking locity component vrr in a range of |φ| ≥ retical of the radial relative velocity pace of maximum vr = 1.0m/s. This requirement gives a component blind spot in −1 at |φ| = φ, ˆ this case is v = 0.17m s which is only of 5.2 m widthrrcentered in front of the vehicle in 50 m about dis- a fifth of the smallest measurable radial velocity of 0.77 m s−1 , tance. This blind spot could be observed by other sensor given by the it radar sensor’s Doppler frequency measurement systems since is usually directly visible. To fulfill these resolution. Nevertheless, in the experiment the requirements, a radar sensor with vrr -measurementrecognition resolurateofofatrleast 39.5%m/s is much higher thanDoppler the expected rate of tion or, respectively, frequency TP =0.052 about rTP ≈ resolution 4%, caused even the pedesmeasurement of by 26.5false Hz isalarms, needed. Theifrequired measurement resolution fifteen times higher than the(10) trian is central in frontisofabout the sensor (φ = 0) where Eq. resolution of 0. theThis radar sensor employed in this gives vrr = leads to the conclusion thatpaper. micro Doppler Antenna side lobe effects:e.g. Parasitic effects because of aneffects (see (He, 2010)), swinging arms and legs, cause tenna side in lobes the intensity (ghost targetsresults. apartefacts the affect DS, which lead to image these classification pear)For as awell as the driver frequency image system, (see Sect. 4.3),case espeparticular assistance a use could cially the surrounding of highly reflective be toinrequire direct measurement (without objects. the help The of miimpact of antenna side lobes on the intensity was modcro Doppler effects) of a lateral walking image pedestrian’s radial eled usingcomponent a measuredvantenna pattern. Although in theory ◦ , given |φ| velocity in a range of ≥ 3 a walkrr compensation this effect be possible, further im−1would ing pace of vof = 1.0m s . This requirement gives a blind r provements sensors concerning lobesinare spot of 5.2ofmtodays widthradar centered in front of theside vehicle 50 m needed. The knowledge of these effects is the key to optimize distance. This blind spot could be observed by other sensor the segmentation algorithm. systems since it is usually directly visible. To fulfill these reSpatial resolution: If a pedestrian is in line of sight and not quirements, a radar sensor with vrr -measurement resolution more than about 35 m distant from the sensor, the radar senof at least 0.052 m s−1 or, respectively, Doppler frequency measurement resolution of 26.5 Hz is needed. The required Adv. Radio Sci., 10, 45–55, 2012

54


measurement resolution is about fifteen times higher than the resolution of the radar sensor employed in this paper. Antenna side lobe effects: Parasitic effects because of antenna side lobes affect the intensity image (ghost targets appear) as well as the frequency image (see Sect. 4.3), especially in the surrounding of highly reflective objects. The impact of antenna side lobes on the intensity image was modeled using a measured antenna pattern. Although in theory compensation of this effect would be possible, further improvements of todays radar sensors concerning side lobes are needed. The knowledge of these effects is the key to optimize the segmentation algorithm. Spatial resolution: If a pedestrian is in line of sight and not more than about 35 m distant from the sensor, the radar sensor’s spatial resolution is sufficient. However, if the pedestrian is occluded behind other objects, the corresponding radar data starts to blur with nearby objects. A finer spatial resolution might improve the separability of the objects in this case. If the pedestrian is further away from the sensor, the spatial resolution cells grow, since their width is a function of the azimuth angle. Hence, a distant pedestrian covers only few resolution cells, resulting in a DS of just few single peaks, which might not represent the DS well. A finer spatial resolution would improve the accuracy of the DS in this matter.

6

Summary and conclusions

Because pedestrians are point targets for radar sensors, object features from the intensity image cannot be used to distinguish between pedestrians and small static objects. Hence, information from the Doppler frequency image is needed. The DS, which is the histogram of the measured radial relative velocities of an object, turned out to be a stable object feature, holding the required information to reliably seperate pedestrians from static objects. A simple classification algorithm, based on five object features, was found with classification results compareable to modern machine learning algorithms. Radial moving pedestrians in line of sight can be recognized up to 95.3 % of the samples. The recognition rate drops to 39.5 % for lateral moving pedestrians and to 29.4 % for lateral moving pedestrians occluded in a gap between two parked cars. In case of misclassifications of pedestrians as static objects, the radar sensor does not deliver statistically different data as from static objects, making it impossible for classification systems (machine learning as well as the proposed solution) to decide for the correct class using only information from single radar raw data frames. Hence, pedestrian recognition using only single radar raw data frames shows poor classification results. The radar sensor’s main limitations in this use case turned out to be insufficient Doppler shift measurement resolution and antenna side lobe effects. Possibly, a finer spatial resolution would also improve the sensor’s performance in recAdv. Radio Sci., 10, 45–55, 2012

ognizing occluded pedestrians. Our findings indicate that pedestrian recognition using this low-level approach is limited. Only radar data with higher Doppler frequency measurement resolution (independent of the used technology, e.g. 24 GHz, 77 GHz) would make major recognition improvements possible. This requirement of future automotive radar sensors is in line with claims in (Rasshofer, 2007). In state-of-the-art research on pedestrian recognition, today’s radar sensors are mostly used to support camera-based systems (Gerónimo, 2010). References Benitez, D. and Zhaozhang, J.: Method for Human Only Activity Detection Based on Radar Signals, U.S. Patent Application US2011/0032139A1, 2011. Büchele, M.: Optimierte Radarsignalverarbeitung für Fahrerassistenzsysteme, M. Sc. thesis, University of Ulm, Institute of Theoretical Computer Science, Ulm, Germany, 2008. Freund, S.: Object Recognition from Radar Raw Data Using Image Processing Methods, Diploma thesis, Ludwigs-MaximiliansUniversität Munich, Institute of Computer Science, Munich, Germany, 2007. Gerónimo, D., López, A. M., Sappa, A. D., and Graf, T.: Survey of Pedestrian Detection for Advanced Driver Assistance Systems, IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, pp.1239–1258, 2010. He, F., Huang, X., Liu, C., Zhou, Z., and Fan, C.: Modeling and Simulation Study on Radar Doppler Signatures Of Pedestrian, in: Proc. of the 2010 IEEE Radar Conference, Washington DC, USA, 10–14 May 2010, 1322–1326, 2010. Hornsteiner, C. and Detlefsen, J.: Characterisation of human gait using a continuous-wave radar at 24 GHz, Adv. Radio Sci., 6, 67–70, doi:10.5194/ars-6-67-2008, 2008. Kim, Y. and Ling, H.: Human Activity Classification Based on Micro-Doppler Signatures Using an Artificial Neural Network, Proc. of the 2008 IEEE Symposium on Antennas and Propagation, San Diego, USA, 5–11 July 2008, 1–4, 2008. Kouemou, G. and Opitz, F.: Impact of Wavelet Based Signal Processing Methods in Radar Classification Systems Using Hidden Markov Models, in: Proc. of the 2008 International Radar Symposium, Wroclaw, Poland, 21–23 May 2008, 1–4, 2008. Nanzer, J. A. and Rogers, R. L.: Bayesian Classification of Humans and Vehicles Using Micro-Doppler Signals From a ScanningBeam Radar, IEEE Microwave and Wireless Components Letters, 19, 338–340, 2009. Rasshofer, R. H.: Functional Requirements of Future Automotive Radar Systems, in: Proc. of the European Radar conference, Munich, Germany, 10–12 October 2007, 259-262, 2007. Ritter, H. and Rohling, H.: Pedestrian Detection Based on Automotive Radar, 2007 IET International conference on Radar Systems, Edinburgh, UK, 15–18 October 2007, 1–4, 2007. Rohling, H., Heuel, S., and Ritter, H.: Pedestrian Detection Procedure Integrated into an 24 GHz Autmotive Radar, in: Proc. of the 2010 IEEE Radar Conference, Washington DC, USA, 10–14 May 2010, 1229–1232, 2010.


A. Bartsch et al.: Pedestrian recognition using automotive radar sensors Wenger, J. and Hahn, S.: Long Range and Ultra-Wideband Short Range Automotive Radar, in: Proc. of the 2007 IEEE International Conference on Ultra-Wideband, Singapore, 24–26 September, 2007, 518–522, 2007.


55

Yamada, N., Tanaka, Y., and Nishikawa, K.: Radar Cross Section for Pedestrian in 76 GHz Band, in: Proc. of the 35th European Microwave Conference, Paris, France, 3–7 October 2005, 1–4, 2005.

Adv. Radio Sci., 10, 45–55, 2012