WiFi positioning and Big Data to monitor flows of people on a wide scale Alfredo Alessandrini1 , Ciro Gioia1 , Francesco Sermi1 , Ioannis Sofos2 , Dario Tarchi1 , Michele Vespe1 1
European Commission, Joint Research Centre (JRC), Directorate for Space, Security and Migration, Demography, Migration & Governance Unit, Via E. Fermi 2749, I-21027 Ispra (VA), Italy 2 National Technical University of Athens, School of Rural and Surveying Engineering, Athens, Greece. Email: {alfredo.alessandrini, dario.tarchi, michele.vespe, ciro.gioia and francesco.sermi }@jrc.ec.europa.eu ,
[email protected] Abstract—The possibility to count the accesses to a site and monitor the internal movements of people can be useful in many different scenarios. In this respect, a WiFi network can be exploited to count accesses and estimate users position. This study extends this principle to a wide spatial area and to a large number of users, introducing synergies between Big Data and localization techniques. The 2016 Open Day of the Joint Research Centre (JRC), Ispra (Italy), was a good opportunity to investigate the potential of Big Data and positioning techniques. During the event, which counted the participation of some 8000 people within an area of about 167 hectares, 20 WiFi access points, scattered across the site, recorded the access of wireless devices, such as smartphones and tablets, belonging to visitors and volunteers. By exploiting the Media Access Control (MAC) address (the device unique identifier) through a data-cleaning process, the data analysis allowed estimating the number of participants to the event and the space/time evolution of their position. Moreover, the visitors flow was reconstructed using a Weigthed Centroid (WeC) algorithm. The results achieved, in terms of number of participants, confirmed the data of the JRC registry report compiled at the entrance points of the area. In addition, the results relative to the people flow within the site were found compatible with the scheduling of the event and with its actual progress. Keywords—Big Data, WiFi, RSS, Localization
I.
I NTRODUCTION
The possibility to count people accessing a given area and to monitor their internal flow is potentially useful in many different scenarios. For instance, if the given area is a city square and the event gathering people is a demonstration, then the real time monitoring of the connections to a distributed network of WiFi access points, not only would allow estimating the number of participants, but it could help preventing critical concentrations in bottlenecks and improving security and safety opreations. Analogously, in case the area is a stadium or a concert arena, the understanding of people access to the site and movements within it would help positioning a series of commercial services and also optimizing the site Wireless Fidelity (WiFi) network according to the users density. Smartphones, tablets and many other devices use wireless
connectivity. Whenever, one of these tools is active, the device broadcasts probe requests to identify known networks. For each device, the probe request contains a unique identifier: the Media Access Control (MAC) address. A MAC address is a 12characters hexadecimal identifier: the first 6 digits identify the manufacturer, while the remaining digits identify the device. This identifier is visible to a network whenever the user is in its range; so, the MAC address can potentially be used to collect information on the activities of users, tracking the movements of the device. For example, if the position of WiFi access points is known, it is possible to determine the location of a user when a device (such as a smart-phone, tablet or laptop) interacts with the WiFi network. MAC address are commonly used to monitor the Received Signal Strength (RSS) and to estimate the distance of the device from the access point. The limits of the RSS measurements are shown in [1]. The authors identify the propagation model as the main issue for the usage of RSS measurements in the indoor positioning. In order to fill this gap, different strategies have been proposed. For example, in [2] WiFi fingerprinting, together with map information, are used for the localization of a robot. This approach requires an a priori knowledge of the environment and a database containing the RSS measurements collected at different points. The fingerprinting concept is further investigated in [3], where the authors develop an application for indoor localization. In [4], indoor localization using WiFi, with centimeter level accuracy, is demonstrated; the algorithm exploits Angle of Arrival (AoA) and a network of 20 WiFi access points to cover an area of 50 × 40 meters. This approach allows to properly identify the location of the users, using a complex algorithm and an expensive network of devices. While, all the mentioned approaches exploit simultaneous measurements, in [5] asynchronous measurements are used to demonstrate indoor positioning with meter accuracy. Moreover, in order to enhance the performance of localization techniques based on RSS measurements, additional sensors have been exploited. For example, in [6], data fusion between RSS and Inertial Measurement Unit (IMU) data is demonstrated, whereas the fusion between asynchronous RSS observables and Global Navigation Satellite System (GNSS) data is investigated in [7].
The above mentioned researches deal with small chunk of data. The added value of this study is the synergy between Big Data and localization techniques. In [8], the author identifies the three Vs as the main characteristics of Big Data: Volume, Velocity and Variety. This study considers more than 12 millions of records collected with various data rate and relative to users with different kinematics and different nature devices. Finally, the considered data-set is unique, because built on data collected within a large open area, such as the Joint Research Centre (JRC) site in Ispra, with a high number of users (almost 8000 people), during the 2016 JRC Open Day [9]). The data collected contains time references, MAC addresses and received power (RSS). Classical RSS localization algorithms cannot be used to localize the users due to the lack of simultaneous measurements: a user is not simultaneously registered by different stations. Moreover, in order to use RSS for trilateration, a model is necessary to map RSSs into distances. The classical model contains unknown parameters related to the transmitted power and to the propagation loss, which are device and environment dependent; hence such model requires an a priori calibration procedure [10], which is unfeasible in this case. Therefore, in order to localize the users, an algorithm based on the Weigthed Centroid (WeC) has been developed. The algorithm is not affected by the synchronization problem, hence no simultaneous measurements are required and no calibration is needed. The results achieved with the proposed approach were compared with the official data reported by the JRC’s security office [9], which were collected by accesses counting at the site’s gates. Among the well known issues about Big Data there are the validation and verification of the analysis. In the propesed case study, validation and verification of the results were possible using the security report and the schedule of the event. The comparison shows that the proposed method leads to a proper identification of the number of users, allowing also to build heat map relative to the users concentration in different time slots. The remainder of this paper is structured as follows: Section II briefly presents the privacy-related aspect of the research; in Section III the algorithm is described; Section IV illustrates the architecture of the system and the data collection; the data and the results are analyzed in Section V; finally Section VI concludes the paper. II.
P RIVACY A SPECTS
Personally Identifiable Information (PII) or personal information is the data content that can be exploited to identify an individual. Although the PII concept is commonly shared, it is very difficult to find a uniform definition of personal information. For example in [11], the Canadian minister of Justice defines personal information as “information about an identifiable individual”. A clarification to this definition has been provided by the Office of the Privacy Commissioner of Canada (OPCC), which includes in the personal information, information that “relates to or concerns” a data subject. Internet Protocol (IP) and MAC addresses are Achilles’ heel of the privacy in the Internet of Things (IoT) era, since they are not easily classifiable as PII. For example, US approach does not provide a clear view on IP and MAC addresses which should be considered personal information. Also, [12]
does not provide a clarification about IP and MAC addresses, which should be considered as personal information when they can be reasonably linked to a particular person. The issue relative to IP and MAC addresses is discussed by the OPCC, which provides a similar conclusion to that presented in [12]: “if they can be associated with an identifiable individual, they should be considered personal information”. In Europe, [13] identifies as personal information “any information relating to an identified or identifiable natural person”. De facto among the EU member states there is no uniformity on IP. However, the European Court of Justice (ECJ) decided that IP addresses are personal information “because they allow users to be precisely identified”. However the identification or not of MAC address as personal information is difficult, because the MAC address is uniquely connected to a device and it can be linked to an individual only through a database. Hence, MAC address may constitute personal information if it is associated with or linked to an identifiable individual. However, often MAC addresses may not be directly linked to individuals, so MAC address does not disclose who actually has possession of the device. Given the difficultis in associating an individual to a MAC address and the practical impossibility to determine PII from MAC addresses, this type of information is used in this research. III.
A LGORITHM
In this section, the algorithm developed for user localization of flow analysis is presented. The diagram of the algorithm is shown in Fig. 1 A dataset containing MAC addresses, Received Signal Strength Indicator (RSSI), the time stamp and the ID of the access points, was collected and stored. In order to analyze the data, different Matlab scripts have been developed. Specifically, the blocks Import data, Data cleaning, Data analysis and Positioning WeC are based on algorithm implemented in Matlab. Whereas, the heat maps are generated using QGIS. The data cleaning procedure and the positioning approach are described in Section III-A and Section III-B, respectively. A. Positioning Due to the lack of simultaneous measurements, an asynchronous RSS positioning algorithm is used. The approach is based on the proximity concept. The most simple form of positioning based on proximity consists in estimating the user position as that of the node associated to the strongest received signal. The WeC approach is an extension of the proximity principle, where the user position is a linear combination of the nodes coordinates: PN −1 wi Psta,i Pu = (xu , yu ) = i=0 , (1) PN −1 i=0 wi where Psta,i = (xi , yi ) is the vector containing the coordinates of the ith station and wi is the weight associated to the ith node. In this work, the weights are related to the RSSI of the received signal, in particular the following weighting function is adopted: wi = 1/ 2 · 10(−RSSI)i /10 , (2)
Data Collection
node. One example of this second case is represented by Apple devices, which, starting from iPhone Operating System (iOS) 8, are able to randomize their MAC address when broadcasting WiFi probes. Both types of records could jeopardize the performed analysis, so they need to be identified and then removed from the data-set. In order to screen out static devices and devices broadcasting fake MAC addresses, two criteria have been applied. In particular, for the first case, a real user is identified only if his device has been recorded at least in three different stations. Whereas, to remove the fake MAC addresses, a user is declared real only if the MAC address of his device has been recorded at least five times. The diagram of the algorithm used to ‘clean’ the data is shown in Fig. 2.
Heat map
Data Extraction .csv file containing MAC, RSSI, time, base ID
Flow Analysis
Import data
Users Position
.mat file containing MAC, RSSI, time, base ID
Data Cleaning
Positioning WeC
IV.
.mat file containing MAC, RSSI, time, base ID
Data Analysis Fig. 1.
Data were collected during the 2016 JRC Open Day . The event was a unique opportunity to analyze people access to a large restricted area and to monitor its flow within the same area. The Ispra site, the third biggest Commission site after Brussels and Luxembourg, covers an area of 167 hectares with 138 buildings.
Statistics
Diagram of the algorithm developed.
meters
where (RSSI)i is the RSS of the ith received signal expressed in dB. Usually RSS-based positioning is carried out using simultaneous measurement. Nevertheless, in the considered case, each user is seen only by one node at the time, hence the classical RSSI-based positioning needs to be modified. The adopted approach considers RSSI measurements collected in a given time interval; specifically a 3 minutes interval is considered. The user position is obtained as the WeC of the nodes coordinates. If the time interval is reduced and only one measurement is obtained in the time intervalconsidered, the WeC solution converges to the proximity solution.
A large amount of data (almost 12 millions of records) was collected, including records not relative to real users. In order to identify the records relative to real users, a first data inspection has been carried out. During this phase, two main issues were encountered: the first one is the presence of data relative to static devices, such as PC printers etc, which tried to connect to the access points; the second one is relative to devices generating fake MAC addresses before connecting to a 50k registered user About 12M of records
7143 Users About 10M of records
User recorded by at least 3 stations
Fig. 2.
Removal of Static Devices
Removal of Fake Devices
User recorded at least 5 times
Diagram of the algorithm used to ‘clean’ the data.
700
Brebbia Gate
Exit Area
External Fencing
B. Data inspection
Collection of Data
E XPERIMENTAL S ETUP
Fig. 3. site.
Entrance Area
Displacement of the 20 Wi-Fi access points within the JRC Ispra
For the occasion, 20 Wi-Fi access points were suitably displaced within the site (Fig. 3). In order to quantify the amount of people entering the site, four access points were strategically positioned: numbers 10 and 11 at the main entrance; number 12 at the gate reserved to the access of volunteers and number 16 in proximity of the Brebbia gate a secondary entrance to the site. Analogously, with the purpose to monitor the people outflow, three access points (numbers 26, 27 and 30) were placed in proximity of the pedestrian gate; which was the only exit gate. The coordinates of the access points are detailed in Table I. The devices were placed before the opening, with an initial installation phase during which all the devices were turned on in the same place and then displaced within the site. The effect of this installation phase is visible in the results discussed in the following section. The devices were equipped with batteries and memory card
C OORDINATES OF THE 20 W I -F I ACCESS POINTS WITHIN THE JRC I SPRA SITE . Longitude 8.629811 8.629255 8.630529 8.637728 8.631023 8.639771 8.633691 8.631397 8.629757 8.627225 8.627685 8.630384 8.629223 8.630108 8.626288 8.624709 8.625349 8.626753 8.623548 8.621281
ID station 10 11 12 13 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
# Entering People
Latitude [deg] 45.803818 45.803819 45.803651 45.808041 45.810159 45.813099 45.809422 45.805916 45.805168 45.806339 45.807266 45.809311 45.812019 45.810419 45.809650 45.809361 45.810600 45.810936 45.811339 45.810057
Exit of Volunteers 5000 4000
Maximum Slope
3000 2000 1000 0 9
10
11
12
13
14 15 Local Time
16
17
18
19
20
10
11
12
13
14 15 Local Time
16
17
18
19
20
6000 5000 # Exiting People
TABLE I.
4000 3000 2000 1000 0 9
Fig. 4. Total number of people as function of the local time. In the upper box, the number of people entering on the site whereas in the lower box the people leaving the site.
to store the data. Although, few devices stopped to work during the experiment, almost all the access points continuously recorded data for the entire duration of the event. The unexpected behavior of the faulty devices did not significantly affect the global results of the experiment. Each device stored the MAC address of the user which connected to the access point, together with the RSSI and the time epoch. This data are used to localize the user position.
3500
3000
# Users
2500
2000
1500
1000
V.
R ESULTS
In this section, results emerged from the data analysis are discussed. From the initial amount of data, more than 50000 users were identified; this number accounts for: mobile devices (smartphones and tablets), static devices (PCs, printers, etc.) and fake MAC addresses. After applying the described datacleaning criteria, a total of 7143 “real” users was identified. This value under-estimates the correct number of people present on the JRC Ispra site during the 2016 Open Day. In fact, according to the security report [9], the final number of guests on the site was 7623, which grows up to almost 8000 considering volunteers and security staff. Hence, almost 10% of people was not included in the real users estimate. The most probable cause of this discrepancy is the presence of guests without a smartphones (i.e. infants or people with old fashion cell phones) and of guests whose smartphone had the WiFi turned off. However, the number identified with the proposed approach is very close to the actual one. In Fig. 4, the total number of people as a function of the local time is shown. From the upper box, showing the number of people entering the site, it clearly emerges that the highest number of accesses was recorded between 10:00 and 11:00. The plot is consistent with that shown in [9]. In the lower box of Fig. 4, which shows the number of people leaving the site, the flow exiting the site is higher than the number of people entering. This phenomenon is probably due to the fact that part of the people that entered the site with the WiFi turned off, activated the connectivity once inside, so their phones MAC was not recorded at the entrance. The plot relative to people leaving the site was not available in [9].
500
0
9
10
11
12
13 14 Local Time
15
16
17
18
Fig. 5. Number of user registered in the proximity of the entrance, station number 10.
The number of users registered per hour is shown in Fig. 5 and Fig. 6. In particular, in Fig. 5, the number of users recorded in proximity of the main entrance (access point number 10) is shown as a function of the local time. It can be noted that, for this access point, the maximum number of people was registered between 10:00 and 11:00, in correspondence with the official opening of the site. A high number of entering people was also registered between 9:00 and 10:00. In fact, while the official opening of the main gate was scheduled for 10:00, an early opening was performed around 9:00 to facilitate the entering in the site. A few people in proximity of the entrance were detected also after 17:00, when the main gate was opened to allow the exit of volunteers. In Fig. 6, the number of users registered in proximity of the exit (station number 27) is shown as a function of the local time. It can be noted that the maximum number of people in proximity of the exit was registered between 15:00 and 16:00. In order to better analyze the flow of people entering and leaving the site, three main areas have been identified: •
entrance area, which includes stations 10, 11 and 12;
•
exit area, which includes stations 26 27 and 30;
•
and finally, central area, which contains all the re-
1000 3000 900 2500
800
Permanence Time
700
# Users
2000
1500
1000
600 500 400 300 200
500 100 0
Fig. 6.
9
10
11
12
13 14 Local Time
15
16
17
0
18
Number of users registered in the proximity of the exit.
Fig. 8.
4
6
8
10 Station
12
10 11 12
100
20
3500
3000
0
8 Installation Phase
9
10
11
12
13 Local Time
14
15
16
17
Central Area
300 200 100 0
8
9
10
11
12
13 Local Time
14
15
16
17
Exit Area 200
2500
2000
500
47 0 0
8
9
10
11
12
13 Local Time
14
15
16
Brebbia Gate
1500
1000
26 27 30
100 0
13 15 16 17 18 20 21 22 23 24 25 29
Number of users
# Users
18
3949
Entrance Area
# Users
16
Gate 4000
200
# Users
14
Median permanence time as function of the station.
Official Opening
Early Opening
2
10
11
12
13
15
16
17
18
19
20 21 Station
22
23
24
25
26
27
28
29
30
17
Fig. 9. Fig. 7. People flows in the three areas: entrance (upper box); central (central box); exit (bottom box).
maining stations. The people flow as a function of local time is shown in Fig. 7 where the number of users was computed considering an interval of 3 minutes. From the figure, the installation phase can be identified; peak common to all the stations is present. For the entrance area a growing number of people was recorded in the first hours of the opening, due to the queues at the gates. From the central box, the anomalies of some access points can be noted: the number of users recorded by the staion number 25 dropped suddenly at 12:30 from 200 to less than 100. The opposite behaviour can be noted before 15:00 when the user number grows istantaneously. The permanence time at each access point is shown in Fig. 8. This parameter can be affected by volunteers, which were almost permanently connected to the same stations. Hence, the median of the permanence time needs to be computed. From the figure, it clearly emerges that the entry of people was properly managed: the median permanence time at station 10 (first bar in Fig. 8) was about 8 minutes. This value can be considered as a median waiting time for the entrance to the site. Higher permanence time, close to 30 minutes, were registered at the stations located in the central area. The ranking of the most popular stations is shown in Fig. 9, where the total number of users for each station is shown.
Total number of users for each station.
From the figure, it emerges that the station visited from more users is the number 22, which registered 3949 users. However, also other stations registered a high number of users. The station with the minimum number of users (only 47) is the one placed in proximity of the Brebbia gate. The total number of user for each station is summarized in Table II. The results are consistent among the different access points. For example, considering stations 24 and 15, at the two opposite sides of alarge plaza, Piazza Lituania, the devices recorded almost the same number of users (with a difference of only 4 users). Sample positioning results are illustrated in Fig. 10. The upper boxes show the user positions in the morning. Users accessing JRC site from the Brebbia gate are visible only in the upper-left box. The Brebbia gate was open only during the first hours of the day, allowing the access of the volunteers only. The main routes within the JRC site can be easily identified from the plots. The upper boxes show a higher concentration of users in the entrance areas, whereas, in the lower boxes, the users are more concentrated in the exit area. Moreover a decreasing number of users can be remarked by comparing the upper boxes with the lower boxes.
TABLE II.
T OTAL NUMBER OF USERS REGISTERED BY THE ACCESS POINTS WITHIN THE JRC I SPRA SITE . ID station 10 11 12 13 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
North [m]
North [m]
400 200 500
1000
Fig. 11. Heat maps of the JRC Ispra site during the 2016 Open Day for different time spots.
show people concentrated in proximity of the exit gate. In particular, the bottom-right box shows: the volunteers leaving from the main gate (South); a concentration of presences in the northern part, where a late party for the volunteers (scheduled after the official closing at 17:30) took place.
400
1000
1000
800
800
600 400 200
-500
0 East [m]
500
1000
17-18
600 400 200
-500
0 East [m]
17 - 18
Volunteers Exit
600
0 -1000 User AP
North [m]
North [m]
16-17
Fig. 10.
16 - 17
200 0 East [m]
Main Gate
1000
600
0 -1000
External Fence
800
-500
10 - 11 Brebbia Gate
10-11 Users from Brebia
800
0 -1000
8-9 Exit
Number of users 3714 3083 550 1764 3455 47 1417 2526 3671 1239 1158 3949 2177 3451 3278 3235 3253 1498 2077 1219
8-9 1000
20 W I F I
500
1000
0 -1000
-500
0 East [m]
500
1000
Users positions computed in different hours.
A. Heat map In this section, the heat maps with the users concentration are presented. In particular the maps related to different time intervals during the day are shown in Fig. 11, where the user displacements can be identified. From the box on the upperleft part of Fig. 11(relative to the early opening) it emerges that the maximum concentration of users is recorded by the access points at the main gate and by those in the north-west part of the site (stations 23, 28 and 29). Moreover, a flow of people approaching the main gate outside the site can be noticed in the proximity of the western fence. Although these people were actually outside the site, they were registered as users because their devices connected to the access points in the proximity of the fencing. However, this phenomenon does not compromise the overall results. The upper-right box is relative to the official opening (10:00 to 11:00) and it shows an increasing concentration of users in the same two spots and both in the center of the site and along via Francia, which goes South to North in the Western part of the site. The lower boxes (16:00 to 17:00 on the left and 17:00 to 18:00 on the right)
VI.
C ONCLUSIONS
This research investigated the potential of WiFi positioning to monitor movement of people on a wide scale. The 2016 Open Day at JRC Ispra site represented an opportunity to reconstruct people flows within a large restricted area. The event attracted a high number of participants: almost 8000 people attended the event. During the event, an experiment was planned in order to demonstrate the feasibility of people flow monitoring through WiFi access points. Twenty access points were scattered across the site, recording the MAC address and the RSS of the connected portable devices (smartphones and tablets). The large number of records relative to different devices and with different data-rate allowed to create a data-set classifiable as Big Data. Among the well known issues about Big Data there are the validation and verification of the results. In the proposed case study, validation and verification of the results were possible using the security report and the schedule of the event. The experiment is significant, also because of the “fully controlled” nature of the site. Since the access points recorded not only connections relative to “real” users, a data-cleaning procedure was required. In fact, within the data-set, three types of users were identified: 1) static devices such as printers, PCs, etc; 2) fake users such as those due to devices able to randomize their MAC address when broadcasting WiFi probes; 3) real users. After the cleaning procedure, the user positions were computed exploiting a simple RSS based positioning algorithm. A total number of 7143 “real” users was identified, underestimating the actual value of guests registered by the security service (about 8000). Moreover the proposed approach allowed to reconstruct the distribution of people within the site in different time spots. Finally, the possibility to validate the results with independent information represents a significant added value to this research.
ACKNOWLEDGMENT The authors would like to thank Pietro Argentieri, Vincenzo Gammieri and Vladimir Kyovtorov for their valuable support. R EFERENCES [1] [2]
[3]
[4]
[5] [6]
[7] [8] [9] [10]
[11] [12] [13]
B. Bobescu and M. Alexandru, “Mobile indoor positioning using WI-FI localization,” Review of the Air Force Academy, 2015. J. Biswas and M. Veloso, “Wifi localization and navigation for autonomous indoor mobile robots,” in In IEEE International Conference on Robotics and Automation, 2010. J. Y. Zhu, A. X. Zheng, J. Xu, and V. O. Li, “Spatio-temporal (ST) Similarity Model for Constructing WIFI-based RSSI Fingerprinting Map for Indoor Localization,” in International Conference on Indoor Positioning and Indoor Navigation, 2014. M. Kotaru, K. Joshi, D. Bharadia, and S. Katti, “SpotFi: Decimeter Level Localization Using WiFi,” in ACM Special Interest Group on Data Communication (SIGCOMM), 2015. D. Borio, C. Gioia, and G. Baldini, “Asynchronous Pseudolite Navigation Using C/N0 Measurements,” Journal of Navigation, 2015. P. Tarrio, J. A. Besada, and J. R. Casar, “Fusion of RSS and Inertial Measurements for Calibration-Free Indoor Pedestrian Tracking ,” in 16th International Conference on Information Fusion, 2013. C. Gioia and D. Borio, “Stand-Alone and Hybrid Positioning Using Asynchronous Pseudolites,” Sensors, 2014. L. Douglas, “3d data management: Controlling data volume, velocity and variety,” in Gartner, 2001. R. P. SOUSA, “Event final numbers jrc open day 2016.” May 2016. D. Borio, C. Gioia, and G. Baldini, “Asynchronous Pseudolite Navigation Using C/N0 Measurements,” TheJournal Of Navigation, vol. 69, pp. 639–658, 2016. C. Minister of Justice, “Personal Information Protection and Electronic Documents Act,” tech. rep., Minister of Justice, 2000. Federal Trade Commission, “Children’s Online Privacy Protection Act,” tech. rep., 1998. European Commission, “Eu directive 95/46/ec - the data protection directive,” tech. rep., European Commission, 2016.
