funded by the European Commission in the eTEN program. ... consolidated technology (such as Business Intelligence applied to a Data Warehouse type model) ...
E2SP: a new environmental reporting platform in ASP mode Mauro Cislaghi1, Gaia Corbetta1, Roberto Mazzilli1, Emanuela Seregni1, Krzysztof Klejnowski2, Agata Bucko-Serafin3, Jerzy Jamrocha 3, Rafal Dunal4 (revisited by Jiri Hrebicek5)
The E2SP project: online reporting and data analysis for environmental agencies This paper aims at describing how the E2SP (Environmental Enterprise Service Provider) system supports the local and regional environmental agencies in fulfilling their daily reporting and data analysis activities, trough providing a cost effective Internet based data processing platform. The E2SP online reporting platform presented in this paper is partly funded by the European Commission in the eTEN program. The DPSIR model (Driving force – Pressure – State – Impact – Response), adopted Europe-wide as the reference model to manage and analyze the environmental issues, represents also the theoretical basis of the E2SP (Environmental Enterprise Service Provider) project funded by the European Commission in the e-TEN program. According to the DPSIR model and the legislative framework, all the environmental agencies in the Member States are in charge of measuring the State of the environment and evaluate the Impact; this implies a data analysis and reporting activity, which constitutes one of the core responsibilities of the agencies. The E2SP project offers a highly original approach to these activities trough the use of cutting edge information technologies, as it is based on the first environmental information system which combines the advantages of first-rate consolidated technology (such as Business Intelligence applied to a Data Warehouse type model) and innovative architecture tailored to environmental data management. The E2SP Environmental Information System supplies a wide range of environmental management tasks (such as integrating data, analysis, calculating indices and indicators, handling emission inventories, reporting, modelling, publishing information over the Internet, etc.) and tools for the integrated management and analysis of the different environmental issues pertinent to the territory (air, water, electromagnetism, emissions…). These features support efficiently the daily activities of different environmental public and private institutions, on different geographic scale from regional to local. Moreover, the sharing of a common set of operative and analytical tools and the usage of the same semantics can foster the adoption of homogeneous procedures for environmental decision support also at transboundary level. The E2SP platform delivers the above listed functionalities as services via Internet, according to the ASP model. The services are supplied by two service centers, whose architecture is based upon a Warehouse platform. They integrate the heterogeneous data coming from the operative level (monitoring systems, laboratory analyses, land registers, etc.) into a remote and homogeneous data repository. These data are processed by the analytical tools, such as the OLAP module (a typical Business Intelligence tool) for advanced reporting and statistical analysis, and by forecasting modelling tools. The access to the E2SP system is realized trough a Web Enterprise Portal.
1
Project Automation S.p.A., viale Elvezia 42, 20052 Monza, Italy (E2SP project coordinator). IPIS Institute of Environmental Engineering of the Polish Academy of Science, Poland. 3 WIOS Regional Environmental Inspectorate of Silesia, Katowice, Poland. 4 ESAprojekt Sp.zo.o., ul. Bytkowska 1B, 40-955 Katowice, Poland. 5 Faculty of Informatics, Masaryk University Brno, Czech Republic. 2
02.11.2005, GaiaCorbetta.doc
1
1.
Introduction
The E2SP project’s objective is to provide innovative and effective services about air quality management to all actors operating in environmental monitoring, planning, governance and assessment activities all over Europe. Target users are the Environmental Agencies, other public bodies (municipalities) engaged in environmental governance, educational and research centres and private firms acting in the environmental assessment consultancy sector. The project involves partners representing a relevant “panel” of operators in Air Quality area: • Regional/Local Agencies: Regional Environmental Inspectorate of Silesia (Poland), Municipality of Bari (Italy). • Research and Educational: University of Bari (Italy), Academy of Science (Poland). • Industrial IT environmental partners (monitoring network and IT solution providers): Project Automation (Italy), ESAPROJEKT (Poland), • Environmental industries (environmental assessment and air pollution modelling) : Aria Technologies (France). • Business partners: Sineura (Italy), GL2006 (UK).
Figure 1: E2SP project consortium.
02.11.2005, GaiaCorbetta.doc
2
E2SP services offer Internet-based user interfaces for: • Configurable query & reporting • Air quality data visualization • Air quality data analysis • Air quality data validation • Data export in XML format • Emission inventories display • Modelling and forecast of air pollution concentrations • Geo-referenced and thematic representation • Knowledge management & information retrieval • Environmental portal • Content management The services are delivered via Internet and wireless communication according to the ASP (Application Service Provider) model, allowing affordable access to data and value added tools.
2.
E2SP service
2.1 Service architecture for online reporting Service architecture for online reporting is based on the cost effective Application Service Provide model, and takes into consideration the mandatory issues of compliance of produced reports to national legislation and to data quality constraints. This paragraph describe the ongoing service deployment. Air quality services in E2SP are currently supplied by two pilot service centres in Italy (Monza at Project Automation premises) and Poland (Katowice at Esaprojekt premises).
"
#
(
!
&& ' $
%
(
Figure 2: E2SP service architecture. 02.11.2005, GaiaCorbetta.doc
3
#
Data from Bari and Silesia air pollution monitoring networks and laboratories are extracted and stored via ETL procedures in the service centres datawarehouses. The only additional requirement to existing infrastructure is a common data connection via modem or ADSL. The environmental operators remotely access E2SP Environmental Information System and related environmental decision support tools via Internet. E2SP services include: data validation, data visualization (graphs, maps and tabular reports), query and reporting, data analysis, pollution simulation and forecast, broadcast of environmental information (actual status and/or forecast) to citizens, data export in XML. The pollution simulation and forecast is realized by mathematical models running on a remote server connected to E2SP system via ADSL line which takes the data from the service centres datawarehouses and makes the simulation results available to users trough the E2SP integrated user interface. The proposed online reporting service start from the same real time data available at user side. Because these data are usually validated in the agencies according to specific validation protocols, the online reporting services will generate reports starting from officially validated data, so compliant with regulation in terms of quality of data. Where no validated data are available, the user can validate them using the data validation functions integrated in the same platform. The service includes the availability of a contact center that supports the users in case of problems.
2.2 Technical architecture for service providers The E2SP service centres are based on state-of-the-art technologies, organised in a multi-tier paradigm which has become a standard reference architecture for data warehouse applications: 1. the data source level, which comprises all those applications responsible for environmental data collection
HDSL 2Mbit
Firewall
Internet
LAN
E2SP WEB/APP SERVER
www.e2sp.pl Telecom
DMZ
Admin Console
Local Air Quality Monitoring Network
E2SP DB
RTC
Figure 3: E2SP service center in Poland: technical architecture.
02.11.2005, GaiaCorbetta.doc
4
2. the integration level, constituted by the Data Warehouse itself, which collects data from the data source level 3. the elaboration and access level, which comprises the E2SP applications that support the services. The service centers are based on the architecture shown in Figure 3. The hardware system architecture is specifically designed in order to guarantee the respect of high levels of availability, scalability, performance and security. A system with such characteristics allows the provided services to be always accessible, with rapid response from the user point of view and guarantees protection of the underlying data. This is typically part of Service Level Agreement for added value service providers. The software configuration of the system components is the following: Integration Level (installed on the DW Servers Cluster in the physical architecture schema): Data Persistency Layer: •
Oracle DB with Clustering, Partitioning, Tuning and Diagnostic options
ETL Layer: •
Oracle Warehouse Builder
Elaboration Level Elaboration Layer (installed on the Application Servers Cluster in the physical architecture schema): •
Oracle Internet Application Server (Oracle Text, Oracle Portal, Oracle Internet File System)
•
BusinessObjects WebIntelligence
•
GIS software
Access Layer (installed on the Web Servers Cluster in the physical architecture schema): •
Oracle Internet Application Server Standard Edition (Oracle Portal)
Back Office Applications •
Antivirus, backup, network and asset management, firewall, DB and ETL administration, users profiling
As for application software (i.e. the applications the technological partners of the Consortium have developed and tested, constituting the E2SP prototype), the system configuration is: Integration Level •
E2RP-Warehouse (Project Automation S.p.A.)
Elaboration Level •
E2RP-OLAP (Project Automation S.p.A.)
•
ARIA Regional (ARIA Technologies SA)
All applications are based on Windows 2000/2003/XP.
02.11.2005, GaiaCorbetta.doc
5
The software applications providing the value added services are built upon web server technology (Oracle), GIS (ESRI) and business intelligence (BusinessObjects) platforms best of breeds in their respective market segments, in order to provide the most robust, flexible and functional services for the users.
3.
Environmental reporting
Environmental quality monitoring networks generate huge amounts of data about pollutants concentrations, regularly collected by the environmental protection agencies. This data are generated automatically by monitoring stations, and manually by laboratories and manual data collection systems. All the information coming from collected data are aggregated to build indicators which are made available trough reports whose first aim is to fulfil legislative requirements and, furtherly, to build an historical view of quality data and, finally, to publish and disseminate accurate information about the environment. These activities are fundamental in the environmental decision process and consequently the reporting phase is equally important, such as the development and availability of reporting tools powerful and easy-to-use as the one proposed in E2SP.
3.1 The Polish context The EU environmental acquis are fully transposed in the Polish (and Italian) legislation. Hereunder there are listed the most important Polish regulations related to air quality control and reporting: •
The Environmental Protection Law Act of April 27th 2001, related acts.
and following instructions: • • •
The Instructions of The Minister of Environment of June 6th, 2002, on admissible levels of selected substances in ambient air, alert levels of selected substances in ambient air and tolerance margins for selected substance. The Legislative Journal 2002, No. 87, Item 796. The Instructions of The Minister of Environment of June 6th, 2002, on evaluation of levels of substances in ambient air. The Legislative Journal 2002, No. 87, Item 798. The Instructions of The Minister of Environment of October 1th, 2002, on environmental information’s publishing rules. The Legislative Journal 2002, No. 176, Item 1453.
The table below presents some examples of fair quality indicators and the corresponding regulations. In the first phase of the pilot installation, the same reports will be available both at the Polish and at the Italian site (in each site the reports will be translated in the corresponding language), as the proposed reports derive from the same European legislation. The list of such reports required by the regulations is presented in the next chapter. The end-users of the system can extend the set of reports with new ones, giving most accurate environmental information.
02.11.2005, GaiaCorbetta.doc
6
Instr. ME L.J.no 176 .1453 2002
Instr. ME L.J.no 87.798. 2002
Instr. ME L.J.no 87.796. 2002
Indicators \ Regulations
Average, Min, Max Moving average AOT Number of exceedances Percentiles Number of valid data Percentage of valid data
Table 1: Air quality indicators and corresponding Polish regulations.
3.2 The Italian context The environmental reporting is a standard activity ruled by Italian and European environmental regulations; according to the current legislations, a set of indicators and indexes is calculated on periodic basis to describe the air quality status and to check the compliance to limit values. The most relevant Italian environmental laws for air quality assessment are: • DPR 24/5/1988 n.203 (this decree rules the atmospheric emissions of all industrial plants and defend the human health, the environment, the ecosystems); • DL 4/8/1999 n. 351 (this decree obliges the regions to identify the zones in which the pollutants levels entail a risk to exceed limit values and alarm thresholds; moreover it recognises the qualified authorities for the risk management); • DM 2/4/2002 n. 60 (this decree establishes the limit values and alarm thresholds, the end in which the limit value must be reached, the criterions to collect air quality data, the lower and the higher evaluation thresholds for nitrogen oxides (NOx), particulate matter (PTS), benzene, carbon monoxide (CO) and lead (Pb)); • DM 1/10/2002 n. 261 (this decree establishes the technical directives for the air quality preliminary evaluation and the criteria to design plans and programmes to reach limit values). An example of air quality indicators is provided in the following table, linked to the corresponding regulations.
02.11.2005, GaiaCorbetta.doc
7
DLgs 183/2004 Ozone
DM 60 2/4/02
DM 25/11/94 Air quality target
DM 25/11/94 Net status
87/6 Istisan 89/10
Indicators \ Regulations
Average, Min, Max Moving average Standard deviation AOT Number of exceedances Percentiles (50, 90, 95, 98, 99) Number of valid data Percentage of valid data Dispersion index Net status (Normal,…) Status frequency
Table 2: Air quality indicators and corresponding Italian regulations. All the information and indicators required by law are summarized in the following reports: • Report/Graph of Concentration Data (Hourly/Daily) • Report/Graph of 10 Minutes Concentration6 per Geography & per Network • Report/Graph of Daily/ Hourly Concentration per Geography & per Network6 • Report/Graph of Standard Day per Geography & per Network • Report/Graph of AOT index -Accumulated Exposure Over Threshold-7 per Geography & per Network • Report/Graph of Number of Exceedances per Geography & per Network • Report/Graph of Percentage of valid data per Geography & per Network • Report/Graph of Standard Deviations per Geography & per Network • Report/Graph of N-th Percentile (50°, 95°, 98°, 99°) per Geography & per Network • Report/Graph of Net Status8 • Report/Graph of Annual Concentration per Geography & per Network6 6
In compliance with DM 60 2/4/2002. In compliance with European Directive 2002/3/CE. 8 In compliance with DM 25/11/1994. 7
02.11.2005, GaiaCorbetta.doc
8
• •
Report/Graph of Assessment Thresholds - Upper and lower6 Report/Graph of Alarm Threshold per Geography & per Network6
3.3 E2SP: online reporting and analytical capabilities In the E2SP project, the reporting functionality are delivered via Internet trough the module E2RP-OLAP that has been designed in order to fulfil the legislative framework described in the previous paragraph. The E2RP-OLAP module performs advanced environmental data analyses that can be used to support environmental management, planning and decision-making; in particular, decision makers can: • • •
create new combinations of operators (environmental indicators and indices) according to various space-time aggregation criteria, investigate data according to “free” and personalized analytical paths, view, build and consult reports in graphic and/or table format in dynamic mode.
One of the most innovative features of E2RP-OLAP is to be Web based: this means that every report and report structure defined by the user is resident in the service centre and do not require any particular IT infrastructure on user side. 3.3.1 Reports structure E2RP-OLAP can generate both standard reports as required by the regulations and „ad hoc“ reports created according to users’ specific needs. Each report is surfable thanks to its tree-structure and can be dynamically explored by simply clicking on tables and graphs; the reports can also be refreshed (trough manual activation or automatically on a periodic basis).
02.11.2005, GaiaCorbetta.doc
9
Figure 4: Example of list of reports about air quality realized in E2RP-OLAP.
Figure 5: Example of E2RP-OLAP report: the report pages can be surfed trough the tree-structure on the left.
02.11.2005, GaiaCorbetta.doc
10
3.3.2 New reports: construction Besides the capability of dynamic exploration of data, the E2RP-OLAP supports the creation of new reports completely built according to specific user requirements. This means that the user can realize the report which exactly fits its analytical needs by selecting all the indicators and analysis dimensions necessary to define his personal analytical path. To this purpose, a set of indicators and analysis dimensions is available to the user for the creation of new reports.
Figure 6: Example of E2RP-OLAP report construction: on left side, the list of indicators and dimensions available for reports construction. On the right side, the indicators and dimensions selected by drag and drop.
02.11.2005, GaiaCorbetta.doc
11
3.3.3 Navigation inside the report: slice and dice In comparison with usual reporting tools, E2RP-OLAP supports the capability of viewing data according to different perspectinìves, that is according to analytical paths which follow different aggregation criteria (geography, time, type of pollutant, name of monitoring station etc.) In practical terms, the user can choose free analytical paths by changing the analysis duimension at every step (for example, starting from geographical aggregations and after focusing on time dimension etc.).
S T A T I O N
Juvara
TIME
Marche Sempione NO2
CO PM10
POLLUTANT
14/01/01 13/01/01 12/01/01
T I M E
12/01/01 13/01/01 PM10 CO NO2 Juvara MarcheSempione
14/01/01
STATION
POLLUTANT
Figure 7: Example of E2RP-OLAP data exploration: data can be analyzed trough different dimensions interchangeable each other according to free analysis paths.
3.3.4 Navigation inside the report: drill down and roll up Another innovative feature of E2RP-OLAP reports is the dynamic exploration of the information provided in the report trough “drill down” and “roll up” capabilities that support very fast aggregations and disaggregations of data. By interacting intuitively with the various components of the report (tables, graphs, etc.), it is possible to perform detailed analyses or new groupings and correlations, such as analytical “zooms” to display detailed data (e.g. average monthly or weekly concentration of a pollutant) starting from extremely synthetic data (e.g. average yearly concentration) simply by clicking with the mouse over parts of graphs and tables.
02.11.2005, GaiaCorbetta.doc
12
IN Q U IN A N T E
S T A Z I O N E 1 6 /0 1 1 2 /0 1 1 4 /0 1 1 5 /0 1 1 3 /0 1
g e n fe b m a r
G IO R N O
M ESE
D rill dow n
TEM PO
2001
ANNO R o ll Up
Figure 8: Example of E2RP-OLAP data exploration: roll up (from left to right) from daily data to annual mean and drill down (from right to left).
3.4 Preliminary steps to E2RP-OLAP online reporting activation The steps needed to activate the online reporting functionality are quite simple and can be summarized as follow: • • • •
availability of a monitoring network and/or other data sources and of a central database containing air quality data updated on a regular basis (typically, for networks, hourly data updated on daily basis) availability of a modem connection or ADSL or wireless connection trough public switched network installation of ETL procedures loading of historical data in the DWH in the Service Center.
Once these requirements are fulfilled, then the online reporting service is available to users.
4.
Benefits
The main advantages offered by E2RP-OLAP are: • • • •
reduction in the time and cost of advanced data analyses, reduction in the time and cost of producing non-predefined reports upgrade and optimisation of the experts’ analytical capabilities vision and integrated analysis of the information (data, indicators and indices) that can be used to manage the environment effectively.
The E2SP system offers the needed flexibility to users in order to adapt it very easily to local needs and “business models” (meant as the institutional mission in case of Environmental Agencies), independently from the geographical coverage, through: • Adherence to EC regulation • Configurable reporting and data analysis common user interface and user procedures 02.11.2005, GaiaCorbetta.doc
13
• •
5.
Sharing of the same data analysis and presentation tools Scalable services and business model (a “pay per use” concept) adaptable to user dimension (sustainability).
Conclusions
The E2SP project represents an innovative proposal of effective services to all actors operating in environmental monitoring, planning, governance and assessment activities all over Europe for the full exploitation of environmental monitoring data. The idea is to supply a common platform for environmental data management and reporting, in order to enforce the process of homogeneization and standardization of such procedures at an European level. This aim is obtained also trough the use of a Web based technology and of the ASP model for the use of the system; both of these choices are in favour of a wide use of the system in an international extent. Future developments of the E2SP system can consist in an enlargement of the number of users and in a wider application in different EU countries, in order to establish a common network for the exchange, reporting, elaboration and publishing of environmental data.
6.
Bibliography
APAT, (2002): Environmental Data Yearbook, APAT, (2003): Environmental Data Yearbook, APAT, (2005): Environmental Data Yearbook, Cislaghi, M. (et al) (2005): E2SP Environmental Enterprise Service Provider, Grant Agreement n° C517375. Corbetta, G. (2004): From monitoring systems to environmental information systems in: Regioni e Ambiente, 6, pp.16-17, Corbetta G. et al., (2004): Environmental information systems: an example of integrated system in: 8th National Conference of the Environmental Agencies “La cultura ambientale come elemento strategico di sviluppo”, Genova, July 2004, Corbetta G., Mazzilli R. et al.(2003): From DPSIR model to operative tools: a proposal for the environmental integrated management in: 7th National Conference of the Environmental Agencies “L’innovazione al servizio della conoscenza e della prevenzione”, Milano, November 2003, European Environment Agency, (1997): European Decision 97/101/CE – Exchange of information., European Environment Agency, (1998): Guidance report on preliminary assessment under EC air quality directives, European Environment Agency, (1999): Criteria for EUROAIRNET. The EEA Air Quality Monitoring and Information System , Mazzilli et al., (1996): Interpretation and validation of air pollution data: integrating knowledge-based technologies with data monitoring networks in: Acts of the conference “Soft Computing in Remote Sensing Data Analysis“, Milano, December 1995, Mazzilli et al., (1993): Organizzazione del sistema informatico della qualità dell' aria: proposte di architetture e tecnologie in: Acts of the conference “I° Simposio Nazionale sulle Strategie e Tecniche di Monitoraggio dell' Atmosfera”, Rome, 1993, Mazzilli et al., (1992): A knowledge based system for air pollution data interpretation and validation in: V° ISAI' 92, "Fifth International Symposium on Artificial Intelligence", Cancùn (Mexico), December 1992.
02.11.2005, GaiaCorbetta.doc
14
