JDemetra+ Java tool for Seasonal Adjustment
Dario Buono
Dominique Ladiray
Eurostat, European Commission
[email protected] @darbuo
INSEE, France
[email protected]
1
Outline • • • • • • • •
Time Series and Seasonality What is JDEMETRA+ History & Governance of JDEMETRA Software Architecture Seasonal adjustment methods Some examples The Seasonal Adjustment Centre of Excellence Users Support
What is a Time Series? A Time Series is a sequence of measures of a given phenomenon taken at regular time intervals such as hourly, daily, weekly, monthly, quarterly, annually, or every so many years
..with at least 3 usual components • Trend/Cycle: the long term evolution of the series • Seasonal pattern: regular fluctuations observed during the year • Irregular: residual and random fluctuations
Cause of Seasonality Seasonality and Climate: variations of the weather/climate (seasons!) Seasonality and Institutions: social habits or to the administrative rules Indirect Seasonality: Seasonality that affects other sectors
Why Seasonal Adjustment? To improve comparability: Over time & Across space Business cycle analysis Reduce noise to facilitate economic reading
It is about smoothing..
5
What is JDEMETRA+? JDEMETRA+ is an Time Series Econometric tool for Seasonal Adjustment • developed by National Bank of Belgium and Bundesbank • supported by EUROSTAT and the European Central Bank Trend identification Outliers treatment Estimation of missing values Calendar Adjustment ARIMA modelling Benchmarking JDemetra+ provides new Java implementations of TRAMO-SEATS and of X12-ARIMA. It is based on the NetBeans platform, is developed under the EUPL license. 6
Some History 2002
Demetra
2010 Demetra+
2015 JDemetra+
• Program to compare X-12-ARIMA and TRAMO/SEATS (1997/98). • Integration of original software in a user-friendly application. • Lack of sufficient product development and handling of errors as a result of a loss of technical knowledge about software.
• Developed in cooperation between Eurostat and the National Bank of Belgium. • Enables the implementation of the ESS Guidelines on SA. • Provides graphical interface and common input/output diagnostics for TRAMO/SEATS and X-12ARIMA. • Includes complex technical solutions. Uses .NET technology and can be used only under Windows.
•Fortran codes re-written in JAVA. •Open source, platform independent. •Extensible graphical interface, based on the NetBeans platform (plugins). •Developed by the National Bank of Belgium, supported by the Deutsche Bundesbank for the X-11 part.
7
Anno Nobili 2015 ESS Guidelines on Seasonal Adjustment Introduced in 2009 and revised in 2015 http://ec.europa.eu/eurostat/documents/ 3859598/6830795/KS-GQ-15-001-ENN.pdf List of options: • A (Recommended) • B (Acceptable) • C ( to be avoided)
8
Official Release of JD+ 2.0.0 Since the 2 of February 2015 JD+ is the official software to be used for Seasonal Adjustment within the European Statistical System for data to be used for Official Statistics
SA user group: review of the SACE work+testing of Jdemetra+
SACoE: Testing of Jdemetra+/Bugs reporting
Jdemetra+ developpers: development of Jdmetra+ and bugs fixing
Official joint ECB/Eurostat Methodological Note published at: http://www.crosportal.eu/content/official-releasejdemetra-software-be-used-seasonaladjustment
SA Expert group proposed the official release of Jdemetra+ to the ESSC.
9
Software layout A rich graphical application (end-users) dedicated to Seasonal Adjustment
10
Software layout But in fact an advanced Java toolkit for time series processing (End users, production, ITteams, researchers)
11
JDemetra+ characteristics Flexibility • Encompasses the leading SA algorithms and can evolve independently
Versatility • Can be used in a rich graphical interface and/or be integrated in other.
Reusability of modules the other circumstances: • Plug-in for temporal disaggregation • Outliers detection, estimation of missing values, Arima forecasts
Extensibility • Additional plug-ins and modules do not change the core engines. • Efficient process of large datasets through: • JWSAcruncher, command line application that allows calling JDemetra+ from other applications;
Web services and Direct call to Java libraries. Open source 12
Architecture JTsToolkit Core algorithms
In house developments Cruncher
External packages
Peripheral modules
Jdemetra-core Jdemetra-app NetBeans
JDemetra+ plug-ins
Third party plug-ins
13
Algorithmic libraries (jtstoolkit) Basic data handling
Basic econometrics
Matrix computation
Arima modelling
RegArima
Complex, polynomials
Linear filters
Seasonal adjustment
Function optimization Basic statistics Utilities...
Tramo
Arima, Ucarima
Structural models...
Benchmarking, temporal disaggregation
X11 Seats
VAR, Dynamic factor model
State space framework
Time series, calendars, regression variables... 14
Generic modules:
Seasonal adjustment Methods SA methods Specific modules
•Analysis •Seasonality tests •Revision analysis •Sliding spans •I/O (common xml schema) •Graphical components •Charts •SI ratios...
Tramo-Seats, X12-Arima...
REGARIMA modules: (X11...)
•Common model •Estimation tools •Automatic modelling routines •Analysis tools (residuals, forecasts...) •Graphical components
REGARIMA pre-processing
Other filters (X11...)
Model-based decomposition (canonical decomposition, structural models...)
Signal extraction tools: •Estimation •Analysis •Graphical components
REGARIMA modelling Model building (Reg. variables)
• Common definitions for Calendar variables, outliers, intervention variables, user variables... • Algorithms for likelihood estimation
Estimation of the model (likelihood, residuals)
• Kalman filter (Tramo-like), • Ansley algorithm (Cholesky on banded matrix) • (modified) Ljung-Box algorithm (X12-like)
• Equivalent results, different performances
• JD+ uses Kalman filter • Up to 4 x faster than Ljung-Box • Ansley in specific cases (outliers detection)
• Optimization procedure Estimation of the parameters (by ML)
• Levenberg-Marquardt. Tramo-Seats, X12 and JD+ use slightly different variants.
Automatic model identification Pre-test (seasonality...) Log/level Calendar effects... Arima (diff. / Arma)
Outliers detection Final estimation Model validation Models comparison
• Independent blocks (dynamically modifiable)
• Specific implementation for Tramo-Seats, X12 Example: X12 modelling with outliers detection from Tramo
Algorithms for signal extraction in JD+ • Wiener-Kolmogorov filters • Burman's algorithm, Maravall's analysis framework (Seats)
• Kalman smoother • Koopman's initialization procedure (disturbance or ordinary smoother)
• Matrix computation • McElroy 's formulae
• Can be applied to any (valid) UCARIMA model
• Results • Estimates: identical • Standard deviations: WK approach yields negligible differences (exception: quasiunit roots in MA polynomial → large differences)
18
State space framework • Key solution for:
• REGARIMA estimation • Signal extraction (Kalman smoother) • Alternative time series modelling (for SA or not) • Structural models...
• Benchmarking • Cholette (including multi-variate extension)
• Temporal disaggregation • Chow-Lin, Fernandez...
• Multi-variate models • VAR, dynamic factor models, SUTSE…
• JD+ provides an advanced implementation of SSF
State space framework (II) Models Atomic models: • • • • • • •
Generic (time invariant or not) Ar(i)ma Ucarima Basic structural White noise Random walk ...
Algorithms Filtering:
Likelihood evaluation:
• • •
Prediction error decomposition
Ordinary filter Fast filter (Chandrasekhar) Array filter (Kailath...)
Diffuse initialization: • • •
Koopman Square root Ad hoc
Others: • •
Derived models: • • • •
Composite Regression variables Aggregation constraints ...
Smoothing:
•
• • •
•
Ordinary Disturbance Fixed point
Univariate handling of multi-variate models Augmented Kalman filter (for reg. model) Extended Kalman filter (for non linear models) ...
Example 1. Comparison tool for different SA algorithms
Example 2. Extensible application through plug-ins
Example 3. Derivation of new outputs Standard deviations of the seasonal component as estimated in SEATS (separate) and as they appear in the full model (stochastic and final=stochastic+calendar). Computation by means of the corresponding state space model.
0.0115 0.0105 0.0095 0.0085 0.0075
Belgium. Imports of goods (monthly series, 1995-2007) Standard deviations of the seasonal components (in logs; includes forecasts)
S (separate) S (stochastic) S (final)
The Seasonal Adjustment Centre of Excellence • A “joint venture” between Eurostat, NSIs and CBs • SACE: Belgium, France, Italy, Latvia, Portugal, UK • Partners: Eurostat, BBk, OECD, ECB, IMF • SAUG: Partners + Denmark, Finland, Hungary, Lithuania, Luxembourg, Romania, Slovakia, Slovenia, Spain • 3-year contract: 03/10/2016 – 02/10/2019
24
On the Agenda • Knowledge sharing and Dissemination • Support to SA practitioners • User Group, Helpdesk, Documentation
• Testing • Plug-ins • Benchmarking, Quality, Analysis of revisions, Weekly and Daily data
• Training, coaching and consultancy.
25
Who is using JD+? A difficult question: Downloads are anonymous Helpdesk
Latvia, Ireland, Czech Republic, Fyrom, Finland, Germany, Greece, Hungary, Portugal, Spain, France, Italy, Malta, Serbia, Luxembourg, Romania, Cyprus, Austria, Denmark, Iceland, Netherlands, Norway, Slovenia, Sweden, Switzerland, Turkey But many other users: ECB, Eurostat, IMF, Algeria, Cameroon, Maroc, Senegal, Tunisia, UEMOA etc.
26
User Support and Documentation The latest JDemetra+ released version can be downloaded at
https://github.com/jdemetra/jdemetra-app/releases Java SE 8 or later versions are required. Modules, code and developers documentation and GitHub • https://github.com/jdemetra for the official modules • https://github.com/nbbrd for NBB resources
All user documentation (JDemetra+ Quick Start, JDemetra+ User Guide, JDemetra+ Reference Manual) can be found here. In addition, beyond seasonal adjustment, the following prototype plug-ins are available for: • Temporal Disaggregation and Benchmarking • Quality/ Validation reporting • Revision Analysis • Nowcasting • Using JDEMETRA+ with R 27
JDEMETRA+ Helpdesk and trainings More info about the SACE http://ec.europa.eu/eurostat/cros/content/seasonal-adjustmentcentre-excellence_en For your Helpdesk queries http://ec.europa.eu/eurostat/cros/content/ess-seasonal-adjustmenthelpdesk_en
For training opportunities visit http://ec.europa.eu/eurostat/web/ess/about-us/estp
Thank you for your attention Dario Buono Eurostat, European Commission
[email protected] @darbuo
Dominique Ladiray INSEE, France
[email protected]
29
