JDemetra+ Java tool for Seasonal Adjustment

94 downloads 6444 Views 1MB Size Report
Integration of original software in a user-friendly application. • Lack of ... jdemetra-software-be-used-seasonal- .... User Group, Helpdesk, Documentation.
JDemetra+ Java tool for Seasonal Adjustment

Dario Buono

Dominique Ladiray

Eurostat, European Commission [email protected] @darbuo

INSEE, France [email protected]

1

Outline • • • • • • • •

Time Series and Seasonality What is JDEMETRA+ History & Governance of JDEMETRA Software Architecture Seasonal adjustment methods Some examples The Seasonal Adjustment Centre of Excellence Users Support

What is a Time Series? A Time Series is a sequence of measures of a given phenomenon taken at regular time intervals such as hourly, daily, weekly, monthly, quarterly, annually, or every so many years

..with at least 3 usual components • Trend/Cycle: the long term evolution of the series • Seasonal pattern: regular fluctuations observed during the year • Irregular: residual and random fluctuations

Cause of Seasonality Seasonality and Climate: variations of the weather/climate (seasons!) Seasonality and Institutions: social habits or to the administrative rules Indirect Seasonality: Seasonality that affects other sectors

Why Seasonal Adjustment? To improve comparability: Over time & Across space Business cycle analysis Reduce noise to facilitate economic reading

It is about smoothing..

5

What is JDEMETRA+? JDEMETRA+ is an Time Series Econometric tool for Seasonal Adjustment • developed by National Bank of Belgium and Bundesbank • supported by EUROSTAT and the European Central Bank Trend identification Outliers treatment Estimation of missing values Calendar Adjustment ARIMA modelling Benchmarking JDemetra+ provides new Java implementations of TRAMO-SEATS and of X12-ARIMA. It is based on the NetBeans platform, is developed under the EUPL license. 6

Some History 2002

Demetra

2010 Demetra+

2015 JDemetra+

• Program to compare X-12-ARIMA and TRAMO/SEATS (1997/98). • Integration of original software in a user-friendly application. • Lack of sufficient product development and handling of errors as a result of a loss of technical knowledge about software.

• Developed in cooperation between Eurostat and the National Bank of Belgium. • Enables the implementation of the ESS Guidelines on SA. • Provides graphical interface and common input/output diagnostics for TRAMO/SEATS and X-12ARIMA. • Includes complex technical solutions. Uses .NET technology and can be used only under Windows.

•Fortran codes re-written in JAVA. •Open source, platform independent. •Extensible graphical interface, based on the NetBeans platform (plugins). •Developed by the National Bank of Belgium, supported by the Deutsche Bundesbank for the X-11 part.

7

Anno Nobili 2015 ESS Guidelines on Seasonal Adjustment Introduced in 2009 and revised in 2015 http://ec.europa.eu/eurostat/documents/ 3859598/6830795/KS-GQ-15-001-ENN.pdf List of options: • A (Recommended) • B (Acceptable) • C ( to be avoided)

8

Official Release of JD+ 2.0.0 Since the 2 of February 2015 JD+ is the official software to be used for Seasonal Adjustment within the European Statistical System for data to be used for Official Statistics

SA user group: review of the SACE work+testing of Jdemetra+

SACoE: Testing of Jdemetra+/Bugs reporting

Jdemetra+ developpers: development of Jdmetra+ and bugs fixing

Official joint ECB/Eurostat Methodological Note published at: http://www.crosportal.eu/content/official-releasejdemetra-software-be-used-seasonaladjustment

SA Expert group proposed the official release of Jdemetra+ to the ESSC.

9

Software layout A rich graphical application (end-users) dedicated to Seasonal Adjustment

10

Software layout But in fact an advanced Java toolkit for time series processing (End users, production, ITteams, researchers)

11

JDemetra+ characteristics Flexibility • Encompasses the leading SA algorithms and can evolve independently

Versatility • Can be used in a rich graphical interface and/or be integrated in other.

Reusability of modules the other circumstances: • Plug-in for temporal disaggregation • Outliers detection, estimation of missing values, Arima forecasts

Extensibility • Additional plug-ins and modules do not change the core engines. • Efficient process of large datasets through: • JWSAcruncher, command line application that allows calling JDemetra+ from other applications;

Web services and Direct call to Java libraries. Open source 12

Architecture JTsToolkit Core algorithms

In house developments Cruncher

External packages

Peripheral modules

Jdemetra-core Jdemetra-app NetBeans

JDemetra+ plug-ins

Third party plug-ins

13

Algorithmic libraries (jtstoolkit) Basic data handling

Basic econometrics

Matrix computation

Arima modelling

RegArima

Complex, polynomials

Linear filters

Seasonal adjustment

Function optimization Basic statistics Utilities...

Tramo

Arima, Ucarima

Structural models...

Benchmarking, temporal disaggregation

X11 Seats

VAR, Dynamic factor model

State space framework

Time series, calendars, regression variables... 14

Generic modules:

Seasonal adjustment Methods SA methods Specific modules

•Analysis •Seasonality tests •Revision analysis •Sliding spans •I/O (common xml schema) •Graphical components •Charts •SI ratios...

Tramo-Seats, X12-Arima...

REGARIMA modules: (X11...)

•Common model •Estimation tools •Automatic modelling routines •Analysis tools (residuals, forecasts...) •Graphical components

REGARIMA pre-processing

Other filters (X11...)

Model-based decomposition (canonical decomposition, structural models...)

Signal extraction tools: •Estimation •Analysis •Graphical components

REGARIMA modelling Model building (Reg. variables)

• Common definitions for Calendar variables, outliers, intervention variables, user variables... • Algorithms for likelihood estimation

Estimation of the model (likelihood, residuals)

• Kalman filter (Tramo-like), • Ansley algorithm (Cholesky on banded matrix) • (modified) Ljung-Box algorithm (X12-like)

• Equivalent results, different performances

• JD+ uses Kalman filter • Up to 4 x faster than Ljung-Box • Ansley in specific cases (outliers detection)

• Optimization procedure Estimation of the parameters (by ML)

• Levenberg-Marquardt. Tramo-Seats, X12 and JD+ use slightly different variants.

Automatic model identification Pre-test (seasonality...) Log/level Calendar effects... Arima (diff. / Arma)

Outliers detection Final estimation Model validation Models comparison

• Independent blocks (dynamically modifiable)

• Specific implementation for Tramo-Seats, X12 Example: X12 modelling with outliers detection from Tramo

Algorithms for signal extraction in JD+ • Wiener-Kolmogorov filters • Burman's algorithm, Maravall's analysis framework (Seats)

• Kalman smoother • Koopman's initialization procedure (disturbance or ordinary smoother)

• Matrix computation • McElroy 's formulae

• Can be applied to any (valid) UCARIMA model

• Results • Estimates: identical • Standard deviations: WK approach yields negligible differences (exception: quasiunit roots in MA polynomial → large differences)

18

State space framework • Key solution for:

• REGARIMA estimation • Signal extraction (Kalman smoother) • Alternative time series modelling (for SA or not) • Structural models...

• Benchmarking • Cholette (including multi-variate extension)

• Temporal disaggregation • Chow-Lin, Fernandez...

• Multi-variate models • VAR, dynamic factor models, SUTSE…

• JD+ provides an advanced implementation of SSF

State space framework (II) Models Atomic models: • • • • • • •

Generic (time invariant or not) Ar(i)ma Ucarima Basic structural White noise Random walk ...

Algorithms Filtering:

Likelihood evaluation:

• • •

Prediction error decomposition

Ordinary filter Fast filter (Chandrasekhar) Array filter (Kailath...)

Diffuse initialization: • • •

Koopman Square root Ad hoc

Others: • •

Derived models: • • • •

Composite Regression variables Aggregation constraints ...

Smoothing:



• • •



Ordinary Disturbance Fixed point

Univariate handling of multi-variate models Augmented Kalman filter (for reg. model) Extended Kalman filter (for non linear models) ...

Example 1. Comparison tool for different SA algorithms

Example 2. Extensible application through plug-ins

Example 3. Derivation of new outputs Standard deviations of the seasonal component as estimated in SEATS (separate) and as they appear in the full model (stochastic and final=stochastic+calendar). Computation by means of the corresponding state space model.

0.0115 0.0105 0.0095 0.0085 0.0075

Belgium. Imports of goods (monthly series, 1995-2007) Standard deviations of the seasonal components (in logs; includes forecasts)

S (separate) S (stochastic) S (final)

The Seasonal Adjustment Centre of Excellence • A “joint venture” between Eurostat, NSIs and CBs • SACE: Belgium, France, Italy, Latvia, Portugal, UK • Partners: Eurostat, BBk, OECD, ECB, IMF • SAUG: Partners + Denmark, Finland, Hungary, Lithuania, Luxembourg, Romania, Slovakia, Slovenia, Spain • 3-year contract: 03/10/2016 – 02/10/2019

24

On the Agenda • Knowledge sharing and Dissemination • Support to SA practitioners • User Group, Helpdesk, Documentation

• Testing • Plug-ins • Benchmarking, Quality, Analysis of revisions, Weekly and Daily data

• Training, coaching and consultancy.

25

Who is using JD+? A difficult question: Downloads are anonymous Helpdesk

Latvia, Ireland, Czech Republic, Fyrom, Finland, Germany, Greece, Hungary, Portugal, Spain, France, Italy, Malta, Serbia, Luxembourg, Romania, Cyprus, Austria, Denmark, Iceland, Netherlands, Norway, Slovenia, Sweden, Switzerland, Turkey But many other users: ECB, Eurostat, IMF, Algeria, Cameroon, Maroc, Senegal, Tunisia, UEMOA etc.

26

User Support and Documentation The latest JDemetra+ released version can be downloaded at

https://github.com/jdemetra/jdemetra-app/releases Java SE 8 or later versions are required. Modules, code and developers documentation and GitHub • https://github.com/jdemetra for the official modules • https://github.com/nbbrd for NBB resources

All user documentation (JDemetra+ Quick Start, JDemetra+ User Guide, JDemetra+ Reference Manual) can be found here. In addition, beyond seasonal adjustment, the following prototype plug-ins are available for: • Temporal Disaggregation and Benchmarking • Quality/ Validation reporting • Revision Analysis • Nowcasting • Using JDEMETRA+ with R 27

JDEMETRA+ Helpdesk and trainings More info about the SACE http://ec.europa.eu/eurostat/cros/content/seasonal-adjustmentcentre-excellence_en For your Helpdesk queries http://ec.europa.eu/eurostat/cros/content/ess-seasonal-adjustmenthelpdesk_en

For training opportunities visit http://ec.europa.eu/eurostat/web/ess/about-us/estp

Thank you for your attention Dario Buono Eurostat, European Commission [email protected] @darbuo

Dominique Ladiray INSEE, France [email protected]

29