The ALMA Science Archive: Implementation

Astronomical Data Analysis Software and Systems XXI ASP Conference Series, Vol. 461 Pascal Ballester, Daniel Egret, and Nuria P. F. Lorente, eds. c 2012 Astronomical Society of the Pacific !

The ALMA Science Archive: Implementation Juande Santander-Vela, Matthias Bauhofer, Holger Meuss, Felix Stoehr, and Alisdair Manning ESO, Karl-Schwarzschild-Str. 2, Garching bei M¨unchen, D-85748, Germany Abstract. The Atacama Large Millimeter Array (ALMA) radio interferometer has started Early Science observations, providing a copious stream of new, high-quality astronomical datasets of unprecedented sensitivity and resolution. We present here how the ALMA Science Archive (ASA) is being implemented, leveraging existing Virtual Observatory (VO) technologies and software packages, together with Web 2.0 techniques, to provide scientists with an easy to use, multi-parameter discovery tool for ALMA data, integrated with the VO.

1. Introduction: the ALMA Science Archive The ALMA Science Archive (ASA) is intended to be the central point for all astronomers, not just radio specialist, to find and retrieve ALMA data. It will contain (Etoka et al. 2007; Stoehr et al. 2012) a transformed copy of the ALMA Front-end Archive (AFA) that can be searched on scientifically meaningful parameters, and not just technical ones. The goal is making ALMA accessible beyond the radio astronomical community, so that multi-wavelength science can be performed by means of non-radio tools, and data discovered through the Virtual Observatory (VO). Figure 1 illustrates the current architecture of the ASA. A harvesting process reads data from the AFA, validates them, estimates different quantities, and writes them into the simplified ASA table structure (Stoehr et al. 2012). The granularity of the ASA Science table is the Field, and supporting Spectral Windows, Projects, and product Dependency tables (for elaborated data products) exist. The interface is a web application, written in Java using the Spring MVC framework. We use the openCADC stack (Dowler et al. 2010), as it provides us with the capability of converting SQL queries into VOTables, and easily support VO protocols. The web interface is described by means of Java Server Pages (JSP), which generate HTML for the browser, and contains JavaScript for Web 2.0 interactivity. Said HTML contains the VOView (Hinshaw 2010), which acts as just another client for the VO Table Access Protocol (Dowler et al. 2011). 2. Query Interface Figure 2 shows the search tab of the query interface. It allows multi-parameter searching on Position, Energy, Time, Polarisation, and Observation provenance parameters, and uses JavaScript and AJAX to allow for dynamic display of help tooltips. Automatic coordinates’ resolution through Sesame is available for object names. 435

Santander-Vela et al.

436

VO Client

Local Apps SAMP-based VO Tools

V O T a b l e

Web Browser

SAMP

VOView

AJAX request

ADQL

Spring MVC openCADC Java Application request

JSP Pages

Database Server

Java Application Server

VO TAP Endpoints

JavaScript

HTML

External Clients

JDBC

ASA Database

Harvester

AFA Database

SQL

Figure 1. Architecture of our ASA implementation. The harvesting process feeds the ASA database from the ALMA Front-end Database, while the Spring-based Java application provides GUI and web services. The web browser, through the embedded VOView, and different clients of the system, request data from the application.

Figure 2. Query interface search tab, displaying a help tooltip with examples of usage for the corresponding field, and in this case resolving coordinates for M31.

Once the search parameters are submitted, the results tab (Figure 3) opens in the same page, where the VOView shows returned data in VOTable format. VOView supports interactive filtering, searching, column reordering, hiding, and sorting, via header clicks, drag&drop, and search operators such as ! (NOT), * (wildcards), , and .. (limits and numeric ranges). ESO is working in allowing the VOView interact with SAMP (see Taylor 2012, in this volume) applications using the SAMP Web Profile, allowing interactivity between local, VO-aware applications and our web interface. Users can select datasets by clicking on check-boxes, and then downloaded using the ALMA version of ESO’s request handler (see Fourniol et al. 2012, in this volume). 3. Harvesting The ASDM format (ALMA Science Data Model; Viallefond 2006) stores all data pertaining to data reduction distributed across many XML tabular entities. However, many

The ALMA Science Archive: Implementation

437

Figure 3. Query interface results tab, with the returned VOTable in a VOView. Column metadata is listed below, including units and descriptions.

scientifically relevant queries are not possible against it, both due to its structure, and that many quantities meaningful for selection are not computed. To provide scientific search capabilities on ASDMs, we harvest (see Figure 4) nonmonitoring data from the ALMA Front-end Archive (AFA).1 Streaming XML APIs, such as EventTree for Python, or XOM for Java, are used for parsing the ASDMs. Once non-observational data have been rejected, the harvester needs to be able to combine the data from the different tables, which have different granularity, a process we call folding. Folding aggregates data from relevant ASDM tables at the Field level: • integration time from Scans is attributed to observed Fields; • weather and water vapour information is averaged; • parameters such as the airmass are estimated from retrieved and derived data; 1

For details on how ALMA data reaches the AFA, see Manning et al. (2012) in this volume.

Santander-Vela et al.

438

NO

Start

Obtain ASDM archive_uid list

Figure 4.

Obtain tables with non-zero rows

Do we have all tables we need?

YES

Obtain and Validate data in needed tables

Fold data per eld

Insert rows per eld

End

The ASDM harvesting process.

• spectral information is aggregated via interval operations, computing characteristic channel numbers, bandwidth, etc.; • Antennas and Station pairs are created, baseline vectors calculated, and then reprojected to estimate spatial resolution and field of view. In total, more than 70 scientific parameters, corresponding to the searchable axes (see Section 2) are retrieved or estimated, and stored per Field. In addition, corresponding Project and SpectralWindow information is inserted if not present. 4. Outlook We have currently internally deployed the ASA query interface, to test all requirements set for the ASA use cases are implemented. The initial reaction from users is very positive, as it provides them with a fast, easy to use tool to explore ALMA data. The time to create the user interface, and the time needed to make changes, has been reduced by the use of the openCADC stack and the VO view, as only one endpoint needs to be maintained, and GUI development is reduced, allowing to provide a better, sophisticated interface with less development effort. Acknowledgments. The Atacama Large Millimeter/submillimeter Array (ALMA), an international astronomy facility, is a partnership of North America, Europe and East Asia in cooperation with the Republic of Chile. References Dowler, P., Rixon, G., & Tody, D. 2011, ArXiv e-prints. 1110.0497 Dowler, P., et al. 2010, openCADC. URL http://code.google.com/p/opencadc/ Etoka, S., Fuller, G., & Wicenec, A. 2007, ALMA Science Archive Requirements, Draft COMP-70.50.00.00-004-A-GEN, Atacama Large Millimeter Array Fourniol, N., et al. 2012, in ADASS XXI, edited by P. Ballester, D. Egret, & N. P. F. Lorente, vol. 461 of ASP Conf. Ser., 669 Hinshaw, D. 2010, VOView. URL http://code.google.com/p/voview/ Manning, A., Wicenec, A., Checcucci, A., & Gonzalez, J. 2012, in ADASS XXI, edited by P. Ballester, D. Egret, & N. P. F. Lorente, vol. 461 of ASP Conf. Ser., 681 Stoehr, F., Leon Tanne, S., Lacy, M., Saito, M., & Santander Vela, J. 2012, in ADASS XXI, edited by P. Ballester, D. Egret, & N. P. F. Lorente, vol. 461 of ASP Conf. Ser., 697 Taylor, M. B. 2012, in ADASS XXI, edited by P. Ballester, D. Egret, & N. P. F. Lorente, vol. 461 of ASP Conf. Ser., 279 Viallefond, F. 2006, in ADASS XV, edited by C. Gabriel, C. Arviset, D. Ponz, & E. Solano, vol. 351 of ASP Conf. Ser., 627