Advanced instrument control and data reduction ...

Advanced instrument control and data reduction software for TIMMI2, the new Mid-Infrared camera for the ESO 3.6m telescope a

b

b

c

a

Helena Relke , Martin Sperl , Josef Hron , Hans Ulrich Käufl , Hendrik Linz , a a Hans-Georg Reimann , and Ralf Wagner a

Astrophysikalisches Institut und Universitäts-Sternwarte Jena, Schillergäßchen 2-3, D-07745 Jena, Germany b Institut für Astronomie der Universität Wien, Türkenschanzstrasse 17, A-1180 Wien Austria c ESO, D-85748 Garching, Germany ABSTRACT

The new ESO Thermal Infrared Multi-Mode Instrument TIMMI2 is described in detail in Reimann1 et al. The TNT Timmi Navigator Terminal is the graphical user interface for TIMMI2. It provides the communication between telescope, instrument and reduction pipeline. The TNT in a very easy way allows the astronomer to prepare and run complex observing programs. The graphical elements are based on Forms Library (A Graphical User Interface Toolkit for X). The TNT is written in C. To facilitate the data analysis and administration, a dedicated reduction pipeline will be used. It is embedded in the MIDAS environment and runs in the background (except for tasks visualising the progress and status of the reduction). The main components are a pre-processor which operates on individual images (e.g. for image cosmetics) and two postprocessors which combine images (e.g. from nod-cycles) and allow further specialised reduction (e.g. for deriving of Stokes parameters from imaging polarimetry). The pipeline also stores and compresses the data for further (re)processing. The reduction steps are controlled by image header keywords and parameter files which can be modified via the TNT. The pipeline can run during the actual observations (on-line reduction with quality control) or during a stand-alone MIDAS session. Keywords: Software, graphical user interface (GUI), steering and control, pipeline reduction, data flow system, image data processing

1. INTRODUCTION TIMMI2 (Reimann1 2 3 et al. ) is designed for a f35 Cassegrain configuration and not only an instrument but an observing system, i. e. it consists of the cryogenic optomechanical instrument a specific calibration system, support hardware for maintenance and operation and the software required for operating the instrument and data reduction. TIMMI2 is a replacement of the previous, simpler thermal infrared facility instrument of ESO, TIMMI4. Instrument control and data reduction software for TIMMI2 consist of three independent components, which communicate with each other by the use of files written in FITS and ASCII format. The main component is the TNT –(Timmi Navigator Terminal), second is the image processing Pipeline and third is IROBS {IROBS(TM) is a product of Wallace Instruments, Madison, Wisconsin, USA, which is the supplier of the infrared detector array read-out electronics}. In Fig. 1 the structure of the TIMMI 2 hardand software components is shown. The hardware consists of the mechanical and optical components of the camera, the electronics, and three Linux-PCs. They are connected by a computer - network (NFS) to exchange command files, data and the images. In these Linux-PCs run the graphical user interfaces –for Pipeline, TNT and IROBS.

1

TIMMI 2 Mechanics and Sensors

Linux PC 1

Acquisition Electronics

Linux PC 2

Dual processor

Linux PC 3

TNT IROBS

Pipeline

PCTCS

TELESCOPE CONTROL SYSTEM

Figure 1. Structure of the TIMMI 2 hard- and software.

2. THE TNT - TIMMI NAVIGATOR TERMINAL There will be two versions of the TNT: one working only at the telescope and used for the actual execution of observing program and a “stand-alone” version for defining and editing the observing program in advance (e.g. at the observers home institute). In Figure 2 the main control panel of the TNT is reproduced. The program can work in one of the following four modes: "Preparation of observing program", "Observation", "Adjustment", "Midas start". 2.1 Preparation of the observing program Following a strategy similar to the one adopted at ESO for VLT instruments for the preparation and execution of observations, TIMMI2 also uses the concept of Observation Blocks (OBs). Each OB specifies an individual observing sequence, which can be scheduled and executed completely without interruption. OBs can specify simple operations like "acquire one N-band image" as well as more complex operations like “obtain a sky image, move telescope to science target, expose science frames, move back to sky position etc.”. However, to keep the operations simple, TIMMI2 does not use the standard ESO software for the preparation and execution of Obs5 but this task is carried out by the TNT. "Preparation of the observing program" (in combination with "Observation Blocks menu") is a part of the software package, which works without communicating with the telescope and camera and provides the possibility to compose OBs for targets

2

Figure 2: Screen Shot of the TIMMI2-Navigator-Terminal, the graphical user interface based on the Forms Library.

3

before or during the observation. It allows to select for the target correctly all necessary parameters for instrument (slit, lens, filters), telescope (coordinates, chopping, nodding, epoch), detector (elementary exposure time, total integration time, number of nodding cycles), guide probe ( select from guide catalogue {TNT allows the creation of a TIMMI2 specific guide star catalogue, which will be growing over the years of observing} a guide star and calculate offset in alpha and delta and guide probe position in X and Y in encoder steps) and own specific set of parameters for the reduction pipeline (see below) for OBs. Currently the following types of OBs are foreseen: • • • •

• • • •

• • •

Imaging: standard imaging with chopping and nodding in the same direction (chop- and nod-angles may be larger than half the field of view) or with an external sky position; Small source imaging: nodding is done perpendicular to the chopping direction and chop- and nod-angles such that the source appears once in each quadrant of the array; Mosaic: series of overlapping images without chopping and nodding but repeated observations of external (i.e. outside the science target) sky positions; Polarimetry: One can consider the imaging polarimetry as an extension to the normal imaging modes. So in addition to the chopping and nodding movements of the mirror and the telescope the rotation of the polarizer has to be included. For the time being we intend to do all four analyzer positions first in one nod positions and then change to the other nod position doing the same four analyzer positions again. Such a nodding sequence has to be repeated several times. See also the sketch of the polarimetry timing sequence in Fig. 4. Long slit spectroscopy: chopping and nodding along slit; Echelle spectroscopy: chopping and nodding along or perpendicular to slit (external sky, avoid overlap of orders from different chop-/nod-positions); Lunar occultation: no chopping, fast readout of array, special mode for accurate positioning of the telescope Day time observation: these have to be done, for obvious reasons, without auto-guiding on a visible star. When using the “small source imaging mode” (see above) this works, apart from not auto-guiding, as night time observing. In case one needs the largest field (“standard imaging” above one has to chop and nod in perpendicular directions (e.g. chopping N-S and nodding E-W) to be able to correct for the chopping offset. Solar system observation: external sky, special mode to follow a non-linear object motion Calibration: special observing sequences for flat-fields, wavelength calibration etc. CCD imaging: transfer of the optical CCD frames acquired by the guiding-CCD to the TIMMI2 data sets.

The parameters for each OB are stored in files in ASCII format. The information about these files of parameters will be stored in a special file called “Observing program” which will be used during the observation. 2.2 Observation The "Observation" mode is the main part of software package which provides the communication between instrument, telescope and reduction pipeline. This program suggests two ways to start the "Observing program": "Classic mode" and "Auto mode". In the "Auto mode" the observer loads the applicable (previously prepared) "Observing program" and activates the program "Start". The TNT defines and executes the necessary commands for the instrument, telescope incl. focus and adapter. This program sends commands to the camera, then requests (scans) the temperature and pressure sensors and waits for the status of the camera. If this status is correct the program at this point sends commands to the telescope waiting for its status. Then the program writes all parameters into a FITS format file and transmits this file to the detector read-out electronics. The electronics accepts this file as fits-header and writes an image. The focusing will be carried out automatically in dependence on the change of the temperature. In the "Classic mode" the observer controls all command functions for telescope, camera, pipeline and electronics and can carry out all eventual programs of observation, that correspond to the technical construction and possibilities of the TIMMI2 camera and that are not scheduled within the standard software package. This may be of particular value for ‘targets-of-opportunity’ such as super-novae or peculiar comets. 2.3 Adjustment The "Adjustments" mode is provided for the control of all command functions of telescope and camera by authorised personnel. This mode allows necessary tests of temperature, pressure and setting of the mechanic functions after the first ‘power-on’ of the TIMMI2. During a first set-up all functions have to be controlled mostly in units such as encoder-steps,

4

whereas later the control is by logical concepts (e.g. polarimetry in/out) or in physical units (offset 5” North). Additionally for calibration purposes the adjustment programs are implemented, that help to generate all standard tables, which will be used in the mode "Observation". The program "Focus" allows to establish a focus calibration table, which will facilitate automatically focusing during the observation. "Adapter calibration" will generate adapter calibration coefficients and offsets introduced by the dichroic. In TIMMI2 particular emphasis has been put on astrometry. Contrary to optical or nearinfrared observations the chances to detect objects in the field-of-view around the scientific target which could be used as secondary astrometric standards to register the observations with e.g. a radio-interferometric map are practically zero. Therefore, the TNT software of TIMMI2 aims to perform all calibrations conceivable, including the telescope and is subsystems, which should allow a registration of frames by ‘dead-reckoning’ to better than the diffraction limit of the telescope (0.6 arcsec). After execution of “Adaptor Calibration” the corresponding calibration tables, necessary for source acquisition and set-up of the guide camera in the adapter will be established. For TIMMI 2 it is required to have the tables, which contain the sky offset values introduced by changing the optical configuration of TIMMI. This will be offset vectors in alpha and delta. The program "Image scale, offset values for lenses/filters" helps to create these tables for all lenses in combination with all filters useful for each lens. The programs "Adapter angle" and "Chopper Throw" allow the calibration of chopping angle versus applied voltage and calibration of the chopper throw versus voltage. These tables with calibration coefficients will be used by the chopping function of the secondary mirror of the 3.6 m telescope. 2.4 Midas start "Midas start" is a software package, which provides the reduction of the data for "Adjustment mode" and can be used during the observation as well as after observation for data reduction without communication with telescope and instrument. All reduction programs are written for MIDAS, TNT defines the parameters for these programs and starts the command file for MIDAS. 2.5 Communication with the 3.6 m telescope The communication between TNT (menu at the button “Telescope control system”) and the 3.6 m telescope is done by means of the software module 'pctcs'6. Communication is achieved via files. The 'pctcs' can handle the following requests from TNT: 1.Send a command to Telescope Control System (TCS). Only one command can be sent at a time to TCS. TNT will write a single command line into the file 'pctcs_cmd'. 2.Get the FITS header from TCS. The TCS provides that the FITS header information is requested - before an exposure starts and - at the end of an exposure. 3.Get status from TCS. Status can be obtained from TCS by writing one or more, so called, "Name Data Items" into the file 'pctcs_status_cmd'. Each item to be queried must be on a separate line. The 'pctcs' will use the Data Query Library of tifCA to get the status, which will be written into the file 'pctcs_status_reply'. The all "Named Data Items" and commands are used by TIMMI 2 based on status and set-up commands to 'tifCA' and keywords for 3.6 m telescope7. 4. Synchronisation via files: TNT is the originator of the communication. The 'pctcs' will wait for the creation of one of the command files described above. The 'pctcs ' polls until one of the command files is created (every 500 ms). TNT writes a command file and waits until the reply file from 'pctcs' is. The 'pctcs' reads the command file, deletes it, sends the command/request to TCS and waits for the reply from TCS. TNT reads the reply file and deletes the reply file afterwards. The entire communications concept has been successfully tested in July 1999 at the telescope with an earlier version of the TNT. 2.6 Communication with the Hardware control electronics For TIMMI2 it is essential to control optical and cryogenic parameters with technical devices. TCP / IP connections as well as TCP /IP – IEEE488 network controller for communication purposes are used to realise access to the camera from external sources. All instruments on the interface bus perform as Talker and Listener. To simplify the communication with the PC, parameters and standard functions are implemented on the devices and can be retrieved by simple IEEE commands. For positioning of the optomechanical components stepper motors are controlled by a IXE-A control unit (supplied by Phytron, Germany). This device contains the regulating programs, which are started by IEEE. It will transfer only the new parameters. So it is possible to reach fast changes of position. Programs for initialization set automatically lens filter and aperture for a basic adjustment. An Edwards Active Gauge Controller (AGC) controls the cold head and vacuum pump and is connected via a RS232 – IEEE converter with the IEEE Bus. Internal relays automatically respond to adjustable parameters. Six external relays of the AGC are used to control the vacuum and cryogenic functions of TIMMI2. These relays can be switched by the TNT via the AGC in order to get the necessary vacuum and cryogenic environment for the camera. Communication between TNT and TIMMI2 camera can be done in two ways:

5

1.Direct control from menu "Camera control system" and "Activate commands for TIMMI2 camera". This way is provided only for authorised personnel. Mouse clicks on the buttons of these menus activate the motors, vacuum pumps and gauges, the Gifford McMahon closed cycle cooler , the functioning of the camera and scanning of the temperature and pressure sensors. 2.Control the camera by "Camera control system interface" ('ccs') that will be activated during the observation and accessible for all users. It provides the communication between TNT and TIMMI2 camera by means of two files. The user selects all necessary parameters for the camera and activates the "Send command to TIMMI” button. All parameters will be written in 'camera.com' file and sent to 'ccs' interface. The 'ccs' analyses this file, compares with status of camera, makes a 'current command file', deletes the 'camera.com file', translates the 'current command file' on command codes, sends to IEEE and waits for status. The status information will be transferred in 'cam.status' file and sent to TNT. TNT analyses this file if necessary gives a message and deletes it afterwards.

3. ACQUISITION ELECTRONICS CONTROL SOFTWARE The acquisition electronics (TIMMI2 is using a commercial product supplied by Wallace Instruments of Madison, Wisconsin, USA) comes with a control software which offers three basic functions to the user: The control of the infrared focal plane array (IRFPA), data acquisition and data storage. It allows remote IRFPA control and monitoring. The user has control of all IRFPA DC biases, clock voltages and clock patterns. Monitoring of all system parameters up to IRFPA signal output load currents, and preamplifier gain and bandwidth is possible. The software offers complete control over the IRFPA operating mode and data acquisition process. The software supports all common observing and operating modes, including staring, chopping, nodding, and correlated and Fowler sampling. FITS format for saving the images is also supported. Included is also a simple image viewer to visualise the raw images as they are acquired. The IR ObserverTM software package runs under RedHat Linux. Software is written in C++ using TCL/TK.

4. REDUCTION PIPELINE Due to the very short integration times in the thermal IR and the chopping/nodding technique, an instrument like TIMMI2 produces a large number of individual frames with quite specific image characteristics. Theoretically, TIMMI2 could produce raw-data at a rate of 64 Mbyte/sec. Even after averaging of frames to 500-1000 Milliseconds, approximately the atmospheric coherence time at 10 micron, a typical data rate of 1-2 Gbytes/hour would arise, which appears undesirable. Therefore, to facilitate handling and the astrophysical interpretation of TIMMI2 data, the TIMMI2 software contains a package for pipeline data reduction. The expected frame rate from the TNT to the pipeline is about 1 frame every 4 seconds. Figure 3 illustrates the main pipeline components, the data flow and also the actual graphical user interface. 4.1 Pipeline components The pipeline consists of the following components: • Pre-processor: receives the raw data from the TNT, modifies the data format, extracts and displays information relevant for a check of the raw data quality (average background, noise), removes/corrects various instrumental effects (bad pixels, flat field etc. ), prepare the data for further processing and store data as FITS datacubes. • Post-processor 1: subtracts sky emission (for unchopped images) and accumulates the corrected raw data (for one nodposition) either by simple addition or via shift & add. • Post-processor 2: combines the frames belonging to different nod-positions, applies further processing depending on the OB used, stores results. • Graphical user interface: displays (graphics and text) the results and status of the different processing steps. • Parameter editor: allows display and editing of the parameter files needed to control the pipeline processing (through a graphical user interface). • MIDAS user session: allows processing of (pipeline processed) data by the user through a standard MIDAS session. • Pipeline feeder: allows to re-process data starting from two stages (raw data, pre-processed) and with modified processing parameters. 4.2 Data flow and processing control There are various data sets to be handled by the TNT and the pipeline: (1) the frames taken at different phases of one chop cycle (on- and off-source images), (2) the sum of several chop-cycles (video frame), (3) the set of video frames belonging to

6

Final data product engineering FITS

raw FITS datacube

TNT

Pre-proc.

FITS

image cosmetics flat fielding quality check Image

Plot

preprocessed FITS datacube

final frame FITS and bdf

Post-proc. 1

bdf

add chop-cycles sky subtraction Image

Plot

Post-proc. 2

bdf

add nod-cycles further processing Image

Plot

Figure 3: Data flow, main components and graphical user interface of the TIMMI2 reduction pipeline. The artificial images correspond to the different processing stages for a small source imaging observation block. The lower left graph displays quantities related to the data quality, the lower middle panel gives an overview of the data received and processed by the pipeline and the lower right panel summarizes the parameters of the currently processed data.

7

one nod-position (4) the sets of frames taken at different nod-positions. In the current version of the pipeline, the difference between on- and off-source frames is computed by IROBS, i.e. the data flow starts with (2) unless an external sky is used. For each video frame there is a FITS file with the data and a FITS header containing the most important keywords. Once per telescope movement (minor number, see below) a second file, the so-called engineering FITS file is written consisting only of an extended FITS header with further information about the status of the instrument (mainly for technical checks etc.). The data flow between the TNT and the pipeline and within the pipeline is controlled through specific FITS keywords. The raw data are stored by the TNT in FITS format. Further processing is done after conversion into the MIDAS bdf-format. The two main pipeline keywords are the major and minor sequence numbers. All frames with the same major number belong to one OB. The different minor numbers correspond to the different nod-positions (or science and external sky observations). Individual short integrations with the same minor number are combined by post-processor 1. Post-processor 2 then combines the (accumulated) frames with the same major number. Further keywords signal the type of the OB and the parameter set to be used by the pipeline. The parameter set is stored in special ASCII files which can be modified with the parameter editor. Each type of OB has its specific parameter set file. Through this editor also the processing level(s) of the data products can be fully controlled. 4.3 Post-processor components In general, the type of post-processing depends on the type of the OB. A specific module for imaging OBs allows the combination of individual video frames through shift & add. This should give an improved angular resolution and S/N. A full two-dimensional correlation between the individual frames is done and tests indicate that it gives reliable results for point sources with S/N better than 10. Tests on a standard PC indicate that the shift & add processing is not much slower than the expected rate of video frames. Efforts were made to integrate a polarisation measurement sequence into the pipeline. Regarding this all the pipeline subprograms within the several pre- and postprocessor stages have to be adapted to the additional division in analyzer positions. Only data belonging to the same position are allowed to be combined. Within the scope of the standard procedure (i.e. the measurement of the flux at analyzer positions of 0°, 45°, 90°, and 135°) one polarimetric sequence contains four wiregrid positions. Therefore the images of the intermediate processing stages are organized in datacubes consisting of four planes. This concept allows quite a compact processing and organisation of related raw frames and interim results respectively. Finally the data of the several planes are combined to compute polarimetric quantities like the Stokes parameters. All these considerations are valid for the imaging polarimetry mode. Since the basic hardware requirements for doing spectropolarimetry are given the integration of a spectropolarimetry mode into the TNT and the pipeline will be examined some time soon. The post-processing of spectral data is currently planned in a way such that the product of the pipeline can be further reduced within the standard spectroscopic contexts of MIDAS.

Figure 4: Timing sequence for one nod position within the imaging polarimetry mode. The horizontal lines indicate the exposure time within a chop position. Slanted lines indicate a change of the chopping and (should the situation arise) polarizer position. The number of chop cycles within a single analyzer position is changeable by the TNT. The times for the exposure and the chopping movements are not to scale in this sketch.

8

4.4 Software environment The pipeline control consists mainly of UNIX shell and MIDAS scripts, the graphical interface is based on TCL/TK. The actual processing is fully done within MIDAS, either through available modules or by dedicated new components. The pipeline is fully self-contained. There is no need to install extra software except MIDAS, all required components are included in the software package. The pipeline operates under Red Hat Linux and has also been tested on other operating systems. For the distributed development and maintenance of the software and documentation the CVS (Concurrent Version System) package is used.

5. CONCLUSIONS The TIMMI2 is an advanced multi-mode instrument for the 10 and 20 micron atmospheric windows. ESO’s 3.6 m telescope on the other side recently has been upgraded to modern (mostly ESO-VLT-standard) hard and software. Similarly the image quality has been improved substantially. TIMMI2 will be equipped with means for a first order calibration of flat fields. The user interface presented here – the TNT – aims to allow ESO’s visiting astronomers to exploit the technical capabilities without excessive complexity. At the same time, by execution of pre-programmed sequences a high productivity of the instrument can be envisaged. TNT and pipeline for TIMMI2 have been tested to the extent this is reasonably possible before commissioning.

ACKNOWLEDGMENTS This project is supported by the BMBF under grant No. 05 3JN204(5), by the Thüringer Ministerium für Wissenschaft, Forschung und Kultur under grant No. B503-95025 and the Austrian Ministery of Science and Transports and the Östereichische Nationalbank (Jubiläumsfonds-Projekt Nummer 6876).

REFERENCES 1 2 3 4 5 6 7

H. G. Reimann , E.Dietzsch, J. Hron et al., “ TIMMI 2: a new Multimode Mid-Infrared Instrument for the ESO 3.6 m telescope”, Proceedings of SPIE Vol. 4008-131, 2000. H.-G. Reimann, U. Weinert, and S. Wagner, “ TIMMI 2: a new MIR multimode instrument for ESO”, Proceedings of SPIE Vol. 3354, pp. 865-876, 1998. E. Dietzsch and H.-G. Reimann: “TIMMI 2: a combined astronomical MIR camera, Spectrometer, and Polarimeter for ESO”, Proceedings of SPIE Vol. 3482, pp.151-160, 1998. H. U. Käufl and B. Delabre, "Improved Design and Prototyping for a 10/20µm Camera/Spectrometer for ESO`s VLT", Proceedings of SPIE Vol. 2198, pp. 1036-1047, 1994. Chavan A. M., Canavan T., Kemp B., Silva D. R., “New generation phase II proposal preparation tool”. Proceedings of SPIE Vol. 4010-42, 2000. P. Biereichel. Private communication. VLT-MAN-ESO-17230-0942 VLT-TCS User .

9