7 Data processing

7.1 IRAC and MIPS Images

All past IR space missions have succeeded in obtaining maps of extended emission in spite of memory effects in the detectors. It is a major ambition of our team to produce SIRTF maps of faint diffuse emission photometrically valid on all angular scales. For this it is necessary to model variations in detector response with past illumination. The standard calibration procedure using the stimulator flashes will only calibrate the instanteneous response which may be sufficient to accurately measure point sources but not the extended emission. Our plans for map-making rely on our ISO expertise. All the ISO detectors were affected by memory and drift effects which badly hampered the quality and photometry of the observations. Based on the analysis of the ISOCAM and ISOPHOT data we found that the detector memory effects can be separated into short-term transients which could be accounted for using a detector model, either physical or empirical (Pérault et al. 1994, Abergel et al. 1999, Coulais and Abergel 1999, Coulais et al. 2000) and longerm-term effects, too complex to be physically modeled, but which could be removed using redundant coverage of the same sky region (Miville-Deschênes et al. 2000). We will follow the same two-step approach on the SIRTF data. The algorithms developed for IRAC and MIPS imaging in the SEDI pipeline will be made available to the SSC.

Figure 6. Interface of the SEDI processing modules and the SSC pipeline.

We will concentrate our efforts on developing a model that will correct for short-term transient variations of MIPS detector response. The 70 and 160 micron MIPS detectors are Ge:Ga photo-conductors of the same type as those used on ISOPHOT. The Si:As 24 micron detector is a new generation BIB detector which is known to be less affected by transient variations than the ISOCAM detector. However, at this wavelength, the brightness contrast of cirrus clouds will be very low with respect to the zodiacal emission. It is thus for this waveband that the requirements on the correction accuracy will be the highest (a fraction of a percent). Therefore, we expect that a model of this detector will also be required to achieve the optimal sensitivity for diffuse emission. Modeling of the IRAC detectors should not be necessary because they are new generation BIB detectors and the contrast with the cirrus emission is much higher than at 24 micron. We will develop a model for the MIPS short-term detector response similar to that we developped for ISOCAM and ISOPHOT. The model parameters will be fine tuned using MIPS test and in-orbit data.

The long-term transient response consist of systematic detector gain drifts appearing as pixel-to-pixel flat-field variations. They also include residual errors from the short-term transient correction and memory effects after cosmic ray hits. Our ISO work has shown that, for raster mode mapping, the use of the spatial redundancy information can be used to separate instrumental artifacts from the sky structure and correct for them. With ISOCAM, by combining detector modeling with redundancy, we succeeded in imaging structures with contrasts less than 1% of the zodiacal background (Miville-Deschênes et al. 2000) as illustrated in Fig 7. We will adapt the ISOCAM data processing to both IRAC and MIPS in developing a module that will make use of the data redundancy to correct for residual errors. In this module the frame-to-frame overlap is used to derive the slow variations of the flat-field along the course of the observation. The limited redundancy of the 160 micron maps, due to the use of the fastest scan speed is not expected to limit our ability to correct efficiently for long-term response drifts within an AOR, since the gaps along a given scan are smaller than the instrument point-spread function and the method used will be limited to regions of the sky with no high spatial frequency variations.

Figure 7. Illustration of the data processing on an ISOCAM observation with faint diffuse emission. The three images correspond to different steps of the processing. Map (A) was obtained after deglitching, dark substraction and short-term transient correction, map (B) after the long-term drift correction, and map (C) after the application of a time variable flat field and the identification of deviant pixels. All the techniques used here take advantage of the fact that a given point of the sky has been observed several times by different pixels.

The transient and redundancy correction, and zodiacal emission removal modules will be used in conjunction with the SSC pipeline as indicated in Fig.6. We will start working on the MIPS detector modeling as soon as the project is accepted. We will collaborate with the MIPS instrument team and the SSC to get access to test data and to produce simulated data including realistic instrumental effects. We will create the maps in two "generations" (Table 2). Simulated data will be used to develop and test the generation 1 mapping pipeline during Stage 1. This pipeline will be ready at the end of IOC and applied to the First-Look Survey data, and to the SEDI data taken during the first year of operation for rapid distribution to the community. A Generation 2 mapping pipeline, including corrections of residual instrumental effects based on the spatial redundancy will be developed during Stage 2. The SIRTF images will be validated by comparing the two independent coverages. This will quantify the relative photometric uncertainties as a function of angular scales. The absolute photometry of the extended emission will be validated by comparing with the DIRBE brightnesses interpolated at the same wavelengths and for the same observing parameters with respect to the zodiacal light.

7.2 Point Source Catalog

The point source catalog from the SEDI project will be an invaluable source for systematic studies of fluxes and colors and searches for rare objects among an estimated 2x10E6 sources. We will also create the catalog in two generations (Table 2). For the first generation, which is intended for rapid distribution to the community, enabling follow-up observations, we will use the data products and software provided by the SSC. First, we will run the extended SSC pipeline to generate new source lists that combine all of the AORs for each sky region. The reliability of our source lists will be validated by comparing data from the two indepedent coverages that we perform for each sky region. A separate, band-merged catalog will be generated for IRAC and MIPS. For the IRAC catalog, we will determine whether each source has a 2MASS counterpart; if so, the 2MASS source name, J, H, and K fluxes, and coordinates will be added to the catalog, extending the wavelength range of the photometry and improving the position accuracy of the catalog to 0.2 arcsec. For the MIPS catalog, we will identify counterparts using a list of IRAS sources. Finally, we will make a band-merged catalog that combines the MIPS and IRAC catalogs by identifying each MIPS source with IRAC sources having positions within the MIPS position uncertainty. For each catalog, we will add flags to indicate the brightness of diffuse emission (using a low-spatial-frequency version of the 70 micron MIPS map) and confusion (based on the source density within 30 beams of each source).

The second generation of the SEDI source catalogs will be developed as follows. During Stages 1 and 2 of this Legacy project, we will be investigating the effectiveness of different source extraction methods on simulated and SIRTF data, and the First-Look Survey data as they become available. We will also undertake the challenging task of measuring source fluxes in regions with bright, extended emission. These algorithms will include: (1) the neural-net Sextractor (Bertin and Arnouts 1996) algorithm; (2) the PSF-fitting DAOPHOT algorithm as used by 2MASS; and (3) spatial filters based on wavelet transforms, then passed through an adapted Sextractor or DAOPHOT algorithm. With the simulations we will test the ability of each method to properly recover simulated sources added to the data in regions with different levels of diffuse emission brightness. We expect that spatially-filtered images such as wavelet transforms will allow the best separation of point-like and diffuse emission, provided that the effective PSF used in subsequent source extraction is processed in the same manner. We will tune each algorithm for each SIRTF detector--a significant effort that will require significant Stage 1 preparation and Stage 2 analysis.

The final source catalogs from the SEDI project will use the final data processing including time-dependent gain corrections together with the best-tuned source extraction method. We will include many additional columns and flags to the catalog, including a comparison of flux derived from different methods, refined astrometric errors, cirrus indicators based on the HI data collected for this proposal, extended emission flags based on the 70 as well as 8 micron data. These flags will be calibrated into reliability indicators, by injecting simulated point sources into regions with different cirrus brightness.

7.3 Scientific validation of point source catalog

The goal of the scientific validation is to check and quantify the reliability, completeness and the photometric and astrometric accuracy of the SEDI point source catalog. It will the responsibility of the Arcetri group under the leadership of Testi to carry out this task. IPAC will provide to Arcetri separate catalogs extracted from the two independent coverages which will be used for a first check of the reliability. Scientific validation will make use of ground based observations including dedicated near-IR observations in test areas selected to cover a range of source density and brightness of the diffuse emission. The Arcetri team will use field stars to quantify the reliability, the completeness and the photometric accuracy of the SEDI point source catalog for the IRAC bands and the 24 micron MIPS band. Stellar models will be used to estimate SIRTF fluxes from optical and near-IR magnitudes.

The 2MASS data goes deep enough to provide near-infrared magnitudes for all field stars that will be detected at 24 micron, 10E4 stars. For the IRAC bands, the Arcetri team will carry out a ground based near-IR observing program in the J and K bands at ESO. For a given field the expected density of field stars that will be detected in our SIRTF observations decreases significantly with increasing wavelength. For example, for the Rho-Halo field, the densities of stellar sources estimated from the 2MASS observations vary from 6x10E4 to 4000 per square degrees between 3.5 to 8 micron bands. These densities are a factor 7 higher in the Serpens field. The near-IR observing program will combine a deep survey and a shallower survey covering a larger area, distributed over different fields, in order to have at all wavelengths ~10E4 sources to investigate completeness, reliability and photometric accuracy as a function of source density and cloud infrared brightness.

An important concern for this SEDI proposal is the relationship of bright extended emission to the accuracy with which sources can be extracted. This is particularly critical for the 70 and 160 micron MIPS data. To test the reliability, completeness and photometric accuracy of the catalog at these two wavelengths we will make use of the simulation tools developed at IPAC to select the extraction algorithms.

7.4 Spectroscopic Imaging Observations

Our reduction of the IRS and MIPS SED data will make use of the SSC pipeline. We will add to this pipeline a module which will correct for variations in the detector response during an AOR. This module will expand the dynamic range of the observations allowing us to make spectra of faint interstellar emission, with little contrast with respect to the zodiacal background, and to fully use the spatial information provided by the spectrometers. It will take as input the BCDs produced by the SSC pipeline and will produce as output a corrected version of the BCDs. This corrected version can then be processed through the extended SSC pipeline. The modules correcting for response variations will be a direct application of the efforts invested at the IAS to model the MIPS detectors in the framework of the mapping pipeline. This is straightforward for the MIPS SED detector which is that used for imaging at 70 micron and the short wavelength IRS detector (5-14 micron) which is identical to the MIPS 24 micron detector. We are hopeful that a model similar to that of the Si-As MIPS detector can be applied to the long wavelength IRS detector (14-40 micron). We will start working on the modeling of this IRS detector as soon as IRS IOC data are available. The set of sources we will choose for IRS spectrosocpy (second-look observations to be started 1 year after the end of IOC) will be selected according to the accuracy with which we will succeed in modeling the detector response. It is for the long wavelength segment of IRS that the small contrast between the interstellar emission and the zodiacal background sets the most stringent requirements on the stability of the response (<1%) if we want to obtain the optimal sensitivity. For the MIS SED data, slight modifications of the SSC extended pipeline will be made in order to process independently the on and off positions without making a subtraction. This is necessary since we will be observing diffuse emission extended on scales larger than the beam throw (< 3'). For both IRS and MIPS SED observations, the sky subtraction will be based on the reference position obseved in cluster mode within the same AOR as the on positions.