Introduction to the Data Products

ObsID Directory

The ObsID directory is the top level in the directory tree created when the tarfile is unpacked. It contains two files and two data directories:

unix% pwd
/intro_data/1843

unix% ls -1
oif.fits
primary/
secondary/
vv.1843.001.1.pdf

Observation index file
oif.fits

The observation index file contains a summary of the data products associated with an observation. It is equivalent to a structured directory listing with descriptive information about the observation.

This file is created by the mkoif tool and is used by the firstlook GUI, which provides a simple means of examining your dataset.

V&V report
vv.1843.001.1.pdf

All standard data products are checked by a CXC scientist before release to ensure data quality and to investigate the cause of any exposure losses or other anomalies. This process is known as "verification and validation", or V&V for short. The V&V report is distributed in PDF format with the data files.

Users should review this information before beginning the analysis to ensure that there aren't any caveats from the V&V scientist.

In the following sections, we describe the contents of the primary and secondary data directories.

Primary Directory

The data products are arranged such that all products necessary for most analyses (e.g. the CIAO threads) are in the primary directory.

unix% cd primary/
unix% pwd
/intro_data/1843/primary

unix% ls -1
acisf01843N001_1_sum2.html
acisf01843N001_2_sum2.html
acisf01843N001_3_sum2.html
acisf01843N001_cntr_img2.fits
acisf01843N001_cntr_img2.jpg
acisf01843N001_evt2.fits
acisf01843N001_full_img2.fits
acisf01843N001_full_img2.jpg
acisf01843N001_src2.fits
acisf01843N001_src_img2.jpg
acisf01843_000N001_bpix1.fits
orbitf084197100N001_eph1.fits
pcadf084271087N001_asol1.fits
[hrc_dtf1.fits]
[acis_plt2.jpg]
[acis_pha2.fits]

The majority of these files can be grouped into three categories:

HTML summary files
sum2.html

The HTML files contain a summary of the observation parameters, such as the instrument, detector, and grating that were used. The images from SDP are included in these pages as well.

Images

Two images are produced for every dataset: a full-field sky image (full_img2.jpg) and a high-resolution image of the center of the field (cntr_img2.jpg).

Imaging observations also have a full field image with the source candidates overlaid (src_img2.jpg).

Grating observations include an image of each spectral component plotted as wavlength vs counts (heg_plt2.jpg, meg_plt2.jpg, and/or leg_plt2.jpg).

Remove those three sets of files from the directory listing and you have the primary data products necessary for most analyses:

acisf01843N001_evt2.fits
acisf01843N001_src2.fits
acisf01843_000N001_bpix1.fits
orbitf084197100N001_eph1.fits
pcadf084271087N001_asol1.fits
[hrc_dtf1.fits]
[acis_pha2.fits]

Level 2 event file
evt2.fits

The level 2 event file is the most important data product you receive. This file is created from the level 1 event list by filtering on the good time intervals (GTI) and status bits. The result is a list of events that is suitable for use in data analysis.

When new calibration is released, it is often necessary to remake the level 2 event file in order to apply the changes. This process is explained in the Create a New Level 2 Event File thread.

Source list
src2.fits

A source candidate list. These detections are obtained from a single run of celldetect with S/N threshold equal to 3 and should not be used directly in your analysis. Users interested in spatial analysis should perform their own detection runs with one of the three Chandra Detect tools: celldetect, wavdetect, and vtpdetect. The Detect Manual describes which tool is suited to a particular analysis.

For instructions on displaying the source list over the data, refer to the Using the Output of Detect Tools thread.

This file is only created for imaging observations.

Bad pixels
bpix1.fits

A list of pixels identified as "bad"; criteria for flagging a pixel are listed in the Bad Pixels dictionary entry. Any tool that reads this file will exclude the bad pixels from its calculations.

The Create a New ACIS Bad Pixel File: Identify ACIS Hot Pixels and Cosmic Ray Afterglows thread contains more information, including instructions on how to make a new ACIS bad pixel file, if necessary. The Creating a New Observation-Specific HRC Bad Pixel File thread explains when and how to make a new HRC bad pixel file.

Orbit ephemeris
orbit_eph1.fits

The definitive orbit ephemeris information for the observation. This file is required if you wish to barycenter-correct the observation times in order to account for the difference in photon arrival times as the Earth and Chandra move around the Sun.

The orbit ephemeris is also needed if you are analyzing an object which was moving with respect to the observer when observed, e.g. a planet or comet.

Aspect solution
pcad_asol1.fits

The aspect solution describes the orientation of the telescope as a function of time. The detected position of an event and the corresponding telescope aspect are combined for an accurate determination of the celestial position of that event.

Since an aspect solution file is created for each stable aspect interval, there is often more than one pcad_asol1.fits file for an observation. All the files must be used whenever a tool requires the aspect solution as input.

Detailed information on the aspect solution files is available from the the Aspect Solution why topic.

Dead time factor
dtf1.fits

The dead time correction factors as a function of time. For ACIS observations, the dead time correction is recorded in the DTCOR header keyword. Since the HRC dead time correction factor varies during the observation, however, the value of DTCOR represents an average and the individual values are stored in the dtf1.fits file.

The Computing Average HRC Dead Time Corrections thread explains when you might need to recompute the deadtime statistics for your observation.

This file is only created for HRC observations.

Type II PHA / Spectra
pha2.fits

The PHA file contains a spectrum for one grating part and order in each row of the file. The number of rows varies depending on the detector and grating combination: e.g. ACIS/HETG data has twelve rows while HRC/LETG only has two.

Detailed information on the pha2.fits file is available from the Examining Grating Spectra and Regions thread.

This file is only created for grating observations.

Secondary Directory

If you are interested in reprocessing your data, you will also need the level 1 files, which are in the secondary directory.

unix% cd ../secondary/
unix% pwd
/intro_data/1843/secondary

unix% ls -1
acisf01843_000N001_aoff1.fits
acisf01843_000N001_evt1.fits
acisf01843_000N001_flt1.fits
[hrc_std_flt1.fits]
acisf01843_000N001_msk1.fits
acisf01843_000N001_mtl1.fits
acisf01843_000N001_soff1.fits
acisf01843_000N001_stat1.fits
[hrc_std_dtfstat1.fits]
acisf084271178N001_0_bias0.fits
acisf084271178N001_1_bias0.fits
acisf084271178N001_2_bias0.fits
acisf084271178N001_3_bias0.fits
acisf084271178N001_4_bias0.fits
acisf084271178N001_5_bias0.fits
acisf084272477N001_pbk0.fits
aspect/
ephem/

Aspect offsets
aoff1.fits

A set of offsets versus time (e.g. ra_off=(ra-ra_nom)) . The one piece of remaining useful info is information about gaps in the aspect records. These are derived from the aspect solution files (pcad_asol1.fits, primary directory) for the observation.

Most users will never need this file in their analysis, instead using the aspect solution files directly.

Level 1 event file
evt1.fits

The level 1 event file contains all the events recorded for the observation. While many of these events have a status bit set to flags them as "bad", none of the information has been removed. This file is filtered on GTIs and status bits to create the level 2 event file.

The evt1.fits file is also the starting point for reprocessing your data, as explained in the Create a New Level 2 Event File thread.

Good time intervals
flt1.fits / std_flt1.fits

The GTI information for the observation, e.g. the start and stop times of all accepted time intervals over the observation. The major contributor to creating GTIs is information about when there is aspect data and when that aspect data is good. When the event file is filtered, the GTIs are stored as extensions of the data file, creating a dynamic record of the time filters applied to the data.

Note that this file is named slightly differently for ACIS (flt1.fits) and HRC (std_flt1.fits) observations.

Mask
msk1.fits

The mask file records the valid part of the detector element - ACIS CCD or HRC plate - used for the observation (i.e. the portion for which events can be telemetered). The active portion of an element may be smaller than the default regions if an observation was performed using subarrays or custom windows. This information is used when creating response files, such as ARFs.

Mission timeline
mtl1.fits

The mission timeline (MTL) consists of all time series data that affects the quality of the data, such as aspect and ephemeris data. This information is used to construct the GTI limits in the flt1.fits file. The MTL ahelp file has more information on the mission timeline.

Most users will never need this file in their analysis.

Alignment offsets
soff1.fits

The science instrument module offsets versus time. These are derived from the aspect solution files (pcad_asol1.fits, primary directory) for the observation. The soff1.fits file is binned, so there's not a one-to-one match to the rows in the pcad_asol1.fits files.

Most users will never need this file in their analysis, instead using the aspect solution files directly.

Exposure statistics
stat1.fits

The exposure statistics file contains up to seven extensions, each pertaining to the sequence of ACIS CCD exposures processed and recorded into telemetry.

Most users will never need this file in their analysis.

This file is only created for ACIS observations.

Dead time factor statistics
std_dtfstat1.fits

The dead time factor statistics information is derived from the HRC deadtime factor file (dtf1.fits, primary directory) for the observation.

Most users will never need this file in their analysis.

This file is only created for HRC observations.

Bias map(s)
bias0.fits

When the bad pixel list (bpix1.fits) is created, each bias map is searched for pixels whose bias values are either too low or too high. There is one bias map for each ACIS chip that was used for the observation.

These files are only created for ACIS observations.

Parameter block
pbk0.fits

The parameter block file is needed in conjunction with the bias maps when creating a new bad pixel list. It is used to determine observational parameters, such as which CCDs are active, the READMODE and DATAMODE, etc.

This file is only created for ACIS observations.

unix% cd aspect/
unix% pwd
/intro_data/1843/secondary/aspect

unix% ls -1
pcadf084265646N001_osol1.fits
pcadf084271087N001_aqual1.fits
pcadf084272206N001_osol1.fits
pcadf084278602N001_osol1.fits

unix% cd ../ephem/
unix% pwd
/intro_data/1843/secondary/ephem

unix% ls -1
anglesf01843_000N001_eph1.fits
lunarf084197100N001_eph1.fits
solarf084197100N001_eph1.fits