CalDB 4.5.0 Public Release Notes

I. INTRODUCTION

CalDB 4.5.0 is an upgrade to the Chandra CalDB, which includes the following items:

• Calbration Validity End Date Corrections for 38 ACIS T_GAIN files, 1 ACIS CTI, 4 HRC-I GMAPs, 3 HRC-I PIBGSPECs, and 1 HRC-S QEU file
• ACIS FP_TEMP Boundary Upper Limit Change for all current ACIS files with FP_TEMP boundary conditions
• HRC-I and HRC-S GAPLOOKUP Upgrades

For the CIAO 4.4.1 / CalDB 4.5.0 release notes see How CalDB 4.5.0 Affects Your Analysis

II. SUMMARY OF CHANGES

A. Calbration Validity End Date Corrections

ACIS T_GAIN files:
Location:	$CALDB/data/chandra/acis/t_gain	Affected OBS_IDs
Filenames:	acisD2000-01-29t_gainN0006.fits acisD2000-05-01t_gainN0006.fits acisD2000-08-01t_gainN0006.fits acisD2001-02-01t_gainN0006.fits acisD2001-05-01t_gainN0006.fits acisD2001-11-01t_gainN0006.fits acisD2002-02-01t_gainN0006.fits acisD2002-05-01t_gainN0006.fits acisD2002-08-01t_gainN0006.fits acisD2002-11-01t_gainN0006.fits acisD2003-05-01t_gainN0006.fits acisD2003-08-01t_gainN0006.fits acisD2003-10-19t_gainN0006.fits acisD2003-11-18t_gainN0006.fits acisD2004-01-01t_gainN0005.fits acisD2004-02-01t_gainN0006.fits acisD2004-08-01t_gainN0006.fits acisD2004-11-01t_gainN0006.fits acisD2005-02-01t_gainN0006.fits acisD2005-11-01t_gainN0006.fits acisD2006-02-01t_gainN0006.fits acisD2006-11-01t_gainN0006.fits acisD2007-02-01t_gainN0006.fits acisD2007-05-01t_gainN0006.fits acisD2007-11-01t_gainN0006.fits acisD2008-02-01t_gainN0006.fits acisD2008-05-01t_gainN0006.fits acisD2008-11-01t_gainN0006.fits acisD2009-02-01t_gainN0006.fits acisD2009-05-01t_gainN0006.fits acisD2009-08-01t_gainN0006.fits acisD2010-02-01t_gainN0006.fits acisD2010-05-01t_gainN0006_revA.fits acisD2010-08-01t_gainN0006.fits acisD2010-11-01t_tgainN0006.fits acisD2011-02-01t_gainN0006.fits acisD2011-05-01t_gainN0006.fits acisD2011-08-01t_gainN0006.fits	649 79 547 2459 1982 3436 3138 2532 3339 4107 3802 3512 5011 4546 4518 5142 4676 6186 4537 6771 5932 7910 7943 8211 7911 9087 9892 10118 10464 10808 10459 11736 11333 13158 12399 12317 12891 13535
ACIS CTI file:
Location:	$CALDB/data/chandra/acis/cti/
Filenames:	acisD2000-01-29ctiN0007.fits	2532
HRC-I GMAP files:			Corresponding HRC-I PIBGSPEC files:
Location:	$CALDB/data/chandra/hrc/gmap/		$CALDB/data/chandra/hrc/pibgspec/
Filenames:	hrciD1998-10-30sampgainN0001.fits hrciD2002-01-26sampgainN0001.fits hrciD2007-09-17sampgainN0001.fits hrciD2009-09-24sampgainN0001.fits	1364 4296 9643 11909	Filenames:	None hrciD2002-01-26pibgspecN0001.fits hrciD2007-09-17pibgspecN0001.fits hrciD2009-09-24pibgspecN0001.fits
HRC-S QEU file:
Location:	$CALDB/data/chandra/hrc/qeu/
Filenames:	hrcsD2000-01-01qeuN0006.fits	3144

It has been discovered recently, and announced on the CalDB4 Caveats Page here, that a problem in the CalDB has rendered certain specific observations unable to find T_GAIN, CTI, or HRC-I GMAP, or HRC-S QEU files, either in SDP or in CIAO. These problems have existed in some cases since CalDB 4.1.0 was released in December of 2008. The quickest fix to the problem is to make small corrections in the calibration validity end dates of the specific CalDB files that should be automatically selected for these specific OBS_IDs. In each case, the change extends the validity date range for the given file by enough to accommodate the DATE-END of each OBS_ID. These fixes are being released in the current CalDB 4.5.0 upgrade. In a later CIAO release there will be a bug fix to the CALDB4 software in CIAO to remove the need for strict validity end dates for time-displaced datasets in CalDB.

Included among these corrections are 38 ACIS T_GAIN files, 1 ACIS time-dependent CTI file, 4 HRC-I GMAP files (and three corresponding PIBGSPECs), and 1 HRC-S QEU dataset. The files (with their corresponding OBS_IDs) are all named above. All the files named have been previously released in CalDB; only the calibration validity end dates have been updated, so their version numbers have not changed. Users of OBS_IDs other than the specific ones named above are not affected by these changes at all, and should seen no change in the CalDB performance.

PIPELINES/TOOLS AFFECTED:

For ACIS T_GAIN and CTI, and HRC-I GMAPs: chandra_repro script

For ACIS T_GAINs and CTI: acis_process_events

For HRC-I GMAPs: hrc_process_events

For HRC-S QEU: fullgarf script, mkexpmap (mkinstmap), fluximage

THREADS AFFECTED:

Reprocessing Data to Create a New Level=2 Event File

Object-centered Aspect Solution and Exposure Map

Reprojecting Images: Making an Exposure-corrected Mosaic

The HRC-I Background Spectra Files

B. ACIS FP_TEMP Boundary Upper Limit Change

Location:	$CALDB/data/chandra/acis/badpix/
Filenames:	acisD2000-11-28badpixN0003.fits acisD2000-11-28badpix_secN0003.fits

Location:

$CALDB/data/chandra/acis/bkgrnd/

Filenames:

acis0D2005-09-01bgstow_ctiN0001.fits acis0iD2005-09-01bkgrnd_ctiN0001.fits acis1D2005-09-01bgstow_ctiN0001.fits acis1iD2005-09-01bkgrnd_ctiN0001A.fits acis2D2005-09-01bgstow_ctiN0001.fits acis2iD2005-09-01bkgrnd_ctiN0001.fits acis2sD2005-09-01bkgrnd_ctiN0001.fits acis2sD2005-09-01bkgrndN0001.fits acis3D2005-09-01bgstow_ctiN0001.fits acis3iD2005-09-01bkgrnd_ctiN0001.fits acis3sD2005-09-01bkgrnd_ctiN0001.fits acis3sD2005-09-01bkgrndN0001.fits acis5D2005-09-01bgstow_ctiN0001.fits acis5sD2005-09-01bkgrnd_ctiN0001.fits acis5sD2005-09-01bkgrndN0001.fits acis6D2005-09-01bgstow_ctiN0001.fits acis6iD2005-09-01bkgrnd_ctiN0001.fits acis6sD2005-09-01bkgrnd_ctiN0001.fits acis6sD2005-09-01bkgrndN0001.fits acis7D2005-09-01bgstow_ctiN0001.fits acis7iD2005-09-01bkgrnd_ctiN0001.fits acis7sD2005-09-01bkgrnd_ctiN0001.fits acis7sD2005-09-01bkgrndN0001.fits acis8sD2005-09-01bkgrnd_ctiN0001.fits acis8sD2005-09-01bkgrndN0001.fits

Location:	$CALDB/data/chandra/acis/cti/
Filenames:	acisD2002-08-01ctiN0005B.fits acisD2002-08-01ctiN0007.fits

Location:	$CALDB/data/chandra/acis/det_gain/
Filenames:	acisD2000-01-29gain_ctiN0005.fits acisD2000-01-29gain_ctiN0006.fits acisD2000-01-29gainN0005.fits

Location:	$CALDB/data/chandra/acis/fef_pha/
Filenames:	acisD2000-01-29fef_pha_ctiN0004.fits acisD2000-01-29fef_phaN0005.fits

Location:	$CALDB/data/chandra/acis/osip/
Filenames:	acisD2000-01-29osip_ctiN0005.fits acisD2000-01-29osip_ctiN0006.fits acisD2000-01-29osipN0006.fits

Location:	$CALDB/data/chandra/acis/p2_resp/
Filenames:	acisD2000-01-29p2_respN0005.fits acisD2000-01-29p2_respN0006.fits

Location:	$CALDB/data/chandra/acis/qeu/
Filenames:	acisD2000-01-29qeuN0005.fits acisD2000-01-29qeuN0006.fits

Location:	$CALDB/data/chandra/acis/t_gain/
Filenames:	acisD2012-02-01t_gain_biN0002.fits acisD2012-02-01t_gainN0002.fits

With the reduction in the Chandra X-ray Observatory operational avoidance window for the radiation zone passes, somewhat higher temperatures may exist on the ACIS focal plane at the start of observations as the mission progresses. There has already been one such observation, currently proprietary, with focal plane temperature (FP_TEMP) starting at 159.74 Kelvin, which resulted in a failure to process in Standard Data Processing (SDP), due to the FP_TEMP boundary limits on ACIS CTI, DET_GAIN, and T_GAIN files. The same would be true in reprocessing and downstream analysis where P2_RESP, QEU, FEF_PHA, OSIP when relevant, and BKGRND may be needed. Hence we have elected to expand the upper limit of the FP_TEMP boundary condition from 159.16K (~-114C) to 164.16K (~-109C) for all currently-valid ACIS calibration files that are bounded on FP_TEMP.

Please note that this change only affects future "warm" observations, and has no affect whatsoever on users whose observations have been taken an nominal or near-nominal temperatures. Recall that users of observations with FP_TEMP values above 159.16K or approximately -114C are advised in their VV reports of this circumstance, because the ACIS temperature-dependent CTI correction calibration has been verified only up to that temperature. So, while CalDB will not fail to deliver calibrations for these warm OBS_IDs, users may still wish to filter their warmer events before analyzing, depending on their specific studies. Currently, an analysis thread to filter out time intervals with out-of-range temperature events is being developed.

Because of the general nature of these changes in CalDB, it will not be useful to specify pipelines, tools, or threads affected by this FP_TEMP boundary change, because the change really only affects certain current and future ACIS observations with higher-than-nominal focal plane temperatures.

C. HRC-I and HRC-S GAPLOOKUP Upgrades

Location:	$CALDB/data/chandra/hrc/
Filenames:	hrciD1999-07-22gaplookupN0004.fits hrcsD1999-07-22gaplookupN0003.fits

The HRC Calibration Team and the HRC Instrument Scientist have developed improved DEGAP solutions, currently formatted in the GAPLOOKUP tables in CalDB, for both the HRC-I and HRC-S detectors. For the HRC-I, the corrections are more modest, with improvement basically in the pre-existing bias toward overestimated degap corrections near the gap boundaries. Hence, users are not likely to see significant changes using the new HRC-I GAPLOOKUP unless they are looking for very small (0.1 arcsecond) deviations particularly for time-dependent data. In this latter case, users should reprocess their data with the new HRC-I GAPLOOKUP table.

For the HRC-S, the telescope aimpoint drift over time has caused the dither pattern to reach the CRSV=100 tap on the detector. The current HRC-S GAPLOOKUP table does not provide an andequate degap solution for this case. The events on the CRSV=100 tap become mislocated, causing a distortion of the image near the aimpoint, for both imaged sources and grating zeroth orders. Users may determine the fraction of events appearing on CRSV=100 using the following datamodel commands in CIAO:

• dmlist evt1file counts
• dmlist evt1file"[CRSV=100]" counts

where the evt1file is filtered for the source(s) of interest, or for the zeroth order image of an LETGS observation. If the fraction is significant, as a general rule, reprocessing using the new HRC-S GAPLOOKUP will be beneficial.

The HRC calibration team has recommended that all archival HRC data be reprocessed with the new GAPLOOKUP tables, including the more recent HRC-S data that have already been repro'd with the previous GAPLOOKUP table. Furthermore they have also recommended that users with HRC-S observations since the beginning of 2010 check their events using a procedure such as the one above, and reprocess their observations if the CRSV=100 events are a significant fraction.

For more information on these updates, see the Technical Details Section.

See the HRC DEGAP why page for more information.

PIPELINES/TOOLS AFFECTED:

hrc_process_events

chandra_repro script

THREADS AFFECTED:

Reprocessing Data to Create a New Level=2 Event File

III. TECHNICAL DETAILS

A. Calbration Validity End Date Corrections

The CalDB Index Files (CIF) store the information necessary for proper file selection, including the calibration validity start dates and times, and validity end dates and times. The CAL_VSD and CAL_VED columns are populated, at least initially, with the CVSD* and CVED* keyword set values in the various CalDB FITS file headers. With the CalDB4 maintenance software it is possible to compare the CalDB file headers with the CalDB index using the "calvalid" command. Hence, when updates to the CIF are desired, the fileheaders for each listing are also changed to keep them consistent.

B. ACIS FP_TEMP Boundary Upper Limit Change

The new standard FP_TEMP boundary range for all currently valid ACIS calibration files in CalDB is FP_TEMP(151.16 - 164.16)K. The current temperature-dependent CTI correction is valid over the range of -122 to -114 C (151.15 - 159.15)K, and in fact was derived using ACIS External Calibration Source (ECS) datasets taken during radiation zone passes. The temperature-dependent CTI correction files were released with CalDB 4.4.0 in November 2010.

The modified FP_TEMP boundary condition allows the SDP and CIAO software to run without failure to find CalDB files essential to the procedures. While these calibrations will not be fully within specifications when applied to observations with out-of-range operating temperatures, steps may be taken to mitigate by filtering the events on FP_TEMP values. In any case, once an appropriate procedure for filtering out time intervals with out-of-temperature-range events is developed, the affect of the high temperature on the data may be determined by comparing results of analyses with both filtered and unfiltered events.

C. HRC-I and HRC-S GAPLOOKUP Upgrades

The HRC calibration teams and HRC instrument scientist Mike Juda have released two new GAPLOOKUP tables for the HRC detectors. For HRC-I there are small improvements, but for HRC-S particularly near the aimpoint (or zeroth order position for LETGS) the improvements are more significant and necessary.

The primary motivation for the recent GAPLOOKUP developments is the difficulty in the previous degap solutions on HRC-S due to the aimpoint drift, which has produced some significant problems in the LETGS zeroth order image, as well as point sources on HRC-S near that aimpoint. These difficulties are documented on the web page:

http://cxc.cfa.harvard.edu/cal/Letg/CRSV100/index.html

Summarizing this issue, aimpoint drift on the HRC-S over time, recent observations using the HRC-S (whether using a grating or not) may have a small fraction of source (or 0th order) events that fall on the CRSV=100 tap. The current degap map (hrcsD1999-07-22gaplookupN0002.fits, from 2005/11/16), which converts event positions from RAW to CHIP (which are then converted to DET and TDET, and ultimately to SKY) has residual errors for that tap, leading to asymmetry and broadening of the PSF. It does not affect the positions of events dispersed by the LETG, although the wavelength calibration may be slightly affected, due to the distortion of the zeroth order, and hence its centroid determination. Fig 1 below illustrates the effect on a zeroth order image:

Fig. 1: 0th order in SKY coordinates using only events with CRSV=98:99 (left) or CRSV=100 (right) for OBS_ID 13025. The green circle (5-pixel radius) is centered on the same pixel in each.

/contrib/juda/memos/hrci_degap_lookup/capella/degap/2012/index.htm

This problem started to show up in late 2010 as the dither pattern of sources at the aim point began to drift onto CRSV=100. We are aware of 10 GO/GTO observations of point sources that have been affected: ObsID 12154, 12203 (just barely affected), 12332, 12324, 12325, 12401, 12916 (probably negligible), 13184, 13240, and 13651. There is no guarantee, however, that this list is complete. Observers concerned about the source/0th order PSF should check any observation made since the beginning of 2010 following the procedures outlined below. In addition, any observation made since the beginning of the mission that observes a field with multiple or extended sources may be affected. Users will need to be informed of a means to determine how their specific observations have been affected, so as to generate a figure such as Fig. 1 above for point sources, or a more general means to use for extended sources.

The development of the improved GAPLOOKUP table for HRC-S (2012) is presented in the web page http://cxc.harvard.edu/contrib/juda/memos/hrcs_degap_lookup/ Summarizing this page, approximately 60 HRC-S observations (some with and some without LETG) have been selected for deriving new improved HRC-S degapping corrections, using the same algorithms as used for HRC-I in 2005. This method uses the aspect solution to predict the location of the aimpoint on the detector and find the mean RAW position of the events for a given modeled position for events in the combined data set. For each ObsID events were extracted for a 20-pixel radius centered on the on-axis source/zeroth-order, as data to provide a comparison to these modeled positions, whence to derive a correction to the HRC-S N0002 degapping correction currently in CalDB. HRC Instrument Scientist Mike Juda used two iterations of the process to derive new corrections.

These new corrections have been loaded into the new GAPLOOKUP file for HRC-S, which has been posted at http://cxc.harvard.edu/contrib/juda/memos/hrcs_degap_lookup/hrcs_gaplookup_test_f.fits and will be installed in CalDB as hrcsD1999-07-22gaplookupN0003.fits.

The derivation information for the HRC-I GAPLOOKUP upgrade for 2012 is given in the following posting: http://cxc.harvard.edu/contrib/juda/memos/hrci_degap_lookup/capella/degap/2012/index.html. Summarizing, the degap look-up calibration that was developed in 2007 (HRC-I GAPLOOKUP version N0003) suffers from a bias to larger than necessary correction near the gap boundaries. This is illustrated in Fig. 2 below which shows the data, information used to derive the degap values and degap values near one of the edges of CRSU = 30. The degap look-up values are determined by calculating the source position on the detector for each event and then determining the mean of the RAW position for a given modeled position with selected coarse position (e.g. CRSU) and AMP_SF value. As the modeled position nears the gap, the deviation is biased to a larger correction as events begin to appear on the adjacent tap.

Fig. 2: Example of the bias in the derived degap look-up values near a gap boundary (dashed vertical line). Dots are from the events from Capella that form the basis of the determination. The red error-bars show the mean deviations of events at a given modeled position re-cast into the RAW space. The solid black curve is the degap look-up devrived from the red-points. The solid blue curve is the update described in the web page above.

In the development of the degap look-up values for near aimpoint region the HRC-S (see above), Mike Juda explored improvements to the derivation of the values in this near-gap region so as to remove or reduce the bias. First, the normal algorithm is used to determine the degap look-up values. Then the values within a padregion relative to the gap are reset to values determined from a linear extrapolation from a region of the same width just inside the pad. The blue curve in Fig. 2 is the result of applying this modification.

The HRC-I 2012 GAPLOOKUP update file has been posted at: http://cxc.harvard.edu/contrib/juda/memos/hrci_degap_lookup/capella/degap/2012/hrci_gaplookup_test_d.fits which will be delivered in CalDB as hrciD1999-07-22gaplookupN0004.fits.

The improvement in the HRC-I degap solution from the new GAPLOOKUP table as applied to OBS_ID13182 is illustrated in Fig. 3 below:

Fig. 3: CHIP coordinate comparison of AR Lac source events from ObsID 13182. Left pane uses degap corrections currently in CalDB (HRC-I GAPLOOKUP version N0003) and that are biased near the gap regions. Right pane uses the new HRC-I GAPLOOKUP N0004 that has the near-gap corrections modified. (Click on the figure to enlarge the image to enhance the illustration.)