Skip to the navigation links
Last modified: 27 September 2010

URL: http://cxc-newtest.cfa.harvard.edu/ciao4.2/releasenotes/ciao_4.2_release.html

CIAO 4.2 Release Notes


CIAO 4.2 was released on 15 December 2009.

[Updated] CALDB was upgraded to v4.3.1 on 28 Sept 2010.

The Sherpa modeling and fitting package has a number of enhancements, such as a new confidence method, full support for wavelength analysis, and a "save_all" command to save the session in ASCII format. Several of Sherpa's computationally intensive functions have been parallelized, to make use of multi-core systems (i.e., laptops or desktops with 2 or 4 cores).

The ChIPS plotting application includes beta support for displaying images and colorbars, as well as the ability to create an editable ASCII save file.

Other notable changes in CIAO 4.2 are:

  • Two new tools - create_bkg_map and dmimgpm - enable users to create spatial background maps equivalent to those in the Chandra Source Catalog.

  • acis_build_badpix allows users more control over customizing the bad pixel files.

  • hrc_process_events, combined with files released in CALDB 4.2.0, can apply time-variable gain calibrations. Using the gain-corrected PI values can reduce the background.

  • tg_create_mask has improved the custom mask settings.



How CALDB 4.3.1 Affects Your Analysis

Some calibration files in CALDB 4.3.1, released on 28 Sept 2010, affect analyses in progress. This section of the release notes describes those changes and which tasks are affected. The changes are arranged by instrument and grating configuration.

ACIS Imaging and Grating Data

  • Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

    There are new time-dependent ACIS Gain (TGAIN) files for 01 May 2010 - 31 July 2010 (Epoch 42):

    acisD2010-02-01t_gainN0005.fits
    acisD2010-02-01t_gainN0006.fits
    acisD2010-05-01t_gainN0002.fits
    acisD2010-05-01t_gain_biN0002.fits

    The files are applicable to -120 C focal plane temperature only. The acisD2010-05-01t_gain_biN0002.fits file is the new default processing file in the CALDB.

    These plots illustrate the degree of change the new files will produce in comparison with the old files.

    The differences in these figures are attributed to whether the T_GAINs are derived from CTI-corrected data or non-CTI data.

    Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

    Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

  • ACIS Blank-sky Background Files for ACIS-8

    Two of the ACIS blank-sky background files for the ACIS-8 chips have been updated with improved streak removal via the destreak tool:

    acis8sD2000-12-01bkgrnd_ctiN0002.fits
    acis8sD2000-12-01bkgrndN0006.fits
    

    The CALDB 4.3.1 release notes contain images of the original and destreaked background files.

    Note that the ACIS background files are not included in the main CALDB tarfile. There is a separate background files tarfile available via ciao-install or from the CALDB Download page.


How CALDB 4.3.0 Affects Your Analysis

Some calibration files in CALDB 4.3.0, released on 10 June 2010, affect analyses in progress. This section of the release notes describes those changes and which tasks are affected. The changes are arranged by instrument and grating configuration.

ACIS Imaging and Grating Data

  • Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

    There are new time-dependent ACIS Gain (TGAIN) files for 01 February 2010 - 30 April 2010 (Epoch 41):

    acisD2009-11-01t_gainN0005.fits
    acisD2009-11-01t_gainN0006.fits
    acisD2010-02-01t_gainN0002.fits
    acisD2010-02-01t_gain_biN0002.fits

    The files are applicable to -120 C focal plane temperature only. The acisD2010-02-01t_gain_biN0002.fits file is the new default processing file in the CALDB.

    These plots illustrate the degree of change the new files will produce in comparison with the old files.

    The differences in these figures are attributed to whether the T_GAINs are derived from CTI-corrected data or non-CTI data.

    Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

    Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

HRC-I Imaging Data

  • HRC-I Gain Maps

    The latest time-dependent gain map (GMAP) for the HRC-I, effective at UTC 2009-09-24T12:00:00, has been released.

    Technical details on the gain maps are described in the memo SUMAMPS-based Gain Maps for the HRC-I.

    Users working with HRC-I data should reprocess to improve the gain calibration in their data.

  • Background Event Files

    The High Resolution Camera (HRC) detectors are occasionally active while they are in a stowed position. Data taken during these active intervals have been released by the HRC calibration team as a set of stowed background event files for the HRC-I detector They may be used for imaging or spatial analyses, e.g. when creating a fluxed image.

    The HRC-I background event files are not included in the main CALDB tarfile. There is a separate HRC background event tarfile available via ciao-install or from the the CALDB Download page.

    unix% ls -1 $CALDB/data/chandra/hrc/bkgrnd/
    hrciD1999-12-06bkgrndN0001.fits
    hrciD2001-01-01bkgrndN0001.fits
    hrciD2002-01-01bkgrndN0001.fits
    hrciD2003-01-01bkgrndN0001.fits
    hrciD2004-01-01bkgrndN0001.fits
    hrciD2005-01-01bkgrndN0001.fits
    hrciD2006-01-01bkgrndN0001.fits
    hrciD2007-01-01bkgrndN0001.fits
    hrciD2008-01-01bkgrndN0001.fits
    hrciD2009-01-01bkgrndN0001.fits
    

    Note that the file hrciD1999-12-06bkgrndN0001.fits is valid from 1999-12-06T00:00:00 to 2001-01-01T00:00:00. It is used for analysis for data taken in 2000. All the other files are valid from 01 January of the year in the filename to 01 January of the following year.

    For instructions on using the background event files, follow the The HRC-I Background Event Files thread.

  • Background Spectra Files

    The HRC calibration team has released a set of background spectra from the HRC-I. These spectra describe the particle background of the detector and vary slowly with time which means that they can be used to significantly improve the signal-to-noise of HRC-I imaging data and may be useful for point source analysis (e.g. fields with many point sources).

    The HRC-I background spectra files are included in the main CALDB tarfile.

    unix% ls -1 $CALDB/data/chandra/hrc/pibgspec/
    hrciD1999-10-04pibgspecN0001.fits
    hrciD2000-12-12pibgspecN0001.fits
    hrciD2002-01-26pibgspecN0001.fits
    hrciD2003-02-22pibgspecN0001.fits
    hrciD2004-11-25pibgspecN0001.fits
    hrciD2005-10-17pibgspecN0001.fits
    hrciD2006-09-20pibgspecN0001.fits
    hrciD2007-09-17pibgspecN0001.fits
    hrciD2008-09-07pibgspecN0001.fits
    hrciD2009-09-24pibgspecN0001.fits
    

    For instructions on using the background event files, follow the The HRC-I Background Spectra Files thread.

HRC-I Grating Data


How CALDB 4.2.2 Affects Your Analysis

There were two errors in CALDB 4.2.1 that have been fixed in the CALDB 4.2.2 release:

  • There was an error in the ACIS index file for one of the TGAIN Epoch 40 datasets. This produced a harmless warning in acis_process_events.

  • An unrelated error existed in the latest ACIS-I1 background file; it contained only one column and was not usable.

With the exception of the corrections to these two files, the release notes are identical to the CALDB 4.2.1 release notes.


How CALDB 4.2.1 Affects Your Analysis

Some calibration files in CALDB 4.2.1, released on 5 April 2010, affect analyses in progress. This section of the release notes describes those changes and which tasks are affected. The changes are arranged by instrument and grating configuration.

Installing CALDB 4.2.1

  • This CALDB release includes an "upgrade" package which updates CALDB 4.2.0 to version 4.2.1. If you already have CALDB 4.2.0 installed, download the CALDB v4.2.1 upgrade package from the CALDB page.

    Users who do not have CALDB 4.2.0 installed should run ciao_install to get the full CALDB 4.2.1 installation.

ACIS Imaging Data

  • Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

    There are new time-dependent ACIS Gain (TGAIN) files for 01 November 2009 - 31 January 2010 (Epoch 40):

    acisD2009-08-01t_gainN0005.fits
    acisD2009-08-01t_gainN0006.fits
    acisD2009-11-01t_gainN0002.fits
    acisD2009-11-01t_gain_biN0002.fits
    

    The files are applicable to -120 C focal plane temperature only. The acisD2009-11-01t_gain_biN0002.fits file is the new default processing file in the CALDB.

    These plots illustrate the degree of change the new files will produce in comparison with the old files.

    The differences in these figures are attributed to whether the T_GAINs are derived from CTI-corrected data or non-CTI data.

    Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

    Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

  • ACIS Blank-Sky Background Files

    A new set of ACIS blank-sky background files have been released for observations starting in mid-2005, when the spectral shape of the ACIS-S3 background (CCD 7) started to change. The spectral shape of the front-illuminated chips from the same period is still constant.

    acis0D2005-09-01bgstow_ctiN0001.fits
    acis0iD2005-09-01bkgrnd_ctiN0001.fits
    acis1D2005-09-01bgstow_ctiN0001.fits
    acis1iD2005-09-01bkgrnd_ctiN0001.fits
    acis2D2005-09-01bgstow_ctiN0001.fits
    acis2iD2005-09-01bkgrnd_ctiN0001.fits
    acis2sD2005-09-01bkgrnd_ctiN0001.fits
    acis2sD2005-09-01bkgrndN0001.fits
    acis3D2005-09-01bgstow_ctiN0001.fits
    acis3iD2005-09-01bkgrnd_ctiN0001.fits
    acis3sD2005-09-01bkgrnd_ctiN0001.fits
    acis3sD2005-09-01bkgrndN0001.fits
    acis5D2005-09-01bgstow_ctiN0001.fits
    acis5sD2005-09-01bkgrnd_ctiN0001.fits
    acis5sD2005-09-01bkgrndN0001.fits
    acis6D2005-09-01bgstow_ctiN0001.fits
    acis6iD2005-09-01bkgrnd_ctiN0001.fits
    acis6sD2005-09-01bkgrnd_ctiN0001.fits
    acis6sD2005-09-01bkgrndN0001.fits
    acis7D2005-09-01bgstow_ctiN0001.fits
    acis7iD2005-09-01bkgrnd_ctiN0001.fits
    acis7sD2005-09-01bkgrnd_ctiN0001.fits
    acis7sD2005-09-01bkgrndN0001.fits
    acis8sD2005-09-01bkgrnd_ctiN0001.fits
    acis8sD2005-09-01bkgrndN0001.fits

    The Using the ACIS "Blank-Sky" Background Files thread (S-Lang or Python) contains instructions on how to select a file and match it to a specific observation.

    Note that the ACIS background files are not included in the main CALDB tarfile. There is a separate background files tarfile available via ciao-install or from the CALDB Download page.

ACIS Grating Data

  • Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

    There are new time-dependent ACIS Gain (TGAIN) files for 01 November 2009 - 31 January 2010 (Epoch 40):

    acisD2009-08-01t_gainN0005.fits
    acisD2009-08-01t_gainN0006.fits
    acisD2009-11-01t_gainN0002.fits
    acisD2009-11-01t_gain_biN0002.fits
    

    The files are applicable to -120 C focal plane temperature only. The acisD2009-11-01t_gain_biN0002.fits file is the new default processing file in the CALDB.

    These plots illustrate the degree of change the new files will produce in comparison with the old files.

    The differences in these figures are attributed to whether the T_GAINs are derived from CTI-corrected data or non-CTI data.

    Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

    Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

  • LETG Grating Efficiency (version N0006)

    The LETG grating efficiency file (GREFF) was updated to be version N0006. The new file will be used by default when creating LETG response files (ARFs). The CIAO tools and scripts which use this file are:

    There is no change in the responses for +/1 order. The revised effective area curves (ARF files) for orders 2-7 will be lower than those created with the previous LETG GREFF calibration by 3-9%.

    The modified grating efficiencies relative to 1st order are estimated to be accurate to +/-10% for m=2 and 3, but efficiencies in 4th order may be too large by 10%, 10-20% for m=5 and 6, and 10-30% in 7th order. Uncertainties for several orders beyond m=7are probably of order +/- 30%.

HRC-S/LETG Grating Data

  • LETG Grating Efficiency (version N0006)

    The LETG grating efficiency file (GREFF) was updated to be version N0006. The new file will be used by default when creating LETG response files (ARFs). The CIAO tools and scripts which use this file are:

    There is no change in the responses for +/1 order. The revised effective area curves (ARF files) for orders 2-7 will be lower than those created with the previous LETG GREFF calibration by 3-9%.

    The modified grating efficiencies relative to 1st order are estimated to be accurate to +/-10% for m=2 and 3, but efficiencies in 4th order may be too large by 10%, 10-20% for m=5 and 6, and 10-30% in 7th order. Uncertainties for several orders beyond m=7are probably of order +/- 30%.


How CIAO 4.2 and CALDB 4.2.0 Affect Your Analysis

Some tool changes and calibration files released in CIAO 4.2 and CALDB 4.2.0 affect analyses in progress. This section of the release notes describes those changes and which tasks are affected.

Compatibility

  • CALDB 4.2.0 is required in order for CIAO 4.2 to work correctly.

    It is not backward-compatible: CALDB 4.2.0 cannot be used with any CIAO software release prior to v4.2.

ACIS Imaging and Grating Data

  • Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

    There are new time-dependent ACIS Gain (TGAIN) files for 01 August 2009 - 30 October 2009 (Epoch 39):

    acisD2009-05-01t_gainN0005.fits
    acisD2009-05-01t_gainN0006.fits
    acisD2009-08-01t_gainN0002.fits
    acisD2009-08-01t_gain_biN0002.fits
    

    The files are applicable to -120 C focal plane temperature only. The acisD2009-08-01t_gain_biN0002.fits file is the new default processing file in the CALDB.

    These plots illustrate the degree of change the new files will produce in comparison with the old files.

    The differences in these figures are attributed to whether the T_GAINs are derived from CTI-corrected data or non-CTI data.

    Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

    Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

  • ACIS QE Contamination Model vN0005

    The ACIS QE contamination model has been upgraded to vN0005. This version of the file contains separate components for the correction on ACIS-S and ACIS-I.

    The changes in the vN0005 file will primarily affect low-energy absorption components between C-K edge (0.283 keV) and 1.2 keV. In general, hard spectra will not be affected much by the calibration change.

    These CIAO response tools automatically apply the contamination file when creating ACIS response files:

    As well as the scripts which use them:

    The following is a summary of the changes for ACIS-S and ACIS-I. For complete technical details, refer to the memo on the update to the contamination model from the CXC Calibration team.

    ACIS-S

    All ACIS-S observations taken after 2005 should use the vN0005 file to create the response files.

    There was an acceleration in the QE degredation, illustrated in the following figure. The plot shows the difference in the measured optical depth between the top and center of ACIS-S3 (red data points) and between the bottom and center of ACIS-S3 (blue data points). The solid line is the model used for version N0005 of the ACIS-S contamination model.

    Model used for the vN0005 of the ACIS-S contamination model.

    The major changes in this new ACIS-S model are:

    1. an adjustment in the time-dependence to account for the accelerated build-up of contaminant on ACIS-S3 relative to extrapolations from the vN0004 file, and

    2. a new empirical determination of the extra absorption component, derived from two ACIS-S3 spectra of the Coma cluster (one taken in 1999, and the other in 2009).

    The following sets of plots illustrate how the ACIS-S change will affect response files.

    Measured fluxes of E0102

    Both figures plot the 0.3-1.16 keV flux in E0102-72 observations on ACIS-S3. Blue points are in full-frame mode, and the red ones are with ACIS-S3 in a sub-array mode. (There is no correction for pile-up applied here, so the sub-array gives a little higher flux.)

    Note the much decreased trends with time, including no systematic deterioration in the intrinsic source flux. For these data, the decrease in the detected flux from E0102 is due to ACIS contamination, and an improved CONTAM model shows this.

    Fitting the Coma Cluster

    The set of three plots shows the fit to Coma Cluster data observed on ACIS-S3 in 1999 and 2009.

    Fitting LETG/ACIS-S Spectrum of Mkn421
    • CONTAM file vN0004: the fitting model used is an absorbed power-law. Note the low-energy residuals, as the model consistently underestimates the data between the C-K edge and the O-K edge.
    • CONTAM file vN0005: fitting with a GARF built using N0005 significantly corrects the ACIS-S QE contamination.

    ACIS-I

    The ACIS-I model change will affect analyses done below 1 keV. ACIS-I does not experience the accelerated build-up of contamination in 2005 as with ACIS-S. However, the new file has adjusted C-O and C-F ratios for all mission times, and hence has a minor affect for all ACIS-I observations relative to the N0004 file.

    The best-applicable model to ACIS-I uses separate analytic functions to model the time-dependence of the elemental and two-level components. The elemental ratios in this new model have been modified from the N0004 file, and are based on the average ratios measured over the course of the mission from gratings data.

    • ACIS-I3 aimpoint effective area: the dashed curves are with the old N0004 CONTAM file and the solid ones are derived using the new N0005 CONTAM file. The black curve is the EA without any CONTAM effect.

      The variations between the two CONTAM files are rather subtle, giving at most several percent differences between any two corresponding curves.

HRC-S Imaging and Grating Data

  • HRC-S Time-dependent Gain Corrections and PI Filter

    A new HRC-S gain map and a pulse-height filter for use with LETG data has been released. Over half of Level 2 background events can be removed at most wavelengths with a loss of ~1.25% of X-ray events. These corrections were previously available as the "ADDSPI and SPIFILTER" contributed software.

    The primary differences from the old gain map and PI filter are:

    • Gain corrections are time dependent; the HRC-S gain is now roughly half what it was at the beginning of the mission.

    • PHA is replaced with scaled SUMAMPS, which creates a much more smoothly-varying gain map.

    • PI is replaced with SAMP-based PI.

    • The old PI filter became less effective over time because it did not account for decreasing gain. Less background is removed at short wavelengths because the filtering threshold must be higher to avoid excluding X-ray events.

    Relevant Plots

    Users working with HRC-S data should reprocess to improve the TGAIN calibration in their data. The PI background filter is applied to HRC-S/LETG data, as explained in the "Apply background filter" section of the LETG/HRC-S Grating Spectra thread.

  • HRC-S QE/QEU Upgrades

    The HRC-S QE has been corrected so that Chandra HRC-S/LETG and ACIS-S/HETG source spectra are consistent. The ACIS QE is based upon ground and in-flight calibration and is believe to be more secure than the HRC-S QE which is more heavily dependent upon in-flight calibration using sources as standard candles.

    The following plots illustrate the improvement in the symmetry of positive and negative orders afforded by the new QEU upgrade for HRC-S. Both plots use the ratio of measured fluxes from the positive and negative orders of LETGS for several observations of HZ43.

    The new QE and QEU will automatically be chosen from the CALDB when creating exposure maps, imaging ARFs, or grating ARFs for HRC-S data.

HRC-I Imaging Data

  • HRC-I Gain Maps

    A new time-dependent gain correction is available for HRC-I data. These new gain maps are based on the scaled sums of the three nearest amplifier signals to the event location, which results a more accurate gain correction. (PHA is the sum of all amplifier signals corresponding to the event in question, and has a much larger small-scale spatial variation than the SUMAMPS metric.)

    Technical details on the new gain maps are described in the memo SUMAMPS-based Gain Maps for the HRC-I.

    Users working with HRC-I data should reprocess to improve the gain calibration in their data.

  • HRC-I Imaging RMF

    This new HRC-I RMF file, hrciD1999-07-22samprmfN0001.fits, is required for use with event data that have PI values derived from the new HRC-I gain maps, also released in CALDB 4.2.0.

    The RMF can be used to calibrate hardness ratios or quantile color-color diagrams (QCCD) to distinguish between gross differences in the spectra.

    We do not advocate using this RMF in spectral fits; the spectral response is not sufficiently constraining to achieve a good fit with reasonable errors. If unavoidable, work in channel space (set_analysis("channel") in Sherpa), not energy space.


Tools

acis_build_badpix

  • Two new parameters, usrfile and bitflag, have been added to acis_build_badpix. The parameter bitflag can be used to choose what types of bad pixels are written to the output bad-pixel file. It can also be used to determine whether or not the pixels adjacent to each of these types of bad pixels is written to the output file. If a supplemental input usrfile is used, then it is possible to provide an explicit list of the bad pixels that should be added to or removed from the output bad-pixel file. These enhancements make it much easier to customize observation-specific bad-pixel files.

acis_process_events

  • The wording of a CC-mode/TIMEDEL warning has been updated:

    WARNING: The values of TIME may be inaccurate because calc_cc_times = yes and TIMEDEL != 0.00285.
    

acis_run_hotpix

  • The acis_run_hotpix script has been updated to support the changes to acis_build_badpix. The changes are transparent to the user and do not affect how the tool is run.

calquiz

  • The telescope and instrument parameters have been added to the tool. They may be used together in place of specifying the infile parameter.

calvalid

  • New Tool!

    calvalid validates the CALDB index file (CIF). The validation process is conducted through three steps:

    • Scan the FITS binary table of the index to determine whether it has a full array of mandatory columns of CALDB index standards and whether the data-types of these columns are up to the par of the standards.
    • Inspect the index if the attributes of the additional columns match those defined in key config file.
    • Check for duplicate rows in the CIF, inaccessible calibration files, or invalid entries in the rows.

create_bkg_map

  • New Tool!

    create_bkg_map is a script which creates a background map for ACIS and HRC from the event file data. These maps can be used for source detection and for detection sensitivity maps.

    This tool replicates the background algorithm that is used in the Chandra Source Catalog.

dmimgpm

  • New Tool!

    dmimgpm computes a low spatial frequency map. A modified Poisson mean is used to create the background map. The tool uses a measure of the distribution of counts for a given sampling region for each point in the map.

    The typical user will run dmimgpm as part of the create_bkg_map script, not as a standalone tool.

    This tool was initially developed for use with the Chandra Source Catalog.

dmjoin

  • Bug fix: If the joinfile had two consecutive rows that were equal, dmjoin would produce a NaN when it linearly interpolated instead of using the exact value.

dmpaste

  • Bug fix: dmpaste would unnecessarily close then reopen the output block in "update" mode, preventing the tool from being able to be piped.

dmcoords

  • For non-Chandra data, the tool only converts from X,Y to RA,DEC and from RA,DEC to X,Y.

    unix% dmcoords IR.fits
    dmcoords>: sky 92 202
    (RA,Dec):     18:38:42.998    -01:08:15.00
    (RA,Dec):      279.67916       -1.13750 deg
    (Logical):          92.00        202.00
    SKY(X,Y):           92.00        202.00
    

    In CIAO 4.1, a number of Chandra-specific conversions were also reported:

    unix% dmcoords IR.fits
    dmcoords>: sky 92 202
    (RA,Dec):     18:38:42.998    -01:08:15.00
    (RA,Dec):      279.67916       -1.13750 deg
    THETA,PHI         47.032'        225.56 deg
    (Logical):          92.00        202.00
    SKY(X,Y):           92.00        202.00
    DETX,DETY           80.32          1.14
    CHIP ACIS-I0     -3178.00      -2956.24 (off chip)
    TDET                 0.00          0.00
    GDPX,GDPY        16478.34       2543.53
    GAC R,D            0.00          0.00
    ENERGY 1.000000
    ZO(RA,Dec):     00:00:00.000    +00:00:00.00
    ZO(RA,Dec):        0.00000        0.00000 deg
    ZO SKY(X,Y):            0.00          0.00
    ZO DETX,DETY            0.00          0.00
    
  • Bug fix: interactive mode wouldn't accept a value of 0 for sim/displace.

eff2evt

  • Two new parameters have been added to eff2evt to allow the ACIS dead area correction to be taken into account: pbkfile and dafile. By default, the correction is "on": the parameter block file should be set in the pbkfile parameter. The default value of the dafile parameter is "CALDB". Users who wish to continue analysis without the dead area correction should set both of these parameters to "NONE":

    unix% pset eff2evt pbkfile=NONE dafile=NONE
    

hrc_dtfstats

  • Bug fix: a porting bug could drop (or include) an extra 2s record.

hrc_process_events

lim_sense

  • Bug fix: The units of the map are now recorded in the output file. ([photons/cm^2/sec] if an exposure map is input and [counts] if the exposure map is not provided).

reproject_image_grid

  • Bug fix: negative declinations were not converted correctly. If the ycenter parameter was specified in sexigesimal format and the declination was negative, the tool did not convert the value correctly.

specextract

  • Improvements were made to how the tool determines which RMF tool to use (mkrmf or mkacisrmf).

  • specextract now uses the WMAP information to lookup the calibration filenames for input to mkarf and mkacisrmf.

srcextent

  • The parameters srcsize and imgsize should be provided in units of arcseconds instead of pixels. This change is to be consistent with the output, which is reported in arcseconds.

tg_create_mask

  • The tool has been modified to allow users to have more control over the custom mask settings. The radii and mask widths are independent when use_user_pars=yes.

    • if any of radius and width parameters are not set and input_psf_tab is specified (CALDB or file), compute the value from the other parameters and the input_psf_tab as usual.
    • if any of raadius and width parameters are not set and input_psf_tab is unspecified ("none" or cannot be found), output a value of 0.
    • if any of raadius and width parameters are set, output that value. The value must be >=0; if <0, an error occurs.
    • if none of these are set and input_psf_tab is unspecified, issue a warning and output a value of 0 for both radius and width.

Parameter Files

A summary of parameter files changes is provided in this section. Refer to the Tools section of these release notes for complete details.

acis_build_badpix

  • Two new parameters, usrfile and bitflag, have been added to acis_build_badpix.

calquiz

eff2evt

  • Two new parameters have been added to eff2evt to allow the ACIS dead area correction to be taken into account: pbkfile and dafile.


Data Model

Bug Fixes

  • If "[opt null=nan]" was used on an integer image, the BLANK keyword would be set to 0, since nan isn't a valid integer. A warning message is now issued.

  • The Data Model would SegV when writing a header comment longer than 80 characters.

  • Boolean array data is properly displayed. Previously, the first value was displayed for all array elements.

ASCII Kernel

  • The CIAO 4.2 release includes limited support for the output format of the Chandra Source Catalog (CSC RDB) produced by the CSCview application. Tables saved from the application can now be processed by the CIAO command-line tools such dmlist, dmcopy, and dmstat, plotted by ChIPS, fitted within Sherpa, or manipulated via the Crates module.

    The CSC and CIAO websites will be updated to include documentation on how to take advantage of this functionality, as well as to point out the current limitations and ways to work around, or avoid them.

  • The Data Model now properly applies scaling to column data in ASCII DTF files.

  • Bug fix: corrected a problem opening ASCII files that did not end in a newline.

Crates Library

  • A number of Crates functions were renamed to avoid conflicts with the ChIPS 4.2 image functionality:

    • get_image() → get_piximg()
    • add_image() → add_piximg())
    • delete_image() → delete_piximg())
    • get_imagevals() → get_piximgvals())
    • set_imagevals() → set_piximgvals())
    • get_image_shape() → get_piximg_shape())

    There is also a new copy_piximgvals (S-Lang or Python help) function which retrieves an array of the values of the specified image within the crate.

  • Updates were made to TABLECrate::get_colnames() to avoid problems extracting multi-word column names from tab-separated files.

  • The version number and string have been added to the pycrates package.

  • The __str__ and __repr__ functions have been defined for CrateKey and CrateData objects to provide a more useful display of object contents.

  • Bug fix: If the EQUINOX key was missing from the file, a segV occured. Now a default value of 2000 is used when the keyword is missing.

    Bug fix: the axis number was being appended to the EQUINOX keyword as a subscript, e.g. "EQUINOX_1".


ChIPS

Startup Options

  • The ChIPS and Sherpa applications use the $HOME/.ipython-ciao/ directory to store configuration information. The system used to update this information has been changed in CIAO 4.2, which means that the first time ChIPS or Sherpa is started you will see a message asking whether you want to update your ipythonrc-chips or ipythonrc-sherpa file.

    Unless you have manually changed your $HOME/.ipython-ciao/ipythonrc-chips (or -sherpa) file, it is safe to answer "Y" here.

    The outdated file is renamed with a timestamp to preserve it.

  • There is a new command-line option which may be specified when ChIPS is started, "-rcfile". This option allows the user to select a specific .chips.rc file to use; refer to "ahelp chipsrc" for details.

Imaging Support

  • ChIPS 4.2 contains limited support for displaying images.

    Images may be added to plots in both pseudo- and true color. The image has an associated colorbar which displays the mapping from pixel value to color intensity. Image support for pseudocolor includes applying and adjusting colormaps and apply transparencies. There is also support for composite true-color images created from three different layers. Images support WCS transformations and allow for overlays of plots and contours.

    For information on working with images, refer to the ChIPS website and the help files, such as:

  • The new command print_image (S-Lang or Python help) prints the specified image to the provided filename. Annotations and other plot objects are not included in the output, but image properties such as alpha channel and thresholding are preserved.

Saving the ChIPS Session

  • make_script (S-Lang or Python help) generates an ASCII Python script that will recreate the state of the current ChIPS session. The session preferences are stored in a related file.

    make_script does not record every ChIPS command that has been issued. Instead it evaluates the current state of ChIPS and builds a set of commands that will replicate it. For instance, if the characteristics of a curve are changed several times, only the final color, symbol, error style etc. are recorded by make_script.

  • The script function logs every command typed in a ChIPS or Sherpa session to an ASCII file.

Setting Multiple Attributes and Toggling Error Bars

  • Chips now supports wild cards (*.) for many of the attributes of ChIPS objects. A property can be set once and applied to all sub-components of an object. For example:

    set_curve("*.color=blue") 
    

    will set all 'color' type properties (i.e. line.color, symbol.color, and err.color) to blue. Other possible attribute wildcards include "*.thickness" and "*.font".

    Additionally, error bars on curves and histograms can be toggled on or off the same way:

    set_curve ("err.*=on")
    set_curve ("err.*=off")
    

Aspect Ratio Information

  • New aspect ratio functionality has been added to ChIPS. Both data and plot areas have the ability to maintain aspect ratios. Data aspect ratio settings allow the user to maintain a consistent unit relationship between axes (i.e. pixel size) as the limits or plot size changes. The plot aspect ratio allows a plot to maintain a consistent shape (i.e. 16:9, 4:3, 1:1, etc...) as the the window or frame changes size.

    Refer to the help files:

Improvements to Printing

  • If the file name used in the print_window call ends in ".ps", ".eps", ".pdf", ".png", ".jpg", or ".jpeg" then the appropriate format will be used. So where in CIAO 4.1 you had to say

    print_window("plot", ["format", "png"]) 
    

    you can now say

    print_window("plot.png")
    

    to create the PNG format file "plot.png".

  • The value "jpeg" can now be used - along with "jpg" - to create a JPEG-format file, e.g.

    print_window("plot.jpeg")
    print_window("plot", ["format", "jpeg"])
    
  • Prior to CIAO 4.2, filled histograms and regions (those with fill.style set to "solid") would show diagonal lines crossing the filled areas in PNG and JPEG output (created by the print_window command). These lines may still be visible when viewing the postscript and PDF versions on screen but should not appear when these formats are printed out.

  • In previous versions of ChIPS, the fill.opacity setting would be ignored by print_window when fill.style was solid and the output format was PNG or JPEG. The opacity is now honored for these bitmap formats. It is still unsupported for the postscript and PDF formats, where the opacity is always set to 1.

  • In CIAO 4.1, plots containing multiple filled regions or histograms would not be printed properly if the fill style was not "solid".

  • It is now possible to create PNG or JPEG format files if the export.fonts setting is false.

  • Prior to CIAO 4.2, the command

    print_window(["fittopage", True])
    

    would have created a file called "fittopage=True" rather than applying these options and sending the output to the default printer.

  • The print_window dialog starts the browse-file location at the working directory.

Additional Enhancements & Bug Fixes

  • The axis.tickformat attribute can now be set to "%0.0z" which provides more readable labels when the axis is displayed on a log scale. See "ahelp set_axis (S-Lang or Python help)" for more information and examples.

  • The size of sub- and super-scripts, relative to the main text, has been increased.

  • Curves and histograms have a new attribute - err.caplength - which controls the length of the cap on error bars. These "caps" are only drawn when the err.style setting is "cap".

  • A histogram with a single bin that starts with a negative value is drawn correctly in ChIPS.

  • Contours of images taken near the celestial poles, which have WCS transforms applied to them, would appear distorted as the axes were gridded uniformly in ChIPS.

  • The window title was not being recorded by the save_state command.

    The prefix setting has been added in CIAO 4.2 which lets you adjust the prefix used to create the window title, which may be displayed by your window manager.

    The geom setting has been added in CIAO 4.2 which lets you tell the window manager where you would like a new window to appear.

  • The linear_scale command may also be called as lin_scale. The log_scale command may also be called as logarithmic_scale.

  • The unbind_axes (S-Lang or Python help) routine will no longer crash the ChIPS server if the input arguments do not refer to a valid axis.

  • The pick_limits (S-Lang or Python help) command would fail when the frame currency was set to "all".

  • pick, get_pick and pick_limits now return an error if used with a window whose display attribute is set to False/0. In earlier versions of ChIPS, it was possible to hang the program with such a call.

  • Occasionally, the first command in ChIPS would not work due to a problem connecting to the ChipServer.


Sherpa

Startup Options

  • The ChIPS and Sherpa applications use the $HOME/.ipython-ciao/ directory to store configuration information. The system used to update this information has been changed in CIAO 4.2, which means that the first time ChIPS or Sherpa is started you will see a message asking whether you want to update your ipythonrc-chips or ipythonrc-sherpa file.

    Unless you have manually changed your $HOME/.ipython-ciao/ipythonrc-chips (or -sherpa) file, it is safe to answer "Y" here.

    The outdated file is renamed with a timestamp to preserve it.

    The new profile is installed read-only. If the user wants to modify how Sherpa uses IPython functions, or call "execfile" to load Python scripts when Sherpa starts, the customization file:

    ~/.ipython-ciao/ipythonrc-sherpa-user 
    

    should be used. If the user wants IPython profiles to be somewhere other than ~/.ipython-ciao, the user can set IPYTHONDIR to point to the desired directory.

  • There are new command-line options which may be specified when Sherpa is started:

    • -rcfile : specify a specific .sherpa.rc file to use; refer to "ahelp sherparc" for details.
    • -norcfile : do not load any .sherpa.rc file; overrides "-rcfile" if both are set.
  • When Sherpa is started, the current directory is no longer searched first for modules. This fixes the Sherpa 4.1 problem where a script with the same name as a module, e.g. "./sherpa.py", would cause the application to fail to load.

    Note that this bug fix does not resolve all conflicts. For instance, a script named "group.py" in the working directory will cause a problem if a Sherpa command that calls the group module (e.g. group_counts) is used.

Parallelization

  • Several computationally intensive functions have been parallelized, to make use of multi-core systems (i.e., laptops or desktops with 2 or 4 cores). Users with multi-core machines should see significant improvements in efficiency compared to previous releases of Sherpa.

    • conf()
    • proj()
    • int_proj()
    • reg_proj()
    • plot_photon_flux()
    • plot_energy_flux()

Loading and Viewing Data

Filtering Data

  • Improved support for filtering grating spectral data, when using RMF or RSP files to define the response matrix.

  • The filter functions notice2d and ignore2d now support FITS region files in addition to ASCII files.

PSF Instrument Models

Models and Parameters

  • The new paramprompt() command controls whether Sherpa prompts the user for initial parameter values when a model component is created. If parameter prompting is turned on, the user is queried for the initial value of each model component parameter when it is established.

  • The "limits" and "values" options have been added to the guess() command. They may be used to restrict which model component values guess affects.

    Reset restores user-defined initial values and limits even after the guess command has been used or a fit was done.

  • There is a new interface for adding user-defined statistic functions: load_user_stat. The function accommodates user-defined functions for a statistic and statistical errors, in addition to defining a list of model parameters and hyperparameters for prior distributions (if prior desired).

  • Sherpa includes the "additive" and "multiplicative" models of XSpec version 12.5.1. (Note that the ahelp files packaged with CIAO incorrectly state the version as "12.5.0aj.")

    New XSpec models included in Sherpa 4.2 are:

  • The get_bkg_unconvolved function has been deprecated; use get_bkg_source() instead.

Optimization Methods

  • In the Levenberg-Marquardt (LMDIF) fitting function, the following vectors are no longer returned by a fit:

    • fvec: an input array, of same size as the data, which must contain the model evaluation at each point in data space.
    • fjac: an output array, which contains an approximation of the Jacobian matrix.
    • ipvt: an output array of integers, that defines a permutation matrix (which is related to the final calculated Jacobian). In the equation fjac * p = q * r, ipvt = p.
    • qtf: an output array, (q transpose) * fvec.

    Users typically don't examine these results, and as they take up much memory, they have been removed from the LMDIF output to make Sherpa's use of memory more efficient.

Fitting

  • A new function, fit_bkg(), fits background data alone when fitting with PHA spectral data. The function's list of arguments is the same as the fit() function. (This function provides functionality equivalent to the Sherpa 3.4 BFIT command.)

  • Primini fit statistic is available as a script in the CIAO Contributed tarfile.

New Confidence Limit Function: conf

  • A new confidence limit function, called conf(), has been added to Sherpa.

    The conf command computes confidence interval bounds for the specified model parameters in the dataset. A given parameter's value is varied along a grid of values while the values of all the other thawed parameters are allowed to float to new best-fit values.

    This function supersedes projection. The projection function has been deprecated and may be removed in the future; users should replace it with the confidence function in their analysis.

Saving the Sherpa Session

  • save_all command saves all the settings which define the current Sherpa session to an ASCII file. The resulting file can be read into a future Sherpa session to recreate the data, models, filter, grouping and quality flags, statistic and method used at that time.

  • The script function logs every command typed in a ChIPS or Sherpa session to an ASCII file.

  • A number of new functions have been added to save specific parts of the session. These commands may be used with both source and background datasets.

Additional Enhancements & Bug Fixes

  • Wavelength analysis is now fully supported.

  • ARF rebinning now occurs when the ARF is at a finer resolution than the RMF or - for grating analysis - the PHA.

  • For spectral PHA data, set_analysis() will not let the user change the units of a data set from 'channel' to 'energy' or 'wavelength' unless the appropriate ARF and RMF instrument files have been loaded.

  • The eqwidth() function now has 'lo' and 'hi' arguments for restricting the calculation of the equivalent width to a subset of the full energy or wavelength range of a data set (similar to the Sherpa calc_*_flux functions).

  • The fake_pha() function now accepts the names of RMF and ARF file as arguments.

  • The dataspace1d command includes an additional argument 'numbins' that specifies the number of bins as an alternative to specifying the step size.


GUIs (Graphical User Interfaces)

ds9 and dax

  • ds9 v5.7 is packaged with CIAO 4.2.

  • Bug fix: when both source and background regions are defined, dax would plot the the count_rate column instead of the net_rate column in lightcurve mode (i.e. CIAO → Histograms → TIME ).

ObsVis

  • Added the ability to manipulate individual field-of-view parameters in the grid dialog.

  • The "Target Name" field now has a "Resolve" button associated with it.

  • The user can edit a saved FoV file and make the Offset-SIMZ values inconsistent between arcmin and mm. When this file is read in, the inconsistent values are adjusted and a warning message is issued.

prism

  • User-configurable data formatting for image and table data columns is available in the prism display. The user can edit the preferences file to set up default data display rules based on column name, extension name, and data type, such as:

    data.format: time,*,double,%e
    data.format: time,events,double,%5.1f
    data.format: phas,events,short,%03d
    data.format: x,*,short,
    data.format: x,*,float,%#+ -4.1f,
    data.format: PRIMARY,PRIMARY,long,%03ld
    

    The formats may modified while the user is viewing the data by selecting the Edit → Format Display Columns option from the data toolbar.

  • Bug fix: if a file is opened in prism with a column filter, that filter is cleared before a histogram is created. In CIAO 4.1, the histogram column was being appended to the original filter.

  • Bug fix: plotting no longer fails when the line thickness is set to 0.5.

peg

  • Bug fix: peg had a problem with running tools where any parameters are set to redirects, e.g. eventdef=")stdlev1" in acis_process_events.

taskmonitor

  • The analysis menu is available from taskmonitor.

  • Added the command-line options:

    • --help: prints help message
    • --version, -v: prints version info
    • --rcfile, -r: allows rcfile to be specified on commandline

Libraries

tcdlib

  • There was a small bug in the tophat kernel in the tcdlib that would cause slight differences on different platforms and optimization levels; e.g. aconvole with "lib:tophat(2,1,3,3)" would give +/-2% differences in pixels around the edge of the tophat.


Environment

ciaover

  • ciaover is a new utility which reports the version of CIAO and/or the versions of each individual CIAO package. This information is useful for reporting to HelpDesk with software questions.

    unix% ciaover
    CIAO 4.2 Monday, November 30, 2009
      bindir      : /soft/ciao-4.2/bin
    
    unix% ciaover -s
     CIAO 4.2 Monday, November 30, 2009
    
    unix% ciaover -v
    The current environment is configured for:
      CIAO        : CIAO 4.2 Monday, November 30, 2009
      Tools       : Package release 1  Monday, November 30, 2009
      Sherpa      : Package release 1  Monday, November 30, 2009
      Chips       : Package release 1  Monday, November 30, 2009
      Prism       : Package release 1  Monday, November 30, 2009
      Obsvis      : Package release 1  Monday, November 30, 2009
      Core        : Package release 1  Monday, November 30, 2009
      Graphics    : Package release 1  Monday, November 30, 2009
      bindir      : /soft/ciao-4.2/bin
      Python path : CIAO
    
    System information:
    Linux buddy 2.6.18-164.2.1.el5.plus #1 SMP Fri Oct 9 12:27:45 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
    

Automatic Update of Configuration Files for Python Users

  • The ChIPS and Sherpa applications use the $HOME/.ipython-ciao/ directory to store configuration information. The system used to update this information has been changed in CIAO 4.2, which means that the first time ChIPS or Sherpa is started you will see a message asking whether you want to update your ipythonrc-chips or ipythonrc-sherpa file.

    Unless you have manually changed your $HOME/.ipython-ciao/ipythonrc-chips (or -sherpa) file, it is safe to answer "Y" here.

    The outdated file is renamed with a timestamp to preserve it.

IPython

  • CIAO 4.2 includes IPython in the CIAO OTS directory which is used by Sherpa and ChIPS to provide command-line user interfaces. These programs create an IPython profile in the directory $HOME/.ipython-ciao.

    If IPython users want any personal customizations to be available when running CIAO, they will have to copy them from $HOME/.ipython to $HOME/.ipython-ciao.

Setting the HEADAS Variable

  • The $HEADAS environment variable is used by Sherpa to locate the XSpec model libraries. This variable used to be defined when CIAO was started, which created a conflict with users running XSpec in the same window.

    In CIAO 4.2, the variable is not redefined until Sherpa is launched, instead of when CIAO is started.

Off-the-Shelf (OTS) Package Versions

  • The following OTS packages are included with CIAO 4.2. For more information on how the OTS packages are built for use with CIAO, refer to the INSTALL_SOURCE file distributed with the software.

    • cfitsio v3.2
    • CCfits v2.1
    • DS9 v5.7
    • fftw v3.2.2
    • ghostscript v8.56
    • gsl v1.12
    • gtkglext v1.0.6
    • GTK v2.16
    • iPython v0.10
    • jpeg v7
    • numpy v1.3.0
    • pysl v0.9.0
    • Python v2.6.2
    • readline v6.0
    • Slang v2.1.4
    • VTK v5.4.2
    • wcssubs v3.7.0
    • xerces-c v3.0.1
    • xpa v2.1.9
    • Xspec v12.5.1 (models only)

    * GTK consists of several sub-packages: pkg-config v0.18.0, gettext v0.17, glib v2.16.6, ATK v1.20.0, pixman v0.12.0, FreeType v2.3.7, expat v2.0.1, libxml v2-2.6.31, tiff v3.8.2, png v1.2.32, fontconfig v2.3.92, cairo v1.6.4, Pango v1.19.4, GTK+ v2.12.12


Analysis Scripts

The CIAO contributed scripts package is available from the Scripts page and is considered a required part of the installation. They can be installed with the ciao-install script when CIAO 4.2 is installed.

Infrastructure Changes

  • The name of the contributed tarfile has been changed to include the release date, e.g. ciao-4.2-contrib-14Dec2009.tar.gz.

  • The tarfile now unpacks into the CIAO root directory, the same as the software files. For instance, if CIAO is installed in /soft/ciao-4.2/, the contributed tarfile should be unpacked in /soft. (Prior to CIAO 4.2, the tarfile had to be unpacked in /soft/ciao-4.2/.)

SherpaCL

  • The SherpaCL application has not yet been updated to work in CIAO 4.2. It is hoped that an updated version will be available in January.

Flux Commands

  • The contributed flux routines (the sherpa_contrib.flux_dist module) are now included as part of the Sherpa application, so there is no need to load the module to use them. These commands are:

    • get_energy_flux_dist()
    • get_photon_flux_dist()
    • plot_energy_flux()
    • plot_photon_flux()
    • sample_energy_flux()
    • sample_photon_flux()

fit_primini

  • This is a Primini iterative-fitting scheme for use in Sherpa, fit_primini. In CIAO 3.4 this method was provided by the "CHI PRIMINI" statistic; for CIAO 4.2 it is provided by the fit_primini() routine, which is a replacement for the fit() command. Note that fit_primini() should only be used with a chi-square statistic (e.g. not "cash", "cstat", or "leastsq").

lc_clean and analyze_ltcrv

  • The scripts lc_clean and analyze_ltcrv were updated and combined into a single script for CIAO 4.1. The new script is available for both Python and S-Lang: lightcurves.sl and lightcurves.py.

    The deprecated versions of the scripts lc_clean and analyze_ltcrv have been removed from the CIAO 4.2 contributed tarfile.

tg_bkg v1.2

  • The tg_bkg script has been updated to v1.2. This update is to correct a minor Data Model syntax problem; the output files from the script are unchanged.


Documentation

Analysis Threads

  • The CIAO, Sherpa, and ChIPS Threads are in the process of being tested and updated for the CIAO 4.2 release. Please check the "Last Update" information at the top of each thread to see if it has been reviewed yet.



Last modified: 27 September 2010