CIAO 3.1 Release Notes

Return to: Version History

• Tools
• Parameter Files
• Data Model
• Sherpa
• User Infrastructure
• S-Lang Modules
• Libraries
• Documentation

Tools

acis_process_events

• New functionality!

  acis_process_events has been modified to include a time-dependent gain adjustment, which is now applied by default. This adjustment is necessary because the "effective gains" of the detectors are drifting with time as the result of an increasing CTI. (The time-dependent gain for ACIS-I2 also includes an adjustment for the corresponding read-out electronics.) After applying the adjustment, the effective gain is consistent with the gain as calibrated for the beginning of the -120 C period (February 2000 through April 2000); this is also the period for which CTI is calibrated.

  Two new parameters have been added so that this adjustment may be applied: apply_tgain and tgainfile. The default values for these parameters are "yes" and "CALDB", respectively. Additionally, the eventdefs now include "l:pha_ro"; the PHA_RO column contains the readout (unadjusted) PHA values.

  How this affects your data analysis:

  • If the observation took place any time after April 2000, applying the time-dependent gain correction may improve the quality of the data.
    
    
  • You do not need to re-correct with acis_process_events if you have already calibrated the data with corr_tgain.

  The Apply the Time-Dependent ACIS Gain Correction thread explains how to apply this correction.

• Added RAND_SKY keyword to indicate amount of randomization applied to chip coordinates when going to sky.

• Added RAND_PI keyword to indicate amount randomization to PHA value when computing Energy/PI.

• Enumerated allowed values for rand_pix_size and stop in the parameter file.

csmooth

• Bug fix: user-supplied backgrounds didn't work reliably.

dmcoords

• Bug fix: returns correct chip coordinates for the case where the position falls off the detector (actually a change to pixlib).

dmextract

• The default error determination now uses Poisson statistics instead of Gaussian statistics; this change affected the error and bkgerror parameter defaults.

• Bug fix: the MEAN_BG_EXP column would contain a non-zero value for the first row only when there are multiple source regions but only one background region.

• Enumerated allowed values for opt in the parameter file.

dmfilth

• Bug fix: the description of the infile parameter no longer mentions event lists as valid input.

dmgroup

• Bug fix: if the GROUPING and QUALITY keywords exist, they will be removed when columns of the same name are created (for XSPEC compatibility).

dmpaste

• Bug fix: tool would segmentation fault if given bad or missing input files.

dmtcalc

• Added minimum and maximum values to verbose in the parameter file.

hrc_process_events

• The default rand_pix_size is changed to zero, meaning that randomization is not applied to the coordinate computations by default. Enumerated allowed values in the parameter file.

  All pixel randomization for HRC data (both imaging and grating data) is turned off by default now, as the randomization has little or no effect on the quality of HRC data.

  How this affects your data analysis:

  • It is not necessary to reprocess your HRC data with randomization turned off.

• Bug fix: changed equation for chip gaps (HRC-LETG rotation).

• Added RAND_SKY keyword to indicate amount of randomization applied to chip coordinates when going to sky.

• Enumerated allowed values for start and stop in the parameter file.

tg_create_mask

• Allow ACIS/HETG and LETG CC mode to use width_ parameters to specify the user mask width.

  Manual control of grating mask parameters in CC mode now allows ad hoc adjustment of the parameters, if necessary. Improvements to CC-mode coordinates, however, also make such situations less likely to arise. More information is available from the Continuous Clocking Mode why topic.

• Bug fix: sign of LEG angle.

tg_resolve_events

• The default rand_pix_size is changed to zero, meaning that randomization is not applied to the coordinate computations by default. Enumerated allowed values in the parameter file.

  Use of unrandomized events in grating analysis yields about 10-20% smaller line FWHM. Downstream analysis of unrandomized coordinates requires use of grating RMFs made with unrandomized LSF parameters. These files are available in the CALDB version 2.27; see the CALDB release notes for details.

  How this affects your data analysis:

  • ACIS-S/HETG data: for users whose science requires the improved spectral resolution for resolving line blends or detection of weak features, we recommend reprocessing without randomization. (Fluxes of relatively isolated lines are not affected.) The Obtain Grating Spectra from HETG/ACIS-S Data thread explains how to do so.
    
    
  • ACIS-S/LETG data: for now, users should keep the randomization, i.e. not reprocess their data. The calibration required to create unrandomized gRMF files is not yet available for this detector combination. The Create Grating RMFs for ACIS-S Observations thread has more details.

• Bug fix: CC mode HETG events all coming out MEG when zero-order off chip.

• Bug fix: CC mode LETG tg_d offset from 0.

• Bug fix: small tg_r/tg_d rotation in CC-mode data (both HETG & LETG) caused by suboptimal chip_y guess.

• Bug fix: sign of LEG angle.

• Added RAND_TG keyword to indicate amount of randomization applied to chip coordinates when going to grating coordinates.

tgextract

• Upgrades to use new header library.

Parameter Files

acis_process_events

• Two new parameters have been added so that the time-dependent gain adjustment may be applied: apply_tgain and tgainfile. The default values for these parameters are "yes" and "CALDB", respectively. Additionally, the eventdefs now include "l:pha_ro"; this allows for column in the output file in which the readout (unadjusted) PHA values are stored.

• Enumerated allowed values for rand_pix_size and stop in the parameter file.

dmextract

• The default error determination now uses Poisson statistics instead of Gaussian statistics; this change affected the error and bkgerror parameter defaults.

• Enumerated allowed values for opt in the parameter file.

dmfilth

• Bug fix: the description of the infile parameter no longer mentions event lists as valid input.

dmtcalc

• Added minimum and maximum values to verbose in the parameter file.

hrc_process_events

• The default rand_pix_size is changed to zero, meaning that randomization is not applied to the coordinate computations by default. Enumerated allowed values in the parameter file.

  All pixel randomization for HRC data (both imaging and grating data) is turned off by default now, as the randomization has little or no effect on the quality of HRC data.

• Enumerated allowed values for start and stop in the parameter file.

tg_resolve_events

• The default rand_pix_size is changed to zero, meaning that randomization is not applied to the coordinate computations by default. Enumerated allowed values in the parameter file.

  Use of unrandomized events in grating analysis yields about 10-20% small line FWHM. Downstream analysis of unrandomized coordinates requires use of grating RMFs made with unrandomized LSF parameters. These files are available in the CALDB version 2.27; see the CALDB release notes for details.

Data Model

Enhancements

• Several changes to DM internals to improve the overall speed performance of the library. The benefit is rather generic, but may be most noticeable in reading table rows.

• Updated region intersection/union handling to recognize identical shapes and consolidate the region.

• Significant improvement to image rebinning functionality.

• Improved "exclude" filter handling.

• Added capability to perform col=NULL filtering.

Bug Fixes

• The Data Model bug page lists all currently known bugs, as well as those fixed in CIAO 3.1.

• When filtering an image, set values outside of region filter bounding box to Null.

• Significant changes were made for image rebinning.

  • Handling logical/physical coordinates with image rebinning.
  • Handle case when creating an image whose dimensions don't exactly divide into the column ranges.
  • Fixed up image rebinning/filtering with integer typed images.
  • Not all subspace components were being set correctly. Image cubes weren't getting the correct coordinate transforms.
  • Use the current bin size as the default when binning with no binsize specified, e.g. "[bin sky]".

• Changes to region intersections and unions so that multiply intersecting/unioning a region with itself will not result in an increasingly large region, but rather just the original region. In intersection it is also possible to check for identical shapes to avoid repetition of identical shapes.

• > and < comparisons against 0 and 0.0 for all floating point type columns.

• Correctly apply an exclude filter on a single component of a vector column.

• Filtering a vector column on ranges when a region subspace is already in place.

• Report a syntax error when attempting to use > or < syntax on an image.

• Handle region strings longer than 1024 characters.

• Proper handling of open ended exclude filter ranges (e.g. counts<2, counts<=2, counts=:2 ). This applies to all real and integer data types. In addition to this, for Integer types, > and >= filters produced the same results.

Sherpa

Instrument Model Changes

• Instrument models are managed slightly differently in CIAO 3.1. After a data set is read in, the user can assign an instrument model to go with that data set, just as in the past. When a new data set is read in, however, the instrument model that went with that old data set is now "unassigned". If the user wishes to use the old instrument model, it must be reassigned to the new data set. This has been done to ensure that users do not run into errors trying to apply unsuitable instrument models to the data sets.

  If the background file is read, but "instrument back" has not been defined, the user can do the following without problems:

  • subtract background
  • filter source data on PHA channel with "ignore filter <range>"
  • filter source data on energy with "ignore source energy <range>" or "notice source energy <range>"
  • fit, either with or without subtracting the background

  However, the user cannot:

  • filter source data on energy with "ignore energy <range>" or "notice energy <range>" (Note that the "source" qualifier has been omitted.)
  • fit background and source data simultaneously
  • plot background in energy or wavelength space

• When an ARF is read in by itself, a dummy RMF is created to help correctly translate from energy space to channel space. However, when the data are grouped, the assignment of the ARF and dummy RMF was (incorrectly) disallowed. This bug has been fixed. A similar bug in the S-Lang load_inst() function (which allows the user to load data from S-Lang arrays or files into instrument models) has also been fixed.

• The user can create one instance of the farf1d model and later read data from a different ARF file into that model. It is not necessary to create a new instance of farf1d to contain the data from the new ARF.

• Bug fix: the configuration variable sherpa.modeloverride is supposed to allow the user to redefine models when the variable is true. This bug did not allow that with certain models (e.g. farf1d).

• ARF files from the Astro-E mission can be read in to Sherpa.

• Bug fix: if kernel.origin is declared before the kernel is defined, Sherpa no longer exits with a segmentation fault.

Evaluation of models

• Prior to CIAO 3.1, when a model had its integrate setting turned off and the data was binned, the model would be evaluated at the left-edge of the bin. As of CIAO 3.1 the model will now be evaluated at the center of the bin, which matches the behavior of the XSPEC "multiplicative" models, such as xsphabs.

  Fitting imaging data

  This update will increase the best-fit locations - e.g. the xpos and ypos parameters of the beta2d model - of models by half a pixel if the model was evaluated with the integrate setting turned off.

  In general the increase in the best-fit position will be +0.5, except for fits made using the PHYSICAL coordinate system and for which the image pixel corresponds to more than one physical pixel. In these cases, the increase will be half the size of the image pixel in physical units; as an example, for an image created by binning an event file using the DM filter "[bin x=::4,y=::4]", the increase in the best-fit position will be 2.

  Fitting spectral data

  This change will not affect most fits since:

  • no change is seen with XSPEC models;
  • most spectral fits are done with the integrate setting set to on for each model component.

  You may see small changes to the best-fit parameter values - in particular those parameters that define a location - if you use other models, such as powlaw1d, in your fit and turn off the integrate setting.

  Fitting radial profiles

  The best-fit values will change slightly if the profile is fit as a binned dataset - i.e. if the minimum and maximum values of each bin are set. If the data is loaded as an unbinned dataset then no changes will be seen.

Grouping and Quality Flags

• The S-Lang function set_groups() can now be used to apply a new grouping scheme to data that had been read in without any groups. A new thread - Re-grouping PHA data in Sherpa - has been created to show how this function can be used to re-group your PHA data within Sherpa on the fly.

• Quality flags can be assigned to data sets using the S-Lang set_quality() function.

• Bug fix: the correct quality flags are now displayed in plots of the quality flags, after the bad quality flags have been ignored.

• Bug fix: Sherpa respects the quality flags for the background dataset.

Other Changes

• XSPEC models may be included in nested models, e.g. the following is a valid expression:

  sherpa> mymodel = const[cs]
  sherpa> source = xsgauss[xg1]{mymodel}
  

• When 2-D models are used and the integration setting is off, the amplitude of the model is calculated from the center of each pixel, instead of from the lower-left corner.

Bug Fixes

• The Sherpa bug pages list all currently known bugs, as well as those fixed in CIAO 3.1.

• General fitting bug: the fit will not fail if a spectrum and the related ARF have a different number of bins, e.g. the output of the add_grating_orders script.

• fakeit: simulated data is written out to the PHA file with the correct grouping.

• import:

  • The ISIS and XSPEC S-Lang modules can be imported into Sherpa without error on OS X.

  • If the Sherpa module is imported, the OS X version of Sherpa will not exit with an error and much extraneous screen output.

• When the instrument was modeled with an ARF only and the flux was calculated at a single energy, a crash resulted. This has been fixed in the get_eflux and get_mcounts commands.

• get_par: will return NULL instead of failing to run when no models are defined.

• load_pha: won't segmentation fault if the background file specified by the BACKFILE header keyword is missing.

• lplot: Sherpa will no longer segmentation fault if you try to plot the quality array of a dataset that doesn't have one.

• read: certain MDL files could cause crashes when read into the Linux version of Sherpa.

• set_ignore() now returns correct values for success or failure.

• xspec abundan: corrected a parse error. The documentation for this command was also updated to mention the "grsa" and "wilm" options.

User Infrastructure

ChIPS

• Bug fix: repeated stores now overwrite, rather than add to, the FITS file.

• Bug fix: plotting of scaled data when an axis is in log scale.

VARMM

• The VARMM functions min, max, and sum were removed; the S-Lang 1.4.9 versions of these functions are now available.

• Bug fix: allows use of colons in directory path.

• Bug fix: properly display non-square arrays images.

• Bug fix: readfile does not cache and re-use cached images.

• Bug fix: print function was failing on OS-X due to differing path specifications.

S-Lang Modules

Module Conflicts

• Bug fix: eliminated multiple symbol definition error on OS X.

Libraries

ardlib

• Upgraded to incorporate the change on data randomization scheme for grating RMF (gRMF). Now the ardlib handles RAND_TG keyword as a part of CALDB query expression, which allows it to distinguish between randomized and unrandomized LSFPARM files.

errlib

• Updated version to reflect CIAO 3.1.

hdrlib

• Changed default TIERABSO keyword from 0.001 to 0.00005 for when new headers are created.

• Updated version to reflect CIAO 3.1.

• Added RAND_SKY, RAND_PI, and RAND_TG to the basic and event headers so they will get propagated to derived products (e.g. spectra, image, etc).

• Updated the new hdrlib to initialize the non-existing keyword values in the same way as the old hdrlib before performing any merging.

• Relaxed the "missing from first file" concept in the WarmOmit merging rules since in general the order of the files is arbitrary. Keywords missing from any of the files will be considered in the overall merge.

• Unsigned long and short data-types are now propagated correctly.

• Added DS_IDENT keyword and appropriate merging rules.

• Fixed problem when writing DATE-OBS, DATE-END, MJD_OBS if certain keywords (TSTART, TSTOP, MJD_REF) were missing.

pixlib

• HRC-S3 no longer returns invalid CHIP_ID in the fpc_to_extended_chip routine.

• The "extended" chip routine returns correct chip coordinates (ccd_id was correct, coordinates were wrong). This is needed for the tg_resolve_events CC mode fixes and fixes a bug in dmcoords.

region lib

• Bug fix: was not recognizing semicolon (;) as a possible line separator when parsing ds9 region files.

tcdlib

• User-supplied backgrounds didn't work reliably. Fixes a bug in csmooth.

Documentation

S-Lang in CIAO

• There is a new section of the CIAO website dedicated to information on using S-Lang in CIAO. These webpages are a collection of the available documentation on the S-Lang scripting language and are intended to help users to learn specifically how use S-Lang within the CIAO software package.

CIAO Threads

• New Threads

  • Apply the Time-Dependent ACIS Gain Correction

• Updated Threads

  • Obtain Grating Spectra from HETG/ACIS-S Data
  • Obtain Grating Spectra from LETG/ACIS-S Data
  • Create Grating RMFs for ACIS-S Observations
  • Create Grating RMFs for HRC-S Observations

Sherpa Threads

• New Threads

  • Changing the grouping scheme of a dataset within Sherpa

• Updated Threads

  • Fitting FITS Image Data

CALDB Website

• The Chandra Calibration Database website has been redesigned. The changes include a sitewide navigation bar with direct links to all sections of the site and streamlined download instructions.