CIAO Release Notes

CIAO 2.1 Release Notes

Contents:

Setup/Packaging
Caveats
Tools
Sherpa
DataModel
GUIs
Infrastructure

Setup/Packaging

    * The CIAO configuration scripts have been reworked, for greater
      simplicity, better compatibility with other packages,
      and increased flexibility. Please consult the CIAO2.1 README
      (located at the top of the distribution) and the configuration
      thread available online at the
      public CIAO page http://cxc.harvard.edu/ciao/
    * CALDB v2.3 must be downloaded with CIAO2.1 due to a new
      calibration file that is needed to run several tools.
    * ATOMDB v1.1.0 must be download in order to use GUIDE in CIAO2.1.
      This is due to a different layout/format of the atomic database
      files that GUIDE uses.

Off-The-Shelf-Software (OTS)

    * Migrated to XPA v2.0.5, for Compaq/Alpha compatibility
    * Migrated to S-Lang v1.4.2

Caveats

    * The SAODS9Engine class explicitly sets region commands sent to the
      ds9 imager into fk(5) (more specifically, J2000) format. This
      should not be an issue for Chandra data in J2000 or ICRS format
      (ds9 currently handles ICRS the same as J2000). In displaying
      images containing fk(4) coordinates or other non-J2000 formats,
      however, users may encounter 'misplaced' regions if they attempt
      to add a region marker to the image by way of sending an XPA
      command to prism.
    * Compaq/Alpha platform-specific information can be found in
      README.tru64 at the top of the CIAO installation tree. Please
      consult it for more information on this platform.
    * Please consult the CIAO2.1 buglist, maintained online at the
      public CIAO page http://cxc.harvard.edu/ciao/ for more specific
      information.

Tools

* acis_detect_afterglow *

	Bug Fixes - mods to support Alpha memory segment problems

* acis_process_events *

Bug Fixes - mods to support Alpha memory allocation problems

* asp_apply_sim *

	Bug Fixes - fixed bug in overwriting an incomplete output
	file.[5314] - fixed bug for few arcseconds shift between expmap and
	image, and between grating spectra and grating ARF's.[5348]
	(Included in CIAO2.0.2 patch)

* asp_calc_offsets *

	Bug Fixes - Fix to catch blank line at end of stack files

* axbary *

	Bug Fixes - Fixed memory initialization problem seen on the alpha

* celldetect *

	Bug Fixes - RA & DEC values in output were sometime incorrect.
	This has been fixed.

* csmooth *

	Bug Fixes - A bug causing SEGV's or bus errors in long smoothing
	operations has been fixed. [5262]

* destreak * *new*

	Summary Destreak is a filtering tool used to remove streaks
	events from ACIS chips (particularly ccd s4 or 8). Streak events
	show up in horizontal rows of pixels with a constant CHIPY, usually
	near a node boundary. When run in a default mode, destreak will
	automatically determine which events to flag (or filter) as streak
	events. This is a datamodel-enable Implementation of John Houck's
	destreak from the software swap page. Includes support to exclude
	regions from analysis and several statistical outputs.

* dmcontour *

	Bug Fixes - Fixed bug in looping over too many pixels.

* dmcoords *

	Bug Fixes - alpha portability changes

* dmcopy *

	Interface & Algorithm Changes - improved QPOE support: dmcopy
	acis.fits acis.qp kernel=iraf should now produce a QPOE file
	compatible with PROS, although PROS does not support different GTIs
	per chip.

	Bug Fixes - fixed bug that caused GTIs to be incorrect when
	applying a filter on CCD_ID that omitted the third or later GTI in
	an ACIS event file but not the first - fixed bug which caused dmcopy
	"evt.fits[ccd_id=4]" new.fits not to have LIVETIME=0 if ccd_id 4 was
	not present in the data. fixed bug which caused problems when
	filtering images. Now dmcopy "img.fits[(#1,#2)=region(logical.reg)]"
	sub1.fits and dmcopy "img.fits[(x,y)=region(physical.reg)]"
	sub2.fits both work where logical.reg is a region file in logical
	("image" in DS9 terminology) coordinates and physical.reg is a
	region file in physical coordinates. Also, filtering on a world
	coordinate region now works.

* dmextract *

	Interface & Algorithm Changes - The original functionality of
	creating histograms and pha files is maintained, but a new
	functionality has been added. The tool can also take a file and bin
	on the regions to calculate the number of counts within the
	specified regions. There are numerous new parameters related to the
	new functionality. The most notable are the "bkg", which allows for
	a background file or value to correct the counts by and the "error",
	which allows the user to determine the type of error calculation for
	the source event counting. Future releases will allow for more cross
	functionality between the current uses of dmextract.

	Bug Fixes - Fixed a rounding error that would artificially
	inflate the number of counts in the first bin.

* dmgroup *

	Interface & Algorithm Changes - tabs now pertain to data in
	column 'tabcolumn' ( new parameter ) - a blank binning specification
	when grouptype=BIN is now a fatal error

	Bug Fixes - grp_stat_err column values were not correctly
	calculated for several of the grouping options. - SEGV in some cases
	using BIN when the step size chosen exceeds the number of rows in
	the input file. - replaces long by double type for some columns .

* dmgti *

	Interface & Algorithm Changes - Uses TIMPIXR to determine time
	interval boundaries for various input files

* dmhedit *

	Interface & Algorithm Changes - added new string parameter
	'datatype' for keyword to be added or updated - added functionality
	to preserve brackets, slashes, and other characters in added keywords

* dmimg2jpg *

	Bug Fixes - minor bug fix to catch allocation problems on Alpha

* dmimgcalc *

	Bug Fixes - dmimgcalc now always propagates the WCS from the
	first input file to the output file. If the WCS in the second input
	file does not match that of the first, a warning is printed.

* dmlist *

	Interface & Algorithm Changes - recision in dmlist output of
	double precision quantities further improved - Improved dmlist
	output so that long string keywords are fully printed

	Bug Fixes - made more robust to certain kinds of non-Chandra file

* dmmakereg *

	Interface & Algorithm Changes - Added ability to append regions
	to existing region file. Useful to add badpixels

* dmmerge *

	Bug Fixes - fixed bug in closing files that affected max number
	of files that could be merged.[5331]

* dmpaste *

	Bug Fixes - fixes for memory problems seen on Alpha

* dmrega2fits * *deprecated*

	Summary As of this release this tool is now deprecated. It is
	still included in the release but will not be distributed in future
	versions of CIAO. The primary use of this tool was to create a FITS
	region file to append onto a grating event list. This is now created
	by tg_create_mask (see below) and tg_resolve_events. The generic
	ability to convert an ASCII region file to an ASC-FITS compliant
	file is now in the tool dmmakereg.

* dmregrid *

	Bug Fixes - initialization problem seen on Alpha

* dmtcalc *

	Bug Fixes - Modifications to support building on the Alpha.

* dmtype2split *

	Interface & Algorithm Changes - tool now copies all keywords in
	input to output.

	Bug Fixes - error-checking adding to check that 'rows' parameter
	contains valid values.

* dmwritefef *

	Bug Fixes - fixes to support alpha

* get_src_region *

	Interface & Algorithm Changes - Made a small fix to the tool to
	ensure that all the output regions will be sorted in the same way
	despite what platform the tool is run on.

* hrc_process_events *

	Bug Fixes - Division by 0 problem fixed for alpha.

* mkgarf *

	Interface & Algorithm Changes - now uses NPOINTS column in OSIP
	table

* mkpsf *

	Bug Fixes - fixed world coordinate system header keywords to
	accurately reflect aspect roll applied to psf image. - fixed output
	psf image scale to default to input image file scale (not psf
	library file scale).

* mkrmf *

	Bug Fixes - support grid FITS binary table input, and a bug of
	file suffix, ".fits", is removed. - remove a bug of an extra file
	unexpectly generated.

* mtl_build_gti *

	Interface & Algorithm Changes - Uses TIMPIXR to determine time
	interval boundaries for various input files

	Bug Fixes - Numerous fixes to support building on Alpha

* pquery *

	Interface & Algorithm Changes - none (Note: readline changed to
	use stderr)

* sso_freeze * *new*

	Summary reprojects sky x and y coordinates from the nominal
	coordinate frame fixed to the celestial sphere to a reference frame
	which is fixed to the moving solar system object (SSO) target. The
	result is that the new events table columns ocx and ocy display
	events as seen from an observer moving with the SSO, eliminating the
	"blurring" seen in the sky x and y events.

* tg_create_mask *

	Interface & Algorithm Changes - The tool will now create a fits
	grating mask file instead of an ascii grating mask file. - Hidden
	dependence on grating.par, acis.par, hrc-s.par, hrc-i.par and
	pix_pixel_plane.par parameter files has been removed. The parameter
	values provided by these files will now be obtained from pixlib.

	Bug Fixes - The tool can now handle HRC-SI data. [5274]

* tg_resolve_events *

	Interface & Algorithm Changes - The grating.par, acis.par,
	hrc-s.par, and hrc-i.par parameter files are no longer needed. The
	parameter values provided by these files will now be obtained from
	pixlib. - The input region file that contains the grating masks can
	now be a fits region file or an ascii region file. If it is a fits
	region file, it will be appended to the output file as the REGION
	extension.

	Bug Fixes - Fixed memory allocation problem on Alpha

* tgextract *

	Interface & Algorithm Changes - Added the DETCHANS keyword to
	the output PHA TYPE I file. DETCHANS keyword's value is equal to the
	total number of bins in the PHA TYPE I file. - The grating.par
	parameter file is no longer needed. tgextract will now obtain the
	values of the grating parameters from pixlib. - Simple command line
	filter can now be used with the input file. For example, the input
	to the infile parameter can be something like
	evt1.fits[time=563800:578800]

	Bug Fixes - The tool can now handle HRC-SI data. [5274]

* tgmatchsrc *

	Bug Fixes - Fixed a compilation error on Alpha by not using
	template in the code.

* vtpdetect *

	Interface & Algorithm Changes - Tests for input having no WCS
	information. In this case, vtpdetect now creates NaN WCS outputs.

* wavdetect *

	Bug Fixes - Now tests that the exposure map dimensions match the
	data set dimensions. This was causing a SEGV. [5277] - Fixed bug in
	handling of "interdir" parameter. [5086] - Clobber=no was not being
	respected with regard to ASCII region file

* wrecon *

	Bug Fixes - Clobber=no was not being respected with regard to
	ASCII region file

* wtransform *

	Bug Fixes - Now tests that the exposure map dimensions match the
	data set dimensions. This was causing a SEGV. [5277]

Sherpa

* Changes to I/O *

    * Added output tells the user if a parameter's best-fit value is
      within 1% of its minimum or maximum.
    * One can now fully specify the instrument model on one line: e.g.
      sherpa> instrument [#] = rsp[modelname](rmf.fits,arf.fits).
    * Sherpa now does limited unit checking when the columns BIN_LO and
      BIN_HI are detected in a PHA file; before, the units were assumed
      to be Angstroms.
    * Sherpa now checks to see if background, RMF, or ARF files named in
      a PHA header can be accessed before stated that their contents are
      being input; if the files do not exist, the user is informed.
    * A bug which prevented NOTICE/IGNORE IMAGE filter regions from
      being listed when the SHOW command was given has been fixed.
    * The SHOW command now outputs both the total number of counts and
      the number of counts within an applied filter.
    * Statements of the form "foo.bar = / S-Lang expression / " are now
      passed on to S-Lang as they should be. Formerly such statements
      were (wrongly) interpreted as pertaining to (non-existent) model
      parameters. (When "foo.bar" does refer to a model parameter,
      Sherpa still correctly interprets this.)

* Changes to General Fitting *

    * The background subtraction algorithm has been updated to correctly
      subtract both BACKGROUND UP and BACKGROUND DOWN, if provided.
    * The algorithm for the simultaneous fitting of background and
      source data has been corrected; this affects the source model
      normalization only.
    * The algorithm for the simultaneous fitting of background and
      source data has been updated so as to correctly treat both
      BACKGROUND UP and BACKGROUND DOWN, if provided.
    * One is no longer allowed to fit both a background and source model
      to only source data, unless the background parameters are frozen.
    * Updated computation of residuals: if the chosen statistic is
      chi-square, sigma residuals ((data-model)/error) are output; if
      Cash or Bayes, regular residuals (data-model) are output.
    * The capabilities of FAKEIT have been expanded to include the
      simulation of background data, if a background model is specified
      and background data have not been previously input.

* Changes to Optimization Methods *

    * The LEVENBERG-MARQUARDT method has been improved: (a) it now
      handles double-precision data; (b) it now behaves correctly if a
      background is being simultaneously fit; (c) repeated array reading
      has been eliminated, leading to faster operation in many cases;
      (d) an error message is thrown if the chosen statistic is Bayes;
      (e) a warning message is thrown if the chosen statistic is Cash.

* Changes to Error Analysis Methods *

    * Corrected bug which led to REG-PROJ sometimes not working
      correctly if parameters were referred to by number (abs.1) and not
      by name (abs.nH).
    * If the chosen statistic is Cash or Bayes, REG-PROJ will use
      SIMPLEX rather LEVENBERG-MARQUARDT as the optimization method.
    * The default for the parameter REG-PROJ.expfac was changed from one
      to three, in an effort to improve the initial appearance of plots.

* Altered Sherpa Models *

    * XSPEC v11 models are now accessible through Sherpa.
    * A new model called "beta" has been added. It is a function of the
      form f(x) = A*(1+[(x-xpos)/r_0]^2)^(-3*beta+1/2).
    * POW: the default slope has been changed from 0 to 1.

* Changes to Plotting/Imaging *

    * For images: one can now specify unfiltered or filtered on the
      command line (default filtered), e.g. sherpa> image [type]
      filtered. If filtered, then images will include only those data in
      the smallest rectangular region that contains all filter regions.
      NOTE 1: This does not yet apply to statistics images, such as
      IMAGE CASH. NOTE 2: Image coordinates are *not* correctly
      preserved when the filtered image is created, a bug that is to be
      corrected.

* GUIDE *

    * GUIDE has been updated to work with ATOMDB 1.1.0.

* Bug Fixes since v. 2.0 (not including items mentioned above) *

* Relating to plotting/imaging *

    * A bug which led to IMAGE ERRORS outputing an image of the input
      data has been corrected.
    * Attempts to IMAGE BACKERRORS when no background data had been read
      in led to SegVs. This has been fixed.
    * A bug affecting the plotting of model amplitudes when histogram
      data were input has been fixed. (The amplitude in the last bin was
      not shown.)
    * Fixed a bug for the Linux version, so that temporary output files
      are now written out for display via DS9.

* Relating to models *

    * A warning is now thrown if a fit is attempted, with an instrument
      RMF defined, and with the analysis command set to "channels".
    * A suboptimal algorithm for guessing Gaussian and power-law
      parameters given negative data (e.g. log-space data) has been
      patched.
    * Fixed a bug in the Sherpa polynomial models, to assure that the
      normalization parameter is always defined.
    * A fix for a bug where, after using an RMF file to define an
      instrument, Sherpa would crash after one tried to use a PSF model
      as an instrument model.

* Relating to GUIDE and S-Lang interaction *

    * The command "import('guide')", if issued after Sherpa source
      models had been defined, caused those source models to become
      undefined. This is now fixed; the user may import the GUIDE
      package at any time during a Sherpa session.
    * ARF data can now be read into Sherpa from either an ARF file or an
      S-Lang variable containing such data.
    * For DEM fitting, fixed a bug which caused errors to be
      overwritten, even when errors had been read from file.
    * When writing out MDL files, it is now ensured that line fluxes are
      always written to file, whether or not the model modeling the line
      is part of the source models being fit.
    * Update function now must be called manually to get current MDL
      information.
    * Increased storage space for source model definition in an MDL file.
    * Fixed a bug that caused Sherpa to crash upon writing out MDL files.

DataModel

    * bug fixes to support Alpha portability
    * Added support for filtering on a region composed of two columns of
      differing data type (e.g. TG_LAM,PI used in grating filtering).
    * fixed memory bug which caused spurious characters in some header
      keywords.
    * fixed bugs that prevented the DM from reading certain non-CXC
      files (e.g. ASCA, make_acisbg output).
    * fixed bug which caused some header keywords to appear out of order.
    * fixed bug: failed to calculate area correctly when virtual file is
      defined by linear ranges e.g. x=4000:5000,y=4000:5000 instead of
      spatial regions, causing incorrect BACKSCAL in dmextract.

GUIs


General

    * The CIAO X resource file reading has been modified, to locate the
      .CXCdefaults file in the following order of preference: $HOME,
      $XENVIRONMENT, and $ASCDS_INSTALL/config/system, and CIAO will
      accept either .CXCdefaults OR CXCdefaults within the
      $ASCDS_INSTALL subtree.
    * Xlib is now initialized properly within multithreaded applications.

Filtwin

    * Appearance changes: Added a 3 line message status area to provide
      users with more feedback; widened block and bin/col text fields;
      renamed the 'Update From:' label to 'Window:'; renamed 'Apply' to
      'Apply to'; renamed the 'Update' button to 'Update From'
    * Increased amount of user feedback provided, and improved content.
    * No longer brings up imager if region filtering (via dmcopy) fails.
    * Prevents pickpoints on 'exclude' filter, since plotter exclusion
      filtering is currently unsupported.
    * When 'Update From' the plotzoom is selected, an instructional
      message informing the user to pick to points from the plot zoom is
      written to the message status window

Prism

    * Print "file loading" messages to the status area during I/O.

Firstlook

    * Now looks for xterm at launch, and will exit if it cannot be found.

CHiPS

    * Improved log-scale plotting: CHiPS now permits log scaling in the
      presence of negative data, lines, or labels (by simply ignoring
      them), whereas previously even 1 such datum would entirely
      prohibit the scaling.
    * Improved multithreaded control.
    * Change in pickpoints function to correct a segmentation fault on
      Linux.
    * Minor updates to fix some DS9 image engine commands, and prevent
      'ximtool' warning messages when more than one instance of ds9 is
      running.

Obsvis

    * Updated ephemeris file and fixed star retrieval bug.

Infrastructure

   History/DmT/Caldb/Tcd Libs
     - Fixed problem seen on alpha with memory allocations.
   Pixlib
     - Now includes information previously stored in grating.par

   Varmm
     - Added I/O support for dmBIT/dmBOOL/dmUSHORT/dmULONG column types