CIAO 2.2 Release Notes
Return to: Version History
Contents:
- Tools
- Sherpa
- User Interface (UI)
- Libraries
- Documentation
Tools
acis_detect_afterglow
. Bug fix for not checking ccdid, could have tagged a few extra
events as 'glowing' that were not.
acis_fef_lookup ** NEW **
. Back in the baseline CIAO distribution.
. Determines the FEF file and filter needed for the source position.
acis_process_events
. New algorithm to compute sky positions for CC mode events. This
works to remove the dither in the CHIP Y direction.
. should stop warning about finding two calibration files for the
grade file.
. New algorithm to check for pixel values in outer 5x5 island (very
faint mode only) greater than split threshold. Will set status bit 23.
AKA "very faint" checking and/or "part55".
. Includes 2 new parameters for very faint checking: check_vf_pha to
determine whether the check is done or not and "trail" which is an
ad-hoc adjustment factor for CTI effects.
. Fix to deal with bad input obs.par file
. Added new parameter to control PHA randomization
. Fixed bug in TE Graded mode where badpixel could have been missed.
(not an issue for SDP, only user supplied badpixels isolted to a
single pixel rather than 3x3 island)
. worked history so that duplicate history records aren't written
out.
. Bug fix to not override TIMPIXR if already exists when using an
obs.par file
acisreadcorr ** NEW **
. Tool to identify the 'streak' of events in the chip read-out
direction that coincides with a bright source. These are events
that are detected during the frame shift/read-out.
. Sets times for those events so some rudimentary time analysis can
be done.
aconvolve
. fixed incomplete error message
acrosscorr
. no significant changes
add_grating_orders ** NEW **
. Previously part of the optional download, this script is now
packaged as part of the CIAO distribution.
. Adds together spectra from different grating orders.
add_grating_spectra ** NEW **
. Previously part of the optional download, this script is now
packaged as part of the CIAO distribution.
. Adds grating spectra (eg MEG and HEG)
apowerspectrum
. no significant changes
asp_apply_sim ** deprecated **
. No longer necessary with the new 'asphist'
. May not be shipped in later CIAO releases.
asp_calc_offsets ** deprecated **
. No longer necessary with the new 'asphist'
. May not be shipped in later CIAO releases.
asphist
. New asphist that works directly from the aspect solution files (asol).
. Will continue to work with the output of asp_apply_sim (backwards
compatible)
. fixed clobbering problems for corrupted outfile
axbary
. This was a fundamental but minor bug that led to errors of the
order of less than 1 us in the barycenter correction.
. There was a error condition related with keeping track of the
position in the orbit ephemeris file that was not properly taken
care of.
. Apply TIMEZERO: a bug that affected XTE corrections
. Output writable
celldetect
. Added source cell locations and double detection flags in the
celldetect output. New output columns CELL_X, CELL_Y, DOUBLE, and
DOUBLE_ID.
. Sources which are on blocking boundaries and are detected in two
successive blocking passes are flagged in DOUBLE and DOUBLE_ID
. checks for successful creation of the logfile
. Fixed src centers for even size cells
. Fixed problem with clobbering files
. Enhanced duplicate suppression based on the .5 pixel centers for
even sized cells. Now it can also reject duplicate sources that
are slightly further apart. User's may see their source lists
shrink slightly.
. Upgraded default parameter file for the latest psfsize table
. Fixed problem with src ellipse axis lengths when data are blocked
. The region output files have the revised ascii region format
acceptable to both ds9 and the DM.
csmooth
. Fixes clobber problem. Also increases length of filename arrays.
. fixed bug for input scale map when only 1 pixel used a certain
scale.
. fixed incomplete history
. Fixed waiting until end to test whether to clobber outputs
destreak
. Updated so clobber messages doesn't appear when verbose=0
dmappend
. Fixed dmappend not returning appropriate exit value
dmarfadd
. no significant changes
dmcontour
. Fix some long standing issues when the contours included regions
that should really be excluded. Regions will now properly be
included/excluded. There where problems in the past when regions
where inside each other, the logic wasn't correct.
. Fixed problem when no contours are found
dmcoords
. fixed bad behavior when bad inputs given for celfmt
dmcopy
. Updated so clobber messages doesn't appear when verbose=0
dmextract
. The POISSERR keyword is properly set in the counts case to reflect
when the errors are Poisson or not.
. proper error case when the RA/Dec cannot be determined.
. Updates to deal with doubles and floats in images
. Updates to catch DBL_EPSILON problems on linux
. Now allow the use of Poisson or Gaussian for the pha/histogram
case.
. Added a warning if the user tries to use a variance file
. Reset the defaults for the error values
. The exposure maps were not correctly accounted for. Dmextract was
weighting the images by the binning factor.
. There was a problem in the way values were being indexed in image
source extraction.
. Fixes problem w/ normalization of exposure maps
. Added reading units from exposure maps to normalize counts
correctly
. Fixed PHA_MIN/PHA_MAX bug
. Support for BKG subtraction in PHA/histogram space
. We are going to be multiplying the (cm^2.s) maps by the livetime
in the file. If the user inputs a exp file, but not a bkg_exp file
or vice-versa, We will ignore the exp values, and calculate
without them and print out an error message.
dmfilth ** NEW **
. "FIL in The Hole"; used to create background maps by excluding
source regions from the image.
. Various method s are presented; some are good for 'counts' images
(eg Poisson), some are good for smoothed images, eg bilinear
interpolation.
dmgroup
. Fixes case where writes out grouping keyword & column
dmgti
. fixed bug when using string keywords (truncating the string by 1
character).
dmhedit
. Added checks against bad input list files
. Made operation, key, etc automatic so user will be queried (if not
filelist is present, eg filelist="")
. Fixed problem when given a file list with nothing in it
dmimg2jpg
. Cleaned up the color lookup tables. There were extra entries at
the end of the files that were not part of the original color
schemes
. changed region support to use region lib parsing for more general
use.
. fixed minor annoyance in clobber output
dmimgcalc
. Added support for infile2="none" to allow simple scalar
multiplication of images
. Get rid of aim point tests for WCS matching.
dmimghist
. Added column for normalized counts
. pixels above and below boundary are not counted in end bins. This is
controlled thru new "strict" parameter.
dmimgthresh ** NEW **
. Sets all values in an image above/below a certain limit to a new
value.
dmjoin ** NEW **
. Interpolates columns from one table onto another by joining the
two files on a common column.
dmkeypar
. Fixed problem on Alpha where string-length was truncated.
dmlist
. Will report "Null" values
. Increase precision of output for doubles/floats.
. Fixed problem dmlisting 1x1 array
dmmakepar
. no significant changes
dmmakereg
. fixed clobbering problem
. better error handling when bad input is supplied
dmmerge
. fixed clobbering problems for corrupted outfile
dmpaste
. Fix problems where error messages were causing segv's.
. Added warnings when columns being pasted from 'pastefile' exist in
'infile'; 'pastefile' columns are skipped.
. Fixed broken error message when no. rows in files not the same.
dmreadpar
. no significant changes
dmrega2fits ** REMOVED **
. Removed from CIAO. Use dmmakereg to create ASC-FITS-Region style
region files. Use tg_create_mask to create grating region files.
dmregrid
. no significant changes
dmsort
. changes, but some are untested. Should continue to only use in
defined CIAO threads.
dmstat
. fixed the incorrect sum problem when centroid=yes.
. Added checking for NaN's and TNULL's in columns
. Added support for NaNs in floating point images. and NULL values in
integer images.
. Changed output format to use "%g" rather than "%f" to get a fixed
precision.
. Removed code that did coord transforms automatically. Users can
now use the datamodel to specify columns with WCS, eg dmstat
infile.fits[cols ra,dec]
. Will now report centroid positions in physical coords in addition to
logical coords.
. Will now report the location (in physical coords) of the min
. only the first instance of the min/max are reported.
. Reports location of min/max regardless of centroid being computed.
. WARNING: the output format has changed slightly, which may break
scripts that use dmstat
dmtcalc
. Fixed divide by zero SIGFPE.
. Fixed scientific notation so that you can enter "3E12" rather than
needing to enter "3.E12"
. Fixed problem with column smoothing and keeping original value
unless explicitly over-written, so foo=goo:3 will have the
original values of "goo" in the outfile, but goo=goo:3 will have
the smoothed values.
. Now propagates the WCS attached to various columns to the output.
. Should also now be setting TLMIN/TLMAX's and also propagating the
DSS (GTIs) properly.
. fixed bug when using string keywords (truncating the string by 1
character).
dmtype2split
. no significant changes
dmwritefef ** REMOVED **
. No longer packaged with CIAO.
get_src_region
. no significant changes
hrc_process_events
. fixed bug that was allowing PI values above 255
. Fix to deal with bad input obs.par file
. worked history so that duplicate history records aren't written
out.
lightcurve
. no significant changes
mkarf
. Ardlib options permitted on the mirror parameter (e.g., AREA=1).
. The name of the current working directory is written to the
history keyword to give meaning to relative path names.
mkexpmap
. The name of the current working directory is written to the
history keyword to give meaning to relative path names.
mkgarf
. CALDB support for the OSIP file.
. IRMXFIL header renamed to OSIPFILE
. The engrid parameter now permits "compute" to be used to
indicate that a default grid is to be used that is consistent
with the resolution of the instrument.
. The name of the current working directory is written to the
history keyword to give meaning to relative path names.
mkgrmf ** NEW **
. This program computes a grating RMF.
mkinstmap
. BUNIT keyword added to the output file.
. The name of the current working directory is written to the
history keyword to give meaning to relative path names.
mkpsf
. If user requests a position outside the current psf library file, the
tool will return the closest match (and issue a warning). Previously,
this was a fatal error.
. Correct bug which sometimes identified two interpolation data
columns when no interpolation was needed.
. Allow alternate filepaths with autonaming for psflib data output
files, check for pre-existing files and add clobber param for
psflib files.
. ability to extract individual psf images in addition to interpolated
one.
. Added additional output header keywords so that detect tools can
run on psf's that are output
. Added naming options for output psflib data files,
. fixed bin/rotate bug for interpolated image
. add ROLL_NOM keyword to output files
. remove user option to specify roll angle.
. fix for not write out RA/DEC in header when in DET coords
. Added CALDB automatic lookup to pick the "best" PSF file ONLY when an
input event file is provided.
. Fixed bug which caused an error in the output interpolated PSF
image when the PSF SIZE parameter was chosen to exactly match the
PSF image size after rebinning.
. Added bug fix for incorrect energy hdr keyword in output psflib
data files when selected energy was out of range.
mkrmf
. Includes new "weights" parameter. When set, a weighted RMF based on
the weights of the FEF regions will be created.
. When "weightfile" is set, any filter appearing in 'infile'
expression will be ignored
. Allows user to input abbreviated "axis1=energy" when weights file
is used since it uses energy grid from weights file
. Still backwards compatible with the existing single FEF inputs.
. Fix for SEGV when logfile name was missing or ""
. Clobber checked at the beginning of run
. Added CALDB lookup when using a weight file (ONLY when using a weight
file).
. Speed enhancements
. add message to stdout to inform multi-region precessing status
. add CHECKSUM/DATASUM/CREATOR keywords
. remove spatial filter from FEFFILE keyword
. quite unnecessary warning message
. fix for incomplete error msg when bad FEF region is requested
mkwarf ** NEW **
. new tool to create a weighted ARF file based on an input WMAP.
. second (necessary) output is a set of weights needed by mkrmf to
create the corresponding weighted RMF.
. Users must supply "[wmap]" on input file name if using a wmap from
dmextract.
mtl_build_gti
. fixed bug when using string keywords (truncating the string by 1
character).
paccess
. no significant changes
pdump
. no significant changes
pget
. no significant changes
pline
. no significant changes
plist
. no significant changes
pquery
. no significant changes
psextract ** NEW **
. Was originally on the optional download list, this is now part of
the basic CIAO installation.
. Extracts the spectra and computes the arf and rmf for the given
source region.
pset
. Now internally sets tool name. The upshot being that you can get
help on the tool you are psetting parameter for.
psf_project_ray ** NEW **
. Post CIAO2.2, the CXC will allow users to generate custom ray
files thru the CXC's mirror model, "SAO-SAC". This will be a web
interface.
. psf_project_ray will take these rays and project them onto the
detector configuration (including non-standard SIM locations) to
produce a PSF at the exact desired location.
punlearn
. no significant changes
reproject_events
. New functionality to set a random time w/in the GTI of the match file.
This allows the "background" datasets to be used w/in CIAO
. New parameter to control time/position randomization.
. various error checking on bad inputs
rmfimg ** NEW **
. Creates a 2D image from an input RMF file. Can be displayed in
imager and includes WCS.
. Will optionally multiply by the ARF.
sso_freeze
. added check to verify at least 2 sso ephem records exist prior to
first event.
stk_build
. no significant changes
stk_count
. no significant changes
stk_read_num
. no significant changes
tg_create_mask
. Fixes problem with HETG mask in CC mode.
. added hetg and letg width factor parameters. width_factor defaults to
redirect of hetg.
. Removed HESF parameters
. Added WPSF keyword to properly catch what CALDB resolved to at
run-time
. made output more consistent with CXC region format so it can be
used as a dm filter.
. Added history information about infile, outfile, and calibration
files used.
tg_resolve_events
. Fix bug where 0th order photons were not getting assigned dispersion
coordinates.
. Fixed problem with CC mode data not being de-dithered if the aspect
solution files are used rather than the aspect offsets.
. Fixed ACIS CC+LETG case
. Changed ACISDE keyword to OSIPFILE to properly catch which CALDB
file was used.
. Now gives out warnings only if GRATING keyword is HETG but the
user enters LETG to the grating_obs parameter or vice versa
. Removed HSEF code to clean things up.
. made changes so it can now handle the cases where the center of
the zero-order source falls in a chip gap
. changed the chip_minrange & chip_maxrange variable from short to
double. Also changed the logic to check if an event's coordinates
lie within the valid range. The changes should make the tool
resolve events that are very close to the edge of the detector.
. fixed minor un-noticed bug that could have effected users iff they
had tried to force the LETG grating type.
. The tool can now sort order (for acis) by using 2 new parameter values
(osort_lo, osort_hi) from the par file. This order-sorting method will
be used only if the osipfile (used to be the rmfile parameter)
parameter is set to "NONE".
. Changes in parameter interface: 1)rename the rmfile parameter to
osipfile 2)add osort_lo and osort_hi parameter
tgdetect
. Fixed problem when no. counts per pixel greater than 32K
tgextract
. made tg_lam,tg_d into a vector column to support region filtering.
. Fixed problem with entering "+" before number
. added MTYPE/MFORM keywords to make output regions be valid FITS
region files.
. Added INREGION keyword to properly catch what CALDB resolved to at
run-time
. deleted the tg_d_units parameter from tgextract.par it can only
handle tg_d in degrees, so it's meaningless to have tg_d_units in
the par file
tgidselectsrc
. no significant changes
tgmatchsrc
. no significant changes
vtpdetect
. The region output files have the revised ascii region format
acceptable to both ds9 and the DM.
wavdetect
. The region output files have the revised ascii region format
acceptable to both ds9 and the DM.
. fixed output file autonaming extensions.
. Fixed problem with src ellipse axis lengths when data are blocked
wrecon
. Fixed autonaming problem
. Fixed problem with src ellipse parameters when data are blocked
. Fixed problem with src ellipse axis lengths when data are blocked
. The region output files have the revised ascii region format
acceptable to both ds9 and the DM.
wtransform
. Fixed segv when exposure map size does not match input image size
Sherpa
New (as opposed to enhanced) functionality:
. EQWIDTH: [command] computes the line equivalent width.
. FTEST: [command] computes the F-test model comparison test significance.
. MLR: [command] computes the Maximum Likelihood Ratio model
comparison test significance.
. UNLINK: [command] undoes previous linkage of a specified parameter.
. GRIDMODEL: [model] allows one to use a numerical model in Sherpa fits;
exposure maps may be specified via GRIDMODEL (see also EXPMAP).
. sherpa_eval() has been added to allow execution of Sherpa commands
in S-lang scripts.
. One may read in systematic errors in addition to statistic errors.
Changes to I/O:
. For PHA files, filtering and fitting can be done in either energy *or*
wavelength space, if an instrument is specified.
. DATASPACE now includes HISTOGRAM (binning) option.
. One may specify additional filters after issuing the command READ FILTER.
. One may specify additional filters after issuing the command IGNORE BAD.
. New functions (e.g., sherpa_update_mdl()) have been written to create MDLs
and store them in S-Lang variables.
Changes to General Fitting:
. Generalized background: source and background datasets no longer need to
be of the same size.
. Background data may be fit alone, with no source data specified.
. Background datasets may have their own instrument models.
. In image analysis, one may specify that the unfiltered background dataset
is the same as the source dataset using the command BACK => DATA.
. PSFFROMFILE: [model] is now faster for 2D convolution and has an improved
interface for 1D convolution.
. PILEUP model stacks may now be specified.
. XSPEC models may be used even when analysis is performed in wavelength
space.
Changes to Sherpa Models:
. New model: JDPILEUP, a pileup model.
. New model: NBETA1D, a normalized beta model for line fitting.
. Sherpa user models can now be written as S-Lang scripts.
. More scripts for doing high resolution spectroscopy (GUIDE).
. One can apply XSPEC v11.1 models.
Changes to Visualization:
. Visualization of RESIDUALS now shows (data - model); DELCHI shows
(data - model)/error.
. More background visualization options: BFIT, BRESIDUALS, etc.
. IMAGE FIT now tiles images in ds9 by default.
. Contour and surface plots updated to handle arbitrarily filtered data;
also, multiple surface, contour plots can be displayed at once.
. New OPLOT command to overplot several curves in one drawing area.
. WRITE has been updated to provided energies or wavelengths of bins;
and filtered or unfiltered data.
. Filtered statistics images can now be visualized.
. Use of new store/restore with chips; the plotted data is stored with the
Chips FITS file.
User Interface (UI)
ChiPS
-----
- introduced chips state object, which signficantly improves
ease and flexibility of chips customization
- will read .chipsrc file at startup, if found
- more streamlined/stable use of S-Lang
- can now be used outside of CIAO, via import("chips") in
any application that embeds S-Lang (like slsh, or ISIS, etc)
- chips_eval() function introduced for executing chips commands
within S-Lang scripts
- store/restore: has been extended to correctly save/restore plots
generated from S-Lang variables
- undo/redo: has been extended to correctly save/restore plots
generated from S-Lang variables
- several enhancements to limits functionality
- new --batch and --slscript startup options
- deprecated 'rename' command
GUIs
----
- cell-based table and image editing has been added to PRISM,
including capability to add rows and columns
- PRISM now maintains a recently-visited files list
- replaced non-editable quicklook plots in PRISM with full chips
plotting functionality
- improved Analysis menu organization in all GUIs
- FILTWIN layout enhanced
- support for new RA/DEC entry formats in OBSVIS
- OBSVIS roll/visibility plot calculations now use auxiliary datafile
rather than hardcoded numbers
Varmm/S-Lang
------------
- writefits(): extended ISIS fitsio functionality and encapsulated it
an within an easy-to-use function, thus providing powerful FITS
table and image writing capability from S-Lang in CIAO
- will read .varmmrc file at startup, if found
- introduced varmm state object, to improve customization of
varmm/s-lang behavior in CIAO
- introduced new VarmmFunc class to encapsulate S-Lang function
as a C++ object
- S-Lang structures returned by varmm I/O functions now use underscore
prefix to denote fields which contain metadata
- structure fields which reflect table column names are no longer
lowercased, but rather reflect the case used in the file on-disk
- min/max/step arrays are now present in the S-Lang structures
generated by the image I/O routines
Miscellaneous
-------------
- ISIS spectral analysis package now bundled with CIAO, and accessible
within ChIPS and SHERPA commandlines via import("isis")
- several enhancements to ahelp (command line help), including
better performance and access to either local or web-based content
- proposal/mission planning tools (OBSVIS and proposal planning tooklit)
merged into CIAO distribution
- significantly improved user documentation
Libraries
datamodel
. Fix for area calculations not being truncated by field size
. Support for row number filtering: dmcopy "evt.fits[#row=100:200,...]"
However [cols #row] will fail.
. Support to access WCS/Coordinate columns as virtual columns dmlist
"foo_evt.fits[cols ra,dec]"
. Fixed problems with string length when complex regions were used
. Fix for bug in rebinning non-square images: dmcopy "image.fits[bin
x=4,y=4]" rebin_image.fits
. DM enhancements to support the definition of a null value when making
images, use "[opt null=value]"
. Problems with many-point-polygon region filtering fixed.
. Can suppress BTYPE/BSCAL FITS rescaling using "[opt scale=no]"
. Can rename columns on-the-fly with "[cols new=old,...]" syntax.
parameter lib
. Fixed some error messages to indicate which parameter files was
being accessed.
regions
. Fixed bug that didn't parse "'" correct for arcmin.
. new "bounds()" functionality to get a bounding rectangle around
whatever region is supplied
caldb
. Added ability to override any/all parameters from the command
line.
stacklib
. Now allows one to escape "," in file names using "\,"
. Blank lines (strlen()==0) in input list files (@foo.lis) are now
ignored.
Documentation
threads (documents_threads.html) . a number of new threads developed . existing threads reviewed and updated to include CIAO 2.2 capability . threads now available as pdf documents scripts (download_scripts.html) . new and major revisions for ciao scripts . most widely used scripts taken from the 'user contributed' area, and incorporated as a part of the standard CIAO build. these scripts are "psextract", "acis_fef_lookup", "add_grating_orders", and "add_grating_spectra". manuals (documents_manuals.html) . significant revisions to the data model manual(s) and detect guide help system (ahelp) (documents_ahelp.html) . added the "-i" option to access the "i"nternet version of the ahelp file, at "http://cxc.harvard.edu/ciao/ahelp/[toolname].html". this complements the existing "-w" flag, which accesses the *local* html version of the ahelp document. . cleaned up the appearance of online ahelp files and created a subject-ordered list . ahelp files now available as pdf documents www . redesigned the ATODMB (../atomdb) web pages
