Skip to the navigation links
Last modified: 16 July 2020

URL: https://cxc.cfa.harvard.edu/ciao/releasenotes/ciao_4.12.1_release.html

CIAO 4.12.1 Release Notes


CIAO 4.12.1 is distributed for the following platforms:

[New] CIAO is now available via the conda package manager: conda builds are available for Linux and Mac for Python 3.7, 3.6, and 3.5. More information can be found in the conda section of the Installation notes below.

There is no support for 32 bit operating systems, older Linux (CentOS 5 era) or older macOS platforms (OS-X El Capitan and earlier). More details can be found on the Platform Support page.



Notable changes and improvements in CIAO 4.12.1

CIAO 4.12.1 is identical with CIAO 4.12, except for a new version of Sherpa (Sherpa 4.12.1). This version provides fixes to bugs that would stop an analysis session or provide incorrect results. The Sherpa bug fixes introduced in CIAO 4.12.1 include:

  • PR 728: load_multi_arfs and load_multi_rmfs caching fixed which would otherwise prevent fit from running when using multi-order responses.

  • PR 756: calc_photon_flux and calc_energy_flux enhanced to better handle monochromatic flux densities and fluxes where the energy bounds fall in between the model grid bin edges. A new model argument is also added to calculate "unabsorbed" fluxes by specifying specific model components.

  • PR 759: ARF model caching now optional, should default caching cause incomplete fitting.

  • PR 725: quash long warning messages generated in resample_data.

  • PR 716: plot_cdf fixed so it will draw a plot instead of throwing an error.

  • PR 745: ensure model parameter limits adhered to even when linked to another model parameter with a different set of limits.

  • PR 747: re-worked handling of regridding user-supplied grid that does not match data grid.


Notable changes and improvements in CIAO 4.12

  • CIAO 4.12 has several important event processing updates including ACIS badpixel updates to exclude frame-store shadow region, HRC updates to gain calibration, HRC degap calibration file changes, and improvements to the ACIS temperature dependent CTI correction.

    [WARNING]
    CALDB 4.9.0 version requirements

    CIAO 4.11 and earlier users analyzing ACIS data should NOT upgrade to CALDB 4.9.0. While the new format of the temperature dependent CTI correction calibration files can be read by earlier versions CIAO, the higher order coefficients are not included. The new columns are ignored resulting in an inaccurate temperature adjustment being applied.

    CIAO 4.12 HRC users must update to CALDB 4.9.0 due to the changes in the degap file. Similarly, CALDB 4.9.0 should not be used with earlier versions of CIAO. In both cases doing so will trigger an earlier degap algorithm that performs badly at the edges of the plates.

  • The ChIPS plotting package has been withdrawn from CIAO 4.12. Users should migrate their sherpa and plotting scripts to use matplotlib. The CXC has provided a conversion guide to assist users to modify their scripts to use matplotlib. prism has been updated to create plots using matplotlib.

  • CIAO can now be installed using the conda package manager. This is still experimental. We have performed our standard set of tests using generic, vanilla conda installations however given the large number of packages and installation options it is possible users may encounter unforseen problems. Please report these to the CXC Helpdesk. Users can install the conda edition with Python 3.7, 3.6, or 3.5 on Linux and Mac.

    Feedback Requested

    The CXC is actively soliciting user feedback on this installation method; users can submit their comments to the CXC Helpdesk.

    Please review the known problems on the conda installation page before contacting the helpdesk.

  • CIAO includes version 8.1 of SAOImage ds9. Users are reminded that they now need to change the Edit mode setting in order to select or create regions in recent versions of ds9. Please see the watchout page for more information on this.

    SAOImageDS9 v8.1 highlights include:

    • New support for overlaying masks based on world, physical or image coordinates. Each mask is rotated, scaled, and centered upon the specified coordinate system. There is no limit to the number of masks which can be displayed.

    • An all new plot tool is now available. Each plot tool dialog can contain one or more graphs, arranged in row, column, or grid order. In addition, a new strip plot layout is also available. Support for line, bar, or scatter graphs is provided. Each graph has its own set of display parameters and can contain one or more data sets. Each plot dialog maybe printed as a postscript image, or exported as tiff, png, jpeg, or gif.

    • The movie feature now supports animated GIFS. 3D data cubes, or a sequence of multiple frames can be animated, and saved as an animated GIF to be displayed on the desktop, in twitter or on a web site.

      rotating ciao log

    • Contours have been reimplemented to provide high quality resolution and have been threaded to take advantage of multiple CPUs, providing fast rendering, especially in the case of 3D data cubes.

  • Sherpa includes both the changes made in the Sherpa 4.11.1 release and Sherpa 4.12.0. Notable changes include:

    • Plotting is now handled by Matplotlib as ChIPS has been removed in CIAO 4.12.

    • Update to version 12.10.1n of the XSPEC Model library, which adds support for the following additive models: xsagnsed, xscph, xskyrline, xsqsosed, xsvcph, xszbknpower and xszlogpar.

      [WARNING]
      Default XSPEC abundance and cross-section settings

      The 12.10.1 XSPEC model library now sets default values based on the contents of your $HOME/.xspec/Xspec.init file if it exists.

      If you do not have this file then the default cross-section table has changed: it now defaults to vern when it was bcmc in CIAO 4.11 and earlier.

    • The plot_xxx family of routines - such as plot_data and plot_fit - can now accept plot settings as keyword arguments. The allowable keywords are the same as those returned by the get_xxx_plot_prefs routines, such as xlog, ylog, and color.

      As an example,

      plot_fit(xlog=True, ylog=True)

      will use logarithmic axes for both axes of the plot.

    • Sherpa now supports the use of a PSF image with a finer resolution than the resolution of 2D data images. 1D models can also be evaluated on a finer grid than the grid of the input data arrays. The models are evaluated on the resolution given by the PSF pixel scale or the 1D arbitrary grid. The evaluated models are rebin to match the original data scales for calculating the statistic during the fit.

    • Asymmetric errors are now supported for calculating parameter uncertainties via bootstrap. The data with asymmetric errors can be entered using a new load_ascii_with_errors function or Data1DAsymmetricErrs class.

    • The paging performed by the show_xxx set of commands - such as show_data and show_all - is now handled by Python rather than using an external command. This means that the output will now appear in Jupyter notebooks, and that the PAGER environment variable is no longer checked.

    As a reminder, Sherpa is also available as a stand alone system, accessible from the Sherpa GitHub repository or from the standalone Sherpa page.

  • The CXC Datamodel has several updates related to reading various ASCII files.

  • Several tools have been updated including

    • Updates to the temperature dependent ACIS CTI correction algorithm in acis_process_events. These changes will be activated by future CALDB updates; until then the temperature dependent CTI corrections will be unchanged.

    • Rows at the bottom of the ACIS CCDs are now being marked as bad due to the "shadow" cast by the frame store covering.

    • HRC updates to support pending gain calibration changes and to fix a technical issue with the previously released updated degap files.

    • wavdetect now creates additional diagnostic columns to help identify duplicate sources.

    • Many tools have been updated to properly copy the image units from the input to the output image.

  • prism, peg, and taskmonitor now use GTK3. prism has been updated to use matplotlib for plotting.

  • There have been several script updates since the CIAO 4.11 release including:

    • chandra_repro updates to support new badpixel CALDB updates (frame store shadow).
    • Updates to several script to use matplotlib for plotting.

    • Updates to dax include using ds9 for plotting, updates to simplify existing analysis tasks, and new radial profile fitting option.

    • Support for Version 2 of the Chandra Source Catalog.


How CALDB 4.9.2.1 Affects Your Analysis

CALDB 4.9.2.1 Release Notes (release 09 July 2020), includes the CalDB 4.9.2 updates.

ACIS Imaging and Grating Data

HRC Data


How CALDB 4.9.1 Affects Your Analysis

CALDB 4.9.1 Release Notes (release 03 April 2020)

ACIS Imaging and Grating Data

HRC Data


How CALDB 4.9.0 Affects Your Analysis

CALDB 4.9.0 Release Notes (release 17 December 2019)

ACIS Imaging and Grating Data


Installation

Users should be aware of these installation items before installing CIAO 4.12. Additional problems which are seen less frequently are listed on the Installation & Smoke Tests bug page.

Supported Platforms

Updates to the ciao-install Installation Script

Installing with the conda package manager

Source builds

IPython Settings

Remove old parameter files


Tools

axbary

acis_build_badpix

acis_process_events

acis_streak_map

addresp

celldetect

dither_region

dmcoords

dmcontour

dmfilth

dmimgadapt

dmimgcalc

dmimgdist

dmimgfilt

dmimgpm

dmimgreproject

dmkeypar

dmmaskbin

dmmaskfill

dmnautilus

dmregrid2

dmsort

hrc_build_badpix

hrc_process_events

reproject_image

reproject_image_grid

skyfov

tgdetect

wavdetect


Parameter Files

A summary of parameter files changes is provided in this section. Refer to the Tools and Analysis Scripts sections of these release notes for complete details.

We recommend deleting all the old parameter files or renaming the parameter-file directory before running any new version of CIAO to avoid conflicts. More information is available in the FAQ.

acis_build_badpix

acis_process_events

skyfov

tgdetect


ChIPS

The CXC has decided to dropped support for ChIPS and move to using Matplotlib, alongside DS9, as the primary plotting and visualization systems in CIAO.

A ChIPS to Matplotlib conversion guide is provided for users who would like to transition from ChIPS to Matplotlib. Please contact the CXC Helpdesk if you have further questions or encounter problems.

The version of Matplotlib and choice of backend (the library that provides the visualization and any interactive capabilities) depends on how CIAO was installed. The ciao-install installation provides version 3.0.2 of Matplotlib and the TkAgg back end. The conda installations, in particular for Python 3.6 and 3.7, provide newer versons.


Sherpa

Sherpa is developed as a community project, and is available on GitHub. Comments, fixes, and additions from the community are welcomed. There are additional releases for this version throughout the year, and can be installed either directly from GitHub or using the conda package manager, as described on the standalone Sherpa documentation page.

Switching to Matplotlib

Plotting improvements

XSpec

Notable changes

Bug fixes


Graphical User Interfaces

prism, peg, and taskmonitor have been updated to use GTK 3. The applications will have a different look than before.

prism

peg

taskmonitor

obsvis


Analysis Scripts

There have been numerous updates to the contributed scripts package since CIAO 4.11 was released. These changes are specific to the CIAO 4.12.1 contributed scripts release.

chandra_repro

check_ciao_version

check_ciao_caldb

combine_grating_spectra

combine_spectra

dax

deflare

make_instmap_weights

tgsplit

ciao_contrib.runtool

lightcurves

ciao_contrib.all

sherpa_contrib.utils

sherpa_contrib.xspec.xsconvolve

chips_contrib


Python Modules

caldb

crates

group

history

paramio

pixlib

psf

region

stk

transform


Libraries

ardlib

cxcparam

dm

dslib and hdrlib

pixlib

region


Environment

Environment variables

Off-the-Shelf (OTS) Package Versions

Compiler versions


Documentation

Many pages have been updated to reflect the changes in this release. Several analysis threads are still under review.