Skip to the navigation links
Last modified: 7 November 2019

URL: http://cxc.harvard.edu/ciao/releasenotes/ciao_4.11_release.html

CIAO 4.11 Release Notes


CIAO 4.11 is distributed for the following platforms:

CIAO 4.11 is the first release to only support Python 3 (it includes Python version 3.5.4), and also to include matplotlib version 2.2.3, the Jupyter notebook system, and support Python package installation into CIAO with pip3.

There is no support for 32 bit operating systems, or old Linux (CentOS 5 era) or macOS platforms (OS-X Yosemite and earlier). More details can be found on the Platform Support page.



Notable changes and improvements in CIAO 4.11

  • This is primarily a maintenance release: bug fixes, supporting new compilers and OTS upgrades.

  • Only Python 3.5 is provided, since support for Python 2.7 in the scientific software ecosystem is diminishing.

  • CIAO now includes some commonly-used Python packages, namely matplotlib version 2.2.3, using the TkAgg backend, and jupyter notebook support.

    A quick guide to converting from ChIPS to Matplotlib is provided.

    Users can also use pip3 to install additional packages into their CIAO distribution, assuming they have write permission to the installation.

    unix% pip3 list --format=columns

    will display the list of Python packages installed with CIAO.

    To support encrypted https:// access in Python - which is required by pip3 - it may be necessary for users to install version 1.0 of the Secure Socket Library (libssl.so.1.0). A simple check is to run (after starting CIAO):

    unix% python -m ssl

    The installation of OpenSSL is only needed if this command errors out.

    Pure Python packages such as astropy and scipy are supported. Python packages which require compiling against C/C++ libraries may not build correctly.

  • CIAO includes version 8.0 of SAOImage ds9. Users are reminded that they now need to change the Edit mode setting in order to select or create regions in recent versions of ds9. Please see the watchout page for more information on this.

    SAOImageDS9 v8.0 highlights include:

    • New backward compatible reimplementation of the interface controlling the command line, XPA, and SAMP. Implemented using formal parsers the API will provide error detection feedback: should an error be detected while processing a command, DS9 will now indicate to the user exactly where the error occurred and suggest options.

    • WCS support has been reimplemented to fully support FITS WCS Papers I, II, and III. In addition to supporting the official WCS standards (including generic non-equatorial celestial systems), a number of non-standard WCS definitions are also supported.

  • Sherpa includes both the changes made in the October 2018 standalone 4.10.1 release and a number of updates since. The XSPEC models have been updated to version 12.10.0e.

    The default option is for Sherpa to use ChIPS for plotting, but the plot_pkg setting in the Sherpa resource file can be changed to select Matplotlib.

    As a reminder, Sherpa is also available as a stand alone system, accessible from the Sherpa GitHub repository or from the standalone Sherpa page.

  • Several tools have been updated including

    • The celldetect tool now requires users supply an input PSF map: an image whose pixel values represent the size of the PSF at each location. For Chandra datasets, user may use the mkpsfmap tool or the updated functionality in fluximage, flux_obs, or merge_obs to create a PSF map.

    • dmextract now compute the mean radius, RMID, column automatically when creating radial profiles.

    • dmhedit and others have been updated to produce errors when asked to modify various read-only files such as compressed files or insufficient permissions.

  • A new Python history module has been added to allow crates to propagate HISTORY records from the input to the output. It also allows scripts to create CXC standard history records compatable with the dmhistory tool.
  • There have been several script updates since the CIAO 4.10 release including:

    • Update to chandra_repro to include additional columns in the evt2 file for HRC datasets.

    • Update to fluximage to create a PSF map for the observation (this is optional), and to the flux_obs and merge_obs scripts to merge these PSF maps (if created). These PSF maps are needed to run wavdetect and celldetect.

    • The fullgarf script can now be used to process MARX-simulated data, by setting the new ardlibqual parameter.

    • Update to find_chandra_obsid to support the new secure CXC servers.

    • The convert_xspec_user_model script has been removed. Please contact the CXC Helpdesk if you find this script useful and would like to see it resurrected.


How CALDB 4.8.5 Affects Your Analysis

CALDB 4.8.5 Release Notes (release 07 November 2019)

ACIS Imaging and Grating Data


How CALDB 4.8.4.1 Affects Your Analysis

CALDB 4.8.4.1 Release Notes (release 12 September 2019)

ACIS Imaging and Grating Data


How CALDB 4.8.4 Affects Your Analysis

CALDB 4.8.4 Release Notes (release 03 September 2019)

ACIS Imaging and Grating Data

HRC Data


How CALDB 4.8.3 Affects Your Analysis

CALDB 4.8.3 Release Notes (release 23 May 2019)

ACIS Imaging and Grating Data

HRC-S Data


How CALDB 4.8.2 Affects Your Analysis

CALDB 4.8.2 Release Notes (release 13 December 2018)

HRC Imaging and Grating Data


Installation

Users should be aware of these installation items before installing CIAO 4.11. Additional problems which are seen less frequently are listed on the Installation & Smoke Tests bug page.

Supported Platforms

Updates to the ciao-install Installation Script

IPython Settings

Remove old parameter files


Tools

acis_process_events

aconvolve

acrosscorr

addresp

arestore

celldetect

csmooth

dmappend

dmdiff

dmextract

dmhedit

dmhistory

dmmakereg

dmreadpar

dmtype2split

hrc_build_badpix

hrc_process_events

reproject_events

srcextent

tg_resolve_events

tg_create_mask

tg_resolve_events

tgdetect


Parameter Files

A summary of parameter files changes is provided in this section. Refer to the Tools and Analysis Scripts sections of these release notes for complete details.

We recommend deleting all the old parameter files or renaming the parameter-file directory before running any new version of CIAO to avoid conflicts. More information is available in the FAQ.

celldetect

hrc_process_events

srcextent

tgdetect

tg_create_mask

tg_resolve_events


IPython, Sherpa, and ChIPS

The prompt for Sherpa and ChIPS has changed from sherpa-n> and chips-n> to sherpa In [n]: and chips In [n]:, and there is now an "output" prompt (which is much closer to the default IPython settings). The prompts are now also colored (green for input and red for output). The on-line documentation has not been updated to this new scheme (where the prompt is normally sherpa> or >>>).

The start-up banner for both applications now includes the Python and IPython version numbers.

The following section shows an example of both changes for Sherpa:

unix% sherpa
-----------------------------------------------------
Welcome to Sherpa: CXC's Modeling and Fitting Package
-----------------------------------------------------
CIAO 4.11 Sherpa version 1 Wednesday, December 5, 2018

Python 3.5.4 (default, Oct 15 2018, 13:47:46) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.

IPython profile: sherpa

sherpa In [1]: get_method_name()
Out[1]: 'levmar'

sherpa In [2]: quit

CIAO 4.11 is the first version to include support for Jupyter notebooks. These allow people to interact with Python sessions using a web-based interface, and to share these notebooks between colleagues. It is also possible to add support for running "bash" notebooks, so that command-line CIAO applications can also easily be run in the notebook environment.

Once CIAO has been started up, the notebook server can be run with the command:

unix% jupyter notebook

and then it may open up a browser window for you automatically (depending on the operating system and configuration). If not, the screen output from the above command will include the URL to use, in a section which looks like

    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=...a rather long hexadecimal number ...

The result of going to this URL will look something like the following (the exact output depends on what files exist in the directory from which the command was run):

Starting the Jupyter notebook server

[A web page showing the "start a new notebook or chose from existing ones" page. In this example there are no existing notebooks to choose from.]
[Print media version: A web page showing the "start a new notebook or chose from existing ones" page. In this example there are no existing notebooks to choose from.]

Starting the Jupyter notebook server

Please contact the CXC Helpdesk if you have problems with the notebook support in CIAO, or need help in using it.


ChIPS

The CXC has decided to freeze support for ChIPS and move to using Matplotlib, alongside DS9, as the primary plotting and visualization systems in CIAO.

In the CIAO 4.11 release, ChIPS is still provided, and is the default plotting system for Sherpa (although Sherpa can be configured to use Matplotlib instead), as well as being used by prism and the ds9 analysis extensions (dax) systems. ChIPS will however be removed in a future release of CIAO.

Version 2.2.3 of Matplotlib is provided with CIAO, using the TkAgg back end for interactive visualizations. A conversion guide is provided for users who would like to transition from ChIPS to Matplotlib. Please contact the CXC Helpdesk if you have further questions or encounter problems.


Sherpa

Sherpa is developed as a community project, and is available on GitHub. Comments, fixes, and additions from the community are welcomed. There are additional releases for this version throughout the year, and can be installed either directly from GitHub or using the conda package manager, as described on the standalone Sherpa page.

Switching to Matplotlib

XSpec

Bug fixes


Graphical User Interfaces

obsvis

peg

taskmonitor


Analysis Scripts

chandra_repro

dax

fluximage

flux_obs

fullgarf

merge_obs

splitroi

ciao_contrib.runtool

sherpa_contrib.xspec.xsconvolve

convert_xspec_user_model

chips_contrib.pyplot


Python Modules

crates

history

cxcdm

region


Libraries

cxcparam

dslib

dm

transforms


Environment

Off-the-Shelf (OTS) Package Versions

Compiler versions


Documentation

Many pages have been updated to reflect the changes in this release. Several analysis threads are still under review.