Last modified: 23 March 2020

URL: https://cxc.cfa.harvard.edu/ciao/watchout.html

CIAO "Watch Out" Page

This page lists noteworthy items and issues about the CIAO release. For the full list of known issues please review the:


ChIPS withdrawn

ChIPS has been withdrawn from CIAO

The ChIPS plotting package has been withdrawn from CIAO. Plotting in sherpa and via prism now uses the Matplotlib plotting package.

The CXC has prepared a ChIPS to Matplotlib conversion guide to assist users to convert their scripts.


Changes to Chandra products in DS10.8.3 requires CIAO 4.12 or workaround

New aspect solution file

Archived Chandra data products created with ASCDSVER=10.8.3 and above now use a different aspect solution file. Users can check the version of the archive products with dmkeypar

% dmkeypar primary/acisf13819N001_evt2.fits ASCDSVER echo+
10.8.3

This new aspect solution file is incompatible with ACIS event processing with CIAO 4.11 and earlier. Users will see the following warning messages when running chandra_repro or acis_process_events

$ chandra_repro 13819 out=13819/repro
...
Running acis_process_events to reprocess the evt1.fits file...
Output from acis_process_events:
# acis_process_events (CIAO 4.11): WARNING: 'CONTENT' keyword '13819/primary/pcadf13819_000N001_asol1.fits' in infile 'ASPSOLOBI' is not valid for aspect solution file, setting file to NONE

# acis_process_events (CIAO 4.11): WARNING: Setting 'stop = none' since acaofffile is none and stop value does not equal none, chip or tdet.

# acis_process_events (CIAO 4.11): WARNING: stop != sky and pix_adj is centroid, edser or randomize, setting pix_adj to 'none'
...

The code will still create an event file, but it will be missing sky coordinates and cannot be used for data analysis.

Workaround

Users are encouraged to upgrade to CIAO 4.12 or later as there are also calibration updates that are incompatible with CIAO 4.11 and earlier. However, users who cannot update to CIAO 4.12 can edit the header of the aspect solution file to change the CONTENT keyword:

$ dmhedit pcadf13819_000N001_asol1.fits file= op=add key=CONTENT value=ASPSOL

If not already done, be sure to gunzip the file before attempting to edit it.


SAOImage DS9

Help, I am unable to create regions in ds9!

By default, ds9 will no longer create regions when it starts; that is, clicking in the ds9 window will not create a region. This is because the ds9 interface has changed in version 7.3.2, as noted above, so that it no longer starts in region editing mode.

To create or edit regions, users must now manually switch to Region mode by selecting

Edit → Region

from the menu bar or the button bar.

Users can make this the default by editing their preferences:

Edit → Preferences ...
Menus and Buttons                  (left panel)
Edit → Menu → Region               (right panel)
Save
Preserve During Load preferences

CIAO users should not enable any of the following options

Menus and Buttons → File → Menu → Preserve During Load → Pan
Menus and Buttons → File → Menu → Preserve During Load → Region

preferences. Due to the way CIAO starts ds9, having these preferences enabled leads to a half-pixel shift in regions. This shift accumulates if the region file is saved and re-loaded.


Installing & Starting CIAO

Python 3.5 support only

In CIAO 4.12 only Python 3.5 is provided. Support for Python 2.7 in the scientific software ecosystem is diminishing.

The Python 3 for Scientists website provides guidance on updating code written for Python 2, and the CXC Helpdesk can provide support.

Users of the beta-conda installation can select python 3.5, 3.6, or 3.7.

CIAO on Mac needs recent XQuartz and XCode

OSX users, particularly those that have upgraded from an earlier OSX version, should ensure that compatible versions of XQuartz (at least version 2.7.11) and XCode (also comes as part of Apple Developer Tools) are installed on the system before proceeding with the CIAO installation.

bash vs. zsh

Many macOSX Catalina (10.15) users will use the new default zsh shell when they run the Terminal application. zsh users should follow the same instructions as bash shell users. This includes the same setup script:

zsh$ source /soft/ciao-4.12/bin/ciao.bash
macOSX Catalina: ls: Operation not permitted

In macOSX Catalina, Apple has restricted access to various folders in the user's home directory including Documents, Desktop, and Downloads. By default the Terminal application and Xquart's xterm application do not have access to these folders. Attempting to list the contents from the command line yeilds the following error message:

$ cd Desktop
$ ls
ls: Operation not permitted

This restriction may impede CIAO users who commonly use these folders.

Users can still navigate and display the contents of these folders using the Finder application. CIAO users who want to use these restricted folders need to grant the Terminal and|or xterm applications "Full Disk Access". The setting can be found in the "Security" settings application.

No support for 32-bit Linux, nor OSX 10.11 (El Capitan) and earlier.

CIAO is no longer available for 32-bit Linux, nor is it available for OSX 10.11 (El Capitan) or earlier versions of OSX. CIAO 4.7 is still available for users who are unable to upgrade their system. The platform support page describes the operating systems that can be used with CIAO 4.12.

Fontconfig error: Cannot load default config file

During the smoke tests some users will see the following message during the obsvis test

Fontconfig error: Cannot load default config file

This error is not critical. As long as the test reports a PASS, then users can ignore that message.

Parameter file conflicts with HEAsoft

Both CIAO and FTOOLS use parameter files to store parameter values for tools, and provide tools for manipulating these files (e.g. plist, pset, punlearn; they have the same name in both systems). The location of the parameter files is taken from the $PFILES environment variables for CIAO and FTOOLS (or $UPARM if $PFILES does not exist).

Common errors are:

  • pget: could not get parameter "foo"
    
  • The following parameter:
        "foo,b,l,no,,,"Keyword""
    has the following error(s):
      o Mode field "l" is invalid.
    
  • The following parameter:
    "foo,i,h,INDEF,INDEF,INDEF,"description""
    has the following error(s):
    o Problem converting minimum field "INDEF" to parameter type "i".
    o Problem converting maximum field "INDEF" to parameter type "i".
    

We recommend that you start FTOOLS before CIAO to minimize conflicts; more information on this problem is available in the CIAO and FTOOLS section of the Starting CIAO thread. Also refer to the related FAQ "The tool is failing because ")sval" has somehow shown up in the parameter value.".

Conflicts with Other Software Packages

There are some known conflicts between CIAO and other software or system libraries and tools. Issues not covered in this "Watch Out" page are listed on the "Other Software Packages" bug page.


Analysis

Incorrect GTIs when time-filtering grating data with dmcopy opt=all

When a time filter is applied to a level=1.5 (evt1a.fits) or level=2 (evt2.fits) grating file and the dmcopy opt=all option is used, the GTI block in the output file is not updated to reflect the correct time range. Time-related header keyword values - such as ontime, livetime, and exposure - will be incorrect because they are calculated from the time ranges in the GTI block.

This bug affects all ACIS and HRC grating data analysis. Read the bug report for details and a workaround.

Specifying Aspect Solution Files

The aspect solution files (pcad..._asol1.fits) are used as inputs to several of the CIAO tools - e.g. acis_process_events, asphist, and tg_resolve_events.

When there is a single aspect solution file, it can be provided directly to the tool:

unix% pset asphist infile=pcadf245733620N002_asol1.fits

In rare cases, there will be more than one aspect solution file for an observation. Then all the files must be provided in chronological order. (The time is in the filename, so "ls" lists them in chronological order.) This may be done as a comma-separated list:

unix% pset asphist infile="pcadf063874624N002_asol1.fits,pcadf063875522N002_asol1.fits,pcadf063902942N002_asol1.fits"

The list of aspect solution files may also be given as a stack. Any number of files, including just one, can be specified in the stack:

unix% cat pcad_asol1.lis
pcadf245733620N002_asol1.fits

unix% pset asphist infile=@pcad_asol1.lis

or more than one:

unix% cat pcad_asol1.lis
pcadf063874624N002_asol1.fits
pcadf063875522N002_asol1.fits
pcadf063902942N002_asol1.fits

unix% pset asphist infile=@pcad_asol1.lis

Note that an at symbol ("@") is used before the filename to indicate that it is a stack file.


Python

Installing 3rd party modules with pip3

Users must be careful when using pip3 to install 3rd party modules such as Astropy and Scipy into CIAO. The latest version of these packages MAY require updated versions of packages that CIAO needs; numpy being the most likely conflict.

pip3 installing source code packages

Many of the python packages available via pip are either pure python or are precompiled binary distributions. However, some packages are provided as source code and will need to compile against the libraries on the users system. Users may experience problems building these packages due to some remaining hard-coded paths in some configuration files.