Last modified: 27 Jan 2023

URL: https://cxc.cfa.harvard.edu/ciao/threads/ciao_install/

Manually Installing CIAO 4.15.2

CIAO 4.15 Science Threads


Overview

Synopsis:

How to manually install version 4.15.2 of CIAO, the Chandra Interactive Analysis of Observations software package.

This thread does not explain how to re-compile CIAO yourself. Refer to the Linux source build or Mac source build instructions for information on that process.

Related Links:

Last Update: 27 Jan 2023 - Updated for CIAO 4.15.1 patch.


Contents


System Requirements

CIAO is supported on a number of operating systems, which are listed on the Platform Support page. Certain system requirements are necessary to successfully install and run CIAO; review these system requirements before continuing with the installation.


Installation Issues

Users should be aware of these installation items before installing CIAO 4.15.2. Additional problems which are seen less frequently are listed on the Installation & Smoke Tests bug page.

Remove old parameter files

With every new CIAO release, some parameter files have changed: new parameters may be added and occasionally old ones removed or renamed. Deleting or renaming the local parameter directory ensures that the correct parameter files will be accessed the first time a tool is run:

unix% rm ~/cxcds_param4/*

Downloading the Software

For a standard CIAO installation, you need the following packages:

  • Binaries: ciao-4.15-bin-<component>-<OS>.tar.gz

    Contains the CIAO tools, libraries, parameter files, and help files. Also contains the pre-compiled off-the-shelf (OTS) programs required to run CIAO.

    The relationship between <OS> and the supported operating systems is given in the following table:

    <OS> Operating system
    Linux all 64-bit Linux w/Python 3.9
    macOS 64-bit macOS 10.15 (Catalina through Ventura) w/Python 3.9
    [IMPORTANT]
    32bit Linux and OSX 10.6.8-10.14: End of Life

    CIAO is no longer available for 32bit Linux nor for OSX 10.14 (Mojave) and earlier. The platform support page describes the operating systems that can be used with CIAO.

  • Chandra Calibration Database (CALDB) v4.10.7

    Some calibration products are packaged separately, due to the larger file size and specific analysis uses for them. Users may opt to include the ACIS background files and/or HRC background files when installing the CALDB. This thread includes only the ACIS background tarfile.

  • Scripts and Modules Package ciao-4.15-contrib-3.tar.gz

The files are retrieved with wget. In this example, the 64-bit Linux files (<OS>="Linux") are retrieved; change the <OS> value to match your operating system. Save the tarfiles to a location - /soft/download/ in this example - that is accessible from where you wish to install CIAO.

unix% mkdir /soft/download/ 
unix% cd /soft/download
unix% wget https://cxc.cfa.harvard.edu/ciao-install/ciao-4.15/Linux/ciao-4.15-bin-core-Linux.tar.gz
unix% wget https://cxc.cfa.harvard.edu/ciao-install/ciao-4.15/Linux/ciao-4.15-bin-obsvis-Linux.tar.gz
unix% wget https://cxc.cfa.harvard.edu/ciao-install/ciao-4.15/Linux/ciao-4.15-bin-sherpa-Linux.tar.gz
unix% wget https://cxc.cfa.harvard.edu/ciao-install/ciao-4.15/Linux/ciao-4.15-bin-tools-Linux.tar.gz
unix% wget https://cxc.cfa.harvard.edu/ciao-install/ciao-4.15/all/ciao-4.15-contrib-3.tar.gz

unix% wget https://cxc.cfa.harvard.edu/ciao-install/ciao-4.15/Linux/ciao-4.15.1-patch-bin-core-Linux.tar.gz
unix% wget https://cxc.cfa.harvard.edu/ciao-install/ciao-4.15/Linux/ciao-4.15.1-patch-bin-sherpa-Linux.tar.gz


unix% wget https://cxc.cfa.harvard.edu/cdaftp/arcftp/caldb/caldb_4.10.7_main.tar.gz

unix% wget https://cxc.cfa.harvard.edu/cdaftp/arcftp/caldb/acis_bkgrnd_4.9.7.tar.gz

The following files were used to install CIAO on a 64-bit Linux (CentOS) machine:

unix% ls -1 /soft/download/
acis_bkgrnd_4.9.7.tar.gz
caldb_4.10.7_main.tar.gz
ciao-4.15-bin-core-Linux.tar.gz
ciao-4.15-bin-obsvis-Linux.tar.gz
ciao-4.15-bin-sherpa-Linux.tar.gz
ciao-4.15-bin-tools-Linux.tar.gz
ciao-4.15-contrib-3.tar.gz

Where should CIAO and CALDB be installed?

The CIAO software package and the Chandra Calibration Database (CALDB) can be installed in any location and do not need to be installed with root or super-user privileges.

CIAO

CIAO unpacks into a directory called ciao-<version>/, i.e. ciao-4.15 for this release. If the current version of CIAO were to be installed into /usr/local/, the full path to CIAO would be /usr/local/ciao-4.15/. This directory (/usr/local/ciao-4.15/) is referred to as the root directory of CIAO in the rest of this thread. It is also the value that the $ASCDS_INSTALL environment variable is set to once CIAO has been started.

In the rest of this thread we will assume CIAO is being installed into the /soft/ directory.


CALDB

CALDB may be installed in two ways:

  • within the CIAO installation, in the directory $ASCDS_INSTALL/CALDB
  • in a separate directory, in which case $ASCDS_INSTALL/CALDB will be a soft link to the actual CALDB location.

This thread assumes that you have installed the CALDB before starting the CIAO installation, as CIAO has to be linked to the location of those files. Installation instructions are available on the CALDB website:


Installing CIAO as the root user

It is is strongly suggested that CIAO be installed as a non-privileged, regular user and not as the root user. If CIAO must be installed in a directory owned by root and you are following the manual process, then

  1. create the ciao-4.15/ directory in the desired location, as the root user;
  2. use the chown utility to change ownership of this directory to the installer's username;
  3. unpack CIAO into the newly-created directory as the installer, not as the root user.

Installing the CIAO Software

1. Change to the CIAO installation directory

The CIAO 4.15.2 tarfiles create a directory named ciao-4.15 for the software installation. It is not necessary to create the CIAO directory before starting the installation.

Move to the directory in which you would like the top level of the CIAO installation to be located. If you are installing the software for a local network, this directory must also be visible to all potential CIAO users.

This thread will use the /soft directory:

unix% cd /soft/

2. Unpack CIAO 4.15.2

From the CIAO location just chosen, untar the CIAO 4.15.2 binary packages. (If you prefer more screen output, add the "v" option to the gtar command.)

unix% pwd
/soft

unix% gtar xzf /soft/download/ciao-4.15-bin-core-Linux.tar.gz

unix% gtar xzf /soft/download/ciao-4.15-bin-obsvis-Linux.tar.gz

unix% gtar xzf /soft/download/ciao-4.15-bin-sherpa-Linux.tar.gz

unix% gtar xzf /soft/download/ciao-4.15-bin-tools-Linux.tar.gz

When all the files are unpacked, the CIAO directory will look like:

unix% ls -F ciao-4.15
COPYING*        README.obsvis     VERSION_sherpa  config.sub*    lib/
COPYING.LIB*    VERSION           VERSION_tools   configure*     ots/
INSTALL_BINARY  VERSION.prop_tk   bin/            configure.in*  param/
INSTALL_SOURCE                    binexe/         contrib/       share/
LICENSE         VERSION_core      ciao-type       data/          test/
LICENSE.SAO     VERSION_graphics  config/         doc/
Makefile        VERSION_obsvis    config.guess*   include/
README          VERSION_prism     config.h.in     install-sh*

3. Scripts & Modules Package

The science scripts are written and maintained by local CIAO users at the CXC. More information on each of the scripts is available from the CIAO 4.15.2 Scripts Page. Check that page for announcements about new scripts and updates to existing ones.

The tarfile is designed to be unpacked from the same directory as the CIAO software.

unix% pwd
/soft/

Unpack the script tarfile:

unix% tar xzvf /soft/download/ciao-4.15-contrib-3.tar.gz
./ciao-4.15/
./ciao-4.15/contrib/
./ciao-4.15/contrib/Changes.CIAO_scripts
./ciao-4.15/contrib/README_CIAO_scripts
./ciao-4.15/contrib/VERSION.CIAO_scripts
./ciao-4.15/contrib/bin/
./ciao-4.15/contrib/bin/acis_bkgrnd_lookup

...

./ciao-4.15/contrib/param/merge_obs.par
./ciao-4.15/contrib/param/specextract.par

The tarfile contains a VERSION.CIAO_scripts file. Compare the timestamp in this file to the "Last Updated" date on the Scripts page to determine when you need to update your local scripts installation.


4. Upack CIAO 4.15.2 Patch

CIAO 4.15.2 is a patch on top of the CIAO 4.15 package and must be unpacked after the base 4.15 files are unpacked. The tarfiles are designed to be unpacked from the same directory as the CIAO software. The contents of the patch tar files will overwrite the existing files.

unix% gunzip -c /soft/download/ciao-4.15.1-patch-bin-core-Linux.tar.gz | tar xf - 
unix% gunzip -c /soft/download/ciao-4.15.1-patch-bin-sherpa-Linux.tar.gz | tar xf - 

5. Run the configuration script

Run the CIAO configuration script:

unix% cd ciao-4.15/

unix% ./configure
Running CXC configure, v4.4.1
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for more... /bin/more
checking for main in -lpthread... no
  --with-imagerdir not specified, defaulting to /soft/ciao-4.15/ots/bin
configure: creating ./config.status
config.status: creating bin/ciao_setup.csh
config.status: creating bin/ciao_setup.sh
config.status: creating ots/lib/python3.9/config-3.9m/Makefile
config.status: creating bin/ciao.sh
config.status: creating bin/ciao.csh
config.status: creating bin/ciao.ksh
config.status: creating bin/ciao.bash
config.status: creating bin/setup_obsvis.sh
config.status: creating bin/setup_obsvis.csh
config.status: creating bin/setup_chips.sh
config.status: creating bin/setup_chips.csh
config.status: creating bin/setup_sherpa.sh
config.status: creating bin/setup_sherpa.csh
config.status: creating bin/setup_tools.sh
config.status: creating bin/setup_tools.csh
config.status: creating bin/ciao-python-fix
config.status: creating test/smoke/bin/setup.sh
config.status: creating test/Makefile

6. Run the ciao-python-fix script

Next, the ciao-python-fix script needs to be run. Despite it's name, it does more than just stuff related to python. Its runs a program that will set the location of various CIAO and off-the-shelf (OTS) libraries.

unix% bash bin/ciao-python-fix

Once this script is run, the CIAO installation cannot be moved or relocated to a different directory without repeating the configure and ciao-python-fix steps.


Create a link to the calibration data, changing the path to match your system installation:

unix% pwd
/soft/ciao-4.15/

unix% ln -s /soft/CALDB CALDB

At this point, your directory should look like:

unix% ls -F
CALDB@          README           binexe/         config.sub*    lib/
COPYING*        README.obsvis    ciao-type       configure*     ots/
COPYING.LIB*    VERSION          ciao_installed  configure.in*  param/
INSTALL_BINARY  VERSION.prop_tk  config/         contrib/       share/
INSTALL_SOURCE  VERSION_core     config.guess*   data/          test/
LICENSE         VERSION_sherpa   config.h.in     doc/
LICENSE.SAO     VERSION_tools    config.log      include/
Makefile        bin/             config.status*  install-sh*

8. Create a CIAO alias

It is strongly suggested that you create a CIAO alias before continuing.

The alias syntax depends on the shell (tcsh, csh, bash); see this FAQ for help in determining which shell you are using.

  • csh/tcsh users should add the following to their $HOME/.cshrc file.
    alias ciao "source /soft/ciao-4.15/bin/ciao.csh"
    
  • bash users should add the following to their $HOME/.bashrc file.
    alias ciao="source /soft/ciao-4.15/bin/ciao.sh"
    
  • ksh users should add the following to their $HOME/.login file.
    alias ciao="source /soft/ciao-4.15/bin/ciao.sh"
    

Users can create the file if one does not already exist. These are simply text files that can be edited with the users favorite text editor (emacs, vi, gedit, etc.).

More information is given in the Running CIAO section of the Starting CIAO thread.

Once the alias is defined, you will be able to simply type

unix% ciao

and CIAO will be setup and ready to use in that window.


Create the Help File Indexes

In order for CIAO to be able to access the help files packaged with the software, it is necessary to create the ahelp database indexes. This step must be run by the user who owns the CIAO installation.

Start CIAO and run ahelp -r:

unix% ciao
CIAO configuration is complete... 
CIAO 4.15 Monday, November 21, 2022
  bindir      : /soft/ciao-4.15/bin
  CALDB       : 4.10.7

unix% ahelp -r

Processing file '/soft/ciao-4.15/doc/xml/dmcopy.xml'...OK
....
Processed Files  : 1117
Succeeded Files  : 1117
Failed Files     : 0
Purged Duplicates: 0

Creating Key Indexes...DONE
Subject DB Size: 1117

Creating Context Indexes...DONE
Context DB Size: 1117

Creating Crossreference DB...DONE
Crossreferences DB Size: 18181

Creating Seealso  DB...DONE
Seealso DB Size: 3914

Run the Smoke Tests

A set of smoke tests are packaged with CIAO 4.15.2. The tests are designed to spot-check the installation for critical problems, not to run a complete regression test of the system. The output created by the new installation of CIAO is compared against a baseline set of files included in the package.

The smoke tests include tests of a number of the GUI applications in CIAO, windows will appear and disappear during these tests. This can make using your computer difficult, since these windows may take the mouse focus or overlap your working windows.

unix% cd $ASCDS_INSTALL/test
unix% pwd
/soft/ciao-4.15/test

The smoke test syntax depends on the shell (tcsh, csh, bash); see this FAQ for help in determining which shell you are using.

  • csh/tcsh users:
    unix% make -k test |& tee ~/ciao_test.log
    
  • bash or ksh users:
    unix% make -k test 2>&1 | tee ~/ciao_test.log
    

The screen output provides a count of how many tests have been run:

unix% make -k test |& tee ~/ciao_test.log

 Requested test suite includes 40 tests
   [1/40] Running test prism-smoke001 ..       
   [2/40] Running test prism-smoke002 ... PASS
   [3/40] Running test ahelp-smoke001 ... PASS
   [4/40] Running test ds9-smoke001 ... PASS
...
   [38/40] Running test tools-dmstat-smoke001 ... PASS
   [39/40] Running test tools-dmstat-smoke002 ... PASS
   [40/40] Running test tools-dmstat-smoke003 ... PASS
 All smoke tests PASSED.

Verifying test successes

A successful test is logged with a "... PASS", while an unsuccessful test reports:

   [7/45] Running test <testname> ... FAIL
         Review /tmp/smoke.username/<testname>/diff.log for details

The smoke tests print a summary of successful and failed tests. If all tests were successful:

All smoke tests PASSED.

or if any failed:

 1 out of 45 smoke tests FAILED.

       !! CIAO MAY HAVE FAILED TO INSTALL PROPERLY !!
 Please check http://cxc.harvard.edu/ciao/bugs/smoke.html
 for a list of known issues before contacting the Help Desk.

Run individual smoke tests

To run (or re-run) an individual smoke test you have to use the runtests Makefile target as shown here

unix% cd $ASCDS_INSTALL/test
unix% make test-PACKAGE runtests

Where PACKAGE can be: ahelp, crates, dm, ds9, obsvis, sherpa, tools, or ui. For example to just re-run the sherpa smoke tests, run the following command:

unix% cd $ASCDS_INSTALL/test
unix% make test-sherpa runtests
 Requested test suite includes 1 test
   [1/1] Running test sherpa-smoke001 ...  PASS

All smoke tests PASSED.


Known Smoke Test Issues

There is a bug page for known smoke test issues. If you experience a problem that isn't listed there, submit it to CXC Helpdesk.


Test the CALDB installation

The scripts package contains a tool - check_ciao_caldb - that performs a simple check of the Chandra CALDB installation. Running the tool should produce screen output like the following:

unix% check_ciao_caldb
CALDB environment variable = /soft/ciao-4.15/CALDB
             CALDB version = 4.10.7
             release date  = 2021-04-23T19:30:00 UTC
CALDB query completed successfully.

For a more-thorough validation please follow the chkcif steps described on the download CALDB page.


Clean Up the Tarfiles

Finally, you may delete the tarfiles from the original download directory:

unix% rm /soft/download/ciao*tar.gz

Notes

Known Conflicts

There are some known conflicts between CIAO and other software or system libraries and tools. They are documented in the following webpages:


Summary

The CIAO installation is now complete. For help getting started with the software, read the Introductory science threads.


History

12 Aug 2005 original version, new for CIAO 3.2.2
14 Nov 2005 updated for CIAO 3.3: added Linux8 package: renaming libgcc_s.so.1 and Mac OS X 10.2: Using ds9 v3.0.3 notes.
12 Jan 2006 updated to include CIAO 3.3.0.1 patch instructions
01 Dec 2006 updated for CIAO 3.4 and CALDB 3.3.0
29 Jan 2007 updated to include CIAO 3.4.1[.1] patch instructions
14 Dec 2007 updated for CIAO 4.0: tarfile creates a directory into which it unpacks; CIAO is shipped with gzip support enabled (no longer need to specify an option in the configure step); smoke tests are included in the AUX tarfile; CALDB 3.4.2 is used
21 Feb 2008 updated to include CIAO 4.0.1 patch instructions
26 Feb 2008 consistently use "/soft" as the installation directory
05 Mar 2008 renamed "Reindexing the Help Files" section to "Create the Help File Indices"
10 Mar 2008 system requirements moved to a separate page; "Create the Help File Indices" should be "Create the Help File Indexes"; CIAO does not need to be installed by the root user; "ahelp -r" must be run as user who owns installation; Mac users need to add xrdb to the PATH; FC7 expected smoke test failure
16 Apr 2008 updated to include CIAO 4.0.2 patch instructions; location of CIAO alias updated for bash users
08 Sep 2008 corrected ChIPS test number in Fedora Core 7 smoke test error (test 3 prints the error, not test 5)
15 Dec 2008 updated for CIAO 4.1: CALDB 4.1.0 is used, new supported platform (FC8 instead of FC7); added Known Smoke Test Issues section; removed the "setting xrdb path" section (needed for .CXCdefaults file, which the GUIs no longer use)
15 Jan 2009 updated to include CIAO 4.1.1 patch instructions; include full alias syntax in CIAO alias section; added bash syntax in Run the Smoke Tests section
29 Jan 2009 added "Installing CIAO as root user" information to Choose a CIAO installation directory section
20 Apr 2009 updated to include CIAO 4.1.2 patch instructions; CALDB version updated to 4.1.2; added the test the CALDB installation step; there is a new tool - ciao-install - that automates the installation of CIAO and CALDB
15 Dec 2009 updated for the CIAO 4.2 release: CALDB 4.2, new supported platforms, updates to filenames, "unzip the files" step replaced by using the "z" option to gtar. The scripts package unpacks from the same level as CIAO software.
30 Dec 2009 Added Snow Leopard workaround
22 Feb 2010 tarfiles are downloaded with wget instead of ciao-install
16 Mar 2010 Added Downloading the CALDB: FTP passive mode note
05 Apr 2010 updated for CALDB 4.2.1: version number of CALDB main and ACIS background tarfiles
19 Apr 2010 updated for CALDB 4.2.2: version number of CALDB main and ACIS background tarfiles
27 Sep 2010 updated for CALDB 4.3.1: version number of CALDB main
15 Dec 2010 updated for the CIAO 4.3 and CALDB 4.4.1 releases: minor changes to screen output
15 Dec 2011 reviewed for the CIAO 4.4 and CALDB 4.4.7 releases: minor changes to screen output
15 Jun 2012 reviewed for the CIAO 4.4.1 and CALDB 4.4.10 releases: minor changes to screen output
31 Jul 2012 updated to use CALDB 4.5.1
15 Oct 2012 updated to use CALDB 4.5.3
03 Dec 2012 Review for CIAO 4.5; updated INSTALL_SOURCE and _BINARY;
05 Mar 2013 updated to use CALDB 4.5.6
11 Jun 2013 updated to use CALDB 4.5.7 and contrib 3
30 Jul 2013 updated to use contrib 4
19 Aug 2013 updated to use CALDB 4.5.8
18 Oct 2013 updated to use contrib 5
25 Nov 2013 Reviewed for CIAO 4.6. Added new section for ciao-python-fix script.
26 Mar 2014 updated for CalDB 4.6.1, Sherpa patch, and contrib 4.6.3
09 Apr 2014 updated for CalDB 4.6.1.1, Sherpa patch, and contrib 4.6.4
16 Jun 2014 updated to use contrib 4.6.5
09 Jul 2014 updated to use CALDB 4.6.2
29 Aug 2014 updated to use CALDB 4.6.3
24 Sep 2014 updated to use contrib 4.6.6
04 Nov 2014 updated to use contrib 4.6.7
11 Dec 2014 Updated for CIAO 4.7. Added information about manually running an individual smoke test.
23 Feb 2015 updated to use CALDB 4.6.7
07 Apr 2015 updated to use contrib 4.7.2
01 Jul 2015 updated to use CALDB 4.6.8 and contrib 4.7.3
23 Sep 2015 updated to use CALDB 4.6.9 and contrib 4.7.4
15 Dec 2015 Updated for CIAO 4.8
27 Jan 2016 updated to use contrib 4.8.2
09 Feb 2016 updated to CIAO 4.8.1 patch
29 Feb 2016 updated to note about the CIAO 4.8.2 patch for OSX El Capitan.
25 Mar 2016 updated to use CALDB 4.7.1
13 Apr 2016 updated to use contrib 4.8.3
27 May 2016 updated to use CALDB 4.7.2
03 Oct 2016 updated to use contrib 4.8.4
14 Dec 2016 updated for CIAO 4.9, CALDB 4.7.3, and contrib 4.9.1.
12 Apr 2017 updated to use contrib 4.9.2
04 May 2017 updated to use CALDB 4.7.4
24 May 2017 updated to use contrib 4.9.3
20 Jul 2017 updated to use CALDB 4.7.5.1 and contrib 4.9.4
18 Aug 2017 updated to use CALDB 4.7.6
14 Dec 2017 updated to use CALDB 4.7.7
09 Apr 2018 Updated for CIAO 4.10.
15 May 2018 updated to use contrib 4.10.2
28 Jun 2018 updated to use CALDB 4.7.9
11 Sep 2018 updated to use CALDB 4.8.0
25 Oct 2018 updated to use CALDB 4.8.1
07 Nov 2018 updated to use contrib 4.10.3
20 Feb 2019 Misc. cleanup.
29 Apr 2019 updated to use contrib 4.11.2
23 May 2019 updated to use contrib 4.11.3 and CALDB 4.8.3
07 Aug 2019 updated to use contrib 4.11.4
03 Sep 2019 updated to use CALDB 4.8.4
11 Dec 2019 CIAO 4.12 updates.
24 Feb 2020 Updated to use https://cxc locations for CIAO-4.12 files.
10 Apr 2020 updated to use CALDB 4.9.1 and contrib 4.12.2
16 Dec 2020 updated for CIAO 4.13, caldb 4.9.4, contrib 4.13.0
29 Apr 2021 updated for CalDB 4.9.5 and contrib 4.13.1
22 Nov 2021 Updated for CIAO 4.14.
24 Mar 2022 Updated for CalDB 4.9.7.
15 Nov 2022 Updated for CalDB 4.10.2 and contrib 4.14.4.
27 Jan 2023 Updated for CIAO 4.15.1 patch.