Last modified: 30 Nov 2023

URL: https://cxc.cfa.harvard.edu/ciao/threads/ciao_build_linux/

Building CIAO 4.16 from source

CIAO 4.16 Science Threads


Overview

Synopsis:

The following instructions let you build CIAO 4.16 from source on either Linux or macOS systems. CIAO 4.16 requires several libraries, or OTS (off-the-shelf) software, which are provided in the conda environment.

Related Links:

Last Update: 30 Nov 2023 - The source code for CIAO 4.16 is packaged and built completely different than before. The source code is now available via conda. The build system has changed from Makefiles to using cmake and meson. The source code has also been reorganized so that the Python code is in a separate package called pyciao. There is also now a single build script that works for both Linux and Mac, so the two separate threads have been combined into this one.


Contents


A few notes before you start

These instructions show how to build CIAO using the ciao-install edition, not the conda edition.

To prevent any old variables from affecting the CIAO build environment, it is advised to open a new terminal window before continuing.

CIAO can only be build on 64-bit Linux and macOS Intel/x86 or macOS ARM/M1/M2/M3 systems.

[TIP]
macOS requires Xcode

Remember that CIAO on macOS requires Apple's Xcode. This is especially true when trying to build from source.

Where should CIAO be installed?

CIAO can be installed in any location and does not need to be installed with any root or super-user privileges. Users are discouraged from installing CIAO as the root user.


CIAO Source Build Quickstart

This section contains a quick start guide on how to build the CIAO system. The remainder of this document provides detailed instructions and explanations of the steps to build CIAO from source.

$ bash ./ciao-install --add ciao-src --add sherpa-src \
  --cache `pwd` --prefix `pwd` 
...
$ source ciao-4.16/bin/ciao.sh     # -or- ciao.csh for tcsh shell users
$ cd ciao-4.16
$ bash ./ciao_build.sh --prefix `pwd`
...
$ bash test/smoke/bin/run_smoke_tests.sh 

Briefly described:

  1. The ciao-install script will install not only the CIAO source code but will also install all the tools necessary to compile CIAO including things like bison, flex, cmake, meson, and the supporting packages they require. It also installs all the OTS packages needed to build and run CIAO.

    $ bash ./ciao-install --add ciao-src --add sherpa-src \
      --cache `pwd` --prefix `pwd` 
    
  2. Source the CIAO setup script. This is needed to setup the PATH to the build tools (compilers, cmake, meson, etc):

    $ source ciao-4.16/bin/ciao.sh     # -or- ciao.csh for tcsh shell users
    
  3. Run the ciao_build.sh script

    $ bash ./ciao_build.sh --prefix `pwd`
    

    While it is possible to build CIAO into a separate directory by changing the --prefix, the OTS libraries and programs will not be available and will cause run-time errors.

    The --prefix must be specified and it must be an absolute, not relative, path. For example using

    --prefix ./
    

    will cause an error when building the python modules, pyciao.

    It is also highly recommended to install CIAO into the same directory. While building CIAO into a separate directory works, the setup scripts assume/requires all the OTS packages to be in the same directory as the CIAO executables.

  4. Run the CIAO smoke tests

    $ bash test/smoke/bin/run_smoke_tests.sh 
    

    A successful test will look like:

            [1/37] Running test FOO-smoke001 ... PASS
    

    where 'FOO' is the name of an individual test. If you see a message that reports:

            [1/37] Running test FOO-smoke001 ... FAIL
    

    then there is an unexpected problem with the tests. Note that if you see additional warnings/errors and the test reports PASS, then the test is good (tests sometimes do test error conditions).

    All the test outputs are written to $ASCDS_WORK_PATH/smoke.$LOGNAME where $LOGNAME is your system login name. The tests need about 60 Mb to run. The test scripts clean-up the disk space upon successful completion. Tests which fail leave their test output so you may check the results. When all of the tests PASS, the smoke test scripts should remove $ASCDS_WORK_PATH/smoke.$LOGNAME and all its sub-directories.

    Note

    $ASCDS_WORK_PATH is set in the $ASCDS_INSTALL/bin/ciao.*sh setup scripts. The default location will be $TMPDIR, if it is set, otherwise it will default to /tmp. If the $ASCDS_WORK_PATH directory does not exist or is not writeable, then users need to override the value. To override the value, simply set ASCDS_WORK_PATH to some other existing, writable directory before sourcing the CIAO setup script

    bash/ksh/zsh$ export ASCDS_WORK_PATH=$HOME/tmp ; mkdir $HOME/tmp
      -or-
    tcsh% setenv ASCDS_WORK_PATH $HOME/tmp ; mkdir $HOME/tmp  
    

    If a test FAILS, you should check the Bugs: Installation & Smoke Tests page for documented problems and solutions; if you run into any other problems, please contact the CXC Helpdesk.


Rebuild an individual directory

[TIP]
Tip

To rebuild individual directories you need to be in the build directory, not the directory where the source code is located.

To rebuild an individual directory, use the cmake --build dir command, where dir is the directory to rebuild. Assuming that users have already run the ciao-build.sh script (which configures the cmake build directory), then for example to rebuild the aconvolve tool users should

$ source ciao-4.16/bin/ciao.sh   # or ciao.csh for tcsh shell users
$ cd $ASCDS_INSTALL/ciao-src/build
$ cmake --build ciaotools/da/analysis/tcdtools/aconvolve 
[  4%] Built target region
[ 48%] Built target ascdm
[ 56%] Built target cxcparam
[ 64%] Built target NewHdr2
[ 68%] Built target err
[ 68%] Built target caldb4
[ 68%] Built target stk
[ 68%] Built target matrix
[ 68%] Built target geom
[ 68%] Built target geomcompat
[ 72%] Built target pix
[ 76%] Built target hist
[ 84%] Built target ds
[ 96%] Built target tcd
[ 96%] Built target tcdio
[100%] Built target aconvolve

Alternatively users can cd into the directory to build and then use "."

$ cd $ASCDS_INSTALL/ciao-src/build/ciaotools/da/analysis/tcdtools/aconvolve
$ cmake --build .
[  4%] Built target region
[ 48%] Built target ascdm
[ 56%] Built target cxcparam
[ 64%] Built target NewHdr2
[ 68%] Built target err
[ 68%] Built target caldb4
[ 68%] Built target stk
[ 68%] Built target matrix
[ 68%] Built target geom
[ 68%] Built target geomcompat
[ 72%] Built target pix
[ 76%] Built target hist
[ 84%] Built target ds
[ 96%] Built target tcd
[ 96%] Built target tcdio
[100%] Built target aconvolve

Use cmake --install dir to install the rebuilt code.

$ cmake --install .
-- Install configuration: "Debug"
-- Up-to-date: /soft/BuildFromSource4.16/ciao-4.16/bin/aconvolve
-- Up-to-date: /soft/BuildFromSource4.16/ciao-4.16/param/aconvolve.par
-- Up-to-date: /soft/BuildFromSource4.16/ciao-4.16/share/doc/xml/aconvolve.xml
-- Up-to-date: /soft/BuildFromSource4.16/ciao-4.16/test/bin/aconvolve.t

Enabling DEBUG build

[TIP]
Tip

The ciao_build.sh script is a wrapper that calls the individual build.sh scripts to build each of the CIAO segments: ciao-src, pyciao-src, and sherpa-src.

By default, CIAO is built without the compiler flags to enable debugging. To enable the compiler debug flags users can modify the build.sh script to add the -DCMAKE_BUILD_TYPE=Debug option. Specifically edit the build.sh file using a text editor and change this line:

cmake .. -DCMAKE_INSTALL_PREFIX=${PREFIX} -DCACHE_INIT=ON || err "Configuration failed."

to:

cmake .. -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=${PREFIX} -DCACHE_INIT=ON || err "Configuration failed."

Users can then either build all of CIAO with the debug flag set or build individual directories using the instructions above.

$ source ciao-4.16/bin/ciao.sh   # or ciao.csh for tcsh shell users
$ cd $ASCDS_INSTALL/ciao-src/build/ciaotools/da/analysis/tcdtools/aconvolve
$ cmake --build  .
[  4%] Built target region
[ 48%] Built target ascdm
[ 56%] Built target cxcparam
[ 64%] Built target NewHdr2
[ 68%] Built target err
[ 68%] Built target caldb4
[ 68%] Built target stk
[ 68%] Built target matrix
[ 68%] Built target geom
[ 68%] Built target geomcompat
[ 72%] Built target pix
[ 76%] Built target hist
[ 84%] Built target ds
[ 96%] Built target tcd
[ 96%] Built target tcdio
[100%] Built target aconvolve

$ gdb ./aconvolve
...

History

08 Dec 2014 First issue. Merges INSTALL_SOURCE text file into CIAO website.
15 Dec 2015 Updated instructions for ciao 4.8 release
07 Jun 2016 Added the fixpc command to Step 6.
28 Nov 2016 Updated instructions for ciao 4.9 release
09 Apr 2018 Updates for CIAO 4.10.
03 Apr 2018 Updates for CIAO 4.11
09 Dec 2019 Updates for CIAO 4.12
30 Nov 2021 Updates for CIAO 4.14. With GTK removed, the instructions are greatly simplified.
27 Jan 2023 Updated for CIAO 4.15.1 patch.
30 Nov 2023 The source code for CIAO 4.16 is packaged and built completely different than before. The source code is now available via conda. The build system has changed from Makefiles to using cmake and meson. The source code has also been reorganized so that the Python code is in a separate package called pyciao. There is also now a single build script that works for both Linux and Mac, so the two separate threads have been combined into this one.