Skip to the navigation links
Search the CALDB website
Last modified: 16 October 2024

URL: https://cxc.cfa.harvard.edu/caldb/downloads/index.html

Downloading CALDB 4.11.5

CalDB 4.11.5 released 16 OCTOBER 2024

[NOTE]
CalDB 4.11.5 is only compatible with CIAO 4.16.0 or later versions! CIAO 4.14-4.15 may use CalDB versions 4.9.0 through 4.10.7. An error in acis_process_events will occur with CalDB 4.11.5 using CIAO 4.14 or 4.15, due to the recent ACIS CTI N0012 files format changes.

Contents:

Determining your current CalDB4 version

The easiest way to determine which version of the Chandra CalDB you currently have installed, is to use the CIAO 4.16 command check_ciao_caldb as follows: 

unix% check_ciao_caldb

Naturally you will need to have loaded CIAO 4.16 in order do do this.

If your version is 4.11.0 or later, you may proceed with the upgrade installation procedure (next instruction).

If the version you get is NOT 4.11.0 or later version, then this upgrade procedure is inappropriate! Do not attempt to upgrade a CalDB of any other version with this upgrade procedure as it will not work. You have some options:

  1. If you are not aware of the CIAO 4.16 release: Please consider upgrading your CIAO software to CIAO 4.16, using ciao-install or Conda, which will also upgrade your CalDB to 4.11.5.

  2. If you are in the process of installing CIAO 4.16 for the first time, manually, you don't want the CalDB upgrade file installation; You need the whole CalDB 4.11.5. Please proceed to the full CalDB 4.11.5 installation instructions below.

Upgrade CalDB 4.11.0 or later version to 4.11.5

If your site has already installed CalDB 4.11.0 or later version, you may upgrade to 4.11.5 by using the "upgrade" tar file in Table 2 of the download page. Follow instructions 1-6 in this section to upgrade your CalDB4 installation.

  1. Change directories to the existing CALDB4 location.

  2. Download the CalDB 4.11.5 "upgrade" gzipped tar file.

  3. Download the recently-upgraded ACIS BKGRND 4.9.7 gzipped tar file, if it has not already been installed in your CalDB area.

  4. Download the recently-upgraded HRC BKGRND 4.7.7 gzipped tar file, if it has not already been installed in your CalDB area.

  5. UNIX: Unpack the gzipped tar ball(s) using gtar xvUfz. The "U" option forces gtar appropriately to reset the links to the index files and your upgrade will be properly installed.

    LINUX: You may omit the U option: gtar xvfz, or else you may get some meaningless error messages about failure to unlink non-empty directories. These errors are harmless.

    If you do not have gtar installed locally, you may use tar xvfz instead. 

    unix% gtar xvUfz caldb_4.11.5_upgrade.tar.gz 
unix% gtar xvUfz acis_bkgrnd_4.9.7.tar.gz
unix% gtar xvUfz hrc_bkgrnd_4.7.7.tar.gz

  6. UPGRADE FINISHED! Proceed to Check the index configuration below.

Install the FULL CALDB 4.11.5

The instructions for installing the CALDB are basically the same whether you are creating a local or sitewide installation.

  1. Select a location for your CALDB at your site. This directory needs to be accessible by all users who are running CIAO 4.16 at your site.

  2. Download CALDB 4.11.5 Full Installation from Table 1. There are three tar ball files in all, including the "main" tar ball.

    The ACIS and HRC Background files have been packaged separately to reduce the space requirements for CalDB when users don't need the background datasets. They can be downloaded rather quickly and unpacked AFTER the MAIN 4.11.5 tarball has been unpacked. We clearly recommend downloading and installing these files on multi-user sites.

  3. From within the CALDB directory, unpack the main tar ball using gtar xvUfz. 

    unix% gtar xvUfz caldb_4.11.5_main.tar.gz

    UNIX: If you use gtar, then you should use the options xvUf (include the "U" option) in case there are filename conflicts which may not be appropriately clobbered. NOTE: In LINUX, the "U" option may cause meaningless errors about failure to unlink directories that are not empty. Ignore this error, or else omit the "U" option. If you do not have access to gtar then you may use tar xvf instead without worry.

    The result will be three directory trees at that level, namely docs, data, and software: 

    unix% ls
data/           docs/           software/

    The docs directory contains the manifests and versioning files. while software/ contains the tools. The data directory is most important, containing the full CALDB data directory tree.

  4. Unzip and untar the ACIS background package, unless you are certain that you will not need any of the blank-field files for making background datasets. Typically observers of extended sources need these files. We recommend multi-user sites to download and install these files in case anyone in your group needs them. NOTE: By unzipping the tarball in the SAME LOCATION AS THE MAIN TARBALL, you get the bkgrnd files exactly where you will want them. 

    unix% gtar xvfz acis_bkgrnd_4.9.7.tar.gz

  5. Unzip and untar the HRC background package, unless you are certain that you will not need any of the HRC-I stowed background files for making background datasets. Typically observers of extended sources need these files. We recommend multi-user sites to download and install these files in case anyone in your group needs them. NOTE: By unzipping the tarball in the SAME LOCATION AS THE MAIN TARBALL, you get the bkgrnd files exactly where you will want them. 

    unix% gtar xvfz hrc_bkgrnd_4.7.7.tar.gz

  6. Delete the tarfiles in order to save space. You may wish to complete the installation and verify that the CALDB is working before doing so.

Check the Index Configuration

To ensure that the index files are linked correctly, run this ls command from your CALDB directory: 

unix% ls -l data/chandra/*/caldb.indx
ls -l data/chandra/*/caldb.indx
lrwxrwxrwx ... data/chandra/acis/caldb.indx -> ./index/caldbN0466.indx
lrwxrwxrwx ... data/chandra/default/caldb.indx -> ./index/caldbN0408.indx 
lrwxrwxrwx ... data/chandra/ephin/caldb.indx -> ./index/caldbN0400.indx
lrwxrwxrwx ... data/chandra/hrc/caldb.indx -> ./index/caldbN0445.indx 
lrwxrwxrwx ... data/chandra/pcad/caldb.indx -> ./index/caldbN0414.indx
lrwxrwxrwx ... data/chandra/pimms/caldb.indx -> ./index/caldbN0416.indx
lrwxrwxrwx ... data/chandra/sim/caldb.indx -> ./index/caldbN0400.indx

where "..." represents the file owner and timestamp. Check that the results from your system match those shown here.

Verify that the CALDB is Working

The first part of this step requires that you have FTOOLS available; if you do not, skip to the check_ciao_caldb example. Note that you do not need FTOOLS in order to use the CALDB with CIAO, only for this verification step.

chkcif

The FTOOL chkcif is the method for verifying the installation of a CALDB after unpacking it. chkcif ("check CALDB index file") reads the index listings and

  • verifies that the index for a given instrument is present and in the proper form.
  • verifies that all ONLINE files in the index are present in the CALDB. If a file that is indexed as ONLINE is missing, the tool reports it to STDOUT and then exits with an error status.
  • lists the OFFLINE files as offline and does not check for them.

A file that is marked OFFLINE is not included in the CalDB directory structure. By the HEASARC standard, such a file might be available in an off-line system such as another disk drive or a web location, and isn't accessible by chkcif. For the Chandra CalDB, "OFFLINE" indicates that a file has been removed from circulation, and is NO LONGER AVAILABLE, and should not be used. Hence, when chkcif returns that a file is OFFLINE, this is normal behavior for the CalDB, and does not indicate any problem with your configuration.

Note that the tool does not verify that each calibration file is whole or uncorrupted, only that it exists in the proper location in the directory structure.

Note that you need to start CIAO in the working window (to define some environment variables) before running chkcif. In this example, CIAO is installed in /soft/ciao: 

unix% source /soft/ciao/bin/ciao.csh
unix% chkcif chandra acis STDOUT
 ** chkcif 1.1.4
 ... Contents of CIF /soft/ciao/CALDB/data/chandra/acis/caldb.indx verified
 ... See STDOUT
     for more information
 ** chkcif 1.1.4 completed successfully

This command needs to be repeated for all the Chandra CALDB instrument branches, namely "hrc", "ephin", "pcad", "sim", and "default".

IMPORTANT: IF the ACIS background files have not been downloaded and unpacked, then the ACIS branch will report errors, as shown in this example. Otherwise, the tool should return the "chkcif ... completed successfully" line for each instrument, verifying that the CALDB is correctly installed. The same holds for the HRC branch if the HRC BKGRND files are not downloaded.

check_ciao_caldb

It is also possible to test the installation using the script "check_ciao_caldb". This routine will locate the CalDB used by your CIAO 4.1.x+ installation, re your current CalDB version, and indicate whether or not a standard test query to the CalDB has worked correctly.

First, start CIAO in the working window. 

unix% check_ciao_caldb

CALDB environment variable = /data/caldb
             CALDB version = 4.11.5
             release date = 2024-10-16T12:00:00 UTC
CALDB query completed successfully.

If the routine does not give you a response equivalent to the above for your machine and directory setup, you may need to reinstall your CalDB, following the Full Installation instructions above.

The tool supports the --latest flag which checks the CALDB version history page to see if your CALDB install is up to date. If it is you will see an extra line in the output: 

unix% check_ciao_caldb --latest
CALDB environment variable = /data/caldb
             CALDB version = 4.11.5
             release date  = 2024-10-16T12:00:00 UTC
CALDB query completed successfully.
The CALDB installation is up to date.

whereas if a newer version is available the tool will fail as shown below. 

unix% check_ciao_caldb --latest
CALDB environment variable = /data/caldb
             CALDB version = 4.7.4
             release date  = 2017-05-04T17:00:00 UTC
CALDB query completed successfully.
# check_ciao_caldb (16 OCTOBER 2024): ERROR The latest available CALDB version is 4.11.5