Downloading CALDB 4.6.3
CalDB 4.6.3 released 29 Aug 2014
The CalDB Manager has detected an error in the HRC branch index file which causes the 2012 and 2013 HRC-I BKGRND files to be selected in reversed order. This means the for 2012 HRC-I OBS_IDs, the 2013 BKGRND file is being selected, whereas with 2013 and later OBS_IDs the 2012 BKGRND file is being selected. This affects HRC-I-related tools such as hrc_bkgrnd_lookup and fluximage.
CalDB version 4.6.3 has been released on 29 Aug 2014 to correct this problem.
- Check your CalDB4 Version
- Upgrade CALDB 4.5.9 or later to 4.6.3
- Install the full CALDB 4.6.3
- Check the Index Configuration
- Verify that the CALDB is Working
NOTE: The PSF Library has been dropped from the CalDB distribution as of the CalDB 4.6.1 release. It was announced beginning with the CIAO 4.5/CalDB 4.5.0 release that the tool mkpsf had been retired, and that CalDB would discontinue the PSFLIB one year after the version 4.5 release. That date passed in December 2013; The very next CalDB release was version 4.6.1.
The easiest way to determine which version of the Chandra CalDB you currently have installed, is to use the CIAO 4.6.1 command check_ciao_caldb as follows:
Naturally you will need to have loaded CIAO 4.6.1 in order do do this.
If your version is 4.5.9 or later, you may proceed with the upgrade installation procedure (next instruction).
If the version you get is NOT 4.5.9 or later version, then this upgrade procedure is inappropriate! Do not attempt to upgrade a CalDB of any other version with this upgrade procedure as it will not work. You have some options:
If you are not aware of the CIAO 4.6.1 release: Please consider upgrading your CIAO software to CIAO 4.6.1, using the ciao-install command, which will also upgrade your CalDB to 4.6.3.
If you are in the process of installing CIAO 4.6.1 for the first time, manually, you don't want the CalDB upgrade file installation; You need the whole CalDB 4.6.3. Please proceed to the full CalDB4 4.6.3 installation instructions below.
If your site has already installed CalDB 4.5.9 or later version, you may upgrade to 4.6.3 by using the "upgrade" tar file in Table 2 of the download page. Follow instructions 1-6 in this section to upgrade your CalDB4 installation.
Change directories to the existing CALDB4 location.
NOTE: With the CalDB 4.6.3 patch release of 29 Aug 2014, there are currently two (2) gzipped tar files needed for the upgrade.
Download the CalDB 4.6.3 "upgrade" gzipped tar file.
Download the ACIS BKGRND files tar ball, which has recently been updated in the CalDB 4.5.7 release, acis_bkgrnd_4.5.7.tar.gz. NOTE: The acis BKGNRD tar file has been updated for consistency with the ACIS CIF in CalDB 4.5.0. No datasets have actually changed, however, since the CalDB 4.3.1 release; only the FP_TEMP boundary range has been changed for the 2005 (GROUP E) datasets.
Download the HRC BKGRND events files tar ball, new in the CalDB 18.104.22.168 release, hrc_bkgrnd_22.214.171.124.tar.gz. NOTE: If you have already installed the HRC BKGRND files with CalDB 4.3.0 in June 2010 or later, you need not do this again.
UNIX: Unpack the gzipped tar ball using gtar xvUfz. The "U" option forces gtar appropriately to reset the links to the index files and your upgrade will be properly installed.
LINUX: You may omit the U option: gtar xvfz, or else you may get some meaningless error messages about failure to unlink non-empty directories. These errors are harmless.
If you do not have gtar installed locally, you may use tar xvfz instead.
unix% gtar xvUfz caldb_4.6.3_upgrade.tar.gz unix% gtar xvfz acis_bkgrnd_4.5.7.tar.gz unix% gtar xvfz hrc_bkgrnd_126.96.36.199.tar.gz
UPGRADE FINISHED! Proceed to Check the index configuration below.
The instructions for installing the CALDB are basically the same whether you are creating a local or sitewide installation.
Select a location for your CALDB at your site. This directory needs to be accessible by all users who are running CIAO 4.6.1 at your site.
Download CALDB 4.6.3 Full Installation from Table 1. There are five tar ball files in all, including the "main" tar ball, and the 4.6.3 upgrade patch tar ball.
The ACIS and HRC Background files have been packaged separately to reduce the space requirements for CalDB when users don't need the background datasets. They can be downloaded rather quickly and unpacked AFTER the MAIN 4.6.3 tarball has been unpacked. We clearly recommend downloading and installing these files on multi-user sites.
From within the CALDB directory, unpack the main tar ball using gtar xvUfz, and then after that unpack the 4.6.3 upgrade patch tar ball.
unix% gtar xvUfz caldb_4.6.3_main.tar.gz
UNIX: If you use gtar, then you should use the options xvUf (include the "U" option) in case there are filename conflicts which may not be appropriately clobbered. NOTE: In LINUX, the "U" option may cause meaningless errors about failure to unlink directories that are not empty. Ignore this error, or else omit the "U" option. If you do not have access to gtar then you may use tar xvf instead without worry.
The result will be three directory trees at that level, namely docs, data, and software:
unix% ls data/ docs/ software/
The docs directory contains the manifests and versioning files. while software/ contains the tools. The data directory is most important, containing the full CALDB data directory tree.
Unzip and untar the ACIS background package, unless you are certain that you will not need any of the blank-field files for making background datasets. Typically observers of extended sources need these files. We recommend multi-user sites to download and install these files in case anyone in your group needs them. NOTE: By unzipping the tarball in the SAME LOCATION AS THE MAIN TARBALL, you get the bkgrnd files exactly where you will want them.
unix% gtar xvfz acis_bkgrnd_4.5.7.tar.gz
Unzip and untar the HRC background package, unless you are certain that you will not need any of the HRC-I stowed background files for making background datasets. Typically observers of extended sources need these files. We recommend multi-user sites to download and install these files in case anyone in your group needs them. NOTE: By unzipping the tarball in the SAME LOCATION AS THE MAIN TARBALL, you get the bkgrnd files exactly where you will want them.
unix% gtar xvfz hrc_bkgrnd_188.8.131.52.tar.gz
Delete the tarfiles in order to save space. You may wish to complete the installation and verify that the CALDB is working before doing so.
To ensure that the index files are linked correctly, run this ls command from your CALDB directory:
unix% ls -l data/chandra/*/caldb.indx ls -l data/chandra/*/caldb.indx lrwxrwxrwx ... data/chandra/acis/caldb.indx -> ./index/caldbN0430.indx lrwxrwxrwx ... data/chandra/default/caldb.indx -> ./index/caldbN0405A.indx lrwxrwxrwx ... data/chandra/ephin/caldb.indx -> ./index/caldbN0400.indx lrwxrwxrwx ... data/chandra/hrc/caldb.indx -> ./index/caldbN0414.indx lrwxrwxrwx ... data/chandra/pcad/caldb.indx -> ./index/caldbN0402.indx lrwxrwxrwx ... data/chandra/pimms/caldb.indx -> ./index/caldbN0406.indx lrwxrwxrwx ... data/chandra/sim/caldb.indx -> ./index/caldbN0400.indx
where "..." represents the file owner and timestamp. Check that the results from your system match those shown here.
The first part of this step requires that you have FTOOLS available; if you do not, skip to the check_ciao_caldb example. Note that you do not need FTOOLS in order to use the CALDB with CIAO, only for this verification step.
The FTOOL chkcif is the method for verifying the installation of a CALDB after unpacking it. chkcif ("check CALDB index file") reads the index listings and
- verifies that the index for a given instrument is present and in the proper form.
- verifies that all ONLINE files in the index are present in the CALDB. If a file that is indexed as ONLINE is missing, the tool reports it to STDOUT and then exits with an error status.
- lists the OFFLINE files as offline and does not check for them.
A file that is marked OFFLINE is not included in the CalDB directory structure. By the HEASARC standard, such a file might be available in an off-line system such as another disk drive or a web location, and isn't accessible by chkcif. For the Chandra CalDB, "OFFLINE" indicates that a file has been removed from circulation, and is NO LONGER AVAILABLE, and should not be used. Hence, when chkcif returns that a file is OFFLINE, this is normal behavior for the CalDB, and does not indicate any problem with your configuration.
Note that the tool does not verify that each calibration file is whole or uncorrupted, only that it exists in the proper location in the directory structure.
Note that you need to start CIAO in the working window (to define some environment variables) before running chkcif. In this example, CIAO is installed in /soft/ciao:
unix% source /soft/ciao/bin/ciao.csh unix% chkcif chandra acis STDOUT ** chkcif 1.1.4 ... Contents of CIF /soft/ciao/CALDB/data/chandra/acis/caldb.indx verified ... See STDOUT for more information ** chkcif 1.1.4 completed successfully
This command needs to be repeated for all the Chandra CALDB instrument branches, namely "hrc", "ephin", "pcad", "sim", and "default".
IMPORTANT: IF the ACIS background files have not been downloaded and unpacked, AND/OR if the ACIS PSF library has not been downloaded and unpacked, then the ACIS branch will report errors, as shown in this example. Otherwise, the tool should return the "chkcif ... completed successfully" line for each instrument, verifying that the CALDB is correctly installed. The same holds for the HRC branch if the HRC BKGRND files and/or PSF library are not downloaded.
It is also possible to test the installation using the script "check_ciao_caldb". This routine will locate the CalDB used by your CIAO 4.1.x+ installation, re your current CalDB version, and indicate whether or not a standard test query to the CalDB has worked correctly.
First, start CIAO in the working window.
unix% check_ciao_caldb CALDB environment variable = /data/CALDB/ciao CALDB version = 4.6.3 release date = TBD UTC CALDB query completed successfully.
If the routine does not give you a response equivalent to the above for your machine and directory setup, you may need to reinstall your CalDB, following the Full Installation instructions above.
The tool supports the --latest flag which checks the CALDB version history page to see if your CALDB install is up to date. If it is you will see an extra line in the output:
unix% check_ciao_caldb --latest CALDB environment variable = /data/CALDB/ciao CALDB version = 4.6.3 release date = TBD UTC CALDB query completed successfully. The CALDB installation is up to date.
whereas if a newer version is available the tool will fail as shown below.
unix% check_ciao_caldb --latest CALDB environment variable = /export/local/caldb42 CALDB version = 4.2.2 release date = 2010-04-19T19:00:00 UTC CALDB query completed successfully. # check_ciao_caldb (29 Aug 2014): ERROR The latest available CALDB version is 4.6.3