A Note on Processing Versions
![[CXC Logo]](../../imgs/cxc-logo.gif)
CIAO 4.13 Science Threads
Overview
Synopsis:
The CIAO Science Threads are designed to work with Chandra data that has been created using recent calibration data. In the past we have referred to this as "reprocessed" data, since early datasets have had to be reprocessed using the new calibration files.
There are two versions of "reprocessed" data that we talk about;
-
data that has been reprocessed by the Chandra Data Archive, because new or improved data and algorithms have been released since the data was originally processed,
-
data that has been reprocessed by a user, either with the chandra_repro script or manually using the acis_process_events or hrc_process_events tools, to apply the latest Calibration Database files.
In this thread we are interested in the first form of "reprocessing"; namely the software and calibration products that were used to create the Chandra data that is available from the Chandra archive.
Last Update: 22 Dec 2014 - Review for CIAO 4.7; updated screen output
Contents
Checking the pipeline version
Download the sample data: 1843 (ACIS-I, G21.5-0.9)
unix% download_chandra_obsid 1843 evt2
The version naming convention of the standard data processing pipeline changed after version R4CU5UPD14 to the "DS" system, starting with DS 6.0.0. Any data with a processing version of 6.0.0 or higher is more recent than R4CU5UPD14 (the "DS" term is not included in the version string).
The processing version information is stored in the ASCDSVER keyword:
unix% dmkeypar acisf01843N003_evt2.fits ASCDSVER echo+ 8.4.5
For an older version of the same dataset:
unix% dmkeypar acisf01843N001_evt2.fits ASCDSVER echo+ R4CU5UPD8.2
Checking the CALDB version
It is also useful to check the version of the CALDB used in the SDP pipeline, since the threads reference the CALDBVER header keyword when discussing processing changes:
unix% dmkeypar acisf01843N003_evt2.fits CALDBVER echo+ 4.5.1.1
![[WARNING]](../../imgs/warning.png)
The CALDBVER header keyword is not updated by the CIAO tools. The value always reflects the version of the CALDB applied in standard data processing (SDP).
To find out which version of the CALDB you are running on your local system, use the check_ciao_caldb script:
unix% check_ciao_caldb --latest CALDB environment variable = /soft/ciao/CALDB which is a link to = /data/CALDB/ciao CALDB version = 4.6.5 release date = 2014-12-16T17:00:00 UTC CALDB query completed successfully. The CALDB installation is up to date.
The --latest flag checks the CALDB web site to see if your installation is up to date.
The information is also stored in the caldb_version.fits file:
unix% dmlist "$CALDB/docs/chandra/caldb_version/caldb_version.fits[cols caldb_ver]" data -------------------------------------------------------------------------------- Data for Table Block CALDBVER -------------------------------------------------------------------------------- ROW CALDB_VER 1 4.1.0 2 4.1.1 3 4.1.2 4 4.1.3 5 4.1.4 6 4.2.0 ... 40 4.6.1.1 41 4.6.2 42 4.6.3 43 4.6.4 44 4.6.5
This file is automatically updated each time the CALDB is upgraded on your system, so the final row always indicates the current version.
What next?
Once you have checked the processing and CALDB versions used to create the archive data, checked the V&V report that is produced for each observation, and reviewed the Data Caveats section, you should reprocess your data to ensure that the latest calibration products have been applied.
History
22 Dec 2004 | updated for CIAO 3.2/CALDB 3.0.0 |
01 Dec 2005 | updated for CIAO 3.3/CALDB 3.2.0: screen output of caldb_version.fits |
01 Dec 2006 | updated for CIAO 3.4 and CALDB 3.3.0: screen output of caldb_version.fits |
09 Jan 2008 | updated for CIAO 4.0 and CALDB 3.4.2: screen output of caldb_version.fits; ObsID 1843 is now version N002 |
30 Dec 2008 | updated for CIAO 4.1 and CALDB 4.1.0: screen output of caldb_version.fits |
25 Jan 2010 | updated for CIAO 4.2 and CALDB 4.2.0: example of using check_ciao_caldb; screen output of caldb_version.fits |
10 Jan 2011 | updated for CIAO 4.3 and CALDB 4.4.1: updates to screen output |
03 Jan 2012 | reviewed for CIAO 4.4 and CALDB 4.4.7: updates to screen output |
03 Dec 2012 | Review for CIAO 4.5; update file version and screen output |
31 May 2013 | Minor updates to distinguish what type of reprocessing is being discussed; added a What next? section. |
10 Dec 2013 | Review for CIAO 4.6. No changes. |
22 Dec 2014 | Review for CIAO 4.7; updated screen output |