Create a New ACIS Bad Pixel File: Identify ACIS Hot Pixels and Cosmic Ray Afterglows
[CIAO 3.4 Science Threads]
OverviewLast Update: 1 Dec 2006 - updated for CIAO 3.4: added "Create a New ACIS Bad Pixel File" to thread title; CIAO version in warning Synopsis: An afterglow is the residual change from the interaction of a cosmic ray in a CCD. Some of the excess charge is captured by charge traps and released in a few to a few dozen subsequent frames. If afterglow events are not removed from the data, they can result in the spurious "detection" of faint sources. New tools were introduced in CIAO 3.2 to identify and flag hot pixel and afterglow events in ACIS observations. acis_run_hotpix is a script that runs the ACIS hot pixel tools in order; see the About the ACIS Hot Pixel Tools section for details. These tools supersede acis_detect_afterglow and replaced that tool in standard data processing (SDP, aka "the pipeline") in DS 7.4.0. acis_run_hotpix is better because it finds some afterglows that acis_detect_afterglow does not, it also finds pixels that are hot or have bad bias values and it misidentifies far fewer legitimate X-ray events as bad. Users should remove any afterglow correction that was applied via acis_detect_afterglow (e.g. during SDP) before running this thread; see the Remove the acis_detect_afterglow Correction thread for instructions. Purpose: To flag hot pixel and afterglow events in the event file, and to create a new bad pixel file with those pixels identified as bad. Read this thread if:
The Get Started section shows how to check the software processing version and mode used for the observation. Related Links:
|
Contents
- About the ACIS Hot Pixel Tools
- Get Started
- Create a New Bad Pixel File
- Applying the File
- Warning message: "File /tmp/hotpixel_list.fits does not have any rows"
- Caveats
- Parameter files:
- History
About the ACIS Hot Pixel Tools
acis_run_hotpix is a script which executes, in this order, acis_build_badpix, acis_find_hotpix, acis_classify_hotpix, and acis_build_badpix a second time.
Collectively, these tools
-
search for pixels where the bias value is too low or too high (acis_build_badpix),
-
search for suspicious pixels where there is an unusually large or unusually small number of events (acis_find_hotpix),
-
classify the events on suspicious pixels as being associated with cosmic ray afterglows, hot pixels or astrophysical sources (acis_classify_hotpix), and
-
write the bad pixels (acis_classify_hotpix) and their neighbors (acis_build_badpix) to the output bad pixel file.
Note that the bad pixels in the input bad pixel file are not all necessarily included in the output bad pixel file. All the bad pixels are recalculated and flagged as appropriate in the output file.
Which status bits are used?
acis_find_hotpix identifies three different kinds of anomalies: hot pixels, afterglow events, and pixels with bright bias. These are represented by the following status bits in the bad pixel file:
Cause | Bad Pixel Status Bit |
---|---|
hot pixel | 14 |
afterglow event | 15 |
bright bias | 16 |
When the bad pixel and event files are input to acis_process_events, different bits are set in the event file to record these same anomalies:
Cause | Event Status Bit |
---|---|
hot pixel | 4 |
afterglow event | 16 |
bright bias | 4 |
For more information on the status bits in the bad pixel file, see "ahelp acis_build_badpix". Technical information on status bits in bad pixel and event files is available from the memos on the MIT CXC Documents page.
Doing pileup analysis
With the acis_detect_afterglow correction, users were advised to filter the data to retain all events tagged as afterglows if the source is piled and you intend use the jdpileup model in Sherpa or XSPEC. The new hot pixel tools are more judicious with respect to throwing away piled source events, so users should be able to analyze the data without having to unfilter events first. The Using A Pileup Model thread has more information on pileup modeling.
VFAINT mode data
The acis_build_badpix tool - which is called by acis_run_hotpix - no longer identifies pixels belonging to the outer 5x5 pixel island in VFAINT mode as bad. It will also clear those bits from an input file.
Originally, all of the outer 24 pixels of a 5x5 island in VFAINT mode were checked to make sure that the "adjacent" pixels to the center were not bad. However, only the center 3x3 is used to compute the pulse height and grade; the outer 16 pixels of the 5x5 are only used to search for potential cosmic-ray events if check_vf_pha=yes. To avoid discarding more events unnecessarily, the code was changed to only consider the central pixel and the eight pixels adjacent to the central pixel when searching for events where one of the pixels is bad.
This change was released on 23 June 2005 in the CIAO 3.2.2 software patch, and became part of SDP at version DS 7.6.
Users working with VFAINT mode data should:
- run acis_build_badpix in CIAO 3.2.2 or higher (assuming that the data was processed with an SDP version lower than DS 7.6)
- reprocess the event file with acis_process_events using the new bad pixel file in the badpixfile parameter
- recreate any images or spectra from that event file
It is not necessary to remake ARFs, instrument maps, or exposure maps, as the outer 16 pixels of a 5x5 island are not accounted for during those processes. You may wish to do so regardless, for the sake of consistency.
Get Started
Sample ObsID used: 459 (HETG/ACIS-S, 3C 273)
File types needed: evt1; bpix1; msk1; bias0; pbk0
The event file (acis_459_reset_evt1.fits) was created by running the Remove the acis_detect_afterglow Correction; it is not the evt1 file from standard processing. Also, it is now possible to to obtain the correct bias and parameter block files from ChaSeR or WebChaSeR; they are included in the secondary data products.
In this thread, we assume that all relevant files are in the same working directory.
Check the mode of the observation:
unix% dmkeypar acis_459_reset_evt1.fits READMODE echo+ TIMED unix% dmkeypar acis_459_reset_evt1.fits DATAMODE echo+ FAINT
This is a TIMED FAINT mode observation, so acis_run_hotpix may be used.
We also need to check the processing version:
unix% dmkeypar acis_459_reset_evt1.fits ASCDSVER echo+ CIAO 3.4
Since this data has already had acis_detect_afterglow correction removed, the ASCDSVER keyword value is the CIAO software version that was used. If you have not removed the correction yet, you should see something like this:
unix% dmkeypar acisf00459_000N002_evt1.fits ASCDSVER echo+ R4CU5UPD11.1
The observation was processed before the tool was included pipeline, i.e. the ASCDSVER is lower than DS 7.4.0. Note that the version naming convention changed after version R4CU5UPD14 to the "DS" system, starting with DS 6.0.0. Before continuing, remove any afterglow correction that was applied via acis_detect_afterglow (e.g. during SDP); see the Remove the acis_detect_afterglow Correction for instructions.
Create a New Bad Pixel File
The acis_run_hotpix tool is run to create the new bad pixel file:
unix% punlearn acis_run_hotpix unix% pset acis_run_hotpix infile=acis_459_reset_evt1.fits unix% pset acis_run_hotpix outfile=acis_459_new_bpix1.fits unix% pset acis_run_hotpix badpixfile=acisf00459_000N002_bpix1.fits unix% pset acis_run_hotpix biasfile=@bias_files.lis unix% pset acis_run_hotpix maskfile=acisf00459_000N002_msk1.fits unix% pset acis_run_hotpix pbkfile=acisf063875928N002_pbk0.fits unix% acis_run_hotpix Input event list (acis_459_reset_evt1.fits): Output bad pixel file (acis_459_new_bpix1.fits): Input mask file ( <filename> | default | none | NONE ) (acisf00459_000N002_msk1.fits): Input bad pixel file (acisf00459_000N002_bpix1.fits): Input bias images (@bias_files.lis):
The badpixfile parameter is set to the observation-specific bad pixel file from the Archive, not a bad pixel file from the CALDB. The latter should not be used when running this tool.
Note that for most observations there will be more than one bias file. All the files must be input to the biasfile parameter, either as a list or as a stack (see ahelp stack for more information). Here we used:
unix% more bias_files.lis acisf063874723N002_1_bias0.fits acisf063874723N002_2_bias0.fits acisf063874723N002_3_bias0.fits acisf063874723N002_4_bias0.fits acisf063874723N002_5_bias0.fits
The contents of the parameter file may be checked using plist acis_run_hotpix.
Applying the File
Now that we have a new bad pixel file, it needs to be applied to the event file. To do so, run acis_process_events as shown in the Create a New Level=2 Event File thread. The bad pixel file is used in the badpixfile parameter:
unix% pset acis_process_events badpixfile=acis_459_new_bpix1.fits
Also, set this as the the observation-specific bad pixel file for any subsequent analysis tasks; see the Use Observation-specific Bad Pixel Files thread for instructions.
Warning message: "File /tmp/hotpixel_list.fits does not have any rows"
Some datasets will return this warning when acis_run_hotpix is run:
# acis_classify_hotpix (CIAO 3.4): WARNING: File /tmp/hotpixel_list.fits does not have any rows.
It just means that no hot pixels were identified in the observation, which is a normal occurrence.
Caveats
There are a few caveats associated with acis_run_hotpix.
-
acis_run_hotpix will not find afterglows and hot pixels if they are embedded in a bright source.
-
The tool can misidentify some legitimate X-ray events as bad. The worst case scenario is a very faint source that has a flare whose duration is much less than the period of the dither (nominally 707 s in the Z direction). The dither why topic has more information.
Parameters for /home/username/cxcds_param/acis_run_hotpix.par infile = acis_459_reset_evt1.fits Input event list outfile = acis_459_new_bpix1.fits Output bad pixel file badpixfile = acisf00459_000N002_bpix1.fits Input bad pixel file biasfile = @bias_files.lis Input bias images maskfile = acisf00459_000N002_msk1.fits Input mask file ( <filename> | none | NONE ) pbkfile = acisf063875928N002_pbk0.fits Input parameter block file (probthresh = 1.0e-3) Minimum significance of potentially bad pixels after trials (e.g. 1 sig = 0.159, 2 sig = 0.0228, and 3 sig = 0.00135) (regwidth = 7) Size of reference region for comparison (e.g. 7 pixels x 7 pixels) (expnothresh = 10) Minimum difference in frame numbers between hot pixels (biasthresh = 6) Minimum bias offset for invalid pixels in adu (tempdir = ${ASCDS_WORK_PATH} -> /tmp) Temporary work directory (verbose = 0) Debug level (clobber = no) Clobber output files? (mode = ql) |
History
10 Feb 2005 | updated for CIAO 3.2.1: new acis_run_hotpix parameter (pbkfile); new interface for retrieving bias files |
16 Feb 2005 | clarified statement in Overview that acis_run_hotpix should not be used on CC-mode data |
23 Jun 2005 | CIAO 3.2.2 patch: added VFAINT mode data section |
21 Jul 2005 | explained "File /tmp/hotpixel_list.fits does not have any rows" Warning |
12 Dec 2005 | updated for CIAO 3.3: users should be able to obtain correct bias and parameter block files from ChaSeR or WebChaSeR; minor changes to acis_run_hotpix parameter file |
22 Mar 2006 | added Which status bits are used? subsection |
29 Mar 2006 | expanded information in Which status bits are used? subsection |
05 Apr 2006 | updated ASCDSVER check in Get Started to show that it may return a CIAO version |
01 Dec 2006 | updated for CIAO 3.4: added "Create a New ACIS Bad Pixel File" to thread title; CIAO version in warning |