About Chandra Archive Proposer Instruments & Calibration Newsletters Data Analysis HelpDesk Calibration Database NASA Archives & Centers Chandra Science Links

Skip the navigation links
Last modified: 1 Dec 2006
Hardcopy (PDF): A4 | Letter

Create a New ACIS Bad Pixel File: Identify ACIS Hot Pixels and Cosmic Ray Afterglows

[CIAO 3.4 Science Threads]



Overview

Last Update: 1 Dec 2006 - updated for CIAO 3.4: added "Create a New ACIS Bad Pixel File" to thread title; CIAO version in warning

Synopsis:

An afterglow is the residual change from the interaction of a cosmic ray in a CCD. Some of the excess charge is captured by charge traps and released in a few to a few dozen subsequent frames. If afterglow events are not removed from the data, they can result in the spurious "detection" of faint sources.

New tools were introduced in CIAO 3.2 to identify and flag hot pixel and afterglow events in ACIS observations. acis_run_hotpix is a script that runs the ACIS hot pixel tools in order; see the About the ACIS Hot Pixel Tools section for details.

These tools supersede acis_detect_afterglow and replaced that tool in standard data processing (SDP, aka "the pipeline") in DS 7.4.0. acis_run_hotpix is better because it finds some afterglows that acis_detect_afterglow does not, it also finds pixels that are hot or have bad bias values and it misidentifies far fewer legitimate X-ray events as bad.

Users should remove any afterglow correction that was applied via acis_detect_afterglow (e.g. during SDP) before running this thread; see the Remove the acis_detect_afterglow Correction thread for instructions.

Purpose:

To flag hot pixel and afterglow events in the event file, and to create a new bad pixel file with those pixels identified as bad.

Read this thread if:

  • you are working with any FAINT or GRADED TIMED exposure mode ACIS observation (imaging or grating) processed with an SDP version lower than DS 7.4.0. Data processed with DS 7.4.0 or higher have had this correction applied in the pipeline.

  • you are working with any VFAINT TIMED exposure mode ACIS observation (imaging or grating) processed with an SDP version lower than DS 7.6. The VFAINT mode data section has details.

  • Do not use acis_run_hotpix on continuous-clocking (CC) mode data. The tools are designed to look for bad pixels, not bad columns, and only the column of an event is known in CC mode.

The Get Started section shows how to check the software processing version and mode used for the observation.

Related Links:

Proceed to the HTML or hardcopy (PDF: A4 | letter) version of the thread.




Contents



About the ACIS Hot Pixel Tools

acis_run_hotpix is a script which executes, in this order, acis_build_badpix, acis_find_hotpix, acis_classify_hotpix, and acis_build_badpix a second time.

Collectively, these tools

  1. search for pixels where the bias value is too low or too high (acis_build_badpix),

  2. search for suspicious pixels where there is an unusually large or unusually small number of events (acis_find_hotpix),

  3. classify the events on suspicious pixels as being associated with cosmic ray afterglows, hot pixels or astrophysical sources (acis_classify_hotpix), and

  4. write the bad pixels (acis_classify_hotpix) and their neighbors (acis_build_badpix) to the output bad pixel file.

Note that the bad pixels in the input bad pixel file are not all necessarily included in the output bad pixel file. All the bad pixels are recalculated and flagged as appropriate in the output file.

Which status bits are used?

acis_find_hotpix identifies three different kinds of anomalies: hot pixels, afterglow events, and pixels with bright bias. These are represented by the following status bits in the bad pixel file:

Cause Bad Pixel Status Bit
hot pixel 14
afterglow event 15
bright bias 16

When the bad pixel and event files are input to acis_process_events, different bits are set in the event file to record these same anomalies:

Cause Event Status Bit
hot pixel 4
afterglow event 16
bright bias 4

For more information on the status bits in the bad pixel file, see "ahelp acis_build_badpix". Technical information on status bits in bad pixel and event files is available from the memos on the MIT CXC Documents page.


Doing pileup analysis

With the acis_detect_afterglow correction, users were advised to filter the data to retain all events tagged as afterglows if the source is piled and you intend use the jdpileup model in Sherpa or XSPEC. The new hot pixel tools are more judicious with respect to throwing away piled source events, so users should be able to analyze the data without having to unfilter events first. The Using A Pileup Model thread has more information on pileup modeling.


VFAINT mode data

The acis_build_badpix tool - which is called by acis_run_hotpix - no longer identifies pixels belonging to the outer 5x5 pixel island in VFAINT mode as bad. It will also clear those bits from an input file.

Originally, all of the outer 24 pixels of a 5x5 island in VFAINT mode were checked to make sure that the "adjacent" pixels to the center were not bad. However, only the center 3x3 is used to compute the pulse height and grade; the outer 16 pixels of the 5x5 are only used to search for potential cosmic-ray events if check_vf_pha=yes. To avoid discarding more events unnecessarily, the code was changed to only consider the central pixel and the eight pixels adjacent to the central pixel when searching for events where one of the pixels is bad.

This change was released on 23 June 2005 in the CIAO 3.2.2 software patch, and became part of SDP at version DS 7.6.

Users working with VFAINT mode data should:

  1. run acis_build_badpix in CIAO 3.2.2 or higher (assuming that the data was processed with an SDP version lower than DS 7.6)
  2. reprocess the event file with acis_process_events using the new bad pixel file in the badpixfile parameter
  3. recreate any images or spectra from that event file

It is not necessary to remake ARFs, instrument maps, or exposure maps, as the outer 16 pixels of a 5x5 island are not accounted for during those processes. You may wish to do so regardless, for the sake of consistency.



Get Started

Sample ObsID used: 459 (HETG/ACIS-S, 3C 273)

File types needed: evt1; bpix1; msk1; bias0; pbk0

The event file (acis_459_reset_evt1.fits) was created by running the Remove the acis_detect_afterglow Correction; it is not the evt1 file from standard processing. Also, it is now possible to to obtain the correct bias and parameter block files from ChaSeR or WebChaSeR; they are included in the secondary data products.

In this thread, we assume that all relevant files are in the same working directory.

Check the mode of the observation:

unix% dmkeypar acis_459_reset_evt1.fits READMODE echo+
TIMED

unix% dmkeypar acis_459_reset_evt1.fits DATAMODE echo+
FAINT

This is a TIMED FAINT mode observation, so acis_run_hotpix may be used.

We also need to check the processing version:

unix% dmkeypar acis_459_reset_evt1.fits ASCDSVER echo+
CIAO 3.4

Since this data has already had acis_detect_afterglow correction removed, the ASCDSVER keyword value is the CIAO software version that was used. If you have not removed the correction yet, you should see something like this:

unix% dmkeypar acisf00459_000N002_evt1.fits ASCDSVER echo+
R4CU5UPD11.1

The observation was processed before the tool was included pipeline, i.e. the ASCDSVER is lower than DS 7.4.0. Note that the version naming convention changed after version R4CU5UPD14 to the "DS" system, starting with DS 6.0.0. Before continuing, remove any afterglow correction that was applied via acis_detect_afterglow (e.g. during SDP); see the Remove the acis_detect_afterglow Correction for instructions.



Create a New Bad Pixel File

The acis_run_hotpix tool is run to create the new bad pixel file:

unix% punlearn acis_run_hotpix
unix% pset acis_run_hotpix infile=acis_459_reset_evt1.fits
unix% pset acis_run_hotpix outfile=acis_459_new_bpix1.fits
unix% pset acis_run_hotpix badpixfile=acisf00459_000N002_bpix1.fits
unix% pset acis_run_hotpix biasfile=@bias_files.lis 
unix% pset acis_run_hotpix maskfile=acisf00459_000N002_msk1.fits
unix% pset acis_run_hotpix pbkfile=acisf063875928N002_pbk0.fits
unix% acis_run_hotpix
Input event list (acis_459_reset_evt1.fits): 
Output bad pixel file (acis_459_new_bpix1.fits): 
Input mask file ( <filename> | default | none | NONE ) (acisf00459_000N002_msk1.fits): 
Input bad pixel file (acisf00459_000N002_bpix1.fits): 
Input bias images (@bias_files.lis): 

The badpixfile parameter is set to the observation-specific bad pixel file from the Archive, not a bad pixel file from the CALDB. The latter should not be used when running this tool.

Note that for most observations there will be more than one bias file. All the files must be input to the biasfile parameter, either as a list or as a stack (see ahelp stack for more information). Here we used:

unix% more bias_files.lis 
acisf063874723N002_1_bias0.fits
acisf063874723N002_2_bias0.fits
acisf063874723N002_3_bias0.fits
acisf063874723N002_4_bias0.fits
acisf063874723N002_5_bias0.fits

The contents of the parameter file may be checked using plist acis_run_hotpix.



Applying the File

Now that we have a new bad pixel file, it needs to be applied to the event file. To do so, run acis_process_events as shown in the Create a New Level=2 Event File thread. The bad pixel file is used in the badpixfile parameter:

unix% pset acis_process_events badpixfile=acis_459_new_bpix1.fits

Also, set this as the the observation-specific bad pixel file for any subsequent analysis tasks; see the Use Observation-specific Bad Pixel Files thread for instructions.



Warning message: "File /tmp/hotpixel_list.fits does not have any rows"

Some datasets will return this warning when acis_run_hotpix is run:

# acis_classify_hotpix (CIAO 3.4): WARNING: File /tmp/hotpixel_list.fits does not have any rows.

It just means that no hot pixels were identified in the observation, which is a normal occurrence.



Caveats

There are a few caveats associated with acis_run_hotpix.

  • acis_run_hotpix will not find afterglows and hot pixels if they are embedded in a bright source.

  • The tool can misidentify some legitimate X-ray events as bad. The worst case scenario is a very faint source that has a flare whose duration is much less than the period of the dither (nominally 707 s in the Z direction). The dither why topic has more information.




Parameters for /home/username/cxcds_param/acis_run_hotpix.par


        infile = acis_459_reset_evt1.fits Input event list
       outfile = acis_459_new_bpix1.fits Output bad pixel file
    badpixfile = acisf00459_000N002_bpix1.fits Input bad pixel file
      biasfile = @bias_files.lis  Input bias images
      maskfile = acisf00459_000N002_msk1.fits Input mask file ( <filename> | none | NONE )
       pbkfile = acisf063875928N002_pbk0.fits Input parameter block file
   (probthresh = 1.0e-3)          Minimum significance of potentially bad pixels after trials (e.g. 1 sig = 0.159, 2 sig = 0.0228, and 3 sig = 0.00135)
     (regwidth = 7)               Size of reference region for comparison (e.g. 7 pixels x 7 pixels)
  (expnothresh = 10)              Minimum difference in frame numbers between hot pixels
   (biasthresh = 6)               Minimum bias offset for invalid pixels in adu
      (tempdir = ${ASCDS_WORK_PATH} -> /tmp) Temporary work directory
      (verbose = 0)               Debug level
      (clobber = no)              Clobber output files?
         (mode = ql)              
    

History

10 Feb 2005 updated for CIAO 3.2.1: new acis_run_hotpix parameter (pbkfile); new interface for retrieving bias files
16 Feb 2005 clarified statement in Overview that acis_run_hotpix should not be used on CC-mode data
23 Jun 2005 CIAO 3.2.2 patch: added VFAINT mode data section
21 Jul 2005 explained "File /tmp/hotpixel_list.fits does not have any rows" Warning
12 Dec 2005 updated for CIAO 3.3: users should be able to obtain correct bias and parameter block files from ChaSeR or WebChaSeR; minor changes to acis_run_hotpix parameter file
22 Mar 2006 added Which status bits are used? subsection
29 Mar 2006 expanded information in Which status bits are used? subsection
05 Apr 2006 updated ASCDSVER check in Get Started to show that it may return a CIAO version
01 Dec 2006 updated for CIAO 3.4: added "Create a New ACIS Bad Pixel File" to thread title; CIAO version in warning

Return to Threads Page: Top | All | Data Prep
Hardcopy (PDF): A4 | Letter
Last modified: 1 Dec 2006


The Chandra X-Ray Center (CXC) is operated for NASA by the Smithsonian Astrophysical Observatory.
60 Garden Street, Cambridge, MA 02138 USA.    Email: cxcweb@head.cfa.harvard.edu
Smithsonian Institution, Copyright © 1998-2004. All rights reserved.