Use merge_all Script to Compute ACIS Exposure Maps and Fluxed Images
[CIAO 3.4 Science Threads]
OverviewLast Update: 16 Apr 2007 - merge_all v3.6: "[subspace -expno]" filter was added to the dmmerge command in merge_all as a workaround for the extra GTIs in merged event files problem Synopsis: mkexpmap generates an exposure map which may be used to convert a counts image of a source to an image in flux units. The computed exposure map is essentially an image of the effective area at each sky position, accounting for the effects of dither motion which are especially important near the edges of the detector. Purpose: To create a wide variety of exposure maps using the script merge_all. Read this thread if: you are working with an ACIS imaging observation and would like to create an exposure map. If you are interested in having an accurate exposure map at the very edge of a CCD, subarray or window, this thread cannot be used. Instead, run one of the step-by-step exposure map threads that include the use of the mask file.
Related Links:
|
Contents
- Getting Started
- Create a Single-Chip, Single Observation Exposure Map and Fluxed Image
- Create a Multiple-Chip, Single Observation Exposure Map and Fluxed Image
- Create a Multiple-Chip, Multiple Observation Exposure Map and Fluxed Image
- Calculate the Source Flux
- Analysis Caveats
- Parameter files:
- History
-
Images
- Single chip, single observation exposure map
- Single chip, single observation fluxed image
- Multiple chip, single observation exposure map
- Multiple chip, single observation fluxed image
- Multiple chip, multiple observation event file
- Multiple chip, multiple observation exposure map
- Multiple chip, multiple observation fluxed image
Getting Started
Sample ObsIDs used: 1842 (ACIS-I, G21.5-09); 1843 (ACIS-I, G21.5-09)
File types needed: evt2; asol1
This thread uses the merge_all script; for information about the script, consult the help file ("ahelp merge_all"). The most recent version of merge_all is v3.6 (11 April 2007):
unix% egrep '(Version|Date)' `which merge_all` # Version: 3.6 # Date: 11 April 2007
Please check that you are using the most recent version before continuing. If you do not have the script installed or need to update to a newer version, please refer to the Scripts page.
Also, make sure that you have set up ardlib to use the bad pixel file for your observation before following this thread. Note that ardlib.par will only accept the bad pixel file for a single observation. If you are merging two or more observations which do not have similar bad pixel files (e.g. Faint vs. Very Faint mode observations), you will have to create the merged exposure map manually. This can be accomplished by following the Multiple Chip example and using dmregrid to combine the final products.
We begin by restricting the energy range from 0.3 keV to 10 keV:
unix% dmcopy "acisf01842N001_evt2.fits[energy=300:10000]" 1842_evt2.fits unix% dmcopy "acisf01843N001_evt2.fits[energy=300:10000]" 1843_evt2.fits
Caveat: extra GTIs in merged event files
When event files that have different ranges of exposure numbers (expno column) are merged, the output file will have more than one GTI block for each chip. This causes incorrect exposure values later in the analysis, as the CIAO tools cannot use multiple GTI blocks for a single chip. Further details are available on the dmmerge bug page.
A "[subspace -expno]" filter was added to the dmmerge command in merge_all v3.6 as a workaround for this problem.
Note that the addition of the subspace filter means that any user who intends to create lightcurves binned on exposure number from the merged output cannot use merge_all, since that information is eliminated from the subspace. (In general, lightcurves are binned on time.)
The ACIS dead area correction
There is a fractional area loss per unit time due to cosmic ray flux incident on the ACIS detector. Calibration to account for this ACIS "dead area" was included in CALDB 3.3.0 on 15 December 2006. The correction is non-zero for the 8 front-illuminated ACIS chips; the effect is not detectable for the BI chips, so the nominal calibration value is 0.0. The resulting chipy-dependent reduction in the EA will be approximately 2.2% at the readout, and 4.0% at the top of the chip. Refer to the ACIS Dead Area Correction why topic for technical details.
Since merge_all runs mkinstmap with a merged event file, it is not appropriate to try to apply the ACIS dead area correction while running this script. Users who wish to apply this calibration should use the step-by-step exposure maps threads available from the Imaging threads page.
Create a Single-Chip, Single Observation Exposure Map and Fluxed Image
Sample ObsID used in this section: 1843, chip I3 (ccd_id=3) only
Since we are only creating an exposure map and fluxed image for a single observation, we do not need to set the parameters refcoord or merged.
The xygrid parameter is chosen so that the resulting exposure map includes the region of the I3 chip and is effectively binned by 2.
Finally, we choose to make the map monoenergetic with an energy of 1 keV (see the Create a Single Chip ACIS Exposure Map Step-by-Step thread for details). You may optionally input an ASCII spectral weights file here, see the Calculating Spectral Weights thread for more information.
Set the necessary parameters and run the script:
unix% punlearn merge_all unix% pset merge_all evtfile=1843_evt2.fits unix% pset merge_all asol=pcadf084271087N001_asol1.fits unix% pset merge_all chip=3 unix% pset merge_all xygrid=2480.5:3680.5:#600,3940.5:5140.5:#600 unix% pset merge_all energy=1 unix% pset merge_all expmap=1843_I3_expmap.fits unix% pset merge_all expcorr=1843_I3_expcorr.fits unix% merge_all Input event file(s) (1843_evt2.fits): Input asol file(s); time-sorted (pcadf084271087N001_asol1.fits): Input Live Time Correction list files for HRC-I (): ACIS CCD ID(s), comma-separated or range; or 'HRC-I' or 'HRC-S' (3): Reference coordinates or evt2 file for reproject_events (): XY grid (e.g., 0.5:8192.5:#1024,0.5:8192.5:#1024) (2480.5:3680.5:#600,3940.5:5140.5:#600): Energy or spectral weights file for mkinstmap (1): Output merged event file (): Output merged exposure map (1843_I3_expmap.fits): Output exposure-corrected image (1843_I3_expcorr.fits): Creating aspect histogram for CCD 3... Making instrument map for CCD 3... ***ARDLIB warning: Filename 1843_evt2.fits does not specify an extension. Assuming the first "interesting" extension. Making exposure map for CCD 3... Exposure map limits: 0.000000e+00, 3.039603e+06 Creating image Normalizing image All done!
The warning may be ignored as it does not adversely affect the output.
In some cases there will be more than one asol1.fits file for an observation. All the files must be input to the asol parameter, either as a list or as a stack (see ahelp stack for more information).
The output exposure map is shown in Figure 1 ![[Link to Image 1: Single chip, single observation exposure map]](../../imgs/imageicon.gif){kind=small}
and the output fluxed image is
shown in Figure 2 .
Check the parameter file that was used with plist merge_all.
Create a Multiple-Chip, Single Observation Exposure Map and Fluxed Image
Sample ObsID used in this section: 1843, all chips:
unix% dmkeypar 1843_evt2.fits DETNAM echo+ ACIS-012367
Since we are only using a single observation, the parameters refcoord and merged do not need to be set.
The xygrid parameter is chosen so that the resulting exposure map is effectively binned by 8.
Finally, we choose to make the map monoenergetic with an energy of 1 keV (see the Create a Single Chip ACIS Exposure Map Step-by-Step thread for details). You may optionally input an ASCII spectral weights file here, see the Calculating Spectral Weights thread for more information.
Set the necessary parameters and run the script:
unix% punlearn merge_all unix% pset merge_all evtfile=1843_evt2.fits unix% pset merge_all asol=pcadf084271087N001_asol1.fits unix% pset merge_all chip=0,1,2,3,6,7 unix% pset merge_all xygrid=0.5:8192.5:#1024,0.5:8192.5:#1024 unix% pset merge_all energy=1 unix% pset merge_all expmap=1843_all_expmap.fits unix% pset merge_all expcorr=1843_all_expcorr.fits unix% merge_all Input event file(s) (1843_evt2.fits): Input asol file(s); time-sorted (pcadf084271087N001_asol1.fits): Input Live Time Correction list files for HRC-I (): ACIS CCD ID(s), comma-separated or range; or 'HRC-I' or 'HRC-S' (0,1,2,3,6,7): Reference coordinates or evt2 file for reproject_events (): XY grid (e.g., 0.5:8192.5:#1024,0.5:8192.5:#1024) (0.5:8192.5:#1024,0.5:8192.5:#1024): Energy or spectrum file for mkinstmap (1): Output merged event file (): Output merged exposure map (1843_all_expmap.fits): Output exposure-corrected image (1843_all_expcorr.fits): Creating aspect histogram for CCD 0... Creating aspect histogram for CCD 1... Creating aspect histogram for CCD 2... Creating aspect histogram for CCD 3... Creating aspect histogram for CCD 6... Creating aspect histogram for CCD 7... Making instrument map for CCD 0... ***ARDLIB warning: Filename 1843_evt2.fits does not specify an extension. Assuming the first "interesting" extension. Making instrument map for CCD 1... ***ARDLIB warning: Filename 1843_evt2.fits does not specify an extension. Assuming the first "interesting" extension. Making instrument map for CCD 2... ***ARDLIB warning: Filename 1843_evt2.fits does not specify an extension. Assuming the first "interesting" extension. Making instrument map for CCD 3... ***ARDLIB warning: Filename 1843_evt2.fits does not specify an extension. Assuming the first "interesting" extension. Making instrument map for CCD 6... ***ARDLIB warning: Filename 1843_evt2.fits does not specify an extension. Assuming the first "interesting" extension. Making instrument map for CCD 7... ***ARDLIB warning: Filename 1843_evt2.fits does not specify an extension. Assuming the first "interesting" extension. Making exposure map for CCD 0... Exposure map limits: 0.000000e+00, 3.115892e+06 Making exposure map for CCD 1... Exposure map limits: 0.000000e+00, 3.154063e+06 Making exposure map for CCD 2... Exposure map limits: 0.000000e+00, 3.070792e+06 Making exposure map for CCD 3... Exposure map limits: 0.000000e+00, 3.037846e+06 Making exposure map for CCD 6... Exposure map limits: 0.000000e+00, 2.560520e+06 Making exposure map for CCD 7... Exposure map limits: 0.000000e+00, 3.584601e+06 Merging exposure maps Creating image Normalizing image All done!
The warnings may be ignored as they do not adversely affect the output.
In some cases there will be more than one asol1.fits file for an observation. All the files must be input to the asol parameter, either as a list or as a stack (see ahelp stack for more information).
The output exposure map is shown in Figure
3 and the output fluxed image is shown in Figure 4 .
Check the parameter file that was used with plist merge_all.
Create a Multiple-Chip, Multiple Observation Exposure Map and Fluxed Image
Sample ObsIDs used in this section: 1842, 1843, chips I0-I3 (ccd_id=0,1,2,3) for both
The script is now used to merge the two observations and to create an exposure map and fluxed image. The second observation (ObsID 1842) is reprojected to the coordinate system of the first. Instead, one could choose to reproject both file to a new new tangent point in R.A. and Dec. [degrees].
The xygrid parameter is chosen so that the resulting exposure map is effectively binned by 8.
Finally, we choose to make the map monoenergetic with an energy of 1 keV (see the Create a Single Chip ACIS Exposure Map Step-by-Step thread for details). You may optionally input an ASCII spectral weights file here, see the Calculating Spectral Weights thread for more information.
Note that it is also possible to use a combined evt2 file (which was created by merge_all) as the evtfile and leave refcoord and merged blank. The script will then produce an exposure map and image using the combined evt2 file. The evt2 file must have been produced by the script since merge_all uses a modified version of the header lookup table to merge the two event files; the tools used by the script need this modified header.
Set the necessary parameters and run the script:
unix% punlearn merge_all unix% pset merge_all evtfile=1843_evt2.fits,1842_evt2.fits unix% pset merge_all asol=pcadf084271087N001_asol1.fits,pcadf084280882N001_asol1.fits unix% pset merge_all chip=0:3 unix% pset merge_all refcoord=1843_evt2.fits unix% pset merge_all xygrid=0.5:8192.5:#1024,0.5:8192.5:#1024 unix% pset merge_all energy=1 unix% pset merge_all merged=merged_evt2.fits unix% pset merge_all expmap=merged_expmap.fits unix% pset merge_all expcorr=merged_expcorr.fits unix% merge_all Input event file(s) (1843_evt2.fits,1842_evt2.fits): Input asol file(s); time-sorted (pcadf084271087N001_asol1.fits,pcadf084280882N001_asol1.fits): Input Live Time Correction list files for HRC-I (): ACIS CCD ID(s), comma-separated or range; or 'HRC-I' or 'HRC-S' (0:3): Reference coordinates or evt2 file for reproject_events (1843_evt2.fits): XY grid (e.g., 0.5:8192.5:#1024,0.5:8192.5:#1024) (0.5:8192.5:#1024,0.5:8192.5:#1024): Energy or spectrum file for mkinstmap (1): Output merged event file (merged_evt2.fits): Output merged exposure map (merged_expmap.fits): Output exposure-corrected image (merged_expcorr.fits): Reprojecting 1842_evt2.fits to 1843_evt2.fits Merging data... SIM_Z of event file: -245.8572931324 SIM_Z of pcadf084271087N001_asol1.fits: -245.8572931324 Difference is 0 mm: SIM_Z differs by less than ~1/4 pixel. SIM_Z of pcadf084280882N001_asol1.fits: -231.2390639692 Difference is 14.6182291632 mm: SIM_Z differs by more than ~1/4 pixel. Modifying DZ in pcadf084280882N001_asol1.fits Creating aspect histogram for CCD 0... Creating aspect histogram for CCD 1... Creating aspect histogram for CCD 2... Creating aspect histogram for CCD 3... Making instrument map for CCD 0... ***ARDLIB warning: Filename merged_evt2.fits does not specify an extension. Assuming the first "interesting" extension. Making instrument map for CCD 1... ***ARDLIB warning: Filename merged_evt2.fits does not specify an extension. Assuming the first "interesting" extension. Making instrument map for CCD 2... ***ARDLIB warning: Filename merged_evt2.fits does not specify an extension. Assuming the first "interesting" extension. Making instrument map for CCD 3... ***ARDLIB warning: Filename merged_evt2.fits does not specify an extension. Assuming the first "interesting" extension. Making exposure map for CCD 0... Exposure map limits: 0.000000e+00, 5.985148e+06 Making exposure map for CCD 1... Exposure map limits: 0.000000e+00, 6.078366e+06 Making exposure map for CCD 2... Exposure map limits: 0.000000e+00, 5.889566e+06 Making exposure map for CCD 3... Exposure map limits: 0.000000e+00, 5.831895e+06 Merging exposure maps Creating image Normalizing image All done!
The warnings may be ignored as they do not adversely affect the output.
In some cases, as shown here, there will be more than one asol1.fits file for an observation. All the files must be input to the asol parameter, either as a list or as a stack (see ahelp stack for more information).
The combined evt2 file is shown in Figure
5 , the exposure map is shown in Figure 6 , and the fluxed image is shown in
Figure 7 .
Note that for these observations, the SIM_Z value
(position of the Scientific Instrument Module with respect to
the aimpoint) differed significantly. This is the reason for
the modification of DZ that the script made, but will not be
the case for all of observations.
Check the parameter file that was used with plist merge_all.
Calculate the Source Flux
Since the units of the fluxed image are [photon/cm2/s/pixel], adding up the pixel values around a source results in the source flux in [photon/cm2/s]. Note that this flux is an approximation - as discussed in An Introduction to Exposure Maps (PS, 12pp) - since a spectral shape was assumed when using mkinstmap (in this thread, a monochromatic source).
The Create a Single Chip ACIS Exposure Map Step-by-Step thread shows how to approximate the source flux from the normalized image.
Analysis Caveats
Users should be cautious about analyzing the data for sources near the edges of the ACIS CCDs.
-
For X-rays passing through the mirrors, the very bottom of each CCD is obscured by the frame store. As a result, some of the events in rows with CHIPY <= 8 are not detected. (The set of rows affected varies from CCD to CCD.) Since the CIAO tools do not compensate for this effect, the ARFs and exposure maps for sources in these regions may be inaccurate.
-
For sources within about thirty-two pixels of any edge of a CCD, the source may be dithered off the CCD during part of an observation. The aspect histogram, which is used to create ARFs and exposure maps, is designed to compensate for this effect.
-
A contaminant has accumulated on the optical-blocking filters of the ACIS detectors, as described in the ACIS QE Degradation why topic. Since there is a gradient in the temperature across the filters (the edges are colder), there is a gradient in the amount of material on the filters. (The contaminant is thicker at the edges.) Within about 100 pixels of the outer edges of the ACIS-I and ACIS-S arrays, the gradient is relatively steep. Therefore, the effective low-energy (' 1 keV) detection efficiency may vary within the dither pattern in this region. The ARF and instrument map tools are designed to read a calibration file which describes this spatial dependence.
Parameters for /home/username/cxcds_param/merge_all.par
##
## Parameter file for merge_all script
##
evtfile = 1843_evt2.fits Input event file(s)
asol = pcadf084271087N001_asol1.fits Input asol file(s); time-sorted
dtffile = Input Live Time Correction list files for HRC-I
chip = 3 ACIS CCD ID(s), comma-separated or range; or 'HRC-I' or 'HRC-S'
refcoord = Reference coordinates or evt2 file for reproject_events
xygrid = 2480.5:3680.5:#600,3940.5:5140.5:#600 XY grid (e.g., 0.5:8192.5:#1024,0.5:8192.5:#1024)
energy = 1 Energy or spectral weights file for mkinstmap
merged = Output merged event file
expmap = 1843_I3_expmap.fits Output merged exposure map
expcorr = 1843_I3_expcorr.fits Output exposure-corrected image
(intdir = /tmp) Directory for intermediate files
(clobber = no) OK to overwrite existing output file(s)?
(mode = ql)
|
Parameters for /home/username/cxcds_param/merge_all.par
##
## Parameter file for merge_all script
##
evtfile = 1843_evt2.fits Input event file(s)
asol = pcadf084271087N001_asol1.fits Input asol file(s); time-sorted
dtffile = Input Live Time Correction list files for HRC-I
chip = 0,1,2,3,6,7 ACIS CCD ID(s), comma-separated or range; or 'HRC-I' or 'HRC-S'
refcoord = Reference coordinates or evt2 file for reproject_events
xygrid = 0.5:8192.5:#1024,0.5:8192.5:#1024 XY grid (e.g., 0.5:8192.5:#1024,0.5:8192.5:#1024)
energy = 1 Energy or spectral weights file for mkinstmap
merged = Output merged event file
expmap = 1843_all_expmap.fits Output merged exposure map
expcorr = 1843_all_expcorr.fits Output exposure-corrected image
(intdir = /tmp) Directory for intermediate files
(clobber = no) OK to overwrite existing output file(s)?
(mode = ql)
|
Parameters for /home/username/cxcds_param/merge_all.par
##
## Parameter file for merge_all script
##
evtfile = 1843_evt2.fits,1842_evt2.fits Input event file(s)
asol = pcadf084271087N001_asol1.fits,pcadf084280882N001_asol1.fits Input asol file(s); time-sorted
dtffile = Input Live Time Correction list files for HRC-I
chip = 0:3 ACIS CCD ID(s), comma-separated or range; or 'HRC-I' or 'HRC-S'
refcoord = 1843_evt2.fits Reference coordinates or evt2 file for reproject_events
xygrid = 0.5:8192.5:#1024,0.5:8192.5:#1024 XY grid (e.g., 0.5:8192.5:#1024,0.5:8192.5:#1024)
energy = 1 Energy or spectral weights file for mkinstmap
merged = merged_evt2.fits Output merged event file
expmap = merged_expmap.fits Output merged exposure map
expcorr = merged_expcorr.fits Output exposure-corrected image
(intdir = /tmp) Directory for intermediate files
(clobber = no) OK to overwrite existing output file(s)?
(mode = ql)
|
History
| 04 Jan 2005 | updated for CIAO 3.2: added Caveats section |
| 12 Jan 2005 | usage warning added to Overview; replaced Create a Multiple-Chip, Multiple Observation Exposure Map and Fluxed Image section with warning note |
| 23 Jun 2005 | CIAO 3.2.2 patch: bug fix to asphist means it is again possible to use the merge_all script to create exposure maps for multiple observations; updated images |
| 19 Dec 2005 | updated for CIAO 3.3: the merge_all script has been updated to version 3.3 for the new asphist tool syntax; corresponding changes to screen output |
| 01 Dec 2006 | updated for CIAO 3.4: version 3.4 of merge_all (removed "punlearn mkinstmap" to allow new CIAO 3.4 mkinstmap parameters (pbkfile, dafile) to be set before running the script, if desired; parameters supplied on the command line will not be prompted) |
| 26 Feb 2007 | merge_all v3.5: added a test to determine whether "energy" value is monochromatic or a spectrum file; temporary filenames have a timestamp value as the root, rather than hardwired names. |
| 03 Mar 2007 | added Caveat: extra GTIs in merged event files; added ACIS dead area correction section |
| 16 Apr 2007 | merge_all v3.6: "[subspace -expno]" filter was added to the dmmerge command in merge_all as a workaround for the extra GTIs in merged event files problem |
