Last modified: December 2022

URL: https://cxc.cfa.harvard.edu/ciao/ahelp/tgextract.html
AHELP for CIAO 4.16

tgextract

Context: Tools::Gratings

Synopsis

Bin event list grating wavelengths column into a one-dimensional counts histogram, by source, grating part, and diffraction order.

Syntax

tgextract infile outfile tg_srcid_list tg_part_list tg_order_list
ancrfile respfile [outfile_type] [inregion_file] [backfile] [rowid]
[bin_units] [min_bin_leg] [max_bin_leg] [bin_size_leg] [num_bins_leg]
[min_bin_meg] [max_bin_meg] [bin_size_meg] [num_bins_meg] [min_bin_heg]
[max_bin_heg] [bin_size_heg] [num_bins_heg] [min_tg_d] [max_tg_d]
[extract_background] [min_upbkg_tg_d] [max_upbkg_tg_d]
[min_downbkg_tg_d] [max_downbkg_tg_d] [geompar] [clobber] [verbose]

Description

"tgextract" filters and bins a grating event file (as output by "tg_resolve_events") into a one-dimensional counts spectrum for each grating part, order, and source, as appropriate. The grating event file contains columns labeling these attributes (tg_part, tg_srcid, tg_m) and the event diffraction coordinates (tg_lam, tg_mlam, tg_r, tg_d; see "ahelp tg_resolve_events") in a spatial region many PSFs wide. "tgextract" bins from a narrow source region in the cross-dispersion direction (either rectangular or via a region file as specified by parameters), and also bins background arrays from wide regions adjacent to the source region. If the background region needs to be specified with a more complex shape than allowed by the "tgextract" parameters, the tool "tgextract2" can be used instead.

The background arrays should be thought of as "related to the background". Background rejection is high for a grating and ACIS. Some proportion of the pseudo-background is due to faint wings of the LSF, and for ACIS, to the frame-shift streak (aliasing of charge in chipy in an amount given by the ratio of the frame-shift-time to the frame-time, or rougly 0.041s/3.200s=0.0128, for nominal timed exposure). Background for HRC will be higher because there is no order-sorting possible via the detector intrinsic energy resolution.

Default output is a "Type II PHA" file, with some CXC addition of columns. This is a standard FITS format in which each row contains array columns. In this case, each row is a unique grating part, source (if multiple), and order. "Type I" are supported via a parameter switch. Both formats are back-compatible with existing software (in particular with xspec and ftools) while providing convenient or necessary extensions.

Note: "tgextract" bins the real-valued diffraction coordinate lists, not detector pixels. Due to aspect dither and instrument module flexure, many detector pixel locations (roughly, a 16x16 arcsec detector region) map to one diffraction coordinate (though at different times).

The default units are Angstroms, since the grating dispersion is linear in wavelength. Default bin order is in ascending energy (decreasing wavelength) for back-compatibility with xspec.

The cross-dispersion coordinates are specified in "grating" angles. That is, using the Rowland spacing as the focal length. The dimensions can be found in the Calibration Database "geom" file (e.g., telD1999-07-23geomN0004.fits), as header keywords or tabulated values. For reference, the current Rowland spacings are 8632.48 mm (HETG) and 8637.0 mm (LETG); ACIS pixels are 0.023987 mm square, HRC-S are 0.0064294 mm. This yields grating angular scales of 0.5731 arcsec/pix (1.592e-04 deg/pix; HETG/ACIS-S), 0.1535 arcsec/pix (4.265e-05 deg/pix; LETG/HRC-S), and 0.5728 arcsec/pix (1.591e-04 deg/pix; LETG/ACIS-S).

RESTRICTIONS: The Type II PHA format is limited to having spectral arrays on each row of the same length. For example, if the MEG spectrum has 8192 bins, starts at 1A, with bin size of 0.005A, then the HEG rows must have 8192 bins. If the HEG were also binned at 0.005A (2*default), then the output array would have to extend for 8192 bins to about 40A, whereas the physical detector edge occurs at about 20A.

The parameters actually over specify the grids, providing four parameters while only three are unique. This allows grid specification in whatever form is more convenient (start:stop:step, or start:binsize:#bins, or start:stop:#bins, for examples). Separate parameters are also given for each grating type.

The binning regions are appended to the output as a FITS binary table of type, REGION.

USER ADVISORY: the source binning region can be adjusted arbitrarily. However, this can affect the effective area and line-spread-function (LSF) by clipping the spatial distribution. A detector pulse-height clipping of orders has already been imposed and cannot be altered by this tool (see "ahelp tg_resolve_events" and "ahelp mkgarf"). The cross-dispersion affect on effective area can be accounted for by mkgrmf.


Examples

Example 1

tgextract acis_hetg_evt2.fits acis_hetg_pha2.fits tg_order_list="-1,1"

Generate a pha_typeII file, acis_hetg_pha2.fits, from acis_hetg_evt2.fits which contains HETG data (that is both MEG and HEG data), for orders -1 and +1. Other parameters are left to the defaults.

Example 2

tgextract hrcs_letg_evt2.fits hrcs_letg_pha2 outfile_type=pha_typeI
inregion_file=letgD1999-07-22regN0001.fits min_bin_leg=1
bin_size_leg=0.1 num_bins_leg=2048 verbose=5

Generate two pha_typeI files, from input file hrc_letg_evt2.fits for "orders" -1 and +1 (HRC-S cannot resolve orders; -1 and 1 refer to the total of negative or positive side counts, respectively). Use the LETG/HRC-S "bow-tie" region file as a spatial filter ("inregion_file") for background reduction (this is in the Calibration Database and is the default for pipeline processing). Set the binning to be 8 times the default (0.0125 A/bin, 16384 bins)

Example 3

tgextract Evt/acisfnnnn_N001_evt2.fits.gz outfile=tst_pha
outfile_type=pha_typeI tg_part_list="HETG" tg_order_list="-1,1"
tg_srcid_list=1 ancrfile=none respfile=none min_bin_meg=1
bin_size_meg=0.04 num_bins_meg=1024 min_bin_heg=1 bin_size_heg=0.04
num_bins_heg=1024 verbose=5 extract_back=no

Extract Type I pha files from a compressed Level 2 event list, only orders -1 and +1. Bin MEG and HEG to the same coarse grid (defaults are 0.005 for MEG and 0.0025 for HEG). Four files are written, with root name "tst_pha", and full names:

(In the output file name, "pha" refers to a file format specification, and the "2" refers to "Level 2" data; "H" and "M" refer to HEG or MEG, and "m" and "p" refer to minus or plus side orders.) Setting the "verbose" parameter to its maximum value of 5 causes much diagnostic material to be written to the output stream. One possibly useful item is the filtering and binning specification into which the grid parameters translate, e.g.:

Opening image:
Evt/acisfnnnn_N001_evt2.fits.gz[tg_srcid=1,tg_part=2,tg_m=1,tg_d=-0.0006
63889:0.000663889,tg_lam=1:41.96][bin tg_lam=1:41.96:0.04] with NUM
BINS for binned_data = 1024

Parameters

name type ftype def min max units reqd stacks autoname
infile file input         yes no  
outfile file output         yes   yes
tg_srcid_list string   all       yes    
tg_part_list string   header_value       yes    
tg_order_list string   default       yes yes  
ancrfile file   none       yes    
respfile file   none       yes    
outfile_type string   pha_typeII       no    
inregion_file file ARD none            
backfile file   none            
rowid string                
bin_units string   angstrom            
min_bin_leg string   compute            
max_bin_leg string   compute            
bin_size_leg string   compute            
num_bins_leg string   compute            
min_bin_meg string   compute            
max_bin_meg string   compute            
bin_size_meg string   compute            
num_bins_meg string   compute            
min_bin_heg string   compute            
max_bin_heg string   compute            
bin_size_heg string   compute            
num_bins_heg string   compute            
min_tg_d string   default     degrees      
max_tg_d string   default     degrees      
extract_background boolean   yes            
min_upbkg_tg_d string   default     degrees      
max_upbkg_tg_d string   default     degrees      
min_downbkg_tg_d string   default     degrees      
max_downbkg_tg_d string   default     degrees      
geompar file   geom            
clobber boolean   no            
verbose integer   0 0 5        

Detailed Parameter Descriptions

Parameter=infile (file required filetype=input stacks=no)

Name of Input event file.

e.g. acisf00001_N0001_evt1a.fits, or hrcf00001_N0002_evt1a.fits.gz.

The file must contains an EVENTS extension which contains the diffraction coordinate columns to be filtered and binned. These coordinates are written by "tg_resolve_events": The diffraction coordinate columns of the event file are:

The input file must also have sky REGION extension which defines the source list (not the diffraction coordinate binning region). The source numbers and zero-order coordinates are copied to an output diffraction coordinates REGION extension.

See "ahelp tg_resolve_events" for details of the file contents.

Parameter=outfile (file required filetype=output autoname=yes)

Output file name or template.

If "outfile_type=pha_typeII", this is either the literal output file name or ".". If ".", the input file name is used as a template to create form the output name. For example, if "infile=root_evt2.fits", then outfile will be "root_pha2.fits".

If "outfile_type=pha_typeI", the name is used as the root name of the output files which will be named as: "root<srcid><part><order>_pha2.fits". As many files as there are source IDs, parts, and orders will be created.

If the output name is given as ".", the input file name is used to create the output file. For example, "infile=root_extension.fits" the output file name will be "root_<srcid><part><order>_pha2.fits".

Parameter=tg_srcid_list (string required default=all)

The list of sources to process. ("all"|string|@filename)

Option "all" will process all the source id's present in the input REGION extension. "@filename" directs the parameter input to the named file, "filename", which must contain a comma separated list or source id's to process.

Parameter=tg_part_list (string required default=header_value)

Grating parts to process: (HETG|HEG|MEG|LETG|header_value). The HETG has types of grating elements which produce spatially separated diffracted photon regions, either or both of which may be processed. The default is "header_value", which uses the input file's GRATING header keyword value, which is either "HETG" or "LETG" (or "NONE", but that is not a valid option).

Parameter=tg_order_list (string required default=default stacks=yes)

Grating diffraction orders to process: ("default"|comma list|range list|@file)

"default" is set to process the following: if ACIS: 1, 2, 3, -1, -2, -3 if HRC: -1, 1.

A "comma list" is a comma separated string list of the orders the user wants to process, ie: "-5, -1, 1, 3"

A range list sets the min and max of the orders to process; all the orders in between, will be processed, ie: "-1..5" will do orders from -1 to +5th order (excluding 0) a range list can be mixed with comma separated list.

"@file" is a pointer to an ASCII file which contains a comma separated list and/or range list of the orders to process.

Parameter=ancrfile (file required default=none)

Ancillary response file name. Default is "none".

This is for back-compatibility with software which expects the the header keyword, "ANCRFILE" to exist, even if its value is "NONE". The keyword and header are written to the output file and do not affect the spectrum.

Parameter=respfile (file required default=none)

Redistribution matrix file name.

This is for back-compatibility with software which expects the the header keyword, "RESPFILE" to exist, even if its value is "NONE". The keyword and header are written to the output file and do not affect the spectrum.

Grating redistribution matrices, which encode the line spread function (LSF) for a given cross-dispersion extraction region can be made with mkgrmf. Nominal versions can be found in the calibration database.

Parameter=outfile_type (string not required default=pha_typeII)

Ouput file type: "pha_typeI" or "pha_typeII" (default). Both type of files contain a SPECTRUM extension and a REGION extension. Each file contains the same information. The Type II has a row for each order and grating part, and the row contains both scalar and array columns. The Type I format does not have array columns; the arrays become columns, and the scalar Type II's columns become header keywords.

The SPECTRUM structure is:

Columns for Table Block SPECTRUM
ColNo Name Unit Type Description
1 SPEC_NUM Int2 Spectrum Number
2 TG_M Int2 Diffraction order (m)
3 TG_PART Int2 Spectral component (HEG, MEG, LEG, HESF parts)
4 TG_SRCID Int2 Source ID, output by detect
5 X pixel Real4 X sky coord of source
6 Y pixel Real4 Y sky coord of source
7 CHANNEL[8192] Int2(8192) Vector of spectral bin numbers.
8 COUNTS[8192] count Int2(8192) Counts array (a spectrum)
9 STAT_ERR[8192] count Real4(8192) Statistical uncertainty (error) on counts colum
10 BACKGROUND_UP[8192] count Int2(8192) Background count vector.
11 BACKGROUND_DOWN[8192] count Int2(8192) Background count vector.
12 BIN_LO[8192] angstrom Real8(8192) Bin boundry, left edge
13 BIN_HI[8192] angstrom Real8(8192) Bin boundry, right edge

The REGION structure is:

Columns for Table Block REGION
ColNo Name Unit Type Description
1 SPEC_NUM Int2 Spectrum number, which points to the SPECTRUM row
2 ROWID String[64] Source or background region identifier
3 SHAPE String[16] Shape of region
4 TG_LAM angstrom Real4 x-coordinate of SHAPE
5 TG_D degrees Real4 y-coordinate of SHAPE
6 R[2] (angstrom, degrees) Real4(2) Generalized Radii of SHAPE
7 ROTANG degrees Real4 CCW Rotation angle for SHAPE
8 TG_PART Int2 Grating part index (HEG=1, MEG=2, LEG=3)
9 TG_SRCID Int2 Source identification number
10 TG_M Int2 Diffraction order
11 COMPONENT Int2 Component index

Parameter=inregion_file (file filetype=ARD default=none)

Spatial filter FITS region file. This may be "none", in which case the rectangular binning region parameters are used.

Users can specify "CALDB" to automatically look up the appropriate region file for the data. However, no region file will be picked up if the data are not HRC-S/LETG data.

NOTE: this file is NOT the same as the grating mask file produced by the tg_create_mask tool; it must be in diffraction coordinates, whereas the other is in sky coordinates.

This file allows binning from other than rectangular spatial regions. This is especially important for HRC-S/LETG, which has high astigmatism which makes the spectrum more "bow-tie" shaped. Following this shape when binning reduces background in the low-wavelength region. (This is the default for pipeline processing.)

CAUTION must be taken with any custom regions, since they could affect the effective area by clipping the cross-dispersion profile. Spatial clipping factors are accomodated by the grating response matrix function utilities (see "ahelp mkgrmf").

The input region file should contain the following columns:
ColNo Name Unit Type
1 ROW_IDX Int2
2 ROWID String[15]
3 SHAPE String[16]
4 dispang(TG_R,TG_D)[7] degrees Real4(7)
5 R[2] degrees Real4(2)
6 ROTANG degrees Real4
7 TG_PART Int2
8 TG_SRCID Int2
9 TG_M Int2
10 BACKSCAL Real4
11 COMPONENT Int2

ROW_IDX: Numerical representation of the region. 1 stands for source region, 2 stands for background-up, and 3 stands for background-down region.

ROWID: String representation of the region. The 3 possible values are SOURCE, BACKGROUND_UP and BACKGROUND_DOWN. (This string is redundant with the integer ROW_IDX.)

SHAPE: Specifies the geometrical shape of the region. The default "bow-tie" is a polygon.

"dispang" is an alias for two vector columns, TG_R and TG_D, of length sufficient to describe the polygon in diffraction coordinates.

R (radius): This column is needed for regions that require a radius parameter in their definition.

ROTANG (rotation angle): This column is needed for rotated regions.

TG_PART: Numerical representation of the grating part. 1 stands for HEG, 2 stands for MEG and 3 stands for LEG.

TG_SRCID: Specifies the source to which the region pertains.

TG_M: Specifies what diffraction order sign that the region refers to. Region with TG_M=1 will be used to filter events that have positive diffraction order, while region with TG_M=-1 will be used to filter events that have negative diffraction order.

BACKSCAL: "Area" scaling term for the region. For grating spectra, the relevant quantity is the width in the cross-dispersion region at any particular wavelength. BACKSCAL is proportional to that width, or to the ratio of the background region width to the source-region width.

The default "bow-tie" has a changing width, but a constant ratio to the source region width. Hence, a scalar is used, which retains back-compatibility with some existing software.

Parameter=backfile (file default=none)

BACKFILE keyword name (for back-compatibility only).

The value given (default="none") is written to the output as a column (Type II) or header keyword (Type I) and does not affect the counts arrays.

Parameter=rowid (string)

The value of ROWID is an optional identifier for each row. It is for comments or other attributes for each row of a Type II PHA file, and does not affect the counts histogram. The same value is written to every row.

Parameter=bin_units (string default=angstrom)

Coordinate units for diffraction axis binning parameters (see below): [angstrom|eV|keV]

Options are: angstrom, keV or eV. It specifies the units of the min_bin_*, max_bin_*, and bin_size_* parameters.

The binning parameters which follow overspecify the grid. Any three may be specified, and the fourth is then determined. This is to provide flexibility in terms of preference for specifying starting or ending coordinates, number of bins, or bin sizes. Each grating has its own set of parameters for maximum flexibility.

Parameter=min_bin_leg (string default=compute)

Minimum dispersion coordinate on which to start binning the +1 or -1 order of the LEG spectrum. ["compute"|real number]. If min_bin_leg is set to "compute", it will be determined from the other three grid parameters (max, size, num). tgextract. This the low-value bin edge of the lowest coordinate bin.

If the detector is HRC_S or HRC_I and the min_bin_leg, max_bin_leg, bin_size_leg and num_bins_leg are all set to compute, these binning parameters for LEG +1 or -1 order will have the following default values:
If the detector is ACIS_S and the min_bin_leg, max_bin_leg, bin_size_leg and num_bins_leg are all set to compute, these binning parameters for LEG +1 or -1 order will have the following default values:

The LEG binning parameters for other order will be scaled by 1/order.

If only the min_bin_leg parameter is set to "compute" and the rest of the binning parameters are set to real values, min_bin_leg will be determined according to the following equation:

min_bin_leg = max_bin_leg - num_bins_leg * bin_size_leg

Parameter=max_bin_leg (string default=compute)

Maximum dispersion coordinate on which to stop binning the LEG +1 or -1 order. ["compute"|real number] This is the high-coordinate edge of the highest-coordinate bin.

A value of "compute" means max_bin_leg will be calculated using the values entered for the other LEG binning parameters. If a specific value for the max_bin_leg parameter is not specified, max_bin_leg will be determined last if possible using the following equation:

max_bin_leg = min_bin_leg + num_bins_leg * bin_size_leg

Parameter=bin_size_leg (string default=compute)

Size of the bins for the LEG +1 or -1 order spectrum. ["compute"|real number]

If bin_size_leg is set to "compute", it will be calculated using the values entered for the other LEG binning parameters. Default value is 0.0125A.

Parameter=num_bins_leg (string default=compute)

Number of bins for the LEG spectra. ["compute"|real number]

If num_bins_leg is set to "compute", it will be calculated using the values entered for other LEG binning parameters. The default of 8192 will be used for ACIS_S, and 16384 for HRC_S and HRC_I.

Parameter=min_bin_meg (string default=compute)

See corresponding leg parameter for the full description.

Defaults:

Parameter=max_bin_meg (string default=compute)

See corresponding leg parameter for the full description.

Default for first orders: 41.96A

Parameter=bin_size_meg (string default=compute)

See corresponding leg parameter for the full description.

Default for first orders: 0.005A

Parameter=num_bins_meg (string default=compute)

See corresponding leg parameter for the full description.

Default: 8192.

If both HEG and MEG data are processed at the same time, num_bins_heg has to be equal to num_bins_meg.

Parameter=min_bin_heg (string default=compute)

See corresponding leg parameter for the full description.

Defaults:

Parameter=max_bin_heg (string default=compute)

See corresponding leg parameter for the full description.

Default for first orders is 21.48A.

Parameter=bin_size_heg (string default=compute)

See corresponding leg parameter for the full description.

Default first orders: 0.0025A.

Parameter=num_bins_heg (string default=compute)

See corresponding leg parameter for the full description.

Default is 8192.

Parameter=min_tg_d (string default=default units=degrees)

The minimum tg_d value to include in the source spectrum. ["default"|real number]

Defaults:

If the source region(s) is given in the input region file (inregion_file), the min_tg_d parameter will be ignored.

Parameter=max_tg_d (string default=default units=degrees)

The maximum tg_d value to include in the source spectrum. ["default"|real number]

Defaults:

Note that if source region(s) is given in the input region file (inregion_file), the max_tg_d parameter will be ignored.

Parameter=extract_background (boolean default=yes)

Indicate whether to bin a local background spectrum.

If set to "yes", events with cross-dispersion from the background region that lies above (positive tg_d) the source region will be binned into a background_up spectrum, while events with cross-dispersion from the background region that lies below (negative tg_d) the source region will be binned into a background_down spectrum. If no background regions are given in the input region file (inregion_file) or if there is no input region file, the min_upbkg_tg_d and max_upbkg_tg_d parameters are used to determine the size of the background_up spectrum. And the min_downbkg_tg_d and max_downbkg_tg_d parameters are used to determine the size of the background_down spectrum.

Parameter=min_upbkg_tg_d (string default=default units=degrees)

Specifies the minimum tg_d value to include in the background_up spectrum. ["default"|real number]

If min_upbkg_tg_d is set to "default", the same value for max_tg_d will be used for min_upbkg_tg_d.

Note that if background_up region(s) is given in the input region file (inregion_file), the min_upbkg_tg_d parameter will be ignored.

Parameter=max_upbkg_tg_d (string default=default units=degrees)

Specifies the maximum tg_d value to include in the background_up spectrum. ["default"|real number]

Defaults:

Note that if background_up region(s) is given in the input region file (inregion_file), the max_upbkg_tg_d parameter will be ignored.

Parameter=min_downbkg_tg_d (string default=default units=degrees)

Specifies the minimum tg_d value to include in the background_down spectrum. ["default"|real number]

The default is to have the background_down region be symmetric to the background_up region. So if min_downbkg_tg_d is set to "default", it will be determined using the equation:

min_downbkg_tg_d = max_downbkg_tg_d - (max_upbkg_tg_d - min_upbkg_tg_d)

Note that if background_down region(s) is given in the input region file (inregion_file), the min_downbkg_tg_d parameter will be ignored.

Parameter=max_downbkg_tg_d (string default=default units=degrees)

Specifies the maximum tg_d value to include in the background_down spectrum. ["default"|real number]

Parameter=geompar (file default=geom)

The name of the Pixlib Geometry parameter file.

Parameter=clobber (boolean default=no)

If "yes", then any existing output files will be over-written.

Parameter=verbose (integer default=0 min=0 max=5)

Program verbosity, 1 (least) to 5 (most).

Changes in CIAO 4.16


Bugs

There are no known bugs for this tool.

See Also

chandra
eventdef
dm
dmbinning
tools::composite
combine_grating_spectra
tools::core
dmcopy, dmextract
tools::gratings
detilt, dewiggle, symmetrize, tg_bkg, tg_choose_method, tg_create_mask, tg_findzo, tg_resolve_events, tgdetect, tgdetect2, tgextract2, tgidselectsrc, tgmask2reg, tgmatchsrc, tgsplit
tools::image
dmfilth, dmimghist, dmregrid
tools::response
mktgresp
tools::table
dmgroup, dmtype2split
tools::utilities
apply_fov_limits, get_fov_limits, get_sky_limits