Last modified: 22 Feb 2022

URL: https://cxc.cfa.harvard.edu/ciao/threads/regions/

Using CIAO Region Files

CIAO 4.16 Science Threads


Overview

Synopsis:

Regions - two-dimensional filters used to include and exclude data - are commonly used in CIAO data analysis. This thread describes how regions are defined, how they are used, and the caveats associated with their use.

We recommend that the "CIAO" region format be used, as it is supported by all of the CIAO data analysis tools.

Related Links:

Last Update: 22 Feb 2022 - Review for CIAO 4.14. Updated for Repro-5/CALDB 4.9.6


Contents


Getting Started

Download the sample data: 1712 (ACIS-S, 3C 273)

unix% download_chandra_obsid 1712 evt2

Coordinate Definitions: Image, Physical/Sky, Celestial, and WCS

There are a few terms needed to discuss coordinate systems:

Image

Image coordinates specify a location in an image. The lower left pixel of the array will have the image coordinates (1,1). The lower left corner of the array will have coordinates (0.5,0.5). Coordinates increase to the right in x and up in y.

Physical/Sky

The physical (x,y) coordinate system is called "sky" in shorthand. This is the system used most often in Chandra data analysis. The sky coordinates of an event are defined as the point where an event crosses the imaginary tangent plane for a given pointing of the telescope's mirrors. This is calculated (by the CIAO tools reproject_events and acis_process_events / hrc_process_events) from knowledge of the location of the event on the detector, the location of the detector in the observatory, and the pointing of the telescope. For Chandra data, the range of sky coordinates is 0.5:8192.5 for ACIS, 0.5:32768.5 for HRC-I, and 0.5:65536.5 for HRC-S.

CIAO regions created in ds9 are saved in physical coordinates by default. This format is recommended to ensure compatibility with the CIAO tools. The CIAO and ds9 region formats section of the Using CIAO Regions thread has a discussion on the ASCII region types available for use.

Celestial

The celestial coordinate system is the standard method of measuring positions in the sky. The transformations from sky (x,y) to celestial (RA, Dec) are described at the end of the event file "columns" listing:

unix% dmlist acisf01712N005_evt2.fits cols
...cut...

8:    EQPOS(RA ) = (+187.2760)[deg] +TAN[(-0.000136667)* (sky(x)-(+4096.50))]
           (DEC)   (+2.0552  )           (+0.000136667)  (   (y) (+4096.50)) 

The two systems are related by the aimpoint of the observation. When events are projected onto the celestial sphere, RA increases to the left (hence the minus sign in the definition) and Dec increases upward. In this example, the celestial coordinates of the reference pixel in degrees (J2000) are (187.2760,2.0552).

CIAO regions may be saved in celestial coordinates, but only sexagesimal (not decimal) coordinates are supported (e.g. circle(17:46:14.065,-28:51:39,1.0')).

WCS

The World Coordinate System (WCS) is a generalized name for the celestial coordinate system. The default coordinate system in CIAO and ds9 is the J2000 system. The physical (x,y) coordinate system is related to WCS by specifying the (RA,Dec) of a reference pixel in the file's header.

A complete description of the WCS standard is given on the WCSTools home page.

For a detailed discussion of the relationships between Chandra instrument coordinate systems, see the Chandra Coordinate Systems: Imaging document (PS, 25pp).


CIAO and ds9 Formats

Regions

There are several region formats available in ds9; the default region format of ds9 is "DS9/Funtools". We recommend that the "CIAO" region format be used, as it is supported by all of the CIAO data analysis tools. A tool will exit with an error if given an unsupported region format.

However, all the basic ds9 region shapes - including circular annuli - will work with the CIAO tools. The more complicated regions - elliptical and box annuli, the panda shape, and format used for writing grids of annuli - will not work properly.

In general, the ds9 region format has many more options for storing display properties (color, coordinate system, comments), while the CIAO region format is a more simplified, stripped down version. To compare the same regions in each format:

  • Load the event file into ds9:

    ds9 acisf01712N005_evt2.fits &
    
  • Left-click and drag on the image to create a region. The shape of the region is set from the "Region → Shape" menu and the color can be changed from "Region → Color". This FAQ explains on how to rotate shapes in ds9. The two regions we defined are shown in Figure 1.

    Figure 1: Image with source and jet regions defined

    [Thumbnail image: The source and jet are marked by a green circle and a red rotated rectangle, respectively.]

    [Version: full-size]

    [Print media version: The source and jet are marked by a green circle and a red rotated rectangle, respectively.]

    Figure 1: Image with source and jet regions defined

    Left-click and drag on the image in the ds9 display to create a region.

  • To save the regions:

    1. Region → Save Regions... → Save As "ds9.reg".
    2. After choosing "OK" in the region filename dialog, a format dialog is opened. Set the format to "ds9" and the coordinate system to "Physical".

    Then repeat:

    1. Region → Save Regions... → Save As "ciao.reg".
    2. After choosing "OK", set the format to "CIAO" and the coordinate system to "Physical".

Now we can compare the two files. First, the ds9 format:

unix% cat ds9.reg 
# Region file format: DS9 version 4.1
global color=green dashlist=8 3 width=1 font="helvetica 10 normal roman" select=1 highlite=1 dash=0 fixed=0 edit=1 move=1 delete=1 include=1 source=1
physical
circle(4082.1758,4076.2896,15.927805)
box(4105.9563,4050.8017,28.609756,12.512195,311.16939) # color=red

The first non-comment line is a declaration of "global" conditions, such as default color and editing flags. The next line indicates that the coordinate system is "physical" (a.k.a. "sky"). Finally, the color for the red box is defined after the pound symbol.

The same regions in CIAO format look like:

unix% cat ciao.reg 
# Region file format: CIAO version 1.0
circle(4082.1758,4076.2896,15.927805)
box(4105.9563,4050.8017,28.609756,12.512195,311.16939)

Since the "physical" coordinate system is assumed for all CIAO regions, it is not indicated for each shape. Notice that the global conditions and colors are also omitted. The ds9 box is called rotbox in CIAO, but they are otherwise identical.


Files

The term "format" may also be used to describe the filetype used to store the region information; the two allowed formats are ASCII and FITS. The region files are loaded in the same way:

  • ASCII files:

    Load the region file from the ds9 menu "Regions" → "Load Regions..." → ascii_file.reg.

  • FITS files

    Load the region file from the ds9 menu "Regions" → "Load Regions..." → fits_file.fits

    If the region definition is saved in the REGION or SRCLIST extension of the FITS file, ds9 will automatically find the correct extension. Otherwise, include it in the filename: "fits_file.fits[blockname]"

Several examples of working with region files are available in the Using the Output of Detect Tools thread.


Regions with Multiple Shapes vs. Region Stacks

A region file can define a single region that has multiple shapes. When you use the region file in an analysis task, CIAO decides whether or not points are in the region, but does not return any information about in which of the shapes a particular point falls. A region stack, however, contains multiple regions. CIAO treats each of the regions separately, essentially running the tool for each region in the stack.

This region file:

unix% cat circles.reg 
# Region file format: CIAO version 1.0
circle(4066,4191,12)
circle(4085,4087,20)
circle(4107,4059,13)

may either be considered a single region composed of three circles or a stack of regions, each of which is a single circle. How CIAO interprets this file depends on the syntax used in inputting it to a tool.

A common use of region stacks is in dmextract, where binning on a region creates a histogram. First we tell dmextract to treat the file as a stack by using the @ syntax:

unix% punlearn dmextract
unix% dmextract "acisf01712N005_evt2.fits[bin sky=@circles.reg]" circles_stack.fits opt=generic

This command operates on the region stack and make three histograms (rows), one for each circle, in the output file:

unix% dmlist circles_stack.fits blocks
 
--------------------------------------------------------------------------------
Dataset: circles_stack.fits
--------------------------------------------------------------------------------
 
     Block Name                          Type         Dimensions
--------------------------------------------------------------------------------
Block    1: PRIMARY                        Null        
Block    2: HISTOGRAM                      Table        18 cols x 3        rows
Block    3: GTI7                           Table         2 cols x 1        rows
Block    4: GTI4                           Table         2 cols x 2        rows
Block    5: GTI5                           Table         2 cols x 1        rows
Block    6: GTI6                           Table         2 cols x 2        rows
Block    7: GTI8                           Table         2 cols x 1        rows
Block    8: GTI9                           Table         2 cols x 1        rows

To compare, replacing the @ with region() tells CIAO to treat the shapes as a single region:

unix% dmextract "acisf01712N005_evt2.fits[bin sky=region(circles.reg)]" circles_region.fits opt=generic

The output file contains only one histogram, instead of three:

unix% dmlist circles_region.fits blocks

--------------------------------------------------------------------------------
Dataset: circles_region.fits
--------------------------------------------------------------------------------
 
     Block Name                          Type         Dimensions
--------------------------------------------------------------------------------
Block    1: PRIMARY                        Null        
Block    2: HISTOGRAM                      Table        18 cols x 1        rows
Block    3: GTI7                           Table         2 cols x 1        rows
Block    4: GTI4                           Table         2 cols x 2        rows
Block    5: GTI5                           Table         2 cols x 1        rows
Block    6: GTI6                           Table         2 cols x 2        rows
Block    7: GTI8                           Table         2 cols x 1        rows
Block    8: GTI9                           Table         2 cols x 1        rows

Excluded Regions

The most important thing to remember when working with excluded regions is that order matters. When generating regions with excluded shapes, the order of shape generation is relevant to how they are interpreted by the software.

For similar reasons, CIAO does not understand a region file in which the first shape is negative. Since the software processes the shapes in their listed order, CIAO attempts to exclude this shape from an area that has not yet been defined. The field() region, which defines the entire field of view, can be added to the beginning of the region file to avoid syntax errors; see "ahelp dmregions" and this FAQ for more information.

The regions are saved by ds9 in the order in which they were created. For example, Figure 2 shows three regions, one of which is marked as excluded (shown by the red strike through it) via the "Region → Properties" menu.

Figure 2: Three regions, one of which is excluded

[Thumbnail image: Three regions, one of which is marked as an excluded region by a red strike through it]

[Version: full-size]

[Print media version: Three regions, one of which is marked as an excluded region by a red strike through it]

Figure 2: Three regions, one of which is excluded

The red strike through region 2 indicates that it is excluded.

If the regions had been defined with the smallest (#1) first, the excluded region (#2) second, and the largest region (#3) third, the file would look like:

unix% cat exclude_one.reg
circle(4086,4084,20)
-circle(4102,4128,100)
circle(4074,4120,200)

This tells CIAO to take the first region (#1), subtract the second (#2), then add in the third (#3). Running dmcopy on this region file:

unix% dmcopy \
      "acisf01712N005_evt2.fits[sky=region(exclude_one.reg)][bin sky]" one_image.fits

unix% ds9 one_image.fits &

produces Figure 3.

Figure 3: Incorrect order of processing regions

[Thumbnail image: A circular section of the field of view with no interior regions filtered out of it.]

[Version: full-size]

[Print media version: A circular section of the field of view with no interior regions filtered out of it.]

Figure 3: Incorrect order of processing regions

If the regions in the region file (.reg) are ordered incorrectly, the excluded region is "lost" because the largest circle is included last.

A more sensible order of regions for this example might be:

unix% cat exclude_two.reg
# Region file format: CIAO version 1.0
circle(4074,4120,200)
-circle(4102,4128,100)
circle(4086,4084,20)

which starts with the largest region (#3), removes the second (#2), and then adds back in the third (#1). Running dmcopy again:

unix% dmcopy \
      "acisf01712N005_evt2.fits[sky=region(exclude_two.reg)][bin sky]" two_image.fits

unix% ds9 two_image.fits &

creates Figure 4.

Figure 4: Correct order of processing regions

[Thumbnail image: A circular section of the field of view with a circular region excluded from it and a smaller circular region included within that.]

[Version: full-size]

[Print media version: A circular section of the field of view with a circular region excluded from it and a smaller circular region included within that.]

Figure 4: Correct order of processing regions

When the regions in the region file (.reg) are ordered properly, the second region is excluded while the data from the first region is included within the excluded section.


Region Area Calculation

CIAO calculates the area of a region differently depending on whether it is considered simple or complex. The calculation of simple areas - those of basic, well-separated shapes - is done analytically. For complicated regions - where shapes overlap or are close enough that it's hard to determine if they are separate - CIAO divides up the area into bins (which may be smaller than a single pixel). It then counts the number of bins which are true for the test "I am inside this region." This approach has a finite error, which is usually on the order of 1% percent of the region area.

A useful way to evaluate region areas is the subspace option in dmlist:

unix% dmlist "acisf01712N005_evt2.fits[sky=circle(4086,4084,20)]" subspace | less
 
--------------------------------------------------------------------------------
Data subspace for block EVENTS: Components: 6 Descriptors: 15 
--------------------------------------------------------------------------------
 
 --- Component 1 --- 
...
   8 sky                  Real4               Circle(4086,4084,20)
   8 sky                  Real4               Field area = 6.71089e+07 Region area = 1256.64
...

The "Region area" calculated for a circle of radius 20 pixels is 1256.64 square pixels, which is equal to the analytical evaluation of the circle's area.

Region areas are used in the BACKSCAL keyword added to PHA and PI spectral files generated by dmextract. The error in the area is almost always small compared to other calibration uncertainties, but be particularly careful when making regions comparable in size to the pixel size. Although filtering does occur in exactly the area specified when applying such a region to a file which contains fractional pixel positions, the user should be careful about interpretation.

dmextract BACKSCAL calculation makes used of a fixed area size. This is included in the $ASCDS_INSTALL/data/cxo.mdb file

unix% cat $ASCDS_INSTALL/data/cxo.mdb
!
! This file is in the spirit of the 
! XSELECT MDB file.
!
! dmextract uses TELESCOP:INSTRUME:colname_binning
!
Chandra:ACIS:pha_binning 1:4096:2
Chandra:ACIS:pi_binning 1:1024:1
Chandra:ACIS:wmap chip=32
Chandra:ACIS:field 8192

History

03 Jan 2005 reviewed for CIAO 3.2: no changes
02 Dec 2005 updated for CIAO 3.3: ds9 v4.0 region format is slightly different than v3.0.
01 Dec 2006 reviewed for CIAO 3.4: no changes
09 Jan 2008 updated for CIAO 4.0: ds9 now automatically looks for the "[REGION]" or "[SRCLIST]" extension in the region file, so it doesn't have to be specified; filename and region files updated for reprocessed data (version N005 event file)
12 Jun 2008 updated image display to place figures inline with text
02 Jan 2009 updated for CIAO 4.1: moved ds9 region comparison to "ds9 regions: v3 vs v4" section; ds9 region output updated to version 4.1
09 Feb 2010 updated for CIAO 4.2: changes to the ds9 region file format menu; removed outdated "ds9 regions: v3 vs v4" section
11 Jan 2011 updated for CIAO 4.3: use opt=generic when making a radial profile to avoid a warning from dmextract
05 May 2011 updated the Coordinate Definitions section
03 Jan 2012 reviewed for CIAO 4.4: no changes
03 Dec 2012 Review for CIAO 4.5; added new dmextract cxo.mdb entry to ensure field size is fixed.
03 Dec 2013 Review for CIAO 4.6; no changes.
22 Dec 2014 Review for CIAO 4.7; no changes.
22 Feb 2022 Review for CIAO 4.14. Updated for Repro-5/CALDB 4.9.6