Chandra Source Catalog Release 2.1

Release 2.1 Introduction

The Chandra Source Catalog version 2.1 (CSC 2.1) expands version 2.0 by adding observations released publicly between January 1, 2015 and December 31, 2021. CSC 2.0 processed observations released publicly through the end of 2014. Compared to CSC 2.0, CSC 2.1 increases the number of sources in the catalog by about one-third, augmenting the total number of observations included from 10,382 to more than 15,000 and the number of observation stacks from 7,287 to about 10,000. A significant fraction of the new observations added represent new sky coverage since CSC 2.0. The total sky coverage of CSC 2.1 is roughly 800 square degrees, compared to ~560 square degrees in CSC 2.0. In Figure 1 we compare the coverage of the two versions.

Figure 1: Unique sources in CSC 2.1 compared to CSC 2.0

[Version: full-size]

Figure 1: Unique sources in CSC 2.1 compared to CSC 2.0

CSC 2.0 (blue) and CSC 2.1 (orange). Each circle represents an observation stack, with the size of the cricle indicating the total number of observations included in the stack.

What is New in CSC 2.1

Besides the significant increase in the number of processed observations, CSC 2.1 includes a number of algorithmic updates specifically aimed at reducing the amount of human intervention in processing, and making source fitting, photometry, and variability estimates more robust. The updates include:

• Stacked observation-level astrometric correction: CSC 2.1 observation stacks with sufficient CSC/Gaia matches (the majority of stacks) are now tied to the Gaia-CRF3 astrometric reference frame, which provides an absolute astrometric reference for the entire catalog. The transformation of coordinates is performed using a weighted least square approach. The resulting coordinate shifts are typically 0.1-0.2 arcsec, but can be greater than 2 arcseconds in a very small percentage of the cases. Figure 2 shows the effect of the stack astrometry correction for one particular extragalactic field, as a reduction of the overall separation between CSC stack detections and Gaia sources. CSC 2.1 sources that have had their astrometry updated will have non-zero or NULL values in the deltax and deltay columns in the Stack Detection Table. Details can be found in the "Tying CSC Astrometry to the Gaia Reference Frame" memo.
• Improved MCMC sampling for aperture photometry using pymc3: For dim sources that fail to converge with the CSC 2.0 photometry algorithms, in CSC 2.1 we sample the marginalized posterior for the fluxes and hardness ratios using a more robust MCMC algorithm (NUTS). This improves our ability to provide fluxes in the low-count regime. Figure 3 shows the effect of this improvement. Details can be found in the "Implementing pymc3 in CSC 2.1 Aperture Photometry" memo.
• Improved MLE fitting: When validating candidate detected sources that are close together, we now fit them simultaneously with a model that accounts for the multiplicity (previously that candidate detections were fitted individually in order of decreasing observed counts). This makes fitting more efficient, with less manual intervention required during quality assurance.

Figure 2: Distribution of Angular Separation Between Gaia & CSC Matched Sources

Figure 2: Distribution of Angular Separation Between Gaia & CSC Matched Sources

The distribution of separations between matching Gaia and CSC sources before (blue) and after (orange) the stack astrometry correction is applied. For separations larger than ~0.7 arcsec the histogram fraction is somewhat overestimated in this figure due to an increased percentage of poor matches.

Figure 3: Improvements in Aperture Photometry with Low-Counts

[Version: full-size]

Figure 3: Improvements in Aperture Photometry with Low-Counts

Improvements in aperture photometry result in better constrained photometry in the low-counts case. The left panel shows the marginalized probability density function (MPDF) for the same low-count detections in CSC 2.0 and CSC 2.1. The CSC 2.1 version is able to constrain a flux, as opposed to providing only an upper limit. The right panel shows the distribution of ultra-soft band fluxes for the two versions. More detections with few counts have constrained fluxes in CSC 2.1 compared to CSC 2.0.

Unmodified, Modified, and New Stacks

Adding new observations implies that some of the observation stacks will be updated to include added observations. As a reminder, observations obtained using the same instrument with pointings that all fall within 1 arcmin of each other are stacked (co-added) prior to source detection. This means that in CSC 2.1 there are three type of stacks, depending on whether they are different from CSC 2.0 stacks:

• Unmodified stacks: These stacks do not have added observations as part of CSC 2.1. At the stack level, they will only have their astrometry corrected to reflect the matching with Gaia. At the master level, and in a minority of cases, if a new stack overlaps the (unmodified) stack in question, the master level properties of detections in the stack might slightly change to reflect information from the added stack(s).
• Modified stacks: These are existing CSC 2.0 stacks that have new observations added to them as part of CSC 2.1. In addition to the coordinate update, the stack and master properties of the sources in these stacks might differ from CSC 2.0, since the addition of new observations can affect detectability, photometric, spectral, and temporal variability properties. Note that some of the added observations might include new sky regions that were not present in the prior CSC 2.0 stack. In CSC 2.1, modified stacks can be identified by stack names that end in stack version _002.
• New stacks: These are stacks made up of observations that do not overlap CSC 2.0 observations within 1 arcmin.

Note

The addition of new observations in a modified stack will result in the detection of new sources in regions of the sky previously covered by CSC 2.0, but also, in rare cases, in the disappearance of some CSC 2.0 sources, due to the change in the detectability conditions resulting from adding new observations.

In Figure 4 we show an example of the effect of new observations being added to a stack. We plot the location of CSC sources detected in original stack acisfJ0956451p284947_001 for CSC 2.0 (upper panels) and modified stack acisfJ0956451p284947_002 for CSC 2.1 (lower panels), color-coded by both master source significance (left) and master source b-band flux (right). The original stack was made up of a single observation in CSC 2.0, but four more observations have been added to the modified stack in CSC 2.1. The gain in the total number of sources resulting from this addition is evident in the plot, but there are also other notable points: first, some newly detected CSC 2.1 sources with considerable significance have master source CSC 2.1 broad energy band fluxes that are upper-limits consistent with zero flux. In all cases, there is at least one other energy band with a non-zero measured flux at the master source level. There are also a couple of cases of sources that were in CSC 2.0 but are not detected in CSC 2.1. As mentioned above, this is due to the detection thresholds changing as a result of new added observations.

Figure 4: Significance and Flux of Stack Detections

[Version: full-size]

Figure 4: Significance and Flux of Stack Detections

Sources detected in CSC 2.0 (upper panels) for original (single observation) stack acisfJ0956451p284947_001 and in CSC 2.1 (lower panels) for modified (five observation) stack acisfJ0956451p284947_002. The dots are color coded for significance (left panels) and broad band flux (right).

CSC 2.1 Processing and the Current Database View

Data that complete processing are made immediately available to the end users via the Current Database view of the catalog (see below). Processing follows a modified chronological order that prioritizes new sky coverage, and that also interleaves processing of stacks with many observations that take longer to complete. The Current Database view is accessible via the CSCview application, and also using VO tools, such as astroquery.

The catalog Current Database view originally contained copies of all of the sources, detections, and data products present in CSC 2.0. As stacks are processed through the CSC 2.1 pipelines, the existing data are replaced in the Current Database view with their CSC 2.1 counterparts. At any instant in time, the Current Database view will therefore contain sources, detections, and data products in different states of completion, depending on the processing status of their associated stacks.

There are three possible processing states for each stack:

1. Pending — the stack has not started processing; all database entries and data products for the sources, stack detections, and per-observation detections match their CSC 2.0 values.
2. Processing — the stack has started but not yet completed processing; database entries and data products for the master sources, stacked-observation detections, and per-observation detections will change as pipelines complete and the data ingest, and may be inconsistent (e.g., some observation-level data/products have been updated but higher level products have not).
3. Completed — the stack has completed processing and all database entries are updated and data products ingested in the CSC 2.1 Current Database, at the observation, stack, and master levels. Most of the data will have their final CSC 2.1 values, although there will be exceptions because final catalog-level QA and updates have not been applied (known expected changes, such as the update to include the systemic astrometric error, will be included in the documentation). Because in a very small fraction of cases there will be stacks that require reprocessing, we provide the date on which the stack processing was "completed".

To support users in their interpretation of Current Database view entries before the full release of CSC 2.1 is completed, we provide a list of all the CSC 2.1 stacks and their current processing status, updated daily. The status list is accessible at: Processing status of CSC 2.1.

Note

Note that even for stacks with no additional observations added as part of CSC 2.1, there will still be an astrometric update applied as part of CSC 2.1, since we have tied the catalog astrometry to the Gaia-CRF3 reference frame. These stacks will still appear as Completed once the astrometric update has been applied.

For sources and detections in stacks that have been modified by the addition of new observations, these sources and detections will have their properties updated in the master sources, stacked-observation detections, and per-observation detections tables, since the addition of new observations will affect the estimations of their average fluxes, variabilities, hardness ratios, etc. There will also be new sources added in CSC 2.1 whose unequivocal detection is only possible with the addition of the new observations, and that were therefore not present in CSC 2.0. Finally, there will be CSC 2.1 sources in new stacks that were also not part of CSC 2.0.

CSC 2.1 Visualization using WWT

We proved a visualization of the stacks according to their processing state in our WWT interface. The color code is as follows:

• yellow stacks have completed processing (state "Completed")
• red stacks are currently processing (state "Processing")
• white stacks have not yet started processing (state "Pending")

The WWT visualization can be accessed via Explore CSC 2.1 using WWT—CSC.

Figure 5: WWT Visualization of CSC 2.1

[Version: full-size]

Figure 5: WWT Visualization of CSC 2.1

A snapshot of the WWT visualization of CSC 2.1. Stacks that have completed processing are shown in yellow. Stacks that are currently processing are marked in red, and stacks that have not yet started processing are indicated in white. By clicking on a stack, the user can see if the stack is new to CSC 2.1, or if it has been modified with respect to CSC 2.0.

How to Find Out the Processing Status of a CSC 2.1 Source

Important Note on Completion of Processing

Note that during CSC 2.1 processing and until a frozen version of the catalog release is provided, the CSC 2.1 Current Database view will include a mix of sources, detections, and data products with different processing states. Therefore, users interested in carrying out statistical studies should not use the Current Database view. Instead, we encourage those users to use the CSC 2.0 release until the final, frozen version of CSC 2.1 has been released.

Suppose you are interested in a particular X-ray source, and you want to know whether this source is new to CSC 2.1, or if it has been processed with the CSC 2.1 pipelines. Here are some steps to find out the processing status of a source.

Using CSCview

1. Get the CSC source name. For example, suppose you are interested in source 2CXO J064004.5-511356.

2. Select the Current Database view in the Catalog tab in CSCview and click the Search button

   

3. In the CSCview Query tab, scroll down to the Master Source/Stacked Observation Detection Associations table in the Source Properties selection area, drag the four properties in the Result Set area of CSCview, and then click the Search button. This will provide the assocation between the source name and the corresponding stack(s) where the source was detected.

   

4. In this example, there are four stacks in which this source was detected. Since these stacks physically overlap each other on the sky, they will all be processed together in CSC 2.1. Those stacks will transition to the "Completed" state only after all overlapping stacks have completed processing and been ingested into the database. (This is necessary because at the Master level, we need information from all stack detections of a given source to compute its properties.)

   

5. Now you can pick one of the stacks and look for the corresponding stack name in the Processing status of CSC 2.1 page. As you can see, this particular stack is new, and has completed processing. Since all stacks that overlap the source must have completed processing, as described above, then the properties of our source of interest, together with the associated stacked-observation detections, per-observation detections, and data products, have been updated and are internally consistent (note that caveats described under the definition "Completed" status above.

   

---

Using VO Interfaces (Jupyter Notebook)

The next section provides more details about accessing CSC 2.1 using VO interfaces. For the purposes of checking the processing status of a source, you can use VO interfaces via a Jupyter notebook to find the source/stack associations:

1. Perform the relevant imports, in particular pyvo.

   

2. Setup up the query

   

3. Run the query with pyvo:

   

4. If preferred, transform into a dataframe with pandas for easier visualization:

   

---

Using WWT

You can also visually look for the source in WWT, and see the status of the stacks that include it. The source position can be derived (approximately) from the source name and entered into WWT. The stack processing status can be accessed by clicking on the corresponding stack:

Accessing CSC 2.1 Through the VO Services (TAP, Cone Search, etc.)

Just like with CSC 2.0, you can access the CSC 2.1 tables and data products using VO-compatible services, such as Table Access Protocol (TAP) and Cone Search, for example, from a Jupyter notebook. The CSC 2.1 interfaces for these services are:

• Table Access Protocol (TAP)
  • The base URL for the Chandra Observation Catalog TAP service is:

    
    		  https://cda.cfa.harvard.edu/csc21_snapshot_tap
    		

  • The base URL for executing asynchronous queries against the Chandra Observation Catalog TAP service is:

    
    		  https://cda.cfa.harvard.edu/csc21_snapshot_tap/async
    		

  • The base URL for executing synchronous queries against the Chandra Observation Catalog TAP service is:

    
    		  https://cda.cfa.harvard.edu/csc21_snapshot_tap/sync
    		

• Simple Cone Search (SCS)

  The base URL of the Chandra Observation Catalog Simple Cone Search is:

  		
  		https://cda.cfa.harvard.edu/csc21_snapshot_scs/coneSearch
  	      

• Simple Image Access Protocol (SIAP)

  The base URL of the Chandra Observation Catalog Simple Image Access is:

  
  		https://cda.cfa.harvard.edu/csc21_snapshot_siap/queryImages
  	      

Note that the table names should be preceded by the schema csc21_snapshot instead of csc2. You can find tutorials on how to use these services using Jupyter notebooks in CSC 2: Accessing release 2.0 of the Chandra Source Catalog with PyVO and CIAO Tools.