Chandra X-Ray Observatory
	(CXC)
Skip to the navigation links
Last modified: 26 November 2017

URL: http://cxc.harvard.edu/csc/cli/index.html

The command-line interface to CSC 2.0


As an alternative to CSCview, tables of catalog data may be downloaded directly through a URL with Unix command-line tools such as cURL and Wget, using the ADQL query synytax or the IVOA-compliant VO Cone Search service.


ADQL Queries

All source properties contained in the catalog may be accessed directly from the following URL in an ADQL command-line query:

http://cda.cfa.harvard.edu/csccli/getProperties

This URL accepts the following required parameters:

version=cur

If the version is not set to cur (or current) then the query will be made against version 1.1 of the catalog.

query=...

The contents of the query which is written using ADQL syntax, which is introduced below. Certain applications, such as curl, let the query be defined in an file.

The following parameters are optional:

nullAppearance=...

The database contains columns which do not have a value for every row (e.g. the flux_aper_b value is not defined for a HRC-I observation). The default return for such values is the empty string, but this can be changed with the nullAppearance option.

floatFormat=...

The options for floating-point values is 'default' or 'native'. The native setting uses contains more significant digits, whereas the default setting may truncate the values.

coordFormat=...

The default format used for celestial coordinates (the RA and Declination colunms in a table) is sexagesimal. The options are 'sexagesimal' and 'decimal'.

outputFormat=...

The default format for the returned data is Tab Separated Values (TSV), using a CSC variant which also encodes metadata about the columns, such as units, description, and display type. The format can be changed to use VOTable TABLEDATA, a XML-based format defined by the IVOA. The format can be changed with the outputFormat parameter, which accepts one of: 'tsv', 'vot', or 'votable'.

The remaining parameters are required when using the cross-match functionality:

UPLOAD=...

Name,URI (Uniform Resource Identifier) pair pointing to user table. Name is required to be 'user_table'. URI maybe an external URL accessible by the CSC web server or an special URI scheme, 'param:', indicating the table is inline content within the HTTP POST request. Note that TSV and VOTable formats are supported for crossmatch.

The special URI scheme 'param:' is used to upload a user table inline. This scheme indicates that the value after the colon will be the name of the inline content. The content type used is multipart/form-data, using a 'file' type input element. The file 'name' attribute must match that used in the UPLOAD parameter.

RA=...

The column of Right Ascension source position coordinates in decimal degree units.

DEC=...

The column of Declination source position coordinates in decimal degree units.

The remaining parameters are optional when using the cross-match functionality:

ID=...

The cross match identification column name in the uploaded user table. If not specified, the row number is used as the user ID. The column can be either an integer value or a string.

SR=...

The search radius in units set by the SR_UNITS parameter, either a single value or name of a column containing multiple values, to apply to each source in the input list of source positions. Defaults to 3.0.

SR_UNITS=...

The units for the SR parameter, which defaults to 'arcmin'. The allowed values are: 'deg', 'arcmin', or 'arcsec'.

SIGMA=...

The 1-sigma radial Gaussian position in SIGMA_UNITS. If specified, a two-sided probability of position match can be calculated. If not specified, a one sided probability is used. Accepts a single value or name of a column containing multiple values, to apply to each source in the input list of source positions.

SIGMA_UNITS=...

The units for the SIGMA parameter, which defaults to 'arcsec'. The allowed values are: 'arcsec' or 'mas' (milli-arcseconds).

A successful call will return the results as formatted text, with the format defined by the value of the outputFormat parameter.


Crossmatch queries

When a user supplies a table for crossmatch via the UPLOAD parameter, an association table between the user_table and the CSC master_source table is called cross_match is available in ADQL.

The user table may contain arbitrary columns not all of which are used. The RA, DEC, and ID parameters map column names from the user table to the RA, DEC, and ID columns used by the crossmatch. The user table is not available for ADQL queries, the ID column is copied and can be queried from the cross_match table.

The cross_match table contains columns named:

Column name Column description
usrid A key copied from the user_table column identified by the ID parameter
msid A key to join on the master_source.msid column.
separation A floating point column representing the separation of the two sources in arcseconds.
probability A floating point column representing the probability that the two sources are the same.

Data product queries

The CSC data products - e.g. spectra, light curves, and images - may be accessed at the command line, but only after interactively querying the catalog within CSCview. After submitting a query in the GUI and selecting the desired data products, users are brought to the Products tab where they are given the option of downloading the selected data products via a download script or tar file. The download script contains a list of Wget commands, one for each file, which can be executed on the Unix command line for a batch download. See the CSC thread Retrieving Data Products to learn how to access catalog data products.


ADQL syntax

An ADQL SELECT statement returns a result set of records from one or more tables of astronomical data, located by the FROM clause. The available tables are:

ADQL name Description
master_source m Master Sources Table
stack_source s Stacked Observation Detections
observation_source o Per-Observation Detections
master_stack_assoc a Master Source/Stack Source Associations
stack_observation_assoc b Stack Source/Observation Source Associations
detect_stack d Detect Stack/Observation Associations
valid_stack v Valid Stack/Per-Observation Detection Associations
likely_stack l Likely Stack/Per-Observation Detection Associations

Some of the optional clauses of a SELECT statement include:

  • TOP specifies the number of rows to retrieve.
  • WHERE specifies which rows to retrieve, according to the search criteria.
  • ORDER BY specifies an order in which to return the rows.

ADQL SELECT, TOP, FROM, WHERE, ORDER BY statements may also be entered into the ADQL window in the Query tab of the data access GUI CSCview. This view is accessed by selecting the menu item View->Query->Show Language while the Query tab is open.


Examples

Perform a basic source property query.

unix% curl -o out.file --form query='SELECT TOP 50 m.name, m.significance, m.flux_aper_b, m.powlaw_gamma FROM master_source m WHERE (m.significance > 10.0 AND m.pileup_flag = 0 AND m.hard_hs > 0.7)' --form version=cur --form outputFormat=votable http://cda.cfa.harvard.edu/csccli/getProperties

unix% wget -O out.file 'http://cda.cfa.harvard.edu/csccli/getProperties?query=SELECT TOP 50 m.name, m.significance, m.flux_aper_b, m.powlaw_gamma FROM master_source m WHERE (m.significance > 10.0 AND m.pileup_flag = 0 AND m.hard_hs > 0.7)&version=cur&outputFormat=votable' 

This query accesses from the catalog the master source name, significance, broad-band aperture energy flux, and power-law model best-fit photon index for the first 50 master sources found with a significance greater than 10.0, a hard-to-soft hardness ratio greater than 0.7, and pileup fraction less than 10%. The result set is returned in VOTable TABLEDATA format.

Perform a basic cone search query.

unix% curl -o out.file --form query='SELECT m.name, m.ra, m.dec, m.flux_aper_b FROM master_source m WHERE dbo.cone_distance(m.ra,m.dec,83.77333,-5.68464)<=10' --form version=cur http://cda.cfa.harvard.edu/csccli/getProperties

unix% wget -O out.file 'http://cda.cfa.harvard.edu/csccli/getProperties?query=SELECT m.name, m.ra, m.dec, m.flux_aper_b FROM master_source m WHERE dbo.cone_distance(m.ra,m.dec,83.77333,-5.68464)<=10&version=cur' 

This query accesses from the catalog the master source name, R.A and Dec., and broad-band aperture energy flux for all sources located within 10 arcminutes of the celestial position R.A.=83.77333, Dec.=-5.68464 (in decimal degrees).

Upload a query from a file to the URL.

unix% curl -o out.file --form query=@query.txt --form version=cur http://cda.cfa.harvard.edu/csccli/getProperties

The file should contain a query written in ADQL 2.0, e.g. a query save file output by CSCview. An example of an ADQL CSC 2.0 query is cscquery.adql.

[NOTE]
Note

The query must be written as a single line - so now new-line or carriage-return characters - for the call to succeed.

Specify how a catalog value of NULL should appear in the output table (instead of a blank space).

unix% curl -o out.file --form nullAppearance=NULL --form query="SELECT DISTINCT o.obsid, o.obi, o.region_id, o.theta, o.mjr_axis_raw_s FROM observation_source o, stack_source s WHERE (o.instrument = 'ACIS' AND o.theta < 0.5 AND o.edge_code = 0 AND s.detect_significance_b > 10.0) ORDER BY theta DESC" --form version=cur http://cda.cfa.harvard.edu/csccli/getProperties

unix% wget -O out.file "http://cda.cfa.harvard.edu/csccli/getProperties?nullAppearance=NULL&query=SELECT DISTINCT o.obsid, o.obi, o.theta, o.mjr_axis_raw_s FROM observation_source o, stack_source s WHERE (o.instrument = 'ACIS' AND o.theta < 0.5 AND o.edge_code = 0 AND s.detect_significance_b > 10.0)&version=cur"

This query accesses from the catalog the ObsID, OBI, off-axis angle, and (full) source ellipse major axis length for all distinctly identified ACIS-observed sources found in the Source Observations Table with an off-axis angle less than 0.5, a detect significance greater than 10.0, and which do not dither off chip boundaries during the observation. Here, what normally would appear as an empty character in the output table will appear as "NULL".

Specify the output R.A. and Dec. coordinate format. Default is sexagesimal.

unix% curl -o out.file --form 'query=SELECT TOP 10 m.name, m.ra, m.dec FROM master_source m WHERE (m.significance > 10.0 AND m.pileup_flag = 0 AND m.hard_hs > 0.7)' --form coordFormat=decimal --form version=cur http://cda.cfa.harvard.edu/csccli/getProperties

unix% wget -O out.file 'http://cda.cfa.harvard.edu/csccli/getProperties?query=SELECT TOP 10 m.name, m.ra, m.dec FROM master_source m WHERE (m.significance > 10.0 AND m.pileup_flag = 0 AND m.hard_hs > 0.7)&version=cur&coordFormat=decimal'

This query accesses from the catalog the master source name, R.A. and Dec. for the first 1000 sources with a master source significance greater than 10.0, a hard-to-soft hardness ratio greater than 0.7, and a pileup fraction less than ~10%. The R.A. and Dec. values are output in decimal degrees.

Specify that floating point numbers should appear in native format.

unix% curl -o out.file --form 'query=SELECT top 10 o.obsid,o.obi,o.targname,o.ra_targ,o.dec_targ,o.ra_pnt,o.dec_pnt,o.roll_pnt,o.ra_nom,o.dec_nom,o.roll_nom,o.gti_start,o.gti_stop,o.gti_elapse,o.gti_end,o.gti_obs,o.gti_mjd_obs,o.mjd_ref,o.instrument,o.grating,o.datamode,o.readmode,o.cycle,o.exptime,o.timing_mode,o.ascdsver,o.caldbver,o.crdate,o.ao FROM observation_source o' --form floatFormat=native --form version=cur http://cda.cfa.harvard.edu/csccli/getProperties

This query accesses from the catalog many observation-specific properties for the first 10 sources found in the Source Observations Table, with source property values output in native floating point format (the "native" catalog format includes numbers with many significant digits, exactly as they are output by the catalog processing pipeline).

Use the current database view as opposed to a specific release view.

[NOTE]
Note

Until the full CSC 2.0 release in 2018, the version parameter must be set to curr (or current) in all queries to access the CSC 2.0 data.

The query has not been updated to match the CSC 2.0 schema.

unix% curl -o out.file --form "query=SELECT top 5 m.name,m.ra,m.dec,m.err_ellipse_r0,o.obsid,o.theta,o.flux_significance_b,o.conf_code,o.extent_code,o.sat_src_flag,o.mjr_axis_raw_b,o.mnr_axis_raw_b,o.pos_angle_raw_b,o.cnts_aper_b,o.src_rate_aper_b,o.src_rate_aper_hilim_b,o.src_rate_aper_lolim_b,o.flux_aper_b,o.flux_aper_hilim_b,o.flux_aper_lolim_b,o.hard_hm,o.hard_ms,o.var_index_b,o.livetime,o.detector,o.posid FROM master_obi_assoc a , master_source m , observation_source o WHERE ((a.match_type = 'u')) AND o.posid=a.posid AND m.msid=a.msid ORDER BY name ASC" --form version=cur http://cda.cfa.harvard.edu/csccli/getProperties

This query accesses source properties from both the Master Sources and Source Observations tables, for the first five unambiguously identified sources in the current catalog database (as opposed to a static 'release' version of the database, frozen in time). The results are sorted on the master source name column; an example of a CSC 2.0 master source name is '2CXO J082250.1+041151'.

Find CSC counterparts to sources of interest by performing a cross-match against the CSC using an input table of source positions.

unix% curl \
  --request POST \
  --form version=cur \
  --form 'query=select m.name,m.ra,m.dec,c.usrid,c.separation,c.probability from cross_match c, master_source m where c.msid = m.msid' \
  --form RA=ra \
  --form DEC=dec \
  --form ID=my_catalog_id \
  --form UPLOAD=user_table,param:my_catalog \
  --form my_catalog="@/data/somewhere/table.xml" \
  http://cda.cfa.harvard.edu/csccli/getProperties

This query accesses from the catalog the separation in arcseconds between the matched catalog master sources and the corresponding sources in a user-input table, along with the names and RA & Dec. positions associated with the catalog sources in the CSC Master Sources Table. The probability value of each source match is also returned, where a value of 1.0 indicates that the returned match is very likely a true match (down to many siginificant digits in source position), and a value of 0.0 represents a very unlikely true match. The my_catalog_id column values in the input table is returned in the c.usrid output column for the matching sources.


VO Cone Search

[IMPORTANT]
Important

The VO Cone Search service is not available for the CSC 2.0 release. It will be made available with the full catalog release in 2018.


Last modified: 26 November 2017
Smithsonian Institute Smithsonian Institute

The Chandra X-Ray Center (CXC) is operated for NASA by the Smithsonian Astrophysical Observatory. 60 Garden Street, Cambridge, MA 02138 USA.   Email:   cxchelp@head.cfa.harvard.edu Smithsonian Institution, Copyright © 1998-2017. All rights reserved.