Chandra X-Ray Observatory
	(CXC)
Skip to the navigation links
Last modified: November 2015

URL: http://cxc.harvard.edu/ciao/ahelp/download_chandra_obsid.html
AHELP for CIAO 4.9

download_chandra_obsid

Context: tools

Synopsis

Download public data by ObsId from the Chandra archive.

Syntax

download_chandra_obsid obsid
download_chandra_obsid obsid1,..,obsidN
download_chandra_obsid obsid filetype
download_chandra_obsid obsid1,..,obsidN filetype1,..,filetypeN

The supported file types are:
aoff aqual asol bias bpix cntr_img dtf eph0 eph1 evt1 evt1a evt2 flt
fov full_img msk mtl oif osol pbk pha2 plt readme adat soff src2
src_img stat sum vv

The -m or --mirror flags allow you to use a mirror of the Chandra Data
Archive.
The -h or --help flags displays information on the command-line options.
The -q or --quiet flags is used to turn off screen output.

Description

The download_chandra_obsid tool will download the data for the given Chandra Observation Id (ObsId) from the public archive. It can not be used to download proprietary data. Multiple values can be given - for both the ObsId and filetype lists - either as a comma-separated list or in a separate file (using the "@" syntax as described in 'ahelp stack').

Where are the files placed?

The files are stored in the current working directory, using the Observation Id as a directory name: so the data for ObsId 1843 would be placed in the directory 1843/. Within this directory the files are placed as they are in the archive, namely a top-level directory containing the oif.fits and V&V report, with the remaining data in the primary/ and secondary/ sub-directories.

How do I download a specific file?

The optional second argument to download_chandra_obsid allows you to restrict the download to one or more files. A comman-separated list (or stack) of file types can be given, so

unix% download_chandra_obsid 1843 evt2,asol,bpix,fov

will just download the evt2, asol, bpix, and fov files for ObsId 1843.

What happens when files already exist?

The tool will skip downloading any files that already exist on disk as long as they are the correct size. If the on-disk version is smaller than the archive version, then download will be resumed rather than started again. A warning message will be displayed if the on-disk version is larger than the archive version.

The behavior described above means that you can interrupt a download and re-start it and will not have to re-download existing data.

Screen output

Unless the -q or --quiet flag was used, each file that is downloaded will be displayed on screen, giving the "type", format, file type before a progress bar of # marks (each # mark indicates 10% of the file). Once the file has been downloaded an average rate is displayed for the download. Examples of the output for two files are shown below:

Type     Format      Size  0........H.........1  Download Time Average Rate
---------------------------------------------------------------------------
vv       pdf        78 Kb  ####################          < 1 s  843.1 kb/s
evt2     fits       17 Mb  ####################            2 s  7170.3 kb/s

If a file has already been downloaded then the message "already downloaded" will be displayed instead of the progress bar.

Example 1

unix% download_chandra_obsid 1843

This will download all the data files for ObsId 1843 to the directory 1843/. The screen output for the run will look something like the following (where many lines have been excluded, indicated by the "..." lines):

Downloading files for ObsId 1843, total size is 81 Mb.

Type   Format   Size  0........H.........1  Download Time Average Rate
----------------------------------------------------------------------
readme ascii   11 Kb  ####################          < 1 s  433.7 kb/s
...
evt2   fits    17 Mb  ####################            4 s  4740.2 kb/s
asol   fits     2 Mb  ####################          < 1 s  5239.1 kb/s
bpix   fits    13 Kb  ####################          < 1 s  575.7 kb/s
fov    fits     6 Kb  ####################          < 1 s  610.6 kb/s
...
osol   fits   354 Kb  ####################          < 1 s  1764.3 kb/s

    Total download size for ObsId 1843 = 81 Mb
    Total download time for ObsId 1843 = 16 s

Example 2

unix% download_chandra_obsid 1843 evt1,bpix,asol,fov,readme

Here we restrict the download to just the evt1, bpix, asol, fov, and readme files from ObsId 1843.

Example 3

unix% download_chandra_obsid 1842,1843

This will download the data for ObsIds 1842 and 1843 to the directories 1842/ and 1843/ respectively.

Example 4

unix% cat obsids
1842
1843
unix% download_chandra_obsid @obsids

Here we use the support for stacks (see 'ahelp stack' for more information) to download the data for the ObsIds listed in the file obsids, in this case 1842 and 1843.

Example 5

unix% download_chandra_obsid 1842,1843 evt1,asol -q

Here we download only the evt1 and asol files from the two ObsIds and use the -q flag to turn off the screen output.

Example 6

unix% download_chandra_obsid -m ftp://cda.cfa.harvard.edu/pub 1842

Download ObsId 1842 from the given CDA mirror site (in this example we have used the location of the actual CDA archive). If the observation is not found at the mirror site then the script will skip it (i.e. it does not fall back to trying the CDA site).

An alternative is to use the CDA_MIRROR_SITE environment variable, so that the above could also be written as:

unix% setenv CDA_MIRROR_SITE ftp://cda.cfa.harvard.edu/pub
unix% download_chandra_obsid 1842

Mirror sites of the Chandra Data Archive

If you have set up, or have access to, a mirror of the Chandra Data Archive (it can be only a partial archive), then you can use the --mirror command-line argument, or the CDA_MIRROR_SITE environment variable, to tell the script to use this mirror instead. The mirror option takes precedence over CDA_MIRROR_SITE if both are set.

If an ObsId can not be found on the mirror site then it is skipped; i.e. the script will not also check the CDA site for the data.

The mirror location should include the URL of the FTP site (the leading ftp:// is required) and the path up to, but not including, the byobsid/ directory. So, for the Chandra Data Archive itself you would use

ftp://cda.cfa.harvard.edu/pub

(although obviously in this case you do not need to use the mirror option). Please see the documentation for the mirror site to find out the correct path to use.

If needed, you can include a username and password in the mirror setting, following RFC3986, for instance

ftp://anonymous:foo@bar.com@cda.cfa.harvard.edu/pub

Changes in the scripts 4.9.3 (May 2017) release

Updated to use ftp://cda.cfa.harvard.edu as the default FTP server.

Changes in the scripts 4.8.1 (December 2015) release

The code has been updated to avoid warning messages from NumPy version 1.9. There is no difference to how the script behaves.

Changes in the scripts 4.6.5 (June 2014) release

Added evt1a and adat file types

It is now possible to select evt1a and adat (PCAD Level 1 ACA image) files using the filetype argument. The adat files can be used with the monitor_photom script.

Changes in the scripts 4.5.4 (August 2013) release

Support for CDA mirror sites

The --mirror command-line argument and support for the CDA_MIRROR_SITE environment variable has been added to allow data access from a mirror of the Chandra Data Archive site.

Support for the 00README files

The new 00README files are included in the download, and can be explicitly selected using the readme filetype.

Notes

This script is not an official part of the CIAO release but is made available as "contributed" software via the CIAO scripts page. Please see the installation instructions page for help on installing the package.

Bugs

There are no known bugs for this tool.

See Also

contrib
cda_data, cda_search
tools
download_obsid_caldb, find_chandra_obsid, list_datasetid, obsid_search_csc, search_csc, splitobs

Last modified: November 2015
Smithsonian Institute Smithsonian Institute

The Chandra X-Ray Center (CXC) is operated for NASA by the Smithsonian Astrophysical Observatory. 60 Garden Street, Cambridge, MA 02138 USA.   Email:   cxchelp@head.cfa.harvard.edu Smithsonian Institution, Copyright © 1998-2017. All rights reserved.