Both PDF and plain text versions of this document are available.

Summary
=======

download_chandra_obsid is a simple command-line script that allows
a user to download public data from Chandra by giving a list of
Observation Ids. The files are downloaded to the current working
directory, and stored using the same layout as the Chandra archive;
for example, ObsId 1843 would be stored in

    1843/...
    1843/primary/...
    1843/secondary/...

An optional list of file types can be given, which restricts the
data to be downloaded (by default all available data for the
Observation Id is downloaded).

WARNING
=======

As described below, this script is not an official product of the
CXC and is provided with limited support. A more-complete solution
is being developed by the Chandra Data Archive for release in
2010.

Download
========

This is version 2010.05.04 of download_chandra_obsid.

Individual files
----------------

-   download_chandra_obsid (use the "Save linked file as" option,
    or equivalent, to save this file as download_chandra_obsid)
-   readme.txt (this file)
-   COPYING

Tar file
--------

-   download_chandra_obsid.tar.gz (version 2010.05.04)

The only file you need to download is the script itself, namely
download_chandra_obsid.

Requirements
============

The script is written in Python and should work using the
CIAO-provided version (CIAO 4.2 provides version 2.6.2). There has
been limited testing of the script using version 2.5.1 of Python.

If it is given the executable bit - e.g.

    unix% chmod u+x download_chandra_obsid

then it can be run as a normal UNIX command, or you can say

    unix% python download_chandra_obsid

The script can be renamed if you object to typing such a long name;
for instance to get_obsid.

The examples below assume that the script is in the current working
directory, but it can be placed anywhere, such as one of the
directories in your $PATH.

Examples
========

Download a single ObsId
-----------------------

    unix% ./download_chandra_obsid 1843

The data will be stored in the directory 1843/ and will contain all
available data.

Download multiple ObsIds
------------------------

    unix% ./download_chandra_obsid 1842,1843

To download multiple ObsIds you can run the tool multiple times or
use a comma-separated list as shown above. In this example the data
will be stored in the directories 1842/ and 1843/.

Note that the ObsId values do not need to be listed in ascending
order.

Restrict the download
---------------------

    unix% ./download_chandra_obsid 1843 fov,bpix,evt1,flt

Here only the FOV, BPIX, EVT1 and FLT files will be downloaded. The
data will be stored in the relevant directores within 1843/ (they
will be created if need be).

Restrict the download (II)
--------------------------

    unix% ./download_chandra_obsid 1843,1842 fov,bpix,evt1,flt

Here we download the FOV, BPIX, EVT1 and FLT files for the
Observation Ids 1842 and 1843.

Usage
=====

A help message will be displayed if the --help option is used, or
the script is called with either no arguments or invalid input. The
message is shown below:

    unix% ./download_chandra_obsid
    Usage: download_chandra_obsid <obsid1>,..,<obsidN> [<type1>,..,<typeN>]
    
     Download public Chandra observations.  The observations to download are given
    as a comma-separated list of ObsId numbers, and an optional comma-separated
    list of file "types" can also be given, which will only download files that
    contain the type strings. So an argument of '1842,1843' will download all data
    for the obs ids 1842 and 1843, whereas '1842,1843 fov,vv,evt2' will only
    download the V&V, fov, and evt2 files for these observations.  The data is
    written to the current directory, with each obsid being saved to its own
    directory (following the layout used by the Chandra archive).
    
    Options:
      -h, --help       show this help message and exit
      -q, --quiet      Download the files without any screen output? [default:
                       False]
      -v, --version    List the version of the script and exit.
      -c, --copyright  List the copyright for the script and exit.
      -t, --filetypes  List the valid file types and exit.
      -d, --debug      Display diagnostic output? [default: False]

In this document the long names of the options will be used, such
as --help, although the short versions listed above (e.g. -h) can
also be used.

Supported file types
--------------------

The --filetypes option lists the valid names for the optional type
argument:

    unix% ./download_chandra_obsid --filetypes
    The list of valid file types is:
    
       aoff aqual asol bias bpix cntr_img dtf eph0 eph1 evt1 evt2 flt fov
       full_img msk mtl oif osol pbk pha2 plt soff src2 src_img stat sum vv

The Chandra Data Products guide provides more information on these
files, including their uses (although there is no guarantee that
the names agree exactly).

Version
-------

The --version option displays the version of the script, which is
the release date in YYYY.MM.DD format:

    unix% ./download_chandra_obsid --version
    2010.05.04

Screen output
-------------

The default behavior is to display output as each file is
downloaded, as shown below. To turn this off use the --quiet flag,
which will restrict screen output to error messages.

When run, the tool will appear to pause as it checks what data is
available in the archive for the ObsId value, and then print a
banner listing the total size available before listing each file as
it is downloaded:

    unix% ./download_chandra_obsid 1842
    Downloading files for ObsId 1842, total size is 69 Mb.
    
      Type     Format      Size  0........H.........1  Download Time Average Rate
      ---------------------------------------------------------------------------
      vv       pdf        88 Kb  ####################          < 1 s  1910.5 kb/s
      oif      fits       23 Kb  ####################          < 1 s  744.5 kb/s
      sum      html        2 Kb  ####################          < 1 s  58.1 kb/s
      sum      ps          2 Mb  ####################            4 s  655.9 kb/s
      sum      html        4 Kb  ####################          < 1 s  199.2 kb/s
      sum      html        3 Kb  ####################          < 1 s  215.1 kb/s
      cntr_img fits      229 Kb  ####################          < 1 s  5794.9 kb/s
      cntr_img jpg       826 Kb  ####################          < 1 s  7064.9 kb/s
      evt2     fits       19 Mb  ####################            3 s  5504.3 kb/s
      full_img fits       90 Kb  ####################          < 1 s  2964.0 kb/s
      full_img jpg        68 Kb  ####################          < 1 s  1613.4 kb/s
      src2     fits       14 Kb  ####################          < 1 s  595.3 kb/s
      src_img  jpg        70 Kb  ####################          < 1 s  1417.6 kb/s
      bpix     fits       10 Kb  ####################          < 1 s  692.3 kb/s
      fov      fits        6 Kb  ####################          < 1 s  414.4 kb/s
      eph1     fits      281 Kb  ####################          < 1 s  6248.2 kb/s
      asol     fits        2 Mb  ####################          < 1 s  2878.2 kb/s
      aoff     fits      889 Kb  ####################          < 1 s  7823.0 kb/s
      evt1     fits       36 Mb  ####################            6 s  5826.7 kb/s
      flt      fits       16 Kb  ####################          < 1 s  953.1 kb/s
      msk      fits        5 Kb  ####################          < 1 s  247.2 kb/s
      mtl      fits      334 Kb  ####################          < 1 s  5525.9 kb/s
      soff     fits        4 Kb  ####################          < 1 s  296.1 kb/s
      stat     fits      276 Kb  ####################          < 1 s  6394.4 kb/s
      bias     fits      426 Kb  ####################          < 1 s  7284.0 kb/s
      bias     fits      494 Kb  ####################          < 1 s  7877.3 kb/s
      bias     fits      430 Kb  ####################          < 1 s  7451.4 kb/s
      bias     fits      423 Kb  ####################          < 1 s  7303.9 kb/s
      bias     fits      427 Kb  ####################          < 1 s  6923.7 kb/s
      bias     fits      427 Kb  ####################          < 1 s  7491.6 kb/s
      pbk      fits        4 Kb  ####################          < 1 s  280.3 kb/s
      vv       pdf         2 Mb  ####################          < 1 s  3041.0 kb/s
      eph1     fits        6 Kb  ####################          < 1 s  312.2 kb/s
      eph1     fits      274 Kb  ####################          < 1 s  601.0 kb/s
      eph1     fits      258 Kb  ####################          < 1 s  5872.6 kb/s
      osol     fits      356 Kb  ####################          < 1 s  5526.5 kb/s
      aqual    fits      132 Kb  ####################          < 1 s  3816.4 kb/s
      osol     fits      360 Kb  ####################          < 1 s  5935.3 kb/s
    
          Total download size for ObsId 1842 = 69 Mb
          Total download time for ObsId 1842 = 17 s

The first three columns give the filetype, format and file size of
the file. The fourth column indicates the fraction of the file that
has been downloaded (each # mark is 5% of the file size, with the H
symbol in the column header indicating the 50% point) and is
updated as the file is downloaded. The last two columns are added
once the file size and list the download time and average download
rate for that file. A summary of the download is provided once all
the files for an ObsId are downloaded.

Previous downloads
------------------

The tool will skip any files that have previously downloaded, so it
is safe to halt the download during a run (using control-c) and
then re-start it. Any partially-downloaded files will resume, so
you do not need to re-download data you already have.

This allows you to do partial downloads, for example:

    unix% ./download_chandra_obsid 1843 evt2,fov
    Downloading files for ObsId 1843, total size is 17 Mb.
    
      Type     Format      Size  0........H.........1  Download Time Average Rate
      ---------------------------------------------------------------------------
      evt2     fits       17 Mb  ####################            6 s  2905.2 kb/s
      fov      fits        6 Kb  ####################          < 1 s  428.0 kb/s
    
          Total download size for ObsId 1843 = 17 Mb
          Total download time for ObsId 1843 = 6 s
    
    unix% ./download_chandra_obsid/download_chandra_obsid 1843 bpix,evt2
    Downloading files for ObsId 1843, total size is 17 Mb.
    
      Type     Format      Size  0........H.........1  Download Time Average Rate
      ---------------------------------------------------------------------------
      evt2     fits       17 Mb    already downloaded
      bpix     fits       11 Kb  ####################          < 1 s  779.5 kb/s
    
          Total download size for ObsId 1843 = 11 Kb
          Total download time for ObsId 1843 = < 1 s

and then just download the whole observation - e.g.
./download_chandra_obsid 1843 - without having to worry about
wasting time on files that you already have.

Compressed files
----------------

The tool does not check that you have an uncompressed version of an
archive file, so in the 1843 output above, if you uncompress the
evt2 file and then re-run the script the compressed version of the
file will be re-downloaded.

File corruption
---------------

There is no check that files have been corrupted during the
download.

Author, Copyright and Support
=============================

The copyright is Smithsonian Astrophysical 2010 and can also be
found using the --copyright option. The information is also
available in the file COPYING.

The author is Doug Burke (dburke@cfa.harvard.edu) and limited
support is provided.

A more-complete solution for downloading Chandra data is being
developed by the Chandra Data Archive for release in 2010.