How to Download Chandra Data from the Archive
[CIAO 3.4 Science Threads]
OverviewLast Update: 1 Dec 2006 - reviewed for CIAO 3.4: no changes Synopsis: ChaSeR is a graphical user interface for browsing and/or obtaining data from the Chandra Data Archive. The Archive contains over 800 public-domain datasets, which consist of all observations for which the proprietary period has expired, as well as calibration data. Note that proprietary data may also be accessed via ChaSeR, but it is password-protected from the general community. Purpose: To download ObsID 1843 (ACIS-I, G21.5-0.9), which is used in all of the Introductory CIAO Threads. |
Contents
Launching ChaSeR
Before beginning, download and install the Java application.
Once installed, launch ChaSeR from the command line:
unix% chaser &
Figure 1 ![[Link to Image 1: The ChaSeR GUI]](../../imgs/imageicon.gif){kind=small}
shows the GUI that will appear.
WebChaSeR
If you do not wish to install ChaSeR, data may still be obtained through WebChaSeR, a web version of ChaSeR. While it has the same search capabilities as ChaSeR, it is not possible to select individual files for download. Using this interface, you can only select complete packages of primary and secondary products; see the "Select" step for information on the different packages.
Due to the slightly different functionality, portions of this thread - mainly the Three Steps to Using ChaSeR section - do not apply to users of WebChaSeR. The two applications are similar enough, however, that the thread may serve as a helpful guide.
The Three Steps to Using ChaSeR
Using ChaSeR to obtain data is a simple 3-step process:
1. Browse
Chandra Archive data may be browsed using any of the fields in the ChaSeR GUI, such as Instrument, Grating, ObsID, and Category. Since we are interested in an ACIS-I observation of G21.5-0.9, leave only the appropriate Instrument (ACIS-I) and Grating (NONE) options checked and enter the Target Name.
Figure 2 shows the completed form.
Click the "Submit" button to enter the request.
2. Select
The Search Results window brings up all the observations that match
our search criteria, as shown in
Figure 3 .
The "Order By..." menu can be used to sort the results;
here we have ordered them by "Observation ID."
Click on ObsID 1843 to highlight the row. After doing so one may:
-
Use the "Detail" button to display specifics about the observation; Figure 4
.
-
Click on the "Images" button to see preview images; Figure 5
.
-
Write the table information to a file ("Save").
Now the desired data products may be selected for download using the "Select primary products", "Select secondary products", and "Select supporting products" buttons.
The data products are arranged such that all products necessary for most analyses (e.g. the CIAO threads) are in the primary directory. If you are interested in reprocessing your data, you will also need the level 1 files, which are in the secondary directory. Each of the threads also includes a list of the files needed to complete it.
By default, all primary and secondary products are pre-selected. If you are only interested in downloading the specific files need for a thread, but need some help determing what those are, check the Standard Data Distribution Contents webpage. These tables list the file extensions (aka "Type") and the "Content"; the content keyword is how the products are labeled in the ChaSeR interface.
The selections are submitted using the "Add to retrieval list" button. You can then highlight a different observation and add the same (or different) data products to the cart for it. Once you have selected all the data you are interested in, proceed to the next step.
3. Retrieve
Click on "View retrieval list" to
view the observations and
data products submitted for selection.
One may see the list of data
filenames and sizes by clicking on the
"Browse selected products" button. The total file
size of the package list is listed at the top of
the screen; use this information when considering available disk space.
The "Retrieve selected products" button is used to start the retrieval
process. The files are compiled into a tarred package which
is placed onto the anonymous FTP server (cda.harvard.edu) by default;
the tarfile can also be saved directly to a local disk. When the
download is complete, the ChaSeR window will look as shown in
Figure 8 . Note that the
location of the logilfe (chaser.log) and the FTP
directory in which the tarfile was placed are both given
at the bottom of the screen.
The name and location of the tarfile is recorded in chaser.log, along with the ObsID that was downloaded:
unix% more ~/chaser.log [16/06/2005 09:53:39:463 EDT] Request retrieval of 1843 [16/06/2005 09:53:40:041 EDT] File(s) package_0_030616095338.tar.* will be available at cda.harvard.edu:/pub/srftp/UoL7nEUa
When you are finished with ChaSeR, use the "Exit" button to close the GUI.
FTP and Unpack the Data
Get the data from the anonymous FTP server; note that the location of the tarfile will be different for your observation:
unix% ftp cda.harvard.edu Connected to cda. 220 cda FTP server (Version wu-2.6.1(1) Mon Aug 7 15:20:43 EDT 2000) ready. Name (cda.harvard.edu:username): anonymous 331 Guest login ok, send your complete e-mail address as password. Password: 230- 230- Welcome to the FTP server at the CXC Science Center 230- 230- This server is cda.harvard.edu 230- 230-If your FTP client crashes or hangs shortly after login please try 230-using a dash (-) as the first character of your password. This will 230-turn off the informational messages that may be confusing your FTP 230-client. 230- 230-Publicly available files are in /pub 230- 230-Problems with this ftp site? Contact cxcftp@head-cfa.harvard.edu 230- 230- +---------------------------------------------------------+ 230- | | 230- | PLEASE NOTE: (ChaSeR users only) | 230- | | 230- | If the directory provided to you by ChaSeR appears | 230- | to be empty, this is a sign that your tar file is | 230- | not yet ready. Please try again in a little while. | 230- | | 230- | If your directory seems to contain several tar | 230- | files, e.g., file.tar.0, file.tar.1, ... | 230- | retrieve all files, then untar as follows: | 230- | cat file.tar.* | tar xvf - | 230- | We are limiting the size of individual files to | 230- | less than 2 GB, but these files need to be | 230- | concatenated before submitting them to tar. | 230- | | 230- +---------------------------------------------------------+ 230- 230 Guest login ok, access restrictions apply. ftp> cd /pub/srftp/UoL7nEUa 250 CWD command successful. ftp> binary 200 Type set to I. ftp> get package_0_030616095338.tar 200 PORT command successful. 150 Opening BINARY mode data connection for package_0_030616095338.tar (65309184 bytes). 226 Transfer complete. local: package_0_030616095338.tar remote: package_0_030616095338.tar 65309184 bytes received in 1.3e+02 seconds (484.05 Kbytes/s) ftp> bye 221-You have transferred 65309184 bytes in 1 files. 221-Total traffic for this session was 65311557 bytes in 2 transfers. 221-Thank you for using the FTP service on cda. 221 Goodbye.
Unpack the tarfile:
unix% tar xvf package_0_030616095338.tar x ., 0 bytes, 0 tape blocks x ./1843, 0 bytes, 0 tape blocks x ./1843/primary, 0 bytes, 0 tape blocks x ./1843/primary/acisf01843N001_src2.fits.gz, 9665 bytes, 19 tape blocks x ./1843/primary/acisf01843N001_evt2.fits.gz, 16289515 bytes, 31816 tape blocks . . (output omited) . x ./1843/vv.1843.001.1.gz, 1985 bytes, 4 tape blocks x ./1843/oif.fits, 25920 bytes, 51 tape blocks x ./tar_030616095338.rpt, 66 bytes, 1 tape blocks unix% ls 1843/ tar_030616095338.rpt package_0_030616095338.tar unix% rm package_0_030616095338.tar
If you downloaded data from multiple ObsIds, a directory will be created for each of them.
The final step is to uncompress the files:
unix% cd 1843 unix% gunzip ./primary/*.gz unix% gunzip ./secondary/*.gz unix% ls * oif.fits vv.1843.001.1.gz primary: acisf01843N001_1_sum2.html acisf01843N001_full_img2.jpg.log acisf01843N001_1_sum2.html.log acisf01843N001_src2.fits acisf01843N001_2_sum2.html acisf01843N001_src_img2.jpg acisf01843N001_2_sum2.html.log acisf01843N001_src_img2.jpg.log acisf01843N001_3_sum2.html acisf01843_000N001_bpix1.fits acisf01843N001_3_sum2.html.log orbitf082728300N001_eph1.fits acisf01843N001_cntr_img2.fits orbitf082987500N001_eph1.fits acisf01843N001_cntr_img2.jpg orbitf083333100N001_eph1.fits acisf01843N001_cntr_img2.jpg.log orbitf083592300N001_eph1.fits acisf01843N001_evt2.fits orbitf083937900N001_eph1.fits acisf01843N001_full_img2.fits orbitf084197100N001_eph1.fits acisf01843N001_full_img2.jpg pcadf084271087N001_asol1.fits secondary: acisf01843_000N001_aoff1.fits acisf01843_000N001_soff1.fits acisf01843_000N001_evt1.fits acisf01843_000N001_stat1.fits acisf01843_000N001_flt1.fits aspect/ acisf01843_000N001_msk1.fits ephem/ acisf01843_000N001_mtl1.fits supporting:
Summary
For a description of the files that you just downloaded, read the Introduction to the Data Products thread.
History
| 03 Jan 2005 | reviewed for CIAO 3.2: no changes |
| 01 Dec 2005 | reviewed for CIAO 3.3: no changes |
| 01 Dec 2006 | reviewed for CIAO 3.4: no changes |
