Last modified: December 2020

URL: https://cxc.cfa.harvard.edu/ciao/ahelp/cratedataset.html
AHELP for CIAO 4.16

cratedataset

Context: crates

Synopsis

The CrateDataset object is used to read and write files with multiple blocks.

Syntax

CrateDataset(input=None, mode='rw')

If input is not None then it gives the name of the file to read in.
The mode argument accepts 'r' (read-only) and 'rw' (read-write).

Description

The CRATES Library uses CrateDataset objects to store a file (dataset) which contains multiple blocks (crates). This is needed if you need to create or use data from multiple blocks in a file or want to ensure that all these blocks are retained if modifying a block.

The read_dataset() command is equivalent to using the CrateDataset object. If you only need to read in data from a single block then you are unlikely to need to use a CrateDataset and read_file() should be sufficient.

The read_pha, read_rmf, write_pha and write_rmf routines use sub-classes of the CrateDataset object - namely PHACrateDataset and RMFCrateDataset - to store all the data they need.

Creating a dataset with multiple blocks

In the following example we create a CrateDataset object and then add two blocks two it: the first an image and the second a table with one column.

import sys
import time
import numpy as np
from pycrates import IMAGECrate, TABLECrate, CrateData, CrateDataset, \
    add_history, set_key
from history import HistoryRecord

toolname = sys.argv[0]

# Convert the time to a format supported by HistoryRecord
ltime = time.localtime()
tooltime = time.strftime('%Y-%m-%dT%H:%M:%S', ltime)

# First block
cr1 = IMAGECrate()
cr1.name = "IMG"
cd1 = CrateData()
cd1.values = np.arange(12).reshape(4,3)
cr1.add_image(cd1)

set_key(cr1, 'CREATOR', toolname,
        desc='tool that created this output')
set_key(cr1, 'DATE', tooltime,
        desc='Date and time of file creation')

# Create a history record which, for this script as it has no
# parameters, is very basic.
#
hist = HistoryRecord(tool=toolname, date=tooltime)
add_history(cr1, hist)

# Second block
cr2 = TABLECrate()
cr2.name = "TBL"
cd2 = CrateData()
cd2.name = "x"
cd2.values = np.arange(20,31)
cd2.unit = 'cm'
cd2.desc = 'Average beard length'
cr2.add_column(cd2)

# The dataset containing both blocks
ds = CrateDataset()
ds.add_crate(cr1)
ds.add_crate(cr2)

ds.write("out.fits", clobber=True)

which creates a file which looks like the following (assuming it's stored in a file called mds.py):

unix% python mds.py
unix% dmlist out.fits blocks
 
--------------------------------------------------------------------------------
Dataset: out.fits
--------------------------------------------------------------------------------
 
     Block Name                          Type         Dimensions
--------------------------------------------------------------------------------
Block    1: IMG                            Image      Int4(3x4)
Block    2: TBL                            Table         1 cols x 11       rows

and the table block contains:

unix% dmlist "out.fits[TBL]" cols
 
--------------------------------------------------------------------------------
Columns for Table Block TBL
--------------------------------------------------------------------------------
 
ColNo  Name                 Unit        Type             Range
   1   x                    cm           Int4           -                    Average beard length

and the header of the image block is

unix% dmlist out.fits header,clean,raw
SIMPLE       = T                    / file does conform to FITS standard
BITPIX       = 32                   / number of bits per data pixel
NAXIS        = 2                    / number of data axes
NAXIS1       = 3                    / length of data axis
NAXIS2       = 4                    / length of data axis
EXTEND       = T                    / FITS dataset may contain extensions
COMMENT      =   FITS (Flexible Image Transport System) format is defined in 'Astronomy /
COMMENT      =   and Astrophysics', volume 376, page 359; bibcode: 2001A&A...376..359H /
HDUNAME      = IMG                  / ASCDM block name
CREATOR      = mds.py               / tool that created this output
DATE         = 2020-11-17T08:25:23  / Date and time of file creation
HISTORY      =  TOOL  :mds.py  2020-11-17T08:25:23                             ASC00001 /

Reading in dataset with multiple blocks

Using the file created above, we can read it in by using either CrateDataset or read_dataset:

>>> cds = CrateDataset('out.fits')
>>> print(cds)
   Crate Dataset:
     File Name:         out.fits
     Read-Write Mode:   rw
     Number of Crates:  2
       1)     Crate Type:        <IMAGECrate>
   Crate Name:        IMG

       2)     Crate Type:        <TABLECrate>
   Crate Name:        TBL
   Ncols:             1
   Nrows:             11

and access its methods such as:

>>> msg = f"{cds.get_filename()} has {cds.get_ncrates()} crates"
>>> print(msg)
out.fits has 2 crates
>>> tcr = cds.get_crate('TBL')
>>> print(tcr)
   Crate Type:        <TABLECrate>
   Crate Name:        TBL
   Ncols:             1
   Nrows:             11
>>> print(tcr.get_column("x").desc)
Average beard length

Loading Crates

The Crates module is automatically imported into Sherpa sessions, otherwise use one of the following:

from pycrates import *

or

import pycrates

The mode argument

When a file is read in, the write permission is checked against the mode argument and, if it does not match (if mode='rw' but the user does not have write permission, or the file is a gzipped file) then a warning is displayed and the mode is set to 'r'.

When is the mode argument used?

The mode argument is only relevant if you call the write method of the crate with no arguments; that is if you say

>>> cr = read_file('tbl.dat', mode='rw')
UserWarning: File 'tbl.dat' does not have write permission. Changing to
read-only mode.
...
>>> cr.write()
IOError: File is not writeable.

It is not used if you want to write to a new file or one that is not write protected. That is, you can read in a file in read-only mode, change its contents, and write it out to a new file:

>>> cr = read_file('img.fits', mode='r')
>>> ivals = cr.get_image().values
>>> ivals += 1
>>> cr.write('modified.fits')

Bugs

See the bug pages on the CIAO website for an up-to-date listing of known bugs.

Refer to the CIAO bug pages for an up-to-date listing of known issues.

See Also

crates
add_crate, crates, get_crate, read_dataset, read_pha, read_rmf, write_dataset, write_pha, write_rmf