Bugs: dmmerge

Caveats

1. Extra GTI blocks in merged event file (16 Apr 2007)

2. The tool sums exposure-related keywords (10 Oct 2007)

Bugs

1. Corrupt output if text columns of different lengths are merged.

Caveats

1. Extra GTI blocks in merged event file (16 Apr 2007)

   This caveat has been updated to use the subspace-editing capability added in CIAO 3.4. The previous title was "When two event files that have different ranges of exposure numbers (expno column) are merged, the output file will have two GTI blocks."

   When merging ACIS event lists with dmmerge, the "data subspace" of the two event lists is combined. Due to a technical subtlety in the way that the EXPNO (ACIS frame exposure number) column is defined, this generates unwanted extra GTI extensions in the output file if the exposure numbers are not equal.

   unix% dmmerge "acisf01587N002_evt2.fits,acisf07073N001_evt2.fits" NGC3379_merge.fits
   
   unix% dmlist NGC3379_merge.fits blocks
    
   --------------------------------------------------------------------------------
   Dataset: NGC3379_merge.fits
   --------------------------------------------------------------------------------
    
        Block Name                          Type         Dimensions
   --------------------------------------------------------------------------------
   Block    1: PRIMARY                        Null        
   Block    2: EVENTS                         Table        15 cols x 1392531  rows
   Block    3: GTI7                           Table         2 cols x 1        rows
   Block    4: GTI2                           Table         2 cols x 3        rows
   Block    5: GTI5                           Table         2 cols x 1        rows
   Block    6: GTI6                           Table         2 cols x 2        rows
   Block    7: GTI3                           Table         2 cols x 2        rows
   Block    8: GTI8                           Table         2 cols x 2        rows
   Block    9: GTI7_CPT7                      Table         2 cols x 1        rows
   Block   10: GTI2_CPT8                      Table         2 cols x 2        rows
   Block   11: GTI5_CPT9                      Table         2 cols x 1        rows
   Block   12: GTI6_CPT10                     Table         2 cols x 1        rows
   Block   13: GTI3_CPT11                     Table         2 cols x 2        rows
   Block   14: GTI8_CPT12                     Table         2 cols x 1        rows
   

   This occurs because the filters could not be combined into a single, valid range for the chip. It is undesireable for further analysis, as the tools cannot use multiple GTI blocks for a single chip, resulting in incorrect exposure values.

   Workaround:

   The subspace-editing capabilities in CIAO 3.4 simplify the workaround for this issue. It is now possible to delete the EXPNO subspace before merging the files:

   unix% dmmerge \
         "acisf01587N002_evt2.fits[subspace -expno],acisf07073N001_evt2.fits[subspace -expno]" \
         NGC3379_merge_new.fits
   
   unix% dmlist NGC3379_merge_new.fits blocks
    
   --------------------------------------------------------------------------------
   Dataset: NGC3379_merge_new.fits
   --------------------------------------------------------------------------------
    
        Block Name                          Type         Dimensions
   --------------------------------------------------------------------------------
   Block    1: PRIMARY                        Null        
   Block    2: EVENTS                         Table        15 cols x 1392531  rows
   Block    3: GTI7                           Table         2 cols x 2        rows
   Block    4: GTI2                           Table         2 cols x 5        rows
   Block    5: GTI5                           Table         2 cols x 2        rows
   Block    6: GTI6                           Table         2 cols x 3        rows
   Block    7: GTI3                           Table         2 cols x 4        rows
   Block    8: GTI8                           Table         2 cols x 3        rows
   

   There are now just six GTI blocks in the output file.

   Merge_all users: a "[subspace -expno]" filter was added to the dmmerge command in merge_all v3.6 as a workaround for this problem. Users no longer need to filter the data before using it as input to merge_all.

   Note that the addition of the subspace filter means that any user who intends to create lightcurves binned on exposure number from the merged output cannot use merge_all, since that information is eliminated from the subspace. (In general, lightcurves are binned on time.)

2. The tool sums exposure-related keywords (10 Oct 2007)

   Since dmmerge was designed to combine multiple observations, it sums exposure-related keywords (e.g. EXPOSURE, LIVTIME, ONTIME).

   When combining an array of chips, e.g. to create a background file for a multi-chip observation, these values should not be summed. The ONTIME/LIVETIME/EXPOSURE keywords should match the value of the aim chip, which is the first GTI block by Chandra convention.

   Here is an example of a merged file in which the values are the sum of the four individual ONTIMEs, EXPOSUREs, and LIVTIMEs (1500000.0*4=6000000.0):

   unix% dmlist merged.fits header | egrep '(ONTIM|EXPOS|LIV)'
   0002 LIVTIME0 1500000.0 [s] Real8 Ontime multiplied by DTCOR
   0003 ONTIME0 1500000.0 [s] Real8 Sum of GTIs
   0004 EXPOSUR0 1500000.0 [s] Real8 Total exposure time, with all known corr. a
   0091 ONTIME 6000000.0 [s] Real8 Sum of GTIs
   0092 LIVETIME 6000000.0 [s] Real8 Ontime multiplied by DTCOR
   0093 EXPOSURE 6000000.0 [s] Real8 Total exposure time, with all kn
   

   Workaround:

   Use dmhedit to correct all three header values, e.g.

   unix% dmhedit merged.fits filelist="" op=add key=EXPOSURE value=1500000.0 unit=s
   

Bugs

1. Corrupt output if text columns of different lengths are merged.

   dmmerge does not check the the length of text columns before merging files. If the columns are different lengths, then the output is corrupted.