Chandra X-Ray Observatory
	(CXC)
Skip to the navigation links
Last modified: December 2014

URL: http://cxc.harvard.edu/ciao/ahelp/merging_rules.html
Jump to: Description ยท See Also


AHELP for CIAO 4.9

merging_rules

Context: concept

Synopsis

A description of the merging rules used when combining header information.

Description

Merging rules have been implemented in the header library and are mainly used by the dmmerge tool. The purpose of this document is to describe what each of the rules means, and also give an example for each rule.

Rules with examples

(1) MAX

Take the maximal value among the keys that are present in the input files. If the key is not found in any of the input files, then no output and warning for the key.

Example:

  • infile1: KEY = 4.
  • infile2: KEY = 2.5
  • infile3: KEY = 3.
  • outfile: KEY = 4.

[ In this case, the output is the maximal value '4'. ]

(2) MIN

Take the minimal value among the keys that are present in the input files. If the key is not found in any of the input files, then no output and warning for the key.

Example:

  • infile1: KEY = 4.
  • infile2: KEY = 2.5
  • infile3: KEY = 3.
  • outfile: KEY = 2.5

[ In this case, the output is the minimal value '2.5'. ]

(3) FAIL

If the key is not present in the first file, or it is present in the first file but the value is different from others, then display a warning and don't write the key to the output. If the key is present in the first file and the value is same as others, then write the value to the output file without a warning.

Example:

  • infile1: KEY = 4.
  • infile2: KEY = 2.5
  • infile3: KEY = 3.
  • outfile: ( KEY not present )

[ In this case, KEY is present in the first file, but the value is different from others. Therefore display a warning: 'KEY values are different...FAIL...', but no output for KEY. ]

(4) FAIL;Default-[value]

First, go through each input file. If the key is not present, set the key value to the default (i.e. [value]). Once each file is checked, compare the values with one another. If they are all same, write it to the output; otherwise, display a warning but no output for the key.

Example using the rule 'FAIL;Default-3':

  • infile1: ( KEY not present )
  • infile2: ( KEY not present )
  • infile3: KEY = 3
  • outfile: KEY = 3

[ In this case, KEY is not present in the infile1 and infile2, so the rule sets both of them to be the default value '3', which happens to be same as the one in infile3. Therefore, it writes '3' to the output without a warning. ]

(5) calcForce

Calculate the value with a special hard-coded rule and write it to the output file. Time keywords with a calcFORCE rule are generally set to "sum the values from the input files". For other keys, this rule will simply copy the first value to the output file.

(6) calcGTI

Calculate the value in the output file from the time subspace (e.g. the good time intervals [GTIs]). This rule is currently applied to ONTIME, LIVETIME, LIVTIMEn, and EXPOSURn keywords.

Notes:

The following are special keys whose merging rules are always treated like 'calcForce' in the dmmerge tool. In other words, they are always computed from the special hard-coded rules:

DATE-OBS DATE-END

(7) FORCE

First, go through each input file. If the key is not present, set the key value to the default (null, 0, blank). Once each file is checked, compare the values with one another. If they are different, write the first one to the output with a warning message; otherwise, output the value without warning.

Example:

  • infile1: KEY = 3
  • infile2: ( KEY not present )
  • infile3: KEY = 5
  • outfile: KEY = 3

[ In this case, the KEY of infile2 is not present, therefore set it to the default (i.e. '0') first. Since KEY values are different, write the first one (i.e. '3') to the output with a warning: 'KEY has 2 different values.' ]

(8) FORCE-[value]

First, go through each input file. If the key is not present, set the value to the default (i.e. [value]). Once each file is checked, compare the values with one another. If they are different, write the first one to the output with a warning message; otherwise, output the value without warning.

Example using the rule 'Force-3':

  • infile1: KEY = 3
  • infile2: ( KEY not present )
  • infile3: KEY = 3
  • outfile: KEY = 3

[ In this case, the KEY of infile2 is not present, therefore set it to [value] (i.e. '3') first. Since KEY values are all same, write it to the output without warning. ]

(9) WarnFirst

If the key is not present in the first file, give a warning and don't write it to the output. Otherwise, copy the first value to the output and display a warning message only if the first value is different from others.

Example:

  • infile1: ( KEY not present )
  • infile2: KEY = 5
  • infile3: KEY = 5
  • outfile: ( KEY not present )

[ In this case, the KEY value of infile1 is not present, therefore no output for KEY except a warning message: "omit - keyword KEY not present in 1st file" ]

(10) WarnFirst;Force-[value]

First, go through each input file. If the key is not present, set the value to the default (i.e. [value]). After each file is checked, copy the first value to the output; if it is different from others, display a warning message.

Example using the rule 'WarnFirst;Force-5':

  • infile1: ( KEY not present )
  • infile2: KEY = 5
  • infile3: KEY = 5
  • outfile: KEY = 5

[ In this case, the KEY value of infile1 is not present, therefore force it to [value] (i.e. '5') first. Since all KEYs have same value, write it to the output without a warning. ]

(11) WarnOmit-[tol]

If the key is not present in the first header, or if it is present in the first header but different from others more than [tol], then don't write the key and give a warning. Otherwise, take the first value.

Example using the rule 'WarnOmit-0.3':

  • infile1: KEY = 5.5
  • infile2: KEY = 5
  • infile3: KEY = 5
  • outfile: ( KEY not present )

[ In this case, the KEY value of infile1 is present but is different from others more than the given tolerance (i.e. 0.3), therefore no output for KEY except a warning: 'omit - KEY values different more than 0.3'. ]

(12) WarnPrefer-[value]

If the key is not present in the first file, or if it is present in the first file but different from others, then write the [value] to the output and display a warning. If all the key values are same, then write it to the output with no warning.

Example using the rule 'WarnPrefer-9.0':

  • infile1: ( KEY not present )
  • infile2: KEY = 5
  • infile3: KEY = 5
  • outfile: KEY = 9.0

[ In this case, KEY is not present in infile1, therefore set it to [value] (i.e. '9.0'). Since the KEY values are different, write the first one to the output with a warning: 'warning: KEY has 2 different values'. ]

(13) Merge-[value1];Force-[value2]

First, go through each input file. If the key is not present, force it to [value2]. Once each file is checked, compare the values with one another. If different, then write [value1] to the output with a warning. If same, write the first value to the output.

Example using the rule 'MERGE-99;FORCE-10':

  • infile1: ( KEY not present )
  • infile2: KEY = 5
  • infile3: KEY = 5
  • outfile: KEY = 99

[ In this case, the KEY is not present in infile1, therefore set it to [value2] (i.e. '10') first. Since the KEY values are different, write [value1] (i.e. '99') to the output and display a warning: 'KEY has different value..merged..'. ]

(14) SKIP

Don't write the key to the output file.

Example:

  • infile1: KEY = 5
  • infile2: KEY = 5
  • infile3: KEY = 5
  • outfile: ( KEY not present. )

[ In this case, always omit KEY from the output file. ]

(15) PUT_STRING-[value]

Force the output value to be [value]. It applies for the key that does not belong to the event header and has the data type of 'char'.

Example using the rule 'PUT_STRING-TEST':

  • infile1: KEY = 'test1'
  • infile2: KEY = 'test2'
  • infile3: KEY = 'test3'
  • outfile: KEY = 'TEST'

[ In this case, force the KEY value to be 'TEST'. ]

(16) PUT_DOUBLE-[value]

Force the output value to be [value]. It applies for the key that does not belong to the event header and has the data type of 'DOUBLE'.

Example using the rule 'PUT_DOUBLE-99.9':

  • infile1: KEY = 100.0
  • infile2: KEY = 200.0
  • infile3: ( KEY not present )
  • outfile: KEY = 99.9

[ In this case, force the KEY value to be 99.9. ]

(17) PUT_LONG-[value]

Force the output value to be [value]. It applies for the key that does not belong to the event header and has the data type of 'LONG'.

Example using the rule 'PUT_LONG-100':

  • infile1: KEY = 99
  • infile2: KEY = 99
  • infile3: KEY = 99
  • outfile: KEY = 100

[ In this case, force the KEY value to be 100. ]

Special merging rules and keywords

(1) All merging rules described in the above section can only be applied for the keys that belong to the event header except the following rules of 1.a, 1.b, and 1.c.

1.a. These three rules should be used by the keys that do not belong to the event header:

  • PUT_STRING-[value]
  • PUT_DOUBLE-[value]
  • PUT_LONG-[value]

1.b. This rule can be used by any key, regardless whether or not it belongs to the event header:

  • SKIP

1.c. The above description for this rule is only applied to an event key. For a non-event key, it will be same as the rule 'PUT_STRING-[value]':

  • FORCE-[value]

(2) The merging rules of the following keys are always treated like 'calcForce' in the dmmerge tool. In other words, they are always computed from the special hard-coded rules:

DATE-OBS DATE-END

About the lookup table

A lookup table is required by the dmmerge tool. It assigns a merging rule to each keyword.

(1) Example of lookup table

Keyword Rule
MISSION Merge-Merged;Force-AXAF
TELESCOP Merge-Merged;Force-Unknown
OBJECT Merge-Merged;Force-Unknown
OBSERVER Merge-Merged;Force-Unknown
TITLE Merge-Merged;Force-Unknown
OBS_ID Merge-Merged;Force-Unknown
INSTRUME Merge-Merged;Force-Unknown
DETNAM Merge-Merged;Force-Unknown
GRATING Merge-Merged;Force-Unknown
OBS_MODE Merge-Merged;Force-Unknown
DATAMODE Merge-Merged;Force-Unknown
ORIGIN WarnFirst;Force-ASC
CONTENT Force
HDUNAME Force
RA_NOM WarnOmit-0.0003
DEC_NOM WarnOmit-0.0003
ROLL_NOM WarnOmit-1.0
SIM_X WarnOmit-0.001
SIM_Y WarnOmit-0.001
SIM_Z WarnOmit-0.001
FOC_LEN WarnOmit-1.0
EQUINOX WarnPrefer-2000.0
RADECSYS WarnPrefer-ICRS
DATACLAS WarnPrefer-Observed
TIMVERSN WarnFirst
CLOCKAPP WarnFirst
MJDREF fail
TIERRELA Max
TIERABSO max
TIMEDEL maX
DATE calcForce
DATE-OBS calcForce
DATE-END calcForce
TSTART calcForce
TSTOP calcForce
ONTIME calcGTI
LIVETIME calcGTI
DTCOR calcForce
TIMEUNIT Fail;Default-s
TIMESYS Fail;Default-UTC
TIMEPIXR Fail;Default-0.5

(2) Lookup Table Format

o) Each row consists of two columns: the keyword and the associated merging rule.

o) The two columns are separated by blanks or tabs.

o) If a default value is assigned to a merging rule, use "-" to separate them.

e.g. mergeRule-defValue

where defValue can be a number or a character string

o) If a keyword has two merging rules together, use ";" as a delimiter between the merging rules.

For example:

mergeRuleOne;mergingRuleTwo

mergeRuleOne-defValueOne;mergeRuleTwo-defValueTwo

(3) Lookup Table Restriction

For the first column:

o) It has to be in capitals.

o) The keyword may be either an exact match or will use the or longest substring match. Thus using 'ONTIME' will apply the same rule to each of the 'ONTIME0', 'ONTIME1', etc. keywords.

For the second column:

o) It has to be one word without any space

o) If a default value is assigned to a merging rule, its data type needs to be same as the keyword. Otherwise, a warning message will be displayed.

o) Currently not able to have a negative number as the numerical default value.

o) For those keywords not found in the user's lookup table, the code will copy the 1st header to the output.

o) Merging rules are limited to those defined in Sec. 3.

o) For any merging rule not found in Sec. 3, the 1st value will be copied to the output.

o) Merging rules are only applied to the values of the keyword, not to the 'comment' of the keyword.

See Also

tools
addresp, dmimgcalc, dmimgfilt, dmmerge, dmregrid2, hrc_dtfstats, reproject_image, reproject_image_grid

Last modified: December 2014
Smithsonian Institute Smithsonian Institute

The Chandra X-Ray Center (CXC) is operated for NASA by the Smithsonian Astrophysical Observatory. 60 Garden Street, Cambridge, MA 02138 USA.   Email:   cxchelp@head.cfa.harvard.edu Smithsonian Institution, Copyright © 1998-2017. All rights reserved.