|
|
Bugs: Data Model
A list of
bugs fixed in CIAO 3.4
is available.
Caveats
The Data Model does not support writing to gzip'ed files.
Bugs
The BLANK header keyword that an older version of
some FTOOLs (e.g. chimgtyp) write to the FITS file
header causes problems for dmcopy.
If you use "[opt null=nan]" on an integer image,
the BLANK keyword is set to 0, since nan isn't a valid
integer.
You may get a seg fault if you try to create a very large
image. What constitutes "very large" depends on the data
type, but for long and float images, 8192x8192 pixels seems
to be the threshold.
# DMCOPY (CIAO 3.4): Bad data type in filter string formatting
The DM does not complain when given a component in a FITS
region file that doesn't exist
(e.g. "file.fits[component=99]" when there's no
component numbered 99).
No more than nine subspace components are allowed.
(01 Dec 2006)
Block names containing blank spaces are treated as bad syntax
Multi-valued string filters don't work
Creating a vector on-the-fly when region filtering
Applying a bit-filter expression to an integer column does
not work, nor does it cause an error.
Using incorrect syntax with the rectangle shape does not
fail when filtering.
Filtering an image on logical coordinates causes problems
when the short cut of omitting a number (i.e. to indicate
the default value) is used.
Trying to exclude a region filter with update=no
will cause the image to be filtered by the region's bounding
box.
There is a bug in region and range filters.
Using "!=" with string filtering causes dmlist to fail.
(05 Mar 2007)
Rebinning an image with different values for the two axes
causes the coordinate information to be lost
Caveats
-
The Data Model does not support writing to gzip'ed files.
The Data Model does not support writing to gzip'ed files,
even if the CIAO installation was configured with
gzip support. General write support for gzip'ed files will
not be added anytime in the near future, so users should
gunzip their files first if it is necessary to modify
them.
Bugs
-
The BLANK header keyword that an older version of
some FTOOLs (e.g. chimgtyp) write to the FITS file
header causes problems for dmcopy.
Workarounds:
-
Use a CIAO tool in place of the FTOOL that created the
BLANK keyword, e.g. instead of chimgtyp,
try
unix% dmcopy input.fits"[opt type=i4,null=-9999]" output.fits
changing the opt type
as appropriate.
Delete the BLANK keyword before passing the file
to dmcopy or any other CIAO tool.
-
If you use "[opt null=nan]" on an integer image,
the BLANK keyword is set to 0, since nan isn't a valid
integer.
Users should specify "[opt null=-999]", or some
other number that is not likely to be a valid pixel
value.
-
You may get a seg fault if you try to create a very large
image. What constitutes "very large" depends on the data
type, but for long and float images, 8192x8192 pixels seems
to be the threshold.
(The image doesn't have to be square; it just needs to
have 8192^2 pixels.)
This condition may be met when the "update=no"
option is used. Normally, when you filter a dataset, the
data subspace (which describes the boundaries of each
column's data and therefore is the intersection of the
initial minima and maxima with any subsequent filters)
gets updated to reflect the filtering. However, when you
give the "update=no" option, you instruct the DM not to
update the subspace to reflect the current filter.
Therefore, the full ranges for x and y are used in the
binning, and you get a 8192x8192 image (and a seg fault,
for the reason described above).
-
# DMCOPY (CIAO 3.4): Bad data type in filter string formatting
Prior to CIAO 3.4, this error message was written as:
# DMCOPY (CIAO 3.3): Filter data type mismatch for DOUBLE
If an unsigned byte column exists in the file, the DM will
generate this warning. The output is unaffected by this
bug.
-
The DM does not complain when given a component in a FITS
region file that doesn't exist
(e.g. "file.fits[component=99]" when there's no
component numbered 99).
Since no rows in the file match that filter, you get an
output file with no rows.
-
No more than nine subspace components are allowed.
(01 Dec 2006)
The Data Model doesn't allow a file to have more than nine
subspace components. If the input file has more than that
number, only the first nine are copied to the output file.
-
Block names containing blank spaces are treated as bad syntax
If the extension in the input file name has a space in it,
the CIAO tools will fail, e.g.
unix% rmfimg "conx-ea-101106-b10.rsp[SPECRESP MATRIX]" rsp.img
# (CIAO 4.0): Failed to open conx-ea-101106-b10.rsp[SPECRESP MATRIX]:
Must have RMF file
Workaround:
For most tools, refering to the block by number instead of
name will allow the tool to run,
e.g. "conx-ea-101106-b10.rsp[3]".
-
Multi-valued string filters don't work
"col=foo" is okay, but "col=foo,bar"
isn't.
Workaround:
Use "col=foo,col=bar" instead.
-
Creating a vector on-the-fly when region filtering
When region-filtering images, you can create a vector on
the fly from any two axes by using a filter like
"(#1,#3)=circle(...)". Although the image is
filtered correctly with a temporary vector, the region
filter isn't recorded in the subspace. Hence, tools that
use the filtered file don't know that pixels outside the
filter region are invalid. As a result, dmstat
reports no nulls in the filtered image (unless you
explicitly tell the DM to set pixels outside the filter to
null by using "opt null=...").
Applying a bit-filter expression to an integer column does
not work, nor does it cause an error.
-
Using incorrect syntax with the rectangle shape does not
fail when filtering.
For example, setting xmax >
xmin and/or ymax > ymin.
Instead it appears that the Data Model simply swaps the min
and max values.
-
Filtering an image on logical coordinates causes problems
when the short cut of omitting a number (i.e. to indicate
the default value) is used.
The exit status of dmcopy is also incorrectly set to 0
(success):
unix% dmcopy "image.fits[#1=1:20,#2=:]" delme.fits
# DMCOPY (CIAO 3.4): [ftColRead]: FITS error 308 bad first element number in
dataset image.fits Block 1 PRIMARY
unix% echo $status
0
Workarounds:
Omit the "#2=:" from the filter
Specify a range for both elements:
[#1=1:20,#2=1:20]
-
Trying to exclude a region filter with update=no
will cause the image to be filtered by the region's bounding
box.
For example:
unix% dmcopy "acis_fimg3.fits[exclude sky=region(src.fits)][opt full,update=no]" remove.fits clob+
The regions are correctly excluded; however, the image is
also clipped at the bounding box around all the excluded
shapes, so the corners of a few chips are removed.
Workaround:
Remove update=no. In this case, the Data Model
internally inverts all exclude filters to be an inclusive
filter, and correctly filters the image.
Be aware that this process is much slower if the region is
large. In that case, it will also add a large region
keyword to the file's header, noticeably slowing down any
operation on that file.
-
There is a bug in region and range filters.
This command:
unix% dmcopy "input.fits[sky=circle(4096,4096,100),y=4020:4100,4250,4350]" \
output.fits
fails to do the y filter altogether. This bug also applies
to exclude filters.
-
Using "!=" with string filtering causes dmlist to fail.
(05 Mar 2007)
For example, the following commands both fail:
unix% dmlist "region.fits[shape!=Annulus]" data
unix% dmlist catalog.fits"[COMMENT!='weak'][cols COMMENT]" data
-
Rebinning an image with different values for the two axes
causes the coordinate information to be lost
For example:
unix% dmcopy acis.img"[bin x=::5,y=::6]" acis5x6.img
Using the same value for both axes works correctly:
unix% dmcopy acis.img"[bin (x,y)=::5]" acis5.img
The following is a list of bugs that were fixed
in the CIAO 3.4
software release.
-
Regions off the image: if one region shape is partly off the
edge of the image, the DM does not exclude all the regions
in the file correctly.
For example:
unix% dmcopy "expmap.fits[exclude sky=region(holes.reg)]" expmap_holes.fits
unix% cat holes.reg
-circle(4713.138184,3794.545410,11.654875)
-circle(4382.204590,4670.012207,10.183500)
-circle(4136.358887,4525.479492,6.161741)
..etc..
Workarounds:
-
Modify the region file to make it legal by adding the line
at the beginning, so it reads
field()
-circle(4713.138184,3794.545410,11.654875)
-circle(4382.204590,4670.012207,10.183500)
-circle(4136.358887,4525.479492,6.161741)
..etc..
then invert the filter to be an include instead:
unix% dmcopy "expmap.fits[sky=region(holes.reg)]" expmap_holes.fits
Omit that shape from the region file
-
Problems occur when you wish to combine the filtering stored
in an external file with additional filters.
Workaround:
Split the filtering up into two steps: the first using
the "@" file and the second the additional filters:
unix% dmcopy "evt2.fits[@src.gti]" tmp.fits
unix% dmcopy "tmp.fits[time=a:b]" evt2.src.fits
-
Due to a DM bug, dmkeypar should not be used to
read data from array columns that are part of a vector.
(01 Jun 2006)
dmkeypar will access the first X value correctly,
but not the first Y value in the vector array column (when
the length of the array is > 1).
Examples of such a column are the "x", "y", and "pos"
columns below:
-----------------------------------------------------------------
Columns for Table Block REGION
-----------------------------------------------------------------
ColNo Name Unit Type Range
1 POS(X,Y)[11] pixel Real8(11) -Inf:+Inf
2 SHAPE String[16]
3 R[2] pixel Real8(2) -Inf:+Inf
4 ROTANG[2] pixel Real8(2) -Inf:+Inf
5 COMPONENT Int2 -
|
![[CIAO Logo]](../imgs/ciao_logo_navbar.gif){kind=logo}