Event processing tools should extract detector geometry information in <detnam>.par directly from pixlib
The level 1 and level 1.5 event processing tools (acis_process_events, hrc_process_events, tg_create_mask, and tg_resolve_events) should extract detector geometry information directly from the pixlib library, rather than using intermediate <detnam>.par, grating.par, and pix_pixel_plane.par parameter files.
Incorrect RA, DEC recorded for source positions in src2 data product (X, Y sky positions OK)
Under some circumstances the RA and DEC coordinates of sources recorded in the src2 data product are wildly incorrect. The corresponding source sky (X, Y) coordinates are correct, however.
ACIS strip file sometimes results in telemetry products with TSTART > TSTOP
Under some circumstances, the ACIS stripper erroneously generates a strip file that results in extracted telemetry data products with incorrect times; in some cases the computed start time (TSTART) is greated than the stop time (TSTOP). This occurred when ACIS was next in line and the data contained a large gap, including a missing science run report.
Add HISTNUM keyword to FITS header to facilitate history processing
Later pipelines/tools that write history information to the FITS header in compliance with the ASC-FITS standard use the FITS keyword HISTNUM to record the number of HISTORY records written to the header. This keyword should be added and populated in level 0 so that HISTORY records written by the extractor will be properly handled downstream.
Merge limits for SIM_Z should be relaxed when combining ObI's in level 2
When combining the Observation Intervals (ObI's) of a multi-ObI observation in level 2, the FITS header should be updated with the level 1 SIM_Z value even if the individual level 1 values differ by as much as 0.1 mm (the previous maximum difference was 0.001 mm). If the values differ by more than this, the SIM_Z value in the level 2 files will be listed as 'Merged'.
REPLICA is incorrectly recorded as 2044 instead of 4 under some circumstances
If the value that should be populated in the REPLICA column produced by the ACA calibration data extractor is 4, it is recorded incorrectly as 2044.
Aspect level 1 pipelines execute very slowly under some circumstances
The tool aca_calc_centr in the Aspect level 1 centroid and standard pipelines executes extremely slowly under some circumstances. This happens when the input files are fed out of order of sampling speed, resulting in excessive FITS record insertions in cfitsio. This enhancement improves performance under these circumstances by a factor of 15-20.
BTIMxxxx keywords not updated when the packet interface selects a new clock correlation
The BTIMxxxx keywords in the FITS headers of the SIM, Engineering, and EPHIN level 0 data products are not updated when the packet interface selects a new clock correlation file to compute times. Subsequent output products should include values for the BTIMxxxx keywords determined from the latest clock correlation being used.
BTIMNULL keyword does not contain correct value based on clock correlation file
The BTIMNULL keyword in the FITS headers of all level 0 telemetry data products should be populated by the value that is used in the clock correlation equation TIME = BTIMNULL + BTIMRATE * VCDU + BTIMDRFT * VCDU * VCDU. That is, BTIMNULL is the TIME corresponding to VCDU 0. Currently, BTIMNULL is incorrectly populated by the TIME corresponding to the (arbitrary) VCDU value which was used when the clock correlation file was created.
Data quality incorrectly assigned as good for gapped multi-syllable MSID by SIM stripper
The SIM telemetry stripper incorrectly assigns good data quality to a multi-syllable MSID that is gapped in the second syllable of the MSID (the first syllable was OK). The stripper should check all syllables of an MSID to establish the data quality. This bug can also affect the EPHIN and HRC strippers.
TLMIN/TLMAX for HRC-S TDET coordinates flipped
The TLMIN/TLMAX values listedin the FITS header for the TDET (tiled detector) coordinates for HRC-S (and HRC-SI) are flipped. The X limits are specified for the Y column and vice-versa.
Add support for high precision event timing for HRC observations with vetos disabled
Implement accurate timing for HRC event data that are obtained with all onboard HRC vetos disabled and that do not saturate telemetry (HRC-S in imaging mode, aka HRC-SI). When the data meet all the necessary requirements, the event times are corrected for the "off by one" error (time of event i is associated with event i + 1 on-board), and corrected for the internal HRC delay. FITS bit 5 of STATUS will be set to 1 to indicate that the event time is corrected. The use use detnam = 'HRC-SI' is temporary pending implementation of HRC window masks.
Add PHA filter to HRC (and HRC grating) level 2 pipelines
Per request from the Calibration Team, modify the HRC level 2 and grating level 2 pipelines (for HRC data) to filter out events with PHA = 255. This significantly reduces the background rate without eliminating true X-ray events.
SIM telemetry extractor exits under some circumstances
The SIM telemetry data extractor fails and exits with a segmentation violation under some rare circumstances.
SIM telemetry extractor exits under some circumstances when extracting diagnostic data
The SIM telemetry data extractor fails and exits incorrectly when extracting SIM diagnostic data if the execution path length is > 74 characters.
Update packet interface (PI2) gap file for ASC-FITS header compliance
The telemetry data gap file (gap0) created by the packet interface (PI2) does not contain ASC-FITS compliant header keywords. Update the FITS header to be ASC-FITS compliant. One notable change is that the header TSTART, TSTOP keywords now represent the start and stop times for the beginning and end of the input raw telemetry data, and not the start time of the first gap, stop time of the last gap.
tg_resolve_events does not handle ACIS CC mode data with LETG
tg_resolve_events does not handle the case of ACIS continuous clocking (CC) mode observations using the LETG. Grating coordinates are computed for the case of 0 order events, but dispersed LETG events are not currently assigned grating coordinates.
Add support to access CALDB directly from Transmission grating tools
The Transmission grating tools should access analysis reference data (ARD) directly from the calibration database (CALDB). Currently, a "helper script" in the pipelines extracts the ARD from the CALDB prior to running the tools. This enhancement provides better support for users running the tools directly in CIAO.
Decouple grating binning parameters between MEG and HEG
Currently, the parameters defining the spectral bin limits for MEG and HEG are not independent. This results in the MEG and HEG data binned on different grids. The grids should be decoupled, and the default grids made the same as the current MEG spectral bin grid. Separate parameters are now required to specify MEG and HEG binning independently.
Convert grating mask file format from ASCII to FITS region
Changes the format of the grating mask created by tg_create_mask to FITS region file format. This will allow grating masks to be manipulated by tools that handle FITS region files. tg_resolve_events is modified to handle either ASCII or FITS region file format masks, and if the latter appends the mask as a REGION extension to the output file.
FITS header: MFORMnnn keyword, FITS header: MTYPEnnn keyword, region: FITS region file
The region file currently created by tgextract does not include MFORMnnn and MTYPEnnn keywords in the region extension. These keywords must be included to construct a valid region extension since they are required by the DataModel.
Incorrect TSTART, TSTOP in SIM level 0.5 files
The TSTART, TSTOP header keyword values in the SIM level 0.5 output data products are incorrect starting with version 6.0.0 of the data system.
Modified all aspect pipelines to obtain analysis reference data from the CALDB
All of the aspect level 0.5 and level 1 pipelines are modified to retrieve analysis reference data directly from the CALDB by using the tool asp_l1_calib. As a result of this change, the aspect interval properties will use the current CALALIGN file to determine the pointing in each constant aspect interval. This could result in small changes in the nominal and pointing RA, Dec, and Roll mean values determined for the observation.
Celldetect reports incorrect source size when using fixed cell size
The size of the detect cell is determined by the size of the PSF at that point (psftable). However when using a fixed cell size, celldetect did not take binning size into account when determining the extent of an off-axis source causing the the tool to report the source size as smaller than it was.
Remove duplicate detections at different scales from celldetect output
Implemented a revised local peak filtering by choosing the best signal-to-noise ratio for the source list. The code now rejects duplicate detections at boundaries of different cell sizes. Added source cell locations and double detection flags for which the new output columns are CELL_X, CELL_Y, and DOUBLE.
Events incorrectly marked bad by acis_detect_afterglow
acis_detect_afterglow incorrectly does not use the CCD_ID (only event chip coordinates and exposure numbers) to determine streak events, potentially causing more events to be marked as afterglows than really were.
Randomization in conversion from PHA to Energy/PI
Added +/- half channel randomization to conversion from PHA to PI in acis_process_events in order to remove quantization effects.
ACIS good time intervals not determined appropriately for interleaved (alternating exposure) mode observations
Revise determination of good time interval boundaries for ACIS alternating exposure (interleaved mode) observations. Previously, the computed GTI boundaries would not match actual exposure boundaries because of the CCD flush cycle. The new algorithm coorectly handles the CCD flush, and the flush time is now recorded in the FLSHTIM{A|B} keywords for the primary and secondary exposures, respectively. There are several consequent changes with the TIMEDEL keyword values. The CCD flush time is no longer included in the TIMEDEL{A|B} keywords. The TIMEDEL value and formula have also changed slightly. Previously TIMEDEL was computed by the following formula:
TIMEDEL = TIMEDELA + (DTYCYCLE * TIMEDELB)
where the DTYCYCLE ranged from 1 to 16. The TIMEDEL formula is now:
TIMEDEL = (TIMEDELA + FLSHTIMA) + DTYCYCLE * (TIMEDELB + FLSHTIMB)
The acis_build_chip_gti tool now uses the FLSHTIM{A|B}, TIMEDEL{A|B}, and TIMEPIXR keywords to calculate the GTI, ONTIME, and DTCOR (will change the resulting LIVETIME).
TIMEPIXR incorrectly overwritten with data from the obspar file
Added check to not overwrite TIMEPIXR with data from an obspar file. Previously, this bug always caused the value to default to 0 rather than maintaining the correct value of 0.5. Consequently, the pipeline/tools would misinterpret the time of events.
Revise handling of dropped exposure records in the exposure statustics file
Changed the way exposure records were marked as dropped. Before, missing exposures were incorrectly identified in both the EXPSTATS extension and the DROPEXP extension. Now missing exposures are automatically marked as dropped, and the EXPSTATS extension only saves exposures that contain data.
Aspect CALDB interface tool asp_l1_calib fails with parse error if "." is in path
When "." is in the file path, the aspect CALDB interface tool asp_l1_calib fails with a parse error and does not retrieve the requested analysis reference data from the CALDB.
Add the smoothed mission time line and GTI limits to the mtl1 file
New script "arc_smooth_mtl" appends (1) the smoothed mission time line and (2) the GTI limits extension to the archived un-smoothed mission time line file (*mtl1.fits). Users can inspect mtl to see which limits were violated.
FITS format of data quality column incorrect in SIM coor0a file
The data quality column (QUALITY) in the SIM coor0a file is incorrectly recorded in 16X format instead of 9X format.
Order resolution does not work correctly if a grating arm falls off the detector
For ACIS continuous clocking mode HETG data, events at the edge of a chip are not correctly resolved due to dithering, and are placed in the wrong bin. This occurs when one arm of the dispersed grating image falls off the detector.
Note: This algorithm did not work as intended, and was updated for data system release 6.5.
Replace ACISDE keyword with OSIPFILE in grating order sorting
Replaced ACISDE keyword with OSIPFILE (Order-Sorting, Integrated Probability File) for consistency. The OSIPFILEs are tables containing expected boundaries in CCD energy vs energy for each position on each CCD. They are used to separate overlapping grating orders.
Order resolution does not work correctly if the zeroth order falls off the detector
Grating event coordinates are computed by using the aspect solution to project the zero-order sky centroid onto the detector at the time of each event, then solving for the diffraction coordinates using the chip coordinates of the detected photon, the zero-order location, and grating geometry. As a consequence, some coordinates did not fall on the chip and the tool was unable to resolve certain events. This triggered the following warning:
"WARNING: Event chip position is outside valid range.
Grating coordinates not computed for this event".
Changes to tg_resolve_events will reduce the number of these warnings generated.
Improve traceability in grating standard data processing
Added the INREGION keyword to properly keep track of which bow-tie extraction file was used and the WPSF to keep track of which psf width table was used in grating standard data processing.
TSTART, TSTOP, DATE-OBS, and DATE-END header keywords not correctly populated in ephemeris viewing angles file
The TSTART and TSTOP keywords in the ephemeris angles file (eph1) contain the start and stop times requested when the tool was run, instead of the actual start and stop times of the data. This change also affects the DATE-OBS and DATE-END keywords.
Bad data in last row of EPHIN housekeeping data file results in erroneous EPHIN light curve
The level 1 EPHIN rates tool, ephin_calc_rate1, interpolates across bad data in the EPHIN level 0 hkp0 data product. However, if the very last row of data is bad, the tool would interpolate between the previous row and (wrap around to) the first row. As a result, the output level 1 lightcurve data product is incorrect. This data product is fed into the mission time line, so the latter is incorrect also in this case. When the very last row of data is bad, no interpolation should be performed.
ASCDS_VERSION environment variable not appearing in FITS header
The ASC-DS processing system release number is not being populated into the ASCDSVER FITS header keyword by any of the level 0 telemetry extractors.
Aspect interval average RA, Dec incorrect for last record of file in non-NPNT modes
The average RA and Dec values calculated for a non-"normal pointing mode" (non-NPNT) aspect interval is incorrect for the last aspect interval in the aspect interval properties (aipr0a) file processed by the aspect level 0.5 pipeline tool aca_make_int.
If counts per pixel overflows short integer then grating zeroth order source detection fails
Change the pixel value data type from a "short" integer to a "long" integer. When pixel values are greater than 32767, they will wrap around to -32768, so celldetect is not able locate the central position of the source detections.
Incorrect pixels used in Aspect CCD background correction
The ROW and COL definitions of offsets for pixel locations in the Aspect camera are swapped. The result is that, in the presence of warm pixels, incorrect background pixels are used in background determination and subtraction.
Aspect image pixel value of 0.0 causes problem with numerical fitting
Aspect image pixels with a value of 0.0 after calibration result in the chisq calculation crashing, causing problems with numerical fitting. This happened in a very rare circumstance where data for a new image slot became available where it wasn't expected, and as a result the calibration data needed was not present, resulting in calibrated values of 0.0. If the pipeline had expected images for that star slot, then the calibration data would be correct and this condition could not occur.
PI2 hangs waiting for a clock correlation file when processing SFDUs
The PI2 pcket interface hangs waiting for a clock correlation when processing SFDU format data, even though a valid clock correlation is supplied. Processing from SFDU format data is used only for Aspect camera calibration data, and not for routine spacecraft telemetry.
MSIDs BGDAVG and BGDRMS should be unsigned integers
MSIDs BGDAVG and BGDRMS are 10 bit unsigned integers; however they are incorrectly decommutated as signed integers. The science impact of this error is negligible, because the background only very rarely gets large enough to wrap around. Furthermore, in those cases the ACA high background status flag will be set, so the pipeline should ignore the data.
Zero order image aspect correction algorithm enhancement for ACIS continuous clocking data
In ACIS continuous clocking readout mode, there is no knowledge of where the events were detected on the chip along the y-axis. This enhancement implements a new algorithm to estimate the sky position based on target location, which allows improved determination of the aspect correction in the CHIPY direction.
Zero order image should use aspect solution instead of aspect offsets
The binned aspect solution (aspect offsets; aoff1 file) is being used to de-dither grating zeroth order for ACIS continuous clocking data, degrading the results. The full aspect solution (asol1 file) should be used instead.
Add MTYPE and MFORM keywords to level 1.5 REGION extension definition
This enhancement adds keywords MTYPE1 (position) and MFORM1 (coordinates in sky (X,Y)) to the level 1.5 REGION definition in order to make it more consistent with CXC region format. These regions can now be used as dm filters.
Order resolution does not work correctly if the zeroth order is dithered into a CCD inter-chip gap
If the dither pattern brings the zero-order centroid into the inter-chip gap, then tg_resolve_events fails to determine the zeroth order position. tg_resolve_events is now capable of resolving events in this case.
Mask is too narrow for ACIS CC mode coordinates
When processing ACIS continuous clocking mode HETG data, tg_create_mask was incorrectly setting the grating mask width, potentially missing the data. This especially happened when RA_TARG and DEC_TARG (target coordinates) were much different than RA_NOM and DEC_NOM (coordinates of the tangent plane). The angle that was getting passed included the MEG grating angle in addition to the roll angle; however, in CC mode, the grating angle should come out of the equation.
Remove the tg_d_units parameter from tgextract (only degree units are supported)
Remove the tg_d_units parameter from the tgextract parameter file, since tgextract does not support any other units than degrees. The grt_lev2 parameter file is also affected by this interface change and has been updated.
"No data" (4x4 image) case in aca_find_hotpix should not kill Aspect level 1 pipelines
When all input ACA files to aca_find_hotpix are 4x4 images, the tool ignores them all - as it should - but then exits with an error, causing ASP level 1 pipeline to crash. Now the tool produces an empty bad pixel file and exits normally.
ObsId incorrectly read from level 0 engineering telemetry by Aspect level 0.5 pipeline
Under some rare circumstances, the Aspect tool aca_make_int would read the ObsId from the level 0 engineering telemetry (FITS column COBSRQID) as a signed integer instead of an unsigned integer, resulting in some very odd (large) numbers while still showing a good quality. This was due to a bug in the Aspect variable I/O layer.
TIMEDEL value for SIM diagnostic data files was set incorrectly
The TIMEDEL value for SIM diagnostic data files is being set incorrectly to 32.8 s (1 major frame). The correct value is is 1049.6 s (32 major frames).
ACIS Data Extractor segv's when bad data are received
Occasionally when the ACIS data extractor (ADE) receives bad data, specifically bad bias maps, it will get a segmentation fault and not process the rest of the strip file. The extractor has been fixed so that it process the entire strip file whether there is a bad bias map or not.
PI2 temp files not being deleted
The packet interface (PI2) was creating a number of temporary files containing the names of different clock correlation files and was not deleting those files after it was finished using them. PI2 now removes the files after it is done with them.
Add new algorithm to flag charged particle background events for VFAINT mode
This release includes a new algorithm in acis_process_events that can be enabled to significantly reduce the ACIS particle background for very faint (VFAINT) data by identifying events with significant flux in the border pixels of the 5x5 event islands. This screening also eliminates almost all of the afterglow events. Several new parameters control the application of this algorithm. check_vf_pha (default="no") controls whether the check is applied; trail (default=0.027) is an adjustment factor for CTI effects. If "check_vf_pha" is yes, then acis_process_events will flag events if any of the 16 border pixels in the 5x5 pixel event island is above the split threshold (parameter spthresh) by setting bit 23 of the event status value (FITS column STATUS).
Added a new parameter to control randomization in conversion from PHA to Energy/PI
The +/- half channel randomization added to the conversion from PHA to PI in acis_process_events is now independently controlled by a new parameter rand_pha (default="yes").
Aspect correction algorithm enhancement for ACIS continuous clocking data
In ACIS continuous clocking readout mode, there is no knowledge of where the events were detected on the chip along the y-axis. This enhancement implements a new algorithm to estimate the sky position based on target location, which allows improved determination of the aspect correction in the CHIPY direction.
HISTORY keywords duplicated in FITS header
acis_process_events and hrc_process_events are writing two exact copies of HISTORY keywords to the FITS file headers.
TIMEPIXR keyword incorrectly overwritten with 0 by acis_process_events
In the level 1 event file, the value of the TIMEPIXR keyword was being incorrectly set to 0 instead of the correct value 0.5. Tool acis_process_events was overwriting the TIMEPIXR keyword from the input file header with the value from the obspar file, which is a dummy value set to 0.0. acis_process_events has now been fixed so that it will not overwrite TIMEPIXR if the latter already exists.
Apply SIM_Z dependent correction to predicted FID positions
For ACIS-I, the relative FID positions change quite a bit (almost 4 arcsec!) when the SIM is moved to off-nominal positions. This is because of imperfections in the fiducial transfer system corner cube. For other scientific instruments the effect is there, but smaller. Now added the ability to calculate a SIM_Z dependent correction to predicted FID positions if and only if correction coefficient columns exist in the CALALIGN file.
It was later discovered that the FID positions were incorrect in this release. The mistake was corrected in release 6.6.
Aspect interval periods less than 100 seconds in length
Per request from SOT Aspect team, identify short aspect intervals less than 100 seconds as bad by setting the asp_sol_status column in the level 1 Aspect quality (aqual1) file as bad. These intervals will be excluded from the event files via the GTI filters.
PI values > 255 should not be present in event file
If the result of the conversion from PHA to PI (Pulse-height Invariant) is greater than 255, then the value should be set to 255 and event status bit 7 should be set.
Change tgdetect default fixed cell size to 15
Per recommendation of SDS grating scientist, change the tgdetect default fixed cell size to 15 for both ACIS and HRC. This will improve the determination of the zeroth order location, and therefore more accurate grating coordinates.
Order resolution does not work correctly if a grating arm falls off the detector (revised)
For ACIS continuous clocking mode HETG data, events at the edge of a chip are not correctly resolved due to dithering, and are placed in the wrong bin. This occurs when one arm of the dispersed grating image falls off the detector.
Note: An earlier version of this algorithm was included in data system release 6.6.1, but did not work as intended.
Integrate P008 telemetry database decommutation templates
Integrate the new telemetry database version P008 into standard data processing. The decommutation templates are pointer tables which contain information about how to derive data from the telemetry stream in order to produce data for scientific use.
Apply SIM_Z dependent correction to predicted FID positions (revised)
For ACIS-I, the relative FID positions change quite a bit (almost 4 arcsec!) when the SIM is moved to off-nominal positions. This is because of imperfections in the fiducial transfer system corner cube. For other scientific instruments the effect is there, but smaller. Now added the ability to calculate a SIM_Z dependent correction to predicted FID positions if and only if correction coefficient columns exist in the CALALIGN file.
This enhancement was a part of release 6.5. However, it was discovered that the FID positions were incorrect. They were corrected for this release.
Spurious gaps in Good Time Intervals
Tool hrc_merge_times was incorrectly setting TSTOP in the evt0a FITS files causing a 3.2 second gap before each file going into level 1.
Mispelled keyword in the ACIS FITS Image header
Due to a mispelled keyword in part of the ACIS Data Extractor code, the STARTBEP keyword was not being populated with a value in the FITS header for ACIS img0 files. Furthermore a new keyword was being added to the header, making the FITS header non-compliant with the ASC-FITS standard. The problem has been fixed and ACIS FITS headers are now correct.
Add HISTORY keyword identifying which pixlib files and parameters are used in data processing
All tools which use pixlib (the CXC pixel coordinates library) now include a HISTORY keyword in their output files indicating which files and parameters are used. These include acis_format_events (stat1 file), acis_process_events (evt1 file), asp_calc_offsets (aoff1 file & soff1 file), hrc_process_events (evt1 file), tg_create_mask (reg1a file), tg_resolve_events (evt1a file), and tgextract (pha2 file).
CYCLE keyword erroneously included in ACIS continuous clocking mode evt2 files
ACIS continuous clocking mode does not support alternating exposure (interleaved) mode observations. However, the header merging rules would set the CYCLE keyword to "Primary" in the evt2 files; the CYCLE keyword should not be present in continuous clocking mode evt2 files.
Fixed bogus "[s]" units for FEP_ID and CCD_ID keywords
In the flt1 files, the CCD_ID and FEP_ID FITS header keywords include '[s]' in the comment part of the FITS record. By FITS convention this indicates that these keywords have units of seconds. In reality, these keywords are unitless.
acis_bkg_ltc not separating background events from source events correctly
Due to an error in calculating the background count rate in acis_bkg_ltc, bright source regions were not being completely excluded from the background field. This was a result of a typographical error in acis_bkg_ltc which was not passing the sigma_factor parameter to the get_src_region tool.
Summary products have incorrect colors
Tool dmimg2jpg makes color JPEG images from three input FITS image files. The color lookup tables were incorrect and as a result the highest pixel values were not scaled to the right colors.
Enhanced PI2 data quality checking algorithm for EHS format telemetry
The PI2 data quality checking algorithm would occassionally lose good data because of an inappropriate error flag in the EHS Secondary Header. The Minor Frame Status byte, which declares that flag, was determined not to be a good method of marking bad data and is no longer used. If the byte does declare a problem the data will now still be processed and a warning message identifying the SFDU listing the ERT and VCDU number will be written to a log file. Only valid SFDUs are supposed to be passed to the CXC from the OCC, so the original PI2 test was too conservative.
PI2 crashes if an EHS packet with the "extra packet" flag set is received
It was discovered that PI2 was receiving a segmentation fault when the EHS Secondary Header extra packet flag was set. PI2 now handles the flag correctly.
ObsId transition time files contain incorrect times
The ObsId transition (obstr) files created when running PI2 did not have the correct VCDU or UT times in them. The VCDU counter time was off by a fraction of a second ahead of what it should have been, while the UT time as a fraction of a second behind what it should have been. The times now correctly reflect the ObsId change.
Degraded event order sorting in the level 1.5 grating pipeline
The full aspect solution should be used by the level 1.5 grating pipeline, rather than the aspect offsets file when resolving event orders in tg_resolve_events. Use of the aspect offsets file degrades the order sorting since the offsets file has lower time resolution. This bug was introduced in data system release 6.7.0.
Analysis reference data from CALDB now identified in FITS headers
Various tools have been updated to access the CALDB (calibration database) directly and to include tool-specific keyword(s) in the FITS header to identify which analysis reference data (ARD) files were used to calibrate the data products.
Double randomization of MEG CHIP pixel positions applied near zeroth order
Around the zeroth order position where the HEG and MEG mask arms overlap, the randomization of the MEG data CHIP pixel coordinates is incorrectly applied twice. Consequently there is some small degradation of the grating coordinates for events in this area. The HEG data are unaffected.
Off-axis angle and azimuthal angle coordinates (THETA, PHI) erroneously reversed
The off-axis angle (THETA) and azimuthal angle (PHI) in the Mirror Spherical Coordinate (MSC) system (the polar coordinate system centered on the HRMA axis) are erroneously reversed in all level 1 event files. Currently, the off-axis angle is encoded in the PHI column and the azimuthal angle in the THETA column.
Incorrect values for DTCOR in interleaved mode
The computation of keyword DTCOR is inaccurate for ACIS interleaved mode observations. This subsequently affects the LIVETIME and EXPOSURE values. A new algorithm for correctly calculating DTCOR has been defined and implemented (see Glenn Allen SDS memo of May 3, 2002).
Incorrect Rowland diameter for LETG
pixlib was using calibration files with the wrong geometry parameter value for the LETG Rowland torus diameter. This may have contributed to the wavelength errors for LETG data. The Rowland spacing value was set to 8632.48 mm (which is correct for HETG) but should have been 8637.0 mm for LETG.
Inappropriate GTI limits for secondary (not on-the-sky) observations
Good time intervals (GTI's) recorded in the level 1 filter (flt1) file for "secondary" mode observations (ie, not on-the-sky internal calibrations) should not exclude as bad time intervals for which aspect, grating, and HRMA optical path limits are violated. Some secondary mode level 2 event (evt2) files do not include all valid data because of violations of limits that are inappropriate for internal calibrations.
Field of view (fov1) file identifies sky region covered by the detector
A new field of view (fov1) file is produced as part of the ACIS (TIMED exposure mode only) and HRC level 1 pipelines for imaging science data. The field of view files record in region format the sky region covered by the active area of the detector, including aspect motion. This file and the level 1 mask (msk1) file are subsequently used in the level 2 pipeline to crop the full field image to a bounding box around the data area.
Level 2 full images and summary product images cropped to detector area
The level 2 pipeline full images and summary page jpeg images are now they are cropped to include only the active area covered by the detectors determined by combining the field of view regions for each observation interval determined by the level 1 skyfov tool. The images will be binned at the same resolution as earlier versions, but will be sized to fit the data (not larger than 1024 x 1024 pixels).
Incorrect mask (msk1) file when using exclude windows
In the case where exclude windows are used, the ACIS level 1 mask (msk1) file wrongly identifies the active detector region. The entire chip outside of the exclude region is active. However, the acis_build_mask tool assumes that an include window for the entire chip will be present in the window block, as well as the exclude window. This is not the case. For example, where a window is used to exclude the zeroth order of a grating observation, the absence of an include window for the entire chip in the window block (as well as the exclude window) results in acis_build_mask not correctly building the window mask file.
Data files now contain full history of all parameters used in processing
Most tools were updated to include the full tool parameter history into the FITS headers. This change affects many Level 0.5, Level 1, Level 1.5, and Level 2 data products.
Enhancements to support AGASC version 1.5
The Aspect level 0.5 tool aca_id_image has been upgraded to support version 1.5 of the AXAF Guide and Acquisition Star Catalog (AGASC) catalog. A FITS non-compliance issue was fixed in AGASC v1.5 (the TFORMnnn keyword in AGASC v1.4 and earlier did not comply with the FITS standard, and required special code to work around). In addition, the version of the AGASC catalog is now recorded in the AGASCVER FITS header keyword in the guide star properties (gspr1) file.
Smooth gyro rates to eliminate noise when identifying Aspect intervals
Excessive gyro readout noise on short time scales can produce gyro rates that momentarily exceed the NUDGE mode transition limits. This results in erroneously splitting constant Aspect intervals. Now, aca_make_int applies box smoothing to the telemetered gyro rates to determine the actual spacecraft body rates. The smoothing interval is controlled by the tool parameter aorate_avg_time, which is defaulted to 15 secs.
id_status erroneously set to blank for FID lights with MARGINAL status
For FID lights determined to have 'MARGINAL' status by tool aca_id_image (measured FID-FID separations between 2.5 and 5 arcsec different from the expected values), the id_status column in the FID properties file (fidpr1) would sometimes be left blank when it should have been set to 'MARGINAL'. This has now been fixed.
Update bad/marginal star image separation distance limits
Update the star-star image separation distance limits (difference between measured and expected separations) as follows: 'BAD' stars > 7.5 arcsec (instead of 5 arcsec); 'MARGINAL' stars > 5 arcsec (instead of 2.5 arcsec). The previous values were too conservative and resulted in occasionally rejecting GOOD stars from contributing to the Aspect solution.
Aspect processing fails for constant Aspect intervals shorter than 32.8 secs
Tool asp_get_calib was failing to identify the record in the SIM coordinates (coor0a) file corresponding to a constant Aspect interval when the interval was shorter than the length of a coor0a bin (32.8 sec) and did not span a bin boundary. As a result, Aspect level 1 processing would fail to compute an aspect solution for the interval.
TIMEDEL and TIMEPIXR keywords not present in Aspect data products
Most of the aspect data products do not include the TIMEDEL and TIMEPIXR FITS header keywords, as required by the ASC-FITS standard. Only the level 1 Aspect solution (asol1) file explicitly includes TIMEDEL. However, even in the this case, TIMEDEL and TIMEPIXR are not being applied to the last record time when determining TSTOP. As of data system version 6.9, all of the binned Aspect data products have been modified to include TIMEDEL and TIMEPIXR keywords and the correct TSTOP time.
aca_corr_ccd fails to handle empty input file list
The tool aca_corr_ccd that applies the CCD level corrections in the Aspect level 1 pipelines crashes when the input data file list is empty.
EPHIN hist0 data product HISTBINS column values incorrect
Prior to data system release 6.9, the HISTBINS values in the level 0 EPHIN histogram (hist0) files are not decommuted correctly, resulting in erroneous data.
EPHIN hkp0 data product SCPHAPRIPTR column values incorrect
Prior to data system release 6.9, the SCPHAPRIPRT values in ithe level 0 EPHIN housekeeping (hkp0) files are erroneously decommuted in reverse byte order, resulting in erroneous data.
HRC-S bad pixels not identified correctly
The hrc_build_badpix tool does not correctly handle bad pixel regions that are defined as spanning all 3 HRC-S plates. In such cases the tool incorrectly reports bad pixels on only one plate, possibly with negative coordinates and/or maximum values less than minimum values.
Revisions in support of new HRC mask data product
Creation of the level 1 HRC window mask (msk1) data product required numerous changes to the observation interval detection pipeline (OBIDET) and associated observation parameter file (obspar) to record the HRC instrument configuration parameters. These changes are implemented for obspar files with version (obspar_ver) >= 4. The changes include
(1) Revised set of HRC configuration parameters in the HRC block. These include trigger_level, range_switch_level, spect_mode, antico_enable, width_enable, width_threshold, uld_enable, upper_level_disc, blank_enable, u_blank_hi, v_blank_hi, u_blank_low, v_blank_low, py_shutter_position, and my_shutter_position.
(2) Extra columns added to the level 0.5 OBIDET HRC egti0a files that record the actual state of the HRC configuration parameters as a function of time.
(3) Additional limits added to the HRC olims0a ObI limits files created by OBIDET, and new HRC configuration state information added to the level 1 mission time line (mtl1) file, to establish good time intervals based on the planned (requested) HRC configuration.
DETNAM = 'HRC-SI' deprecated
Starting with data system version 6.9, DETNAM = 'HRC-SI' to indicate HRC-S in imaging mode will no longer be used. In these cases, DETNAM will be set to 'HRC-S'. The new HRC level 1 mask (msk1) data product will record the active area of the detector.
New HRC mask data product identifies detector active area
The HRC level 1 pipeline uses the hrc_build_mask tool to create a new window mask (msk1) data product. The mask file records the active area of the detector in a format similar to the ACIS window mask file, and will allow users to account correctly for the active detector area in data analysis. The hrc_build_mask tool extracts the actual HRC configuration from telemetry via the (version 4 or higher; parameter obspar_ver >= 4) observation parameter file (obspar). If hrc_build_mask is fed a pre-version 4 obspar file as input, the tool will still function; however, the mask will cover the entire detector area and not just the active area. The active area of the detector is determined from the HRC center and edge blanking parameters and, for HRC-S, whether the detector is in normal or imaging mode (the latter was previously identified with detnam = 'HRC-SI'). The use of 'HRC-SI' is now deprecated.
ACA calibration data stripper may SEGV after a data gap
Under some circumstances, such as gaps in the raw data, the ACA calibration data stripper would not correctly resynchronize with the data strean, resulting in a segmentation violation (SEGV). This can occur because the synch pattern (FF00 hex) can appear in valid data. A real synch pattern can be identified by the fact that (AAAA hex) must appear 14 bytes later in the data stream.
FITS header COMMENTs in level 0 data products do not comply with ASC-FITS standard
FITS header COMMENTs in level 0 data products do not comply with the ASC-FITS standard. According to the standard, the "/" should appear in column 32. However, in the level 0 data products the "/" appears in column 30. The data products are compliant with the NOST FITS standard.