Overview: Detecting Sources in Imaging Observations
CIAO 4.15 Science Threads
An overview and comparison of the three Detect tools: celldetect, vtpdetect, and wavdetect. A complete description of these algorithms and their uses is available from the Detect manual.
Last Update: 13 Jan 2020 - Reviewed for CIAO 4.14. No changes.
- The CIAO Detect Package
- Comparison of the Detect Tools
- Comparison of Detect Methods
- Running The CIAO Detect Tools
- Output of Detect: _src Files
The CIAO Detect Package
The detection of significant features in 2-D images is at the heart of much of astronomy, as well as of other areas of science. X-ray astronomy poses some special challenges (e.g. large sparse arrays, Poisson statistics) that mandate specially developed methods for source detection. Moreover, the extremely small beam area of the Chandra mission can resolve tight clusters of sources. However, the small beam also means that more diffuse sources will also be resolved, making them more difficult to detect.
The CIAO Detect package includes three different source detection programs which identify statistically significant brightness enhancements, deriving from both unresolved (point) and resolved (extended) x-ray sources:
celldetect uses a sliding square detect cell whose size is matched to the instrument PSF (or fixed by the user) to search for statistically significant enhancements over the background. This tool works well in simple fields with well-separated point sources.
More detailed information about celldetect is available from the Detect manual, including:
vtpdetect uses a Voronoi tessellation to determine individual densities, or fluxes, for every occupied pixel. The tool then analyzes the distribution of densities for significant source enhancements. This is the only detect tool that is scale-free.
More detailed information about vtpdetect is available from the Detect manual, including:
wavdetect correlates the image with wavelets of different scales (selected by the user) and then searches the results for significant correlations. Note that wavdetect is a wrapper for the tools wtransform and wrecon.
Based on extensive simulations to determine the dependence of the number of pixels per false source on source significance and background (expressed in the new sigcalfile parameter), the tool determines the significance threshold for source detection itself, if the user inputs directly the desired false source rate (e.g., desired number of false sources per field per scale). Both threshold algorithms are fully supported, and users need make no changes if they still wish to use the current algorithm. The algorithm is restricted to symmetric wavelets but can be applied to non-Chandra data.
More detailed information about wavdetect is available from the Detect manual, including:
Comparison of the Detect Tools
In general, wavdetect and vtpdetect offer more capability for complex fields with both point and extended sources, but place a significantly heavier demand on computer resources. For wavdetect, it is recommended that you restrict the image space as much as possible; see Section 1.5 - Data Manipulation (DM) Filtering - of the Detect manual for information on how to do so.
Comparison of Detect Methods
A. Factors Affecting Performance and Runtime
The primary factor that influences the runtime of celldetect is the size of the dataset. Two secondary factors are the number of cell sizes to examine and the actual number of detected sources.
The three most important factors that affect the runtime of wavdetect are:
- the size of the dataset
- the number and size of wavelet scales
- the number of background cleaning iterations.
The first two factors influence both wtransform and wrecon, the third only affects wtransform .
For large wavelet sizes, the dataset needs to be padded. This causes the size of the data to be artificially augmented.
wavdetect also requires a lot of memory. Data structures for a 512x512 image use up 36 Mb. A 2048x2048 images requires over 300 Mb. For images larger than 2048, at least 1 Gb of memory is recommended. Datasets that do not fit in physical memory will page heavily to disk and processing will run very slowly.
The runtime of vtpdetect does not depend on the spatial size of the dataset, but depends on many other factors and is rather unpredictable. The most important factors are the number of unique photon locations and the overall number of events.
vtpdetect will run quickly if the number of photons is low and there is a high contrast between background and sources. In the opposite scenario (i.e. when one has a large number of photons and a large number of faint sources), the vtpdetect run can be very long, since fitting background becomes an arduous task. In these situations, our tests have shown that vtpdetect becomes very slow if the observation contains more than ~105 photons.
B. Comparative Results For The Chandra Deep Field South
As described in section 2.4 of the Detect manual, we chose to use ObsID 2405 (part of Chandra Deep Field South) as a test field because it contributes 10% of the total exposure time of the Deep Field South. From running detect on the full exposure, we know the locations of all real sources which could be detected from the short exposure.
For this comparison, we have run the tools with their default parameter values. For celldetect, we used a S/N threshold of 3, an encircled energy of 0.9, and the PSF library of 2001 April. For wavdetect, we used scales of 1,2,4,8, & 16 and sigthresh = E-6. We found 71 detections for the block=2 image and 26 for the inner 1024 x 1024 area at block=1; of these, only 4 were unique. For vtpdetect, we used scale = 1 and a maximum probability of being a false source (limit) equal to E-5 and E-6. The results are shown in the following table.
|# of Edge
|# of Spurious
|# of Valid
The distinction between "spurious" and "edge" is somewhat subjective. In the case of wavdetect , the edge detections were close to, but not actually on, the edge. The percent spurious assumes that these detections don't count; i.e. if we had used an exposure map, they probably would not have been detected. By adjusting various parameters, each tool could be made to perform better or worse; the values presented here are indicative of the norm.
Running The CIAO Detect Tools
Separate threads have been created for each of the CIAO Detect tools. Each thread illustrates running the tool, selecting appropriate parameter values, etc.:
Output of Detect: _src Files
All three Detect programs have a common output format, a _src file.
For each source, these FITS files contain the RA, Dec of each source (with uncertainties), the net source counts and background (with uncertainties), signal-to-noise ratio, the region descriptor for the source and a number of parameters that depend on the detect method employed. The Detect manual gives a complete list of the file columns and their meanings in the "Data Product Description" sections.
The Using the Output of Detect Tools thread illustrates how to use the source list in CIAO data analysis.
The CIAO detect tools are solely intended to detect candidate sources. The source properties from the different tools, including position, net counts, morphology (point vs. extended), are only approximations. Users should perform detailed analysis of sources and not rely solely on the output from the detect tools.
|14 Dec 2004||reviewed for CIAO 3.2: no changes|
|03 Jun 2005||updated links for CIAO 3.2 version of Detect manual|
|12 Dec 2005||reviewed for CIAO 3.3: no changes|
|01 Dec 2006||reviewed for CIAO 3.4: no changes|
|16 Jan 2008||updated for CIAO 4.0: added information about new wavdetect algorithm|
|12 Jan 2009||updated for CIAO 4.1: wavdetect algorithm was improved in this release; updated wavdetect memory requirements|
|29 Jan 2010||updated for CIAO 4.2: removed outdated runtime information|
|13 Jan 2011||reviewed for CIAO 4.3: no changes|
|09 Jan 2012||reviewed for CIAO 4.4: no changes|
|03 Dec 2012||Review for CIAO 4.5; no changes|
|25 Nov 2013||Review for CIAO 4.6. No changes.|
|17 Dec 2014||Review for CIAO 4.7. Added a short note about source properties.|
|13 Jan 2020||Reviewed for CIAO 4.14. No changes.|