This document highlights important changes and additions to Sherpa functionality in the CIAO 4.4 Sherpa release.
- Bayesian Analysis with pyBLoCXS
pyBLoCXS, a sophisticated Markov chain Monte Carlo based algorithm designed to carry out Bayesian Low-Count X-ray Spectral (BLoCXS) analysis in the Sherpa environment, is available via the functions listed below. (For more information on BLoCXS, see van Dyk, D.A., Connors, A., Kashyap, V.L., & Siemiginowska, A. 2001, Ap.J., 548, 224, "Analysis of Energy Spectra with Low Photon Counts via Bayesian Posterior Simulation")
The code is a Python extension to Sherpa that explores parameter space at a suspected minimum using a predefined Sherpa model to high-energy X-ray spectral data. The get_draws function runs a pyBLoCXS chain using fit information associated with the specified data set(s) and the currently set sampler and parameter priors, for a specified number of iterations. It returns an array of statistic values, an array of acceptance Booleans, and a 2-D array of associated parameter values.
- Evaluating Fit Results
Likelihood Ratio Test
Sherpa includes the new function plot_pvalue for performing a likelihood ratio test to compare a fit to data done with a simple, null model versus a more complex, alternative model. This function plots a histogram of likelihood ratios comparing fits done with a specified null model to fits done with the alternative model, using data simulated with Poisson noise. It computes the likelihood ratio and the p-value (value used to reject or accept the null model) using the observed data.
The get_pvalue_plot command accesses the histogram plot of the likelihood ratios comparing fits of the null model to fits of the alternative model using faked data with Poisson noise.
Random Sampling of Thawed Model Parameters
The following new functions are available in Sherpa for sampling the current set of thawed model parameters from a specific distribution, and computing the current statistic value on the set.
- Visualizing Statistical Distributions
There are new statistics plot functions associated with the pyBLoCXS module to visualize statistical distribution functions.
The functions listed above create plots of the probability density function (pdf) of a specified array of histogram values, the cumulative pdf, the trace plot, and a scatter plot of x and y data values, respectively. The associated "get" functions provide access to the data arrays and prefences which define the plots.
- New Models
CIAO 4.4.1 (27 June 2012)
The CIAO 4.4.1 version of XSpec has been updated to the current XSpec 12.7.1 release, and the Sherpa XSpec module has been updated to link to the XSpec 12.7.1 model libraries. In addition, new Sherpa interfaces have been added to provide access to the following XSpec additive models:
(As of the date of this entry, HEASARC has released XSpec patches 12.7.1a and 12.7.1b. These patches are not included in CIAO 4.4.1, as neither patch affects the XSpec model libraries to which Sherpa links.)
CIAO 4.4 (15 December 2011)
The following optical models are now available in Sherpa for fitting spectra or SEDs at optical wavelengths. These include continuum models, absorption and emission features, and extinction curves for the ISM at optical and UV wavelengths. The position, line width, and other similar model parameters have units of Angstroms where relevant.
New XSpec models included in Sherpa are:
The XSPEC model library has been upgraded to version 12.7.0. (As of 10/11/11, XSPEC patches up to 12.7.0o have been released, but none affect the model library. Therefore, none of the 12.7.0 patches are included in CIAO 4.4.)
The ATOMDB v2.0.1 database is now distributed with CIAO 4.4 and can be used with XSpec models (such as the APEC models) that depend on the database. The set_xsxset command may be used to specify the version of the ATOMDB atomic database which should be accessed by XSpec models in Sherpa.
The save command now records the XSpec-related settings, such as abundance and xsect.
- Table Models
Table models now support linear, nearest-neighbor, and polynomial interpolation. (Interpolation is used by both table and template models to match the data grid to the model grid, which must match before the fit statistics can be calculated for fitting.)
- Template Models
Sherpa supports a new template model (an extension of table model). Sherpa can now read in a collection of templates from a directory full of template files, and can compare a data set to all the templates in that collection. Sherpa finds the template that best matches the data, and reports back the parameter values associated with that template.
The files that contain template models can be read into Sherpa via the new load_template_model function, which has a similar signature to load_table_model. In the simplest case, the user will have a directory containing a set of ASCII template model files, along with a single ASCII index file which lists the various templates in that directory. The index file is specified in the "templatefile" parameter of load_template_model, and the various templates are loaded into a Sherpa model instance assigned the ID specified in the "modelname" parameter.
The gridsearch optimization method will select the best template model by trying all of the templates. This method evaluates the fit statistic for each point in the parameter space grid; the best match is the grid point with the lowest value of the fit statistic. gridsearch reports back the parameter values associated with this point.
The Sherpa template model supports nearest-neighbor and polynomial interpolation.
- Optimization Methods
There is a new grid searching optimization method, called gridsearch, designed for use with the Sherpa template model. The user must provide this method with a parameter space grid to search. The method will evaluate the fit statistic for each point in the grid; the best match is the grid point with the lowest value of the fit statistic. The grid search method will report back the parameter values associated with this point.
The Monte Carlo optimization method has been significantly improved, and can now find more reasonable parameter values in cases where it formerly came up against parameter limits.
The LMDIF optimization method has also been updated to avoid getting "stuck" at the parameter space boundaries for certain cases.
- Grouping and Filtering Data
In Sherpa in CIAO 4.4, it is no longer necessary to re-apply a filter to a data set after grouping the data channels with one of the Sherpa group commands. In previous versions of the software, grouping a data set would remove any filter which had been applied to the data with one of the ignore/notice commands.
- Additional Enhancements & Bug Fixes
PSF Instrument Models
- 2-D PSF kernels can now be larger than the source data (previously, the kernel was restricted to be equal to or smaller than the source). FFT padding 2n occurs using the largest of the two sizes. The PSF padding is removed down to the size of the source data.
- A fix to the PSF utility 'normalize': it no longer returns NaN when the PSF kernel is all zeros.
- The WCS in 2-D PSFs kernels is changed according to subkernel extraction. In previous versions of Sherpa, the WCS of subkernel images is wrong. The image sources now line up according to the WCS, consistent with CIAO 3.4. NOTE: this fix only affects imaging. The kernel WCS is not used during fitting.
- The normalization of a kernel now occurs before kernel extraction for both file-based and model-based PSFs.
- The coordinate system of kernel plots/contours is now based on the kernel or source data coordinate system. Originally, the coordinate system was based on the number of bins in each dimension.
- The location of model-based PSFs is now updated based on PSF parameters when set_psf is used to associate the kernel with a source model. This is simply a convenience if the user forgets to update the kernel model parameters.
- 1-D PSF plots (from file and model) are exclusively scatter plots. (Model PSFs are no longer plotted as a red line).
- 2-D PSF contours (from file and model) are exclusively scatter plots. (Model PSFs are no longer plotted as a red contour lines).
Manually Definining Models
The OGIP AREASCAL keyword within Type I and II PHA files is now supported in Sherpa. Like XSpec, Sherpa incorporates the PHA AREASCAL header keyword value into the background scaling factor used during background subtraction and simultaneous modeling of source and background spectra.
There is a new math function from the CEPHES library, "inverse erf", in the sherpa.utils module. This function is useful for converting between confidence limits and confidence intervals (sigma/percent).
PyFITS 3.0 is now supported in the alternate PyFITS I/O backend.
CIAO 4.4.1 (27 June 2012)
Sherpa contains two updates for reading in files in the CIAO 4.4.1 patch release. These updates fix critical bugs in Sherpa's interface to the PyCRATES module.
- The first CIAO 4.4 bug in Sherpa was in the load_pha() function. For some PHA files, load_pha() found the image instead of the PHA spectrum. This happened when the primary image was not null, and the name of the image extension was anything other than WMAP. For PHA files with such images, Sherpa tried and failed to load the image instead of the PHA spectrum. This patch ensures that the PHA spectrum is always loaded from a PHA file, no matter what image is contained in the first extension of the file.
- The second CIAO 4.4 bug involved cases where a file's Data Model syntax was not passed from Sherpa scope to PyCRATES, as it should have been. In such cases, extra, unwanted columns were read in from file and used in the Sherpa data set, even though the user had indicated via DM syntax that such columns were to be excluded. This patch ensures that Sherpa passes Data Model syntax to PyCRATES.
CIAO 4.4 (15 December 2011)
- Several significant bugs associated with the functions set_full_model() and set_bkg_full_model() have been fixed. These functions are intended to allow users to define more complex models (i.e. models with some components that are folded through an RMF or a PSF and other components that are not).
- The axis scale of plots output by the plot_model_component and plot_source_component commands could not be changed to log scale with the set_xlog and set_ylog commands.
- The log keyword argument was not working correctly for the reg_proj and reg_unc commands.
- The conf function was not using the 'numcores' setting in the .sherpa.rc file.