About Sherpa

Sherpa is the CIAO modeling and fitting application; it is provided for interactive analysis of Chandra X-ray data. However, it is general enough to be equally useful for analysis of images, spectra and time series from many other missions (even from optical missions such as Hubble).

The first version of the redesigned Sherpa became available with the CIAO 4.1 release. There are several new features in this version of Sherpa, as well as a few missing in comparison to Sherpa in CIAO 3.4; this document highlights these items. For important updates to Sherpa functionality which become available with each CIAO release, see the Sherpa page "Latest Updates", as well as the "Sherpa" section of the CIAO Release Notes.

The redesigned Sherpa is meant to be flexible, modular and extensible. As such, Sherpa is an importable module for both the Python and S-Lang scripting languages, with user interfaces in both languages. Sherpa model, optimization and statistic functions are also available via both C++ and Python for software developers wishing to link such functions directly to their own compiled code. The CIAO 4.1 release of Sherpa also contains many improvements over the beta release of Sherpa that was part of CIAO 4.0.

The CIAO/Sherpa 4.1 release supports fitting 1-D X-ray spectra from Chandra and other X-ray missions. It also supports fitting of 1-D non-X-ray data, including ASCII data arrays, radial profiles, and lightcurves. The options for grating data analysis include fitting the spectrum with multiple response files required for overlapping orders in LETG observations. Modeling of 2-D spatial data is fully supported, including the PSF and exposure maps.

New Features

Python and S-Lang environments

The Sherpa interface is compatible with both the S-Lang and Python scripting languages. Users can select which language they prefer for data analysis; see the page Choosing Python or S-Lang and the Sherpa Startup thread for more information.

Python and S-Lang allow scripting, mathematical calculations, and access to libraries of scripts written by the community. The new ChIPS plotting package has a compatible interface, so that both applications can be used within the same environment. The Crates interface (S-Lang or Python help) provides a way to read FITS and ASCII data into Python or S-Lang from files, retaining all the relevant metadata and providing CIAO Data Model filtering and binning capabilities.

Startup Options

There are four new command-line options which may be specified when Sherpa is started:

• -x : launch chips/sherpa shell in separate display terminal.
• -b : run in batch mode
• -l : specifies which language to use, "slang" or "python".
• -n : do not print banner

The same options may be used with ChIPS.

Accessing and manipulating data structures

In addition to the normal mode of loading data from files, Sherpa allows you to manipulate the data arrays internally at the Python/S-Lang prompt and then "re-load" them before fitting and modeling further. The resulting data may be manipulated further without writing it out to a file, making it easy to incorporate into scripts.

The analysis of data from modern instruments requires a flexible modeling and fitting system. The new infrastructure provides the user with an environment to develop complex and sophisticated analysis that goes beyond the default Sherpa functionality.

Optimization methods

There are three optimization methods in Sherpa 4.1: "levmar", a modification of the Levenberg-Marquardt algorithm which uses the LMDIF algorithm; "neldermead", A Nelder-Mead Simplex direct search; and "moncar", a Monte Carlo method - differential evolution - based on the paper by Hedar & Fukushima. These algorithms are a complete replacement of the OPTIM routines; they are more robust and show very good convergence required by modern data analysis.

Statistics

The XSpec variance method has been added to the available statistics in Sherpa; named "chi2xspecvar", it is a χ² method with data variance = 1.0 if the data are less than or equal to 0.0.

The least-squares method, "leastsq", is available for modeling data.

In addition, the new interface provides an easy way to define new statistics required for analysis by the user.

Parameter Uncertainties and Confidence Regions

Covariance and Projection, based on the CIAO 3.4 algorithm, calculate the confidence range for the best-fit model parameters; interval-projection and region-projection provide a way to visualize the model constraints on the best-fit parameter values. The results are easily accessible with the user functions get_proj_results (S-Lang or Python help) and get_covar_results (S-Lang or Python help).

User Interface

There are many new functions available for accessing internal data, model data, fit and projection results, plotting objects etc., in a Sherpa session. These provide the user with easy access and interaction with the system required by modern data analysis.

Missing or Changed Functionality with respect to CIAO 3.4

Commands and algorithms

Initial model parameter values are no longer automatically guessed in Sherpa; the guess command (S-Lang or Python help) is available for this purpose, but only for simple (i.e. single-component) models. This command makes an initial guess at parameter values to ensure convergence, but users should check that the initial range of parameter values (soft limits) is sensible for their data.

Grouping: additional functions from dmGroup that group on the x-grid will be added in a future release.

Certain keywords such as "data", "model", "fit", "bkg", can no longer be used as data set IDs.

Statistics

The "chi primini" and "bayes" statistics are not implemented.

Fitting

Multiplication of two integrated models, each with its own normalization, is not handled correctly. It results in incorrect fit parameters and an incorrect number of counts in fake_pha (S-Lang or Python help) simulations.

The sigma rejection method is not available.

User interface

There is no interface to the GUIDE atomic transition catalog.

Plotting

Overplotting of fit and model components: the ability to plot the contribution of individual model components over the best fit model is not an option among the plot functions. Users can use an overplotting argument and add additional model curves to the default plot.

Sherpa and ChIPS no longer have the ability to do surface plots of 2-D data. However, the support for contour plots is improved in the new version of ChIPS.

Plotting model-fitted spectra in photon flux units: there is no function in Sherpa 4.1 equivalent to the Sherpa 3.4 function "lplot UFIT", which plots fitted data in photon flux units; see Sherpa Plotting FAQ #2 for a Sherpa 4.1 routine which mimics its behaviour.

General infrastructure

The save command (S-Lang or Python help) creates a binary file that can be restored in Sherpa with the restore command (S-Lang or Python help). In the future, the save file will be written as an ASCII file which can be inspected and edited by the user. The Sherpa 4.1 save functions will evolve towards the Sherpa 3.4 functionality in a future release.

The ability to save fits in a format that may be manipulated by the user will also be provided (similar to the CIAO 3.4 MDL files). The support in CIAO3.4 was not entirely satisfactory, and that approach will be reviewed.