Skip to the navigation links
Search http://cxc.harvard.edu/sherpa/ Contact the CXC HelpDesk
Last modified: 12 December 2018

URL: http://cxc.harvard.edu/sherpa/index.html
[Sherpa logo]

CIAO's modeling and fitting package

WHAT'S NEW | WATCH OUT
Analysis Threads | Ahelp | Download CIAO || CIAO | ChIPS | PSF Central

Sherpa is the CIAO modeling and fitting application. It enables the user to construct complex models from simple definitions and fit those models to data, using a variety of statistics and optimization methods (see the Gallery of Examples).

Latest Version

Sherpa version for CIAO 4.11 was released on December 13, 2018. Sherpa in CIAO runs under Python 3.5. The major updates in this release include:

  • Update to version 12.10.0e of the XSPEC model library, which adds support for the following additive models: xsbrnei, xsbvrnei, xsbvvrnei, xsgrbcomp, xsjet, xsssa, and xszcutoffpl. The default value of the Gamma parameter of the xsoptxagnf model has been updated from 0.5 to 1.05 to match the change in XSPEC.

  • The implementation of the Levenberg-Marquardt optimiser has been changed from FORTRAN to C++. This should have no significant difference to users except that the optimiser now returns an estimate of the covariance matrix, which is shown on screen (after the parameter value) and in the covar field in the object returned by get_fit_results.

    An example of the new fit output, taken from the introductory Sherpa thread is shown below, where the new error estimates are shown in bold: 

    sherpa> fit()
Dataset               = 1
Method                = levmar
Statistic             = chi2
Initial fit statistic = 2815.14 
Final fit statistic   = 151.827 at function evaluation 6
Data points           = 11
Degrees of freedom    = 9
Probability [Q-value] = 3.68798e-28
Reduced statistic     = 16.8697
Change in statistic   = 2663.31 
   model1.c0      1.58227      +/- 0.0248858
   model1.c1      0.198455     +/- 0.00392255

    The CXC recommends that the conf function is used to estimate the errors, with the values reported above used as a guide.

  • The eqwidth function can now calculate an error estimate for the equivalent width; new routines - create_arf and create_rmf - have been added to simplify creation of ARFs and RMFs in simulations; and there is no experimental support for evaluting models on a different grid than the data, which can be used to evaluate higher-resolution models - which is then down-sampled to the data - or to include signal outside the data range, for use by convolution-style models.

  • CIAO 4.11 contains Matplotlib version 2.2.3 as well as ChIPS, and Sherpa can use either for plotting. The default behavior for Sherpa is to use ChIPS, but this can be set to Matplotlib by editing the ~/.sherpa.rc file so that the plot_pkg line is: 

    unix% grep plot_pkg ~/.sherpa.rc
plot_pkg : pylab

    The existing Sherpa documention still focuses on using ChIPS, but will be updated to reference Matplotlib. The Matplotlib for ChIPS users conversion guide may be useful.

The CIAO release of Sherpa is based on the GitHub-developed version of Sherpa. Please consider contributing to Sherpa development, whether by adding code, fixing bugs, or changing documentation. The documentation for this "stand alone" version of Sherpa may be useful, but is not aimed at CIAO users.

Sherpa lets you:

  • fit 1-D data sets (simultaneously or individually), including:
    spectra, surface brightness profiles, light curves, general ASCII arrays;

  • fit 2-D images/surfaces in the Poisson/Gaussian regime;

  • visualize the data with ChIPS or Matplotlib;

  • access the internal data arrays;

  • build complex model expressions;

  • import and use your own models;

  • choose appropriate statistics for modeling Poisson or Gaussian data;

  • import new statistics, with priors if required by analysis;

  • visualize a parameter space with simulations or using 1-D/2-D cuts of
    the parameter space;

  • calculate confidence levels on the best-fit model parameters;

  • choose a robust optimization method for the fit: Levenberg-Marquardt,
    Nelder-Mead Simplex or Monte Carlo/Differential Evolution;

  • perform Bayesian analysis with Poisson Likelihood and priors, using
    Metropolis or Metropolis-Hastings algorithm in the MCMC (Markov-Chain Monte Carlo);

  • and use Python to create complex analysis and modeling functions,
    build the batch mode analysis or extend the provided functionality
    to meet the required needs.
[] []
[] []
[] []

The Sherpa infrastructure greatly enhances the default Sherpa functions, and provides users with an environment for developing complex and sophisticated analysis.

Sherpa is designed for use in a variety of modes: as a user-interactive application and in batch mode. Sherpa is an importable module for the Python the scripting language and is available as a C/C++ library for software developers. In addition, users may write their own Python scripts for use in Sherpa.

The About Sherpa page outlines key features of the software, and the Latest Updates page describes new functionality and recent changes. See also the SciPy2009 conference proceedings for detailed information about Sherpa's capabilities.

Please send feedback and questions on Sherpa to the CXC Helpdesk or the Sherpa Issues list on GitHub.

Citing Sherpa in a Publication

If you are writing a paper and would like to cite Sherpa, we recommend the following papers and presentations:

Sherpa: a mission-independent data analysis application (ADS)
P. E. Freeman, S. Doe, A. Siemiginowska
SPIE Proceedings, Vol. 4477, p.76, 2001 

\bibitem[Freeman et al.(2001)]{2001SPIE.4477...76F} Freeman, P., Doe, S., 
\& Siemiginowska, A.\ 2001, \procspie, 4477, 76

The specific version of CIAO and CALDB (if applicable) used for the analysis should be mentioned as well.

A reference for the Python interface to Sherpa is also available:

Developing Sherpa with Python (ADS)
S. Doe, et al.
Astronomical Data Analysis Software and Systems XVI, 376, 543 

\bibitem[Doe et al.(2007)]{2007ASPC..376..543D} Doe, S., et al.\ 2007, 
Astronomical Data Analysis Software and Systems XVI, 376, 543

Sherpa: 1D/2D modeling and fitting in Python (SciPy 2009)
B. Refsdal, S. Doe, D. Nguyen, A. Siemiginowska, N. Bonaventura, D. Burke, I. Evans, J. Evans, A. Fruscione, E. Galle, J. Houck, M. Karovska, N. Lee, M. Nowak
Proceedings of the 8th Python in Science Conference (SciPy 2009), G. Varoquaux, S. van der Walt, J. Millman (Eds.), pp. 51-57 2009

Fitting and Estimating Parameter Confidence Limits with Sherpa (SciPy 2011),
B. Refsdal, S. Doe, D. Nguyen, A. Siemiginowska, V. Kashyap
Proceedings of the 19th Python in Science Conference (SciPy 2011), S. van der Walt, J. Millman (Eds.), pp. 4-10 2011

Further guidelines are available from the Acknowledgment of Use of Chandra Resources.