Skip to the navigation links
Last modified: 14 December 2020

URL: https://cxc.cfa.harvard.edu/sherpa/index.html

CIAO's modeling and fitting package


Sherpa is the CIAO modeling and fitting application. It enables the user to construct complex models from simple definitions and fit those models to data, using a variety of statistics and optimization methods (see the Gallery of Examples).

CIAO 4.13

Sherpa version for CIAO 4.13 was released on December 15, 2020. Sherpa in CIAO runs under Python 3.8, 3.7, or 3.6 (when installed using the conda package manager) and Python 3.7 (when installed with ciao-install). The full list of the Sherpa updates is given in the Release Notes. The major updates in this release include:

XSPEC

Update to version 12.10.1s of the XSPEC model library, from version 12.10.1n in CIAO 4.12 (so there is essentially no change).

Users who want to use XSPEC 12.11.1 will need to build Sherpa themselves.

Support for XSPEC convolution models has been added: xscflux, xsclumin, xscpflux, xsgsmooth, xsireflect, xskdblur, xskdblur2, xskerrconv, xskyconv, xslsmooth, xspartcov, xsrdblur, xsreflect, xsrfxconv, xsrgsxsrc, xssimpl, xsvashift, xsvmshift, xsxilconv, xszashift, and xszmshift.

Changes to the Sherpa configuration file

The default Sherpa configuration file has been updated for CIAO 4.13 to add the multiprocessing section, which contains the multiprocessing_start_method, and to remove the chips section, as it is no-longer used.

If you have trouble running Sherpa it is worth removing your existing resource file and then re-starting Sherpa (which will create a new version) to see if this fixes problems: for example

unix% mv ~/.sherpa.rc ~/.sherpa.old.rc
unix% sherpa
...
Multiprocessing, Python 3.8, and macOS

In Python 3.8 on macOS the multiprocessing module has changed how it starts processes. As the new default does not work for Sherpa, the multiprocessing.multiprocessing_start_method setting has been added to the Sherpa configuration file to allow this setting to be changed, but please contact the CXC Helpdesk for help if you find you need to change this setting.

Plotting improvements

CIAO 4.13 continues the improvements made in CIAO 4.12 to the plotting functionality, and includes:

  • changes to the display of PHA and histogram data and models;
  • the transparency of the data or model can be changed by setting the alpha argument to a value between 0 and 1;
  • the plot function can now accept keyword arguments, with the settings applied to each plot;
  • and the overplot option now works for the plot_fit_xxx and plot_bkg_fit_xxx family of commands (e.g. plot_fit_ratio and plot_bkg_fit_resid).
Flux calculations

Extending the CIAO 4.12.1 changes to the calc_energy_flux and calc_photon_flux commands, there have been a number of improvements to the sample_energy_flux and sample_photon_flux commands. The main change is the introduction of the model parameter, so you can now calculate the flux for a component (e.g. the unabsorbed flux). The documentation has been improved and the support for the scales array no correctly handles both NumPy and list inputs.

The How do we calculate a flux in Sherpa Jupyter notebook has been updated to discuss this new functionality.

PHA data handling

There have been a number of improvements and fixes to PHA data handling. In particular fitting a model to the background dataset has been re-worked and can result in different results if the source and background PHA datasets have different exposure times.

The load_pha command will now use the id argument when loading in a PHA2 file: it defines the first dataset which will be loaded whereas previously it always started at 1.

The ungroup and unsubtract commands will no-longer error out if used on ungrouped or unsubtracted data.

Documentation

The Sherpa ahelp files have been updated to match the Python docstrings. Each command now has its own ahelp file, rather than combining multiple commands into a single file.

Jupyter notebooks

Many of the objects created by Sherpa will now take advantage of Jupyter notebooks to display a HTML table or an actual plot. As an example, load a PHA file with load_pha and then call get_data, or call get_source to display a model.

Handling asymmetric errors

The resample_data routine, used to estimate the effect of asymmetric errors, has seen a number of improvements in behavior and the data that is returned.

Numpy masked arrays

Numpy masked arrays can now be used to create a dataset with load_arrays or any of the Data classes.

Model combination

The dimensionality of models is now checked, so it is no longer to add a 1D model to a 2D model.

The voigt1d and pseudovoigt1d models

The absorbtionvoigt and EmissionVoigt models were problematic and have been removed. They have been replaced by the more-generic voigt1d and pseudovoigt1d models.

Sherpa lets you:
  • fit 1-D data sets (simultaneously or individually), including:
    spectra, surface brightness profiles, light curves, general ASCII arrays;

  • fit 2-D images/surfaces in the Poisson/Gaussian regime;

  • visualize the data with Matplotlib and DS9;

  • access the internal data arrays;

  • build complex model expressions;

  • import and use your own models;

  • choose appropriate statistics for modeling Poisson or Gaussian data;

  • import new statistics, with priors if required by analysis;

  • visualize a parameter space with simulations or using 1-D/2-D cuts of
    the parameter space;

  • calculate confidence levels on the best-fit model parameters;

  • choose a robust optimization method for the fit: Levenberg-Marquardt,
    Nelder-Mead Simplex or Monte Carlo/Differential Evolution;

  • perform Bayesian analysis with Poisson Likelihood and priors, using
    Metropolis or Metropolis-Hastings algorithm in the MCMC (Markov-Chain Monte Carlo);

  • and use Python to create complex analysis and modeling functions,
    build the batch mode analysis or extend the provided functionality
    to meet the required needs.

[] []
[] []
[] []

The Sherpa infrastructure greatly enhances the default Sherpa functions, and provides users with an environment for developing complex and sophisticated analysis.

Sherpa is designed for use in a variety of modes: as a user-interactive application and in batch mode. Sherpa is an importable module for the Python the scripting language. In addition, users may write their own Python scripts for use in Sherpa.

The About Sherpa page outlines key features of the software, and the Latest Updates page describes new functionality and recent changes. See also the SciPy2009 conference proceedings for detailed information about Sherpa's capabilities.

If you have ideas about how to enhance or improve Sherpa, please contribute ideas (and code) to the Sherpa GitHub repository.

Please send feedback and questions on Sherpa to the CXC Helpdesk or the Sherpa Issues list on GitHub.


Citing Sherpa in a Publication

[New] New in CIAO 4.13 is the sherpa.citation method which will return infromation from the latest release on Zenodo:

sherpa> sherpa.citation('latest')

If you are writing a paper and would like to cite Sherpa, we recommend the following papers and presentations:

Sherpa: a mission-independent data analysis application (ADS)
P. E. Freeman, S. Doe, A. Siemiginowska
SPIE Proceedings, Vol. 4477, p.76, 2001

\bibitem[Freeman et al.(2001)]{2001SPIE.4477...76F} Freeman, P., Doe, S., 
\& Siemiginowska, A.\ 2001, \procspie, 4477, 76

The specific version of CIAO and CALDB (if applicable) used for the analysis should be mentioned as well.

A reference for the Python interface to Sherpa is also available:

Developing Sherpa with Python (ADS)
S. Doe, et al.
Astronomical Data Analysis Software and Systems XVI, 376, 543

\bibitem[Doe et al.(2007)]{2007ASPC..376..543D} Doe, S., et al.\ 2007, 
Astronomical Data Analysis Software and Systems XVI, 376, 543

Sherpa: 1D/2D modeling and fitting in Python (SciPy 2009)
B. Refsdal, S. Doe, D. Nguyen, A. Siemiginowska, N. Bonaventura, D. Burke, I. Evans, J. Evans, A. Fruscione, E. Galle, J. Houck, M. Karovska, N. Lee, M. Nowak
Proceedings of the 8th Python in Science Conference (SciPy 2009), G. Varoquaux, S. van der Walt, J. Millman (Eds.), pp. 51-57 2009

Fitting and Estimating Parameter Confidence Limits with Sherpa (SciPy 2011),
B. Refsdal, S. Doe, D. Nguyen, A. Siemiginowska, V. Kashyap
Proceedings of the 19th Python in Science Conference (SciPy 2011), S. van der Walt, J. Millman (Eds.), pp. 4-10 2011

Further guidelines are available from the Acknowledgment of Use of Chandra Resources.