Description

GOODNESS reports to the user information about how well specified models fit to the data. At a minimum, it reports: the choice of statistic; the number of bins in the fit; the number of degrees of freedom (dof), i.e., the number of bins minus the number of free parameters; and the statistic value. (See the documentation on the command STATISTIC for more information on how to set the current statistic within Sherpa.)

If the chosen statistic is one of the chi-square statistics, or the CSTAT statistic, then more information is shown: the reduced statistic, i.e., the statistic value divided by the number of dof; and the probability, or Q-value:

Q = (integral)_(X^2_obs)^(infinity) dX^2 p(X^2 | N-P) ,

where X^2 is the chi-square statistic, X^2_obs represents a specific observed value of chi-square (e.g., resulting from a fit), N-P is the number of degrees of freedom (number of bins minus number of free parameters), and p(X^2 | N-P) is the chi-square probability sampling distribution.

Q measures the probability that one would observe the value X^2_obs, or a larger value, if the assumed model is true and the best-fit model parameters are the true parameter values. A value that is too small (e.g., Q < 0.05) indicates that the selected model does not accurately portray the data, while a value that is too large (Q -< 1) indicates that the fit is ``too good." The usual cause of a fit that is too good is an overestimation of the errors (e.g., by using CHI GEHRELS in the low-counts regime (see note below), or by adding in too much systematic error). Increasing the errors decreases X^2_obs, and increases Q.

Note that the accuracy of Q is dependent upon whether the selected statistic is actually sampled from the chi-square distribution! This may not be the case if the number of counts in any bin is too small (< 5-10).

The information output by GOODNESS may be retrieved using the Sherpa/S-Lang module function get_goodness.