Last modified: December 2013

Jump to: Description · Example · Bugs · See Also

AHELP for CIAO 4.11 Sherpa v1


Context: utilities


Calculate the significance using the F test


calc_ftest(dof_1, stat_1, dof_2, stat_2)


The calc_ftest command computes the significance using the F test with the degrees of freedom of the simple model (dof_1) and its best-fit statistic (stat_1), along with the degrees of freedom of the complex model (dof_2) and its best-fit statistic (stat_2).

The F-test is a model comparison test. Model comparison tests are used to select from two competing models which best describes a particular data set. A model comparison test statistic, T, is created from the best-fit statistics of each fit; as with all statistics, it is sampled from a probability distribution p(T). The test significance is defined as the integral of p(T) from the observed value of T to infinity. The significance quantifies the probability that one would select the more complex model when in fact the null hypothesis is correct. A standard threshold for selecting the more complex model is significance < 0.05 (the "95% criterion" of statistics).

calc_ftest uses the ratio of the reduced chi2 which follows the F-distribution, (chi2_1/dof_1)/(chi2_2/dof_2). When calculating the significance, Sherpa uses the incomplete Beta function to obtain the integral of the tail of the F-distribution. The significance, or p-value, is returned by calc_ftest. If significance is < 0.05, the more complex model is selected.

The F-test may be used if:

If these conditions are fulfilled, then the observed F statistic is sampled from the F distribution, whose shape is a function of dof_1 and dof_2. (The tail integral may be computed analytically using an incomplete beta function; see any basic statistics text for details.) If these conditions are not fulfilled, then the F-test significance may not be accurate.


sherpa> calc_ftest(2, 20.28, 34, 33.63)

Calculate the F statistic for where the simple model has 2 degrees of freedom and a best-fit statistic of 20.28 and the complex model has 34 degrees of freedom and a best-fit statistic of 33.63.


See the bugs pages on the Sherpa website for an up-to-date listing of known bugs.

See Also

copy_data, dataspace1d, dataspace2d, datastack, delete_data, fake, get_axes, get_bkg_plot, get_counts, get_data, get_data_plot, get_dep, get_dims, get_error, get_quality, get_specresp, get_staterror, get_syserror, group, load_ascii, load_data, load_grouping, load_quality, set_data, set_quality, ungroup, unpack_ascii, unpack_data
get_filter, load_filter, set_filter
get_default_id, list_data_ids, list_response_ids
plot_data, set_xlinear, set_xlog, set_ylinear, set_ylog
save_error, save_filter, save_grouping, save_quality, save_staterror, save_syserror
calc_data_sum, calc_data_sum2d, calc_kcorr, calc_mlr, calc_model_sum2d, calc_source_sum2d, get_rate
contour, contour_data, contour_ratio, get_ratio, get_resid, histogram1d, histogram2d, image_data, rebin