Next: , Previous: Implementation, Up: Top


Appendix C Verification and Validation

spatquant has undergone testing on several fronts to ensure that it works as advertised. Tests can be divided into those which test the front end (the bin manipulation routines) and those which deal with the back ends, which provide information about which bin a ray goes in. Some tests are applicable to both groups.

The following set of tests were performed with a set of rays which uniformly filled a square with sides of length 1.5. The rays were generated by genphot

  1. This test checks edge effects for the ‘rectrect’ grid and binning scheme. It checks that all rays are assigned to the correct bin, an operation which is sensitive to round-off error at bin edges. It also tests if the front end correctly deals with clipping in sparse and core modes. Because clipping is turned on, the backend makes decisions about which rays are to be accumulated.

    Clipping was turned on, and the grid was set to have bin widths of 0.5 and heights of 1.5. This exactly covers the input rays. The weights of the accumulated rays in each bin was compared against those expected from an independent summation. All input rays were properly accumulated (none were lost, and the correct number was found in each bin). A similar test, but switching the dimensions and number of bins between x and y, was performed, with the same results. These tests were performed with spatquant in both core and sparse mode (the latter turned on by setting bufsz to 0).

  2. The same test as in the previous item was performed with clipping off. The backend no longer enforces limits in this setup. It results in the same output as the previous test (as it should).
  3. When in sparse mode, the handling of bins is not symmetric in i and j. In order to ensure that the internal trees are being built correctly, a set of tests were run with the ‘rectrect’ gridding and binning scheme. Each test involved comparing a set of rays filtered through spatquant in ‘mask’ filter mode and a set with their x and y values swapped before and after the pass through spatquant. Both sets of rays were sorted in the order of increasing y and then x.

    In order to ensure that the ‘rectrect’ scheme had no problems of its own, the test was run with clipping turned off.

    The remaining tests were run with clipping on. Sparse mode was turned on by setting bufsz to 0. Core mode was ensured by setting bufsz to a value larger than the size of the grid requirements. Tests were made for core mode, sparse mode, and a cross-comparison, to ensure that they both returned the same results. Comparison was done on a ray by ray basis for equivalence of position and weight.

  4. The rays were passed through spatquant with filter_mode set to ‘mask’ and ‘bin’ with clipping on and off, for schemes ‘rectrect’, ‘hexcirc’, and ‘rectcirc’. For ‘rectrect’ the bins were square, with sides of 0.1. For ‘hexcirc’ and ‘rectcirc’ the diameter was 0.01. When clipping, n_x and n_y were set to 40.

    The total weight and number of rays output for each permutation was tallied. An image was created with format ‘binary-hdr’ and translated into fake rays with the same weight for each permutation. The weights of the images were compared to those of the output rays. For the cases where accumulation took place, the number of nonzero pixels in the images were compared to the number of output rays.