+----------------------------------------------------+
| CALL HDIFFB (ID1,ID2,TOL,NBINS,CHOPT,NBAD*,DIFFS*) |
+----------------------------------------------------+
Action: Compare two histograms, bin by bin. For each bin, return the probability that the contents are from the same distribution. For details of the method see below.
The comparison may be done between two 1-dimensional histograms, two 2-dimensional histograms, or between two profile histograms.
HDIFFBWhen to use HDIFFB instead of HDIFF:
Thus HDIFFB is appropriate when:
A plot of hits versus detector element, where the detector elements cover some angular range, is an example of a histogram which might be considered with either comparison utility. The choice depends on the question you wish to answer:
If you choose 0.05 for TOL, you should expect 5 or so bad bins per trial from a histogram with 100 channels. For monitoring, you must compromise between the number of false messages you can tolerate (based on the total number of channels you monitor), and the amount of data you will need to collect to claim a channel is bad. In general, a somewhat smaller fraction of channels than TOL will be flagged as bad, since for discrete distributions (Poisson statistics), the probability is quantized. For example, the probability might be 0.053 for 4 entries, and 0.021 for 3. If TOL=0.05, only bins with 3 or fewer entries would be flagged as bad.
The S option should be used when both histograms are filled with statistical data, for example a momentum distribution from two successive data runs. Using the S option when comparing data to a function or known reference yields poor results because it attributes errors to both histograms. In this case, the C option should be selected.
The C option assumes that the reference histogram contains the theoretically expected values with no (or negligible) errors. Examples might be a flat distribution hand-inserted as the expectation for a phi distribution, or a long data run to be compared with shorter data runs.
The A option can be used as an equivalent to the C option by choosing TOL in terms of standard deviations instead of probability, and returns z values in DIFFS for each bin. The A option is intended for setting by hand absolute minima and maxima. To restrict an efficiency between 80 and 100%, load the reference histogram with a mean of 0.9 (via HPAK) and the error bar of 0.1 (via HPAKE), and use HDIFFB with TOL=1.0 and the A option. The N option should also be selected for this application.
This is in general undesirable, as it forces you into the less accurate Gaussian approximation. Thus it is preferable, for example, to have unweighted Monte Carlo events if you need to use HDIFFB to compare with data. The only useful case is if the weighted histogram is the reference histogram in the C comparison, which only makes sense if you have much better accuracy than your data.
The N option is irrelevant for profile histograms. The overflow/underflow options are illegal for profile histograms because insufficient information is stored to calculate the error bars. None of the test options (S, C, or A) check on the number of entries in a profile histogram bin. (To do that, make a separate 1-dimensional histogram.) This has an unexpected effect when the number of entries are small. Bins with no entries always pass the S and C options (no data is compatible with any distribution), so in such cases more bins pass than called for by TOL.
The value of DIFFS may depend somewhat on the value of TOL chosen, as the approximation chosen to calculate DIFFS depends on both the number of entries and on the size of TOL (how accurately DIFFS must be calcuated).
The S option sometimes returns a confidence level of 1.0 in the small statistics calculation, i.e. there is no probability that the two numbers came from different distributions. This is due to finite precision. Values slightly higher than 1.0 will be returned when the two content values are identical, since no statistical test could claim they came from different distributions.
The normalization scaling (used unless N option selected) is based on channel contents for all channels requested (including overflow/underflow), provided you select one of the overflow/underflow options.
Negative bin contents are flagged as bad bins in S, C options.
The methods used for the S and C options are correct for unweighted events and Poisson statistics for 1- or 2- dimensional histograms. Errors may result in either the S and C options for small tolerances if bin contents are greater than the largest allowed integer.
For the S option with unweighted events, the test (which is uniformly most powerful) treats N = sum of the two bin contents as having chosen via a binomial distibution which histogram to enter. The binomial parameter p is given by the relative normalization of the histograms (0.5 if the total number of entries in each histogram was the same). For DIFFS values greater than TOL, the first two digits are correct. For values less than TOL, the two digits to the right of the first non-zero TOL digit are significant, i.e. for TOL=0.0001, 0.000xxx are significant. One can force higher accuracy by setting TOL smaller (or even 0), but calculation time will increase, and warning messages will be issued. A Gaussian approximation is used when there are 25 or more events in each bin, and TOL>0.001.
The C option for unweighted events in the data histogram simply calculates the Poisson probability of finding n, the ID2 bin value, given a mean equal to the bin value of ID1. A Gaussian 6 approximation is used when the the mean is 10 or larger, and TOL is 0.001 or larger. Given the expected mean, the choice of TOL implies bounds (n ,n ) on n (i.e. n within these bounds < >
passes). An error occurs when the approximations used in calculating DIFFS give an incorrect value for n or n . No such errors occur < > 5 -15 for mean <10 and TOL >10 . The errors in n or n are less < > 6 -6 7 than 2 for mean <10 , TOL >10 , or mean <10 , TOL -5 >10 . There is a maximum n beyond which DIFFS returns zero, so
7 bins with n>n always fail. For mean <10 , this is irrelevant for max -9 values of TOL >10 .
For the profile histogram S option, HDIFFB calculates the t test probability that both bin means were produced from a population with the same mean. The C option calculates the probability of finding the value in ID1 given a Gaussian with mu and sigma given by the ID2 contents. Small numbers of entries for either test give DIFFS values which are too large, and HDIFFB will reject too many events in profile histograms.
For weighted events, the S and C options use a Gaussian approximation. This results in DIFFS values which are too low. HDIFFB rejects too many bins for weighted events, particularly for small numbers of equivalent events.
The methods used for the S and C mode are correct for unweighted events and Poisson statistics for one or two-dimensional histograms. For weighted events, a Gaussian approximation is used, which results in DIFFS values which are too low when there are fewer than 25 or so ``equivalent events'' (defined under HSTATI) per bin. This is caused by either few entries or by wide fluctuation in weights. The result is that HDIFFB rejects to many bins in this case.
Comparisons for profile histograms assume Gaussian statistics for the S and C mode comparisons of the channel mean. Fewer that 25 or so events will result in DIFFS values which are too large. The result is that HDIFFB rejects too many event in these low statistic cases.