A quasi-exact method for the confidence intervals of the difference of two independent binomial proportions in small sample cases

In this paper we propose a quasi-exact alternative to the exact unconditional method by Chan and Zhang (1999) estimating confidence intervals for the difference of two independent binomial proportions in small sample cases. The quasi-exact method is an approximation to a modified version of Chan and Zhang's method, where the two-sided p-value of an observation is defined by adding to the one-sided p-value the sum of all probabilities of more "extreme" events in the unobserved tail. We show that distinctively less conservative interval estimates can be derived following the modified definition of the two-sided p-value. The approximations applied in the quasi-exact method help to simplify the computations greatly, while the resulting infringements to the nominal level are low. Compared with other approximate methods, including the mid-p quasi-exact methods and the Miettinen and Nurminen (M&N) asymptotic method, our quasi-exact method demonstrates much better reliability in small sample cases.