Abstract
A common problem in statistics is making multiple tests of hypotheses without controlling for the type I error rate. SAS has identified several different methods to adjust p-values for multiple testing. To compare the effect of these methods, an animal health dataset that deals with the treatment of cattle lice was examined. Clinical trials were conducted in Illinois and Wisconsin to evaluate the efficacy of two formulations of a new product Spinosad, two commercially available positive controls, and an untreated negative control. A baseline lice count was recorded prior to the treatment. After treatment, weekly measurements of lice counts were taken for 8 weeks. Counts of 4 lice species were recorded separately. A linear mixed model analysis was conducted for each species of lice after transforming the counts with a natural logarithm transformation. Simple contrasts between treatment groups at each week were performed. Treatment differences were also compared using 5 multiple testing methods: Bonferroni, Sidak, Holm’s step-down Bonferroni, Hochberg's step-up Bonferroni, and false discovery rate. Seventy-one out of 96 simple tests showed significant differences among the treatment groups. The five multiple testing methods confirmed only 48-67 significances out of the 96 tests. Comparatively, Bonferroni and Sidak methods provided similar and the most conservative multiplicity test results, i.e. fewest significant differences. The Holm’s step-down and Hochberg's step-up Bonferroni methods provided similar but less conservative results. Finally, the false discovery rate method provided the least conservative results.
Keywords
multiple tests, Bonferroni, Sidak, Holm’s step-down Bonferroni, Hochberg's step-up Bonferroni, false discovery rate
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
Recommended Citation
Cui, Zhanglin; Mowrey, Daniel H.; Zimmermann, Alan G.; and Hutchens, Douglas E.
(2006).
"A COMPARISON OF MULTIPLE TESTING METHODS: SPINOSAD AS A TREATMENT FOR LICE ON CATTLE,"
Conference on Applied Statistics in Agriculture.
https://doi.org/10.4148/2475-7772.1117
A COMPARISON OF MULTIPLE TESTING METHODS: SPINOSAD AS A TREATMENT FOR LICE ON CATTLE
A common problem in statistics is making multiple tests of hypotheses without controlling for the type I error rate. SAS has identified several different methods to adjust p-values for multiple testing. To compare the effect of these methods, an animal health dataset that deals with the treatment of cattle lice was examined. Clinical trials were conducted in Illinois and Wisconsin to evaluate the efficacy of two formulations of a new product Spinosad, two commercially available positive controls, and an untreated negative control. A baseline lice count was recorded prior to the treatment. After treatment, weekly measurements of lice counts were taken for 8 weeks. Counts of 4 lice species were recorded separately. A linear mixed model analysis was conducted for each species of lice after transforming the counts with a natural logarithm transformation. Simple contrasts between treatment groups at each week were performed. Treatment differences were also compared using 5 multiple testing methods: Bonferroni, Sidak, Holm’s step-down Bonferroni, Hochberg's step-up Bonferroni, and false discovery rate. Seventy-one out of 96 simple tests showed significant differences among the treatment groups. The five multiple testing methods confirmed only 48-67 significances out of the 96 tests. Comparatively, Bonferroni and Sidak methods provided similar and the most conservative multiplicity test results, i.e. fewest significant differences. The Holm’s step-down and Hochberg's step-up Bonferroni methods provided similar but less conservative results. Finally, the false discovery rate method provided the least conservative results.
