Article
Version 1
This version is not peer-reviewed
Meta-Analysis of Genuine and Fake p-Values
Version 1
: Received: 23 July 2024 / Approved: 24 July 2024 / Online: 24 July 2024 (12:53:18 CEST)
How to cite: Brilhante, M. F.; Gomes, M. I.; Mendonça, S.; Pestana, D.; Santos, R. Meta-Analysis of Genuine and Fake p-Values. Preprints 2024, 2024071927. https://doi.org/10.20944/preprints202407.1927.v1 Brilhante, M. F.; Gomes, M. I.; Mendonça, S.; Pestana, D.; Santos, R. Meta-Analysis of Genuine and Fake p-Values. Preprints 2024, 2024071927. https://doi.org/10.20944/preprints202407.1927.v1
Abstract
For Sir Ronald Fisher, it is important to consistently obtain significant p-values to support an experimental hypothesis. So, replicating experiments to obtain independent p-values is a legitimate and desirable research practice. Several simple statistics have been proposed to meta-analyze p-values, all assuming that they are genuine, i.e. observations from independent standard Uniform random variables. But, as publication bias favors the studies that report "significant" p-values, when a p>0.05 is obtained for the outcome of an experiment, some researchers will "fall into temptation" and decide to replicate the experiment in the hope of getting a smaller second p-value, ideally a significant one. Consequently, if the smallest of two p-values is reported, this is a Beta(1,2) distributed "fake" p-value, not a uniformly distributed genuine p-value. This is an unacceptable scientific research practice, and moreover the detection of fake p-values is unpractical. Even when it is possible, the analytic results to accommodate their existence in combined tests are cumbersome. For an informed decision, inclusive when the presence of fake p-values in a sample of p-values to be meta-analyzed is probable, tables with simulated critical values for the usual combined testing are supplied. This will also allow comparisons to be made between several combined tests.
Keywords
combined p-values; fake p-values; Mendel random variables; meta-analysis
Subject
Computer Science and Mathematics, Probability and Statistics
Copyright: This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Comments (0)
We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.
Leave a public commentSend a private comment to the author(s)
* All users must log in before leaving a comment
