Abstract

When designing a clinical study to compare the efficacy of an experimental treatment with a control, the primary endpoint is selected to measure the treatment effect, along with a clinically meaningful effect defined for that endpoint. At the end of the study, success is typically determined by statistical significance, based on a test statistic and its associated p-value calculated from the observed data. In addition to statistical testing, the observed treatment effect is compared with the pre-specified clinically meaningful threshold to assess clinical significance. The sample size is calculated to ensure high statistical power that is, a probability of detecting statistical significance for the specified clinically meaningful effect. However, the probability of achieving clinical significance is generally lower than the statistical power. As a result, it is not uncommon for a study to demonstrate statistical significance without clinical significance. In this article, we examine the relationship between clinical and statistical significance under various scenarios. This analysis helps explain why statistically significant results frequently lack clinical significance. If we seek a higher probability of clinical significance, our studies must be designed accordingly.

Keywords

clinically meaningful effect size, primary endpoint, strength of evidence, sample size calculation, standardized effect size