The research question of a trial usually involves a single primary response variable, although researchers often analyze others that were not originally planned, especially when the outcomes of the initial response variable are not as expected when the hypothesis was proposed. Other variables are sought or newly constructed from existing variables to “see if something statistically significant emerges”. This increases the risk of finding false positives.

To prevent this, authors should avoid including unplanned outcome variables and, if they do, make it clear that the analysis is post-hoc. Moreover, they must show all comparisons made,2 to avoid the selective publication of statistically significant results. To avoid biased interpretation due to selective publication, readers, reviewers and editors must check that the outcome variables were stated a priori in the trial methodology and registration. Multiple trial registries have been set up,4–6 but sometimes (depending on the country and year of completion) details on the construction of the outcome variables may be insufficient.

The categorization of response variables is also prone to post-hoc opportunistic selection based on the most striking results. Therefore, it would be interesting to establish cut-off points in the protocol or to use internationally accepted standards.7

Unlike group analysis (defined a priori by the research question), in subgroup analysis, subgroups are usually defined afterwards (after data collection is complete). The subgroup analysis evaluates the effects of treatment on subpopulations of patients according to baseline clinical (e.g. diabetes, severity) or personal (sex, age, genetics) characteristics. The aim is to assess whether the treatment effect varies depending on certain patient characteristics.

Controversy exists over whether these evaluations should be performed8: not performing them may deprive us of new scientific knowledge, while performing them increases the possibility of obtaining false positives by the use of multiple comparisons.

Recommendations for considering subgroup analyses are9:

• (1)

  If the subgroup analysis is included in the protocol, and the numbers undergoing this analysis are reduced.

• (2)

  Use interaction calculations to assess heterogeneity between groups. The level of statistical significance of interactions can be used to assess whether chance can explain differences between subgroups.

• (3)

  The results are consistent with other studies and are biologically plausible.

These are data analyses performed before the end of the study in order to generate preliminary results for deciding whether or not to discontinue the study. The main reasons for stopping a study are9: 1) a drug is determined to be harmful or 2) highly beneficial, or 3) the probability of demonstrating efficacy is negligible. This decision should be made by an independent data monitoring committee and not by the investigators themselves. In any case, it should be emphasized that even in this case, and despite the correction of the level of significance by statistical techniques such as O’Brien-Fleming, Peto or Pocock,9 estimates of the treatment effect will always be biased.

Other methodological aspects that can increase the type I error of a trial are the approach to missing data (different statistical methods can be used) and losses to follow-up (intent-to-treat or per-protocol).