"˜Regression toward the mean" in replications of fundamental research studies on homeopathic high potencies?

Abstract

Around 100 high-potency studies, each one replicating earlier studies, were located in a bibliometric study on fundamental research [1]. 70% of the replication studies yielded the same result as the original study, 10% the opposite result, and 20% found no difference between the test and control groups. This is to say, the final standing was 70 positive, 10 negative and 20 undecided results. Among the replication studies performed by the original laboratory, the outcomes were 83 positive and 5 negative (12 undecided); in the ones performed in laboratories of independent researchers invited to participate in a multicentric study the corresponding outcomes were 75 and 11 (14), and the ones of studies performed by external researchers 48 and 14 (38). Had the experiments been performed on purely random phenomena the most probable outcomes would have been 33 : 33 : 33. In a follow-up study we sought to establish whether the outcomes of the replication studies might be attributed to regression toward the means. As is known, this term describes the situation in which a random experiment yields an extreme result, while subsequent replications give average results quite independent from the initial one. By contrast, when the experiment is not entirely based entirely on chance, subsequent replications tend to yield results that deviate from the average in the same direction as the initial one. We thus grouped the studies into 3 clusters of at least 7 articles according to their date of publication. The relative frequencies of positive, undecided and negative results were graphically represented through by curves to allow for visual comparison of the initial result versus the results of the replication studies for each of the 3 clusters. Regression towards the mean was neither observed in the results of the replication studies taken together nor in the group of laboratory-internal replication studies. By contrast, the group of multicentric replication studies exhibited increase of the relative frequency of undecided results and decrease of the one of positive results. In the group of external replication studies the relative frequency of undecided results increased and the one of positive and of negative results decreased. The statistical probability of drawing a ‘positive result’ from a total of 3 equally probable options (positive, undecided, negative) is 33.3%. By contrast, within the entirety of replication studies the frequency of positive results was 77% on the first replication, 66.7% on the second and 66.7% on the third. Unlike the results from the group of laboratory-internal studies, the ones of multicentric and external replications are suggestive of regression towards the mean. However the magnitude of this effect is far weaker than would be expected from a purely random distribution of results. One question that remains is why there appears to have been practically no regression in the laboratory-internal replications. This should not be cheaply attributed to publication bias. Especially in experiments on homoeopathic high potencies, it could well be that numerous, in some part hidden factors of influence are involved, which internal laboratories – possibly unwittingly through unreflected working habits– are better at controlling than external laboratories, regardless of the effort invested to this end. Keywords: high dilution, replication experiments, regression to the mean References [1] Endler PC, Bellavite P, Bonamin L, Jäger T. Mazon S. Replications of fundamental research models in ultra high dilutions, 1994 and 2015: an update on a bibliometric study. Homeopathy 2015;104:235-245