Unfounded criticism of a parapsychology book in Nature

Highly regarded journal refuses to publish correction of error in review: a case of censorship?

"Any time a reputable news organisation gives its readers or viewers details that later turn out not to be true, they are obligated to tell the truth". -- Don Hewitt of 60 Minutes, CBS.

---

In the highly regarded journal Nature (Oct. 23rd. 1997, p.806), I.J. Good gives a strongly critical review (now available on the Web, courtesy of Nick Herbert) of Dean Radin's The Conscious Universe. The Journal refused to publish the rebuttal of the criticisms printed below. It has also since that time refused to publish a request from the author of the book for a correction to be noted (see also Apr. 9th. 1998 update: a grossly inadequate 'correction' is now published).

Update of Aug. 14, 1998: In its correspondence pages (Nature 394, 413 (30 July 1998)), the journal has now belatedly published Radin's letter (with the omission, no doubt not surprising given the Journal's record, of his closing remark 'I hope this note motivates readers to study the evidence for themselves'). A comeback by Good, still conforming to Rossman's strictures of 'obsolescent critique', is appended to Radin's letter. This is more than six months after the author originally requested of the journal publication of a note concerning the error, and more then eight months subsequent to my own similar request. The charge of censorship on the journal's part is hardly affected by this belated response, perhaps made only in response to widespread complaints.

See also comment by Michael Rossman.

Should the Journal offer a response to the criticisms contained on this web page, such a response will be posted here.

1. Effectively ignoring current practices in the field designed to counter such criticism, Good suggests that positive results in parapsychological research can be explained in terms of fraud and self-deception.
2. A calculation of the 'file-drawer factor' (see below) giving a result greatly different from Radin's, implying both faulty statistical analysis on the part of the latter and the possibility that the existence of a sufficiently large collection of unpublished negative results could explain away the deviations from chance found by the analysis. But investigation of this matter reveals that Good used an incorrect number in his own analysis. His negative conclusion is therefore invalid.
3. Quotation of a misapprehension concerning the method of combination of results from more than one experiment. In his book Radin quoted, without comment, an extract from a book by Hansel containing this misapprehension. Good mentions this in his review in order to back up his claim that Radin's statistical abilities are of a low standard. This criticism is also misconceived. Radin comments in the letter reprinted below: 'I am well aware [of the problem with the criticised statement] but did not think it necessary to correct Hansel's mistake in a book aimed towards a general readership.' The Journal increased the confusion by adding an editorial 'clarification' that was itself mistaken, viz. attributing some of the erroneous extract to Radin rather than Hansel.

The journal's response to requests for a correction to be published has been (by ordinary standards) bizarre. It has published a correction of its own editorial error but has failed to acknowledge the far more serious difficulty with the review that the file-drawer analysis, central to Good's criticisms, is based on an incorrect assumption.

For a journal not to correct a known error of any significance, for whatever reason, discredits more than just that journal. The reputation of science is also at stake, because the integrity and validity of the scientific enterprise demand that individual journals adhere to the accepted practices of science. Indeed, it is hard to draw a clear line between a journal's remaining silent about the fact that a criticism to which it has given the authority that goes with publication in its pages is unfounded, and an individual submitting for publication results that he or she is aware are probably invalid.

I.J. Good's review of Radin's survey of the evidence for paranormal phenomena, The Conscious Universe [1], misleads by its selective approach to parapsychological research, combined with claims of error on the author's part that are invalid. As the book indicates, possibilities for fraud and unintentional error are much reduced by present day techniques so that what may or may not have happened in the case of Soal is essentially irrelevant (unless one believes in extensive collusive cheating among apparently reputable individuals, a hypothesis I find implausible). For example, readings are nowadays normally not written down by the experimenter, but recorded and analysed automatically. Such improvements have not made the effects go away, giving one some reason to consider that they are real.

Regarding the claims of error, Good may have been misled by certain simplifications in the book that Radin has explained as having been necessitated by the requirement that it be attractive to the general reader as well as informative to the interested scientist, on which grounds he omitted comment on the error of Hansel that Good highlights in his review. Again, for the general reader's benefit, he described a P-value which was actually of order 1 in 10^2000 simply as 'odds of more than a billion trillion to one against chance', so that the latter number does not approximate to the actual P-value as Good assumed. clarification

Thus investigation shows Good's claims of mistakes on the author's part to be unfounded. Radin has, as Good admits, provided a well-written account of the arguments supporting the existence of ESP, while the very frequently misconceived nature of the arguments of sceptics may have justified giving less space to them than Good would have liked.

[1] Good, I.J., Nature 389, 806-7, 1997.

Brian D. Josephson, Cavendish Laboratory, Madingley Road, Cambridge CB3 0HE, U.K.

2. There follows the letter sent subsequently by Radin (17 Jan 1998):

Dear Editor,

I have heard that numerous people have gained an unfavorable impression of my book, "The Conscious Universe" (1997, HarperEdge), as a result of a review published in your journal on Oct. 23rd, p.806. Prof. I. J. Good's review involved factual errors, not merely differences in opinion, and I would like the journal to publish a correction in view of this.

---------

In his review (Oct. 23rd, p.806) of my book The Conscious Universe (1997, HarperEdge), I.J. Good suggests, based on certain misunderstandings on his part, that something "must be" wrong with my statistical arguments in favor of the reality of psychic phenomena. His inability to reproduce my estimate that 3,300 unpublished, unsuccessful experiments would be required for each published ESP card experiment (to nullify the cumulative outcome) follows from his incorrect assumption that my words "more than", used in connection with the cumulative odds against chance for the 186 experiments listed in Pratt et al (1966, "ESP after 60 years", as noted in my book), could safely be replaced by "approximately equal to." The actual p-value is approximately 10^-2000, to which application of standard methods (Rosenthal, 1991, "Meta-analytic procedures for social research") gives my reported figure. Secondly, Good attributed to me an inaccurate statement actually made by the skeptical psychologist Mark Hansel. I am well aware that combining the statistical outcomes of experiments is not the same as combining the probabilities of independent events, but did not think it necessary to correct Hansel's mistake in a book aimed towards a general readership.

I am encouraged by the fact that a statistician of Good's repute did not discover any genuine flaws in my comprehensive analysis of the empirical evidence for psi phenomena, and I hope this note motivates readers to study the evidence for themselves.

Dean Radin, Ph.D.

Once again, the Journal did not see fit to inform its readers that there was a problem with its critical review.

This characterisation may surprise those who have read the review concerned, which after a discussion of the way people misjudge coincidences, and a famous case of alleged fraud, goes on to discuss some apparent 'mistakes' on the author's part (I remind the reader that, as noted above, proper investigation shows such claims of mistakes to be unfounded), concluding with the comment:

"So Radin's method for evaluating the file drawer effect, whatever that method may be, must be misguided. This conclusion largely undermines Radin's meta-analysis which is central to his case for ESP"

True, the above quote was immediately followed by the assertion that the book is "well written and provides a good summary of the arguments supporting the existence of ESP". But still, if someone writes of a thesis or paper "it is well-written, though unfortunately the arguments in it are faulty" (and I find it rather difficult to see the above quote as meaning anything other than 'the arguments in the book are faulty'), what does one make of that? Is it being characterised as a good thesis or paper because it is well-written -- or the reverse, because the arguments are faulty? I tend to assume the latter myself but perhaps (taking account of the reponse by Nature in this particular case) I should be less certain than I am that the same would be the general view.

But if I am right in seeing the statement that the review was supportive of the book as mere ad hoc and post hoc justification for a decision that it would be hard to justify on rational and scientific grounds, what does this mean? Is there after all, despite the intense claims that have been made to the effect that science is concerned only with matters of truth, a social and political dimension to science?

See also further comment on scepticism, and letter to Nature by Michael Rossman.

• Nature eventually gives in and publishes a genuine correction (30 July 1998)

Comments for myself to bdj10@cam.ac.uk

Return to Home page