| Dr. Ellen Tarr recently posted some thoughts on UFO-related survey results as conducted and presented by FREE (Foundation for Research into Extraterrestrial Encounters). Tarr holds a Ph.D. in Molecular Biology and Immunology, is an Associate Professor at Midwestern University, and graciously offers analysis from time to time on such topics as Project Core and alleged Sasquatch DNA.
She interpreted survey results as reported by FREE to be unclear on details like numbers of respondents and exactly how FREE arrived at some of its figures. Tarr’s pointed observations included “the myriad problems with the survey itself and the analysis,” as well as “the lack
of controlling which respondents answer follow-up questions.” As she explained:
There are numerous cases within the survey where more people responded to follow-up questions about a specific type of experience than had claimed to have had the experience. For example, 211 respondents reported having sex with an ET and 236 gave answers regarding what type of ET they had sex with. The likelihood that many items include responses from people who did not have the experience calls many results into question.
Tarr also noted survey results were represented by FREE as specifically including people who reported UFO-related contact experiences with a non-human intelligence, yet it is unclear if all who responded actually interpreted that to be the case. For instance, fewer people reported a craft or ship associated with their experiences than participated in the survey.
|Budd Hopkins and David Jacobs at a
2004 Intruders Foundation Seminar
Such challenges with surveys and their interpretations have long plagued the UFO community. The design of a 1991 Roper Poll funded by Robert Bigelow and conducted by Budd Hopkins, David Jacobs and Ron Westrum was competently called into question by qualified professionals. The trio arrived at the stunning conclusion 3.7 million Americans had been abducted by aliens through a survey of less than 6,000 people who were never even asked. Instead, those surveyed were subjected to a series of questions of which Hopkins and Jacobs felt themselves qualified to interpret if the responses indicated abductions had occurred. To directly ask respondents if they’d ever been abducted, it was rather incredibly rationalized, would give false results because many people were unaware of their abductions until after hypnosis.
Of a total of 5,947 people interviewed, 119, or two percent, were identified as likely alien abductees. It was from there the conclusion was drawn that about two percent of the American population, which at the time equated to 3.7 million people, had been abducted by aliens.
Critical review was provided by parapsychologist Susan Blackmore and sociologist Ted Goertzel, among others. The work of the late psychologist Robyn M. Dawes and political scientist Matthew Mulford, the latter of which became an expert in research methodology at the London School of Economics and Political Science, showed how questions on the survey were poorly constructed in ways known to produce flawed results. Goertzel wrote:
This conclusion is also strongly supported by Dawes and Mulford’s (1993) innovative study at the University of Oregon which demonstrated that the dual nature of Hopkins, Jacobs and Westrum’s first item, which asked about waking up paralyzed and about sensing a strange person in the room in the same item, actually led to an increased recollection of unusual phenomena as compared to a properly constructed single-issue survey item. Textbooks on questionnaire writing universally warn against “double-barreled” questions of this sort because they are known to give bad results. Dawes and Mulford confirm this and further offer the explanation that the combination of the two issues in one item causes a conjunction effect in memory which increases the likelihood of false recollection.
While the Hopkins, Jacobs and Westrum scale is not a valid measure of UFO abduction, they have inadvertently constructed a useful measure of another phenomenon: the tendency to have false memories.
The poll and its questionably interpreted conclusions continue to be cited in UFO circles in spite of its flawed construction. The problematic aspects of its methodologies are typically not addressed when claims are made of some 4 million Americans being abducted by aliens. The objectivity of Budd Hopkins was further questioned due to such circumstances as his claims surrounding alleged alien symbols purported to have been seen by abductees while aboard alien craft. His questionable interpretations and desire to “stack the deck,” as he put it, were documented in the 13-minute video clip below shot by Carol Rainey.
An important point, in my opinion, is that Dr. Tarr and other qualified experts demonstrate a willingness to address the UFO phenomenon and offer review of research produced by ufology. The scientific community is often criticized for dismissing the topic out of hand, and the complaint may be justified at times, but there are clearly exceptions.
Furthermore, it should be noted that such critical review is part and parcel of the path to establishing fact-based evidence. The critiques of qualified professionals should be embraced and addressed, not discarded with aversion. It is when standards of evidence are recognized, and professional research protocols are collectively respected and implemented, that the UFO community will mature and begin to gain the credibility it has long claimed to seek.
Please join me this summer in Roswell at a conference themed 70 Years Later: Modern Challenges to the Extraterrestrial Hypothesis. I’ll be discussing exploitation in ufology, the intersection of the UFO and intelligence communities, and related topics.