Paraspsychology: When Regular Psychology Isn't Soft Enough.

SLAM: debunk creationism, pseudoscience, and superstitions. Discuss logic and morality.

Moderator: Alyrium Denryle

Post Reply
User avatar
adam_grif
Sith Devotee
Posts: 2755
Joined: 2009-12-19 08:27am
Location: Tasmania, Australia

Paraspsychology: When Regular Psychology Isn't Soft Enough.

Post by adam_grif »

Is this evidence that we can see the future?

Extraordinary claims don't come much more extraordinary than this: events that haven't yet happened can influence our behaviour.

Parapsychologists have made outlandish claims about precognition – knowledge of unpredictable future events – for years. But the fringe phenomenon is about to get a mainstream airing: a paper providing evidence for its existence has been accepted for publication by the leading social psychology journal.

What's more, sceptical psychologists who have pored over a preprint of the paper say they can't find any significant flaws. "My personal view is that this is ridiculous and can't be true," says Joachim Krueger of Brown University in Providence, Rhode Island, who has blogged about the work on the Psychology Today website. "Going after the methodology and the experimental design is the first line of attack. But frankly, I didn't see anything. Everything seemed to be in good order."
Critical mass

The paper, due to appear in the Journal of Personality and Social Psychology before the end of the year, is the culmination of eight years' work by Daryl Bem of Cornell University in Ithaca, New York. "I purposely waited until I thought there was a critical mass that wasn't a statistical fluke," he says.

It describes a series of experiments involving more than 1000 student volunteers. In most of the tests, Bem took well-studied psychological phenomena and simply reversed the sequence, so that the event generally interpreted as the cause happened after the tested behaviour rather than before it.

In one experiment, students were shown a list of words and then asked to recall words from it, after which they were told to type words that were randomly selected from the same list. Spookily, the students were better at recalling words that they would later type.

In another study, Bem adapted research on "priming" – the effect of a subliminally presented word on a person's response to an image. For instance, if someone is momentarily flashed the word "ugly", it will take them longer to decide that a picture of a kitten is pleasant than if "beautiful" had been flashed. Running the experiment back-to-front, Bem found that the priming effect seemed to work backwards in time as well as forwards.
'Stroke of genius'

Exploring time-reversed versions of established psychological phenomena was "a stroke of genius", says the sceptical Krueger. Previous research in parapsychology has used idiosyncratic set-ups such as Ganzfeld experiments, in which volunteers listen to white noise and are presented with a uniform visual field to create a state allegedly conducive to effects including clairvoyance and telepathy. By contrast, Bem set out to provide tests that mainstream psychologists could readily evaluate.

The effects he recorded were small but statistically significant. In another test, for instance, volunteers were told that an erotic image was going to appear on a computer screen in one of two positions, and asked to guess in advance which position that would be. The image's eventual position was selected at random, but volunteers guessed correctly 53.1 per cent of the time.

That may sound unimpressive – truly random guesses would have been right 50 per cent of the time, after all. But well-established phenomena such as the ability of low-dose aspirin to prevent heart attacks are based on similarly small effects, notes Melissa Burkley of Oklahoma State University in Stillwater, who has also blogged about Bem's work at Psychology Today.
Respect for a maverick

So far, the paper has held up to scrutiny. "This paper went through a series of reviews from some of our most trusted reviewers," says Charles Judd of the University of Colorado at Boulder, who heads the section of the Journal of Personality and Social Psychology editorial board that handled the paper.

Indeed, although Bem is a self-described "maverick" with a long-standing interest in paranormal phenomena, he is also a respected psychologist with a reputation for running careful experiments. He is best known for the theory of self-perception, which argues that people infer their attitudes from their own behaviour in much the same way as they assess the attitudes of others.

Bem says his paper was reviewed by four experts who proposed amendments, but still recommended publication. Still, the journal will publish a sceptical editorial commentary alongside the paper, says Judd. "We hope it spurs people to try to replicate these effects."

One failed attempt at replication has already been posted online. In this study, Jeff Galak of Carnegie Mellon University in Pittsburgh, Pennsylvania, and Leif Nelson of the University of California, Berkeley, employed an online panel called Consumer Behavior Lab in an effort to repeat Bem's findings on the recall of words.

Bem argues that online surveys are inconclusive, because it's impossible to know whether volunteers have paid sufficient attention to the task. Galak concedes that this is a limitation of the initial study, but says he is now planning a follow-up involving student volunteers that will more closely repeat the design of Bem's word-recall experiment.

This seems certain to be just the first exchange in a lively debate: Bem says that dozens of researchers have already contacted him requesting details of the work.

tl;dr, after decades of trying they finally get a good result. A tiny-but-statistically-significant effect was found (p < 0.001 across the experiments), one attempt to replicate findings has already failed, and the paper hasn't even been published yet..

I think the funniest part of the study (which can be found here) is where he states that people may be psychically influencing random number generators in the computers running the tests as opposed to seeing the future.

I can't wait for the scramble of psychologists to replicate the tests over the next few years and turn up with nothing notable. None of which will get reported in mainstream outlets :)
A scientist once gave a public lecture on astronomy. He described how the Earth orbits around the sun and how the sun, in turn, orbits around the centre of a vast collection of stars called our galaxy.

At the end of the lecture, a little old lady at the back of the room got up and said: 'What you have told us is rubbish. The world is really a flat plate supported on the back of a giant tortoise.

The scientist gave a superior smile before replying, 'What is the tortoise standing on?'

'You're very clever, young man, very clever,' said the old lady. 'But it's turtles all the way down.'
User avatar
Formless
Sith Marauder
Posts: 4143
Joined: 2008-11-10 08:59pm
Location: the beginning and end of the Present

<because scoring easy points is pretentious asshattery>

Post by Formless »

Congratulations. You've discovered why New Scientist and other pop-science magazines are considered a joke. Have a cookie.
"Still, I would love to see human beings, and their constituent organ systems, trivialized and commercialized to the same extent as damn iPods and other crappy consumer products. It would be absolutely horrific, yet so wonderful." — Shroom Man 777
"To Err is Human; to Arrr is Pirate." — Skallagrim
“I would suggest "Schmuckulating", which is what Futurists do and, by extension, what they are." — Commenter "Rayneau"
The Magic Eight Ball Conspiracy.
User avatar
adam_grif
Sith Devotee
Posts: 2755
Joined: 2009-12-19 08:27am
Location: Tasmania, Australia

Re: <because scoring easy points is pretentious asshattery>

Post by adam_grif »

Formless wrote:Congratulations. You've discovered why New Scientist and other pop-science magazines are considered a joke. Have a cookie.
Did you miss the part where this has passed peer review and is going to be published in the next issue of an APA Journal? The fact that it's New Scientist reporting this is quite irrelevant. The actual paper is right there for you to read.
A scientist once gave a public lecture on astronomy. He described how the Earth orbits around the sun and how the sun, in turn, orbits around the centre of a vast collection of stars called our galaxy.

At the end of the lecture, a little old lady at the back of the room got up and said: 'What you have told us is rubbish. The world is really a flat plate supported on the back of a giant tortoise.

The scientist gave a superior smile before replying, 'What is the tortoise standing on?'

'You're very clever, young man, very clever,' said the old lady. 'But it's turtles all the way down.'
User avatar
Formless
Sith Marauder
Posts: 4143
Joined: 2008-11-10 08:59pm
Location: the beginning and end of the Present

Re: <because scoring easy points is pretentious asshattery>

Post by Formless »

Someone needs to get their eyes checked, because I wasn't talking about whether or not it was peer reviewed. I'm talking about the idiocy of reporting on a paper that hasn't been published yet. This is just them looking for something sensational to get people's attention, like they always do. Its not news, and its sadly not unusual.
"Still, I would love to see human beings, and their constituent organ systems, trivialized and commercialized to the same extent as damn iPods and other crappy consumer products. It would be absolutely horrific, yet so wonderful." — Shroom Man 777
"To Err is Human; to Arrr is Pirate." — Skallagrim
“I would suggest "Schmuckulating", which is what Futurists do and, by extension, what they are." — Commenter "Rayneau"
The Magic Eight Ball Conspiracy.
User avatar
Wyrm
Jedi Council Member
Posts: 2206
Joined: 2005-09-02 01:10pm
Location: In the sand, pooping hallucinogenic goodness.

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Wyrm »

What's more, sceptical psychologists who have pored over a preprint of the paper say they can't find any significant flaws. "My personal view is that this is ridiculous and can't be true," says Joachim Krueger of Brown University in Providence, Rhode Island, who has blogged about the work on the Psychology Today website. "Going after the methodology and the experimental design is the first line of attack. But frankly, I didn't see anything. Everything seemed to be in good order."
Seems to be in good working order, squiffy? Then what's this in the very next section?
Critical mass

The paper, due to appear in the Journal of Personality and Social Psychology before the end of the year, is the culmination of eight years' work by Daryl Bem of Cornell University in Ithaca, New York. "I purposely waited until I thought there was a critical mass that wasn't a statistical fluke," he says.
Sounds like not specifying your stopping rule ahead of time to me, a common blunder in these circles. If you wait until you get a "critical mass" before you publish, the probability of getting one approaches 1 through normal noise.
Darth Wong on Strollers vs. Assholes: "There were days when I wished that my stroller had weapons on it."
wilfulton on Bible genetics: "If two screaming lunatics copulate in front of another screaming lunatic, the result will be yet another screaming lunatic. 8)"
SirNitram: "The nation of France is a theory, not a fact. It should therefore be approached with an open mind, and critically debated and considered."

Cornivore! | BAN-WATCH CANE: XVII | WWJDFAKB? - What Would Jesus Do... For a Klondike Bar? | Evil Bayesian Conspiracy
User avatar
Maj
Youngling
Posts: 75
Joined: 2010-08-26 12:08am
Location: Olympia, WA, USA

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Maj »

Wyrm wrote:Sounds like not specifying your stopping rule ahead of time to me, a common blunder in these circles. If you wait until you get a "critical mass" before you publish, the probability of getting one approaches 1 through normal noise.
It sounded to me like the guy was just saying he wanted an adequate sample size. If he based his conclusions on just a few people, not having enough people would be the criticism thrown at him. So now he has a lot of people involved, and the criticism is that he was just waiting until the numbers randomly got significant.

Translation: The guy is screwed. Regardless of the actual scientific procedure that went into eight years of research and experimentation, Bem is just a crackpot who couldn't possibly turn up anything relevant.

Gotcha.
Psychic_Sandwich
Padawan Learner
Posts: 416
Joined: 2007-03-12 12:19pm

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Psychic_Sandwich »

Translation: The guy is screwed. Regardless of the actual scientific procedure that went into eight years of research and experimentation, Bem is just a crackpot who couldn't possibly turn up anything relevant.

Gotcha.
If his results can't be replicated, then that's exactly what he is. Or, he's just unlucky and randomly got a statistically significant fluke despite actually trying to do everything properly. That, after all, is why research is published like this; so other people can try to replicate it and tell everybody if they fail.
User avatar
Archaic`
Jedi Council Member
Posts: 1647
Joined: 2002-10-01 01:19am
Location: Brisbane, Australia
Contact:

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Archaic` »

From what I've read, he did 8 identical studies (except for small variations in n), 1 each year for the past 8 years. 7 were statistically significant, one was not. The "replication" study was done online, rather than face-to-face. What impact that change of context might have had on the results is unknown, but Bem's criticism of it is probably valid, so I'm looking forward to seeing what Galak does with his second go at replication.
Veni Vidi Castravi Illegitimos
User avatar
Maj
Youngling
Posts: 75
Joined: 2010-08-26 12:08am
Location: Olympia, WA, USA

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Maj »

Thanks, Archaic`.
Psychic Sandwich wrote:If his results can't be replicated, then that's exactly what he is.
His experiment wasn't replicated. From the article above, emphasis mine:
One failed attempt at replication has already been posted online. In this study, Jeff Galak of Carnegie Mellon University in Pittsburgh, Pennsylvania, and Leif Nelson of the University of California, Berkeley, employed an online panel called Consumer Behavior Lab in an effort to repeat Bem's findings on the recall of words.

Bem argues that online surveys are inconclusive, because it's impossible to know whether volunteers have paid sufficient attention to the task. Galak concedes that this is a limitation of the initial study, but says he is now planning a follow-up involving student volunteers that will more closely repeat the design of Bem's word-recall experiment.
If the repeat experiment isn't the same, how can its failure allow anyone to draw adequate conclusions on the original study?

I don't have a problem with skeptics who want to repeat the experiments to verify the results, but they need to follow the recipe at least once before they try making up a dish on their own.
User avatar
Wyrm
Jedi Council Member
Posts: 2206
Joined: 2005-09-02 01:10pm
Location: In the sand, pooping hallucinogenic goodness.

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Wyrm »

Maj wrote:
One failed attempt at replication has already been posted online. In this study, Jeff Galak of Carnegie Mellon University in Pittsburgh, Pennsylvania, and Leif Nelson of the University of California, Berkeley, employed an online panel called Consumer Behavior Lab in an effort to repeat Bem's findings on the recall of words.

Bem argues that online surveys are inconclusive, because it's impossible to know whether volunteers have paid sufficient attention to the task. Galak concedes that this is a limitation of the initial study, but says he is now planning a follow-up involving student volunteers that will more closely repeat the design of Bem's word-recall experiment.
If the repeat experiment isn't the same, how can its failure allow anyone to draw adequate conclusions on the original study?

I don't have a problem with skeptics who want to repeat the experiments to verify the results, but they need to follow the recipe at least once before they try making up a dish on their own.
There's something more serious going on here, I think. The effect may have vanished because the controls have tightened. Refining the experiment, after all, is one of the way you thoroughly investigate the phenomenon in question. You can control for variables the original authors hadn't thought of, for example. It also allows you to catch the difference between a genuine new phenomenon and an artifact of the particular experimental setup.

You see this all the time in paranormal studies: when the controls are tightened in replication, the effect size goes to zero. Perhaps this is what we're seeing here.
Darth Wong on Strollers vs. Assholes: "There were days when I wished that my stroller had weapons on it."
wilfulton on Bible genetics: "If two screaming lunatics copulate in front of another screaming lunatic, the result will be yet another screaming lunatic. 8)"
SirNitram: "The nation of France is a theory, not a fact. It should therefore be approached with an open mind, and critically debated and considered."

Cornivore! | BAN-WATCH CANE: XVII | WWJDFAKB? - What Would Jesus Do... For a Klondike Bar? | Evil Bayesian Conspiracy
User avatar
Maj
Youngling
Posts: 75
Joined: 2010-08-26 12:08am
Location: Olympia, WA, USA

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Maj »

No offense, but conducting a study over the internet versus in person just innately smacks to me of less controls, not more.
User avatar
Wyrm
Jedi Council Member
Posts: 2206
Joined: 2005-09-02 01:10pm
Location: In the sand, pooping hallucinogenic goodness.

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Wyrm »

Not necessarily. While it is in principle possible for it to be easier for two people over the internet to collaborate, in practice it can be seriously confounded by making it extremely unlikely that you have or can ever even contacted your partner by any means of communication outside the study (and even inside the study), such as randomizing and concealing the identity of your pair. On the internet, your partner might be on the other side of the globe, after all. In order for collaboration to work, the two would basically need to be psychic anyway!
Darth Wong on Strollers vs. Assholes: "There were days when I wished that my stroller had weapons on it."
wilfulton on Bible genetics: "If two screaming lunatics copulate in front of another screaming lunatic, the result will be yet another screaming lunatic. 8)"
SirNitram: "The nation of France is a theory, not a fact. It should therefore be approached with an open mind, and critically debated and considered."

Cornivore! | BAN-WATCH CANE: XVII | WWJDFAKB? - What Would Jesus Do... For a Klondike Bar? | Evil Bayesian Conspiracy
User avatar
Maj
Youngling
Posts: 75
Joined: 2010-08-26 12:08am
Location: Olympia, WA, USA

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Maj »

I guess I don't really understand what you're saying... The article indicates that the test subjects were participating in the experiment online. But if the researchers can't see their subjects, how is that better?
User avatar
Wyrm
Jedi Council Member
Posts: 2206
Joined: 2005-09-02 01:10pm
Location: In the sand, pooping hallucinogenic goodness.

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Wyrm »

Why is seeing necessary here? Why can't this experiment be done just as well with a faceless cgi script?
Darth Wong on Strollers vs. Assholes: "There were days when I wished that my stroller had weapons on it."
wilfulton on Bible genetics: "If two screaming lunatics copulate in front of another screaming lunatic, the result will be yet another screaming lunatic. 8)"
SirNitram: "The nation of France is a theory, not a fact. It should therefore be approached with an open mind, and critically debated and considered."

Cornivore! | BAN-WATCH CANE: XVII | WWJDFAKB? - What Would Jesus Do... For a Klondike Bar? | Evil Bayesian Conspiracy
User avatar
Zaune
Emperor's Hand
Posts: 7517
Joined: 2010-06-21 11:05am
Location: In Transit
Contact:

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Zaune »

I don't think better or worse is really the point here. It's a condition of the experiment that differs, quite significantly, from the original. I might be an English major, but I'm fairly sure that when replicating an experiment, the only thing you're supposed to change is the size of the sample set.
There are hardly any excesses of the most crazed psychopath that cannot easily be duplicated by a normal kindly family man who just comes in to work every day and has a job to do.
-- (Terry Pratchett, Small Gods)


Replace "ginger" with "n*gger," and suddenly it become a lot less funny, doesn't it?
-- fgalkin


Like my writing? Tip me on Patreon

I Have A Blog
User avatar
Wyrm
Jedi Council Member
Posts: 2206
Joined: 2005-09-02 01:10pm
Location: In the sand, pooping hallucinogenic goodness.

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Wyrm »

It is impossible to replicate an experiment and not change any more than the sample size. For one thing, you're not going to be able to use the same subjects for the replication as the original study. The room may not be the same color, or contain "goats" (skeptics seem to have an anti-psychic effect — I wonder why). The exact details of experiment will be different in a replication.

However, the original author should have already teased out the irrelevancies to produce a specification that, if followed, will reliably generate the same results. Any experiment staying within that specification should be an equivalent experiment. Now, if Bem didn't specify in his original paper that the study couldn't be done online, or any other specification to the same effect, then that's an omission on his part, and not the fault of Galak. It's also something that he is expected to control for to see if, yes, it makes a difference whether or not the study is conducted online. After all, we're conducting an experiment in precognition.

Galak's replication is indeed very probably a replication as specified in the paper, and as such, Bem's objection is an ad hoc rationalization for its failure. That's a foul on his part, and completely unsurprising.
Darth Wong on Strollers vs. Assholes: "There were days when I wished that my stroller had weapons on it."
wilfulton on Bible genetics: "If two screaming lunatics copulate in front of another screaming lunatic, the result will be yet another screaming lunatic. 8)"
SirNitram: "The nation of France is a theory, not a fact. It should therefore be approached with an open mind, and critically debated and considered."

Cornivore! | BAN-WATCH CANE: XVII | WWJDFAKB? - What Would Jesus Do... For a Klondike Bar? | Evil Bayesian Conspiracy
User avatar
Zaune
Emperor's Hand
Posts: 7517
Joined: 2010-06-21 11:05am
Location: In Transit
Contact:

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Zaune »

Point taken, though the presence or absence of a flesh-and-blood observer seems a bit more fundamental than the colour of the room. I'll reserve judgement until I've seen the paper itself.

And come to think of it, psychology is one field of scientific study where there is a case for an observer or participant with a preconceived expectation of the outcome skewing the results in some circumstances.
There are hardly any excesses of the most crazed psychopath that cannot easily be duplicated by a normal kindly family man who just comes in to work every day and has a job to do.
-- (Terry Pratchett, Small Gods)


Replace "ginger" with "n*gger," and suddenly it become a lot less funny, doesn't it?
-- fgalkin


Like my writing? Tip me on Patreon

I Have A Blog
User avatar
aimlessgun
Redshirt
Posts: 9
Joined: 2010-11-29 10:06am

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by aimlessgun »

His reason for objection was that they couldn't know if 'sufficient attention' was paid by the test subjects. Is that an actual issue with online tests or does it just sound like BS?
User avatar
Archaic`
Jedi Council Member
Posts: 1647
Joined: 2002-10-01 01:19am
Location: Brisbane, Australia
Contact:

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Archaic` »

In the context of the study, it does sound kind of reasonable. If it's an online study, you can't exactly peer over their shoulder to make sure they're not just semi-randomly filling it in for their $5 or whatever it was they got for participation.
Veni Vidi Castravi Illegitimos
User avatar
Wyrm
Jedi Council Member
Posts: 2206
Joined: 2005-09-02 01:10pm
Location: In the sand, pooping hallucinogenic goodness.

Re: Paraspsychology: When Regular Psychology Isn't Soft Enou

Post by Wyrm »

Zaune wrote:And come to think of it, psychology is one field of scientific study where there is a case for an observer or participant with a preconceived expectation of the outcome skewing the results in some circumstances.
There are several good reasons to believe that the "case" for psi-missing is complete bullshit, and the sheep-goat effect is nothing more than mundane effects:
  • If the experimenter is a believer, then he likely has less than adequate controls on the experiment, as such leaves it vulnerable to mundane effects that mess up the results. Skeptical researchers usually have tighter controls, not because they want to disconfirm the effect, but because they know that they can be fooled by mischief, and unsurprisingly do not find the phenomenon.
  • If the subject is a believer, then that comes with a set of mundane skills that enhances performance, such as cues, sensory leakage, counting cards, and sometimes outright cheating. (But only because he was "weak" that day!) After all, they believe they have some special power, so they practice — and the skill they acquire will be at least in part in mundane skills that extract real information. And of course, a believing researcher will often leave those holes open to exploit. Skeptics don't develop those skills because they don't practice, and so they miss the holes and get less impressive scores. Also, they ignore those skills they have developed because the recognize those sources as not legitimate to the experiment.
  • In many experiments, psi-missing is not a part of the experimental design in which it shows up. Instead, it shows up after looking at the data and seeing a depressed effect. This renders the explanation an ad hoc rationalization, by stating that the experiment is a success because it did "worse than chance." No. A failure is a failure, and there's no differentiation between grades of fail. The "worse than chance" result is a negative result, not a positive result for a negative effect.
  • Even for those experiments where "psi-missing" is part of the design, it is much harder to establish the negative effect. Because of the nature of 'unconsciously missing' the mark on psi tests (guessing wrong on Zener card tests, for example), it should take much more evidence to establish the "psi-missing" phenomenon than "psi-present". But no, the "psi-missing" phenomenon are often about the same size as the "psi-present."
Despite all the above, as a rule psi researchers do not control for sheep-goat or psi-missing, instead claiming their manifestation to be psi. This is shenannigans. The sheep-goat effect and psi-missing can be sufficiently explained by statistical fluctuations, lack of adequate controls, or other mundane effects.

=====
Archaic` wrote:In the context of the study, it does sound kind of reasonable. If it's an online study, you can't exactly peer over their shoulder to make sure they're not just semi-randomly filling it in for their $5 or whatever it was they got for participation.
So a researcher can peer over the shoulder and tell that someone is just semi-randomy filling in, rather than genuinely receiving psychic precognative flashes. Right.
Darth Wong on Strollers vs. Assholes: "There were days when I wished that my stroller had weapons on it."
wilfulton on Bible genetics: "If two screaming lunatics copulate in front of another screaming lunatic, the result will be yet another screaming lunatic. 8)"
SirNitram: "The nation of France is a theory, not a fact. It should therefore be approached with an open mind, and critically debated and considered."

Cornivore! | BAN-WATCH CANE: XVII | WWJDFAKB? - What Would Jesus Do... For a Klondike Bar? | Evil Bayesian Conspiracy
Post Reply