Wigwam Jones
Caesar non supra grammati
Well, this thread isn't any fun anymore. You guys have fun storming the castle.
"Expectation bias and plecebo effect are much stronger than people care to admit."
- Thread starter Mid-Fi
- Start date
- Status
robgmn
Super Member
EDITED as the post I was quoting has been removed from thread.
BTW - I am amazed that positron has a proprietary method of listening that took years to develop and he won't reveal publicly.
It must be something above and beyond those natural appendages known as ears.
I wonder if he's into genetic engineering?
I wonder if he plans to sell it? (I hope it's not a "Whisper 2000")
BTW - I am amazed that positron has a proprietary method of listening that took years to develop and he won't reveal publicly.
It must be something above and beyond those natural appendages known as ears.
I wonder if he's into genetic engineering?
I wonder if he plans to sell it? (I hope it's not a "Whisper 2000")
Last edited:
Tom Bombadil
AK Member
When I did my recent A v. B on my tube preamps, using my pink noise track, I set the SPL to 75 at my listening position. That worked for me.
Back years ago, I know it was in that range on some of the comparisons, but not all.
Glutton for punishment .
dogscanskate
Super Member
I'll try to be more precise on this subject and by the way I'm not taking sides either.
What I'm trying to explain as trickery is the fact that you have 2 components A and B. You have a switching device to allow you to compare both components. The main focus of the test is to compare A to B in such a way as the listener does expect the first switch to be from A to B. If you switch from A to A, that is deceitful.
Can I hear a difference from A to A? It is quite possible as I get accustomed to the system, the music program and the acoustics. If just one new nuance is picked up on a repeated verse or chorus, why am I called an audiofool? Because I may have heard something that was not apparent on the first take and on the same device under test? Please let the experts tell me this is impossible.
The test may be unfair because the expected switching amongst listeners is to go from A to B sequentially. At least that is what I expect from such tests. Switching from A to A to A to B to A to A to B to B to B makes the test a different animal unless the subjects are informed of this possibility in which case I doubt you would find enough people to do it.
I'm not arguing for the sake of argument. Why not have a simple and fair test with no surprises where you can only switch from A to B (and B back to A) sequentially?
As for the power of suggestion, there are components I have acquired that were a huge let down in spite of my wanting them to be an upgrade. Where does that fall in the placebo effect?
BTW - I am amazed that positron has a proprietary method of listening that took years to develop and he won't reveal publicly.
It must be something above and beyond those natural appendages known as ears.
I wonder if he's into genetic engineering?
I wonder if he plans to sell it? (I hope it's not a "Whisper 2000")
Expectation bias. If you know that your switch is going to change the DUTs and you expect a difference then you will most likely find one, even if there isn't one. That's why you set up a switching scheme that may or may not change the DUT and let the test subject try to pick out a difference.
dogscanskate
Super Member
That is exactly the premise of the test, A vs B. Random switching brings about infinite permutation possibilities as far as I can tell Ray. You have an infinite choice of settings and still even if you never switch to the other DUT, is it still not possible to hear differences (albeit slight ones) on the same DUT?
It just complicates matters. The way you present it, ''expectation bias'' is negated by random switching.
Just as an example, I once bought an a/c wall outlet (high end at the time), and I wanted it to be an upgrade. After a week of fiddling around with it, I brought it back to the store. The salesman told me I was the first one to ever do that and that I must have done something wrong, it just made the system sound awful, I much preferred the 99 cent a/c outlet. It didn't meet my expectations for sure but it wasn't good at all in spite of being very expensive.
I don't see why you want to suppress expectation between switching DUT's. You hear it or you don't. With random switching, it's like walking in a mine field blindfolded with a cane. I expect nobody that is human could ever pass that type of test. You can call it fair, it's your choice
!BTW - I am amazed that positron has a proprietary method of listening that took years to develop and he won't reveal publicly.
It must be something above and beyond those natural appendages known as ears.
I wonder if he's into genetic engineering?
I wonder if he plans to sell it? (I hope it's not a "Whisper 2000")
:lmao:
JBL GUY
Addicted Member
That is not exactly correct. In medical trials, the subjects are not told in advance whether they are taking the true drug or not.
Just exactly how I described it in my previous post.
This protocol is used precisely because of the "placebo effect".
Depending on the variability of the population of the test subjects, the affect of the placebo effect (expectation bias) may range from a small percent to a statically significant percentage.
This is simply the hard science.
Again this indicates a misunderstanding of the needed protocols to generate statistically accurate results.
The placebo effect is real and documented in many tightly controlled and peer reviewed trials.
OK, it has been shown that even when the test is switching A to A a certain percentage of test subjects will report that they perceived a change.
Tell me how you would deal with this. Tell me how you believe this would affect the statistical accuracy of any tests.
Exactly, the placebo effect (expectation bias) is variable and depends of many things including the individual.
Are you trying to say that your one experience defines totally, accurately and completely everything for everyone including the accepted methodology used other trials?
Are you saying that based on only your experience you are ready to suggest that all of the tried, true and peer reviewed protocols in use are totally and completely incorrect?
These protocols have have been developed to reduce the affect of the placebo effect on the results of the trial.
If one wants to develop and implement alternative protocols that by definition include skewing of the statistical results by including the placebo effect, that is fine as long as the protocols used are clearly defined.
By most standards currently in use (in many fields of study, not only our hobby), this will results in less than the most accurate results.
Again, this is not rocket science. No mystery here.
dogscanskate, just saw your recent post.
That is the problem, a test subject may perceive a change, when there was no change to perceive. If one wants to make choices based on this, that is their prerogative.
The reason that this is done in normal scientific testing, which you seem to be indicating is flawed, is to improve the statistical accuracy of the tests.
It is clear that you believe that scientific testing is flawed and that all of the peer reviewed results of tests in any field that take into account expectation bias, are flawed and inaccurate.
That puts you in a very, very small minority.
You basic logic is flawed and you are setting up straw man arguments to support you position.
The basis of blind testing is to account for expectation bias and remove or reduce its affects on the results.
Expectation bias has been scientifically proven to exist in many tests in many fields of research over many years of study. If such is not the case, just show me the peer reviewed research, not just anecdotal one off, non-scientific examples.
Scientific research that takes into account expectation bias is different than selecting a piece of gear based on an individual perception.
It is possible in a blind test for a subject to choose item A over item B when both examples in the test were item A. That preference is valid for that subject and that subject may be happy as clam with the choice.
You may say that this is cheating or being untruthful, but that is the point of taking expectation bias into account.
It is to demonstrate and take into account that our perceptions are fallible. Unfortunately it appears that Ego may enter into this. That is, my perception of events is true and accurate and I will not accept anything that might indicate otherwise.
To be clear, this statement is not directed as anyone in particular, just a general observation.
I for one, if possible would like to know that any choice that I make is based on a real change not expectation bias, however I understand that this may be rather difficult when related to subjective results.
But this in no way indicates that expectation bias does not exist.
dogscanskate
Super Member
http://en.wikipedia.org/wiki/Placebo
It seems I may not be alone my friend! Scroll down to placebo effects please.
I truly value your points JBL but there are scientists that dispute the value of placebo effect too.
Mind you, I'm no expert in the field at all.
It seems I may not be alone my friend
! Scroll down to placebo effects please. I truly value your points JBL but there are scientists that dispute the value of placebo effect too.
Mind you, I'm no expert in the field at all.
gvalent
Well-Known Member
http://en.wikipedia.org/wiki/Placebo
It seems I may not be alone my friend
I truly value your points JBL but there are scientists that dispute the value of placebo effect too.
Mind you, I'm no expert in the field at all.
Which section are you referring to?
I think you misunderstood my post. The medical trials and placebo explanation were two different things.
JBL GUY
Addicted Member
I reread your post and with your following post, understand your point.
In medical trials, it is thinking that you might have been given the true medicine.
Your point validates expectation bias.
From the conclusions of the research by Hróbjartsson & Gøtzsche, published as a 2010 Cochrane systematic review(which is what I believe dogscanskate is referring to), meta-regression analyses showed that, among other conclusions, larger effects of placebo interventions were associated with patient-involved outcomes (patient-reported outcomes and observer-reported outcomes involving patient cooperation).
And they acknowledge that large effects of placebo interventions may occur in certain situations.
Not that it never occurred.
In other words, subjective evaluation of the outcome by the test subject.
Similar to what might be expected in our DBT/ABX testing.
To be clear, in the context of our hobby, I am not stating that expectation bias is strong enough to over ride the results in all instances. It is not. But in situations were differences may be very small or nonexistent, it may play a part.
And some research using functional imaging has shown measurable physiological changes.
And in research by Daniel Houser, Kevin McCabe, Rodney K. M. Hopson, in 2008 using caffeine pills and and a placebo, monitoring blood pressure several times during a 90 minute interval, they came to the following conclusion "Our evidence supports the existence of physiological placebo effects and provides compelling evidence that these effects are mediated by expectations".
I am not going to list their entire protocol, suffice it to say that they were very careful not to include anything that might skew the results.
This even extended to having all of the test subjects remain in the same not stressful environment and read the same magazine during the 90 minute time interval. Blood pressure monitoring was via automated equipment.
Even given the research by Hróbjartsson & Gøtzsche, it appears that that placebo effect/expectation bias does exist.
As stated previously, the outcomes may depend on the circumstances, knowledge and expectations of the test subject.
Wigwam Jones
Caesar non supra grammati
It really boggles me that so many people apparently have difficulty actually understanding what the placebo effect is. They think they know, but they most definitely do not.
Look, aspirin makes headaches better, right? Pretty much established for a long time now. You get a headache, if you take aspirin, your headache may get better. And if it does, we assume it is because that's what aspirin does. There might be other causes, but we certain expect the aspirin to make headaches better.
Now, if someone sneaks into your house and replaces your aspirin tablets with an identical-looking tablet, but they are inert and do nothing at all, then these faux aspirin tablets are now placebos.
If you take them, thinking that they are aspirin, and your headache gets better, you may be experiencing the placebo effect. Of course, it may be that your headache got better for other reasons; just like when you take the real thing.
But carefully-controlled studies have shown that this effect is real. People who take placebos thinking they are aspirin have an expectation that their headaches will go away, and they often do. This is not fake, the headache is really and truly gone.
Note that it does not work for broken legs. You can't take a pill after being told that it cures broken legs instantly and have your leg suddenly healed up. But pain, unlike broken legs, is experienced internally; it takes place in the mind, although it may have a physical cause.
What sciences does not know is by what mechanism this occurs. It appears that our own minds have an amazing capacity to change our perceptions. In the example above, the perception of pain.
Now, since we are talking about perception and how the mind can alter it, we come to audio equipment.
If a person has an expectation that they are hearing something that they believe and have been told is superior to some other thing, they may well experience the placebo effect and indeed hear it as superior.
That does NOT mean the sound they hear *IS* superior. Just that they may experience it that way, due to having been told to expect a better experience.
Nothing about the placebo effect, or expectation bias, changes reality. Reality is reality. But it may affect how some experience that reality; the part that takes place inside their own heads. Like pain from a headache, our minds can also control the perception of enjoyment.
It really should not be that hard to grasp. This is simple, logical, stuff. It's not an attack on anyone or any manufacturer or any kit, tweak, or other audio-related item. It's all taking place (in some cases) inside the mind of the listener. That is a real effect, but it does not change reality. It's mental.
Look, aspirin makes headaches better, right? Pretty much established for a long time now. You get a headache, if you take aspirin, your headache may get better. And if it does, we assume it is because that's what aspirin does. There might be other causes, but we certain expect the aspirin to make headaches better.
Now, if someone sneaks into your house and replaces your aspirin tablets with an identical-looking tablet, but they are inert and do nothing at all, then these faux aspirin tablets are now placebos.
If you take them, thinking that they are aspirin, and your headache gets better, you may be experiencing the placebo effect. Of course, it may be that your headache got better for other reasons; just like when you take the real thing.
But carefully-controlled studies have shown that this effect is real. People who take placebos thinking they are aspirin have an expectation that their headaches will go away, and they often do. This is not fake, the headache is really and truly gone.
Note that it does not work for broken legs. You can't take a pill after being told that it cures broken legs instantly and have your leg suddenly healed up. But pain, unlike broken legs, is experienced internally; it takes place in the mind, although it may have a physical cause.
What sciences does not know is by what mechanism this occurs. It appears that our own minds have an amazing capacity to change our perceptions. In the example above, the perception of pain.
Now, since we are talking about perception and how the mind can alter it, we come to audio equipment.
If a person has an expectation that they are hearing something that they believe and have been told is superior to some other thing, they may well experience the placebo effect and indeed hear it as superior.
That does NOT mean the sound they hear *IS* superior. Just that they may experience it that way, due to having been told to expect a better experience.
Nothing about the placebo effect, or expectation bias, changes reality. Reality is reality. But it may affect how some experience that reality; the part that takes place inside their own heads. Like pain from a headache, our minds can also control the perception of enjoyment.
It really should not be that hard to grasp. This is simple, logical, stuff. It's not an attack on anyone or any manufacturer or any kit, tweak, or other audio-related item. It's all taking place (in some cases) inside the mind of the listener. That is a real effect, but it does not change reality. It's mental.
Tom Bombadil
AK Member
I find expectation bias to be such a fascinating topic. And one that is very difficult to avoid falling prey.
I related the story about the store owner who stocked the same cable in 3 different colors but described them in three different ways and how nearly all of his customers over a several year period, bought into his descriptions.
And the story about when I switched cables and yet the salesman never picked up on it. He expected Cable A to have certain characteristics.
I also did a little bit more of this, but on a very small sample size and not in any controlled way. During the course of some of the in-store comparisons, other customers would wander by and ask about what we were doing. I would be flipping switches and somethings replaying the same music passages over and over. A few times I told the random customer that I was running a comparison on the input selector of the preamp. As in connecting to "CD" vs "AUX." That they were supposed to sound alike. To others, I said I was comparing an expensive interconnect which was supposed to "open up the sound" vs a cheap interconnect.
In all cases, the people to whom I said it was supposed to sound alike, agreed that it did sound alike. And in all cases when I said one was supposed to sound better, they said it did sound different.
When I ran sighted tests in my own system, between cables which were supposed to "add air around instruments", "enhance the soundstage", etc., it was very difficult to not perceive a difference. Even after failing 3 or 4 blind cable comparisons, I would listen to a new one and think that just maybe I could pass the next one with it, as it sure seemed to "lift a veil" in my sighted comparison. Then I would I fail that one too.
Vendors/manufacturers also use the same psychology with their recommended "break in" periods. Don't form any opinions until you have used the product for 100 hours. They know the odds of someone eventually convincing themselves that they spent their money wisely and buying into all of the product claims over 100 hours of use, are extremely high. Even if there was no difference at all at first or after 100 hours. For along with "expectation bias" there is also "justification bias."
I related the story about the store owner who stocked the same cable in 3 different colors but described them in three different ways and how nearly all of his customers over a several year period, bought into his descriptions.
And the story about when I switched cables and yet the salesman never picked up on it. He expected Cable A to have certain characteristics.
I also did a little bit more of this, but on a very small sample size and not in any controlled way. During the course of some of the in-store comparisons, other customers would wander by and ask about what we were doing. I would be flipping switches and somethings replaying the same music passages over and over. A few times I told the random customer that I was running a comparison on the input selector of the preamp. As in connecting to "CD" vs "AUX." That they were supposed to sound alike. To others, I said I was comparing an expensive interconnect which was supposed to "open up the sound" vs a cheap interconnect.
In all cases, the people to whom I said it was supposed to sound alike, agreed that it did sound alike. And in all cases when I said one was supposed to sound better, they said it did sound different.
When I ran sighted tests in my own system, between cables which were supposed to "add air around instruments", "enhance the soundstage", etc., it was very difficult to not perceive a difference. Even after failing 3 or 4 blind cable comparisons, I would listen to a new one and think that just maybe I could pass the next one with it, as it sure seemed to "lift a veil" in my sighted comparison. Then I would I fail that one too.
Vendors/manufacturers also use the same psychology with their recommended "break in" periods. Don't form any opinions until you have used the product for 100 hours. They know the odds of someone eventually convincing themselves that they spent their money wisely and buying into all of the product claims over 100 hours of use, are extremely high. Even if there was no difference at all at first or after 100 hours. For along with "expectation bias" there is also "justification bias."
Tom Bombadil
AK Member
I read of a study of a migraine medication. All participants were people who had regularly suffered from migraines for at least 5 years. They separated the participants into the control groups, with one of them taking a placebo but being told that they were taking a powerful new migraine medication.
Something like 20% of the group taking placebos were "cured." I found this fascinating. That these people whose lives had been so severely impacted by migraines and who had desperately wanted to be rid of them, and had taken other medication for years, to get rid of them, were now "cured" by a placebo.
robgmn
Super Member
I just got a copy of "The Mind-Body Connection" by Dr. John Sarnow.
After hearing his statement that back pain was a relatively unknown thing before WWII, it got me to really wondering just how much of what we deal with is all "in our minds".
I can't wait to start reading it.l
gvalent
Well-Known Member
Amen to that!
After reading the Wiki article which stated something to the effect that about 30% of people are subseptible to the placebo effect it reminded me of some reading I'd done years ago on hypnosis. Hypnosis has about the same percentage of subseptibilty and there is a strong correlation between hypnosis and the placebo effect.
These are two very fascinating subjects.
- Status
