"Expectation bias and plecebo effect are much stronger than people care to admit."

[robgmn]

Absolutely correct GV. Since this string is winding down, I would like to make a couple of comments.

1) Notice GV was the only one who knew what a "confound" was. So how could Rob or any other "scientific" gents address the subject in a knowing and thoughtful manner when they do not even understand the subject.

2) The meyers and moran paper was posted as an example by the "scientific", but again demonstrates the lack of scientific understanding of the subject.



"A" is somewhat self revealing. One is testing for ultra sonics yet uses typical CDs? Pretty weird.

"B" shows how one can manipulate, knowing or unknowingly, the test. Notice the authors did not separate the "golden ears" from others, so any "golden ears" was buried. It would seem to me that the test is as close to rigged as one can get.



And at least one place, they used 85db spl, so we don't know how much cochlea fatigue was involved in skewing the results.

And good question, how did these basic, major flaws get past the referee(s), AES. HMMM.

But the public is suppose to do in home audio dbt/abx testing and be accurate when no one explains the confounds. Keep the public ignorant. That should
rig the test towards no sonic difference every time.

Viewers, the lesson should be, to be very careful who to believe. Just because they claim to be "scientific" does not mean they are.
Agendas and marketing ($) are involved as we just read above.

Cheers and so long.

ps. I see Rob refused to answer if he is affiliated with a company or has a friend(s) who owned a company. He could have simply replied no if such
were the case.

I didn't realize you asked the question, but I am not involved nor affiliated with ANY manufacturer of ANY audio equipment in ANY business capacity (other than buying equipment that ultimately puts money in their coffers).

Many years ago, I USED to sell equipment produced by a variety of competing manufacturers.
I pushed the equipment for each sale that I thought best served the consumer for their specific application. Of course, I pushed the brands that I sold and not brands that I didn't stock.

You may think this is "trickery" or "dishonesty" on my part, but it made good business sense for me.

To your discussion of "confounds", the words "confounding variables" cannot be combined to make the word "confounds". The definition of "confound", per Merriam Webster (please don't ask me to prove the Merriam Webster Dictionary exists. Just take my word for it. ):
Definition of CONFOUND

1
a archaic : to bring to ruin : destroy
b : baffle, frustrate <conferences … are not for accomplishment but to confound knavish tricks — J. K. Galbraith>
2
obsolete : consume, waste
3
a : to put to shame : discomfit <a performance that confounded the critics>
b : refute <sought to confound his arguments>
4
: damn
5
: to throw (a person) into confusion or perplexity
6
a : to fail to discern differences between : mix up
b : to increase the confusion of

Another reference: con·found
kənˈfound/Submit
verb
3rd person present: confounds
1.
cause surprise or confusion in (someone), esp. by acting against their expectations.
"the inflation figure confounded economic analysts"
synonyms: amaze, astonish, dumbfound, stagger, surprise, startle, stun, throw, shake, discompose, bewilder, bedazzle, baffle, mystify, bemuse, perplex, puzzle, confuse; More
prove (a theory, expectation, or prediction) wrong.
"the rise in prices confounded expectations"
synonyms: contradict, counter, invalidate, negate, go against, quash, explode, demolish, shoot down, destroy, disprove; More
defeat (a plan, aim, or hope).
"we will confound these tactics by the pressure groups"
archaic
overthrow (an enemy).
2.
mix up (something) with something else so that the individual elements become difficult to distinguish.
"'nuke' is now a cooking technique, as microwave radiation is confounded with nuclear radiation".

You use "confounds" as a noun, of which no noun seems to exist.

Again I ask if you are using "confounds" (incorrectly) to represent "confounding variables"?

And again, when are you going to post the PhD information you have in which it has been proven the ABX/DBT testing is an invalid method?
If you can provide the scientific prove that these types of tests are not valid, I will gladly accept it.

Yes, I do fully understand what DBT and ABX testing are. They are an incredibly easy to understand method of testing that has been around since the late 1700's. They can be applied to pretty much anything that we can determine a difference between, using our known senses (sight, hearing, taste, smell, touch).

They can be used to support the primary topic of this thread, which is that Expectation Bias and the Placebo Effect can influence our preferences in audio equipment (or food, the color of paint, perfume, textiles, etc., etc.).

Lastly, you stated yourself that "Expectation bias can and will alter one's perception".
If this is true, why is it not possible to perform a test that eliminates the bias?
You claim testing is too stressful. Wouldn't you think that paying 10x for a new cable than the the old one would add a significant stressor to your own comparisons?
Remove that stressor and you have a much better shot at making a decision based on what you HEAR and not what your mind THINKS you should hear.
No?

 

[ofajen]

1) Notice GV was the only one who knew what a "confound" was.

Please don't make assumptions like this. While I wince at the gratuitous generation of nouns from perfectly self-respecting verbs, the meaning in this thread, though new to me, was immediately obvious from the context. My eyes glazed over due to the tedium of the back and forth exchanges, not due to uncertainty over the use of that word.

Cheers,

Otto

 

[Tom Bombadil]

Can anyone point to any DBT/ABX test where there were no incorrect responses when a switch was thrown, but no change was actually made?

I am unaware of any.

I know in the tests I conducted that it was commonplace for participants to make this error.

I also observed that participants were more nervous/anxious when I included the possibility of a switch not actually being a switch. They were more comfortable if there was a guarantee that a "switch" actually switched the gear being compared. When I asked why, they admitted that when they knew there was a switch that they felt they at least had a better chance of guessing it correctly.

I related the one comparison, of interconnects in a high end system of which the participant was very familiar with, where I didn't even ask the participant to try to identify which cable was which. I asked him to tell me when an actual switch was made. And he was allowed to skip as many "switches" as he wanted. All he had to do was after hearing a click, say "yes" if he thought the cable was actually switched or to remain silent and wait for the next click. He had about a 50% success rate.

I did this same thing a few other times. I think it's a great test. If you can't tell at all when the switch is made, then how much difference could there really be?

 

[dogscanskate]

I didn't realize you asked the question, but I am not involved nor affiliated with ANY manufacturer of ANY audio equipment in ANY business capacity (other than buying equipment that ultimately puts money in their coffers).

Many years ago, I USED to sell equipment produced by a variety of competing manufacturers.
I pushed the equipment for each sale that I thought best served the consumer for their specific application. Of course, I pushed the brands that I sold and not brands that I didn't stock.

You may think this is "trickery" or "dishonesty" on my part, but it made good business sense for me.

To your discussion of "confounds", the words "confounding variables" cannot be combined to make the word "confounds". The definition of "confound", per Merriam Webster (please don't ask me to prove the Merriam Webster Dictionary exists. Just take my word for it. ):
Definition of CONFOUND

1
a archaic : to bring to ruin : destroy
b : baffle, frustrate <conferences … are not for accomplishment but to confound knavish tricks — J. K. Galbraith>
2
obsolete : consume, waste
3
a : to put to shame : discomfit <a performance that confounded the critics>
b : refute <sought to confound his arguments>
4
: damn
5
: to throw (a person) into confusion or perplexity
6
a : to fail to discern differences between : mix up
b : to increase the confusion of

Another reference: con·found
kənˈfound/Submit
verb
3rd person present: confounds
1.
cause surprise or confusion in (someone), esp. by acting against their expectations.
"the inflation figure confounded economic analysts"
synonyms: amaze, astonish, dumbfound, stagger, surprise, startle, stun, throw, shake, discompose, bewilder, bedazzle, baffle, mystify, bemuse, perplex, puzzle, confuse; More
prove (a theory, expectation, or prediction) wrong.
"the rise in prices confounded expectations"
synonyms: contradict, counter, invalidate, negate, go against, quash, explode, demolish, shoot down, destroy, disprove; More
defeat (a plan, aim, or hope).
"we will confound these tactics by the pressure groups"
archaic
overthrow (an enemy).
2.
mix up (something) with something else so that the individual elements become difficult to distinguish.
"'nuke' is now a cooking technique, as microwave radiation is confounded with nuclear radiation".

You use "confounds" as a noun, of which no noun seems to exist.

Again I ask if you are using "confounds" (incorrectly) to represent "confounding variables"?

And again, when are you going to post the PhD information you have in which it has been proven the ABX/DBT testing is an invalid method?
If you can provide the scientific prove that these types of tests are not valid, I will gladly accept it.

Yes, I do fully understand what DBT and ABX testing are. They are an incredibly easy to understand method of testing that has been around since the late 1700's. They can be applied to pretty much anything that we can determine a difference between, using our known senses (sight, hearing, taste, smell, touch).

They can be used to support the primary topic of this thread, which is that Expectation Bias and the Placebo Effect can influence our preferences in audio equipment (or food, the color of paint, perfume, textiles, etc., etc.).

I'm still waiting for your proof rob ! Why do you ask for scientific proof when you don't want to show the science behind your assumptions, you are not exempt.

Talk in circles to me about the moon and the stars all you like but back up what you say, I wanna see them 1700 placebo tests. Make your buddies feel fuzzy and warm please ! But back it up .

If you can't show the data,then, you can post something absolutely prejudiced and biased like post # 250 (page 17) by your buddy Tom and shame us into the stigmatized position we deserve to be in because some of us do not agree with you and some of your cohorts.

We could have had a lot of fun exchanging different ideas from both sides of the coin. It's easier to bash the high end, manufacturers, their customers...


Btw, DBT' ABX are tools that can stand refinement. Nobody's against them as they could be useful under the correct conditions.

Also funny that post from Tom didn't get anybody riled up here. I read it and figured it was better to let it go and see how many agree with that !

 

[RayW]

A moderator has already stated that this thread is in its death throes. Please try to keep the discussion to placebo effect and expectation bias or it will be locked and left to die a peaceful death.

 

[robgmn]

I'm still waiting for your proof rob ! Why do you ask for scientific proof when you don't want to show the science behind your assumptions, you are not exempt.

Talk in circles to me about the moon and the stars all you like but back up what you say, I wanna see them 1700 placebo tests. Make your buddies feel fuzzy and warm please ! But back it up .

If you can't show the data,then, you can post something absolutely prejudiced and biased like post # 250 (page 17) by your buddy Tom and shame us into the stigmatized position we deserve to be in because some of us do not agree with you and some of your cohorts.

We could have had a lot of fun exchanging different ideas from both sides of the coin. It's easier to bash the high end, manufacturers, their customers...


Btw, DBT' ABX are tools that can stand refinement. Nobody's against them as they could be useful under the correct conditions.

Also funny that post from Tom didn't get anybody riled up here. I read it and figured it was better to let it go and see how many agree with that !

I'm asking for posi to post the PhD proofs that he keeps jabbering on about and he refuses to do it in spite of continuing to reference them.
Here's mine.

Merriam Webster definition:
Definition of DOUBLE-BLIND
: of, relating to, or being an experimental procedure in which neither the subjects nor the experimenters know which subjects are in the test and control groups during the actual course of the experiments — compare open-label, single-blind

This proves that these tests exist (else you accuse them of defining something that doesn't exist).

I won't post the actual tests, but if you would like proof that tests have been done in the past, here are some references:
Rivers WHR and Webber HN. “The action of caffeine on the capacity for muscular work” Journal of Physiology 36: 33-47: 1907 (August).

A very early example of randomisation and double blinding was an evaluation of homeopathy conducted in Nuremberg in 1835. it was known as the "Nuremburg salt trials".

Though it's from Wiki and very likely a lie (not), "The French Academy of Sciences originated the first recorded blind experiments in 1784: the Academy set up a commission to investigate the claims of animal magnetism proposed by Franz Mesmer."

A test was done for national television on PBS in 1997 to determine whether a person who claimed he could successfully "dowse" was actually able to do it.
In the blind test, his success rate was far worse than what would be expected by random chance (10%). He actually got NONE of his dowsings correct.

A well-publicized example of a test was the "Pepsi Challenge". You may dispute the results as it was sponsored by Pepsi, but cannot dispute the tests happened.

J Clin Invest. 1996 February 15; 97(4): 1129–1133.
doi: 10.1172/JCI118507
PMCID: PMC507162
The effect of docosahexaenoic acid on aggression in young adults. A placebo-controlled double-blind study.

March II 1981, Volume 73, Issue 1, pp 95-96
A preliminary double-blind study on the efficacy of carbamazepine in prophylaxis of manic-depressive illness

Mayo Clinic Proceedings
Volume 78, Issue 6 , Pages 687-695, June 2003
Prospective, Randomized, Double-Blind Study of the Efficacy and Tolerability of the Extended-Release Formulations of Oxybutynin and Tolterodine for Overactive Bladder: Results of the OPERA Trial

Yes dog, the tests really do exist in history and have been performed.

 

[dogscanskate]

Rob, thanks for your efforts, I appreciate that. While we may be worlds apart, we may be striving towards the same ultimate goal. We can gracefully disagree without spreading ridicule towards each other.

I still believe that there are many hurdles to overcome before we can collect DBT data that can be repeatable and accurate as far as audio is concerned. In placebo tests, the patient receives the same support as the pill patient that is ''the expectation to heal'' bias. In audio, random DBT switching suppresses it. Could it be possible that this will trigger another type of bias? Having to sort X from Y and add the task of deciphering which signal is on, X or Y at the flip of the switch. Are you good at multi- tasking? The listener is given 2 targets to identify instead of just concentrating on X or Y.

Pandovsky linked the Edgar Vilchur video on his gear vs live. In the video, Vilchur said that recordings will capture the ambiance (acoustics) of the venue whether it be in studio or elsewhere. That would cause smearing of the signal as the natural acoustics captured in the recording will be added to the acoustics of the listening room when played back. He therefore recorded outdoors to avoid hall acoustics to have a chance at having his gear sound indistinguishable from the live performers.

That seems to narrow music selection tremendously for scientific DBT tests. That could be a reason nobody can hear a difference between wires, connectors, isolation devices, shelves to support gear...

As in the Pepsi test, they monitored the brain to see which parts were excited during the tests. Which part of the brain would be triggered in DBT tests for those who successfully pass the test? Is that repeatable for all successes? The same could be said for those who fail.

Why not test seeing people vs blind people in a DBT. We could learn something there (maybe not).

I feel there is still much more to learn about ABX/DBT. You may be satisfied with what is happening now, I feel it has room to grow. It could be a wonderful tool if we learn to use it correctly and wisely.

The concept of audio is to listen to music not necessarily to the differences. If DBT can be used to enhance your listening pleasure because you learn from it, who would stand against it?

 

[Jon_Logan]

Thanks Pandovski for the SAS linkage.

Yeah there's more history to these tirades between Ethan and Steve.

Google sammett/frog/ethan

It seems there's some never ending "I hate you, I hate you more"'s.

 

[Gang-Twanger]

Placebo-schmebo...

I don't multi-task very-well... Too-complicated.

 

[Tom Bombadil]

My own experiences in participating in blind and double blind tests were very positive. I learned a great deal. I felt very little to no pressure to pass the tests, even on equipment where I had some reason to justify my expenditures. Perhaps some of this is due to my being a Myers-Briggs INTP, where one of the characteristics is that we are not very defensive about our present opinion/belief and are willing to switch that position should another present a stronger argument.

Did I find that I had spent money on items which I could not distinguish from less expensive gear? Yes! Did I "save" myself from making more purchases of equipment which I could not distinguish from gear I already owned? Yes!

I feel it was a great tool towards building my system (well, 3 systems) in a cost efficient way to great sound (by my standards). And by involving several others, it gave me some insight, and confidence, into the decisions I made as well as on the topic of "expectation bias."

If I were in the market to purchase another major component, I would certainly employ blind testing again. Based upon my own experience with "expectation bias" I don't trust my own ears in sighted comparisons. I would rather depend solely upon my ears to make these decisions.

 

[RayW]

I still believe that there are many hurdles to overcome before we can collect DBT data that can be repeatable and accurate as far as audio is concerned. In placebo tests, the patient receives the same support as the pill patient that is ''the expectation to heal'' bias. In audio, random DBT switching suppresses it.

If you want a blind audio test to mirror medical placebo testing then you have a switch that you tell them changes from A to B but make the switch not change anything. That is the analog of the medical test, the expectation of something that is in reality, nothing. However you have already voiced your concerns over such deceit so it seems that you will not be satisfied either way.

Could it be possible that this will trigger another type of bias? Having to sort X from Y and add the task of deciphering which signal is on, X or Y at the flip of the switch. Are you good at multi- tasking? The listener is given 2 targets to identify instead of just concentrating on X or Y.

In most tests the listener is not trying to identify the 2 DUTs, only the difference. The question asked is, "Is there a difference between A and B?" That is not multitasking, it is simply comparison.

 

[dogscanskate]

In most tests the listener is not trying to identify the 2 DUTs, only the difference. The question asked is, "Is there a difference between A and B?" That is not multitasking, it is simply comparison.

My point being that if the test is random, I might be getting A vs A when all I am asked is if I hear a difference between A and B. If I can't hear differences between A and B the point is moot. If there is a possible difference to be heard, I have to identify that I am getting A vs B or B vs A first and foremost then identify the difference. That is multitasking. I would be stressed trying to figure out on each switch if it is indeed A or B first.

Even though it's not about the 2 DUTs, the random test is also about them whether you like it or not. That is a ''confound''. If you want me to tell the difference between A and B, do so, without the random switching.

If you are so sure about the results before the test is actually done, at least give the listener a fair chance. Go do that with golf clubs, same difference.

On the first point Ray, I see where you have me boxed in ! Now my understanding is that should we be able to hear no differences? The original test would be for you to tell me that speaker B (as an example, POS speaker) is just as good as speaker A (let's say your TOTL design), by positive reinforcement of the ''placebo/expectations effect''.

If you absolutely want to remove the ''expectations effect'', which makes sense for audio purposes, then randomizing the test is also needless because all you want to know is if I hear the difference. Why compound the task?

 

[fpitas]

This thread is turning out like I thought it would. Another example of expectation bias? :scratch2:

 

[I.R.Brainiac]

was immediately obvious from the context.

Thank you. :thmbsp:

 

[Tom Bombadil]

If you absolutely want to remove the ''expectations effect'', which makes sense for audio purposes, then randomizing the test is also needless because all you want to know is if I hear the difference. Why compound the task?

Because it strikes at the very heart of the issue.

If you know A & B are different and when you hear the switch take place, you can simply say "I hear a difference" and you will be correct 100% of the time. It really isn't a test at all. It just means you heard the switch click.

But if the switch click can be between A & A or B & B, then you have to rely upon your ears to discern a difference. If the objects being compared actually sound different, then you should be able to tell if the sound did or did not change. And if it turns out that you really can't tell the difference, that will show up in your results.

This type of test works most effectively when it is conducted in a system the participant is quite familiar with. Using music that they select. Say to go into their home system and set up the comparison with just one change to the system vs the stock system. If we change one cable, does it make a difference? If we change the CD player or DAC or preamp, does it make a discernible difference?

 

[aldread]

Positron:

Maybe I did go a little far, but with Rob and others insulting so much, it is difficult to tell who knew and who did not.

If you didn't think we understood the term (which I and I'm sure everyone else did), why did you continue to use it endlessly?

 

[Pandovski]

Thanks Pandovski for the SAS linkage.

Yeah there's more history to these tirades between Ethan and Steve.

Google sammett/frog/ethan

It seems there's some never ending "I hate you, I hate you more"'s.

Ahaa I see. http://www.stereophile.com/content/steve-sammettfrogethan-triangle-needs-stop-1

Apparently this Placebo thread of ours has been taken over by people of the industry.

I'm cheering for

Ethan Winer

.

 

[Jon_Logan]

That is a good one. The subject of the string is Ethan attacking one of his competitors on Stereophile. An ethics violation. So what does that string have to do with presenting science on this string Pandovski and Jon.[?]



Cheers.

From post 1. http://www.youtube.com/watch?v=BYTlN6wjcvQ

 

[gvalent]

positron, is the problem with the OP's video regarding JJ's portion of the video?*
Is this what all the back and forth on thread is all about about?

 

[Mid-Fi]

I'm truly sorry I started this thread.