"Expectation bias and plecebo effect are much stronger than people care to admit."

[gvalent]

In positron's defence, he is using the word correctly in this context.

Used in statistics and scientific evaluation the definition is as follows:

"Confounding variables (aka third variables) are variables that the researcher failed to control, or eliminate, damaging the internal validity of an experiment."

 

[JBL GUY]

In positron's defence, he is using the word correctly in this context.

Used in statistics and scientific evaluation the definition is as follows:

"Confounding variables (aka third variables) are variables that the researcher failed to control, or eliminate, damaging the internal validity of an experiment."

Thank you gvalent.

I did not realize how many did not understand the correct meaning as used in this context.

This may help keep the thread on track.

 

[robgmn]

In positron's defence, he is using the word correctly in this context.

Used in statistics and scientific evaluation the definition is as follows:

"Confounding variables (aka third variables) are variables that the researcher failed to control, or eliminate, damaging the internal validity of an experiment."

Yes, we are getting off topic, but it's a little tough to discuss something when words are used that do not in any way apply to the discussion.

I wouldn't discuss speakers and describe a tweeter as a Finch.
Confounding variables were not discussed (unless someone created their own word by combining and shortening two, then creating a definition for it. We don't get to do that in English. )

Now, I wonder if there would ever be mass acceptance of the results of a DBT/ABX test if sponsored and administered by a third party with input from both believers and non-believers onthe setup of the test?

I wonder how many variables would be addressed before the test would be agreeable to all?

 

[Tom Bombadil]

Now, I wonder if there would ever be mass acceptance of the results of a DBT/ABX test if sponsored and administered by a third party with input from both believers and non-believers onthe setup of the test?

I wonder how many variables would be addressed before the test would be agreeable to all?

I don't think this will ever happen. People have been calling for this over the past 25-30 years and I feel we are further apart now than back then.

We have to understand that it is not in the best interest of the audio sales industry to have such a thing. Let's say there were standardized double blind tests, accepted by manufacturers, reviewers, and consumers. Not all of them, but a respected consortium of them.

This would certainly not be in the best interests of manufacturers, for their very being is based upon creating products which sound different/better. What if a $5000 preamp was compared to an $800 preamp and found to be indistinguishable in 16 out of 20 blind tests? Or a $25,000 amplifier vs a $1500 one? To say nothing of cables, isolation devices, break-in periods, etc.

This would make selling a product far more difficult. Manufacturers might even turn toward making their products sound different, rather than better (or at least seemingly better).

Likewise it would not be in the best interests of audio store owners. They need the "magic" of the allure of higher cost products and enhancements and their associated higher profit margins, to stay in business. They "sell" their own expertise on identifying which products are the best in their opinions.

If there was a Consumer Reports type publication which published all of these blind test results, which carried the validity of being backed by major manufacturers, reviewers, leading audiophiles, etc., then the store salespeople would lose much of their perceived "added value."

Need I say anything about the reviewers side of this? Their existence depends upon convincing consumers that their opinions carry weight. Even if some of their claims are preposterous and make manufacturers roll their eyes, the manufacturers know they have to live with it.

Consumers have a large say in this too. The high end audiophile consumer wants to read how all of this makes a difference. There is little demand from them for reviews which state that there are little to no differences between many products.

So it just isn't going to happen. Even if there are (and I believe there are) perfectly valid methods to perform double blind audio tests, there is no reason for the audio industry to accept them. They are much better served by continuing to belittle and cast doubt upon such things.

 

[Pandovski]

I highly value all tests and research data coming from the Harman labs done by either Dr. Sean Olive or Dr. Floyd E. Toole.

I somewhat trust data published by a company with an R&D department and not so much cottage industry company with no R&D, one oscilloscope, one multimeter and 50 pages writings on electron movement. Exception is BOSE, I don't trust anything they say despite them having a R&D department.

I also doubt any credible and authorized tests will reduce snake oil marketing and buyers irrational decisions. Maybe, but I am skeptical.

Sound quality
In the audiophile community, the sound from the SACD format is thought to be significantly better than older Red Book CD-DA format CD recordings. For example, one supplier claims that "The DSD process used for producing SACDs captures more of the nuances from a performance and reproduces them with a clarity and transparency not possible with CD.

Comparison with CD
In September 2007 the Audio Engineering Society published the results of a year-long trial, in which a range of subjects including professional recording engineers were asked to discern the difference between SACD and compact disc audio (44.1 kHz/16 bit) under double blind test conditions. Out of 554 trials, there were 276 correct answers, a 49.8 % success rate corresponding almost exactly to the 50 % that would have been expected by chance guessing alone. The authors commented:

Now, it is very difficult to use negative results to prove the inaudibility of any given phenomenon or process. There is always the remote possibility that a different system or more finely attuned pair of ears would reveal a difference. But we have gathered enough data, using sufficiently varied and capable systems and listeners, to state that the burden of proof has now shifted. Further claims that careful 16/44.1 encoding audibly degrades high resolution signals must be supported by properly controlled double-blind tests.
Following criticism that the original published results of the study were not sufficiently detailed, the AES published a list of the audio equipment and recordings used during the tests.

Comparison with DVD-A
Double-blind listening tests in 2004 between DSD and 24-bit, 176.4 kHz PCM recordings reported that among test subjects no significant differences could be heard. DSD advocates and equipment manufacturers continue to assert an improvement in sound quality above PCM 24-bit 176.4 kHz. Despite both formats' extended frequency responses, it has been shown people cannot distinguish audio with information above 21 kHz from audio without such high-frequency content.

Source: http://en.wikipedia.org/wiki/Super_Audio_CD

 

[I.R.Brainiac]

I don't think this will ever happen. People have been calling for this over the past 25-30 years and I feel we are further apart now than back then.

We have to understand that it is not in the best interest of the audio sales industry to have such a thing. Let's say there were standardized double blind tests, accepted by manufacturers, reviewers, and consumers. Not all of them, but a respected consortium of them.

This would certainly not be in the best interests of manufacturers, for their very being is based upon creating products which sound different/better. What if a $5000 preamp was compared to an $800 preamp and found to be indistinguishable in 16 out of 20 blind tests? Or a $25,000 amplifier vs a $1500 one? To say nothing of cables, isolation devices, break-in periods, etc.

This would make selling a product far more difficult. Manufacturers might even turn toward making their products sound different, rather than better (or at least seemingly better).

Likewise it would not be in the best interests of audio store owners. They need the "magic" of the allure of higher cost products and enhancements and their associating higher profit margins, to stay in business. They "sell" their own expertise on identifying which products are the best in their opinions.

If there was a Consumer Reports type publication which published all of these blind test results, which carried the validity of being backed by major manufacturers, reviewers, leading audiophiles, etc., then the store salespeople would lose much of their perceived "added value."

Need I say anything about the reviewers side of this? Their existence depends upon convincing consumers that their opinions carry weight. Even if some of their claims are preposterous and make manufacturers roll their eyes, the manufacturers know they have to live with it.

Even consumers have a large say in this. The high end audiophile consumer wants to read how all of this makes a difference. There is little demand from them for reviews which state that there are little to no differences between many products.

So it just isn't going to happen. Even if there are (and I believe there are) perfectly valid methods to perform double blind audio tests, there is no reason for the audio industry to accept them. They are much better served by continuing to belittle and cast doubt upon such things.

I...I think I love you. :tresbon:

 

[Hyperion]

I...I think I love you. :tresbon:

I am not sure I'd go that far - but it was compelling reading indeed.

 

[gvalent]

I don't think this will ever happen. People have been calling for this over the past 25-30 years and I feel we are further apart now than back then.

We have to understand that it is not in the best interest of the audio sales industry to have such a thing. Let's say there were standardized double blind tests, accepted by manufacturers, reviewers, and consumers. Not all of them, but a respected consortium of them.

This would certainly not be in the best interests of manufacturers, for their very being is based upon creating products which sound different/better. What if a $5000 preamp was compared to an $800 preamp and found to be indistinguishable in 16 out of 20 blind tests? Or a $25,000 amplifier vs a $1500 one? To say nothing of cables, isolation devices, break-in periods, etc.

This would make selling a product far more difficult. Manufacturers might even turn toward making their products sound different, rather than better (or at least seemingly better).

Likewise it would not be in the best interests of audio store owners. They need the "magic" of the allure of higher cost products and enhancements and their associating higher profit margins, to stay in business. They "sell" their own expertise on identifying which products are the best in their opinions.

If there was a Consumer Reports type publication which published all of these blind test results, which carried the validity of being backed by major manufacturers, reviewers, leading audiophiles, etc., then the store salespeople would lose much of their perceived "added value."

Need I say anything about the reviewers side of this? Their existence depends upon convincing consumers that their opinions carry weight. Even if some of their claims are preposterous and make manufacturers roll their eyes, the manufacturers know they have to live with it.

Even consumers have a large say in this. The high end audiophile consumer wants to read how all of this makes a difference. There is little demand from them for reviews which state that there are little to no differences between many products.

So it just isn't going to happen. Even if there are (and I believe there are) perfectly valid methods to perform double blind audio tests, there is no reason for the audio industry to accept them. They are much better served by continuing to belittle and cast doubt upon such things.

I pretty much feel the same way.
I know a lot of people would disagree with this, but seriously, how much of a sonic difference is there between a well made $1,000 amp compared to a well made $5,000 amp?
Not much if any in my opinion.

 

[I.R.Brainiac]

Mine are not assumptions. They are proofs that already exist.
The proof that double-blind and ABX testing is valid scientific method.

Do you believe in the laws of physics because you have tested every one of the for yourself, or do you accept things that other people have already proven scientifically?
How about the earth? Have you ever PROVEN the circumference to yourself, or do you accept the proofs that others have established?
How about your audio system? have you PROVEN that the signal going to your speaker is AC current, or have you accepted what others have already proven?

I don't think it should be necessary for me to prove double-blind and ABX testing are valid when it has been done so many times before over the past 200+ years, do you?



I didn't state AUDIO double-blind testing has existed, I stated the method ITSELF has existed (in response to your statement that it has never been proven as a valid method of testing).

Big difference.

As for cochlear fatigue:
If you claim that cochlear fatigue can make the results of double blind and/or ABX testing invalid, then you are stating that no personal preference can have ANY solid basis.

How would you possibly know that you made the correct choice?
Maybe your cochlear fatigue caused you to err and choose the wrong (wire, speaker, cart, amp, etc.).

You invalidate your own argument against testing.

You can't have it both ways, chief.

And, please stop referencing all of your special sources and incidental contacts over the years.

Les Paul was a customer of mine and I sat in is house and talked audio with him.
It doesn't make me any more knowledgeable about playing a guitar or creating music than your "contacts" make YOU about this discussion.

A friend of mine is an Oncologist.
It doesn't qualify me to diagnose cancer in someone.

Stick to the facts, and don't expect me to think you are knowledgeable because you once met or talked to someone who IS.

This also made me feel a warm fuzzy.

 

[Pandovski]

I remember Jeremy Clarkson's rationale on hyper cars, why we buy them if they are impractical and we will never exploit their full potential as long as we live.

He said when I buy a watch that can sustain 1000ft under water and work with molecular precision and has 5000 parts crammed in a schilling sized case, I am actually buying a promise by the manufacturer. I can never test, prove or disprove their proclaim. I will never dive that deep or count 1/1000000000th of a second or disassemble the watch in 5000 bits. If they lie no harm done as long as we all agree to shut up about it and never check.

I am buying a story. I want space age paper cone woofers, military grade aluminum alloy boxes and beryllium twitters that are highly poisonous when they break in my living room and come with evacuation procedure in the manual. I want that.

 

[I.R.Brainiac]

and beryllium twitters that are highly poisonous when they break in my living room and come with evacuation procedure in the manual. I want that.

I have those...they rock.

 

[robgmn]

I have those...they rock.

Prove it


I agree about the difficulty of making the test happen, thanks to the vendors AND consumers and actually find it disappointing.
In essence, some consumers are actually supporting the fact that they MAY be paying a huge increase in the cost of equipment merely for the cache of a boutique name.

Can anyone think of other products that this happens with?

 

[chicks]

Can anyone think of other products that this happens with?

Check out DeBeers history and massive marketing campaigns of diamonds for a very similar story.

 

[RayW]

1) Notice GV was the only one who knew what a "confound" was.

The only one? Are you trying to be deliberately insulting or do you really believe that nobody else understood that?

 

[Pandovski]

I wish positron explained non-Herzian waves or the Voynich Manuscript to me. I have troubles sleeping tonight.

 

[RayW]

Read the previous posts, and show me one who explained it before GV did.
(If you understood, that is good.)

Cheers.

That's pretty much absurd. The fact that nobody explained it has nothing to do with who, or how many, understood. You've been doing your best to show everyone how you have a better handle on this than anyone else. Insulting all of them is not the correct way to show that.

 

[Pandovski]

Unfortunately, that does not explain why you posted obviously false information on the unsuspecting public.


Pandovski:


Cheers.

Dude, go and fix it over at Wikipedia. People are being misinformed. It is your responsibility to do something!

 

[JBL GUY]

BTW, I understood the use of a confound in the context of this discussion.

But more to the point and quite simple per the title of this thread "Expectation bias and plecebo effect are much stronger than people care to admit."

Can anyone point to any DBT/ABX test where there were no incorrect responses when a switch was thrown, but no change was actually made?

Simple test, is A the same as B or different? Any test, anywhere, where all of the test subjects were 100 percent accurate all of the time?

If it is not expectation bias, what is it?

And no matter what it is, how does it affect the results of the tests?

 

[JBL GUY]

Thank you.

 

[Pandovski]

And none of your scientific friends objected to your posting false information that has been known for years.

So you and your friends post clearly false information while attacking me for posting clearly scientific, PHD, information. Interesting.


Cheers.

:scratch2: