Absolute Proof that blind testing is flawed !!!
- Thread starter gvalent
- Start date
voltcontrol
Fizheuer Zieheuer
:tresbon:
It is fun to be from the place that is always mentioned as the worst possible place, except maybe for Flint.:yes:
I assume you've seen the Kentucky Fried Movie?
Pennington: These are the Hartz Mountains of Asia. A terrain so rugged, so treacherous, no country will claim it.
Asquith: Worse then Detroit?
Pennington: I'm afraid so.
Dr. Klahn: Take him to... Detroit!
CIA Agent: No! No, not Detroit! No! No, please! Anything but that! No! No!
In reality, I think Gary, Indiana beats Detroit. They're so depopulated that even the crime left. :lmao:
Who's "noone"??
Scuzzer
Fixed Bias
no one knows
"Did you know that during the course of World War 11,..."
World War 11? When did that happen?
bs
Frank's OCR software is probably to blame for that typo. The original article ran in Audio Basics so many years ago that it was probably created on a typewriter (remember them?) rather than a PC with a word-processing program.
Also, don't knock Van Alstine until or unless you've heard his gear. He can be difficult and opinionated, but there's no denying that he makes some great equipment.
dr*audio
Fish fingers and custard!
I participated in an interesting blind test, at the CES Las Vegas, in the 1980's. BASF was introducing their new Chrome cassette tape. They played a CD and simultaneously recorded it on a Nakamichi Dragon (I think.) They randomly switched between the direct playback and the playback from the Nak, and you had to try to determine which it was. I got it right most of the time. They were a bit surprised, and said only one or two other people had been able to tell the difference. I told them the bias adjustment was off on the deck. Got a free BASF Chrome tape for passing the test. Very good tape; too bad they stopped making it.
Anyway, the point I am making here is that blind tests can be revealing and if you are the type of person who gets stressed out in a situation like that, then you will be distracted, and the test is invalid, but it doesn't mean that blind testing is invalid, only that you should have been disqualified as a subject.
Anyway, the point I am making here is that blind tests can be revealing and if you are the type of person who gets stressed out in a situation like that, then you will be distracted, and the test is invalid, but it doesn't mean that blind testing is invalid, only that you should have been disqualified as a subject.
dr*audio
Fish fingers and custard!
Some food for thought:
Here's a little test I do to evaluate the sound of a preamp. I connect the Aux input of the preamp to the record output of my system, and the preamp output to the play input of the tape loop in my system. Then I use the preamp output level control to match the level when the tape monitor switch in my system is switched from source to tape. I used the power meters on my power amp to measure the level, with a 1KHz tone from a test CD.
Then I listen to music, switching back and forth from source to tape. If I hear any changes to the sound, I know that the preamp is not neutral, and is altering the sound.
I first tried this with a Crown preamp. A friend had wanted to know if I could improve the sound. Every time the switch was set to tape, the soundstage collapsed, and the highs got mushy sounding.
I upgraded all the opamps in the Crown and repeated the test. Now, when the Crown was switched in, absolutely no difference could be heard. I then performed this test for my friend and he came up with the same observation.
I realize this test was not blind for me, but it was for my friend. It is also not the same protocol as a blind or double blind test. I am simply making the point that you can hear differences between components in a test where you switch them back and forth immediately, and it isn't necessary to listen for hours or days and then switch things around and go by memory. Also, there are reasons why components sound different; such as the performance of the opamps used.
Here's a little test I do to evaluate the sound of a preamp. I connect the Aux input of the preamp to the record output of my system, and the preamp output to the play input of the tape loop in my system. Then I use the preamp output level control to match the level when the tape monitor switch in my system is switched from source to tape. I used the power meters on my power amp to measure the level, with a 1KHz tone from a test CD.
Then I listen to music, switching back and forth from source to tape. If I hear any changes to the sound, I know that the preamp is not neutral, and is altering the sound.
I first tried this with a Crown preamp. A friend had wanted to know if I could improve the sound. Every time the switch was set to tape, the soundstage collapsed, and the highs got mushy sounding.
I upgraded all the opamps in the Crown and repeated the test. Now, when the Crown was switched in, absolutely no difference could be heard. I then performed this test for my friend and he came up with the same observation.
I realize this test was not blind for me, but it was for my friend. It is also not the same protocol as a blind or double blind test. I am simply making the point that you can hear differences between components in a test where you switch them back and forth immediately, and it isn't necessary to listen for hours or days and then switch things around and go by memory. Also, there are reasons why components sound different; such as the performance of the opamps used.
stereofanboy
Addicted Member
One thing that could be gained is a little jolt of satisfaction for folks who are sick of getting a condescending attitude from the golden eared select. This is assuming that the golden eared select are not very successful in their choices. More usefully, a sort of winnowing through the chaff, that can help, if not with buying descisions, then at least with shopping descisions.
It seems reasonable to me that a person could hear immediately if there was a difference in sound between two components. I think it would take longer to definitely say which was the better of the two.
A common argument against blind testing is that a person can't make an informed decision based upon a quick analysis. I think that listening to a wide variety of material is necessary, otherwise the test could be skewed by the choices of listening material. That takes a little time. The golden ears say it takes a couple of weeks, or a month. I say fine, take a couple of weeks or a month, just don't peek during that time. It would have to be double blind because the more time you spend with the switching guy, the more his prejudices will rub off on you, even if they are unexpressed.
This would be too cost and logistics prohibitive to have a large enough sampling to be statistically indicative, but it would be dandy to use one of those trained audio reviewers. That's what he would be doing all day anyway, right? The only difference is they would need to order some extra sandwiches to feed the switcher guy and the reviewer couldn't see, but only hear what he was testing. Oh, and he probably shouldn't be doing this at home. It needs to be a place the reviewer can't access when no one else is around.
I don't deny differences between amps or speakers or even wires. I don't even really believe that the magazine reviewers don't have a developed skill at listening that surpasses that of most of humanity, including me. I just don't believe that a listener, any listener, can banish all his prejudices by an act of will. I think those prejudices will lurk in his subconscious mind, subtly altering what the conscious mind thinks. If you believe that you are so in control that prejudices don't sway you, or that you aen't prejudiced, then you are a candidate for double blind testing.
So...are all you objective-uber-alles guys this much fun to hang out with in real life? Because I've been encountering a lot of you folks on different threads lately and I have to wonder if this is how you live every aspect of your lives.
Say we're at a party, and I say, "Man, I'm having a blast!"
Would you say, "Well, it has yet to be determined whether you are, indeed having a blast. I must run the numbers and ask 20 people if they are also having a blast. If a majority of those people, randomly chosen of course, are indeed having a blast..without seeing that others are having a blast, mind you (wouldn't want them to be swayed)...then I will give you the idea that perhaps you are indeed having a blast. Until then, I'll be at the computer."
Because I don't see how listening to my stereo, enjoying the music, is much different than having fun at a party.
Say we're at a party, and I say, "Man, I'm having a blast!"
Would you say, "Well, it has yet to be determined whether you are, indeed having a blast. I must run the numbers and ask 20 people if they are also having a blast. If a majority of those people, randomly chosen of course, are indeed having a blast..without seeing that others are having a blast, mind you (wouldn't want them to be swayed)...then I will give you the idea that perhaps you are indeed having a blast. Until then, I'll be at the computer."
Because I don't see how listening to my stereo, enjoying the music, is much different than having fun at a party.
The purpose of a double blind A/B/X test is that bias is eliminated, since it is impossible for bias to affect the listeners' choices. The double blind A/B/X test does not test for quality or preference, it simply tests to see if a difference can be detected. Listeners aren't asked "Which one do you like?".
Listen to the first sample, listen to the second sample, listen to the third sample; is the third the same as the first or the second? Listeners can't tell.
Well, apparently not. And the purpose of double blind testing is not to say which is better, only to test if the listeners can hear a difference.
Which, apparently, listeners cannot do. That's what is so interesting about this topic.:yes:
I'm afraid the determining factor is the quality of the dip.
We'd have to double blind test the dip. I'll be first in line for that, by the way. Love me some dip. Unless it is sour cream based... which I am SURE I could tell with my eyes closed and blindfolded because I really can't stand the stuff. Give me some Guacamole....
DonQuixote99
just give me some truth
Listening without measuring is blind.
Measuring without listening is deaf.
Measuring without listening is deaf.
Anything that doesn't start with peeling the lid off the grocery store container is good. When I get the basement done I'll show you some serious guac.:yes:
:thmbsp: