Blind tests

[JohnVF]

DBT/ABX are useful when someone needs to prove that something sounds different. Proof is something that is sorely lacking in the marketing of audio products. Proof is exceptionally lacking on some of the more outlandish audio products, especially the ones that try to use pseudo science to convince the undereducated that there's something real behind the curtain.

And my position is..it's not proof. It can be useful in design, as I fully admit, and if you really want, choosing things for yourself if you really want every variable taken out of the equation. I get that, but disagree that it is proof. I think the tests, as they have been done, have many MORE variables (new room, different gear than I'm used to, etc etc) than i would experience myself in my own home with my eyes open. Which is also why I said it would be fun to do it in my home, over a long audition, with perhaps two amps. But that's just my opinion. After all, I'm all strawman all the time...feel free to call me the scarecrow.

Unfortunately, most (not all..that would be a strawman again...I'm just going off of the course all these threads seem to take) of the times that DBT/ABX testing is brought up, the discussion starts to lean towards furthering some tin foil hat theory that all amps sound the same/the entirety of the high end exists to rip you off claim. Not all of them, but many of them do and it's not that black and white. I'm NOT anti science, nor do I think everything in the high end is on the level (I'm an advertising creative director...like Don Draper....trust me, I wouldn't believe anything a marketer told me, I probably know more about what goes on behind the doors in marketing than 95% of this board). The amp on my radar is on their because of what's under the casing, not the name ON it. I think $5000 cables are ludicrous, especially since that crap further paints the industry I work in as a bunch of slippery liars. I like my speakers because of their time and phase alignment and the fact that there are no parallel sides in their cabinets. I don't see the voodoo in any of this. For some reason, anytime anybody professes to care only on a subjective level, we're lumped in with all the cable lovers and those who put stones on their amps or whatever. That's as accurate as me thinking everybody who believes in DBT/ABX testing listens to Large Advents and '70s monster receivers thinking it's the height of audio technology. We're all much more varied than the stereotypes let on. We're all much more complicated in our choices than the extremists believe. I think I'm pretty fair on most accounts, but apparently I'm not.

 

[willysan]

I never thought my OP might spark such a rich debate. Let me add a personal experience about a "non-blind" test.
Some years ago I bought an amplifier I´d been longing for but couldn´t afford. However, this amplifier had a tag price much lower. My first thought was "It´s a piece of junk" but I decided to check it. The amplifier had a very good cosmetic appearance; the owner turned it on and the sound was excellent. I couldn´t imagine why he was asking such a low price. He told me: "I need the money; I´ve just bought the new model. Listen to it! It´s wonderful".
I listened to the new model and it didn´t sound that great. It was O.K. but too expensive. Thus, at the same time, two men were thinking: "My amplifier is the best".
Contradiction? I don´t think so. We both had gotten what we had been waiting for and we were happy with our choices. I love my amplifier but I must admit I´ve put too much feelings on it. The result of this "non-blind test" was two happy men, but a well-conducted DBT might have revealed more objective information.

 

[JohnVF]

I never thought my OP might spark such a rich debate. Let me add a personal experience about a "non-blind" test.
Some years ago I bought an amplifier I´d been longing for but couldn´t afford. However, this amplifier had a tag price much lower. My first thought was "It´s a piece of junk" but I decided to check it. The amplifier had a very good cosmetic appearance; the owner turned it on and the sound was excellent. I couldn´t imagine why he was asking such a low price. He told me: "I need the money; I´ve just bought the new model. Listen to it! It´s wonderful".
I listened to the new model and it didn´t sound that great. It was O.K. but too expensive. Thus, at the same time, two men were thinking: "My amplifier is the best".
Contradiction? I don´t think so. We both had gotten what we had been waiting for and we were happy with our choices. I love my amplifier but I must admit I´ve put too much feelings on it. The result of this "non-blind test" was two happy men, but a well-conducted DBT might have revealed more objective information.

The fact is, there are left brained and right brained people. We're all different. Some of us are more inclined to want objective proof of something, and others like myself are more inclined to just kind of take things on an emotional level. That's not an accusation against either side, but I think a huge reason why these discussions never really get resolves is that we are all wired VERY differently.

 

[Scuzzer]

Unfortunately, most (not all..that would be a strawman again...I'm just going off of the course all these threads seem to take) of the times that DBT/ABX testing is brought up, the discussion starts to lean towards furthering some tin foil hat theory that all amps sound the same

That's a spectacularly difficult thing for an ABX test to prove. It can be used to show that there is a difference but it can't be used to show that there isn't. If 500 people participate in an ABX test and none can discern a difference (based on statistical relevance) it doesn't prove there is no difference. It can be used show that the "night and day" differences between two items are not nearly as clear as the marketing hype says but it can't be used to say that there are no differences.

The rest of your post is similar to my line of thinking. The average individual doesn't need anything more than a "hey, I like that" type thought to justify a purchase. It's when they start spouting the marketing pseudo-science as fact that I tend to get my panties in a wad.

 

[JohnVF]

It's when they start spouting the marketing pseudo-science as fact that I tend to get my panties in a wad.

I hate that stuff, too. But I've decided myself that it's often a bunch of hooey, without blind testing. I also think that any test designed with a motive is inherently flawed from the outset. Ironically...for probably the same reasons that the advocates of blind testing think I can't choose what sounds better unless all influence is removed. I think that anybody setting up a DBT to say that there aren't big differences in, say, cables, is not immune from setting the test up in a way that favors their preferred outcome. Man, did the waters just get muddy there or what? haha. While that might sound hypocritical, what I mean by that is that I believe it's possible for a casual listener to be totally unbiased while looking at two things, and I'm not so sure that its possible for somebody with a vendetta against wrongful marketing practices to be unbiased. Maybe they can be...

I would like to add that one man's night and day difference is anothers very small difference. Things that bother me now in my system are very very minute compared to the problems I had when i started. But the bug me about the same.

 

[Scuzzer]

I think that anybody setting up a DBT to say that there aren't big differences in, say, cables, is not immune from setting the test up in a way that favors their preferred outcome.

If that was their stated goal then I would be skeptical as well. The goal should always be to test for positives not for negatives. Set up the test so that positives have as good a chance as they can possibly have while keeping all the other variables as similar as possible. Cable tests should be one of the easiest ABX tests to set up.

I would like to add that one man's night and day difference is anothers very small difference.

Then they need to come up with a better term for it. Everyone can tell the difference between night and day.

 

[marc mc]

As a matter of fact, they do exist and they do serve as a design tool. I've done DBT in designing speakers. Between our electronics tech and I we rigged a switchinator that allowed us to switch back and forth between 2 speakers that were screened off so we couldn't see what was playing. We grabbed people out of the office as well as us (the lab rats) and let everyone pick what they liked.

Ray Your reading to much into my post. I should have stated "for all practical terms they are rare and virtually none existant.

The designers that I have read and met (personally) did not use them. And I have never met an audio entusiast (thats for you whoaru99 ) that used or cared about any form of blind testing.

My overall point is if a certain segmemt never mentioned blind testing it would never come up and has never been the standard this industry has used for design or use. And this hobby does not have enough legs left for blind testing to ever be accepted.

BTW if I had for what ever reason believed that ABX testing was valid and had intent on introducing more audiophiles to its importance. I would have never gone about it the way the ABX camp now uses. I would have bought the gear and ran test after test and posted the results. The fact that the futterman example is still used is proof of the ABX camps total apathy in "Putting your money were your mouth is.

I wanted some friends to hear the benifits of active systems. So I packed an entire set up into my car and let some friends hear it in their houses. All they had to do was open their front door and say "Hi". I set everything up and left it all their for a couple weeks. The proof is in the pudding guys. I really got a kick out of the looks on their faces when I went back.

marc mc

 

[JohnVF]

Then they need to come up with a better term for it. Everyone can tell the difference between night and day.

Yes, even the difference between Krell driving Magicos and a Sansui G-9000 driving DQ-10s is probably a difference of 'afternoon to dusk' to most people but I've learned to expect exaggeration as part of listening to different gear with audio folks. "That's amazing!" often seems to actually mean "that's a very subtle improvement in the specificity of the imaging."

 

[RayW]

Ray Your reading to much into my post. I should have stated "for all practical terms they are rare and virtually none existant.

I just read what you wrote, but I admit I took off and ran with it.

I wanted some friends to hear the benifits of active systems. So I packed an entire set up into my car and let some friends hear it in their houses. All they had to do was open their front door and say "Hi". I set everything up and left it all their for a couple weeks.

I'll PM you directions to my place.

 

[JohnVF]

Ray, you should have blind folded us when you introduced the Chicago crowd to your mushroom speakers. I can't unsee them! They're burned into my retinas.

I want a yard just so I can use a pair.

 

[sheltie dave]

No, DBTs or ABXs do not serve as design tools. When are you guys going to get it! They don't serve as any tool! They don't exist. You can't find one. Don't even know where to look for one. Oh look there it is! No, thats not one either.

Do you get it? It's all theory because they are not and never have been a part of this hobby. Scream all you want about the reasons why no one does them but that will change nothing. Because no one does them. In almost 30 years I have never seen one, been asked to participate in one, or met another audiophile that did.

You guys have used the same argument forever and yet as I keep pointing out! You guys yatter and yatter and yatter on about these tests but, YOU NEVER GET AROUND TO ACTUALLY SETTING ONE UP. And even if you did it wouldn't change Jack about this hobby. So get over it and do something more productive with your keystrokes.

Audiophiles don't use this form of testing for the same reason that Zoo's don't have Unicorns. They don't exist.



marc mc

Marc, after you visit Ray, you are more than welcome to swing over to St. Louis and we can run another series of blind tests here. We had 25 folks over at the last DBT, and it was a lot of fun...namely because we weren't trying to trick anyone, or embarrass or demean any equipment.

We put some heavyweights into the fray, and let the chips fall where they may.

I enjoyed hosting it, because it allowed me a chance to get feedback from others on what they felt were strengths and weaknesses of the various amps, and it also generated a quick ranking of what a small mob thought of the varied amps. It provided some measure of validation about my opinions of the amps, too.

The real interesting finding was that there was a relatively large difference across the board in presentation, soundstage, sometimes frequency response, and often quick consensus decisions on which amp folks liked in the shootout.

While it may not have been what audiophiles would have done, and not necessarily attended by only audiophiles, the Levinson #27, Audio Research D115, Marantz 8B, Monarchy Audio SM70 Pros, custom Horus SET amps, and T Chip amp certainly are all used by audiophiles somewhere, so I would say there are DBTs that are both scientifically rigorous and competently run in a manner to generate useful comparison data.

 

[whoaru99]

.. the discussion starts to lean towards furthering some tin foil hat theory that all amps sound the same....

This is one I read a fair bit too, but I'm not sure I recall anyone ever actually saying that. Usually, there are tons of stipulations that essentially boil down to amps of similar style and used within the least common denominator.

Perhaps it's splitting hairs and part of the hangup but, to me, many (maybe even most of a general style) amps sounding so similar that few if any people could tell them apart is completely different than saying all amps sound the same. :scratch2:

I've spent a fair bit of coin on amps from Levinson, Audio Research, some pro gear, etc. and I'd like to think there are some differences. But, at times, I'm not entirely sure of that in certain context.

 

[Scuzzer]

This is one I read a fair bit too, but I'm not sure I recall anyone ever actually saying that. Usually, there are tons of stipulations that essentially boil down to amps of similar style and used within the least common denominator.

Yeah in the fine print it will normally mention something about the amps being competently designed and used at a reasonable level. Then everyone gets to discuss what those terms mean and the whole discussion turns to poo. They obviously aren't talking about comparing tube amps to SS units or old unrecapped silver faced Pioneers vs. the latest high zoot offering.

Everything sounds the same is more of a CD player comment than an amp one.

 

[JohnVF]

Yeah in the fine print it will normally mention something about the amps being competently designed and used at a reasonable level. Then everyone gets to discuss what those terms mean and the whole discussion turns to poo. They obviously aren't talking about comparing tube amps to SS units or old unrecapped silver faced Pioneers vs. the latest high zoot offering.

Everything sounds the same is more of a CD player comment than an amp one.

I think this could also be a case, then, of the original intent of the test being lost upon those who bring the test up later on to prove a point. I agree that, in large part, my competently designed gear that is operating according to original spec, doesn't sound completely night and day different. I recently switched out my little Sony Class-A solid state amp for a dynaco stereo 70, and to the person who wasn't as familiar with both of those amps as much as I was, they probably sounded more alike than different. But I also chose them both because I like a particular sound.

I guess it wasn't brought up in this thread...they all seem so similar that they blend from one into another. We'll get there at some point....

take care..

 

[Mr. Lin]

Everything sounds the same is more of a CD player comment than an amp one.

And one that's possibly even more incorrect.

 

[Brett a]

Originally Posted by Scuzzer
Everything sounds the same is more of a CD player comment than an amp one.

And one that's possibly even more incorrect.

I think if all CD players sound the same, Scuzzer, there are other issues with your system.

I've rotated a small stack of CD players through my system (just for yuks) and they all sound different from each other.

 

[Scuzzer]

I think if all CD players sound the same, Scuzzer, there are other issues with your system.

The "they all sound the same" argument is used more often in regards to CD players than to amps or other equipment. Please note that I didn't make the argument nor am I here to defend it. It is simply an observation based on my perusal of objectivist writings.

 

[Mr. Lin]

Please note that I didn't make the argument nor am I here to defend it. It is simply an observation based on my perusal of objectivist writings.

I kind of figured that. You're right too, there's definitely a group of people who actually think that almost all CD players sound the same, and the high-end ones that do sound different are simply colored to make the format more palatable.

 

[whoaru99]

I kind of figured that. You're right too, there's definitely a group of people who actually think that almost all CD players sound the same, and the high-end ones that do sound different are simply colored to make the format more palatable.

I'll state again...."sounding the same" and sounding so similar that few people can tell them apart are two different things. Usually, the argument is in regard to the latter but perverted into the former.

I kind of figured that. You're right too, there's definitely a group of people who actually think that almost all CD players sound the same, and the high-end ones that do sound different are simply colored to make the format more palatable.

I'm not clear on this. Are you saying/implying that high end players are categorically more true to the source?

 

[Brett a]

The "they all sound the same" argument is used more often in regards to CD players than to amps or other equipment. Please note that I didn't make the argument nor am I here to defend it. It is simply an observation based on my perusal of objectivist writings.

Duly noted.

..and it's the first time I've hear the opinion that CD players sound more alike than amps. In my experience reading and talking about it, it's the amp (just the amp, not the pre section) that is regarded to have the least affect on the quality of the final output.

I don't say this to be argumentative in the least, just a little surprise by what you say.