Blind tests

[Mr. Lin]

I'll state again...."sounding the same" and sounding so similar that few people can tell them apart are two different things. Usually, the argument is in regard to the latter but perverted into the former.

Either one is flat out wrong, doesn't make a difference to me how it's said.

I'm not clear on this. Are you saying/implying that high end players are categorically more true to the source?

They certainly should be, I never said they always are.

 

[whoaru99]

Either one is flat out wrong, doesn't make a difference to me how it's said.

So you have no problem with a blind test since the audible differences are clear?

They certainly should be, I never said they always are.

One should hope, but I don't necessarily believe it...mainly just wanted to be sure I understood what you were really saying.

 

[Mr. Lin]

So you have no problem with a blind test since the audible differences are clear?

No, I do have a problem with blind testing when it comes to audio, I think it's a seriously flawed way to determine much of anything useful. The differences between CD players (IME), while quite evident, are heard over a period of time, in a relaxed and familiar setting, with your own system and music. Switching quickly between CD players disrupts the way we naturally listen to and process music and sound - and that goes for any other kind of source, not just CD players.

But I have to mention something: I just got a new CD player today, and within about 15 seconds of listening to the first CD I could hear some things it does differently than my other three CD players. In most cases these differences are a little less immediate, though not at all insignificant.

 

[whoaru99]

But I have to mention something: I just got a new CD player today, and within about 15 seconds of listening to the first CD I could hear some things it does differently than my other three CD players. In most cases these differences are a little less immediate, though not at all insignificant.

I probably should leave well enough alone, but since you indicate the difference is clear, do you think you could tell them apart in a blind test with 90% or better confidence, in your system?

It's really sort of a yes or no question. I personally don't care what the answer is, but it strikes to the whole point of the topic, I think. If we listen with our ears, what does it matter if we can't see the equipment? The sound is the same either way, yes, therefore the differences should be just the same and just as easy to pick out?

 

[Scuzzer]

The differences between CD players (IME), while quite evident, are heard over a period of time

The second part directly contradicts the first part. If something is quite evident then it is plainly obvious and doesn't require a period of time to discern. Other folks might use the word subtle to describe something that needs a long period of time to hear. Perhaps we're dealing with the semantics issue that JohnVF alluded to a few posts ago.

BTW, there's no rule in DBT or ABX that says you have only a short period of time to listen to each one. You could ABX cables over a period of months if you wanted to. Obviously it would be a bit more tricky to do that with a CD player but it's still possible to use fairly long time frames.

 

[Sam Cogley]

2) Once you reach that point of clarity, it is relatively easy to distinguish basic qualities and attributes in only ten to fifteen minutes of listening.

I was quite amazed at how quickly I was able to pick out the tube and solid state amps in your test. I think the longest time I spent wondering which was which was about 30 seconds.

I'm not so sure... the mind is a funny thing and you may not fully know what subconscious motivations you might have that predispose your preferences. I'm not doubting that there were obvious differences in this particular exercise, but I don't think this is a good example to prove that blind testing isn't ever valid.

I fully believe that's what some of these blind testing fanatics think is going on. They think we're all completely incapable of deciding whether or not we enjoy listening to one thing over another if we know anything at all about it.

What happens later on when I open my eyes or the curtain is pulled back or box lifted off? It's not like I've never changed my mind on a piece of gear, so we're back to square one. So do I have to always keep the piece of gear I like covered? "Don't open that box I might become biased! Nooooooooooo!"

Humans are weird. How can we all be wired so differently? I guess that's part of the fun.

The human brain is a really weird device, and there has been all sorts of research done on perception biases (of which I'm sure you're aware, John). A thorough discussion of the subject would be interesting but would most likely devour all of the board's bandwidth. And storage space.

 

[Brett a]

It's really sort of a yes or no question. I personally don't care what the answer is, but it strikes to the whole point of the topic, I think. If we listen with our ears, what does it matter if we can't see the equipment? The sound is the same either way, yes, therefore the differences should be just the same and just as easy to pick out?

I'm increasingly comfortable with admitting that this is a multi-sensual hobby. I'm pretty sure it involves all of them except taste and smell (although it might for some of you here ).

For me, sight runs a close second to hearing. I really do care what my gear looks like and how nicely it's set up. I would enjoy my system much less if it was behind a curtain. That said, I did not like the looks of my Shanling amp when I auditioned it, but it sounded so good compared to the other 4 amps I tried, I decided to buy it any way. Since then, I've grown to really love its looks.

Also, there's no debate that the skin as a sense organ is involved in "listening"; or sensing air movement. ---so lets not leave that out. Touch is also important; how many times have you heard someone comment about how much a piece weighs or how it feels when you use it ("It's built like a tank").

I also consider the mind to be a sense; thought formations, thinking, planning, visualizing. We all do a lot of that in this hobby, I'm sure.

So what I'm saying is: in our efforts to determine the qualities of sound in a unit, let's not forget these other important senses. They do deserve a place at the table IMO.--they are not simply distractions.

 

[whoaru99]

No debate that all those things can influence our ultimate choice. Still, on the basis of "A" sounds different than "B", sight shouldn't play a factor to that end, IMO.

 

[Brett a]

No debate that all those things can influence our ultimate choice. Still, on the basis of "A" sounds different than "B", sight shouldn't play a factor to that end, IMO.

Yeh, I get that.

I was simply putting in a plug for the validity of the other senses when evaluating, or even placing $$ value on a piece of gear. I've even seen a few Mac owners sheepishly admit they were in it mostly for the blue lights!.

Blind tests can be an important exercise to isolate the specific experience of of sound, but i would never make a decision based on blind tests alone (as I think some people imply you should).

 

[Sam Cogley]

No debate that all those things can influence our ultimate choice. Still, on the basis of "A" sounds different than "B", sight shouldn't play a factor to that end, IMO.

You're misstating the concept. Sight itself isn't the issue, it's the knowledge attached to the visual cues. If you know that amplifier "A" cost "X" dollars, and that amplifier "B" has a better reputation and cost more than "A," there will often (nearly always) be a bias based on the assumed superiority.

 

[whoaru99]

Yes, not quite a fully accurate description, but I assumed everyone knows the implications of what I said. I guess it's always good to be as clear and correct as possible though.

 

[JohnVF]

You're misstating the concept. Sight itself isn't the issue, it's the knowledge attached to the visual cues. If you know that amplifier "A" cost "X" dollars, and that amplifier "B" has a better reputation and cost more than "A," there will often (nearly always) be a bias based on the assumed superiority.

This is exactly what I disagree with, at least for me. I think the effect of this would vary from person to person. But for myself, I honestly don't care what anything costs, it doesn't make me feel better or worse to have valuable stereo gear...if it did, I'd just buy it. Also...there have been numerous times where I've heard something or tried it out and have been surprised how good it sounded, given the manufacturer. Just recently, I really thought my Fisher 300 would wipe the floor with my 'cheaper' Dynaco ST-70, and it turned out to be the other way around and my eyes were open the whole time. My ears said 'no the dynaco sounds a lot better'. But according to the whole blind test theory, I should have been swayed the other way. And it was not a 'night and day' difference...it was actually way more subtle, but still noticeable.

I see these blind test ideas thrown around by a lot of the same people (not you, Sam) that are really hung up on the cost of things. I'm of the opinion that THEY are the ones who judge things based on cost, not the people who they say need to do the blind testing.

 

[Brett a]

... for myself, I honestly don't care what anything costs, it doesn't make me feel better or worse to have valuable stereo gear...if it did, I'd just buy it. Also...there have been numerous times where I've heard something or tried it out and have been surprised how good it sounded, given the manufacturer.

This is an excellent point that i see getting left out of the conversation most of the time.

When considering what integrated to buy, i took home and tried 5 different models priced between an $850 Cambridge and a $2,400 Krell. The Krell had the least desirable sound out of all of them.

I buy the best sounding gear I can afford and some of my choices have come in quite under my budgeted amount.

I'm not saying I don't have biases. a big one I know I have is based on personal experience after years (decades) of running "vintage" gear (silver-faced Yamaha, AR speakers, etc).

I find new/modern gear to be worth the extra two weeks pay per piece simply based on its sound quality. I don't admit this much around here, because i don't want to offend, but I have a bias that whenever i see something involving a wood cabinet, i think it's going to lack soundstage and headroom, because all the "vintage" gear i have ever encountered had these characteristics. I still like to hook vintage gear up in my system from time to time, just for fun. The old Sony STR-7045 I have does better than i thought it would, but still not in the same league as my main amp.

I'd honestly love to have a different experience with old gear someday. Maybe a blind test will be the place.

 

[JonL]

I think it is possible to have a bias towards the cheaper and more unassuming gear. I can easily see someone feeling that their own senses are superior to any marketing, pricing, and appearance influences, and thus be subconsciously predisposed towards choosing the underdog. It as if the subconscious might want to say, "See, I told you I don't need a blind test... I chose the cheaper ugly duckling after all. I'm immune to the influences of cost and appearance." I'm not saying this necessarily applies to anyone posting here, but it certainly is a possibility. That's what I was getting at when I wrote:

"I'm not so sure... the mind is a funny thing and you may not fully know what subconscious motivations you might have that predispose your preferences."

in an earlier post.

 

[benb]

Blind tests are important because people always have biases, conscious and unconscious. Sure, some people might not need to do tests blind, but a blind test GUARANTEES there's no bias from anything except the sound.

I was on a BBS many years (a couple of decades!) ago which had an audio forum, and someone there described testing speakers. A high quality recording is made either in an anechoic chamber or an open field of a guitar (presumably with the mic maybe eight feet away, to record the sound it develops at that distance). A chord being strummed and let ring out can be adequate.

This recording is played back through the speaker (only one, not a pair) in question. The same guitar is also present (strummed by the same person as in the recording, at the same height as the speaker, and both about eight feet from the listener), and the same chord is played. The idea of course is to tweak the speaker so the recording sounds as close as possible to the sound of the live guitar.

But the ultimate test is to play the guitar strum and the recording of the guitar strum through the speaker in a blind test, one after the other, telling the listeners one is live and one is recorded, and asking which is which. A correct answer rate of 50 percent means the speaker (and the whole recording/reproduction chain) is indistinguishable from the source (for those listeners). A higher percentage of correct answers indicates lack of perfection, roughly to the extent reflected in the percentage of correct answers.

My point here is this example cannot be done except as a blind test. It's impossible to honestly answer "which one sounds most like a guitar played live" when you can see whether or not the guitar is being strummed.

 

[Sam Cogley]

This is exactly what I disagree with, at least for me. I think the effect of this would vary from person to person. But for myself, I honestly don't care what anything costs, it doesn't make me feel better or worse to have valuable stereo gear...if it did, I'd just buy it.

The problem isn't conscious biases, it's unconscious biases. The nagging, back of the brain, "this cost more, shouldn't it sound better than that cheap stuff?" sorts of biases. Basic marketing stuff, really.

What I learned from blind testing at Dave's:
1) it's extremely easy for me to pick out a solid state amp.
2) I really like tubes.
3) the Marantz 8B is an incredible amplifier.
4) battradio can build amps that would make Saul Marantz stop and perk up his ears.

 

[JohnVF]

So with the Dynaco/Fisher comparison I mentioned...it happened last week, and a friend of mine was over. I hadn't yet unpacked all my gear from a move and my friend was loving my system. I said "well...I have an even better amp in a box in the closet!". So I unpacked the Fisher, hooked it up, and let it warm up for about 1/2 an hour. Then we sat down for some serious listening. And our excitement just deflated like a balloon with a slow leak. It just wasn't the same, it lacked the engaging sound of the Dynaco, and we stopped paying attention to the music and just started talking about other things. Finally, I said "it doesn't sound nearly as good, does it?" and he completely agreed, though he admitted to still wishing it was his.

So what would have changed in this scenario had it been a blind test? The Fisher SA-300b was:

1. More expensive. I have actually 3 times the monetary investment in this amp as the Dynaco, and I also told my friend that they sell for quite a bit of money when asked about its price.
2. Bigger and cooler looking.
3. Had a better reputation.
4. Was talked up by me as being superior.

Also, neither was rebuilt by me, and I actually know more about the (great) skill of the guy that rebuilt the Fisher than I do about the guy that rebuilt the Dynaco.

So every single thing that I can think of should have biased us towards the Fisher. But it was removed after another 1/2 an hour or 45 minutes of listening, and we happily went back to the Dynaco. It instantly got us engaged in the music again.

 

[JohnVF]

The problem isn't conscious biases, it's unconscious biases. The nagging, back of the brain, "this cost more, shouldn't it sound better than that cheap stuff?" sorts of biases. Basic marketing stuff, really.

What I learned from blind testing at Dave's:
1) it's extremely easy for me to pick out a solid state amp.
2) I really like tubes.
3) the Marantz 8B is an incredible amplifier.
4) battradio can build amps that would make Saul Marantz stop and perk up his ears.

And all I'm saying is that many people, possibly including yourself, could have come to these same conclusions without it being a blind test. I think your test was really interesting, and personally would have loved to have heard all of these things blind just for fun. I just don't think that it's necessary for everybody to be able to choose the gear they like with a blind test, however fun it might be (and I'm sure it was a ton of fun...quite literally, too, as all those amps had to be heavy!).

 

[Sam Cogley]

And all I'm saying is that many people, possibly including yourself, could have come to these same conclusions without it being a blind test. I think your test was really interesting, and personally would have loved to have heard all of these things blind just for fun. I just don't think that it's necessary for everybody to be able to choose the gear they like with a blind test, however fun it might be (and I'm sure it was a ton of fun...quite literally, too, as all those amps had to be heavy!).

That's quite possible, although it was very interesting to just listen to what was coming out of the speakers without thinking about anything else.

 

[whoaru99]

When I really, really "listen" to the sound, I often close my eyes and try to relax and hear into the details, FWIW.