Blind tests

[whoaru99]

Besides, I don't know why someone would want this info anyway.

In a round about way, I agree, inasmuch as I think there are plenty of people with vested interest to avoid DBT/ABX for at least a few reasons.

If a person fails to notice a difference that is there, I say the test has failed.

But isn't that a straw man of sorts, to infer there is a difference in all reported scenarios therefore the test is flawed.

 

[Brett a]

Originally Posted by Brett a
Besides, I don't know why someone would want this info anyway.

In a round about way, I agree, inasmuch as I think there are plenty of people with vested interest to avoid DBT/ABX for at least a few reasons.

Sure, I wouldn't deny that. My comment was admittedly flippent as I just get tired of the senseless fight.

I buy gear based on sound, price, and looks. If all 3 check out for me, It's mine---I'm happy. DBTs abd ABXs mean nothing to this process.

I am very fortunate to live near a high-quality two-channel shop that allows me to take gear home for half a week in the process of deciding. No one is taking advantage of me, which I know is some people's noble concern.

Originally Posted by Brett a
If a person fails to notice a difference that is there, I say the test has failed.

But isn't that a straw man of sorts, to infer there is absolutely a difference.

That's certainly another useful way of looking at it. And points me back to the view that everything is perception. So why not indulge inasmuch as it feels good and is not financially detrimental? (That can be a rhetorical question)
:yes:

 

[willysan]

Sure, I wouldn't deny that. My comment was admittedly flippent as I just get tired of the senseless fight.

I buy gear based on sound, price, and looks. If all 3 check out for me, It's mine---I'm happy. DBTs abd ABXs mean nothing to this process.

I am very fortunate to live near a high-quality two-channel shop that allows me to take gear home for half a week in the process of deciding. No one is taking advantage of me, which I know is some people's noble concern.


That's certainly another useful way of looking at it. And points me back to the view that everything is perception. So why not indulge inasmuch as it feels good and is not financially detrimental? (That can be a rhetorical question)
:yes:

Buying based on sound, price and looks is a most sensible choice for an individual.
However, DBTs or ABXs serve as design tools. If a large number of people can spot which equipment is A or B, then you know you have a comparison tool. I don't mean you have succeeded, the new gear may be worse than the previous one, but you can be sure the difference has been recognized. In my opinion, improving a design calls for one of these tests. Otherwise, the resultas become fuzzy, merged with subjective judgement.

 

[sheltie dave]

We ran a series of double blind tests with amps a few weeks back, and it drove home a few points, namely -

1) The testing is only valid if the preamplifier and speakers are of sufficient resolution and low distortion to allow the amplifier's "sound signature" to present itself properly.

2) Once you reach that point of clarity, it is relatively easy to distinguish basic qualities and attributes in only ten to fifteen minutes of listening.

3) Snapshot listening spans like this do not adequately allow time to discern all about an amplifier, but they do allow folks to choose an amp they can readily listen to for extended periods of time with little thought.

http://www.audiokarma.org/forums/showthread.php?t=283164

 

[ev13wt]

now you presume to be "training" the people here? :scratch2:

Of course not individual members. Hmmm. I get the point. I think I am mistaking members here for my general perception of the "audio masses", and will control myself.

 

[marc mc]

Buying based on sound, price and looks is a most sensible choice for an individual.
However, DBTs or ABXs serve as design tools. If a large number of people can spot which equipment is A or B, then you know you have a comparison tool. I don't mean you have succeeded, the new gear may be worse than the previous one, but you can be sure the difference has been recognized. In my opinion, improving a design calls for one of these tests. Otherwise, the resultas become fuzzy, merged with subjective judgement.

No, DBTs or ABXs do not serve as design tools. When are you guys going to get it! They don't serve as any tool! They don't exist. You can't find one. Don't even know where to look for one. Oh look there it is! No, thats not one either.

Do you get it? It's all theory because they are not and never have been a part of this hobby. Scream all you want about the reasons why no one does them but that will change nothing. Because no one does them. In almost 30 years I have never seen one, been asked to participate in one, or met another audiophile that did.

You guys have used the same argument forever and yet as I keep pointing out! You guys yatter and yatter and yatter on about these tests but, YOU NEVER GET AROUND TO ACTUALLY SETTING ONE UP. And even if you did it wouldn't change Jack about this hobby. So get over it and do something more productive with your keystrokes.

Audiophiles don't use this form of testing for the same reason that Zoo's don't have Unicorns. They don't exist.



marc mc

 

[dpastern]

Well, I'll step into this discussion, hopefully bringing some logic to the thread.

1) people's eyes vary from person to person. That's pretty much a given. Some people are shortsighted, others long sighted, others colour blind. Variation in eyesight is a well established fact. Does everyone agree on this?

2) If 1) is a yes, then isn't it logical to believe that other organs (hearing, smell, touch) vary from person to person? If this is the case...

3) then it isn't logical (and reasonably expected), that different people will hear different things? I think so. Just because person A doesn't hear a difference, doesn't mean that person B doesn't.

Now, I've only considered the physical aspects from a logical point of view. Does the brain play tricks on our senses? Of course it does. One of the most well known scenarios of the brain playing tricks on our senses, is on our eyes, and is optical illusions. Anyone care to argue with this logic? If optical illusions can fool our eyes, then it is logical that such brain interprations can also affect the other senses, hearing included.

The difficult part is working out how much of what we hear is dictated by physical abilities, and how much is the brain influencing what we hear.

Personally, I've got more to do in my life than countless blind tests, I'd rather just listen to the music. If my foot taps, and my head nods, then the stereo combination is doing something right imho. Is it imperative to milk the absolute last bit of detail out of a recording? Personally, I don't think so. It's overkill. Who cares what that chair scraping in the back of the recording studio sounds like? I certainly don't. Is it of utter importance to the enjoyment of music? Not to my ears.

Dave

 

[whoaru99]

They don't exist.

marc mc

Actually they do.

Here's a machine that was comercially developed specifically for it. ABX Comparator Wish they were still available. I think it would be a neat gadget to play with.

A fellow by the name of Steve Herrala, who owns a "stereo shop" Sound Video in Anoka, MN has (or at least had one).

And, I fully agree, that most "audiophiles" (your word, not mine) shy away from anything like this. Usually, but not always, people are smart enough to avoid situations they know are a recipe for failure - especially in front of the jury of peers. Still doesn't discount the fact that I hear all the time "just use your ears". Well, blind testing is the epitome of "just using your ears", yet it's avoided like the plague. As I've said before; curious, is it not?

I've seen this discussion just as many times as you, most likely, and at the end of the day, it's still about a 50/50 split so neither camp really as a minority or majority. Nor is it, IMO, about being right. It's just one of those things that "is" if you know what I mean.

 

[CUlater]

IMHO, aural 'accuracy' is a knee curve that asymptotically approaches, but never reaches, perfection. At the low end of the 'equipment quality' scale, its easy to hear gross level inaccuracies and differences, but the curve eventually hits the 'knee' and starts leveling out - the differences between components become much harder to hear and identify, and the incremental improvement in accuracy become miniscule. This is where the incremental $/improvement ratio starts going through the roof.

So, as the audible improvements get harder to hear and identify it is my heartfelt opinion that it becomes easier to start being swayed by non-auditory aspects - such as the look, the ergonomics, the cost, the prestige, the peer recommendations, etc., etc., ad nauseum.

Hey, I'm not afraid to admit that a large part of my enjoyment of this hobby is non-auditory: the look, the feel, the nostalgia - all play a part in what I select and use. Doesn't mean that the sound isn't important, or that I can't hear differences. It just means that I recognize that given equal, or nearly equal auditory choices, I have secondary assessment factors that start to increasingly weigh in.

Double blind testing takes away all the secondary factors that, for many, are also important factors in their overall satisfaction and enjoyment. But, being of the scientific/engineering persuasion, I think double blind testing if administered properly is the only way to consistently and accurately separate and sort out the ultimate and incremental audible differences between components.

That said, I would think double blind testing should be fine and agreeable to those who claim that they are 'only' about the accuracy of the sound, all other factors be damned. And yet, IMHO, its those very folk that tend to be the most opposed to it; that has always struck me as exceedingly odd. :scratch2:

Ah, well, all is good and well, as each of us ultimately has to scratch our inner itch in whatever way is most personally satisfying.

Hope you all are having as much fun following along on these types of discussions as I am!

 

[JohnVF]

I did some 'sighted' testing last night in my place with a friend. A gear shootout, just for kicks. Frankly, the differences were so glaringly obvious between everything we tried that I just don't see the point in what I'm gaining by not seeing the choices. If I'd been swayed by anything, I'd have wanted the speakers I just bought to 'win' (they came in 4th out of 4 ) and the tube amp I put $1200 into to sound better than the one I'd payed $400 for. It didn't. Sounds were discussed, my friend and I both heard the same things every time, though he liked the midrange better in the speakers that finished last than I did. His ears are also 30 years older than mine, so I'd expected some differences in preference. Frankly, more than there were.

So what would have changed had I not seen them? According to the advocates of blind testing, I could only have come up with results opposite of what I did, because the human mind is supposedly incapable of not caring who wins a fight.

 

[JonL]

So what would have changed had I not seen them? According to the advocates of blind testing, I could only have come up with results opposite of what I did, because the human mind is supposedly incapable of not caring who wins a fight.

I'm not so sure... the mind is a funny thing and you may not fully know what subconscious motivations you might have that predispose your preferences. I'm not doubting that there were obvious differences in this particular exercise, but I don't think this is a good example to prove that blind testing isn't ever valid.

Please understand that I don't think that blind testing of audio components is an easy thing to do. I don't think it can be done in the usually suggested manner, sitting and listening for a few minutes to "A" and then switching to "B," or randomly switching between the two. I know that I personally need a lot of time listening to something to decide if I like it more or less than what I had before. It takes a lot of listening under various conditions for me to judge if I enjoy the music more or less. The smaller the change, the longer it takes, though I have made some changes that were immediately obvious.

 

[ehoove]

That is all very subjective. Girls are really smart and will also tell you that you have a big one. (a joke dude, try to follow along here xD) Its not like she is stupid and doesn't know what your into... If she says she can't hear it, you will talk for one hour about stuff she doesn't really care about. If she tells you that there is a difference, ya never know if she is actually hearing something, or saying it first so you don't talk for an hour, but smile big and kiss her and then you listen to some music together, etc etc etc.

Not saying there is no hearable difference in your cables! But all your statement states as "scientific fact" CAN all be based on mind processes. This means your sound difference could be based on mind processes as well.

I don't want to fight, my aim is only to train you for studio ears. Not golden or wooden ones. Armed with knowledge about your own minds "hearing" and your ears, schooled through the hated blind tests to perfection, you are actually a mile above a bunch of people.

Listen to some active studio monitors. I suggest Geithain 901k and 922k. They might just blow you away, correctly positioned, in a proper room and measured in. Fairly priced studio perfection.

After 40 years in the audio game, I think I'll choose my own trainer should I feel the need!:scratch2:
Regards,
Jim

 

[JohnVF]

I'm not so sure... the mind is a funny thing and you may not fully know what subconscious motivations you might have that predispose your preferences. I'm not doubting that there were obvious differences in this particular exercise, but I don't think this is a good example to prove that blind testing isn't ever valid.

Please understand that I don't think that blind testing of audio components is an easy thing to do. I don't think it can be done in the usually suggested manner, sitting and listening for a few minutes to "A" and then switching to "B," or randomly switching between the two. I know that I personally need a lot of time listening to something to decide if I like it more or less than what I had before. It takes a lot of listening under various conditions for me to judge if I enjoy the music more or less. The smaller the change, the longer it takes, though I have made some changes that were immediately obvious.

I agree with almost everything you've said here, especially the idea that the longer amount of time you spend with something, the more the true differences come to light. What I was doing last night was just for fun, and wasn't really meant to be a sighted test..it was just two guys eating pizza and talking about the differences in the gear I had just unpacked from a move. The fact that the Dynaco had been running on and off for a week and the Fisher was cold for 6 months surely played into our assessment (though we let it warm up for 45 minutes). I think the very casual nature of what we were doing helped us to have a pretty unbiased view of the gear.

I guess I just generally disagree with the idea that the 'general' differences in audio gear can't be decided upon just listening to it. I'm not as firm a believer in the subconscious altering my perception of something that is so much a part of the physical world as sound is, at least to the point of not being able to make a gut, quick assessment. Like you, I need to spend months with something to truly understand if I like it or not.

I've heard people say that hearing-memory is short. That doesn't gel with my experience. My high school girlfriend called me once, 15 years out of the blue, and I knew the moment she opened her mouth who it was. Granted, that was more of a timbre-memory but it is still sound.

But besides all of this, and I've said this many times before..I just don't care about blind tests. I'm not trying to prove anything to myself. I simply don't care one ounce about whether or not I'm making the most accurate call when I choose one thing over another. That's just how I live my life, I really don't stress or dwell or worry that much about the little things. And for some reason, a lot of gear-heads just can't fathom that. Yet, in all honestly, my system has sounded, to me, much better by a long shot than any I've heard set up by somebody obsessed with scientific measurement. (But..I suppose they'd want me to blind test that result in hopes of getting a fair shake ).

 

[RayW]

No, DBTs or ABXs do not serve as design tools. When are you guys going to get it! They don't serve as any tool! They don't exist. You can't find one. Don't even know where to look for one. Oh look there it is! No, thats not one either.

As a matter of fact, they do exist and they do serve as a design tool. I've done DBT in designing speakers. Between our electronics tech and I we rigged a switchinator that allowed us to switch back and forth between 2 speakers that were screened off so we couldn't see what was playing. We grabbed people out of the office as well as us (the lab rats) and let everyone pick what they liked.

No, we didn't listen for months as it wasn't practical. The listening environment was far from ideal and most of the listeners were not trained for it. However, the fact that nobody knew what was playing makes it a DBT. We used "generic" people in a generic setting because that's where the speakers were going to end up.

 

[whoaru99]

So what would have changed had I not seen them? According to the advocates of blind testing, I could only have come up with results opposite of what I did, because the human mind is supposedly incapable of not caring who wins a fight.

Another straw man arugment, of sorts.

 

[JohnVF]

Another straw man arugment, of sorts.

I fully believe that's what some of these blind testing fanatics think is going on. They think we're all completely incapable of deciding whether or not we enjoy listening to one thing over another if we know anything at all about it.

What happens later on when I open my eyes or the curtain is pulled back or box lifted off? It's not like I've never changed my mind on a piece of gear, so we're back to square one. So do I have to always keep the piece of gear I like covered? "Don't open that box I might become biased! Nooooooooooo!"

Humans are weird. How can we all be wired so differently? I guess that's part of the fun.

 

[marqueemoon]

I think I'd be most interested in doing some blind testing of two pieces of completely unfamiliar gear. I feel like the sound of my own stuff (for better or worse) is pretty well burned into my brain (but I could be wrong about that too).

 

[whoaru99]

I've been guilty in presenting "extremist" or straw man arguments on this topic too, generally from the advocate camp (as if that's not clear ). Moreso in the past I'd like to think but, anyway, IMO, it's that sort of thing that truly prevents both camps from coming to common ground on this topic.

 

[JohnVF]

I've been guilty in presenting "extremist" or straw man arguments on this topic too, generally from the advocate camp (as if that's not clear ). Moreso in the past I'd like to think but, anyway, IMO, it's that sort of thing that truly prevents both camps from coming to common ground on this topic.

I agree, and I don't honestly mean to be extreme. There's probably more middle ground than either camp realizes...for example, in another thread I said I'd actually enjoy a month-long blind test in my own home with my own gear and a 'mystery' amp or two. And I have read of some very competent speakers designers (in addition to the very competent RayW) using blind tests to check their upgrades to crossovers, etc. But that is a lab, where I expect a lot of things to go on that I just don't deem as being necessary to me, in my home. At that point, I'd expect due diligence to have been done to the design, and my job as just being able to enjoy or not enjoy. But I'm not a DIY guy and nor do I really care about somethings' cost one way or another. I'll happily listen to a $2 amp if it sounds good to me, and most of my friends probably wouldn't be able to tell the difference enough to be impressed, or not, anyway.

 

[Scuzzer]

I fully believe that's what some of these blind testing fanatics think is going on.

You're all strawman all the time. Instead of worrying about (or painting a caricature of) what the other side thinks perhaps it would be better to stick with what you know and talk about why you like to do it your way. Most of the problems with threads like these tend to crop up when one camp portrays the other as morons seemingly focused on only one aspect of a complex issue.

I agree with you that you don't have to prove anything to anyone and you can like whichever equipment you like. Trading anecdotes about the pros and cons of audio products is the bread and butter of a forum like AK. It would turn into a boring but factually relevant repository like Hydrogen Audio if everyone had to prove every claim in order to have an opinion.

DBT/ABX are useful when someone needs to prove that something sounds different. Proof is something that is sorely lacking in the marketing of audio products. Proof is exceptionally lacking on some of the more outlandish audio products, especially the ones that try to use pseudo science to convince the undereducated that there's something real behind the curtain.