Double-blind testing - why are we not using it?

[Holst]

The problem with double blind testing from my stand point is a lack of facilities and staff. Someone would have to set up the test and keep the process secret from me or the double blind thing would be moot.

Otherwise, I'd be game to be a guinea pig.

 

[Dusty Chalk]

Y'all realize, I hope, that this may be the single most civil thread on this topic ever posted on the internet.

Quite probably.

 

[GordonW]

I think, an exchange between Jon Risch and David Clark, way back at the 1991 AES convention sums up some of the problems with ABX testing.

To summarize: an ABX test CAN confirm differences between components. An ABX test CANNOT DETERMINE THAT THERE ARE NO DIFFERENCES between components.

Basically, it's a derivative of the old scientific methodology credo "you can't prove a negative"...

In short, an ABX test can be quite effective at VERIFYING the presence of a difference. However, it's NOT correct to say, if a difference IS NOT DETECTED in an ABX test, that NO difference exists. It may just be that the specific test setup does not have ENOUGH SENSITIVITY or PROPER METHODOLGY (ie, testing procedure, measurement equipment, etc) for the difference to be detected...

To put it even shorter: There's still some things that can be sensed by the human ear-brain system, that we can't reliably quantify in numbers yet. We're getting constantly better at it, but we're not at that point yet, and probably not even really CLOSE yet...

Makes speaker design a lot of fun. Frustrating, hair-pulling, teeth-grinding fun sometimes, but fun nonetheless.

Regards,
Gordon.

 

[Bigerik]

Ah! <revealing light from above> See, I thought the burning questions were about judging the relative accuracy of the reproduction, the quest for the perfect reproduction. Instead, I now realize the whole cables/tweaks/mods thing is about judging the resulting 'relative attractiveness' of the reproduction. DOH!

I'm really not trying to be funny, now I think I understand TOL better now - its sort of like a place where you can talk about whether you like your music with the tone controls centered, or loudness on and EQ switched in. With 16 ga. zip cord, or 10 ga. audiophile cables. Or, 'I think it sounds better because...it just does!'. Its about individuals' subjective response to whatever the tweak/mod does to their listening experience. Thanks guys!

Not really.
Certainly not that easy. TOL is actually a forum where people can discuss the results of tweaks and actually have the opportunity to test various tweaks in their own systems, without fear of being attacked for whatever they are trying to describe or what they have heard. No snobbery hear. VERY unique to TOL is the fact that we have started putting together an incredible set of reviews of various pieces of equipment, tested in all manner of systems. Never have I seen that tried before.
To be honest, I don't know a single forum on AK, nor have I seen one in any of the audio sites that I have been to that is totally devoted to "the quest for the perfect reproduction.". I have heard some really big buck systems and none of them have ever fooled me into believing I am hearing a perfect reproduction of live music. I have hear some great audio through them tho.

 

[CUlater]

:scratch2: Hmmm. If not pursuing more accurate reproduction, then what purpose are 'tweaks' serving?

If they are just about different ways to 'alter' the reproduction, then tweaks fall into the same category as tone controls (albeit, much more subtle and sophisticated)? No way! Judging from the passionate descriptions employed ('more focused', 'an increased sense of space', 'graininess removed', 'more fuzziness', etc.) and level of interest in the things discussed in the TOL forum, I've always taken away a sense that more accurate reproduction is what is being pursued.

IMHO, the Hi-fi hobby has always been about the continuing quest for perfect, accurate reproduction. <"Our five-year mission: to seek out new life and ... better hi-fi! :yes: "> That's why, to demonstrate how close their products were to approaching this nirvana, both RCA and AR pulled marketing 'stunts' (I don't like the connotation of that word, but I don't know another one to use here) where they transitioned from a live orchestra to their speakers in front of an unsuspecting audience of audiophiles. Why engineers pursue vanishingly low distortion figures and ruler flat frequency response in their TOTL products. Where the phrase 'straight line with gain' arose in trying to describe the 'perfect' system. Remember "Is it real, or is it Memorex?" If not accurate reproduction, then what the heck are we audiophiles pursuing? Otherwise, it's paint by numbers, only without the numbers...

Now, I will admit that when achieved, accurate reproduction can be disappointing - hence, tone controls, reverb units, bass synthesizers, spatial expanders, etc. are there to 'tailor' the sound more to personal liking. The audio equivalent of rose colored glasses - not accurate, but pleasing. I've always considered that being 'an audiophile' means that one is trying to attain an as-close-as-possible objective baseline of accurate reproduction, upon which 'modifications' can be made to get the desired subjective experience.

Perhaps a truism regarding our nature can be found in 'The Matrix' when Agent Smith notes that, after the failure of the initial 'perfect Garden of Eden' version of the Matrix, he's concluded that humanity defines reality only through suffering... Perhaps we must 'suffer' for our audio experience to mean anything? :thmbsp:

<unbeckoned, a Saturday Night sketch comes to mind: a horrible club musician states at the outset of his performance: "I've suffered for years for my music, now its your turn...">

This is fun, guys! I truly appreciate the opportunity share my thoughts, and your viewpoints and thoughts... I hope you all are enjoying this as well!

 

[Artie]

:scratch2: Hmmm. If not pursuing more accurate reproduction, then what purpose are 'tweaks' serving?

If they are just about different ways to 'alter' the reproduction, then tweaks fall into the same category as tone controls . . .

One could argue that thats precisiely what "tweaks" are. Consider two people, with the exact same system, but one has heavy drapes and thick carpet and the other has no curtains and hardwood floors. They both may be striving for the same "flat" sound but will adjust their tone controls differently to achieve it.

Now lets consider our components as a "system". (Which we do.) Ideally, one engineer would design the stereo from phono cartridge to speaker coil as a single unit. He'ld have total control on all the electrical aspects and interactions of the selected components. He could make whatever adjustments were needed to get flat response from end to end. But in the hi-fi world that doesn't work. We want to pick and choose our own components. This isn't that far off from picking our own power supply capacitors. The designer has to make his preamp's output impedance match up with another designers amp input impedance. In the real world, they don't often match. Take for example the old Dynaco ST-120 amp. It had an input impedance of around 500k, if I recall correctly, as opposed to a more typical 10k to 100k input impedance.

So, without getting too long-winded here, lets say your preamp/amp combo has an input/output mismatch that causes a slight frequency dip at the point that might be the third harmonic of a cymbol. You've altered the "sound" of that cymbol. Now, lets say you change to a new interconnect, (which has its own LRC values, regardless of how small), that has a "peak" at that same point. You've just "tweaked" the tone by replacing an electrical component in the signal path, rather than turning a knob.

Same goes for speaker cables, which have their own LCR values, or vibration dampners that prevent standing waves. Its all about minute "tone" control.

 

[luvvinvinyl]

Haven't you just defined "system synergy"?

 

[Artie]

I'm not sure. I just meant that "tweaks" are minute tone adjustments.

 

[Lefty]

I'm not sure. I just meant that "tweaks" are minute tone adjustments.

They may be minute but they sure can be expensive. I'll stick with the tone knobs as my tweet outlet :thmbsp:

Lefty

 

[Negotiableterms]

I've written this all before on AK, so if you'll all pardon me for repeating myself:

The problem with any form of human listening tests of audio gear is the inherent limitations of the human, in which two well-documented effects collide:

1. Our brains can only remember what they heard for a very short time.
2. The signal process analysis system in our brains takes quite a long time to work on subtle differences.

So, for gross differences, ABX testing works just fine. For subtle differences, by the time we've figured out what one sound is like, we've forgotten what the other really sounded like. Next thing you know, we guess, with predictably poor results.

In the long run, you can just buy the stuff that makes you happy, and be done with it.

 

[DougMac]

A couple of questions

The designer has to make his preamp's output impedance match up with another designers amp input impedance. In the real world, they don't often match. Take for example the old Dynaco ST-120 amp. It had an input impedance of around 500k, if I recall correctly, as opposed to a more typical 10k to 100k input impedance.

So, without getting too long-winded here, lets say your preamp/amp combo has an input/output mismatch that causes a slight frequency dip at the point that might be the third harmonic of a cymbol. You've altered the "sound" of that cymbol. Now, lets say you change to a new interconnect, (which has its own LRC values, regardless of how small), that has a "peak" at that same point. You've just "tweaked" the tone by replacing an electrical component in the signal path, rather than turning a knob.

To be honest, I've never thought much about matching impedences between my pre-amp and amp. Help me learn how to correct this. I guess the first thing I need to do is find out the impedences of the two. I looked up some specs on some pre-amps and amps and found the output impedence of the former and the input impedence of the latter. If they are not the same, how much difference does there need to be for it to have an audible effect?

Furthermore, using your cymbol example, how to I determine what effect this impedence mismatching is having on my system's EQ? How do I now that it is causing a dip in a third order harmonic and how do I know what frequency range this is occuring?

Let's say I've determined that there is enough of an impedence mismatch to cause an audible change and I'd like to correct this by using an appropriate interconnect. What information regarding the interconnect should I look for to know that I'm choosing one that will give me the desired results? What calculation should I use to determine what my needs are? I'm concerned that if I buy the incorrect interconnect, no matter how much I pay, without the ability to (a) determine my problem and (b) calculate my needs and (c) use this information to obtain the correct interconnect that I may end up exacerbating the problem instead of alleviating it. Please provide the methodology I need.

Same goes for speaker cables, which have their own LCR values, or vibration dampners that prevent standing waves. Its all about minute "tone" control.

I guess I'm showing my ignorance, but I'm not sure what kind of vibration to which you refer and what kind of standing waves you're talking about. Are you talking about the speaker cable vibrating? Are you talking about the room, the speakers or items in the room vibrating? Are you talking about standing waves in the room? Again, if it's audio standing waves in the room, please tell me (a) how to determine them and (b) how to use this information to determine the correct speaker cable and (c) what specifications I need to look for in the cable to address the problem.

Thanks,

Doug

 

[Spasticteapot]

To be honest, I've never thought much about matching impedences between my pre-amp and amp. Help me learn how to correct this. I guess the first thing I need to do is find out the impedences of the two. I looked up some specs on some pre-amps and amps and found the output impedence of the former and the input impedence of the latter. If they are not the same, how much difference does there need to be for it to have an audible effect?

Furthermore, using your cymbol example, how to I determine what effect this impedence mismatching is having on my system's EQ? How do I now that it is causing a dip in a third order harmonic and how do I know what frequency range this is occuring?

Let's say I've determined that there is enough of an impedence mismatch to cause an audible change and I'd like to correct this by using an appropriate interconnect. What information regarding the interconnect should I look for to know that I'm choosing one that will give me the desired results? What calculation should I use to determine what my needs are? I'm concerned that if I buy the incorrect interconnect, no matter how much I pay, without the ability to (a) determine my problem and (b) calculate my needs and (c) use this information to obtain the correct interconnect that I may end up exacerbating the problem instead of alleviating it. Please provide the methodology I need.




I guess I'm showing my ignorance, but I'm not sure what kind of vibration to which you refer and what kind of standing waves you're talking about. Are you talking about the speaker cable vibrating? Are you talking about the room, the speakers or items in the room vibrating? Are you talking about standing waves in the room? Again, if it's audio standing waves in the room, please tell me (a) how to determine them and (b) how to use this information to determine the correct speaker cable and (c) what specifications I need to look for in the cable to address the problem.

Thanks,

Doug

From what I understand, most speaker cables are too small to support standing waves in anything below the megahertz band. We can't hear that high, anyway.

Also, impeadance matching is easy for us solid-state folks. (Especially considering that the amplifier I'm building is essentially just a REALLY big op-amp.)

 

[Holst]

They may be minute but they sure can be expensive. I'll stick with the tone knobs as my tweet outlet

Tweeks can alter more than tone, I stuck a two dollar 50 pond concrete chunk under my turntable and some felt pads for moving furniture under that and it made a nice difference. Vibrations from the speaker stopped muddining up the sound, plus, my daughter can now run up the staits behind my audio rack and I don't hear footfalls.

1. Our brains can only remember what they heard for a very short time.

Sonic memory is something like 15 seconds.

2. The signal process analysis system in our brains takes quite a long time to work on subtle differences.

Unless you are in a room that you’ve never seen, blindfolded and listening to unknown (to you) equipment, you will have more than one sense effecting your opinion. With regard to a turntable base, as I did research I found people who had tried different things said granite had a hard sound, wood sounded warm.

Really... or does the wood just look warmer (especially if it’s on fire).

In the long run, you can just buy the stuff that makes you happy, and be done with it.

Amen.

As I said in my earlier post, I would love to be plunked down in a double blind test, if nothing else, just to try it, and hopefully, learn something. Or unlearn something. I’ve been messing with audio gear since I was four years old, I’m 49 now. I’ve owned gear, listened to gear, sold audio gear, but it took a heck of a long time to start listening to the music instead, at which point a started seeing that my audio gear opinions were overly opinionated and rather limited.

I love TOL and the test programs because I get to fiddle with something myself and form my own opinion. It's a lot of fun. Plus, I can read someone else's opinion on the same thing and maybe see things from a different perspective.

I’m still learning

JJ

 

[surething]

:scratch2: Hmmm. If not pursuing more accurate reproduction, then what purpose are 'tweaks' serving? If they are just about different ways to 'alter' the reproduction, then tweaks fall into the same category as tone controls (albeit, much more subtle and sophisticated)?

As a consumate tweaker for over 25 years, I think I may be qualified to answer that. First, tweaks have nothing to do with tone controls. Tweaks can alter perceived or measurable freq response, but tweaking can alter EVERY possible aspect of a system, just as changing components can (Which itself is another way of altering the reproduction, like tone controls). Tweaks often don't bring about changes as "heavy" as changing components, but they can, particularly if you do heavy tweaking. Why tweak? Because you can change components to your heart (or wallet's) content and still not achieve the right sound for you, and tweaking is a much more direct way of doing that.

As for "accurate reproduction", I don't shoot for that goal per se, because its almost impossible to define. How do you know what's accurate, when you're reproducing music through media? Some say you go by the master tape. Okay.... on WHAT system? Every single component of every single stereo system in the world has its colorations and will lend its sonic signature to the sound. Its meaningless to use the master as the standard, if there's no absolute standard as to what it sounds like.

Some say you judge accuracy by the sound of live music. Absolute nonsense! WHAT live music? Live music WHERE?! Now you're talking about differences in halls, equipment used to play the music, and probably singers and musicicans as well. That's truly comparing apples to oranges.

Some might argue you go by the live music used to make the recording. Okay, now you're a little closer to an absolute standard of accuracy for the recording in question. But how many people can judge your system that were there at the original performance? And dont they all have different ears? If they're using them to judge your stereo, that's hardly an objective standard, is it. Point being, there is no absolute objective standard for accuracy of a music reproduction system. Plus there's no system in the world that can reproduce music as good as live music ("real instruments" and voices at least). So the best you can do is go by whatever -your- standard of accuracy is (which may not be an accurate interpretation of proper music reproduction, but its meaningful for you).