I really, honestly, think it would be a great time. That was quite a murderer's row of amps you had in line there. Nice stuff!
Blind tests
- Thread starter willysan
- Start date
I really, honestly, think it would be a great time. That was quite a murderer's row of amps you had in line there. Nice stuff!
Mr. Lin
Lunatic Member
Yes, I'm sure I could.
No I definitely disagree, especially with the part I highlighted. In fact you could also use the word subtle, which many people would likely assume means 'small,' but the essential differences between different components do not always present themselves in an immediately obvious way. Just because it takes time to grasp the big picture of what a component is about doesn't necessarily make the differences smaller or less significant.
If you're going to do a test where a substantial amount of time is spent listening to one component before switching to the next one, then perhaps that would work, I don't know. I'm not saying blind testing will never work with hi-fi audio, I just think it's value and reliability here are both pretty low. In fact, I usually stay out of these discussions because blind testing is of no use to me since I have no need to prove what I'm hearing scientifically, to myself, or anyone else.
whoaru99
Epic Member
That's cool. One should stand by their convictions.
I'll bet that's not the response you were expecting.
Mr. Lin
Lunatic Member
That's cool. One should stand by their convictions.
I'll bet that's not the response you were expecting.
I've learned to expect the unexpected. :yes:
Off topic, but seriously this Naim CDP is incredible (the one I referred to in an earlier post).
Another thing I have noticed when switching gear around recently, as I've been doing a lot of that, is that I am often reacting to the 'change' in sound instead of the overall picture, and it's that overall sound that takes a bit longer to form an opinion on. The other day I put my Sony TA-e86 preamp in place of my tubed Fisher 400c, and immediately noticed that the soundstage was more precise...instruments were easier to pinpoint, etc. I really liked that for a while. The next day, I started to realize that I just wasn't enjoying the system as a whole as much. I put the Fisher back in...and the accurate timbre portrayal on the Fisher that I like was back. I think what this really means is that I want a preamp that is more precise in imaging than that the Fisher, and more 'real' sounding than the Sony..so the quest continues. I'll, um 'keep my eyes open' for a new preamp (ha ha).
Mr. Lin
Lunatic Member
John, that is exactly what I'm talking about.
For those that want to spend some time learning the truth about duo-trio balanced reference tests (ABX) as it pertains to audio the following are a good read.
"Sensory Evaluation Techniques" book and the "Sensory Testing Methods" manual which is published by the American Society for Testing and Materials. Also read, "Perceptual Audio Evaluation-Theory, Method and Application."
For those that just want to cut to the chase, summed up by a well respected member of the audio community,
"The peer-reviewed scientific literature and textbooks authored by credible, reputable and respected professionals in the sensory science and audio performance evaluation fields indicate that blind tests, ABX tests, duo-trio balanced reference tests or any tests of that ilk are scientifically invalid and totally inappropriate for sound stimuli."
And,
"The ABX test (actually called a duo-trio balanced reference test) is not indicated for the class of sensory stimuli in which audio equipment evaluation falls. In other words, applying ABX testing to audio equipment evaluation is actually within the realm of pseudo-science because it inevitably leads to erroneous results. Therefore, the descriptor "believer" would more appropriately be applied to the naysayer or anti-audiophile since their thought processes are driven by, and based on, data which they "believe" to be right, but which has no verifiable basis in reality or in accurate and correct scientific procedure and theory.
It is ironic that a group of individuals who rail the hardest against "pseudo-science" and who scream the loudest for scientific verification of audiophile claims insist on using a test that is scientifically inappropriate and scientifically invalid for the evaluation of sonic differences in audiophile grade equipment. Even more tragic is the fact that the ABX test is highly error prone even for the sensory phenomena it is appropriate for."
"Sensory Evaluation Techniques" book and the "Sensory Testing Methods" manual which is published by the American Society for Testing and Materials. Also read, "Perceptual Audio Evaluation-Theory, Method and Application."
For those that just want to cut to the chase, summed up by a well respected member of the audio community,
"The peer-reviewed scientific literature and textbooks authored by credible, reputable and respected professionals in the sensory science and audio performance evaluation fields indicate that blind tests, ABX tests, duo-trio balanced reference tests or any tests of that ilk are scientifically invalid and totally inappropriate for sound stimuli."
And,
"The ABX test (actually called a duo-trio balanced reference test) is not indicated for the class of sensory stimuli in which audio equipment evaluation falls. In other words, applying ABX testing to audio equipment evaluation is actually within the realm of pseudo-science because it inevitably leads to erroneous results. Therefore, the descriptor "believer" would more appropriately be applied to the naysayer or anti-audiophile since their thought processes are driven by, and based on, data which they "believe" to be right, but which has no verifiable basis in reality or in accurate and correct scientific procedure and theory.
It is ironic that a group of individuals who rail the hardest against "pseudo-science" and who scream the loudest for scientific verification of audiophile claims insist on using a test that is scientifically inappropriate and scientifically invalid for the evaluation of sonic differences in audiophile grade equipment. Even more tragic is the fact that the ABX test is highly error prone even for the sensory phenomena it is appropriate for."
whoaru99
Epic Member
Got any links to main articles where the bolded information resides? I'd like to read it for myself mainly because it doesn't seem to say what is the end goal of the testing in this context...identifying a difference or making preference decisons as in which one sounds better; a big difference.
Last edited:
cwall99
Addicted Member
I'm with you on this. Give us a source we can check out (just like when you write a research paper). Something a little more concrete than "a well-respected member of the audio community." Obviously, you have some information as you cited the titles from two books (and, I presume, we can check it out).
But I think we're on the cusp of a red herring here. One part of the conversation here is talking about using ABX testing to make scientific determinations about what sounds best while the other is restricting their comments to a more useful application of ABX testing as an audio consumer.
To me, the notion of the double-blind test is to remove as much noise as possible from the signal path in the decision-making process. For instance, if I'm comparing two speakers at a shop. I think it'd be cool to spend some time blindfolded and listening to speakers. I can't see them. They should be positioned as near to each other as possible, and the only thing the sales person is doing is, at my request, flipping the switch between speaker a and speaker b.
I'm trying to sort out what sounds best to me, and I'm trying to remove any sorts of clues or cues that may bias me.
JonL
Lunatic Member
I think the criticism of ABX testing in those quotes really reflects the difficulty in ANY audio comparisons. It's impossible to examine music side-by-side, the way you could photographs. Often, a change in audio gear sounds "better," at least for a few minutes, simply because it's different. For me, I often can't hear differences until I listen for a while, and I frequently can't decide on a preference until I live with a new component for a while - sometimes weeks. It's not clear to me why being "blind" to the change would make the comparison any less valid than being sighted. ABX testing of audio gear may not be perfect but I cannot understand why it would be inferior to sighted tests of similar duration and methodology.
whoaru99
Epic Member
I'm not so sure the main argument is being blind, it seems moreso that stress, etc. can change the situation and outcome.
I'm sure it probably can but tests of all kind are stressful. Should we ignore failure of a Bar exam or medical exam/final because the test was stressful?
Then again, there are many who do great at the exams, and suck as lawyers and doctors.
cwall99
Addicted Member
The point that the one poster was claiming that ABX testing is not necessarily a valid method to use if you want to claim that one product is superior to another. It may be that current methodologies and technologies don't allow for it.
That's not to say that it can't be done, but to do it in a lab or on a showroom floor might require an investment in technology that's cost-prohibitive.
The other issue is that some of what you're trying to do comes down to a particular person's bias. You can, I presume, produce a graphic image of a sound and compare that image three ways: the original sound, reproduced by condition A, reproduced by condition B. The condition whose graph most closely matches the original may be said to have the highest degree of fidelity.
But that doesn't address whether or not it sounds "better."
Think about Iron Chef or Iron Chef America. Basically, when it comes down to picking the winner, you have three judges to impress, and most of the time, it seems as if those three judges have lots of disagreements over what tastes better.
The food may be prepared the best way possible by the most highly skilled chefs available, but if I don't like mushrooms or yellow mustard, I don't like them, and if those are ingredients in your food, I won't like the food either. I don't care if it's the most expensive mustard ever made. I still won't like it if it's yellow.
In short, we all have our own biases and that will make it impossible, really, to prove anything conclusively when it comes to making a universal statement about how something sounds.
But, as I mentioned earlier, if I want to pick which speakers or which speaker wires sound the best to me, it's best if I can remove as much noise from my signal path (metaphorically speaking here) as possible. In short, I want to try and get as many of my biases out of the way as possible.
ABX testing probably works best on an individual level when you're trying to choose a new speaker or if you're trying to determine whether it's worth dropping $13,500 on a pair of speaker wires.
cwall99
Addicted Member
That's more an indication of the validity of the test instrument than it is of the concept of testing.
Or vice-versa.
sheltie dave
Addicted Member
As an instrument that is fair to everyone, the blind test also rankles people who don't like a rigged test.
I like having as much information as possible if I am going to trade, purchase, or sell gear.
Can anyone tell me which amp will sound better in this listening test?
I like having as much information as possible if I am going to trade, purchase, or sell gear.
Can anyone tell me which amp will sound better in this listening test?
marc mc
Super Member
Man talk about a kick in the ass. I would love to see the real article myself. I have said before that I wondered why an test that was designed for drug testing has been usurped for audio.
Anyway I would love to see the article and mybe we can put to bed the debate once and for all. If its not a valid test then we will all just have to live with it.
marc mc
Anyway I would love to see the article and mybe we can put to bed the debate once and for all. If its not a valid test then we will all just have to live with it.
marc mc
Blind testing wasn't invented only for drug testing, it's used anywhere there might be an observer bias effect.
How is a blind test more "rigged" than a sighted one?
marc mc
Super Member
Ok let me say it a different way.
If "The ABX test (actually called a duo-trio balanced reference test) is not indicated for the class of sensory stimuli in which audio equipment evaluation falls.
Then we will have to live with it.
But I think the main point of the claim is if ABX testing has ever had "Board approval" as a test for audio in the first place. If not then thats what is known as a good old fashion kick in the ass. Agreed?
And if ABX testing (for audio gear) is in fact an accepted method by the scientific community (not the guys that posture on forums) then I will personally accept that as the standard and will change my subjective ways. In fact I will make a hunting store my first order of buisness and get me some cam gear. I figure I might as well use the opportunity to look for Bigfoot at the same time because ABX tests are just as rare. In fact I might even take bets on which one I will find first. Personally I think the smart moneys on Bigfoot.
Guys I'm just having fun!
marc mc
Last edited: