• Check out the Annual AK Secret Santa exchange Program, signup before November 23rd in the AK Charities & Christmas forum!

Share Your Double Blind Testing Experineces Here.....

M

Mister Pig

Pigamus Maximus
Ak is a big big place. There are many supporters of DBT testing here, so it would seem that there should be plenty of stories to be told about this subject. Why not gather them in one thread for all to enjoy?

So for all AK'ers who have participated in DBT testing. Please let us know the particulars! Can you share the methodology, the gear being tested, and how you participated?

Also if you used DBT testing to select your personal stereo gear, can you share those stories?

I look forward to reading all the great posts, and how things worked out.

Regards
Mister Pig
 
Register to hide this ad
Some years ago I was in a speaker building club and at a meeting they did DBTs of speaker wires with an ABX device. No one was able to identify differences in the wires tested, none of which I remember.

Of course the fly in this ointment could be (could) that those predisposed to not tell a difference might be as liable to not identify differences that actually exist as those who are predisposed to tell differences would be to identify differences that don't. I think DBTs would be more able to filter the later group than the former.
 
Tom Brennan said:
Of course the fly in this ointment could be (could) that those predisposed to not tell a difference might be as liable to not identify differences that actually exist as those who are predisposed to tell differences would be to identify differences that don't. I think DBTs would be more able to filter the later group than the former.
Click to expand...

Great post, Tom. You just nailed the issue. The people who believe in dbts are the ones who are likely trying to disprove a difference. The others will just sit down and listen and make up their own minds.
 
I did a quasi-double-blind testing (many times actually) between my JBl 4430s and quasi-L300s.

I simply used a center off DPDT switch to switch between them. (Just flip them off then flip it one way or the other without looking so you really didn't know which were playing. The sensitivities were so close that you could not use that to judge.

The 4430s have a better sound stage and image better, but are a bit rough at the top of their range and tubby in the upper bass. The L300s are smoother with a better high end, but don't have that pinpoint imaging of the 4430s nor is the soundstage as nice.
 
I used the ABX plug-in in Foobar2000 a while ago to see if I could tell the difference between 320kbps MP3 files and the FLAC files I created the MP3 files from. With a sample size of 50 plays, I could not tell the difference on my 4311s running from my Luxman rig, fed by the hacky RCA output of my Xonar DX sound card.
 
Bigerik said:
Great post, Tom. You just nailed the issue. The people who believe in dbts are the ones who are likely trying to disprove a difference. The others will just sit down and listen and make up their own minds.
Click to expand...

This really isn't a thread designed to take the path of DBT is it right or wrong. Certainly members have their opinion, even those who participate in it. But I don't want the thread to derail into the tired old argument.

What I am interested in seeing is how many people have participated in a true DBT, and how it was put together. It sounds like Tom's was a true DBT, although I personally don't think Toddalin's example meets the criteria set up for one.

Personally I have never participated in a true DBT. I have been in a good single blind test on power cords. I discovered that I liked the cheapest cord from Audio Magic, and I did buy and use them for several years. But it was not a true DBT.

Regards
Mister Pig
 
I am (genuinely) curious - do folks who don't "believe in" double blind testing in the context of hifi also reject the use of double-blind testing in sociology, psychology, and the pharmaceutical sciences? If not, why not?
 
I believe in DDT (Double Donut Testing), myself. What this involves is two "radically" different donuts--one in each hand. First you listen to A and eat some from your right hand (after being blindfolded, of course!), then you listen to B and eat from the left, then repeat in reverse; listen to A and eat from the left, now B and eat from the right. By the time you're done, at the very least, you'll need some coffee in order to decide. Do you think anyone "doubts" the donuts are "different", and that responses might be affected by this dislike? Yeah, it's been that kind of day!

And yes, we do use this kind of control in the social sciences in response to Mark's question above.

cnh2
 
Last edited:
Tom Brennan said:
Some years ago I was in a speaker building club and at a meeting they did DBTs of speaker wires with an ABX device. No one was able to identify differences in the wires tested, none of which I remember.

Of course the fly in this ointment could be (could) that those predisposed to not tell a difference might be as liable to not identify differences that actually exist as those who are predisposed to tell differences would be to identify differences that don't. I think DBTs would be more able to filter the later group than the former.
Click to expand...

Was it an actual DBT though? Did the person throwing the ABX switch know, at any given time, what the switch was connected to?

One of the biggest problems with the audio world's use of the term "double blind" is that it usually gets incorrectly applied.
 
Old_School said:
Was it an actual DBT though? Did the person throwing the ABX switch know, at any given time, what the switch was connected to?

One of the biggest problems with the audio world's use of the term "double blind" is that it usually gets incorrectly applied.
Click to expand...

If the switcher knows, it's single-blind.

Donuts. I often feel that I missed my true calling in law enforcement... as I love donuts.
 
I just did this:

ABXMP3V0VSFLAC2.png


I really didn't want that result. I've been quite happy listening to MP3s up until now. :no: Granted, I had to set the treble knob on my speakers to 10, as well as the treble knob on my amplifier. There was probably at least 20dB more treble than there's supposed to be in the test.
 
Well, that seems to indicate (to me) that there really is a difference in the high frequency resolution of the different file formats, and that you can clearly hear it when the difference is exaggerated, but that it is maybe masked in most listening situations?

Eh?
 
BTW, I did something like that a couple of years ago and ended up re-ripping most of my Jazz CDs at lossless. Even if I don't notice it in every situation, it makes me feel better.
 
DC said:
Well, that seems to indicate (to me) that there really is a difference in the high frequency resolution of the different file formats, and that you can clearly hear it when the difference is exaggerated, but that it is maybe masked in most listening situations?

Eh?
Click to expand...

That there's a difference is not up for discussion; there most definitely is. The question is whether or not it's audible. To me, the distinction was a lot easier to make when I boosted the treble as much as I could, but I also know that I've got better high-frequency hearing than most people; I hear tones up to about 22kHz. Someone with more normal high-range hearing ability might listen for other things than I did.
 
Freezer7Pro said:
That there's a difference is not up for discussion; there most definitely is. The question is whether or not it's audible. To me, the distinction was a lot easier to make when I boosted the treble as much as I could, but I also know that I've got better high-frequency hearing than most people; I hear tones up to about 22kHz. Someone with more normal high-range hearing ability might listen for other things than I did.
Click to expand...

My high-frequency hearing is probably not as good as yours, but I heard differences in "resolution" (I think) when I did a similar test some time back. It was enough to convince me to go back and re-digitize some CDs at lossless format. I remember being satisfied with the difference.

Anyway, sorry to derail the thread! :)
 
Well, it wasn't technically a DBT, but the listeners could neither see nor hear the person making the change - we listened to a cut all the way through once, changed speaker cables, listened to that cut through again (used in-line banana M/F connectors rather than a switch), and listeners noted which cable they thought was in use - it was the standard buck-a-foot Monster cable of the time vs. a $10/ft Audioquest. There were six of us, and we took turns being the switcher for five cuts each (so ten trials each).

System included a Well Tempered Record Player with Denon LOMC, Audio Research SP-11 Mk II, D-250 monoblocs, and Duntech Crown Prince speakers. Results of the test were statistically insignificant according to the math whiz of the group; we may as well have been flipping coins. I don't think any of us were firmly in a skeptic/believer camp at the time - all but me were young college students.

A few weeks later, we used a similar format for comparing a $250 Sony CD player with an almost-two-grand Denon. The only person with a statistically-significant result was me - I mis-identified the players in every single case. At least it proved I heard a difference.
 
Last edited:
Actually, double blind is not absolutely necessary. We have set them up where everyone was sitting facing the speakers, with the person doing the switching behind those judging. He wrote down the sequence in which he would play the individual pieces, and then write down our responses without making any comments or sounds of any kind during the test. In other words, as long as you cannot SEE the tester, nor allow him to make any responses during the test, he should not figure into the equation.

Trying to compare vinyl to digital is somewhat of a problem, though..due to the inherent noise from one. However, one test in which I participated was done by recording a very good, clean vinyl record to CD, and then comparing that original record with it's CD copy. None of us were ever able to consistently pick which was the vinyl and which the CD while they were playing and being switched back and forth.

Also, my friends did a test where a very expensive CD player was compared to a cheap department store model. The owner of the expensive one picked the cheap one nearly every time as being the best sounding. He was absolutely sure he was picking his. Needless to say, he was a very unhappy camper when shown the results. By the way, no one else there could reliably pick between the two.
 
You must log in or register to reply here.
Back
Top Bottom