As far as blind testing goes, no I am not surprised. Identifying an amp through blind comparison testing is very difficult. However, determining which is smoother, which is faster and more articulate, which is less fatiguing, etc., etc., is very possible.
We must be talking different objectives for the experiment.
The objective of my blind test experiment, and those I've seen discussed, never has been to identify a specific piece of gear. In other words, the goal is not to guess if it is the Pioneer or the McIntosh, the goal is to match an unknown, random selection to itself.
The type of test experiment I did, and that I'd suggest, is the ABX methodology. Meaning, there are two pieces of gear in comparison, one piece is A, the other is B. A and B remain A and B thoughout the duration of the test. Then, at random, either A or B is played, you are not told and cannot see which it is. This becomes X.
So, you listen to X then you get to change/switch between listening to X and A or X and B, as many times as you want, but eventually have to choose whether X=A or X=B.
For any given test (one choice) the relationship of X to A or X to B does not change. Once you choose X=A or X=B, the relationship of X to A or X to B is again randomized prior to the listen and the subsequent choice.
This randomization and matching/selection process is repeated as often as desired. Typically a minimum of ten times.
At the end, you get your "score" of how many times you correctly matched X to A or B.
The premise, of course, is that if there is an audible difference between A and B, matching X (which is either A or B) to itself should be easy.
