I assume the systems that sounded different did not include the same speakers placed in the same location in the same room.
That's just it...you go out and listen to this stuff, and you realize that its not that hard to tell things apart. I can take my preamp into another system that I'm not familiar with, and I can tell if its a good preamp. I won't be able to tell exactly what it would sound like in my own system, but I can tell its good. There are extremes- I've auditioned some gear that I knew was good, in systems that made it sound bad. But with a base level of competency in gear, its not hard to tell the good from the bad. And you can tell largely where things are falling apart, be it the speakers or preamp or source. Unless everything is a total mess. The problems of speakers tend to stand out in a different manner than the problems with source gear. To a certain degree, at least. And while I don't think there are golden ears, I think listening is something that can be perfected by going out and listening to as much of a variety of gear as you can. I would think that would almost be an obsession for people in this hobby, but many seem to have an aversion to listening to anything different at all, much less listening for differences between things.