I don't think those kinds of comments are called for. I'm not singling you out; I have made similar comments in the past and I'm trying to not do that sort of thing anymore. It's bad for AK and unnecessarily hurtful to some.
Let's say that the logic is that if a person is very much involved in their hobby, to the point of what some might consider an obsession, and those persons are given a name (like 'audiophile'), other who also are into the same hobby but not to that extreme sometimes don't like to be thought of that way by those not involved in the hobby at all. One might say "Yes, I'm an audiophile, but not to the extent that some are. I just like good music reproduction, I don't spend a fortune on it or sweat every single detail."
There is nothing wrong with people who choose to spend their money and effort to take it as far as it can be taken, and who believe that certain things improve the sound they're hearing that others think of as impossible. At at certain point, we're talking about the right of people to spend their money as they please and enjoy what they enjoy. Leave them be, live and let live. It is no skin off my back if someone thinks that something I find ludicrous improves their system. Fair enough, it's their system. They are audiophiles, I am an audiophile, and it's OK that we're not the same.