I listen to six-figure systems fairly regularly. I enjoy it, not least because almost always, I come home to a system that sounds amazingly close or even better (usually I'd say different: better in some respects, perhaps not as good in others, but definitely competitive), but cost me a fraction of what those did.
The one system I heard some years ago that really did "spoil" me for a while is one that I'd have some reservations about hearing again, for that reason. I know it is head and shoulders above the usual "high end" stuff. (That system was built around IRS Vs.) But I'll go to the high-end audio show here, at which many of the world's most expensive audiophile manufacturers demo their newest and best gear, without fear. I know it won't "spoil" my enjoyment of my own system.
Plus, as others have noted, listening to good stuff educates your ears, so to speak. You become a more discriminating listener, when you listen more to various high-end systems. That is not a bad thing when you are trying to improve your own system.
As for rules, I think I have one predominant one right now: I don't buy any gear, unless it is a clear upgrade to what I have already (which is getting harder and harder to do, as time passes and my system improves), or something really unusual and "special", that I really love. No more accumulating vintage mid-fi stuff.

Now I'm much more likely to plan a purchase, save up for it, and pounce when the deal comes along.
That brings me to another rule: I don't pay above market price, and never pay the high "eBay" type of prices for things. My goal is to AT LEAST break even, if I ever resell, even though most of the gear I now buy I don't intend to sell (at least not any time soon). Actually, I'd probably be willing to break this rule for the "right" item, but I can't think of what such an item would be.
I weigh more than 200 pounds. There aren't too many units that weigh more than me (mostly speakers), but if I were lucky enough to find a good deal on a good unit that heavy that I wanted, I'd just hire someone to help me bring it home. I hire a van driver, anyway, and I'd hire a moving company to help move a big sofa, piano, heavy appliances, etc.. so what real difference does it make? I wouldn't be able to grab the stereo and save it in the event of a fire, anyway, right?