It's all about tinkering. Be it cars, stereos, boats, planes, guns, computers, etc. For audio it's all about trying this and that and seeing what works together, what doesn't, be it thrift store finds or $200,000 gear, it's in search of the same thrill. Finding that one change that takes the sound to the next level, be it from a new kind of cable, speakers, rack, or a change in any other one of the countless variables that exist in a system. It's all about the quest for something better and the thrill of finding it. If someone spent $200,000 on a succession of bass boats and fishing gear over the years or on a series of muscle cars nobody would bat an eye. If someone spends $150,000 and waits 10 years to get a custom made Italian shotgun it's an investment. But, if someone spends that on an audio system people think they are strange. Shows what people know. Everyone needs something to get jazzed about. Some kind of hobby to occupy their time, mind, and to bring them happiness. So long as they are not financially irresponsible and don't blow the mortgage or food money on speakers then I say have fun, and please invite me over to hear the new setup! I'll bring the cocktails.
Oh, women are not immune. They just tend to not be enamored with technical stuff like guys. They collect shoes and craft stuff instead. My wife has an entire room filled with stuff for scrap booking, sewing, beading, knitting, etc. She also has every kitchen appliance known to man (and she knows how to use them!). If you want to have a long discussion about different brands of pots and pans, food processors, juicers, or mixers she is a walking encyclopedia and will gladly talk about any of them. But if I mention stereos, guitars, or guns her eyes glaze over.