Once you change a driver or make modifications that significantly change the crossover behavior, it's really no longer the same speaker. That can be fine if you like it better, but it is significantly different, and any changes should be noted if you sell the speakers. My son recently bought some EPI speakers and the woofers had been replaced with some generic Radio Shack woofers, so not even an upgrade....unfortunately that was never mentioned to him, but should have been.
I've upgraded and made minor mods to many speakers over the years, but most were fairly pedestrian mid-fi level speakers. I would think twice before modifying a really well designed and/or expensive speaker like a Vandersteen, Magico, Merlin, etc. Most of the changes I've made were upgrades to caps, wires, connections ,etc., but have also made tweaks to tweeter output, cabinet bracing, etc. to some that I felt needed it. Replacing electrolytics with decent poly caps has been worth it every time IMHO, even if the old caps were in spec....they're just clearer, but it can change ESR which is another variable to be mindful of.
The vast majority of big name speakers are from large corporations and are built for 20-25% of the MSRP, meaning a $1000 pair of speakers costs $200-$250 to build, which is not much considering what a good cabinet, good crossover parts, and drivers cost. Many corners are cut to meet that price point, and many less than ideal parts are used....I have no issue making upgrades to crossover parts and basic cabinet damping and/or bracing. It's wise to be mindful that the sound will be different, and might not always be preferred....ie: a significant portion of the output can be from cabinet resonance, which can be perceived as warm sounding even though it's a coloration. It's a matter of preference regardless if it's technically better.