I used to be pretty deep into car audio. Often found myself pulled over on the side of the road, or in a random parking lot, running to the trunk to tweak/adjust. Drive 5 minutes then pull over again. Etc, etc. Similar with home audio... set it, enjoy it, hate it, "fix" it, hate it more, "fix" it again, nope not quite right, blah, blah blah. Been happier without tone controls in my system. True, some albums, and even songs on a given album, will sound better than others - it's the nature of the beast. I will admit to boosting the bass knob (digital on phone) a notch or two for my headphone setup. That's my story and I'm sticking to it.
Enjoy the music as you see fit!!