By default, when you subscribe to a thread, under notifications, "no email notification" is selected.
Well, that's the point of subscribing, right?
So, I make sure "instant notification by email" is selected.
What was odd was that I was getting notifications for threads I hadn't subscribed to, only replied to.
Thanks for helping me find that setting!
I'm pretty sure I didn't change it, I didn't even know it was there.