Yeah very true. I suppose I'm just very paranoid as a novice, due to some people saying they have huge collections of used records and never get IGD. Not doubting them, maybe they're very lucky with buying used. Also I was once told that even though another pressing of the same album hasn't got IGD, it's not an accurate test unless the writing on the runout is identical. Sigh.
Now I tend to notice any little bit of distortion on albums. Sometimes on harsh guitar hits, for example, I'll hear a faint 'buzzing' in the left or right channel as it fades away, if you know what I mean. Stuff that you probably wouldn't hear on speakers but I hear on headphones. Stuff that although not overpowering, makes you question your setup and whether you're damaging records.
Like this for example but not as bad