I recall buying tapes and CDs at Sears way back. Around the time that CDs were just becoming the thing for regular folk.
There's a regional drug store chain around here that sells music. The CDs are mostly popular stuff, but you can find some gems. Decent vinyl selection, too.
I score a nice collection of CDs there a few years back because of this thread:
http://audiokarma.org/forums/index.php?threads/jazzplus-reissues.546768/
Interesting; I read the thread pretty quickly, but I get the impression it was a Canada-only thing. I've never seen anything quite like that in a US drug store.
I noticed that the Real Gone Jazz stuff was mentioned as an aside; are those available in record stores in Canada? I've never seen one in the US, but I did pick an Ornette Coleman set up a few years ago when my wife and I were in Italy. It was only about $15 US for six albums, so it was too good to resist, but I was leery about the sound quality, and something seemed a little sketchy about it; nonetheless, they sound pretty good to me. I learned later that these albums are public domain material in Europe, so perfectly legal there, and are made by digitally processing LPs. But they are still technically barred by US copyright law, so of questionable legality here, although I have seen other, similar sets brand new here. Maybe Canadian law mirrors European law on this point.