Does this happen with every tape you put in?
I think pretty much all walkmans in the 90's were just clipped together. As careful as possible, pry it open, slowly working around all edges AND not letting previously separated edges snap back together by mistake.
When you say the speed is unstable, is it just slow or fast, or does it "flutter"?
If there is a lot of flutter - most likely just needs a new belt.
If it is stable but slow or fast -
Sometimes there is a pinhole somewhere on the back (possibly hiding under the back label if there is one, but not likely) where you can put in a tiny flat head and adjust speed, of course to get it right you need a calibration tape and something to measure frequency like an oscilloscope or a multimeter that measures frequency (which is what i use)
OR when adjusting speed, you could basically adjust until "that sounds about right".
Sometimes the stupid speed adjustment screw is not accessible unless the cover is off.