It's not unusual for a motor to get a touchy speed-adjustment potentiometer INSIDE the motor. Squirt some Caig DeoxIT into the hole in the back of the motor, and turn the pot up and down about twenty times before doing the final speed adjustment.
The easy way to do this is to order a low-cost test tape with some simple alignment tones on it, such a 400Hz and 1000Hz. Many common DMM's have a basic frequency counter built in. Play back the 1000Hz tone, for example, and measure the frequency at the LINE OUT jacks. Adjust the speed to match.
I like to use a jeweler's screwdriver to make the adjustment, however, I put a little shrink tubing over the shaft so that the metal of the screwdriver doesn't touch the case of the motor.
Best,
Fred