While an oscilloscope is a superior tool for chasing hum/ripple, a DMM set to AC volts will show an ~RMS value for ripple. All that you have to do is place the probes across the + and - lugs of the capacitor to measure the ripple across it. As mbz says, several tens to perhaps a hundred mv (RMS in this case) is normal, and should be about the same for the caps on each side of the power supply. While you are in there, you can also measure the DC volts across the same lugs; these values should match the PS spec for + and - volts DC, and also should be about equal but of opposite polarity for the two sides. In my experience with failed PS filtering caps, if you have significant ripple, the DC voltage on that side will be significantly reduced. The more the AC ripple, the more the reduction in DC voltage.
But the fact that the OP has already reported that there is no buzz through the speakers already indicates, IMO, it is unlikely there is excessive ripple across the suspected cap(s).
The other question that bears asking is if the OP is absolutely sure it is that one cap that is buzzing, and not the nearby transformer? Mechanical buzz of the transformer, without buzz through the speakers, is not uncommon.
In my experience, if you need to place your ear next to something in order to localize it, distances of a couple of inches can be hard to distinguish. Also, knowing the frequency of the buzz can help: 60Hz for the transformer, 120Hz for post-rectification (e.g., the caps). Again, an oscilloscope is superior tool for this, but an audio frequency plotting app like Spectroid on a phone or tablet can tell you what you want to know. It also has the benefit that you can place the microphone in the phone more precisely than you can place your ear, giving grater confidence in identifying the actual source of the buzz.