Well, expect to spend some money to join the club. R2R has a somewhat high admission fee, as you've already discovered. As for what machines to look at, it kinda boils down to what you want the reel to reel to do. R2R came with different tape widths, track configurations and reel sizes. From your post it sounds like you want to start basic. That would be quarter track, 1/4 inch tape at the standard consumer speeds. (3 3/4 and 7 1/2 ips.) Now, what size tapes do you want to play? Most pre-recorded reels come in the 7" size. As it would happen, 7" blanks are still the cheapest option around. New tape is still made and you can find it online from a couple of sellers, like usrecording or splicit.
http://usrecordingmedia.com/
http://splicit.com/
There are others, but those are the first two i thought of. Next thing to remember is reel to reel decks haven't been made in 25 to 30 + years now. (Unlike turntables and even cassette decks, which have always been produced to one quality level or another. R2R has not enjoyed this support.) Most decks will need some maintenance, if only just oiling, belts and maybe new rollers. Other decks will need ALOT more. But often when your question is asked i see some very common answers repeated. There is alot of love for the Pioneer RT 707 and 701. They are well built, sound good and even at this age are usually still running well as is. Parts are easy to find, and units are still fairly cheap. There is also a large number of reccs for the TEAC A series decks. Often needing some new lube and belts to run, but easy to repair and well built. Also, common as can be, so easy parts. Teac even still has some! In general, avoid the really old portable units with speakers built in or attachable.
Thats my condensed observations. I'm sure others will be along soon to give more suggestions from first hand use. If buying old tape, stick with Maxell. Its proven to be the most reliable of the vintage tapes. Stay away from Ampex, as much of it is the opposite of Maxell on aging with grace. (Look up sticky shed syndrome)