Chris, where are you located? With a lower power, a horn loaded or a number of vintage speakers might be the answer.
Wharfedale W series will work. They can be had for well below your budget and while they will likely need caps even refurbished they will leave you a lot of excess cash.
Klipsch Heresys and Cornwalls also will work but, the Forte and Forte II may be more to your liking. Add Bob Crites magic and they are hard to beat. If lucky, you might even be able to locate a beatup set of LaScalas. Physical condition of these is not a big deal as most are unfinished. Again, Crites is your flight to person. There are other companies using horns you should also consider but, Klipsch is the most mainstream.
You also can go a might esoteric and find set of single drive full range systems such as the Lowther. These operate on flea power and provided you have an open mind, are really an amazing experience in the audio range you can hear. There is a whole world out there about these types of speakers and it should not be ignored. One of the best speakers I ever heard with female vocals was a Lowther. I would own the set today if I already was not so deeply vested with high powered amps.
Many vintage speakers from the tube period will easily meet your needs as back then 5 watt amps were very common. There is no need to apologize for what you have. There are a lot of us using console tube amps revamped as a power amp and they look and sound excellent. You may want to do a search on Magnavox tube amps as there are some threads about them.
It is easy to dump huge sums and buy big components to buy into a good system but, it can be far more satisfying to build a nice low cost, low powered system. I just downsized from a 100/chl system to a 20watt/chl system and love it as much and even better maybe.