Not really increasing that much. As I said, there are some (few) examples of these cars fetching high prices. Mostly, they fetch high asking prices, not selling price.
I've owned mine for nearly seven years, so I've been tuned in to the market to some extent. Just look at ebay. Some of those have been relisted several times. There's a '66 for $12K asking - it's been listed for awhile. Recently, a pretty pristine '69 was listed at one of the big auction companies, Barrett-Jackson I think? - Anyhow, it no-saled at somewhere around $12K.
I don't know about the Riviera though, but I haven't seen sky-high prices for those either when I've looked.
Why? They're not the holy grail cars such as the Cudas, Mustangs, Chevelles, Chargers, etc. They're also not easy to restore or customize. You have to remember, the Toro was the ONLY FWD car out there at least American - at the time. So you aren't souping up the tranny (or swapping it) - though it is in essence just a Turbo 400 3-speed. You can barely buy wheels for it - unless you have some custom made. The offset from the FWD is ridiculous.
Also, there are very few reproduced parts. You aren't buying a fender out of the JC Whitney catalog. You also aren't buying anything from GM Obsolete. You either have it or you scrounge for years - trust me, I know.
Some other issues are the oil pan - has to be one for a Toro. There's a little ole' driveshaft that goes underneath the pan creating a curved hump. Can't just slap a big old Moroso pan under there. The intake manifold is more flattened. You're also not bolting on a huge double-pumper Holley without cutting a hole in the hood. There's not much room between the air cleaner and hood. And the 455 block is pretty specific to the Toro. In short, they're not a very easy car to restore, let alone hot-rod.
They're also considered an old guys car, not a hot rod. They're a personal luxury coupe - and those just don't bring the same money. Tell me, when have you seen a $50,000 Monte Carlo, Grand Prix or other car like that? Not often. Same deal here.
Having said that, mine is a 455 4 barrel - from factory 385 HP with nearly 400 lb ft of torque to the front wheels. Pretty good numbers, though you're right, it's a 5,000+ lb car.