(...) I think perhaps the Sony unit had a few more bells and whistles that might account for a higher price point and it got very good reviews as I recall. (...)
Actually, to me it would rather seem the other way around (regarding the originally intended usage as digital field recorder, that is), due to the aforementioned dual recording feature of the Tascam. But of course these dual recording modes are of little interest for us hifi folks, who are primarily interested in line-in recording in plain stereo. Pretty much the same would go for the somewhat better (= closer to noise-free) mic-ins of the Sony.
Not sure, whether the Sony also offers a higher SNR on it's line-ins, though. I.e., on paper it doesn't, but the problem with those specs is, that both (Tascam & Sony) are for line-in to line-out, while I know from a review (done by good, old Reinhard Paprotka for the German Stereoplay mag), that the Sony does quite a good bit better (than specified for line-in to line-out), when one just looks at line input to the recording medium, in which case the PCM-M10 manages to achieve the specified ADC noise level of its CS42L52. However, due to the lack of a comparative line-in to medium result for the DR-2d I can't tell, whether the Tascam does just as well in that regard.
So what might really have made the Sony more expensive (a possible "It's a Sony" premium aside

), might be its extra 4 GByte of internal flash memory and the software bundle, I could imagine.
You said "-4 dBV, which translates to just a tad over 0.63 V(eff.) - while the line-in of the DR-2d could still digest up to 2 V(eff.)" If you would explain how that "translation" is done, or direct me to a place where I can read about it myself, I would be grateful.
Oh, well, that pretty easy actually. Often enough foremost in the realm of prosumer audio, but sometimes also in the realm of hifi audio, levels aren't given in plain (effictive) voltages, but in dBV or dBu values - so one has to convert these, unless one already is familiar with these anyway. Reference for dBV is 1 V(eff.) - whereas for dBu it's not quite 0.775 V(eff.), the origin of this reference value being the voltage required for a power of 1 mW in combination with a 600 Ohm load.
So for example for converting -4 dBV into a plain voltage you'd calculate 10^(-4/20) *1 V(eff.) = ca. 0.631 V(eff.), for converting -4 dBu it would be 10^(-4/20) * 0.775 V(eff.) = ca. 0.489 V(eff.). So that's simply the other way around from the dB formula for voltages (i.e., dB value = 20 * log (voltage to be expressed in dB / reference voltage), with the reference voltage being 1 V for dBV or ca. 0.775 V for dBu, as already mentioned above).
Personally I typically convert such values with the Windows calculator (switched to scientific mode), so I don't forget, how to do it (in which case I typically first calculate the x/20 value and then use the inverse log function to apply the 10^ to that value, as I deem that more convenient, but that's just a personal preference...), but of course you could just as well use an online calculator for that purpose - like for example the one provided by good, old Mr. Sengpiel (rip!) over there:
http://www.sengpielaudio.com/Rechner-db-volt.htm
Do you know model numbers for any of those "earlier among the very compact models"? (...)
Well, from my own plus a friend's experience I'd primarily suggest to look for the Sony PCM-M10 and Roland R-05 - but if Andreas says, he's happy with the DR-2d, that's probably a good one, too.
Whereas I'm not so sure about the also earlier Olympus LS-10, LS-11 and LS-5, which also still featured a separate line-in, iirc - but I think all three had a somewhat vague line-in level spec of "-6 dBv" (= either ca. 0.5 V (in case it's a typo, and they actually meant dBV) or ca. 0.388 mV (in case they used dBv as a synonym for dBu)) and I never found any good info back then, whether that's the minimum 0-dBfs-level at maximal gain or the maximum 0-dBfs-level at minimal gain (= maximum line-in level) or someting in between or some kind of rated level. So these three, albeit otherwise also being attractive models, were out for me back then, as I wanted to make sure, that my recorder of choice would have no problem with modern sources with an output level of ca. 2 V.
Greetings from Munich!
Manfred / lini