Agree on the 44.1/48 thing; a niggle at most.
HDMI only for output is definitely a problem and the reason I keep the AirPorts around; they speak Ethernet/WiFi data as well as optical and analog music, and will therefore connect almost anything to anything else.
You're shitting me on the remote, right? The flat one that is impossible to pick up off a flat surface without dropping it? The one that interprets rubbing on its sides as command inputs? The one that has vastly different responses to horizontal vs. vertical swipes? The one that forget to include command repeat when a "button" is held down continuously? The one that is always going to sleep so that you have to press MENU and wait for communications to be re-established before you can once again use it for control? THAT remote?
And the ATV 4K 32GB runs tvOS (unlike the previous generation) so there's little likelihood Apple will port, um, improvements to the 4/4K units back to the 3(?) series. Mine is constantly locking up, as well as routinely forgetting its links to my iTunes library running on a pretty robust Mac mini. A measure of the flakiness of the unit is that I can now pretty much restart the goddam thing with that horrible remote without even having to turn the TV on (once comm is reestablished, naturally). I have to unplug the ATV4K at least once every couple weeks when it totally locks up and becomes unresponsive to its remote, network "pokes", or WiFi pleas from my IOS devices. BTW, this is all with the thing hardwired to my 1Gb Ethernet, so there's no WiFi complications involved (the IOS devices WiFi to my Time Capsule which has a hardwired connection to the network backbone as does the ATV4K—a complete and total piece of shit in my book. Boo, Apple.
Of course and as always, YMMV...