The stylus, indeed, makes a huge difference. Conical styli won't read as much detail as an eliptical, which in turn is worse than a hyper eliptical, shibata or fine line.
And the cart the stylus is mounted to makes a big difference. Some are just better at frequency response, channel separation, clear sound, etc.
And of course, the cart isn't flying on thin air, but is mounted on a tonearm. The tonearm needs to have appropriate mass to match the cart giving a favorable resonance frequency for the "sytstem" or tracking will suffer. And the bearings on the tonearm need to have miniscule friction to allow the tonearm to follow the groove under the influence of the stylus riding in it. And don't forget tonearm geometry. Some have it wrong and are worse than others.
And, of course, the tonearm isn't itself resting on thin air. It's rigidly mounted on the top part of the turntable or plinth, with a motor whirring below, causing vibrations that can and will reach the platter and the tonearm. And the plinth is sitting on a shelf or desk where it can pick up vibrations from people walking around or the speakers themselves. So, an isolated plinth makes a difference. As does the way the motor is mounted internally. And the mass of the platter as well as they way it's mounted (sub-chasis over springs, high mass, etc.).
So, there are a lot of variables in a mechanical system that is influenced both by internal and external sources of unwanted events. The way such a system is designed, the materials and way it's put together make the differences.
And to give you two extreme examples:
In my youth, I mounted an expensive (for me at the time) Stanton 681-EES cartridge to a Dual 1009. It didn't sound good, it tracked awfully and I preferred the lowly M44MG cart that came with the 1009. That same cart sounds just great on a Technics SL-1210.
Back in 2008, when I came back to listening to records, I scored a Revox B791 that came with an Ortofon MC20MkII and an MC step up transformer, all for the price of 250 euros. The sound was - and still is - phenomenal. It was my first moving coil cart and I never knew what I didn't know - how nice they can sound.