This topic will never disappear, it seems. Round and round we go, like musical chairs, and with every post of every thread, someone else is the one temporarily left without a seat.
Let me try to focus a few thoughts on this subject:
1. Yes, you cannot (with current technology) improve on the signal from a source. You can sometimes make it sound better, though. For example, if your original recording has an overly-pronounced midrange, then a cable, amp or speaker that subdues the midrange somewhat, and lets more of the highs and lows through, may make the overall balance seem better. The result can subjectively sound better. However, it hasn't really improved things, because what you really have accomplished is simply to remove some of the sound that was originally there (from the midrange). The old adage remains, "garbage in, garbage out", even if you make the garbage a little tidier. So in a broad theoretical sense, the source could be considered the most important component.
2. However, that does not necessarily mean that in the real world, the source is the area we should focus most on, or even spend the most on. A system is just that: a system. All components must work together. The speed with which a signal moves through a system may be considered as instantaneous, much as the light we see when we switch on a lamp is, effectively, instantaneous. All parts affect the sound. If you doubt this, take any really outstanding system, and replace just one cable on each channel with, say, a piece of string. Now how does it sound? You just lost 100 percent of your sound quality, from just cable! Does that mean that cables are the most important part of the system? They are, if one or two of them are just pieces of string!
3. In practical terms, then, the most important part of YOUR system --that is, of any given system you are considering-- is whatever component is the "weakest link", the one that is holding back the other parts of the system the most. It is the one which, when it is replaced or upgraded, makes the biggest positive impact on the final sound. In one system, perhaps an outstanding, highly revealing system, it may be the original recording, whose weaknesses are clearly audible. In another system, it might be the pre-amp, or even the cables.
4. BUT in MOST systems, especially digital ones, it is likely to be the speakers. I've heard far greater differences between speakers than between any other components, in systems that were otherwise the same. When Yamaha introduced the NS-1000 monitors, they declared it "the end of the weak link", for precisely this reason: speakers are usually the weakest link in an audio system, the one place where the biggest improvement can be found when upgrading.
5. It is still true that, with truly outstanding, revealing speakers, the source flaws will be evident, and the importance of the source will be revealed. But those same recordings that sound flawed, would not sound so with lesser speakers, and even the best sources cannot sound truly superb without outstanding speakers. Both are important: good speakers can distinguish between good and bad sources, and better sources will sound better through virtually any speakers. But speakers are where the biggest real-world improvement can usually be found in most systems.
6. Let any one component be substantially inferior to the others in a system, and the decrease in sound quality will be evident (but no where else as much as with speakers), so in a particular system, any "weak link" component can be said to be the most important part of that system. Just remember the strings for cables!
7. The reason we focus on the beginning and end of the chain --sources and speakers-- is, as has been said, because they are where energy is converted from mechanical or sonic vibration to/from electrical; these are the transducers, where the highest demands are being placed and where the most room is for flaws to be introduced. They are the most difficult parts to make well and good. Amplifiers in-between can alter the sound in varying ways, but --at least among the better ones-- tend to alter it much less than either cartridges or speakers.
8. Don't forget the importance of the listening environment, or even the listener him/herself, to the overall equation. If someone is partially deaf, they become arguably "the weakest link". Perhaps an equalizer or hearing aid could be the "most important" component in a system for someone with imparied hearing.
9. Ultimately, this whole discussion is a bit silly, as a system cannot function without each of its parts functioning in some degree of harmony (pun intended

) with the others.