The question, as posed, is, "Are you legally bound to keep a physical copy of your music media after you have recorded or ripped for personal use?"
There is no statute that specifically says you have to retain a physical copy of the media if you created another copy. Such a statute is not necessary because the statutes describe what is prohibited and then give exceptions for special cases.
17 U.S. Code § 501 - Infringement of copyright, is one place to start. 17 USC 501(a) says, "Anyone who violates any of the exclusive rights of the copyright owner as provided by sections 106 through 122 or of the author as provided in section 106A(a), or who imports copies or phonorecords into the United States in violation of section 602, is an infringer of the copyright or right of the author, as the case may be."
To understand that statute, the "exclusive rights of the copyright owner" need to be defined. 17 USC 106 lists the exclusive rights in copyrighted works. Sub-section (1) identifies one of those exclusive rights as the right "to reproduce the copyrighted work in copies or phonorecords."
To understand that statute, the definition of "phonorecord" must be understood. 17 USC 101 contains definitions: “'Phonorecords' are material objects in which sounds, other than those accompanying a motion picture or other audiovisual work, are fixed by any method now known or later developed, and from which the sounds can be perceived, reproduced, or otherwise communicated, either directly or with the aid of a machine or device. The term 'phonorecords' includes the material object in which the sounds are first fixed." In short, a CD is a phonorecord.
The plain language of the statutes, as described above, prohibits making copies of CDs, for any reason. So we need to look for exceptions, of which there are many dispersed among the statutes and case law.
Let's quickly look at the exceptions. One is 17 USC 107, which is the Fair Use limitation on the exclusive rights. There is a history of case law regarding the right of owners of a phonorecord to make copies for their personal use, and some of that case law has been codified. None of the case law nor the statutes allow for making or owning copies of a phonorecord without also owning the phonorecord. See, for example, 17 USC 109(d). There should be no question that selling or even giving away the CD transfers ownership to someone else. Destroying a CD is not the same as transferring ownership.
Keeping a physical copy is just one way to prove ownership. It is also the easiest way to prove ownership. But the question of ownership only arises during an infringement action. Such an infringement action would likely be very rare unless a person is doing more than just keeping copies for his own personal use.
The answer to the question is "no, you are not legally required to keep a physical copy of a CD if you make a copy of the contents for personal use, but you must maintain OWNERSHIP of that CD," unless someone can identify a statute or case law that specifically allows keeping a copy of phonorecord without also keeping ownership of the phonorecord.
tl;dr: The law is that having a copy of a CD for personal use without concurrently owning the CD is copyright infringement unless there is a specific exception to the law. I know of no known exception.
PS: (fine print disclaimer) The above is not a legal opinion, but is only brief explanation of the law offered for educational and information purposes.
