text: Initially, the album was going to include a duet between the band's frontman and Garbage vocalist Shirley Manson. The track – a cover of The Human League's "Don't You Want Me" (1981) – was recorded after the two artists met at a Yeah Yeah Yeahs concert and enjoyed each other's company. After the cover's completion, Marilyn admitted that he "wasn't in the best frame of mind when I did it". Shirley commented on the cover: "It's really cool but neither of us felt comfortable putting it on our records so I don't know if it will ever come out. I hope it does. Our voices sound fantastic in a very Beauty and the Beast kind of way." Both singers wanted to try to collaborate again sometime in the future, and Manson said that he enjoyed working with Shirley.
  After making the decision not to include the duet on Lest We Forget, Marilyn Manson took a break from making music. After receiving fellatio with a rosary wrapped around his penis, Manson had the idea to cover Depeche Mode's "Personal Jesus" (1989) for the album. Manson said that he wanted to cover "Personal Jesus" because he found Depeche Mode's music hypnotic, sexy and inspirational. Manson told MTV that "I thought if I had to write a song, [the lyrics of 'Personal Jesus' are] exactly what I would say. And that's why I picked this song, because I think it takes a little more of an ironic tone when you put it in context with what's going on today." The band's version of "Personal Jesus" is the only new song on the album, and was released as a single.