His infamous quote about "on the younger side" was in New York magazine in 2002, a year before the birthday message