I promise you that he only remembered how much he loved you in the end. I know it hurts buddy, but try not to dwell on any rough patches. You gave him a wonderful life. That's what truly matters.