I think it comes down to if you plan on reselling it or keeping it. I usually keep the film on, but thinking of pulling some off. I don't see how they can become too damaged just by pulling the film off. It would seem to me they would get a few wear scratches like Chrome cards do unless you put them in a sleeve or something though. In the end though, if you never plan on selling it, then peel it off and enjoy its beauty.
Royal Card Review is my blog if you feel like checking it out, thanks if you do!- royalcardreview.blogspot.com/
In the process of updating my collection so don't trust any of my lists right now.