I care to the extent that I would like anything I add to be in similar condition to the rest of the set I have.
If I wasn't matching a few cards to a set I had most of it isn't so much of an issue. As a matter of fact, if I were starting a pre-1980 set from a near starting point I could see me looking for lower grade cards to make it affordable.
One of my favorite cards I own is a Hank Aaron that is in pretty good condition with also pretty good centering and color. The only issue it has is a pinhole top center. I imagine someone pinned it to their wall at one time. It's a great card and due to this cost me around $20.