I never really thought about it from that perspective, so thanks for sharing how you feel.
I have always been motivated by people actually being able to see my accomplishments. I never took it as prying or invasive and I really don't think most people intend to be rude, etc. I know I have not meant to be when I have asked someone that question. However, if you don't like it for whatever reason that is fine but you have to let people know. There are wide spectrum of comments from "none of your business" to "thank you for noticing, but the actual number is really personal to me." You have to make the decision how to respond.