Could we decrease the HN points counter to 0 for new comments?

This is something that bugs me a little about Hacker News. It's a very minor issue, but I'm bringing it up because it's Hacker News.

Let's say I make a comment, and it has 14 upvotes. That does not mean 14 people saw it and liked it, it means 13 people did. A new submission with 2,834 upvotes only had 2,833 people who actually upvoted it. I know that there are downvotes which make things not a perfect match, but I don't think that affects my view.

In my ideal world, every submission and comment would start at 0. And they'd either go up or down from there.

This would also be adjusted in the "fading" for downvoted comments, with them starting to fade at -2 instead of -1.