Just tested it on 15 different builds. New builds were the only ones that I experienced this with. So when I would go to upvote it would go plus 2 or plus 3. The builds that aren't new or popular don't have this problem so I guarantee that it's just refreshing and you're seeing the upvotes from other people as well.
Edit: The page doesn't automatically update like you're thinking, the only time it refreshes would be if you manually refreshed the page or if you clicked something e.g. the upvote button.