It recovers fairly quickly. It's like a mana/energy bar in a computer game. If your voting power wasn't finite in some way, you could use an automated script to give yourself unlimited influence, and that wouldn't be fair to other users.