If no whales vote, then all "minor" votes will be worth multiple times their normal value because the reward pool is fixed.

Think of it this way: There are 1000$ per day to reward users and 80% of them are decided by whales and 20% by smaller votes.

If the whales don't participate in the voting, the entire reward pool (1000$) will be decided by the smaller votes.