Defending yourself from attack is not ruling by force.

As long as nobody is aggressing against you there is no need to defend yourself with violence.

Absent those folks forcing these folks to comply there can be no violent conflicts.