The people are difficult to govern because of the (excessive) agency of their superiors (in governing them). It is through this that they are difficult to govern.