Why voting against directors matters

A couple of excerpts from Power Without Property by Adolf Berle

Concentrated economic power, whether held by private organisations and directed by their chiefs, or by the State and its chiefs, raises at once the question of "legitimacy".
Why be concerned with "legitimacy"? What difference does it make? Power comes to rest in the hands of specific groups; why not accept the fact, dealing with it merely as a fact? The answer is deeply rooted in immemorial custom and human experience. Power is a fact; but it also a fact that the human mind apparently cannot be wholly or permanently inhibited from asking certain questions. Why should this man, or this group, hold power - instead of some other, possibly more attractive, individual or group? The human animal has always endeavoured to answer his own question: power lies there because the holder is entitled to it by some test or standard. This carries a necessary corollary: the holder can be deprived of it if demonstration is made that there is no title or right in his possession of it.
....
[A]t present... the men vested with economic power - because they hold a position in a corporate or other economic organisation - got it by a method which the community recognises: through prescribed or accepted process or ritual. Boards of directors of large corporations are recognised as representing the stockholders because they have received the votes of holders of a sufficient number of shares to elect. The ritual of their election is ordinarily casting of a ballot for them, at a stockholders' meeting held for that purpose. It is not an impressive ritual... This is a far cry from the ritual coronation of a king, claiming power by inheritance and the grace of God, and assuming it in an abbey or cathedral by dramatic ceremony. Yet the rationale of the stockholders' meeting and of the coronation is the same. In both cases the ceremony is intended to affirm that this man or this group legitimately holds powder under accepted conditions. It is designed to induce (as indeed it does) general acceptance that the power has been well and truly located in the specific individual or individuals involved.

Whatever blah asset managers come out with to explain why we shouldn't read too much into voting decisions, ultimately if you vote for a director you reaffirm in public their right to power. Looking at Sports Direct, for example, the company has strong grounds for arguing that - by reference to the legitimising function of the AGM - the current board has no need of change. If you voted for the re-election of the current board don't be surprised if you get more of this kind of thing.