The more I study, the more I marvel at the wisdom of the Rabbis. In פרקי אבות it is written, אהוב את המלאכה ושנא את הרבנות. A loose translation would be, "Love the work, but hate the glory."
The Mishna points out that Yosef lived a shorter life than his brothers (only 110 years old ) because he was in the public eye and held a position of authority. The Mishna is also hinting to the fact that power corrupts.
When one is in a position of power, it is a huge challenge to remain humble and honest. This is the reason why we hear so many cases of either misuse of funds at the disposal of such a powerful person, or sexual improprieties because everything is coming to him.
The Mishna is emphasizing that working for the benefit of the community is a noble task. But we should focus on the work alone and hate the fact that certain notoriety and power may come with such work. And that could be dangerous and lead to the downfall of that person. Brilliant advice from our very wise sages.