If you're a top manager, a founder or C-level type, you still have to deal with loneliness (since the power you have over the people below you in the org chart inevitably crimps any relationships you form with them), but you at least get the compensating benefit of being able, to some degree, to determine your own destiny: you make decisions, the company pivots to follow them.
Middle managers don't even get that; they're sandwiched between the people above them, who actually get to decide what the "company line" is, and those below them, who just have to follow it. Middle managers have to not just follow it, but enthusiastically agree with it, even if they think it's stupid or counterproductive. (Or at least they have to do so whenever the people they manage are watching.) Failure to do so, to participate eagerly in whatever fad thinking the people at the top are high on this week, is what I used to hear described when I worked on an Air Force base as "a career-limiting maneuver."
And on top of that, they have to work in an environment where even the limited power they do have to make a difference can be thrown out the window at any time. All it takes is for someone higher up the food chain to take an interest in your team and suddenly all the decisions you've made start getting overridden. Their reasons for doing so can be completely arbitrary, but that doesn't matter. If you're lucky, the person doing the overriding is at least generous enough not to do that to you in front of your team. If you're lucky.
This is part of the reason why the traditional career path of promoting senior developers into middle management is so destructive. The usual critique of that model revolves around programmers often not having the kinds of skills and experiences that make good managers, and that's true. But it's also true that middle management just generally sucks as a job, regardless of who you put in it.
Sometimes (depending on the sensitivity of the issue) they would also tell the team what they were doing to try and get the decision changed.
I have been fortunate to work places where the higher levels of management welcomed dissenting opinions - maybe that is one of the key bits to look for when determining if a (traditionally-structured) company is a good place to be a mid-level manager.
Secondly, what I heard in your comment (and agree with) but don't see in the original post is anything about engaging to be an advocate for your employees up the chain, giving them an opportunity to tell you what they need, the questions they have, the things you can do. To a certain extent this might make you lonelier on the "who do I talk to?" side, but helps a great deal as a reminder of the purpose of the job.
A middle manager is two APIs: one for upper management to interface with the workers, and one for workers to interface with upper management. The two APIs have distinct areas of responsibility, but both of them should include sanity-checking on their inputs rather than just passing them on to the layer above/below, and sometimes they should definitely throw exceptions.
This is what makes middle management a difficult role: it is literally two-faced. Internally, however, there should be a single state machine that maintains some kind of consistency, balancing the current corporate direction with the well-being of team members and the desired career track of the middle manager.
Unfortunately the ideal balance often involves all parties being about equally unhappy.
Reminds me, when I was offered such a job by my boss and didn't get it, because the rest of the management team didn't think I was "loyal" enough.
I never heard such a justification for positions I had before.
Most of the time people give me jobs because I knew to do something they didn't or didn't have the time to do it on their own.
In the end some "more loyal" person got the job. And I thought, oh so they really just wanted a person who didn't disagree with them...
Well, in the end I packed my things and left.
Clearly, my colleagues and I are doing it wrong, then. If you enthusiastically agree with everything the higher levels hand down, you are completely superfluous. In any decent organization, it is your job to disagree when necessary.
You are higher up than your reports, so you have a bit more context that helps communicate/explain things that come down from "on high". At the same time, you're still close enough to the trenches that you can speak up about issues that just weren't visible at the level above you.
Yes, ultimately higher management can override all your decisions. But if they are any good at their job, they listen to your input. That's the whole point of having layers of management - separate abstraction levels, and the ability to communicate important concerns up and down the abstraction ladder.
Yes, the manager who's just the megaphone to "his master's voice" is a common pattern, but it's not the pattern of a good manager. The same way that the copy/paste "programmer" is a common pattern, but not the pattern of a good programmer.
On the positive side, when I do have the opportunity to assimilate those decisions and the motivations behind them, I feel suddenly far more powerful – then I can engage positively both up the chain and down. Which makes the impasse a middle manager so often finds him or herself in so disappointingly counterproductive – my imaginative self feels there must be another way.
[This is a concept I first heard about in Elliot Jaques' thinking around what he calls Requisite Organizations... I don't agree with all of his thinking, but this context-setting piece has always seemed crucial to high-performing larger organizations]
This has two big benefits: first, it should make it easier to assimilate higher-level managers' decisions and understand the motivations behind them and second, it puts everyone in a position to make better-informed decisions on their own.
As a simplistic (and not particularly realistic) example, if I am a dev manager trying to decide whether to devote time to new feature development or a refactoring effort, making the 'right' decision is dependent on knowing whether we have things in place to put a marketing push behind new features and generate new sales, whether we are at risk of losing existing customers due to latency issues that the refactoring would resolve, etc.
If the right context-communication process is in place, I know that answer and can easily both make my decision and communicate to the team why it is the 'right' one.
On the other hand, I had all kinds of problems working for upper management I didn't believe in. When I didn't trust the folks above me, it was very difficult for me to implement even fairly benign changes.
Middle management should also have this relationship with the upper managers. Upper management should trust the middle managers are capable of leading their team and will raise issue doing so with them. Middle management should trust that upper management respects their ability to understand the needs of their team and will take every effort to maintain a suitable environment for them to lead their team.
The role of middle management is to have relationships on both sides. If middle management is lonely this chain of trust is broken.
This is an undoubtedly challenging position to be in because they are required to maintain relationships on both sides and the trust of each relationship must go both ways. If middle management is unable to express their concerns to upper management, either because upper management doesn't trust the middle manager is raising a important concern or because the middle manager doesn't trust upper management will listen (or raising the issue will threaten their career), they do not have a relationship of trust. The same goes for middle managers and their employees.
But my job can suck too. I've have also been asked to find solutions to problems with variables outside of my control, or take on a project bigger than anything I've worked on before, or to do something outside my experience and knowledge. Two valuable lessons that have come from of these challenge for me have been 1) asking for help and 2) saying no. Both of these lessons are about communication, and communicating with other people is the best cure for loneliness.
Right now I have a great position now where I still do a significant amount of development but also have authority and a fancy title and a management structure above me that's mostly concerned with business decisions and lets me make the technical ones. It took me longer to get the title but it was worth it to wait. I couldn't have done it at a large company.
Before learning to say no life was miserable. After I did it was heaven. Learn to say reasonable noes to your customers, to your upper managers, to your partner,to your kids, and your life will be much better.
Forget the advice of that stupid blog. You should not always support upper management, you are not a drone. If you work in the army and they tell you to kill every civilian inside a village, you must say no.
When upper management wants to do something that is stupid and you will have to support and be responsible for it, you should say no(after you double proof that it is not you who is wrong).
Get a mentor you respect and admire. Create a mastermind of the best managers you could find. You will discover management is not what you think it is(or better said, it does not have to be that way).
The Manager Tools podcast gives insight into the language and thought processes of top-down thinkers. Useful for learning how how to work within and around such structures: http://www.manager-tools.com/2013/04/politics-101-chapter-3-... & http://www.manager-tools.com/2013/05/politics-101-chapter-3-... . This should be balanced by authors like Tim Ferris, Chris Malburg (http://www.amazon.com/How-Fire-Boss-Chris-Malburg/dp/0425127...), or your favorite historical barbarian.
There's a Heinz von Foerster essay on free will within deterministic systems, "Only those questions that are in principle undecidable, we can decide. Why? Simply because the decidable questions are already decided by the choice of the framework in which they are asked, and by the choice of rules of how to connect what we call "the question" with what we may take for an "answer."", http://web.stanford.edu/group/SHR/4-2/text/foerster.html & http://kiriakakis.net/comics/mused/a-day-at-the-park
Complex bureacracies (academia, airlines, government, large enterprises) illustrate that thinking humans are most valuable at the boundary of system determinism. Any human who just does what the "rules" or "book" or "computer" says, will eventually be replaced by a mindless algorithm/app. But airline customer service people in First Class, or executive assistants who have worked with the same CEO for decades, or great managers - define human free will and agency, with their decision-making capacity for handling the entropy of "irregular operations".
If we view large organizations as semi-deterministic programs for which we have lost the source code, how would that change our view of management?
And this is why middle management is next to useless and why there have been so many cuts over the last 20 years. If you aren't honest with your direct reports then you will be seen as untrustworthy and/or clueless. Either way I'm certainly not going to feel inclined to be honest and straightforward with you.
The reality is as a manager you're an influencer and leader. You also need to have personal integrity or you'll just end up depressed. If you have no ability to influence upper management, you need to work on how to connect with your boss.