I have a whole lot to say on this issue so I'll try and keep it briefish...context, I'm finishing up my PhD in Engineering Education at a 'top tier' university. I also have a BS and MS in an engineering discipline.
Several friends in my program, as well as myself, have been applying for faculty jobs this year. Two of us have experience as technical founders, as working engineers, have prior experience teaching engineering, and now have PhDs in how to teach engineering well.
More than one teaching focused university has specifically told us that our degrees in teaching engineering are DISQUALIFYING to teach engineering at their schools. They will only take PhDs in that field of engineering. Those students, because of the nature of modern research universities, are often explicitly discouraged from developing teaching expertise - it distracts from the research their advisers need accomplished to get tenure there. So you have schools that produce future faculty discouraging them from getting the preparation needed for the majority of faculty jobs, and the schools that do the majority of teaching not interested in educational expertise.
One department head specifically told me during a phone interview they didn't see how a degree in how to teach engineering would be relevant to what they do. That school advertises that it's focused entirely on undergraduate engineering education, does not have graduate students, and rewards tenure almost entirely based on teaching performance. I pointed out that I had two degree in their discipline, worked as an engineer in the field for years, had already been teaching engineering before I returned to graduate school, and went back to school specifically to learn how to teach better. Silence. The last person they hired (who has become a friend/mentee of mine) went straight through from their BS to MS to PhD in a single engineering discipline. No teaching experience and no industry experience - but a PhD in the field.
The trend there doesn't just apply to teaching, it applies to research and to broader visions of what a university can or should be.
Really, summarized, the issues we are seeing are issues of strategy - but that is a dirty word at universities. The schools have all, teaching as well as research focused, managed to entirely divorce Schein's three levels of culture from each other. They have artifacts they use for promotion, tenure, and measuring the school's activity. Many have ceased to try and argue those artifacts are linked to the values that they espouse. They simply state they are linked and then stigmatize anyone who points out otherwise.
At many schools, the faculty pathologically fear losing input on governance yet are unwilling to contribute to running of the university in a meaningful way. They complain about perpetually rising tenure standards, but they are the ones enforcing those standards. Trying to engage in these discussions is a dirty word because that would require them to confront things outside of their bubble. Trying to individualize isn't within the capacity of the faculty because ongoing disciplinary ossification, narrowing, and inlooking. It also isn't in the capacity of the outside managers they bring in to run the 'business' side - because they don't have the foggiest understanding of how education works, and get stuck on traditional metrics for business success and solutions that prevent any real change.
Its easy to blame sports, and they sure don't help, but the culture of the way universities are run - lacking in any reflection, self-awareness, or broader world view. It isn't complying with title IX, but thats a convenient scapegoat. It isn't building nice dorms, that's a symptom not a cause. Each school looks up the ladder of rankings and then does their damnedest to make themselves look like the school above them. In an attempt to 'compete' they all actively commodify themselves.
As an academic I find it sad. As an engineer/entrepreneur I find it frustrating. As a citizen I find it actively dangerous. In the broadest sense, these aren't just problems of education, they are problems of a society that (on the whole) seeks a black and white understanding of the world and seeks an education system that reproduces what it thinks it wants, rather than what it could be or might need later.
I think one of the biggest issues in faculty hiring is that research experience and a strong publication record outweigh any form of teaching experience.
But this doesn't make sense at all, simply because ~30-50% of a full faculty member's time is spent teaching (including prep time, office hours, etc). More importantly, the fact that you have a couple of publications in top tier conferences does not imply that you know how to teach!
>>I hear that the transition from full-time to grad school is pretty challenging. What were the issues you faced, if any?
Its...interesting. The big issue was ego. I'm sure I have one I'm not super aware of but I make a really itnentional effort not to talk out my ass. I sat in a meeting where people were describing the 'needs' of engineering employers. No one in that room besides me had ever worked as an engineer, hired an engineer, or managed an engineer. But because I was a grad student they didn't see my input as useful. That pattern, of role based power supplanting all, is commonly used. It is also a pretty strong indicator of competence - those who use it are less likely to be so.
I would add that adherence to process (but not formal process, always implicit process) is hugely valued in my experience. Even more so than when I worked in manufacturing.
Would you take a machine learning course from an unknown, likely outdated professor when you could listen to Andrew Ng for free?
What the internet has brought is a 'winner takes all' for education.
Compare this to when I tried to take an AI class online offered by stanford, and I had to drop it because it was impossible to grok.
The difference? One professor actually taught, and the other simply wrote inscrutable symbols on the board and expected you to interpret it yourself. When we informed the professor on our lack of requisite statistics background, the unknown one simply adjusted his lessons to introduce the necessary concepts, while the Massive online course simply required a thorough background in probability to watch even the first few lectures.
Independent learning can kind of scale, in half-hearted ways, but actual TEACHING does not scale. Real teaching requires one on one effort.
I went to a low tier university because I had zero chance of getting actual access to an MIT professor, and the local ones were good enough to teach me everything I needed to have a real chance in the field.
The schools don't create value for students because they are bad at teaching. Having all the materials online is not learning, and its not good education.
But, the fact that students can create a better learning environment themselves than can a faculty member who knows the content and works at a university teaching the conten should be shocking.
Schools always want to hire 'collegiality' candidates...who get along with folks. They have warped the definition of collegiality (https://en.wikipedia.org/wiki/Recuperation_(politics)) to mean doesn't buck the system rather than respectfully disagree in furtherance of a shared purpose.