Ask HN: Raising high level vision concerns as a junior, internal org takeover?

[jamboca]

Hello peers,

I am currently on a high visibility project at my company involving collaboration of many data scientists and data engineers in a non-purely engineering domain.

I am a non-senior data engineer supporting this project and i have a lot of thoughts about the direction of it, how things could be improved, and where we are going wrong. however, i don't have the space or title to be heard out. data scientists feel like they see the platform team as an enemy rather than a power multiplier. additionally, the senior platform engineer supporting this project is spread thin across many projects and doesn't have their hands on the data. yet, they are directing us on how to work on this project. i get very annoyed working with these people as i see and know what the project needs, but instead i am just instructed to follow the lead of a senior engineer who is not really in touch with the needs of the project but thinks they are.

currently we use a complex pipeline which i was instructed to build. i am the most in touch with the underlying data and intimately understand the needs on both the platform and DS side despite not receiving super concrete requirements from the side of the "scientists."

i would like to implement some visionary functionality, high level architectural differences in parallel with current development/platform support work, and then one day say something like "hey, i built a copy of what we are doing but faster, better, etc. here is the proof and let me know what you think." however, it would basically automate the jobs of the people on the platform side, making enemies with at least a few people and especially the lead engineer who makes these calls which i disagree with. at least, i would like the stakeholders to see how our current approach is hindering results and how i have a vision for a better one. however, it's not my place or my title to do so.

i want to just show a director or our manager that i can fix this project without doing it in a hostile way. but i don't see how to do this given the organizational structure/conditions. would it be better to leave and go to another company in hopes of a better atmosphere where i can be heard out for my ideas?

Stuck, Data Engineer

[benoau]

They are executing something they have planned and budgeted over many meetings and if everything is going on-schedule then everybody accountable for the project is in the process of chalking up a win on an important project - nobody is going to want to radically change the plan now.

[jamboca → benoau]

But what i am saying is like what if i showed the director that i can do the work that a team has taken months/years to do. And is still failing to do. Seems like best way to proceed is just put my head down which sucks and makes me want to leave

[JojoFatsani → jamboca]

Well if you put your head down you’re doing what a good junior does which is just sharpen your technical skills and learn best practices cs anti patterns. You can have some good war stories in your future interviews.

The way to effect this change is to psy-op your teammates/bosses into thinking they came up with it.

[benoau → jamboca]

If the consensus is the project is failing share your thoughts on how to turn it around in your next one-on-one.

If the project is not failing, talk about how you would have done things differently in your next one-on-one and make it be known you'd like to be part of the planning next time.

It's normal to wish things were tailored more to your preference, it's normal to look back on how something was done and see a better way it could have been done. If you have retrospectives that's another good place to talk about these thoughts.

[pavel_lishin → jamboca]

> Seems like best way to proceed is just put my head down which sucks and makes me want to leave

It sounds like a bad culture fit for you, to be honest. Nothing wrong with looking for another job; you're not likely to be able to steer this ship around, if nobody is interested in supporting you.

[panny → jamboca]

You're working where they want cogs, interchangable cogs. They don't want you to build something special and be the only one who knows how it works. You could leave, and then they are screwed trying to support the thing you wrote. It sounds like you've already done some of this by your own description. You're not a senior, and you're busy trying to prove you are senior material, right? But if you really want to be senior, you need to do dull things that you are skipping. Documenting, collaborating, keeping everyone in the loop, sharing all the information you have as best you can.

Want to really be a senior? Think this way: Everything I build as a senior developer is an attempt to make myself obsolete. I should be able to give a two weeks notice and walk away with no harm to the company. When I build things at work, I never worry about job security, not even a little. If there is something I built that they can't support without me, I've failed.

And you will ultimately know you failed when you can't even go on vacation without carrying the company laptop with you. I know you're proud of that thing you built and how you're the only one who knows how it all works, but that's the core problem.