
Ask HN: Beyond CRUD and ETL – How to Grow Professionally? - ugenetics
I have near 10 years of experience by now. I have worked on ETL , Data migration , CRUD applications.  I am well familiar with data warehousing and BI concepts and have worked on many projects involving both.  Since last 3 years I have been working with Big Data applications on Hadoop platform mostly writing code in Java and using abstract language&#x2F;platform such as Pig, Spark , RedShift , Hive.<p>I am tired of ETL , data cleaning , slicing and dicing projects now.<p>From business standpoint I have good knowledge of healthcare , insurance, advertising and little bit of finance.<p>I want to contribute to something big and grow professionally.  I work as manager for a mid size company in bay area.  I want to join next company as Director or higher level.<p>What are things I should learn to grow professionally ?<p>What are things I should keep tab on beyond technical knowledge for getting higher roles as Director or VP ?   How do I cultivate the change ?
======
dragonwriter
I think if you want a Director or better kind of position in the technical
side of any organization, what you need probably isn't necessarily more
technical "depth", but a demonstrated track record of delivering value by
proactively applying both technical and organizational/managerial skills to
solve business problems (or capitalize on business opportunities; two sides of
the same thing), preferably problems or opportunities that you identified for
your organization. (You'll probably _also_ gain knowledge, at a high level at
least, of all the technical areas involved in that solution in the course of
doing that, and you can dive into those more deeply if they are interesting to
you, but they aren't the things that are likely to be most critical at getting
the positions you seem to be looking for the industries you are interested
in.)

And, of course, the people skills to sell an organization on both your past
track record of doing that and your ability to do the same in their
organization.

------
davismwfl
You say you want to get to a director level or higher. Given the industries
you listed no technical skill is going to put you in the Director or Exec
seats at a larger organization. If you were talking SMB then yes, technical
skills could assist you to getting that next level, but even then it is more
about people skills, planning skills, budget management and most of all how to
affect revenue.

What I would say is that people are finally waking up and realizing the data
we collect is where the gold is at, not in selling a widget for $5.99. The
data tells us what to sell, when to sell it, what to price it etc. So
everything you know is driving business today, especially in insurance,
finance and advertising. So don't run from what you know, change your focus
into leading others to do the "tasks" of what it takes and switch your brain
to architecting solutions that make companies more money. Don't be a manager
of people, be a thought leader, an inventor that can turn an idea into action
and quickly.

As for what to do to grow. Seriously, if you don't know a lot about
accounting, learn the basics of how a company runs accounting. Learn how what
you are involved in might affect revenue and how you can personally help
affect revenue with your team. Stop just taking orders to create this next ETL
or report, and invest some extra time to figure out how you can invent a
solution (doesn't have to be fancy) that can help senior execs or help the
company make more money. That is how you can move up, even if it isn't in your
current organization you will be able to say, I thought of, designed and
implemented X that brought the company Y more revenue through Z means. Or I
created X report that allowed the senior team to see data in a way that they
weren't prior and it has lead to Y changes etc. As a last thought, go spend
time with sales teams, they will complain a lot about things they need and
aren't getting. Help them get some of those done and they will love you and
brag about it. They will also then want to see you move up to help them more.

~~~
astazangasta
>The data tells us what to sell, when to sell it, what to price it etc.

Does it actually? I believe this is conventional wisdom in Silicon Valley,
hence the billion analytics startups, but where is the actual paydirt? How do
we know that data analytics produces anything other than cool graphs?

~~~
davismwfl
Well a few points below, but probably the most important one is you are right,
a pretty graph is useless without context and a person to put it into
perspective, which is why doing the analysis and then selling the perspective
of how it impacts the business is critical and can help advance a career.

But for example, Wal-Mart. They use the data in near real-time to make a huge
amount of decisions, among them is pricing, what is selling, where it is
selling best and with a little work they know the price sensitivity for a
product in each market allowing them to maximize revenue. I think there are
some Wal-Mart devs on here that know far better than I do, if they can
comment.

Or another, Ford. I was on a team that built analytics for them that drove
their supply chain decisions in near real time based upon a huge number of
inputs and it saved them massive amounts of money by reducing supply chain
inventory.

Insurance analysis is done via collected and publicly sourced data to
determine risk profiles, rates and a host of things. So yes, the data is
driving what policies to sell, when its best to sell it and how much they
should charge to maximize revenue. In addition market analysis tells them
which policies to offer where.

A number of the big box hardware type stores use analytics to determine where
to move product and and what time of year to move it to maximize product
revenue.

This can go on and on. Silicon Valley is isn't really where I would look for
valid references to how this works in the real world, because I think to your
point their view is sometimes skewed towards producing the analysis but not
converting that into action. But enterprises use data everyday, and are
getting better at it to make these decisions. So when you can show a company
how to use data they already collect to make a better decision or maximize
revenue you can absolutely make an impact and help your own career.

------
jedberg
This is intended as no way offensive, but your written English isn't great
(I'm thinking it isn't your first language?). The first step to getting an
executive position would be to improve in that area.

The second thing about being a Director or VP is that it's all about team
building, not technical skills. Technical skills are a baseline, but a good
executive is an expert at attracting, hiring and retaining good talent. So
that is another area where you'd need to prove yourself out (you didn't
mention anything about hiring or recruiting).

~~~
ugenetics
That is correct. English is not my first language.

Just curious, since you found out from my writing style what particular
sentence stood out as odd to you.

Do you know any resources to assimilate to American English in verbal and
written cases ?

~~~
emmelaich
Briefly

    
    
        near -> nearly
    
        well familiar - delete the 'well'
    
        Since last 3 years -> For the last three years.
    
        I work as manager -> I work as a manager *or* I am a manager *or* I have worked as a manager.
    
        What are things -> What are the things
    

Beyond that there's some minor style issues. The misuse of 'since' is common
among German and Dutch speakers so that would be my guess of your native
language. I don't think any if this is a blocker for a director position but
get yourself a good partner / secretary.

~~~
dllthomas
"Well familiar" isn't too strange a construct, though removing the well does
certainly improve things here.

~~~
tonyarkles
"quite familiar" seems more natural to me. Also, the odd comma spacing
immediately jumps out at me.

~~~
dllthomas
Certainly "quite familiar" is more common.

------
arosenbaum
You are a manager but you didn't state anything in your story about the people
you hired and developed, the projects you led or rescued, or how you
contributed to your peers, managers and companies success. Do that.

------
GFischer
I don't think you'll get a role as Director or VP simply on technical
knowledge.

This is what the Harvard Business Review has to say (pretty light on
actionable advice though):

[https://hbr.org/2011/03/the-new-path-to-the-c-
suite](https://hbr.org/2011/03/the-new-path-to-the-c-suite)

Another article:

[http://www.computerweekly.com/feature/Career-management-
The-...](http://www.computerweekly.com/feature/Career-management-The-path-to-
CIO)

I guess an MBA could be of help, but it's useless unless you already have
connections and all the other requirements in place.

[http://www.informationweek.com/it-leadership/6-must-have-
ski...](http://www.informationweek.com/it-leadership/6-must-have-skills-for-
aspiring-cios/d/d-id/1103925)?

A recurring piece of advice is having been a consultant.

The other is building your relationship-building prowess.:

"Strong project management skills are critical to running a successful IT
shop. But if you're hoping to climb that corporate ladder, it's important to
know how to build relationships up, down, and sideways. In fact, according to
a recent poll from SearchCIO, conducted of 875 senior and mid-level IT
executives, CIOs who earn the highest salaries make building relationships
with top executives more of a priority than managing IT projects.

"Being a CIO requires an ability to develop relationships in all directions--
with your boss, outward with one's peers, with other C-level executives, heads
of business units and relationships downwards as well,"

~~~
gaius
This is a recurring theme in that article

 _she 's only ever written one program in anger and that was "functionally a
brilliant success but performance-wise it was lousy_

Once you have a reputation as "technical" you will _never_ be promoted to
senior ranks, esp. not in the UK.

------
memracom
Lots of IT shops nowadays are struggling to contain a BIG MESS while still
providing new features that the business needs. So the skills that are most
useful at director level and above, are how to cope with the BIG MESS and how
to extricate the business from the BIG MESS. Most likely this involves lots of
compromises and the usual techniques of managing in the midst of chaos by
evolving bit by bit to get closer to some overall architectural vision.

As far as the technology goes, there are many ways to have an IT
infrastructure without a BIG MESS. So technology does not help you here. What
matters most is deciding on an architectural vision and leading the whole
company towards that "better place". If you believe in silver bullets, then
you will fail.

If you read Peopleware and it made sense to you, then you are probably on the
right track. There is a huge literature on management techniques to guide you.
Pick a path that you can walk and just do it.

------
pkinsky
Stream processing is pretty cool. Instead of doing batch jobs try to set up
something real time using Spark.

~~~
ugenetics
Yes, stream processing is awesome. I will look into it.

------
ak39

      "I am tired of ETL , data cleaning , slicing and dicing projects now."
    

My instinct says that one has to love what one does (the dirty bits
especially) for one to evolve naturally into the next greater phase of one's
career.

------
sharemywin
Just remember this the hire up you go the more it's about sales. so with that
in mind. You need to network with people that can hire a director and impress
the shit out of them. Then, when they are looking for one... your the guy.

------
polskibus
Work on a product, instead of doing one-off projects for other companies.

~~~
ashmud
That's what we (company I work for) have done. 95%+ work we do is on our core
product. Made work much more meaningful.

------
yanilkr
How to grow professionally? I have no clue. I made some observations of my
own.

If you work at a company, you would expect the hierarchy to notice your
ambition and guide you to next level. In practice this does not happen or
happens to only few people for variety of reasons. The company does not
produce enough market value, there are too many skilled people with ambition,
there are no networking opportunities, you were not noticeable that nobody
took you under their wing, you are too odd to fit in, the industry is too
young etc. One could only prepare self to look good, talk the talk and wait
for an opportunity to arise. When I see so many managers ass-kissing their
bosses, I believe they are doing the same, preparing for receiving. I once
noticed an ambitious director making a spectacle while taking pictures of his
boss when the boss was speaking to an audience. I really have no clue if these
social engineering techniques work, but I notice so many people do it.

If you are perceived technically strong, the hierarchy does not see you as
leadership materiel. I witnessed this event that changed my perspective about
over stressing my technical abilities. A bunch of people, almost all of them
out of touch with technology sat down and the conversation started like this
with someone saying -

"I know this large company which takes this amazingly smart technical people
and made them managers and directors and they are facing difficulty, in our
awesome org we should never do that”.

A bunch of other people around agree and they give their own experiences of
horror stories. Such things gain wider acceptance because these meetings have
less representation from someone who still see themselves as technically
competent and current. I was in such a meeting and asked the group, if
leadership is a skill that people are born with and what should technically
strong people do for their career advancement? The answers were more
ridiculous. Technical skill is seen as something you only need to have up to a
limit and any more than that is not useful to the company. There is a ceiling
if you are technically smart.

The environment is self selecting what is needed for their business to strive.
I found it smart to adapt sometimes but then I see a lot of these young people
in silicon valley do amazing things and give themselves great titles. We both
know that skills are transferrable between companies but titles may not be. I
have more respect for these young people who create their own reality and make
everybody believe theirs. Now Zuckerberg is still a CEO and they are still
teaching people in MBA the skills to be a CEO or CIO or whatever it is and the
magazines writing about the rules to be a CEO or CTO or CIO.

------
Omnipresent
off topic. I am a step behind the OP. I am trying to get into the role that
the OP describes he is in for the last 3 years. What would be some ways to
move into big data and BI other than quitting my current job that doesn't deal
much with Big data. Is the solution to this to contribute to Open Source?

------
tmaly
how about learning something related to DevOps?

~~~
ugenetics
DevOps is cool but I find it too bothersome at times.

I personally like product design , development and system engineering aspects.

