Hacker News new | past | comments | ask | show | jobs | submit login
The Time I Lied to the CTO and Saved the Day (grumpyolddev.com)
507 points by mundanerality 11 days ago | hide | past | favorite | 389 comments





If you are a "delivery driven" person who cancels holidays to keep working, I would say to you, from own experience: stop being stupid.

I know, that, specially when you get recognised for the hard work it is hard, but you will regret every minute.

If you work for a company that counts on your holidays and vacation to sell their products, you are helping to create the problematic world with have today. If many do that, more will need to do. If none do it, and act like it is an absurd to request it (and it is) nobody is forced to do it and the company is forced to do real estimates even if that hurts your fav CEO pocket


Unless you are getting the economics of saving the day (you are a large equity holder / partner) then structuring your life completely around work is a fools errand.

Communicate your holiday plans, arrange coverage, put it team calendars, do hand-offs.. and then take it. Projects come & go, dates slip on their own. If you move holidays for project dates you will never take holidays.

The only exceptions would be if you are in a role with seasonality and its generally understood end of year / end of quarter / tax time / something is well known busy season and its in poor taste to disappear during it.


> Unless you are getting the economics of saving the day (you are a large equity holder / partner) ...

Even if you are a large equity partner it's at best short-sighted. I have written about this before[1] but the gist of it is that even when individual heroics saves a project it has robbed the project manager of valuable feedback regarding how the planning was deficit.

[1]: https://two-wrongs.com/hidden-cost-of-heroics.html


>The only exceptions would be if you are in a role with seasonality and its generally understood end of year / end of quarter / tax time / something is well known busy season and its in poor taste to disappear during it.

This is exactly how you get guilted into working during vacation; everything is sold as an exception.


The seasonality is not an exception. It's not a case of "just this once". It's a well understood thing of "every year, before the tax deadline we have crunch-time, so don't plan on going on holiday in that time". The repeated, unplanned, "just this once" exceptions are a whole different thing.

If your whole year depends on you having a product ready to sell on August 1 and you don’t finish it until the middle of July, then the company fucked up. If the whole enterprise depends on that date, then the thing should be done well ahead of that date. The remaining time is to handle training and documentation and shipping. Not still working on planned features, or doing bug fixes where the devs played chicken with the drop dead date.

These are all management failure, because nine times out of ten you ran out of time because some manager or exec had a great reason why his change was so important that we should break the rules to get it in. Or that what he said isn’t what he meant and now we need to fix it no matter what. This is the sort of exception that makes me upset.

The problem is these children in suits don’t understand that “we have 9 months” means you plan for six months of work. Tops. They can’t stand the “waste” because none of them have read a real management book since college, and they only skimmed it for the test.


We're talking about real seasonality, not "we have to launch this quarter" fake board room seasonality.

If you want to do people's taxes for a living you have to acknowledge that Feb/March/April are not times where it is appropriate or acceptable for you to schedule a 3-week vacation. This is not the same as "we do database updates in August so no summer holidays."


I thought we were talking about real seasonality? A tax deadline arbitrarily chosen in the legislative chamber is as much fake seasonality as summer database updates or product launches.

If that database update is well know to be every August or that product launch is something like Apple's WWDC then sure, that's the distinction people are making.

If it's just we want to launch our latest product in the second week of June, because that's the one-off date we came up with when we started the project, that's different. It's not seasonality it's just failing to account for staff availablity/unrealistic planning.


If you miss your database update or product launch deadline you can't get arrested for it (in the vast majority of cases). Yes, all deadlines are arbitrary, but the consequences for missing them are different for different deadlines!

Being arrested is still just the arbitrary choice of people. It's not real in the sense defined earlier. It is not like, say, needing to harvest food before winter sets in.

Look, I love Baudrillard as much as anyone, and we can talk about simulacra all day, but it's simply not germane. We're not talking about whether these restrictions are societal constructs but rather whether they are predictable, reasonable, and controllable.

The tax deadline is predictable. A two-month restriction from holidays is relatively reasonable. The deadline is not realistically controllable. (Your employer cannot eliminate tax filing.)


Stop being deliberately stupid

How is one of those more real? They are both actions that people undertake to accomplish an objective, but there is nothing tangible or factual about either, which appears to be your working definition.

What makes you think any are more real than any other? If you had taken the time to read the thread you would have learned that the whole idea of a fake deadline is nonsensical.

No, only your idea of it is.

Huh? I have no idea. If I had an idea, why would I spend my time on a message forum with it? You have clearly not thought this through.

You had the idea that "the whole idea of a fake deadline is nonsensical". That could only have come to you because your idea of what a fake deadline is is wrong.[1] Because they exist, so the idea of them is not "nonsensical". HTH!

___

[1]: So that's at least two ideas that you obviously have; putting the lie to your "I have no idea".


The classical notion of "idea" would exclude the output of software, no? Perhaps you are redefining "idea" here?

"I thought we were talking about real seasonality? A climate deadline arbitrarily chosen by the Sun and Earth's orbit around it is as much fake seasonality as any whim by a corporate executive."

Yeah, that's idiotic. But only about as much as your version.


I don't know if it's normal, but my sister is a CPA and while tax season is a thing, so is working 1-4 hours per day in the summer.

That's really the key. I don't resent working into the night if I'm a) engaged by an interesting an worthwhile problem and b) know that I can cut out early next week and go for a bike ride after school with my kids.

Endless crunch is a problem, and I don't crunch to meet made up internal deadlines or for work that is administrative nonsense.


In fact if you are working on a sector with high seasonality nobody is going to have to skip vacation during holidays, because those days you are not going to have vacation to begin with

I think the point there is vacation schedules can differ wildly from the seeming social norm depending on the occupation.

Some lines of work simply require you to be working when everyone else is on vacation.


Right it's not about moving holidays its about planning them for a reasonable time if you have strong seasonality to your job.

If your job is some giant e-commerce retailer and you probably shouldn't take Black Friday or the week before Christmas off. If you work in accounting or some operations role then quarter end is likely bad. If you work in tax, then late March thru April probably bad. Etc.

There's plenty of time in the year to take off around whatever your jobs regular busy season is. Don't set yourself up for failure.


This is fine IMO if you get compensated. For example my old job I got double baseline pay for Sundays and holiday work. This was fair.

Nowadays working in software I sometimes do stuff on the weekend - but only because I know I can take longer breaks during the week since part of the work is already done.

“Good enough” is my goal and it should be generally. IMO Norway has the best working culture in that regard, although I only know if from the outside/stories.


> structuring your life completely around work is a fools errand.

Structuring your life completely around anything is a fools errand — work, video games, family, food, etc.

Unless it’s not. Maybe the only thing you care about in the world is your family and you’ll ignore everything else. Maybe video games are your truest passion and you’ll do whatever it takes to play them 80 hours a week.

Maybe I love CRUD apps and Jira as much as my next door neighbor loves his wife. Who’s to say?

If you want to devote your entire life to one thing, go nuts. It’s probably going to have some consequences whether that thing is work or not.


I structure my life around my gym routine, my sleep schedule, and my main hobby/passion!

Its useful to do the math around what your equity would be worth, if you are the 50th hire you are probably getting less than .1%, so even a crazy moonshot billion dollar acquisition would leave you with a few 100k at most. Grinding yourself down for a slim chance at making that amount isn't worth it.

I have had managers say things like: "this could be the last job we ever work" as if none of us are capable of doing math.

That said we are all a bit prideful so exhausting yourself to get that fleeting feeling of having achieved something clever and good is definitely on the table.


> this could be the last job we ever work

I interpret that as "This is the job that kills you."


This is why I said large equity holder.

Most startups unless you are founding team this is not the case.

Certainly sub-1% ownership is far from large.

A situation where you have sub-$1M equity that is illiquid for a decade until a hoped-for exit is not large equity.


> exhausting yourself ... is definitely on the table.

No. It's not.


Companies will also never remember your hard work when it comes to layoffs. My cousin worked until 1am almost every day from September 2023-January 2024 to complete a poorly managed project. He only had Christmas off because the director thought employees would sue as it is a religious holiday. He missed important events and lost about 20 pounds, because of stress.

The company had a hard deadline to move over to a brand new system. They had 5 years notice and decided not to start it until the year before. If they didn't make the deadline, it would mean millions of dollars in costs because it was outside the contract.

Him and his team made the deadline. His reward was being laid off a few weeks later as his position was eliminated. Now he's over 50 and trying to find work in a terrible time to look for a job.


The company didn't remember his long hours, but I bet his family will remember it for a long time.

This is awful in so many ways

The messiah complex is rampant with both developers and operations people.

The are addicted to feeling valuable and important.

When they get home from work, they have nothing else to do.


It’s not that I have nothing else to do. It’s that the dopamine hit of fixing a thing that affects my team often outweighs any other choice.

Working out frequently, getting a family, getting a side project off the ramp, and reducing to part-time have all helped me with my addiction to feeling valuable and important when nothing else is gained than becoming known for fixing stuff in weekends free of charge.


> It’s not that I have nothing else to do. It’s that the dopamine hit of fixing a thing that affects my team often outweighs any other choice.

I know first hand about the "fixer" mindset that a lot of engineering-minded men have.

Do yourself a favor and fix things mostly for your family/friends, and your company a distant third. Every single person will be better off in this scenario. It's a hard learned sometimes counter-intuitive lesson, but it must be learned.


I will help you. The truth is that you are an asset. Even if they make you feel important, you are as important as a good chair. They will replace you without thinking twice if they want.

This is not a great take. I have plenty of things to do outside of work and I will still do what's "right" in my mind, especially if there's enough evidence that it will make my and other people's life easier in the long run.

There are plenty of people like that in a company but they normally don't cluster together, so it feels like "nobody else cares". That's false. At times that's how exactly how products, or even entire companies, are saved. Because a bunch of individuals who think like that get together and work hard to accomplish something.

Whether that effort is recognized or not depends on the company. If it's not then, I agree, it would be irrational to keep pushing. But you can't know until you try.

(Edit: to be clear, I'm not advocating prioritizing work over anything else, that really depends on you, but there's a fine line between "working hard" and being obsessed. The latter will more often than not result in a net negative in most cases. The same level of effort and/or prioritization should be applied to all areas of life)


I come from a country where work culture has gotten to the point where no one cares. It's a self-fullfilling prophecy of delays, lies, and lower compensation.

The correct approach is to be flexible, recognise that your family is worth more than the company, but still give a shit about your work and getting it done.


My hack was to start my own product, that way (at least for two hours a day) I get to play messiah for 100% of my own equity

Own product is great, although it becomes difficult to justify bikeshedding variable names and a possible 3rd Rust rewrite when you should be grinding out more hours on marketing and sales.

Or writing documentation for your product.

[flagged]


Yeah, doing alright - might end up employing me eventually.

I think this is due to my environment. That 'Insecure Overacheiver' idea resonated with me.

Had 1-2 childhood events that shaped you, and now you are afraid to do anything except 'be high value'.

I'm aware of it, I'm even aware of philosophies like Absurdism, good luck changing me. Its either in my DNA or faux-PTSD.


I heartily agree, but it's poor form when the threshold for Messiah is so low.

I remember a company meeting where a series of employee recognition awards were given out, and the story for about four awards in a row was an employee who spent nights and weekends heroically fixing a giant mess, at least enough to make a deadline. In a couple of cases the employee who got the award was also partly responsible for the mess. After about the fourth one the top managers in the room finally got it that the company had a problem.

I remember an awards occasion where the employees who received the recognition for actually heroically fixing a mess could not make it to the stage because they had been laid off.

Awkward silence ensued. ...


this is probably why they are grumpy now :)

>If you are a "delivery driven" person who cancels holidays to keep working, I would say to you, from own experience: stop being stupid.

You should put that on a coffee mug and sell it. It would make a great gift to a bunch of people that I know.


[edit: I retract my comment that it's bad advice to not cancel holidays to work hard. I misread the comment that I replied to -- I had thought the commenter was saying don't work extra hard in your role as it's never worth it. That's not what the commenter said though. In my experience, people who produce more value in the world are more valuable and get rewarded more than those who don't.]

I think this is poor and dangerous advice for anyone who wants to get ahead in life. If you are happy to coast by and not achieve much in life, sure, don’t work hard. But if you want to be one of the few who either rise to the top in your field or to create value in the world, then don’t feel bad about wanting to work hard. People generally learn by doing and those who do a lot learn a lot. Of course, don’t prioritize it over things that are important to you (physical health, family etc) but don’t feel it’s wrong to prioritize it above stuff isn’t important to you (eg Netflix and YouTube shorts).


I think the difference between someone who "gets ahead in life" and someone who doesn't usually isn't the number of hours spent in the office. Usually its more related to how well you make decisions in your career, and the relationships you build along the way.

Go to therapy. Learn about yourself. Work on your communication skills. Figure out what matters to you and invest in it. (For most people, family and friends are high on that list).

By all means work hard, but be strategic about it. Martyring yourself for your company won't make people respect you.


I definitely thinks it's a factor. You really don't see CEO who worked part time during their rise to power. Maybe there is something to that.

Agreed but there is a big energetic difference between a future ceo personality working long hours vs a supporting staffer who is just sacrificing themselves inefficiently thinking someone will care. People care about results and the appearance of some effort. Long hours are a basic requirement but healthy emotional boundaries are what get someone to the top.

There's several game studios out there whose last two tweets were "we've just won an industry award for our game!" / "our studio is being shut down immediately".

It's worthwhile working for yourself if you do think you're learning, but in today's corporate environment loyalty is just showing your willingness to be exploited.


What do you mean by "getting ahead in life"? That you pass more time doing stuff at work so you are closer to your death without having done the things you like, spent time with the people you love and visiting the places that fill you with joy? Or do you mean money? I am pretty sure if you are good at your job and you are not in a dying profession there will be enough places that show you the respect you would extend to them.

Shilling for work place abuse and unpaid overtime isn't getting you ahead in life the same way begging for forgivness with an abuser will make your life better.

If your corp can't manage people's time realistically, why would you expect them to manage anything realistically? Get out and go to a real place that knows how to run projects.


Don’t ruin your health for someone else. People who throw themselves on this fire in BigCo don’t get recognition or reward, they get abused by the political animals that will take advantage.

Work hard for yourself to build your own. But don’t think for one moment that death marches will result in a swift rise to the top.


Working hard is the only reliable route to excellence, the tough part is making sure you don't get exploited along the way - 50% equity or walk.

This is the sound advice.

Nothing wrong with working hard, but make sure you get something about of it.

If you’re being asked to cancel a holiday because some clown can’t get anything done without setting realistic timelines, don’t do it.

But if you take a job where your boss says “we need to get this done by this date and we need people who are willing to get it done even if that means sacrificing work-life balance” and you get paid like you should for a job like that, then do it.


> get paid like you should for a job like that

Does that happen very often though?


I've been the person working stupidly long hours for many years. Eventually I stopped doing it, and I started to set boundaries around my downtime, and I found out something really valuable: Nobody noticed a difference in my productivity. I was doing far less, but nobody noticed. They all assumed I was still as busy, and doing as much, and it's because the things we all do to be super busy and productive ARN'T productive. You might ship some things faster, but the quality suffers, changes are harder to make later and everyone's tired all the time.

So no, "working hard" won't get you ahead. The people who really get that far ahead in life are the ones networking and learning the political games, not really working. And why do you need to "get ahead" to be successful? I'm ambitious, and I love growing in my career. I'm not someone who is happy with the bog standard things, but I've also learned that when all you value is being ahead of your peers, you miss out on more than you gain. And on top of it, you all end up in similar roles anyway.


One can work hard and still not cancel holidays for work. Plenty of people work through holidays and don't "achieve much in life". I've known many people in higher positions who use all of their time off.

> one of the few who either rise to the top in your field or to create value in the world

"rising to the top" is pure ego-inflation

"create value in the world" -- you'd better make sure that you're creating value for the world in something that's important and meaningful to you, not just "create value for shareholders" (who don't give a f about you) which is what "creating value" means 90% of the time if you're in industry


Work hard != Work on your free time

Achieving a lot in life is not necessary about working hard. Its more about working smart or having a bit of luck. None of the successful (money wise) people arround me worked really hard. The working hard sentence is really nonsense.

I don't think it's nonsense but it's incomplete. A lot of people work really hard and achieve nothing notable. A lot of people sort of phone it in and do alright just by shrewd positioning (or luck). There is no substitute for combining both though: work smart and hard.

The trick is to make sure you're always investing in yourself. Even when you're working for others you be either "learning or earning", ideally both.


>I think this is poor and dangerous advice for anyone who wants to get ahead in life.

Oh yeah, every new-comer to the tech world thinks like this, and 99.99% of them don't get any further "ahead" in life than the rest of us. In fact, many end up further behind.


Why should that person sacrifice themselves when they are happy working? Just so other people are not held to that standard? That seems like a management issues. Don't expect the same things from everyone.

If I won't do it, somebody else will. And we're back and square one.

For some people it is a chosen lifestyle I think.

With 30+ years of personal experience, my qualified statement to the next reader is that the above is 100% accurate.

Any impressionable kids reading... this story playing out like it did is highly company-dependent, as well as luck-dependent.

In a healthy company/organization:

* That outsourcing implementation approach probably wouldn't have happened, because an experienced person could've guessed how that would turn out, before it was started (it's such a cliche).

* Earlier on, people would tell the CTO it wasn't working out, instead of lying and saying the opposite.

* Whenever they needed to do something smarter/creative to rescue the project, they could work with the CTO on that, and maybe even with the customer.

* There wouldn't be marathon whip-cracking over the holidays, especially not after the team was already burning out.

* A manager/lead would go to bat for the success of the project and the health of the team, and insist upwards that the team needed a break over the holidays, if that needed to be clarified.

On that last point, in a "half-healthy" organization, a manager might intentionally be very vague, and omit information. That happens, and might or might not be a good idea, depending.

But the way this story is told, it sounds like the manager/lead outright lied repeatedly up the chain of command, which is generally considered very-bad, in both healthy and unhealthy companies.

I'll add that these things are vastly easier to armchair-quarterback. Certainly we're all going to make mistakes, especially when put in difficult positions, and/or when overextended/fatigued. But it helps to look at scenarios, to try to learn from them, and to think from a distance how they might've been handled better, so you're armed with that "experience", the next time you're tossed into a rough situation that has some similarities.


All of the bad stuff in the article is a consequence of the CTO not being able to call technical bullshit and encouraging a culture of yes-reports.

If you ever find yourself in a company like this, start looking for a new job.

It's impossible to fix rot that bad at the top, and they're not going to make you CTO.


The worst problem is, every success will be credited to CTO and every failure will be aimed towards the manager because it's a disobedience. In this case, the CTO also reward the credit to his friend because he/she doesn't know the backend.

In the short time you may save the company, in the long time you'll lose and the company will lose too.


The CTO certainly seems not the most competent.

"Because the CTO had a yearly turnover of his direct reports, every status call about the project took some variation of “great idea, boss” even though literally no one involved thought it was even a good idea. Or even a mediocre idea. It was a bad idea"

But if everyone confirms his bad decisions instead of pointing flaws out, then they are all part of the problem.


> But if everyone confirms his bad decisions instead of pointing flaws out, then they are all part of the problem.

They are part of the problem but they don’t have to be part of the solution. It’s the CTOs job to realize they’re not getting accurate information from below and work to fix it, that’s why they’re getting paid the big bucks. It’s not like it’s especially hard to figure out but building the trust to create an atmosphere of psychological safety is the job of executives and takes a lot of hard internal reflection but that’s what you sign up to when you sign up to do the job.


Thats the key. If a CTO doesnt set a stage of “look be fully frank with me” then nobody will just say “look bossman, we could have finished this three months ago.” When it really could have been less effort.

> The CTO certainly seems not the most competent.

Viewed from another angle, the CTO successfully psychologically manipulated his team and almost literally beat a working, cheap solution out of them, and will almost certainly get full credit for this victory done on the backs of others. SO, was he truly not competent? Or was he extremely, dangerously competent?

Not only that, but while the CTO is busy counting his bonus money and accepting his new shares of equity from a satisfied performance condition, he managed to get the rest of the team to simply be proud of doing it for a pat on the back and the feeling of doing "live theater" or whatever.


That is how I'd read it. I would actually say that OP got played, successfully.

Yep, the whole article was infuriating, yet I kept expecting a big windfall for the team in the end to justify it all, but instead we got:

  "But you also want to feel like a rock star when all your hard work hits real users for the first time and you feel that thrill of I did that. People like what I did. I overcame the impossible."
Y'all got rewarded with an attaboy and a 'rock star thrill' while the CTO probably blasts off in his career and collects his $millions. Well done!

>> every status call about the project took some variation of “great idea, boss”

I read that in two ways.

1. The CTO / management let that fly, instead of reflecting and challenging people when they repeated receive nothing but agreement.

2. People were apathetic, which means they weren't really engaged in meetings, which means the meetings were the wrong format shouldn't have been happening.

Good companies encourage engagement (say the uncomfortable thing), empowerment (servant-leader), and reality (if anything is said that's not true, say something).


That is true, but the times I have seen such dynamics happen it was usually do to... emotional instability. If every time a subordinate brings you anything other than stellar news you get all emotional, angry and start blaming people guess what:

Nobody is going to give you the truth anymore — as A) it seems you can't handle it anyways and B) there is no incentive for them to give you the truth while there are many incentives to not give you the truth.

This is just bad leadership. The most important thing to make good decisions is correct and meaningful information. If you punish people for telling you things that you don't want to see, you will from now on never have a good picture of what is going on. Good luck making decisions that way.

And this isn't even rocket science, it is basic common sense.


Yes, that is bad leadership. But the article does not say that this CTO did this:

"If you punish people for telling you things that you don't want to see"

It is implied, but I rather see direct evidence of a culture of yes saying and grumbling behind the back, something I learned to hate. Most people in reality do not face a existential crisis, if they speak up and the boss gets mad. It might cause inconvenience, but this is already enough for many to just nod along and venting out that frustration later. And wondering why nothing ever changes.

Those things work both ways.


Agreed, this is all speculation and I have seen corporate culture that worked despite bosses like that, but these were also corporate culutures which would have worked even better without the interruptions and "great ideas" from the top.

The main thing a boss can do is ensure the incentives for the behavior you want to see are there and there are disincentives for behavior you don't want to see. And with this over time corporate culture can be changed. But the peculiar thing is that you are part of corporate culture, and a part with special reach and importance — so you better act as you say.

And this is rarely the case.


>But if everyone confirms his bad decisions instead of pointing flaws out, then they are all part of the problem.

The nail that sticks out gets hammered down.

If I'm in that position, my first priority is keeping my job. Correcting my boss in a culture where they are not accustomed to that is not helpful to job security.


I report to a CTO and certainly can point to people who are more like this, but even the most agreeable still pushes back sometimes. If you understand how CTOs think and what they want/need, it's not someone to poke holes in their plans, but propose adjustments and alternatives that fill the holes you see. It's almost like good improv

Also CTOs (especially in supposed F500 companies like presented here) don't typically drive the project execution and interact with the development team in a more than cursory basis. THis is another reason I'm super-skeptical of the entire story. The smallest company on the current list is over 16K employees.


Depends on the problem to solve. If they are trying to solve the problem of improving the CTO leadership skills, then yes, they are part of the problem. If they are solving the problem of putting food on the table, like most people are; then you might argue that this is sub-optimal, but I don't think putting their heads on the chop block is a better way to solve it.

> "Because the CTO had a yearly turnover of his direct reports"

Yet another red flag entry. Employee turnover should not be this high. It should be the #1 metric for detecting bad managers.


I worked at a shop where 2 layers of management decided they didn't like to see red on RAG status reports and wanted to discuss "changing the definition of done" for projects to preclude you know.. actually shipping them to prod.

That is when I ramped up my job search to 11.


> encouraging a culture of yes-reports

I get the impression that most large corps are of this culture at this point


Even small ones.

I have run into many people that work for companies, where vendor relationships are based on That Guy With the Really Good Coke at Burning Man, or, in more staid organizations, That Guy With the Prada Pimp Suit at the Country Club.

Many times, the higher-level managers are working on a culture, where they are The Big Decision-Makers, With A True Knowledge of The Big Picture, and everyone else is an interchangeable peon.

This is usually reinforced by their peers, and the culture is embedded like a tick. Pretty much impossible to dislodge, without damaging the entire company.


>I have run into many people that work for companies, where vendor relationships are based on That Guy With the Really Good Coke at Burning Man

This is the cynical view, but you're right.

But why does this surprise developers? The more optimistic view is: people want to work with people they like. I want to work with people I like.

There's a trope in the tech world of the ornery IT guy that everyone tolerates because he's smart. Well, a lot of times these people aren't as smart as they think they are, and will be the first to be replaced when it's possible. Life is too short to work with these kinds of people.


I believe in teams. I was a manager, for many years, and having a strong, healthy team was of paramount importance.

My team was pretty damn good. Almost everyone in it had decades of software development experience, but it was also difficult to manage. I had to deal with every member, individually, and had to sometimes be Bad Cop, but it worked.

When I interviewed candidates, I don't think I ever made a technical mistake, but I did hire people that broke the team, and they didn't last.

Teams are how we do big stuff. Individuals can be extremely productive, but there's an upper limit to how much they can do. If you can get a good team working, there's really no limit.


Could you elaborate what you mean when you say they broke the team?

Generally, it was "personality fit."

Being self-centered, dishonest, not taking responsibility, blaming others, etc.

Each person had to be very reliable, and that included admitting challenges, and asking for help, as long as it wasn't asking all the time.

Selfishness, where they would not compromise for the team, was a dealbreaker.

I gave each of my engineers a great deal of agency, and expected them to deliver, as opposed to having to ride them. They were grown-ups, and I needed them to act as if they were.

Personal Integrity and Honesty was a big deal for me, as was a sense of accountability.

Most managers "cop out," and only hire people that "fit the culture."

The problem is that homogeneity breeds mediocrity. If you want good, innovative stuff, you need to hire and manage people that don't "fit the mold." That's a challenge.

Everyone seems to get caught up on technical merit, but a good tech can generally be trained to do anything. During my tenure (almost 27 years, 25, as a manager), we went through many iterations of technology, programming languages, etc.

When we want a good, heterogeneous team, we need to hire for team cohesiveness, as well as technical merit. Almost no one we hired was able to just do the job, out of the starting gate. The tech was too specialized. We needed people that could be trained, and that would stay around (When they rolled up my team, the person with the least tenure had a decade).


>Even small ones.

Which points a larger problem: The vast majority of people are incapable of wisely handling the complexities of modern life.

>Pretty much impossible to dislodge, without damaging the entire company.

You are correct, the only way a good company can be built (if it can be built) is ground up.

In practice this does not happen. A sort of boiling frog phenomenon. The creep sets in because the founders hired or promoted people who have better soft skills (and sometime low on principles and hard skill) over people with hard skills. Sieving and assessing multitudes of prospective candidates for a job is a very time consuming, exhausting work, and people are often willing to settle for less, especially if the organization is really successful and the founders feel the need to expand rapidly.


Nope - Well run companies want to know the truth, expect mistakes, and want people to learn from them. In most of the teams I have been on, being a yes person is no t rewarded because yes people cannot deliver.

Most large companies aren't particularly well run.

>Most large companies aren't particularly well run.

What does it mean to be "well run" if our largest and most successful companies are not that?


They were well run when they were small, then as they grew large they could live on just being large and dominant and started to accrue organizational debt like this. After a couple of decades almost no company is particularly well run, they grow until they are no longer well run or until they captured the entire market.

Well, I'd suggest you start by reviewing your idea of "success" and how companies achieve it.

They deliver pressure, crunch and obedience. There are no well run companies. The defunct is part of the process as a load bearing pillar.

What is the size of your company?

Mine has 1.2k people and is the same. Very unionized, it's the IT part of a big energy company.

> In a healthy company/organization

This mythical beast, where can one find one? I haven't, in the past 25 years of work.


They do exist, temporarily, in the space of 10 to 50 employees, and are often led by people who have been burnt in the past.

Scotland. The home of the true Scotsman.

Former company of mine got a board member who pushed his "solution" on us, and it fucked up so much of our ordering processes, things we didn't even SELL were being shown. It took a year and a half to fix this, meanwhile it also meant we were unable to reverse a sale because "underwear" was shipped out and we can't cancel it (meanwhile, we don't sell underwear, just networking services). So we had to wait for the system to close out the ordering process and THEN they could assist the customer. Of course, that board member left a year later, Probably very pleased with himself for suckering our idiot CEO.

But that's ok cuz I got laid off of there, and I have no sympathy for that bumbling company. Idiots trying to outsource "solutions" that only added more pain for everyone.

Sometimes these are actual problems that need solutions but half the time ... It's just more crap to make themselves feel like they're "saving money but not building in house". Yeah just now you have to pay contracts to support the crap you never built in the first place. Good job. Golden parachutes for all the leaders, I guess.


Where do you find this mythical organization you call a healthy company?

To be honest, I'm wondering where all these cartoonishly unhealthy companies come from. I've worked in a bunch of companies in my career, and sure, at pretty much all of them at one time or another I may have thought "are you f'ing kidding me?", but that's really just the nature of large organizations.

And perhaps I've just been lucky, but in 25 years in tech I've never seen the level of gross incompetence described in this post. I'm truly envious of the vast majority of senior leaders and execs I've worked with - not because they're geniuses or anything, but because they excel at things that I find very challenging (and I know from my stint in management) and I learned a lot from them. Again, not everyone, but I've certainly had more good bosses than bad.


> To be honest, I'm wondering where all these cartoonishly unhealthy companies come from.

Simply take the norms of one industry, and apply them in a radically different industry.

Take a manager from the construction industry who knows a bricklayer with an assistant can lay 2 tonnes of bricks in an 8 hour shift, and if they didn't it's probably because they took a 3 hour lunch break, and apply it to the software world.

Take a manager from the food service industry, who expects workers to clock in before their shift starts, and that a worker who's even two minutes late is letting down the team and needs immediate attitude adjustment.

Take a manager from precision manufacturing, where Zero Defects is the norm, and "bugs" don't exist, and failing to deliver precisely what was promised is a big embarrassment.

Take a manager from the call centre industry, who thinks if you take a lax attitude to sick leave people will start falsely calling in sick all the time, anyone who calls in sick should be interviewed by HR upon their return.

Take a manager from a paperwork-heavy industry where work is simple but precision is important - like data entry for paper forms, where a worker who makes even minor typos just isn't cut out for the work.

Take a drill sergeant from the army who knows the most important thing in inducting a new employee is yelling in their face and bullying them, thus letting them bond with their peers....


Man I didn’t know all my managers had side gigs

I didn’t know that my manager had such an extensive resume.

I definitely agree with that. Probably the worst company experience I've had was with senior leadership who fundamentally didn't understand how software development works.

But still, I've never experienced a CTO who was that clueless. Other members of the senior leadership team, certainly, and I've certainly seen CTOs who I thought were poor, but never really CTOs who were as clueless as the one described in this article.


> where all these cartoonishly unhealthy companies come from.

From MBA people.


Let's not kid ourselves, engineers can and do fumble management matters

probably more like too entrenced in their trade and don't focus on other ppl.

There are effective and ineffective MBA's.

Except MBA claims to be a signal of management capability. With that prior assumption, it is a catastrophic failure.

Jack Welch was chemical engineer…

Perspective matters IME. I held both perspectives, this org is dysfunctional and healthy, at the same position, organization and exact same circumstances.

In one I was sure I was right and in the other I entertained the notion I actual might not be and things are not that simple.


Yeah, I've generally had good bosses and colleagues, including some outstanding great ones. (Actually, I had such overall good colleagues earlier in my career, I was totally unprepared the first time I ran into someone dishonest. It took too long to believe they would behave like they did, which ended up extremely costly.)

Despite overall good experiences, I've heard of dysfunction like this article describes, and even worse, in numerous real-world companies. Talking about particular instances can be very delicate when you have insider info. But I think there's enough frequent dysfunction in industry, and some very common tropes that we keep hearing from people at other companies, that senior engineers will tend to be able to immediately recognize some of it.


My experience matches yours. I have had very few bad bosses and almost all teams I have worked on have been healthy. I have been on a few bad teams and groups. They usually failed. They were not bad because they failed but because lying was rewarded, political skill was rewarded, and solving the customer's problem was not valued.

When I see questions like "where are the healthy companies?", I think either the poster has been very unlucky, or the poster might be the problem. When I say the poster is the problem, I mean they typically fall into one of the following buckets:

1. The person is very critical and cannot accept humans for what they are. They demand perfection, demand their coworkers are the best in the field, etc. They may also minimize the positives.

2. They have a very cynical or negative outlook.

3. They do not like their field (computers, sales, accounting, medicine, etc.). As a result, they are always unhappy.

The main point is something inside the person causes them to view every organization as screwed up and awful. This includes organizations which are OK, good, or even outstanding.


4. For myself, I was incompetent as a developer. Therefore I always landed in dysfunctional organizations. I was invested a lot, but in learning the wrong things.

Since I became competent (it was 2006, there were no Youtube tutorials for everything), I work with awesome people. It also means my managers at the time didn’t coach me properly (unsurprising for dysfunctional orgs). Life is much easier when you’re on top of things, and much harder when you’re unlucky. Unluckiness compounds.


Understand that luck works in the same way as "unluck".

Your experience is not a scientific experiment so you also have to consider that you might have been lucky. Perhaps other people have been unlucky.

For example, I could paint your post in a negative light: "A poster who blames the victim perhaps wants to feel good about the company they work in and ignore the experiences of others, or perhaps they are now in a responsible position and don't want to think that they might be part of a problem."

This would be unempathetic but so is trying to blame the people who describe their bad experiences.


Have you mostly worked for younger tech companies or were they "traditional" businesses who had to be forced to adopt technology? The places I've seen with the biggest challenges are all large old and well established companies. Think insurance, transportation, agriculture or healthcare. Places where the people who managed the entire system out of file cabinets a decade ago now manage the document repositories they don't understand and they are still bitter about it.

The insurance company was known for hiring contractors and keeping them on the bench for months to years just in case they needed them. I also remember while working there a particular Big Three consultant kept showing up in people's meetings and never speaking up. No one could figure out what he did or why he was there, but the agency was billing $500 / hour for his time and no one could figure out how to get rid of him. It was a complete shit show. This doesn't even touch on some of the major technical blunders they made throughout the years. Just a few small personal anecdotes.

The transportation company was even worse. Literally the worst company I've ever spent time at. It only employees around 15k people, so it's quite a bit smaller than the above insurance company. All signs indicate that this company was well run by the founder. But not by his sons who inherited it. They had a driver turnover rate of over 100%. If they needed 100 drivers for the year, they would have to hire 105 drivers throughout the year. They just constantly churn through brand new drivers, train them up and lose them to other companies through incompetence. One of the consequences of constantly burning through brand new CDL drivers is you have higher accident rates. The project I was brought in to consult on was a driver monitoring system. So if the driver braked too hard or swerved too fast it would create an incident and the driver would have to talk to someone after their route to explain what went wrong and how to prevent it in the future. I can't imagine why they had turnover problems! Their entire IT org was run in a similar dysfunctional way.


Don't get a job at IBM. ;)

It's aspirational, and people who aspire will achieve it to varying degrees.

For starters, we can all aspire to work in a company where people wouldn't be outright lying, nor feel that they needed to.


> we can all aspire to work in a company where people wouldn't be outright lying, nor feel that they needed to.

I think you should consider this might be antithetical to the very concept of a company - although don't misunderstand me to mean I think all companies are "dishonest" (even if many are).

"Working" in a "company" implying making some profit, usually implies something proprietary, which implies some secret, or with-holding of information/resources. So then, most, if not all companies, function off of scarcity-of-information/resources, and probably wouldn't be successful without.

Withholding of information or resources, is awfully close to the next step i.e. "lying". In fact, it is frequently requisite to "lie" i.e. tell half-truths in order to conceal/confuse/distort whereabouts, processes, etc. So while the stated goal of many companies may be transparency, this is actually incompatible with their own goals in many ways. And so this can be seen repeated internally, where "resources" usually imply "promotions", "sales leads", "job responsibilities".

This may explain why nearly every company seems to slowly become worse over time...


Many of the top companies you know were that healthy company once. You basically need to be one at that magical inflection point where the growth will crush you if your engineering is not empowered and on point. Especially back when clouds didn't exist, or were far less featureful, so turning dollars into horizontal growth was not a thing.

The dark secret is that being a healthy company is just a moment, not something a company is at all times. Staying a healthy company as you grown when you are actually successful is very hard, and once the health is gone, good luck regaining it, because now you have a lot of people that thrive in unhealthy environments.


>being a healthy company is just a moment

Bang on. This is why I find these other comments which amount to "work in an unhealthy company? Just don't!" to be so naive. You're only ever one departure away from a shakeup which can totally change your work environment. If you aren't equipped or prepared to play the big game, then your options are

A) suffer

B) leave and roll the dice on the next joint


So true, when looking back at the great companies I've worked at, none of them are still like that any more.

I have close to 15y of experience now, and I've mostly only worked on healthy companies - I can think of 1 that wasn't. I have worked in ~8 companies give or take.

That seems quite a lot of changing the companies. Why so frequent switches?

Not OP, but I have a similar track record. Frequent switches because boredom, better pay (especially the first half of my career), and always searching for that amazing moment of confluence where I'm the dumbest guy in the room and working on an incredibly interesting/complex problem. Two years is about right to tackle something important and deliver, and also feel out if there are other opportunities at the current company. It has mostly worked out for me. You get really good at on-boarding yourself and getting up to speed quickly. I sell my labor as being an expert generalist and have professionally worked with half a dozen different languages, numerous different stacks, in a few different industries, and at big, small and in-between sized companies.

I feel like this approach wouldn't be able to make me truly valuable at any larger company for example, which in this case would be the ones considered unhealthy? Because there's enough complexity that takes years to understand. I think as an engineer your ability to provide value climbs in multiples the more you understand the product and what the company itself exactly values, besides the tech. You can solve meaningless problems using tech, but if you understand what is exactly worth solving, this is when your value can skyrocket, especially the larger the company is. Because you will have the understanding of marketing, leadership and product people while having technical capability to know what can be done.

And also I feel like if it's better to switch companies every 2 years because of better pay, it implies that the current company is not actually healthy, because if they were, they would understand the value you provide, you should be able to provide more value at their company rather than starting from scratch in another company.

While I don't feel my current company is healthy, at least I feel like I've been able to climb through promos and compensation faster than if I were to switch every 2 years. I have been there for 6 years.


Maybe you found a great company from the get go, those exist as well !

My pay bumps went something like: 1.5x, 1.5x, 2x, 2x, 1.5x, 1.5x, 2x (note that I changed countries twice so some of the bumps of 2x were also for a more expensive cost of life )

In only one of the companies I stayed more than 2y and was pretty great, I went from senior to senior manager within 4y. Now I’m at 2y again at my current company and am pretty happy, unlikely that I will change again.

The healthy part described by the parent post was along the lines of healthy work environment, not pay. Apart from my current company I’ve never worked somewhere that would give more then 5-10% increases per year, they were more like 0-5


>Because there's enough complexity that takes years to understand. I think as an engineer your ability to provide value climbs in multiples the more you understand the product and what the company itself exactly values, besides the tech. You can solve meaningless problems using tech, but if you understand what is exactly worth solving, this is when your value can skyrocket, especially the larger the company is. Because you will have the understanding of marketing, leadership and product people while having technical capability to know what can be done.

Yes, exactly. I am lucky that I am auto-didactic and grok things quickly. This is what I meant by complex/interesting problems: those are the kinds of problems that management is interested in because those make money. The more skilled management is, the more able they are to recognize those opportunities and allocate skilled labor to make it happen. The friction and need for finding a new company is when you work for under-skilled management that don't.

>And also I feel like if it's better to switch companies every 2 years because of better pay, it implies that the current company is not actually healthy, because if they were, they would understand the value you provide, you should be able to provide more value at their company rather than starting from scratch in another company.

Those companies only deal with the market reality when hiring new. Yes, it would make sense to retain your valuable people and reward them accordingly in an equitable situation, but when your shareholders are pressuring you to reduce labor costs, it is too tempting to give only CoL adjustments in spite of record profit quarters. They bank on retention through other means besides monetary (inertia, "career progression" cult, etc). To them, labor is a resource that is fungible rather than the core pillar of their business. They simply won't value your labor appropriately without negotiation.

Starting from scratch is really only a limitation if you don't ramp up quickly. And that means building social capital and demonstrating clear wins early.

> While I don't feel my current company is healthy, at least I feel like I've been able to climb through promos and compensation faster than if I were to switch every 2 years. I have been there for 6 years.

And how broad or deep has your experience been in those 6 years? Any serious migrations/rewrites/stack changes/scalability challenges/greenfield? Do you really feel you've been challenged professionally in that time? Being on the new hire cusp, especially for new initiatives is where all the action is and what has management's attention and capital expenditure. Being a backfill hire on a feature factory or vendor implementation team is definitely not where growth is going to happen.

And have you accumulated enough wins to make your next interview process easier? Thing is one of the gigs with the most impact in my career was one where I was able to find new opportunities within the org to deliver value, but it was the confluence of smart, motivated people working on interesting/complex things. If you aren't getting that, your 6 years in one place doing the same thing is working against you because all tech ages and decays.


In the beginning of my career I changed jobs quite often for better pay. That gave me a lot of big increases. In one of them I stayed less than 6 months.

I mean, nowhere's perfect, but I don't think I've ever worked anywhere as dysfunctional as the company depicted in the article. That's really quite bad. If the place you work is like that, consider applying elsewhere; this is not typical.

> Earlier on, people would tell the CTO it wasn't working out, instead of lying and saying the opposite.

I have done this, also have brass balls and dont give a fuck. EDIT: I mean tell the truth, so rare to not hide the fuck up. See MS security memo.

> There wouldn't be marathon whip-cracking over the holidays, especially not after the team was already burning out.

Ha ha ha ha ha... Ok this is partly true. IF you work in a BANK or in a FAANG then yes. If you work in any company that has "startup culture" forget it. The thing is that skin in the game (real skin) will change your attitude about work pretty quick. I dont think there are many companies where you will get it and have the opportunity to see that.

> But the way this story is told, it sounds like the manager/lead outright lied repeatedly up the chain of command, which is generally considered very-bad, in both healthy and unhealthy companies.

Do you know how many times I have been the cowboy who got some shit done in the dark of night cause it needed to happen? Most of the people I know who are good have gone this route. Hell I have seen these sorts of things happen AT A BANK.

Bend all the rules past breaking but tell no one. As long as you aren't making a huge fiscal bet (aka more than the company or your teams value) then your gonna get away with a lot. You either do it and get promoted or you get a new job (cause you limited your ability to get promoted).


> If you work in any company that has "startup culture" forget it.

Depends what "startup culture" is.

I actually did an early startup work marathon over Christmas, to help pull off a minor miracle, and make a very hard launch date successful.

But during the same period, I also managed the tasking/planning to shield a key teammate who'd gotten sick, and take stress off of them.

> Bend all the rules past breaking but tell no one.

I think that depends on the company, and the kind of rule.

In some companies, rules tend to be there for a reason, and if you break a rule, you generally have to disclose that, and maybe discuss it (ideally beforehand).


Even the healthiest large companies do this sort of thing, I've worked in a few and heck I've sold to a few too. Low code solutions that somehow need a team of contractors to deliver business functionality, delayed projects that drop key integrations to deliver something on time but functionally useless, CTOs whose next job depends on giving work to supplier. Worst of all, organisations that insist that they buy not build but then have larger software teams than their suppliers.

It wouldn’t happen today because front end development is such a mess.

> Earlier on, people would tell the CTO it wasn't working out, instead of lying and saying the opposite.

That's the part of the story that gets me. Everyone is acts so gutless.


> Not my circus not my monkeys

Some ppl just dont care coz in one year they will be in a completely different place


> The vendor product stored every customer transaction as a json record in a giant json document.... Adding a new transaction involved reading the entire json document out of the database, then appending the new record to the end.

Does this sound insane to you? Obviously it should, but in the vein of "insane things" I was helping do tech diligence of a potential investment for a fund. The startup's users table also contained the "tickets/booking" data. Each ticket was a column. So if your most active user had 5 tickets in total history, you needed 5 columns. These guys had 500+ columns by the time I reviewed it, and were looking for an investment to "scale".

Of course, it's a solvable problem, but as you can imagine everything was designed in some backwards, janky way. That was just the most obvious "wtf" moment.

They did not get the investment.


> Does this sound insane to you? Obviously it should,

I dealt with the exact same thing in my 2nd ever job. Entire customer and product database (with plaintext passwords) stored in a multi-meg public-facing .js flat file that had to finish loading before the app could do anything (with early-2000s internet). And to top it off, the app was a single monolithic file, in a directory with names like index.1.js, index.final.js, index.newest.js, index.45.js, etc etc.

I had enough experience with best practices to go to our CEO and get the CTO fired, and went about rewriting it with git, mysql, and server-side logic with an actual architecture. And then, the windows server it was all running on got pwned into a porn server, and somehow it was my responsibility despite me never having seen this server and not even having admin permissions.

Man, my first few jobs were educational!


I heard rumor back in the late `90's of an e-commerce site that stored the prices for it's products in the browser cookie. Presumably a motivated buyer could go in and edit the cookie before checkout, although I don't know if that ever actually happened.

The one I’m thinking of stored prices in the HTML form as a hidden field. It trusted the client-submitted value when deciding how much to charge you.

The justification I heard for the ethics of enjoying the home-rolled discount was that it was similar to haggling. The store chose to accept the offer.

Not sure I totally accepted that, but appreciated the creative thinking.


Low hanging fruit that seemed somewhat common back in the day was not verifying prices of items on the backend. So a somewhat technical user could edit the price in the html field of the item to be $10 instead of $100.

Between this and everything being non https - what a time to be alive.


IIRC, an early version of myspace actually used a GET form for login. It immediately redirected, but if your browser was wide enough, and your eye was trained on the address bar and knew what to look for, you'd see the password flash by in plaintext.

why is that an issue? just in case someone is peering over your shoulder? the pw will be sent in the request body as plaintext on a post...

> just in case someone is peering over your shoulder

Yes, exactly that. That's why I highlighted the risk of someone looking in the right spot at the right time.

> the pw will be sent in the request body as plaintext on a post

Which is how every single login form in the world works, today, for this very reason.


This brings back memory: This was the case for a gold-buying website for the Runescape game in the 2000s. You could edit your cookies or other front-end facing information to change the price of items in your cart, so you could buy gold or items for much cheaper than the market rate. At some point, while the vulnerability remained, they started cancelling orders abusing this and manually checking the orders.

I think you could still find some old youtube videos or threads on obscure forums with enough digging about that topic, that's how I learned of it initially.

So this was a real thing!


Back in these days I've seen one learning system that stored the correct answer in the value attribute of the radio button input tag.

I worked at <large tech company everyone hates> and we had this exact architecture. The senior engineer (who went to Stanford and was proud of it) designed this system. I had long arguments with him about how this system wouldn't scale when we launched it, and used real production data to prove that it would happen. No one listened to me, it launched and imploded within a few weeks, and soon after I transferred off the team. The worst part is, that senior engineer was eventually promoted and that system was passed off to an entirely new team to battle with. That entire system was terribly designed, and I think you can guess why.

Show me your tables, and I won't usually need your flowchart; it'll be obvious." -- Fred Brooks, The Mythical Man Month (1975)

It astonished me that people can ever think this is a good idea (everything in a single json document) or that anyone who thought it was could ever get funding or even get through any basic customer due diligence.


One part of my work entails entering companies on their death bed and going through their software development and related investments. I see a lot of JSON-storage in the initially well funded ones. They cough up some crap on Firebase, get some tens of millions of dollars in venture funding, burn through, and never leave Firebase.

It's really, really hard to sell systems implemented with these sexylicious clown technologies. Like, extremely hard. Apparently no sane person with money wants to pay a non-trivial amount for a pile of locked in JSON and Python scripts, and I haven't managed to add any insane ones to my network yet.

The small companies without funding that topple over and end up in my way, more commonly they instead put some stuff on a VPS or two, and typically the market values them quite a bit higher.

Fashionable technology choices burn through a lot of investment money. Seems to me like maybe a sign that money has been a bit too cheap a bit too long.


It is not that firebase is bad, it is that most types of data for most types of applications are relational and belong in a relational database

People bending document stores to fit every use case are insane.


The lock-in is very bad, especially in the auth and user management but migrating out of the JSON-storage isn't easy either. It also encourages bad patterns, like developing stuff straight into cloud functions rather than locally with version control.

Personally I prefer SQL-RDBMS-style data storage in part because it has these algebraic qualities that makes transitions between 'schema'-versions fairly robust and predictable. Extending and transforming JSON-storage is in my experience rather brittle, even if we ignore the hurdles and indirections around export and import in the clown solutions.


You can go with firebase for version N of your application when you have all relations between entities defined. If version N+1 redefines what your documents are, you're looking to write a migration script. That's not so different from SQL with all the joins. But there are people who can't design a data schema for neither a SQL database nor a document store.

So which is preferable storage from an eventual valuation perspective, organized files with a set of scripts, or a relational database?

Generally something like Postgres or MySQL would have been a better choice in these cases. You can migrate those, as well as handle growth through sharding or clustering, and more easily adapt to changing business or market requirements.

You'll need to recruit differently to build that way, though. Have someone that knows a bit of nginx and Linux, someone able to stitch together a bit of auth, things like that.

Languages with explicit types are also preferable, as a prospective buyer you'd want to have your people become somewhat efficient fast when they start working with the system. If obvious type constraints you kind of have a guarantee that IDE tooling will help with this, whereas Python, Ruby, JavaScript can be rather inscrutable at first glance.

Now, I have a financial interest in these things, it helps me if a system is well built and somewhat easy to evaluate, change and sell, so I have a bias. But in practice it seems that the basic setup of auth, storage and initial business logic takes very little time compared to the time it takes to flesh out the entire system needed to start making a profit, and since most (IT-)companies fail within three to five years it's probably a good bet to try and make the software you build be reusable and/or resellable.


It works great at a small scale. It's really convenient to have all the data at hand and be able to manipulate it directly in your programming language without having to go through SQL, which, no matter how comfortable you may get with it over years of experience and/or no matter how much library/ORM/etc. you throw at it, is always a foreign imposition into your programming language.

If you know beyond a shadow of a doubt that you will never leave "small scale", it's honestly not a terrible approach. It's also easy to define how to backup "the one file that is everything". You also ought to be confident that the worst case of the One File becoming corrupted is something you can deal with. (Although this is an often-overlooked issue in systems design anyhow.)

I've got some systems that have bits and pieces of this idea... but I'm really, really confident they'll never scale any bigger because for the systems in question it's simply impossible they ever would. I've got one that is, for instance, bounded on the set of wiki pages of a certain type that for it to grow into the 10,000s of pages where the approach would finally start to creak (from the "dozens" it has been in for the last several years and to all appearances will remain indefinitely) would require so much human effort to create those 10,000s of pages that I'd certainly have the budget to upgrade the approach anyhow as it would be negligible compared to the rest of the effort.

It also helps if this not something you're loading per HTTP request; in my case it's one-time on startup of a persistent service.

But you need to know you understand the relevant scales, because this is an immense trap if you don't do the math (it's not hard math, but it's math). Your prototypes work well and are easy to write and then are just total trash when they go to production, and presumably, if the engineer didn't do the math in the first place, it will emerge that they don't understand why the system is crashing in production either. Huge, huge trap. I've not quite seen this exact one with JSON files but I've definitely witnessed the "prototype that any experienced engineer could have absolutely guaranteed would not scale in about 5 minutes of examination".


They read that column-oriented DBs delivered great performance.

That sounds like some Bad CaRMa that I read about once...

https://www.red-gate.com/simple-talk/opinion/opinion-pieces/...

... which while there were other red-gate posts out there, I can't find that one ever submitted as a dup to link the comments about it (this shall be rectified).


Writing "RUN LIKE HELL" on a white board is insane, and should not be ignored.

Yeah, column-oriented works best for musical performance because it makes it easy to switch from Ionian to Dorian mode.

thanks for explaining on what are column-oriented DBs are in a ELI5 way

... In case this is serious, the joke is that that is absolutely not what that term means.

It’s genius.

Schmooze the execs with fancy dinners and trips. Deliver a horrible product so they need you for the foreseeable future.

The story itself says the CTO is unaware of everything. From his perspective, everything worked out fine in the end.

Win for everyone except for the devs who put in 80 hour weeks.


You missed the part where they say they felt like a rock star. Different people are motivated by different things. The key is, if you are very hard working try to channel it in a way that will actually benefit you.

Yes, don't let other people channel it towards themselves! You're in a company, - they will fire you when it becomes convenient to do so - don't burn yourself out for them!

Man, I’ve had one of those “I’m an idiot and only an idiot would rely on me to build things” kind of days. This made me feel better.

I worked very briefly for a company (that I won't name) that did this with elasticsearch. Every customer basically had a customized form for data entry, and every field on that form became a field in ES, even if that same field name and type was on every other form in the system. Eventually they had to contract with another company to run a customized build of ES on a bespoke cluster because they blew past the default 10k limit on the number of fields in ES.

When it became clear that they had zero interest in fixing this, I tendered my resignation and quickly found a new gig with sane architectural principals.


Hopefully those sane principals implemented sane principles.

touché…

How about reading the entire Json document out of the database, making a copy of that document, updating the copy, and saving that copy to the database...

Is how someone on my old team designed an internal tool


Depending on load patterns, for an internal tool that might be perfectly reasonable; you can trivially look at the history, for one thing.

Is this not how you update a JSON column on a db that doesn't have (or you choose to not use for reasons) a native JSON type with a sprinkling of data should be immutable?

Well without some form of locking you have a race condition. Two updates done at the same time can have the overwrite the other

One team in my company insists on storing everything related to a target in an MB-size Elasticsearch document and then do all the aggregation client-side, because they already use ES for everything else and don't know how to use a relational database.

> Does this sound insane to you? Early in my career, a client came to us with their internally developed perl web app that was very slow and asked us to help them. They were storing all of the information as csv files and would do exactly that, read the entire file, add a new row etc...

We explained to them that we should rewrite it for them :)


I don’t understand the confidence that leads to such misuse. I’m NOT a db guy by any means, and yet when I need one I’m carefully looking up things like “what’s the space complexity of compound indices”, at a development stage. Defensive and paranoid. I guess there are a lot of cowboys out there.

Most people who use an index wrong dont understand space and time tradeoffs at all, how to measure if there is a problem, or how to identify what a thing should be in the first place.

Most people who at least think a bit get pretty far before the vendor specific gotchas start biting.


This is something I’d do it a ship it tomorrow, trash it the day after MVP.

In a consultant project with nearly a year to deliver, no way.


Would it help, though, even in the short term? Making an actual `ticket` table really isn't that much work. If for some reason it is, you can always use an array type, or even shove JSON into a text column. Even if you're not thinking further than an MVP—hell, further than 5pm—one column=one ticket seems the most awkward and difficult solution I can think of.

My best guess is it started as a single ticket column, then maybe a manual migration to quickly add a second when a rat's nest of code was built around the single column already.

the most far-fetched justification I can muster up for this: the UI/user requried a tabular display of all tickets as columns with one row per user ... and they had a UI that only displays DB table contents as-is.

Nah, even for a 5pm demo that just sounds absurd (with up to 500 tickets per user).


> Each ticket was a column. So if your most active user had 5 tickets in total history, you needed 5 columns.

I was thinking like, "Oh yeah, they have 5 tickets, so need 5 rows. What's wrong?". And I read it again and found it's the word "column". It must be a big fun to write a query for such tables.


I bet it was SELECT * FROM customers; followed by a for loop. Easy-peasy.

It would be _slightly_ better if the DBMS have array types. Often they are not scalable, but it is better than thousands lines of copy-and-pasted code.

Honestly this is not necessarily a good reason not to invest. Sure this needs to be fixed, plus whoever allowed this to happen should be replaced but analyzing the business as an investment opportunity and passing because of this is just as insane in my opinion

Execution is what matters in startups, if they either can't find developers who are halfway decent or if a cofounder wrote this, that's a major red flag that they don't know how to execute.

Those types of design, especially the giant json document, has to be part of the reason some companies pushed pretty hard against the GDPR. Imagine trying to remove personal data from such a system.

Everything in this story is dysfunctional, including the protagonist's approach. It is completely unacceptable for a team leader to give their people time off and lie about it and he is well in to territory where the company could fire him. I wouldn't be surprised if it was a fire-for-cause situation, in fact. Although upper management sounds so off the rails that he would probably get away with it and get a commendation of some sort; it does seem to be a play to the environment he's in.

A tip for new team leads; there is nothing here to feel proud of and a better play is to kick up a huge stink about people are working overtime and insist that the vendor be held to sane standards or project scopes reconsidered to fit in a normal work week schedule. Trying to make this sort of situation work is going to burn people out for nothing, or get people fired with no potential upside.

Unless I were truly desperate for work to keep my family going, a team lead has a responsibility to protect their team's reasonable work hours from insane requests. That is a hill to get sacked over, but not to lie for.


> It is completely unacceptable for a team leader to give their people time off

It is more unreasonable for the company to cancel already-given time off.


Because throwing the CTO under the bus is always career enhancing ...

That isn't throwing anyone under the bus. I've had literally that conversation and it went fine. If you're optimising for dev productivity, you let them take holidays and work them to a plan that lets them get a good night's sleep and some time to recover.

You literally had what conversation? You went over the CTOs head, to the CEO, problem got fixed, and your career stayed on track, and it was a company of > 5 people?

#1) That's a miracle.

#2) That's before we get to the actual situation.

It's December and work is due to the client in 1 month.

99 times out of 100, the CEO is going to decide you're an insubordinate loose cannon showing up inappropriately late to make confusing complaints about outsourcing decisions that were made 10 months ago.

Conversely, lets say you made a big stink about this 10 months ago, when the outsourcing decision was made. 999 times out of 1000, the CEO is going to assume you're an insubordinate loose cannon showing up inappropriately to make confusing complaints about outsourcing that are above your pay grade.

Hell is other people, and I find it very hard to believe you have extensive experience and can breezily handwave about "literally having that conversation", in any form. I can think of a reason why at every point in this entire story, the person complaining would be blamed. I don't think the outcome is great, I'd definitely be looking for other jobs, but again, 36 years old and every experience I've had everywhere is that these situations are bad news and you wanna avoid conflict, unless you have nothing to lose.


The "kick up a huge stink about people are working overtime and insist that the vendor be held to sane standards or project scopes reconsidered to fit in a normal work week schedule" one. I'm not sure where you're getting "went over the CTOs head" from; I never suggested doing that. It sounds like a stupid thing to do. You kick up a stink by talking to the CTO, 1:1. Or whoever you report to.

Ummm...let me think here:

Poster: Because throwing the CTO under the bus is always career enhancing ...

You: That isn't throwing anyone under the bus. I've literally had that conversation.

Me: What do you mean? You complained to the CEO about the CTO?

You can't "throw the CTO under the bus" when talking to the CTO, "throwing X under the bus" colloquially means "blame X for the problem when talking to authority figure Y"

That's why your comments are confusing.

The thread leading up to your comment is presupposing tattling on the CTO to some authority figure.

You were very terse and dismissive in your reply. Apparently you were trying to communicate "No, I was smart, I didn't talk to an authority figure", instead you said you've "literally had that conversation", which would further imply you tattled to a authority figure, and that you only wanted to disagree on whether this constituted throwing the CTO under the bus to the authority figure.


Should have thought one step further back: The weirdness comes in going from "a better play is to kick up a huge stink about people are working overtime and insist that the vendor be held to sane standards" to "throwing the CTO under the bus". The huge stink was kicked up with the CTO, so there never was any "throwing under the bus". It's your "Poster", the next commenter after OP, who introduced the confusion -- not OP (your "you"). HTH!

ETA, after looking up names: No, the thread up to then was not about "tattling to a [higher] authority figure". Poster roenxi was the OP, with "raising a huge stink". Then speedbird2 misunderstood that as "throwing under the bus", and you're for some reason taking that misunderstanding as defining what the thread was about.


He never said he goes to the CEO.

This story is about the CTO having a dumb idea and literally no one pushing back on the plan, with the only people suffering are the (relative) grunts. There's ways to respectively raise flags here.

The answer isn't to go to the CEO, the answer is to not be a drone-like yes-person.


So who do you have this conversation with if history has shown that yes people are the ones who stay close to CTO and others get filtered?

Be honest. Be kind. Be quick. Be brutal. Be thoughtful. Be diplomatic.

But be honest. Honesty will be a great adventure of your life. If you get punished for it, so be it. Those who punished you for no reason and for ill-will will receive their reward, even if you never see it happen.


>Those who punished you for no reason and for ill-will will receive their reward, even if you never see it happen.

Your comment sounds very nice.

However, when that punishment you receive is losing your job and you have to feed and house yourself (and maybe a family!), there is no solace to be had in they hypothetical divine justice you speak of.


Being honest is not easy. If it was, the world would probably have more honesty and truth in it.

If your fired for honesty, shake the dust from your feet, pick yourself up, and move forward with your life away from those tyrants. Will it suck? Yes, but only until you come out the other side.

The alternative, perpetually lieing to keep your job, letting those lies spread into the rest of your life, and eventually being caught in your web of lies, will suck much more, and there is no light at the end of that tunnel. Small lies turn into big lies, and some lies turn into several lies. Eventually, you will lose yourself in your lies.

And on top of that, now you were to one who perpetuated and invited evil to continue. You are the creature you wished to prevent by lieing in the first place.


>If your fired for honesty, shake the dust from your feet, pick yourself up, and move forward with your life away from those tyrants. Will it suck? Yes, but only until you come out the other side.

If the "will it suck" = struggling to feed yourself or your kids, I feel like you are massively downplaying the "suck" part.

In a perfect world, I agree with everything you said. Unfortunately, we don't live in a perfect world, so sometimes you have to be a bit more realistic.

And, realistically, I would rather lie and keep my job than be honest and have to explain to my wife that we get to choose between electricity and eating.


If you would get fired for being honest, then yes, go home to your wife, tell her you lost your job, and you'll find a better job. If you are a dependable worker, you'll find a job. And being honest isn't an acceptable reason to avoid unemployment.

Having said that, being honest isn't being stupid. It doesn't mean you can't have another job already lined up when you hand in your notice because you realized your employer doesn't like truth. Or that you need to always be blunt. Honesty is about being firm snd diplomatic.

In an alternative world, would you like to lie so much that you begin cheating on your spouse? After all, it starts with one small lie. And that your kids never trust you when they are older because they know how much you lied to them?

And that job interviews go poorly because they already know the previous lies at your previous employer?


I generally agree with what you're saying, and I've lived it, but the examples are a touch extreme and muddy the message.

I left Google after 6 years by carefully, over months, thinking through what was happening, what my limits was, what I'd do next, and whether I could show senior leadership at the company and my division was aligned with me, even if immediate leadership was acting like it didn't matter and playing games and never talked directly.

Regardless, people are awful, people who previously over-the-top treated me like I walked on water and we needed to get the project done by any means necessary, treated me like I was having a sudden mental health issue by not finding a magical way to force someone who had found 1000 reasons to delay and not compromise.

I don't know that I'll ever work _for_ someone else again, and even with A) my full knowledge going in and B) support from professional mentors in the company and mental health professionals, it's taken me almost a year to get back to 90% okay with the world. There's no karma here, they stayed there getting checks.

I agree with your point, and have lived it, but I think you are doing yourself and others a disservice by painting in such stark absolutist terms.

I wouldn't be able to make the decision to accept leaving without numerous previous decisions to bite my tongue and vest.


>In an alternative world, would you like to lie so much that you begin cheating on your spouse? After all, it starts with one small lie. And that your kids never trust you when they are older because they know how much you lied to them?

And that job interviews go poorly because they already know the previous lies at your previous employer?

These are absolutely wild examples that share so little relation to the conversation that I am actually left a bit confused and unsure how to respond.


I totally get where you're coming from. Sometimes we gotta just lay low. But at some point in your life, if you're always afraid of pushing back against demands because you're afraid of getting fired, its more of a "you" problem than a "them" problem.

I'm trying to figure out where you are getting this "go above the CTO's head to the CEO" situation from the message you are replying to?

I am not the person you are replying to, but I've definitely made a stink my boss about my people being overworked. If the person from the original article went to the CTO and said "look, my people need a week off, but we will still have the software delivered on schedule", that would have been the right solution, not lying to your boss.


A: Because throwing the CTO under the bus is always career enhancing ...

B: That isn't throwing anyone under the bus. I've literally had that conversation.

This led me to believe B had a literal conversation where a literal CTO was not-literally thrown "under the bus", meaning "authority figure was told the CTO was to blame for $X". Authority figure for a CTO is generally CEO.


> This led me to believe B had a literal conversation where a literal CTO was not-literally thrown "under the bus", meaning "authority figure was told the CTO was to blame for $X". Authority figure for a CTO is generally CEO.

Yeah, that whole scenario was made-up by you or speedbird2, or both in unison. You read words that weren't there. Here's what the thread really said:

ronxio: Raise a huge stink.

speedbird2: Throwing the CTO under the bus!

ronxio again: That isn't throwing anyone under the bus. I've literally had that conversation.

ronxio never said the huge stink was raised with anyone else than the CTO. speedbird2 doesn't get to define the terms of ronxio's anecdote; that's not how it works.


Then you risk with CTO saying "no, we just can't afford it."

And then it feels like they might more actively look into that.


sometimes you burn the CTO but you better make sure you brought ample firewood because guys at top can take some serious heat before melting and most low levels underestimate the amount of firewood it takes resulting in getting roasted by the leader instead.

Once I worked for a company that had an ex-googler on the board who tried to act as CTO. Eventually I tried desperately to throw that person under the bus but the CEO bizarrely moved the bus around instead of letting the bad influence on the company take a hit.

Took a year of trying, then I left and one of the other two developers did as well. Would have stayed for longer if they payed me more which I made very clear, but they were stingy, adding like ten percent to an already pretty low salary and presenting it as such an effort on their part.


The team leader reports to the CTO. The team only reports to the team leader. Therefore, it's perfectly acceptable for the team leader to give their people the time off, and even lie about it, since it didn't affect performance.

The part you are missing here is that the work was already done and none of that progress was shared with management. After all what would they do? Accelerate the project!

Stick it to the man when they try to get others to work harder to glorify themselves. Weak bastards.


Saved whose day?

The shitty vendor who delivered crap?

The CTO who surrounded himself with yes men and was clearly completely unaware of anything happening in the company?

The devs who were worked to the bone, but don’t worry I gave them a week off?

The protagonist who lied to everyone to hit an arbitrary date for a company that clearly doesn’t give a shit about them?

This story made me cringe at every turn. I’m a hard worker and I have put in extra hours to make a launch go smoothly for a client here and there but this story is pure insanity. I’m willing to put in the extra time on occasion because of the “relationship” I have with my boss and knowing that I can always tell them the truth. In fact that’s a core concept of having a no-blame culture, you only get that IF everyone is telling the truth.

Lying like crazy to meet a deadline for an idiot CTO is quite literally insane. If you find yourself in a situation like this then GTFO of there and find a better place to work.


> The devs who were worked to the bone, but don’t worry I gave them a week off?

Thats the part that piss me off the most. What devs get from this? Feeling of "pride and accomplishment" and burnout?

> And we hit our dates in January, went live with a great launch, and were rock stars for a bit. Maybe more like Herman’s Hermits than The Beatles. But it still felt good.

Thats the part where "we" definitely means "me".


And the CTO comes out looking like a genius for getting an optimal solution he didn't even know existed.

Maybe I'm lucky but I've never been fired for telling the truth and the truth is easier to align to. "There's a bug in a third party library that's part of the critical path "We can make it harder to hit the bug but can't fix it until the vendor fixes it", or "user growth has revealed performance issues no one thought we'd encounter this soon, we can spend three times as much money on infrastructure to mitigate it for the 2 months it will take is to fix it or lose customers due to performance" or "our biggest customer didn't know what they wanted until we delivered the first iteration, now they want something that isn't at all what we thought we'd be building. We can build it and be profitable or follow our dreams and die".

Again maybe I've been lucky but honesty has worked well for me.


People that surround themselves with “yes” people often fail to digest the truth. So as others have mentioned, they will “take it under advisement” and then you will slowly get phased out or put into “busy work”.

Second option is to hold your tongue, watch them struggle, and eventually fail over a long period of time. Usually 1 year but have seen it fold in less than 2-3 months and the leadership effectively canned the next quarter.


This is not really about honesty - if someone asks your opinion you can explain your concerns diplomatically - but if they don't then you can also keep quiet.

It's more about whether you're going to actively point out a problem with a plan that someone higher up than you is trying to take credit for. i.e. they will feel personally attacked - that their judgement is questioned - when you speak up.

It's an extremely difficult thing to do without making enemies and enemies last a long time and 1 enemy is more damaging than can be made up for by lots of friends.


I’ve never been fired for being honest but I have been managed out of a team for it.

I have not been fired, but my output has been ignored or heavily filtered by layers between high leadership and myself.

And I have been given feedback in perf reviews that I have been disagreeing too much.

And in those cases it did turn out later that I was right, so I wasn't being difficult for no reason.


In many environments, trust matters more than truth, and trust is gained by only telling nice things. So you have to play the game.

> I dial in every morning for the mandatory death march status call with the CTO and I lie.

> “The team is working hard. Today we hit milestone integration point #73.”

> “The team made good progress yesterday, we finished another web service.”

> Every day I showed up and told the big boss that we were hard at work on stuff that we had already completed over the previous month.

I think it's a really important detail that he was lying about work that was already done. That looks like "underpromise and overdeliver" from a certain angle.

I'd feel worse about lying about work that's not done yet. Certainly riskier, and maybe not the best feeling for the team when they get back: "Here are your tickets, I told the CTO they were already done, so hurry up."


Developer interactions with management suffer from huge information imbalances, and a lack of trust.

For example, I'm working on a project now to update a code base running on a (very) old compiler. We've been working for a year, and large chunks of the system are done. One (fairly major) part is yet to be completed.

Management doesn't understand the process, or have confidence that the project will ultimately be successful. I don't blame them for being skittish, software projects (especially conversions) have a long history of failure.

We had weekly meetings with them on progress - which are now twice-weekly (because that will speed things up.) Mostly they don't attend directly, a middle manager acts as a go-between.

The developer view is that of course this will work (personally I've never been in doubt on that front) but the time-scale is uncertain because the system is large and old. But we're talking months left, not years. Yes, I'm aware of the 80/20 principle, but we're well into the 20% already.

Of course to management it's a binary state, done or not-done. It's hard for them to guage progress. There's no "trust" in what we say.

Which makes sense to me. We might have done 0 work for a year. If all we did was the meetings -they wouldn't know-. (Its a fixed price job, so it doesn't make sense for us to drag it out.)

Of course the risk is all on them. They've spent a bunch of money and are nervous. They made (good) tech decisions (based on the advice of outside people who understand the tech) but they feel very uncertain. Ultimately if the project fails they'll take a hit (not so much us.)

There's no easy fix here - you can't just argue that managers should be technical (the tech is not their core business), and more "consultants" won't make them feel warm and fuzzy. The best we can do is push through and deliver.


>> One (fairly major) part is yet to be completed.

Break this down. Break it down into granular chunks even if they are a month long. Divide everything up into sub tasks. Even if they aren't perfect even if you have rough edges.

Name every chunk something friendly and fun. I recommend classic dances... tango, cha cha, waltz.

Call a meeting with your managers (including the high up ones). As for middle managers to start attending (auditing) daily standup. Make everyone STAND for them (so they stay short) and track against the tasks on the list in the wave.

Again, a piece being added, or a piece being late isnt going to freak any one out as long as you are close....

We both know that going live is gonna be the big hurdle just dont tell them that till your ready.


> Make everyone STAND for them (so they stay short)

Hard to see on Teams or Slack or whatever.


>> We both know that going live is gonna be the big hurdle just dont tell them that till your ready.

Yeah. There are over 5000 stores that this will roll out to, so it'll go slow there. Fortunately there's a dedicated (separate) testing team so thats good.

>> Break this down. Break it down into granular chunks even if they are a month long

Part of the problem is that it's not easy to break down at this point. We are aware that "it doesn't run". (There are underlying classes which affect things.) What we don't know is "how many class issues are outstanding". We knock them off, and we move to the next one. We "feel" like its close now. But any attempt at quantification is simply speculation.

We have tried -not- to introduce speculation into the process because that's not data. When we're pushed on providing speculate data (when will this be done?) we ecplain why that's impossible for us to determine and suggest (to the middle manager) that he guesses instead. He tends to not push too hard there.

Part of building trust (at least as far as him) is in not making-stuff-up. We're clear on what we do know, and clear on what we don't know.

Obviously I don't know yo what extent that is passed on upstream. Upstream are keen on dates, targets, reports on "accomplishments", all the usual management. "We don't know" is good data for them, but not what they want to hear, and perhaps not helpful to them.


> We are aware that "it doesn't run".

...what? You have an application that literally doesn't run and you've been working on it for months?


Presumably there is some legacy integration or blocker why it doesn't run.

Even so, working as an engineer on a project that doesn't run as a whole for more than a week, let alone months.. It's very understandable that management is nervous.

Cut things away until you have something that runs is step one...


It's probably an over-simplification to say "it doesn't run".

Firstly, this is a conversion of existing code to a later version of the compiler. (a 25-year later version). So it's not "writing a new app" - it's making it all compile (and run) under the new compiler.

To be clear - it does compile. That was a pretty small part of it. And it does "run" in the sense that the Exe starts, gets through authentication and so on. All the Utilities run, and the 2nd-largest system is running.

The primary program consists of around 15 threads all running simultaneously. The code was originally written using a co-operative threading model, and it's now going to use a preemptive model. The program that "doesn't run" is effectively (not yet) starting all the threads correctly, in the right order, and at the right pace. (There is a lot of cross-thread communication and cross-thread UI interactions.)

So yeah, it's a giant big ball of smoosh. This is not a project for the faint-hearted. But we're actually nearing the end of the tunnel now - every day progress is being made.

But until we can show the "whole program" running, to management progress seems to be binary. (It either works or it doesn't.)


> But until we can show the "whole program" running, to management progress seems to be binary. (It either works or it doesn't.)

That's insufficient communication from delivery (= you), imho.

When a tunnel gets dug, you also don't always know what you're going to encounter (surprise water table? rock? skeletons?), but you make a plan to the best of your current knowledge, track your progress against it, and adjust where needed.

If you cannot do that, it looks like you're just trying to power through the project blind based on gut feeling alone.


Same as if you're a contractor. You propose a plan so they can track progress, estimate it so they plan for dependent stuff, and adjust the plan/timetable when issues arise (and communicate plenty and plainly). While I don't know GP's project context, I'd go with a test checklist and measure against that.

> We had weekly meetings with them on progress - which are now twice-weekly (because that will speed things up.)

I've been on more than one project with delays. Delays make management nervous. I understand that, but having more meetings about it will never make things go faster.

Having a half-hour check-in with the PM every day so that they can "provide me with support and give me what I need to get the project back on track" is not helpful. There is nothing they can give me that I need other than fewer meetings. There are no obstacles but time, and the only reason that is an obstacle is because upper management insisted on an unrealistic timeline to begin with.


> They made (good) tech decisions (based on the advice of outside people who understand the tech) but they feel very uncertain.

If they took advice of internal people and then made the same internal people work on it, this trust issue might get solved.

Ultimately by showing lack of trust in their developers, they are showing lack of trust in their management skills for having selected the correct developers. They are doing one core part of their job very badly.


The problem with hiring good developers is that you need good developers and a solid corporate tech culture to hire them.

If you don't have anyone you can trust to interview technical folks, how will you avoid hiring bad developers?

And similarly, if you hire good developers but your corporate culture is bad, they'll leave, and eventually the only remaining developers will be bad ones.

It's very hard for non-tech companies to hire and retain quality talent.


> If you don't have anyone you can trust to interview technical folks

If you have an IT or SWE department, you let them hire people they need. Which means you put technical people in charge that knows what is required. If you don't have trust in your department, no one can fix that.

> And similarly, if you hire good developers but your corporate culture is bad

That's on the company. And I found it's when they want to applies a standard policy across all departments. Like the same standard issue computers and a management software that eats half the ram. Peter can work fine in Word, but Julie's IDE is freezing every hour.

> It's very hard for non-tech companies to hire and retain quality talent.

It's very easy. Pay them what's they're worth and let them work on your problems.


If you have incompetent people in your IT / SWE department, they're not going to make good hiring choices.

Similarly, corporate culture is very little about stuff and more about processes. If hardware or software environment problems exist, will they ever be fixed? At a lot of companies, no.

And "pay them what they're worth" is a relative metric. Non-tech companies aren't going to be able to afford to pay a cost center what tech product companies can pay a profit center.


> If you have incompetent people in your IT / SWE department.

Same for every other deparment. Incompetent people won't hire competent people.

> Non-tech companies aren't going to be able to afford to pay a cost center.

Isn't that a mislabeling issue? If software being better implied more profit, then it's a profit center. If your legal department being good means fewer legal fumbles, then isn't it worth it? Same for software. Yeah you can deliver mediocre software and still retain users, but that just leaves you vulnerable to competitors.


Profit- vs cost-center is pretty ingrained into a company's management DNA, for all companies.

From top-down perspective, it's a question of "If I +$1 to this department, what +revenue do I realize?"

In non-tech companies, that's a long or abstract link: "How does funding SWE help me sell more furniture?" In tech product companies, it's trivially obvious.

Tl;dr - always work for a profit center, never work for a cost center.


Alas, it's almost never that simple.

Firstly because the tech team has varied enormously over the last 25 years. Some have been around a while, some are new.

Secondly because middle management (and for all I know upper management) has cycled a lot over the last 25 years. (We've had 3 different middle managers on this project in the 2.5 years since the project was first proposed.)

So, speaking generally now, it's common for the people who did the hiring not to be the ones who now have trust issues.

Equally decades of failed software projects (industry wide) have lead to a (well deserved) reputation for IT not being trustworthy.


They probably didn't pick devs that they trusted, they picked the ones they thought were the cheapest.

I think very few execs care about the cost of individuals. Firstly it's not their money, secondly they're not rewarded for "hiring cheap" but they are penalized for their failure to perform.

Throw in that hiring in general is hard, and that hiring technical people when you are non-technical is even harder, and it's hard to have lots of confidence right out the gate.


> I think very few execs care about the cost of individuals.

From what I've seen, it's always about cost. They would post salary ranges if that weren't the case.

> hiring technical people when you are non-technical

Non-technical people are always hiring technical people. How do houses get built? Who hire lawyers?... It's either references or researching the person's professional experience. And yes, people can lie. But that happens everywhere not just in tech.


And so again failed at their jobs.

If management has no visibility into the project progress, that seems like a pretty major problem.

It absolutely shouldn't be "binary." Any year-long project should have tractable progress metrics. Upper management doesn't need to be technical, but someone in the chain absolutely should be capable of translating progress into a digestible format.


When you build a building, it starts by digging a hole and then filling it up. Then you see walls go up etc. Although progress is visible, visible can lie.

Obviously the project had milestones. But it is in the nature of big conversions/updates that you will encounter things you don't know. We're in that stage now of killing the unknowns.

It's not a linear process and the quantity of them is unknown. So at this point metrics become hard. (We hit all the targets for the "knowns", and we continue to squash the unknowns, so the dev team is confident, but management is nervous.)

Progress is easy to digest. But (for now) there's no easy "finish line" to see, which makes their lives hard.

There's no "quick fix" here where "better paperwork" would improve things. As much as you (and indeed management) would like that.


It is a major problem, but it is the big unsolved problem of software engineering that has resisted a remarkable number of attempts by unreasonably capable people. Under normal conditions the metrics you are talking to boil down to "the dev says we're making progress". The level of formality with which they say that changes, but not the underlying process of how the metric is generated.

The only way to improve on that is for managers to go in to git and check what is happening. Technical managers can get deep into tracking progress. Non-technical managers cannot.


> It is a major problem, but it is the big unsolved problem of software engineering that has resisted a remarkable number of attempts by unreasonably capable people.

That seems like a cop out. Knowing precisely how long a project will take is famously hard, but knowing the rough progress being made towards milestones is absolutely solvable.

Git is the best place to look, but you don't need that level of detail to see new things being shipped.

Everywhere I've worked (from small startups up to Google) it is absolutely the norm to have clear milestones to work towards on at least a quarterly basis.


> knowing the rough progress being made towards milestones is absolutely solvable

Yeah, but the process for knowing that is ask the dev and they will tell you. Or ask the PM who will ask the dev. Or ask a tech lead who will ask the relevant dev. Or read the spreadsheet entry that the dev updated. All roads lead through conversation with the dev working on a feature. There are occasional exceptions, but they are rare.

If the dev is willing to lie, or honestly mistaken, or just too inexperienced to estimate how they are doing then it is close to impossible to gauge progress.

> That seems like a cop out

You would probably believe the number of people who say that.


> If the dev is willing to lie

Isn't that a firing offense?

> or honestly mistaken

Estimates can be wrong, but I think a battery of test is pretty objective.

> or just too inexperienced to estimate how they are doing

That's hiring the wrong people


> Isn't that a firing offense?

Yeah. But the dude in the article lied about progress so I have to include it.

> Estimates can be wrong, but I think a battery of test is pretty objective.

Yeah. But if 80% of the tests are passing, is the project 80% of the way through the timeline to completion? No. They are useful because they measure correctness but they don't measure progress.

I also don't think it is reasonable to expect non-technical managers to interpret technical tests; even as a rough measure of progress.


If 80% of the tests are passing, then another week of work get 2 more passing, that's progress. It may not help predict the end of the project, but I dare say that a manager would like to know that instead of a vague "We're still working on it, but we don't know when will be done". Even if it's a bug you're investigating, a more helpful report would be "I had three hypothesis, I managed to eliminate two" instead of "I'm still on this ticket".

> If 80% of the tests are passing, then another week of work get 2 more passing, that's progress.

Sure. But that is the sort of information you get by ... talking to the responsible dev.

You can get the dev to describe their progress in any one of a myriad of ways. But you aren't escaping from the fundamental issue here of the dev self-reporting and management ultimately having to have confidence in the dev's ability to self-report. There is no more information for non-technical management in this metric than if the dev says "I think I'm about 60% done".

I'd have more respect for a dev that estimated this way than one who just guessed at random, but it isn't making it easier for non-technical people to gauge progress. It is actually running a serious risk of backfiring, because on the date that the metric suggests the project should complete, the dev might quite reasonably discover something that needs another X amount of time to work on something that the test cases didn't cover. Then management may well feel deceived because it turns out the metric wasn't tracking progress accurately.


I agree with you, but it's not that different with any other department. Sales can project 8k sold units for a quarter, but with 3k only being sold after 2 months. It's a more tangible metrics but unexpected things happen and and you just have to trust your subordinates to report them to you so you can react properly.

Managing a software project is not that different than directing a movie or building an house apart from that inspections are easier as the result is visible. But you can make software work visible by having milestones, tasks lists, and tests. Not by following SCRUM methodologies.


That argument is avoiding the point of contention though. It observes that sales can be behind a metric and a dev team likewise. So far so good, but that isn't the problem with the dev metric.

If the sales team lead started the quarter with an 8k goal and ended with 6k actual, they are a quarter closer to being fired. Their job is to hit the metric because the metric is in itself commercially important.

If a software team lead starts the quarter with milestones, tasks lists, and tests, and at the end of the quarter has pushed back a milestone for something unexpected of higher priority, did different tasks because that turned out to be more efficient and refactored the test suite to better reflect quality it isn't at all reasonable to use that as evidence they should be fired. A company that uses those things as a progress measure is going to be bad at software because it can't learn fast enough to pivot effectively. The company might fire the responsible lead anyway under the same logic as they used with the sales team lead, but that would result in them firing their high-performing leads who are more interested in good commercial outcomes than achieving bad metrics.

Some of the highest performing software team leads are happy pivot on a dime mid-quarter and call it a major success even if the initial targets were reasonable. There are no high performing sales team leads who think missing a reasonable target is a win. Acceptable, maybe, but not a win. This is a major problem for non-technical managers and means that milestones, tasks lists, and tests can't be used to track progress.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: