* The internal software was built up over a number of years as a series of hacks that rarely if ever got unwound. The people running the business were blind to the necessity of this and a lot of the engineers were not motivated to fix it because they were experts on the rabbit hole and were angling for job security.
* The internal software's infrastructure aged badly and we were stuck in a perpetual upgrade hell. This was again because the people running the business did not really understand the accretion of technical debt. Their abilities centered more around, say, pricing insurance and regulatory issues.
* They didn't know how to hire engineers and they instituted corporate processes that ensured that the engineers were both pissed off and unable to properly do their work and they were in a geographical area where it was hard to hire. Good engineers largely left, bad engineers stayed behind and continued building rabbit holes.
* They outsourced critical functions to cheap overseas labor and got crap as a result.
* Their response to slowed development and deadlines was to throw more engineers at the problem.
* They're also the only client I've had to sue for nonpayment of an invoice.
You indicated a large amount of technical debt and issues, which the techie in me 101% empathizes with and shares your pain. You also indicated morale and hiring practices issue, which the person in me empathises with.
What you didn't indicate however was the business outcomes: Were there unhappy clients that left? Did the costs rise? Business fold?
Everything you mentioned, unfortunately, is also the case for some wildly profitable/successful businesses.... :-<
It certainly wasn't the case that this project was wildly profitable. Not sure if it will ever make money at all in fact.
I have worked for profitable businesses who have behaved like this too and it's usually the case that they have some kind of inbuilt unfair advantage (legal, governmental, monopoly/size/clout, etc.) that compensated for sucking at tech.
You have to know how to price software, know how it is built and know what are challenges. It is just like I would want to open restaurant right now. I have enough money to do it, but I am going to be ripped by suppliers, chefs, staff.
Bad devs just sucking out salaries over time is one thing. Then you get silly project managers who want to prioritize FCKN EVERYTHING and don't understand project and don't care, just want their features in. Maybe even conflicting feature requests from multiple customers.
You can do 99.5% of what most non-technology businesses do via a combo of SaaS services. Those keep getting better over time. It’s cheaper to hire a large team of coders/consultants for a few months to migrate you than to pay the same said team in perpetuity.
What is is with tech people and pulling numbers out of thin air to justify their simplified straw man of users?
I agree that pretty much any large organization writing their own email or payroll system in this day and age is probably certifiably insane.
But even large "non-tech" companies have fairly significant custom needs even if it's just integrations and customizations of SaaS. Just because you're using Salesforce doesn't mean you just point everyone in the company at your Salesforce instance and call it a day.
What you're forgetting is that insurance companies still need their custom insurance software with all of their custom risk models built in. That's the entire value proposition of the business so your estimate that it's only "0.5%" of what the company needs is way off.
I don’t have any additional context besides my consulting experience but I’m willing to bet several engineers did their best to make everyone aware of the technical debt and were ignored.
And WTW are a risk-management firm. The failure to recognise technology as risk to be managed seems particularly perverse.
25 years in and still not getting it.
Amazon has always been and will always be in the business of cheap, scalable logistics. Not selling books. Not selling the cloud. Selling access to low cost scale.
For example, they signed a bunch of exclusive supplier agreements for their store. Then, when they launched Marketplace, they got sued by some of those suppliers for breaking exclusivity.
E-commerce success created the cashflow and in-house expertise that allowed Bezos to explore other markets. Some succeeded, like Marketplace and AWS. Some did not, like the A9 search engine and Fire phone.
And maybe to make a larger point, I think there's been bad consequences of constantly overhyping the innovation happening in web technology and software, while underestimating the innovation in manufacturing, aeorospace, defense, logistics and so on. There's much more overlap between the general economy and the latter, and it's much more regionally diverse. I think we could have done a little better if more talent went into those 'boring' sectors, rather than building the next sharing app in the valley.
I have seen egregious uses of the term 'tech' used to just mean specifically silicon valley internet software, but this case is more a marginal oversight of one example in passing, rather than a deliberate or misleading misuse.
The cloud, communications and enterprise solutions are new fields for this _hardware manufacturer_.
The reality is vastly broader.
As if travel companies spinning off CRS systems is something new... every single train/airline/hotel reservation system out there is a direct or indirect spinoff from an internal company product
My data points, though limited in number, indicate a strong tendency on the part of these kinds of shops to build solutions that are over-fit to their particular workflows, and to under-resource their development. I was, in one of these environments, explicitly told that version control and separate development/test/production — or even just dev and prod — environments were a silly extravagance.
I'm sure there exist shops that don't do it that badly. I'm more sure, however, that they're not the norm.
The thing happened in China over last 5 years:
- Restaurants run ordering APIs for food delivery companies
- Restaurants run ordering APIs for online menu companies
- Laundries run restocking, and order ready notification APIs
- Logistic companies pretty much became a few layers deep API business with most companies having 0 user facing "frontend"
- Hotels - some will deny you booking if you don't checkin online. Some went even further, and check in you remotely
- Construction companies - now run with client facing construction site management software, and run joint CAD projects with
- All kind of odd job service aggregators now do what is called o2o: dog walkers, handymen, porters, cleaners etc
- Many business service companies simply went out of business because their entire businesses were turned into free apps. You can file your taxes with a free app that scans QR code or electronic invoices for example
- All kind of retail store interact with wholesale suppliers only through some EDI stockkeeping or a more advanced API
- And for commerce, things are all self explanatory: JD and Alibaba
And kids in the US rightly focus on the sexy stuff because when you compare just Apple to the entire Indian IT sector revenues are 3x more. Why would you want to go work on laundromat or construction company APIs when you have a shot at getting into Apple or Google.
The US spends about that length of time discussing on whether or not to build a new tower in a historic neighborhood.
I don't know why this is the case, but I think it has something to do with a rapidly growing economy versus an already grown economy. If someone could explain to me why that's the case, I'm all ears.
My off-the-cuff guess: if a country is growing, the easiest way to make money is hop into a growing industry and building as fast as you can. If a country is grown, the easiest way to make money is to siphon it from somewhere else. So in developed economies you have stagnation and inefficiency because the elites are more interested in redistributing the wealth (to them), instead of creating new wealth.
The citizens in the US and their political power are the source of most of the inertia.
You could replace "country" with "company" and describe why startups are continuing to disrupt existing industries.
How to say this without an ad hominem?
> Why would you want to go work on laundromat or construction company APIs when you have a shot at getting into Apple or Google.
Why would one be even bothered by that? Apple or not, anybody more or less seasoned professional knows his price and capabilities. My experience in the West shows that MNCs and dotcoms are full of kids just wanting to be abused by their employer for: 1. nice record in the resume; 2. sense of "social status"; 3. getting that "artsy startup cred" approval mark.
If laundromat API pays 3/4 of Apple's wage for an equivalent duration of work, but is many, many times easier to get than a job at Apple, I think there is not question what gig to pick up.
That's a typical wishful thinking of a "creative class wannabe" who has tons of insecurity about his claim to entitlement coming with that "artsy high class person" label.
Chinese devs may be taking someones job in the West, but it has no dependence whatsoever on them being evaluated for their "creativity." This has much more to do about ones productivity, and track record of successfully shipped products.
On that record, the comparison is not in favour of their American counterparts: a typical twenty something "code drone" will very likely score more on that than a senior dev or tech manager in the West.
I am checking Linkedins of engineering SVPs in Google and Apple now, near all have just 3-4 shipped products. I know a 25 years old who now counts his 21st engineering project in his career.
But more than that, that an everyday for an average migrant worker in a big city in China.
But man having an IT team or just tacking on sales, account management, support, legal, customization, billing ... and doing it well is a big lift. Some of those things are "soft of" like what folks do internally, but man with another company it can be WAY different. Let alone problems like a company who hasn't done those things fighting its own momentum to do it right....
I'm reminded of a story from Amazon someone mentioned on HN (or linked it) where they were working on early AWS like stuff and supposedly Bezos shows up for a meeting / retreat late. He asks what everyone is talking about. They were talking about "more cooperation" between the folks working on early AWS stuff and their own storefront people and the challenges the storefront folks had using this early form of AWS. Lots of cooperation talk.
Jeff didn't like it and said something about how he didn't like this cooperation idea (more harshly than I said it IIRC). I'll get this next part wrong to some extent but I think I've got the spirit of his next actions right: He then setup a system where the back end AWS folks and the storefront IT folks were actually discouraged from communicating directly. Rather the AWS folks were required to write documentation to reduce the number of interactions directly with the the people running the Amazon storefront. If the storefront folks couldn't figure out how to do something and opened a support ticket (no more direct calls) it counted against the AWS team.
The general idea here was to establish the same systems / practices you would need to deal with outside customers who would one day use AWS... and hopefully keep using it.
It's also a good example about how working on internal applications and external are way way different.
I wouldn't be surprised if there were MANY versions of the same story as a lot of people would have been impacted by the change in approach to everything, but they would have seen the implementation and results in different ways.
However, in my experience, internal software development is often tailored very tightly to specific environments and other tools that are used in this particular situation, and the pressure from higher-ups is to build something faster rather than generic (and even if it's built to be generic, that usually doesn't work well unless there are real use cases exercising it).
What I'd really like to see is several municipalities joining their forces and writing common open source software that works for their use cases. Would be soooo much cheaper in the long run than each procuring on their own.
A lot of legacy business remains of course and this seems big business for the likes of IBM and Oracle. But I don't know of any young companies signing contracts with either and would recommend to actively avoid that kind of thing to anyone.
I would love to sell access to this API as a service. Sure, you can write your own. Sure, you can buy your own EC2 instance with a gig of RAM to run Chrome. It's easy! I did it myself! Or you could pay us, say, $0.01 per PDF generated and not spend $30/month on that EC2 instance and $200/hour on writing the service and maintaining it.
I think it's a great idea to outsource as much as possible. Some stuff isn't written yet, however, or it's bad. So you have no choice but to do it yourself. The people that have done it themselves and then charge money for it are how outsourcing is possible in the first place. (In fact, that's how EC2 got its start. I feel like that "not focusing on their core business" of selling books over the Internet turned out pretty great for Amazon.)
Of course Coase would say that's a risky bet (the spun out business could fail for ordinary business reasons and then you'd have nothing).
The problem is that many companies incorrectly make that exact judgment, assuming that their competitors are, on average, buffoons.
But if you've spun it off, and especially if you got in outside investors, you've both partly cashed in and largely hedged yourself against that risk. Now you can just treat them as an external vendor and if they fail, buy elsewhere.
Right. And if there are no alternatives on the market already, that tells you that charging for it is not going to be viable. The way to make the most of that investment is to keep it alive, even if this means giving stuff away!
Or you could spend $0 and let the bits just rot if they're not in use - no lawyers reviewing licenses, no code reviews, nothing. Or you could send it through engineering to make sure there's no secret sauce, then through corporate legal (at $250/hour), then through business development...
It's almost always better to just smother the code with a pillow than it is to give it away, because giving it away actually costs money, and will bring you no profit. Screw that, if it's important enough, let someone else spend the time and money writing it.
Sure, but the underlying play is that clearing the code for release will at least allow you to share support costs with other actors in the same or closely-related industries. In many ways, it's a marginal choice that sits awkwardly inbetween "smother it with a pillow" and "make it a full-blown proprietary product that can actually bring sizeable revenue". That doesn't mean it cannot work in some cases.
If you, as Coase did, generally believe that markets are efficient it is not obvious why companies form where the effect of the free market is basically removed inside of the firm. Inside each firm some kind of technocracy/meritocracy rule instead, and resources are not allocated in the same way as outside the firm. If we believe that the free market is the best way to price and allocate a resource (say steel or something), we don't we use the free market to price and allocate resources inside the firm (say workers or machine capacity in a car plant)?
Of course he talks about things that we now consider "transaction cost", it would be a pain in the butt to outsource everything and do nothing "in house". But another reason he talks about is the risk of your subcontractor going out of business.
So to loop back to the question here, if the software is sold to another company which we then buy the services form, we have the risk that the company could fail and we would be without our services.
Econtalk I find excellent - a range of guests and a reasonable grilling given even if they are on his (right hand) side of the world.