Hacker News new | past | comments | ask | show | jobs | submit login
Companies outside the tech industry are spinning off internal software (wsj.com)
122 points by metaphysics 6 days ago | hide | past | web | favorite | 84 comments

I worked at a company that tried to do this - Willis Towers Watson. It went pretty badly for a number of reasons:

* The internal software was built up over a number of years as a series of hacks that rarely if ever got unwound. The people running the business were blind to the necessity of this and a lot of the engineers were not motivated to fix it because they were experts on the rabbit hole and were angling for job security.

* The internal software's infrastructure aged badly and we were stuck in a perpetual upgrade hell. This was again because the people running the business did not really understand the accretion of technical debt. Their abilities centered more around, say, pricing insurance and regulatory issues.

* They didn't know how to hire engineers and they instituted corporate processes that ensured that the engineers were both pissed off and unable to properly do their work and they were in a geographical area where it was hard to hire. Good engineers largely left, bad engineers stayed behind and continued building rabbit holes.

* They outsourced critical functions to cheap overseas labor and got crap as a result.

* Their response to slowed development and deadlines was to throw more engineers at the problem.

* They're also the only client I've had to sue for nonpayment of an invoice.

Let me ask an unpopular question: in which sense did it "go badly"?

You indicated a large amount of technical debt and issues, which the techie in me 101% empathizes with and shares your pain. You also indicated morale and hiring practices issue, which the person in me empathises with.

What you didn't indicate however was the business outcomes: Were there unhappy clients that left? Did the costs rise? Business fold?

Everything you mentioned, unfortunately, is also the case for some wildly profitable/successful businesses.... :-<

They had one large customer lined up. I didn't really follow what happened with them after I left but I don't think that customer was particularly happy and calls with this customer were fraught and somewhat terrifying.

It certainly wasn't the case that this project was wildly profitable. Not sure if it will ever make money at all in fact.

I have worked for profitable businesses who have behaved like this too and it's usually the case that they have some kind of inbuilt unfair advantage (legal, governmental, monopoly/size/clout, etc.) that compensated for sucking at tech.

...Their abilities centered more around, say, pricing insurance and regulatory issues...

You have to know how to price software, know how it is built and know what are challenges. It is just like I would want to open restaurant right now. I have enough money to do it, but I am going to be ripped by suppliers, chefs, staff.

Bad devs just sucking out salaries over time is one thing. Then you get silly project managers who want to prioritize FCKN EVERYTHING and don't understand project and don't care, just want their features in. Maybe even conflicting feature requests from multiple customers.

Sounds like costs rose if they threw more engineers at the problem.

Now imagine that you're buying software from a third party that's still doing all that, but you can't fix any of it because you don't own any of it, and you can't switch to another vendor because all your processes for the past half decade have been tied to this product.

I feel like this is overblown in this day and age, and certainly doesn’t justify the insane cost of having a dev team to be building you custom software that is anywhere near competent.

You can do 99.5% of what most non-technology businesses do via a combo of SaaS services. Those keep getting better over time. It’s cheaper to hire a large team of coders/consultants for a few months to migrate you than to pay the same said team in perpetuity.

> You can do 99.5% of what most non-technology businesses do via a combo of SaaS services.

What is is with tech people and pulling numbers out of thin air to justify their simplified straw man of users?

"just about everything" is the point, don't obsess over the number. Payroll, payments, IT, finance, sales, HR, benefits, inventory management, etc. are all available by subscription. Many of those used to require a team of developers in-house. Its easier than ever to run a business on these, and its FAR cheaper than building out software yourself.

Well, there is a big difference between 99.5% (i.e. essentially everything) and 60% or whatever. You rattled off a bunch of functions most of which are both quite complex and pretty standardized across large business.

I agree that pretty much any large organization writing their own email or payroll system in this day and age is probably certifiably insane.

But even large "non-tech" companies have fairly significant custom needs even if it's just integrations and customizations of SaaS. Just because you're using Salesforce doesn't mean you just point everyone in the company at your Salesforce instance and call it a day.

Just about everything except for the business itself. Cool, you outsourced a bunch of operational stuff.

What you're forgetting is that insurance companies still need their custom insurance software with all of their custom risk models built in. That's the entire value proposition of the business so your estimate that it's only "0.5%" of what the company needs is way off.

Eh maybe. Technology can be a major differentiator for business success even for a non-tech player. I am not sure what a non-tech player is anymore.. Dominoes is pretty tech savvy as is Chick-fil-A for instance.

There can be a need for a technology team on a big player, but for most its not necessary. Dick's Drive-In in Seattle doesn't need much more than a mailing list, a shopify site for merch and an HTML site. They use doordash for online orders. They use an off-the-shelf PoS system and they can buy into whatever timecard system they want.

In other words, Big Ball of Mud https://en.wikipedia.org/wiki/Big_ball_of_mud

Not just that though. Also the existence of a somewhat toxic work environment which functioned sufficiently where all they needed was cheaper, less skilled, easy to hire employees doing more rote kinds of work but which hampered both creativity and getting shit done and meant employees with other options simply left.

Template zombies, as coined by Mark Schwartz in the book "A seat at the table". Managers who think that they can turn everything into a template, and then hire monkeys to take those templates and fill in the blanks.

Everyone knows the manager is the most valuable person in the room. They have the ideas. The rest of us are just implementing their vision. So easy.


Right. This approach becomes evident when a manager refers to developers as "resources".

The term is "amateurization", and it can lead to efficiencies since the lesser skilled labor pool is larger.

You'd be surprised how many companies operate with this biz model today. I call em zombie companies. management just sucks out the profit while they slowly crumble. Sad for all involved. Spinning out internal CRUD from a zombie co. is a whole nother ball game though

Yup, and backdoor galore!

This was again because the people running the business did not really understand the accretion of technical debt.

I don’t have any additional context besides my consulting experience but I’m willing to bet several engineers did their best to make everyone aware of the technical debt and were ignored.

the business did not really understand the accretion of technical debt

And WTW are a risk-management firm. The failure to recognise technology as risk to be managed seems particularly perverse.

The irony was everywhere. I wasn't paid because of a bug in their HR IT system. They also sell HR IT systems.

> While the basic strategy isn’t new— Amazon.com Inc. was an online bookstore long before it became a tech industry powerhouse

25 years in and still not getting it.

Amazon has always been and will always be in the business of cheap, scalable logistics. Not selling books. Not selling the cloud. Selling access to low cost scale.

This is hindsight. When Amazon started, it was a pure e-commerce play. It later evolved into the company you describe.

For example, they signed a bunch of exclusive supplier agreements for their store. Then, when they launched Marketplace, they got sued by some of those suppliers for breaking exclusivity.

E-commerce success created the cashflow and in-house expertise that allowed Bezos to explore other markets. Some succeeded, like Marketplace and AWS. Some did not, like the A9 search engine and Fire phone.

In what sense can Celestica, a company which produces 'Communications, Enterprise and Cloud Solutions, Industrial, Aerospace and Defense, Renewable Energy, HealthTech and Capital Equipment', be considered 'outside the tech industry'?

annoyingly in some circles the phrase 'tech' became synonymous with "user facing software" around the last 10 years or so, so now everyone is baffled when Walmart writes software.

And maybe to make a larger point, I think there's been bad consequences of constantly overhyping the innovation happening in web technology and software, while underestimating the innovation in manufacturing, aeorospace, defense, logistics and so on. There's much more overlap between the general economy and the latter, and it's much more regionally diverse. I think we could have done a little better if more talent went into those 'boring' sectors, rather than building the next sharing app in the valley.

Agreed, but it seems that the web tech startups are the ones that attract the most VC attention. Maybe they've optimized their pitches more towards VC risk models? Maybe the aerospace etc. don't have the same unicorn promise of exponential growth?

Web offers a chance at scaling that's hard to achieve in other technical work. VCs are looking for one of their investments to scale massively to compensate for all the others.

I think that's being really nitpicky. None of the other firms given as examples are in any way tech firms. Celestica is mentioned in a single sentence as part of a list of firms. The overall theme of companies outside the software industry creating software and marketing it is pretty clearly communicated and unambiguous.

I have seen egregious uses of the term 'tech' used to just mean specifically silicon valley internet software, but this case is more a marginal oversight of one example in passing, rather than a deliberate or misleading misuse.

In the sense that they weren't doing all of those things before they started doing them.

The cloud, communications and enterprise solutions are new fields for this _hardware manufacturer_.

"Tech" is not a synonym for "software". Companies that design electronics are tech companies.

While I agree, strongly, with you, software (or more pointedly, Web-and-mobile information technology) is what the general awareness of "tech" seems to be.

The reality is vastly broader.

Celestica has been around for decades. They are one of the original tech companies, the way Apple or IBM or Microsoft or DEC or Compaq are.

They were doing wireless communications in the early 2000s.

It’s worth noting that nothing Amazon built internally was ever directly sold through AWS. Rather, Amazon built something similar internally then used its engineer expertise to build a completely new product for AWS. Selling your big ball of mud directly is a recipe for disaster.

That, but I also have a theory about Version 2. By the time something hits v2, it's popular enough to justify the work to get to v2, and the lessons that were learned from v1 can be applied, even if that leads to breaking changes.

Not sure I understand the negativity. Yes non tech companies might produce horrible programs, but at least this is a breath of fresh air vs the standard “vendors only” IT approach. There are plenty of horror stories of companies trusting “enterprise” vendors to avoid doing their own programming. For instance I have yet to see a salesforce implementation that works well and is adopted by salespeople, meanwhile something a bit more custom will most likely have automatic adoption. Anyway, I think extremes are always bad so I welcome companies moving away from the “standard vendor products” mantra.

> Other companies to spin out businesses from internal tech efforts include Choice Hotels International Inc., which offers a cloud-based reservation system for hotelier

As if travel companies spinning off CRS systems is something new... every single train/airline/hotel reservation system out there is a direct or indirect spinoff from an internal company product

Having worked for a few shops where the internally developed software I helped write was seen as an expense, and a tool to facilitate the company's real work, I'm skeptical this will work all that well in the general case.

My data points, though limited in number, indicate a strong tendency on the part of these kinds of shops to build solutions that are over-fit to their particular workflows, and to under-resource their development. I was, in one of these environments, explicitly told that version control and separate development/test/production — or even just dev and prod — environments were a silly extravagance.

I'm sure there exist shops that don't do it that badly. I'm more sure, however, that they're not the norm.

Collapsing dev, test, and prod into one environment sounds like a 300% increase in efficiency - now that’s a number management can really get behind!

I think US companies are just now realising just how much of their business activity can fit into few lines of code.

The thing happened in China over last 5 years:

- Restaurants run ordering APIs for food delivery companies

- Restaurants run ordering APIs for online menu companies

- Laundries run restocking, and order ready notification APIs

- Logistic companies pretty much became a few layers deep API business with most companies having 0 user facing "frontend"

- Hotels - some will deny you booking if you don't checkin online. Some went even further, and check in you remotely

- Construction companies - now run with client facing construction site management software, and run joint CAD projects with clients

- All kind of odd job service aggregators now do what is called o2o: dog walkers, handymen, porters, cleaners etc

- Many business service companies simply went out of business because their entire businesses were turned into free apps. You can file your taxes with a free app that scans QR code or electronic invoices for example

- All kind of retail store interact with wholesale suppliers only through some EDI stockkeeping or a more advanced API

- And for commerce, things are all self explanatory: JD and Alibaba

You make it sound as if the Chinese are more imaginative than Americans. That's not the case. They just have more people. If I am sitting in NYC and need 10 programmers overnight for a boring IT project... no problem. But if I am sitting anywhere else in the US it's going to take more time. Not the case in China. Or India. The quantum of activity is directly proportional to how much labor you have access too especially in "unsexy" IT work.

And kids in the US rightly focus on the sexy stuff because when you compare just Apple to the entire Indian IT sector revenues are 3x more. Why would you want to go work on laundromat or construction company APIs when you have a shot at getting into Apple or Google.

I don't think China is more imaginative, but they are far more nimble. For instance, if a city is growing, they just plop down a ton of apartment buildings, throw in 8 new subway lines, connect a bullet train and they're done.

The US spends about that length of time discussing on whether or not to build a new tower in a historic neighborhood.

I don't know why this is the case, but I think it has something to do with a rapidly growing economy versus an already grown economy. If someone could explain to me why that's the case, I'm all ears.

My off-the-cuff guess: if a country is growing, the easiest way to make money is hop into a growing industry and building as fast as you can. If a country is grown, the easiest way to make money is to siphon it from somewhere else. So in developed economies you have stagnation and inefficiency because the elites are more interested in redistributing the wealth (to them), instead of creating new wealth.

The other big factor is the government can make unilateral decisions because they're not a democracy and their human rights are a joke. If they want to tear down your building to build a new one they'll buy you off or if you refuse that kick you out by force, historical value be damned.

The citizens in the US and their political power are the source of most of the inertia.

They don't always kick owners out by force. Search for "nail house". Sometimes they just go on about their development as though the holdouts are not there. Sometimes they cut the water and power in the middle of the night.


> If they want to tear down your building to build a new one they'll buy you off or if you refuse that kick you out by force, historical value be damned.


My old college is doing this to an old employer/business owner/friend. If anyone feels passionate about Eminent Domain being theft; check out their petition.


> My off-the-cuff guess: if a country is growing, the easiest way to make money is hop into a growing industry and building as fast as you can. If a country is grown, the easiest way to make money is to siphon it from somewhere else.

You could replace "country" with "company" and describe why startups are continuing to disrupt existing industries.

This is typically framed as "Grow the Pie" vs "Split the Pie" https://en.wikipedia.org/wiki/Grow_the_Pie_(phrase)

> You make it sound as if the Chinese are more imaginative than Americans. That's not the case. They just have more people.

How to say this without an ad hominem?

> Why would you want to go work on laundromat or construction company APIs when you have a shot at getting into Apple or Google.

Why would one be even bothered by that? Apple or not, anybody more or less seasoned professional knows his price and capabilities. My experience in the West shows that MNCs and dotcoms are full of kids just wanting to be abused by their employer for: 1. nice record in the resume; 2. sense of "social status"; 3. getting that "artsy startup cred" approval mark.

If laundromat API pays 3/4 of Apple's wage for an equivalent duration of work, but is many, many times easier to get than a job at Apple, I think there is not question what gig to pick up.

That's a typical wishful thinking of a "creative class wannabe" who has tons of insecurity about his claim to entitlement coming with that "artsy high class person" label.

Chinese devs may be taking someones job in the West, but it has no dependence whatsoever on them being evaluated for their "creativity." This has much more to do about ones productivity, and track record of successfully shipped products.

On that record, the comparison is not in favour of their American counterparts: a typical twenty something "code drone" will very likely score more on that than a senior dev or tech manager in the West.

I am checking Linkedins of engineering SVPs in Google and Apple now, near all have just 3-4 shipped products. I know a 25 years old who now counts his 21st engineering project in his career.

Honestly that laundromat job sounds interesting assuming its bunch of IoT devices sync'ing data to the cloud (eg: CoF payments, push notifications when laundry is done, etc), retrofitting machines with SBC devices to identify when a cycle is done, triggering payments etc.

Are there any english language sources that cover trends in the tech ecosystem like this in China?

Only like one bit here and there. Most of "sputnik moment" styled youtube videos about China internetising everything are more or less representative of the state on the ground.

But more than that, that an everyday for an average migrant worker in a big city in China.

This is fascinating. Would love to read more about it.

This is somewhat a sequel to WSJ's "Why Do the Biggest Companies Keep Getting Bigger? It’s How They Spend on Tech"


Certainly there are lots of internal IT teams developing solutions that could be useful elsewhere.

But man having an IT team or just tacking on sales, account management, support, legal, customization, billing ... and doing it well is a big lift. Some of those things are "soft of" like what folks do internally, but man with another company it can be WAY different. Let alone problems like a company who hasn't done those things fighting its own momentum to do it right....

I'm reminded of a story from Amazon someone mentioned on HN (or linked it) where they were working on early AWS like stuff and supposedly Bezos shows up for a meeting / retreat late. He asks what everyone is talking about. They were talking about "more cooperation" between the folks working on early AWS stuff and their own storefront people and the challenges the storefront folks had using this early form of AWS. Lots of cooperation talk.

Jeff didn't like it and said something about how he didn't like this cooperation idea (more harshly than I said it IIRC). I'll get this next part wrong to some extent but I think I've got the spirit of his next actions right: He then setup a system where the back end AWS folks and the storefront IT folks were actually discouraged from communicating directly. Rather the AWS folks were required to write documentation to reduce the number of interactions directly with the the people running the Amazon storefront. If the storefront folks couldn't figure out how to do something and opened a support ticket (no more direct calls) it counted against the AWS team.

The general idea here was to establish the same systems / practices you would need to deal with outside customers who would one day use AWS... and hopefully keep using it.

It's also a good example about how working on internal applications and external are way way different.

That story sounds similar to Bezos' mandate in Steve Yegge's Google platform rant: https://gist.github.com/chitchcock/1281611

That sounds very similar.

I wouldn't be surprised if there were MANY versions of the same story as a lot of people would have been impacted by the change in approach to everything, but they would have seen the implementation and results in different ways.

Not the same thing, but I still find it amazing that Django was created by the Lawrence Journal-World.

If it works, it's great. There are a lot of companies doing similar things that duplicate lots of effort building similar applications.

However, in my experience, internal software development is often tailored very tightly to specific environments and other tools that are used in this particular situation, and the pressure from higher-ups is to build something faster rather than generic (and even if it's built to be generic, that usually doesn't work well unless there are real use cases exercising it).

What I'd really like to see is several municipalities joining their forces and writing common open source software that works for their use cases. Would be soooo much cheaper in the long run than each procuring on their own.

Many airlines could very well be tech companies at this point. Same for some other "old school" stalwarts like UPS and Fedex.

I worked for a company 15 years ago that had build a nice collaboration website like Confluence/Jira for internal use. It was built by an internal team of 5 engineers. Very nicely done UI. The company had a lot of things going for it but it missed a few strategy opportunities and was bought for a song and split up. In hindsight, I feel like that collaboration software could have been a business of its own but nobody recognized it. Maybe 10 years too early.

One of the things I'm most sick of in technology is home-grown software. I've been re-writing and re-implementing the same solutions for 15 years, because apparently nothing anyone creates is good enough. It's like we're in the business of selling door to door, but to do that we make our own cheap shoes that fall apart after a few years and have to be re-built again.

Most companies can't afford a proper IT department and that just rules out a whole range of things they can do. IT is expensive; good IT is much more expensive. If it's not mission critical, SAAS is a great solution. Once you go SAAS, you can outsource some of the people using those solutions as well. You can outsource IT, HR, accounting, etc. Even sales, marketing, and support functions can be outsourced.

A lot of legacy business remains of course and this seems big business for the likes of IBM and Oracle. But I don't know of any young companies signing contracts with either and would recommend to actively avoid that kind of thing to anyone.

But but but... what about focusing on their core business? What about outsourcing everything else? Not only that, this would require management to recognize value in something else. Were not in a Dilbert book here.

I think a lot of incidental tools are generated by focusing on your core business. For example, I work at an ISP. Something we do is generate Letters of Agreement for cross-connect at shared sites. This involves making a nice-looking PDF from some HTML that the customer's information is substituted into. So I wrote something to use Chrome to generate the PDF, and we couldn't get the page to look nice by using wkhtmltopdf's antiquated rendering engine. Is that our core business? Nope. But it's something we had to do because people want a PDF they can print out and give to their datacenter. No PDF, they can't give us their money. Incorrect PDF, and they get fed up and switch to a competitor. So I think it was worth the day or whatever I spent on it.

I would love to sell access to this API as a service. Sure, you can write your own. Sure, you can buy your own EC2 instance with a gig of RAM to run Chrome. It's easy! I did it myself! Or you could pay us, say, $0.01 per PDF generated and not spend $30/month on that EC2 instance and $200/hour on writing the service and maintaining it.

I think it's a great idea to outsource as much as possible. Some stuff isn't written yet, however, or it's bad. So you have no choice but to do it yourself. The people that have done it themselves and then charge money for it are how outsourcing is possible in the first place. (In fact, that's how EC2 got its start. I feel like that "not focusing on their core business" of selling books over the Internet turned out pretty great for Amazon.)

There are plenty of HTML-to-PDF web APIs though so if you want to spin a SaaS off of it you need to spend more time than just host it somewhere, e.g. some kind of market differentiation, SEO, work out pricing etc. If you're a lone developer sure, you can probably get to beer money profit quickly and kinda leave it there but your company needs to weigh the perspectives of a spin off against resources they remove from their "core expertise".

It's the other way around: if my business is making furniture and the company made a scheduling program I'm better off spinning it out to someone who will focus on it, and just continue to use the product. So instead of it being just a cost center the company gets a bit of possible financial upside and hopefully the product becomes better via market exposure.

Of course Coase would say that's a risky bet (the spun out business could fail for ordinary business reasons and then you'd have nothing).

If that scheduling program is proprietary and a significant competitive advantage to your particular furniture making business, you may prefer to pay the higher cost to keep it internal.

The problem is that many companies incorrectly make that exact judgment, assuming that their competitors are, on average, buffoons.

>Of course Coase would say that's a risky bet (the spun out business could fail for ordinary business reasons and then you'd have nothing).

But if you've spun it off, and especially if you got in outside investors, you've both partly cashed in and largely hedged yourself against that risk. Now you can just treat them as an external vendor and if they fail, buy elsewhere.

If they could buy elsewhere why did they build the software to begin with?

If you can't buy elsewhere, and don't care about keeping the software internal as your unique selling point, arguably the right thing to do is to open source it. That way you make sure it stays around, and attract third parties to support it. But there's also a lot of internal software around that wouldn't yield any benefit from either spinning off a profit-seeking business to commercialize it, or open sourcing.

The right thing to do is whatever makes the most of your investment. Don't give it away - charge for it.

> The right thing to do is whatever makes the most of your investment.

Right. And if there are no alternatives on the market already, that tells you that charging for it is not going to be viable. The way to make the most of that investment is to keep it alive, even if this means giving stuff away!

Releasing things usually comes with overhead - you have to verify that everything you're releasing doesn't have anything proprietary, that all other code is following a license that allows for release, etc.

Or you could spend $0 and let the bits just rot if they're not in use - no lawyers reviewing licenses, no code reviews, nothing. Or you could send it through engineering to make sure there's no secret sauce, then through corporate legal (at $250/hour), then through business development...

It's almost always better to just smother the code with a pillow than it is to give it away, because giving it away actually costs money, and will bring you no profit. Screw that, if it's important enough, let someone else spend the time and money writing it.

> ...It's almost always better to just smother the code with a pillow than it is to give it away

Sure, but the underlying play is that clearing the code for release will at least allow you to share support costs with other actors in the same or closely-related industries. In many ways, it's a marginal choice that sits awkwardly inbetween "smother it with a pillow" and "make it a full-blown proprietary product that can actually bring sizeable revenue". That doesn't mean it cannot work in some cases.

Well, how long is a piece of string? There could be a million reasons, depending why the product failed. Maybe a competitor came in and wiped out their market. Maybe, maybe. I'm just saying spinning it off doesn't have to be a source of increased risk and can be a way to hedge risk.

Coase and theory of the firm? How does that turn into a risky bet? (Asking for understanding not for challenge)

Ronald Coase, who kind of invented the field about "the theory of the firm", asked in his 1937 paper "Nature of the Firm" why companies exists basically.

If you, as Coase did, generally believe that markets are efficient it is not obvious why companies form where the effect of the free market is basically removed inside of the firm. Inside each firm some kind of technocracy/meritocracy rule instead, and resources are not allocated in the same way as outside the firm. If we believe that the free market is the best way to price and allocate a resource (say steel or something), we don't we use the free market to price and allocate resources inside the firm (say workers or machine capacity in a car plant)?

Of course he talks about things that we now consider "transaction cost", it would be a pain in the butt to outsource everything and do nothing "in house". But another reason he talks about is the risk of your subcontractor going out of business.

So to loop back to the question here, if the software is sold to another company which we then buy the services form, we have the risk that the company could fail and we would be without our services.

https://en.wikipedia.org/wiki/Theory_of_the_firm https://en.wikipedia.org/wiki/Transaction_cost http://www.econtalk.org/munger-on-the-nature-of-the-firm/

Thank you

Econtalk I find excellent - a range of guests and a reasonable grilling given even if they are on his (right hand) side of the world.

Samsung started as a grocery store.

Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact