Oh boy, been there. This is Conway’s law in action - they are about to go and create the same code soup again - because the org won’t have changed. Having been in the same boat my lesson is positive engineering initiatives can only come top down - through some very senior champion. You can’t change an organisation from the bottom up - and it’s the organisation that produces the code base.
Top down comes with its own risks. The bottom line is big ideas and big change can only happen with both the champion at the top, as well as good understanding at the bottom, and a critical mass of good alignment and competence throughout the middle layers. Though Conway's Law is immutable, it doesn't necessarily depend on a formal org chart, it can be worked with by simply creating ad-hoc communication structures between the right tech leads and competent managers. A handful of technically weak or empire-building managers in the middle can fuck the whole thing pretty easily though, and depending on the lifecycle of your company it may be a lost cause due to Pournelle's Iron Law of Bureaucracy.
Although it’s not always necessary to go through the middle - you can do the old switcheroo and build a parallel org in your image, emerge from the shadows and duke it out with the old structure.
If you come at the king you best not miss. Fail at taking down the old structure and those relationships are so destroyed they will come after you. I’ve seen groups of consultants become unemployable across the whole industry by failing to take down the old guard. Word spreads.
> as well as good understanding at the bottom, and a critical mass of
good alignment and competence throughout the middle layers.
How do you hold the middle together? Especially under such turbulence?
"Strong" leadership isn't enough.
Reflective practice [0] is one way to build that. It's common in
medical and some safety-critical civil/military spheres but strangely
missing from many corporate structures. I talk about it here with
regard to a more mature cybersecurity stance [1]. It intersects with
Agile ideas but doesn't make much noise in software engineering
project management as much as it should? Anyone have this in their
org?
That highlights the greatest repeated tragedy to befall software: poor leadership. For some bizarre reason developers almost always think they can fix people problems with better tools. For example if all your developers suck give them a popular framework. It’s just an excuse to avoid dealing with people that provides license for the children to run the daycare.
If you want excellence set high standards defined by rules that to hold people accountable and impose ownership (liability/reward). It’s not complicated but it requires not being terrified of assertiveness and confrontation from the top down.
> It’s not complicated but it requires not being terrified of assertiveness and confrontation from the top down.
It requires the professional communication skills that are not taught in a STEM education. That's why this is all so difficult: we are not taught how to communicate effectively with one another, and the entire tech landscape has poor communicators misinforming and misleading one another.
That cannot be over-stated enough. I learned my communication skills in the military while building my software career. Those worlds are so incredibly different that I have spent most of my professional life lost.
On one hand my expectations for excellence in software are alien to my peers who just want to complete a JIRA task by any means possible. Yes, it is possible to deliver superior results for an entire day's effort in about 2 hours provided you aren't framed by all kinds of institutional roadblocks and unnecessary abstractions. I am scared to death to actually communicate my opinions regarding software because people tend to whine about how hard life is. For this reason I have abandoned my JavaScript career.
Then on the military side I have to unlearn the high agreeableness that is so necessary for not getting fired in software. By high agreeableness I mean keeping your opinions to yourself, not voicing concerns, treating everything with kid gloves, and making everything as easy as possible. The military side is a much safer environment that expects brutal honesty. Nobody there believes themselves to be god's gift to the universe and they are always in a learning mode, which means they are coach-able and will not whine about corrections and mentorship.
Although I do not have military experience, I've spent some time writing software deployed to military security clients, and my interactions with technical armed forces staff has had remarkably good communications. To the degree it changed my attitude about the armed forces in a significant positive direction, and not just the tech side.
was referring to the fact that military personell tend to be trained fighters with a heightened ability to inflict physical damage if threatened or distressed.
I do not believe so. This lack of communication skills is also due to a lack of recognition of the value good communications engenders. An undergraduate business school education pragmatically touches on professional communications, but only as far as how to motivate and control people, not how to effectively communicate - as in a shared synchronization of understanding, which is what professional communications is all about. Colleges of communications teach professional communications as their foundation, before the various media majors one graduates within. The more I look at the levels of communications in general within society, our civilization has not yet recognized how critical communications are to every aspect of life, not just business and careers. Good communications are like fire, capable of destroying and capable of being the foundation greater things are built, and the lack of this recognition amazes me.
Interesting, as that is what my current work is all about: integrating LLMs into ordinary software people use everyday, and those LLMs serve multiple purposes: a) they know the software they are integrated and can offer support in how to use the software, b) a 'psychologist AI' looks at what the user is doing and makes an assessment of how well they understand both the software they are using and the content of their use of the software, that assessment is used to modify the amount of help and the language used when helping, c) both the assessment from b and the content from whatever they are authoring is used to generate an expert in the subject matter of their content, whom is then available in a chat interface with access to rewrite the content the user is authoring upon their request. And d) when a "completed" document is viewed (not in an editor) a slightly different expert is generated based on the context of the document, whom is available for extended Q&A of that document and it's subject.
I'm calling this work 'human comprehension support'.
I've witnessed what Casey calls "Conway's Nightmare" (at 32:04 in his video) firsthand- it is definitely real. Basically, a software system isn't necessarily just a reflection of an organization, it's an amalgamation of all of the organizations that worked on the software system in the past.
Does Conway’s law imply that an organization of size 1 produces a system with components that are optimally integrated?
I sometimes wonder what would happen if a company hired 100 software engineers, gave them all the same task to program the entire thing independently on their own, and then paid them as a monotonic function of the ranking of their output against their peers. (Obviously, there are limits on what type of product could be developed here, but I think this still encompasses far more company products than you would suspect.)
> Does Conway’s law imply that an organization of size 1 produces a system with components that are optimally integrated?
It’s creative work, like anything else. 1 is not necessarily always the right size, but 100 certainly isn’t. If you look at musical bands usually a sweet spot is between 1-6. You can add more in eg an orchestra or choir but everyone’s not involved in the creative work.
Then it depends on how well you vibe with people. If you have extremely unique ideas and thoughts, you may be better off alone. If you find peers with similar values, preferences and experiences, you might be fine with 5 people.
This feels like something that is just crazy enough to work.
Most teams I've been on or managed were at their optimal efficiency with three people. Just enough to keep any one person from making too many poor decisions, but not to many to spend half a day in meetings.
An organization of size 1 has other constraints, like time.. But also, Conway's law applies outside the organization as well, communication constraints shape society at large and the way a society is organized limits the communication channels available.
On the topic of Microsoft/Linkedin, aren't they the ones who used to or maybe still assign the same task to multiple teams and let the teams compete for who can deliver? That does sound vaguely like what you propose.
I remember one time we had to provide some important numbers to auditors. A data analyst on the team wrote some queries to collect this information, but my manager decided to write his own queries independently as a consistency check. The numbers didn’t match at all.
So he asked me to write my own queries as well to see who was right. Lo and behold, my numbers didn’t match either of theirs at all.
So clever! The brief answer is you have many staff who have knowledge but not creative authority. If you think about it for a few minutes I think you’ll find some options.
This video explains everything I've seen in the enterprise IT of a huge government org that was the result of a long series of back-to-back mergers and splits.
It's exactly this!
Conway's law, but over time.
Dysfunction not just because of the large hierarchy, but also because of the built up layers of history. Some elements still limping along, some essentially amputated and mere phantom limbs of the org tree now, some outright dead and spreading miasma and corruption from their decay.
I knew or suspected most of this, but to hear it said out loud so clearly has crystallised my understanding of it.
Yeah once it hits, it really explains so much of the issues in large organizations (and explains a lot about small organizations / indie development as well). For example, why is it that mergers and buyouts rarely work out? Conway's law largely explains it! When a product is bought by an organization, the result is an integration over time of both organization org-charts combined. If the companies are both large, this immediately becomes an intractable mess. If the original company was small and the new company is large, it _also_ becomes intractable because now different teams will have to communicate to manage or split components that don't fit the communication patterns of the new organization, and will lead to large refactors and rewrites and in the end the essential qualities of the original software will inevitably change.
Even if you know this, there is nothing you can do - you have to organize yourself somehow and how you do that can't be left up to just mirror whatever this piece of software that was brought in happens to be organized - because _that too_ doesn't reflect an actual org chart but is an integration over time of all org charts that have touched it.
I don't even know how to think about the implications this has for the use of open source software and how open source is developed, like... I think there are huge opportunities for research into Conway's law and how it relates to software development practices, software quality, etc.
I have this naively optimistic hope that AIs will allow orgs to scale units past Dunbar’s number.
We humans can’t effectively communicate with groups larger than about 7 people, but AIs have no such limits. We could all simply talk to one manager that can integrate everything and everyone into a unified whole.
It’s like the ship Minds in the Culture series having hundreds or even thousands of conversations at once.
The more the initial fade of AI assisted work sets in, and given the inherent vagueness and unpredictability of managing, I'm eagerly awaiting not my job, but my bosses job being replaced by AI. There's no need for exactness, but superficial clarity, decisiveness and seeming coherence.
That's an interesting thought, yeah... technology and communication are definitely interwoven, things are possible today that were impossible before computers simply due to communication constraints.
One could imagine a large number of practically autonomous small teams organized "automatically", more mirroring a neural network than a hierarchy.
The problem is always communication because it is the means to cooperate. The root of many issues in software development is the simple fact that instead of letting the required communication pathways define the organization, it is the organization which defines the pathways and through that causes communication obstructions.
"Not Just Bikes" has a good few videos, including https://www.youtube.com/watch?v=n94-_yE4IeU and a couple more that talk about problems that larger roads effectively cause more traffic ("induced demand", "traffic generation"). Organizational structures are like roads, and like roads they can get overloaded, which in turn means traffic needs to be reduced. There is even communication jam, and to combat that something like enforced communication reduction (lower information throughput), etc. to keep this manageable. That also causes decision making being done with less and less information the more steps are included in a communication chain (like upwards/downwards in a hierarchy), which in turn means the quality of decision making is severely hampered by it.
This whole mess is also the reason why the agile manifesto puts humans before processes and other such things, in fact it implies you change even the organizational setup to fit the project, not the other way around. But in the space of "managerial feudalism" (David Graeber) this is pretty much impossible to pull off.
The tragedy of agile is that the practices that are labelled agile in practice tend to exemplify the exact opposite of everything that the manifesto advocates..
You might be correct, but the AI minds that you are contemplating don't exist yet, and there is no reason to think that they will be developed from current LLMs.
Once again, seizing the term AI to mean LLMs and other current generative techniques has poisoned clear thinking. When we think "AI", we are thinking about HAL and Cortana and C3PO, which is not what we actually have.
interestingly positive, I think I might agree with you. It seems the nr.1 problem of good leaders in organizations is that they can't be everywhere at once. But an AI can be.
I've watched a lot of Casey Muratori's other presentations but just happened to find this one last week and I wholeheartedly agree. Like many people I'd heard of Conway's Law but always imagined it as a pithy truism and hadn't thought that the effects could run so deep.
Casey's example is focused on a (frankly, ridiculously) large organisation but I've seen the same thing in numerous small companies with just 20-30 developers, and it's hard not to imagine that this is universal, which is a pretty depressing thought.
Recently I've been involved in a new project where teams were set up to be 'domain-specific' with the idea that this somehow avoided the issues of Conway's Law, but this just feels like exactly the same problem because team structures and the software that they end up producing is siloed in exactly the same way as the organisation structures that existed at the time.
Casey's point that the final design is inherently limited by the lack of high-bandwidth communication between the groups of people that need it most is also fairly demotivating, personally.
Tangentially, having been writing more Golang recently this made me think of Golang's mantra (or was it Rob Pike's quote) of "Don't communicate by sharing memory; share memory by communicating.". Go/CSPs concurrency and sharing model is interesting, but I wonder if it's a fair analogue to compare sharing memory by communicating as low-bandwidth communication, and communication by sharing memory as the high-bandwidth communication that seems to be needed for effective designs.
To be fair, it’s only a web application if you want it to be. I have all the usual office products installed locally. When I get a link that opens a document in the browser I click open in app and off it goes.
Microsoft has moved very quickly to add LLM functionality to multiple products
Literally the opposite of LinkedIn. I’m approximately user 5000 on linkedin, and since I joined almost nothing useful has been added, despite a huge increase in complexity
LinkedIn and Microsoft are the same, that said I personally think the rush to push LLMs into everything is entirely consistent with my prediction and with the internal structure of Microsoft (current and historical).
Come on now, that's not true. Every week LinkedIn adds a new email marketing category which you haven't opted-out of yet, because it didn't exist before.
Exactly. Remember that we are not customers, we are the product. Presumably, LinkedIn is adding a lot of useful features for their actual customers, they're just not visible to us.
I think it is more nuanced, you can also change the organization bottom up, but it needs to be new stuff that does not exist or has it infancy. Changing existing stuff is very hard even top-down.
Have you changed an organisation from the bottom up? In two decades I’ve never seen it happen in organisations larger than 50 people.
You can change something, like how an engineering team works and how an organisation does DevOps and other things that management doesn’t really know anything about and trust their employees on. But moving an organisation into something like team topologies which is a more modern expansion on Conway, is virtually impossible from the bottom up in my experience. Change management in general is very hard both directions, but it’s much harder going upwards because going upwards means you don’t really have the support of management above you. Maybe they’ll humour you but to make an actual impact you’re going to need them onboard. You’ll even have to reach pretty far up top if you want to inflict lasting culture changes as things carried by a single (or few) managers often dies with them.
My career has sort of put me in a role where I help non-tech startups transition from that to medium or enterprise size. I solely focus on the tech journey though, as I know I won’t have too much impact on culture or work processes on the grander scale. Often this leads to a semi-team topologies approach as the tech teams divide systems into independent business related services, but as a whole I don’t expect any changes to reach outside of the development departments.
I'm not disagreeing on your overall point, but it can be done (I've done it with 130+, so there's a lot of incumbent process and numerous senior managers to align).
The key is having a champion who can tie change in with higher level desires.
In these orgs there's often an external (as in stakeholders external to prod eng) perception that "engineering is slow, product doesn't deliver, they're preventing us delivering our OKRs", and that can be used as leverage for change.
You can't go dark and stop delivery, but you can usually carve out a 10 - 20% allowance for change (that doesn't sound much, but is 1 - 2 days every 2 weeks, which quickly adds up). Start small, show success in impacting metrics that external stakeholders care about, then next quarter push for more.
I've focused myself in a similar area as you, but actually lean into processes and people, whilst still guiding tech - maybe we should chat!
> new stuff that does not exist or has it infancy.
If it was new stuff, then the people doing it must not be very low at the bottom (by virtue of the org being quite shallow - such as a startup).
Even new stuff in a big company with lots of layers of management cannot make new stuff with excellence. Eventually, probably sooner rather than later, it gets polluted.
Not to mention that in a big org, the people at the bottom do not get rewarded for making excellence in engineering, if the top doesn't appreciate it. SO why go that extra mile, when a mediocre job will earn you the same salary? You'd be more incentivized to move jobs to grow your pay instead, and use this as a stepping stone.
As somebody who has only ever worked at places small enough for individuals to all roughly understand eachother's impact, could I even get a job at a big org like that? I don't even understand the mindset
Yes of course. You have to demonstrate that you are very skilled at building with the particular bricks they use. Don't mention anything else to show your commitment to particular brick usage.
Sometimes, I daydream about being a nameless drone in a cubefarm, taking home a nice paycheck and leaving my work at work. But, yeah, the dream passes quickly because I know I'd hate it with a burning passion.
If you want to seek meaning in your work, then yes, cubicle farms will not give you fulfilment. But if you're just after a steady paycheque, and don't give a damn about the business, your work nor its purpose, then this is fine. And it gives you free time to live outside of work too.
Conway's Law isn't a "law" - it is an interesting observation and there are definitely interactions (both ways) between organizational structure and software "design". But people weigh it too heavily. There are plenty of counterexamples of, for instances, fine-grained teams in one company that build a monolith, and the same type of teams in another company building a microservices architecture. Complex software is complex, and once your require more than a handful of people, that complexity requires a complex organization structure to handle the challenges of communication and responsibility. But it isn't a purely deterministic relationship between organizational structure and software structure. More that certain organizational structures are really bad at building complex software, and some are better.
It is interesting to think about, but there is this tendency to invoke "Conway's Law" as some simple method of fixing the extraordinary complex problem of developing and evolving complex software over time.
Really what it is is that organizations from the 70s, 80s, and 90s were built around building physical things, for the most part. Manufacturing and construction, mostly. And taking those organizations and applying them to software development fundamentally was a terrible match. We had to find new structures that were better suited, and we have gotten a lot better, but we are still working on it. Management was taught by those that managed manual laborers, who's output was easy to measure, and whom you could force to work a certain amount without a significant drop in their output. The same isn't true for knowledge workers. And so new management had to be created, a process we still are working on as well, but many managers these days are way better than managers from 30 years ago.
Read Conway's original paper, it is interesting, but it isn't a physical Law that cannot be violated!
> But people weigh it too heavily. There are plenty of counterexamples of, for instances, fine-grained teams in one company that build a monolith, and the same type of teams in another company building a microservices architecture.
These aren't counterexamples because Conway's law makes no comment on how the teams will choose to deploy what they build. It talks about the overall design of the system, the solution space that was explored. Windows is a monolithic application if the alternative is microservices, but anyone who gives it more than a cursory glance can see the org chart (both past and present!) reflected in the disjointed UX.
> And so new management had to be created, a process we still are working on as well, but many managers these days are way better than managers from 30 years ago.
I'm not sure why this makes Conway's law not applicable anymore—what you're describing is that new management does a better job of creating communication structures that lend themselves well to creating software. That may well be correct, but the resulting improvements validate Conway's law! If we're getting better at software, it may well be because people are talking about and accounting for the impact that communication structures have on the output.
It’s not a physical law - it relates to people - “law” has a wider meaning outside the field of physics.
I recommend watching the talk! He specifically addresses the very points you bring up. Of course reality is messy and people especially so, but Conway’s law is one of very few theories within CS that has held up in replication studies.
> There are plenty of counterexamples of, for instances, fine-grained teams in one company that build a monolith, and the same type of teams in another company building a microservices architecture.
Oracle certainly has. Several GBUs realized they were working towards the same goal of a microservices base layer for future service hosting (for handling auth, logging, databases, etc.) and combined their efforts, and I happened to work on one of the resulting teams.
I'm not sure how public they've been about all this, but you certainly can teach an old dog new tricks, provided there is management buy-in (which we certainly had - at one point, our teams were moved between GBUs to save us from budget cuts).
That's the thing that stumped me here: how does a _Senior Staff Engineer_ doesn't understand their job isn't to build software, but to enable the rest of the organization to build software? (Paraphrasing on that old Toyota principle)
Not just one layer, look at the stack: Type script is MS's Conways law. API/microsevice everything is more of a google thing (vs FB monolith)... So you have two other major product of Conways Law that your trying to jam into your own org.
On top of that a 5 year plan is never going to digestible by (almost) any company. How many people stay in one place for that long? How much has the front end changed in 5 years? Is any choice you make in the JS world relevant in 5 years? Though sell there too!
