Tons of unstructured data that needs to be cleaned up in so many different ways that are hard to predict. Knowledge distributed in the heads of greybeards, handwritten papers und excel, sharepoint and dozens other locations like ancient proprietary software. Which you have to repeatedly ask for and discover more piece by piece. Then clean up, discover issues, make the customer gather more data/correct it, etc.
No, but if we go from data to experiment faster we see earlier if something is wrong. Once we get data we quickly can fix the data pipeline and get results. If there are issues we can quickly show what/why and discuss it with them in detail and make changes based on their response faster.
Yes, but they tell things and maybe forget to mention certain things or contradict other data they have given you. Now we may not notice it initially until we make an experiment and get the results. Something is wrong and we have to find out what. Talk to the customer again. Fix experiment. Repeat. Faster experiments means the latency from new data to results is less and we finish the project faster. Errors become less expensive and cause less delays. People make errors and data quality is often horrible so the shorter iterations are nice.
It's not all boilerplate of course. But iteration cycle is shorter and the latency matters. We are trying find a formulation, so a lot of experimenting, fine-tuning, checking errors (first via simulations, then real-world with measurements). Then we see what needs to be changed, repeat until error is in desired bounds. But the real-world is messy so we see a problem (lets say adding some certain color does not change the final properties of paint the way predicted) and we go back to the customer, some expert remembers temperature/moisture of the air was different on this day and that affects printing results. Now we need to measure this too and add it to the simulation and add it to the data pipeline. But we may have to wait until the real world has implemented that too (create a process, buy hardware, whatever) and then a while later we finally get real data and can start doing our part. Reducing our latency makes things faster, as you may do something else in between and need to load everything in you head again. The LLM gives results in sometimes minutes: frontend, backend and database. We now have the feature branch where you can upload the data like the customer would do, start the calculation, see the predicted formula. From idea to working prototype. We can focus on making assumptions and checking them quickly. Previously they can make the changes in their Python scripts but getting it in a presentable form took much more work. Now we can go to the customer and present a pretty custom made UI like an end-user would see same day they give us new data or report issues.
The stack certainly could need improvement, but much of the overhead comes from the process being pretty serial because each step depends on the previous. If someone we needs to talk to goes on vacation, sick or has a full calendar this delays it much further. Iteration speed is not linear in such cases. We do a lot of custom solutions and have to implement custom parsers or interface with undocumented ancient software or simply no existing software. Everything is custom, because previously they have no processes, no IT infrastructure or simply only data intended for humans (roughly structured, informally written, ever changing naming schema, typos).
Oh yes. Verification needs to be done. Some statistics and validation is absolutely required. Also most customers don’t want their private data uploaded anywhere, so ai gets only test data and writes the script to transform it. As long as it is just XML, CSV or similar it’s pretty reliable. Handwritten notes and bad scans however…
A friend of mine runs a business similar to this for systems migration, which is a fairly tough and tedious "short-put" job (you have to know both old and new system fairly well, and if all goes well, you get the same job done on the new system, while if something goes wrong, you're in a world of pain...):
The AI analyses sample data from old and new system, and then builds a prototype deterministic mapping pipeline. The experts can then inspect said pipeline and with their domain expertise catch edge cases/exceptions/uncovered cases, fix them and improve things, and then you use the vetted deterministic pipeline for migration. Yet, the initial AI draft saves enormous time.
> handwritten papers und excel, sharepoint and dozens other locations like ancient proprietary software
This area is one which AI has shown to be pretty good (as long as some error rate is acceptable). Character recognition, voice transcribing, converting documents from one standard to an other. I can see an argument that if people spent most time doing those things, AI will speed it up (with a bit of added errors).
I am reminded however that in translations and AI generated subtitles, there is still a lot to be desired compared to the human touch. Handwritten papers in a different language, using local short hand and assumptions, may result in some odd and error prone AI transcription.
Boilerplate becomes burdensome as complexity increases. Perhaps sometimes we don’t give ourselves enough credit for how much applications and their infrastructure are actually taking care of as part of boilerplate.
I look at stuff like terraform code or AWS infrastructure and it’s annoying and repetitive as hell sometimes but at the same time it’s incredibly impressive that you can just describe something as complex as your own private company’s network existing on global data centers and on a relative level you can just bake in pretty complex stuff like a spiderweb of IAM rules, disaster recovery/backup/databases you barely have to think about, horizontal scaling, security policy and all kinds of other stuff I’m forgetting off the top of my head.
It’s heavy boilerplate because like you said you can basically design an infinitely complex corporation as code and once you’re done with that deploy it instantly which used to be pretty much unheard of.
And yet, many companies today are spinning up full stack teams, asking them to be experts from user interviewing to database design, architecture, hosting, devops, API management, data product, and also keep an eye on GDPR compliance.
I see dozens of teams that are doing cloud to the wazoo when really they need to just be told “we’ll load your routes from main.py, build your assets in /static, you need a DB click the button to provision it”.
It boggles the mind that so many teams are reinventing the wheel a million times over to publish their APIs, register AWS accounts, setup CI, decide on ECS vs Lamda, etc.
This sort of activity has been the source of income for a very large number of people. You can't really fault them for trying to extend that gravy train's run.
Probably a controversial take - but in the pursuit of elegance. 'Uhh, so I need to hook this connector up to that receiver using this adapter and then uh, I pass it to the connector manager which sends it to the scheduler that... @#%! FML.' And then it's all so polymorphicized that going through the debugger you spend an eternity just trying to figure out what the types even mean and why/where they came from.
When you look at old-school code that just hacked things together, it was ugly, but also very easy to in turn further hack. There was no elegance or grand base types with 80 levels of inheritance to enable clean extension at every possible juncture of types. And no spending forever to find out the exact function you want to be able to override, only to discover that it's outside your modifiable codebase and wasn't marked virtual. Everything just worked.
Well, that's a funny thing. The "well engineered" code can actually be harder to work on. And maintainability is one aspect of "well engineered".
What I've seen (at least at one place I worked) is that the code may be very well engineered for further modification - but only by people who have worked quite a bit on that code. For someone coming from outside, it's almost impossible. It takes forever to find your way around, and figure out where the magic place is that you need to modify in order to work within the existing design.
Well, you say, that's what the design documents are for! But they didn't have enough detail. They described the major objects, but not all the ancillary helper objects. They didn't describe the flow of data through the system in enough detail to know what to modify. The knowledge was there, in peoples' heads, but it was hard to get those peoples' time.
So, yeah. "Well engineered", but not engineered for maintenance by people who are not those already in the group.
This reminds me of a nice quote about engineering: 'Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands.' Another is that 'If people knew how many assumptions went into designing a bridge, they’d never dare cross one.'
Engineering isn't about creating some perfect thing that is prepared for any conceivable scenario, but about solving a problem as efficiently as possible.
At what stage did the engineering fail?