> I said "if that's relevant" and you jumped to "this guy just makes websites". I do many things.
I didn't. I made a guess. I might be wrong, that's OK. I love to be wrong, because I learn things by being wrong. Also no offense was intended, and I don't consider webdev inferior anything. What I tried to mean is, if AI has more training data for a domain, it does better. If you fire the same model on a niche domain, it falls flat.
> If you know that the "environment keeps on working", there must be some way for you to observe that fact.
Yes.
> And if it is an octopus, it is likely difficult and/or to change the conditions and observe the correctness with respect to those changes.
Nope. On the contrary, because there's so much innate knowledge that is required to know what to do, simulating in mind, deploying and testing on real world is much easier and faster than letting loose an ML model on it. You need real data, real data comes in slow, but you can catch problems early and easily.
Considering it's a niche area, AI also doesn't have much training on that domain, so it's doubly inapplicable for what we do.
> but for me it totally changes the calculation on when it makes sense to automate something ... (snipped for brevity)
It's great that if it works for you, but YMMV part is way more correct than people want to accept and want to learn. AI is a pneumatic hammer, but not everything is a nail which can be driven in with that.
When it works, it works. When it doesn't, well people still pretend it does or insists it shall. We must accept the limitations.
> deploying and testing on real world is much easier and faster than letting loose an ML model on it.
No this is what you're not getting. It is "doing the things I would do to deploy and test in the real world, but faster and in the background while I do other things", it is not "letting loose an ML model on it". This is the new capability. If you have any process like "do {action}, wait until {something}, check {something}, determine if it matches expectation", it is now possible to run that loop way more times in way more variants without either spending the time on it synchronously oneself or writing a script to do it. (If you do that specific action loop often enough, it's probably worth writing the script anyway, but that's also much quicker to do now.)
The AI doesn't need training on the domain, it just needs to be told "these are the things I would do, please do them for me and report back".
I'm sympathetic to not everything being nail-like, but I really think you're leaving a lot of chips on the table if you can't imagine any of this kind of action-check-evaluate loop you have that you could offload.
I'm saying you could do more things. Instead of waiting until a new thing is merged so that it can be deployed and waiting for monitoring to catch issues, you could be deploying pre-merge to testbeds, using different variants of the code or different configurations of the whole system or different inputs to exercise edge and corner cases. Maybe it's too complex of a system to set up that kind of testbed or simulation? Well, it's easier to do that now too! I can spin up and down environments, either with containers on my workstation, or in cloud deployments, that I would have not attempted in the past, because it would have taken me too long to set them up. But now it doesn't take that long, and I find it super valuable to be able to try more things out. (Cost is still a real constraint, I'm not saying that constraints magically no longer exist.)
Obviously I have no idea what your work looks like! But what I'm saying is that time savings are not just time savings. There can be a point at which the time savings bring you under an "activation energy" such that it unlocks a new capability, not just a speedup. And some of those unlocked capabilities can be directed toward improving the quality of software. And I think that's awesome and useful, is my prevailing point here. I won't claim that it will usher in an industry wide improvement in quality or anything, but for me personally, I'm making better software more quickly now, and I'm very pleased that I can do that.
I didn't. I made a guess. I might be wrong, that's OK. I love to be wrong, because I learn things by being wrong. Also no offense was intended, and I don't consider webdev inferior anything. What I tried to mean is, if AI has more training data for a domain, it does better. If you fire the same model on a niche domain, it falls flat.
> If you know that the "environment keeps on working", there must be some way for you to observe that fact.
Yes.
> And if it is an octopus, it is likely difficult and/or to change the conditions and observe the correctness with respect to those changes.
Nope. On the contrary, because there's so much innate knowledge that is required to know what to do, simulating in mind, deploying and testing on real world is much easier and faster than letting loose an ML model on it. You need real data, real data comes in slow, but you can catch problems early and easily.
Considering it's a niche area, AI also doesn't have much training on that domain, so it's doubly inapplicable for what we do.
> but for me it totally changes the calculation on when it makes sense to automate something ... (snipped for brevity)
It's great that if it works for you, but YMMV part is way more correct than people want to accept and want to learn. AI is a pneumatic hammer, but not everything is a nail which can be driven in with that.
When it works, it works. When it doesn't, well people still pretend it does or insists it shall. We must accept the limitations.