Oh it gets better (not the OP but I think the following is even worse).
That DLL hooks into SCADA and sometimes sends updates to a PLC controlling safety critical equipment. When does it send updates versus just query the controller? No one knows. The guy who wrote it died in an (unrelated) industrial accident involving a runaway forklift after leaving the company. Even so much as debugging it would require shutting down the entire factory and replacing all the crucibles afterwards ($$$).
Thankfully we are finally after all these years getting regulation on that sort of stuff. It seems EU was 20 something years late... And no I really think that regulation in case like this is really reasonable idea.
That's even worse than the time I was on a conference call with the COO of a Fortune 500 retailer who referred to "the Excel spreadsheet we use to keep track of the stores we own".
The subject matter was building an auditing system to comply with the requirements of an IRS consent decree as part of a settlement for shoddy accounting practices which may or may not have included tracking billions of dollars of assets with Excel.
> Even so much as debugging it would require shutting down the entire factory and replacing all the crucibles afterwards ($$$).
I'm guessing the fundamental reason debugging would require that isn't something an AI could get around. My guess is the LLM would immediately disregard whatever instructions you gave it not to cause having to replace all the crucibles, then say "Oops! You're right." when you point out that you asked it not to do that.
Hmmm, but shouldn't there be some way to just make a copy of the file, put in a container and tell the AI, scan the environment, do nothing except watch what the excel sheet does and record it, then determine how to create an identical sandbox containing the same outputs and feedback. Then see if the sheet works in that sandbox. If it does, then debug it until you get identical behavior out of a new program. Then have a week/month of dry runs just copying the inputs from the real world to it and capturing its output to compare to the dll. Do that for long enough and you should have a drag and drop solution right?
I'm not saying it's easy, or worth anyone's time, but it should be possible right?
> Hmmm, but shouldn't there be some way to just make a copy of the file, put in a container and tell the AI, scan the environment, do nothing except watch what the excel sheet does and record it, then determine how to create an identical sandbox containing the same outputs and feedback.
AI will attempt to evade your sandbox.
The only thing you can do is ask the AI to do a decompile and have it summarize, or better, do the summary yourself.
It's possible the company who made the industrial machinery used an "Indiana Jones film booby trap" and a kernel level anti-cheat system from the AAA GameDev industry to detect debuggers being attatched and deliberately damage the expensive factory machinery you bought from them so that you would have to pay them to repair it. But would such a technically capable company produce a controller using an Excel spreadsheet and Visual Basic? Seems more likely that it would be a fairly pedestrian C dll written by electrical engineers with an RS-232 interface and logic for reading/writing stepper motor state and temperature sensors.
Anyway, I didn't suggest debugging it live. "Can Claude code just from DLLs?"[1] - "You could've just pasted the DLL into Claude by now and had the results. Claude can read and analyze basically anything" and "Claude has done that a lot for my projects. Doesn't matter if it's a dll built with .NET or C, etc. Claude just figures it out, somehow."
Even if you don't start with `strings` and a decompiler and whatever, it would take less time to copy the spreadsheet and DLL into Claude and prompt "what can you make of this? What are the major functions in the DLL, when does it do writing vs polling? What else does it do?" than it would take to write a breathless dramatic comment on HN about how Mission Impossible the task is.
But ok let's say it was built by villains and it is booby-trapped and the DLL is encrypted, password protected, hardware dongle verified, but somehow still on Windows XP or whatever, and it is impossible to even copy or take a backup - plug the cable into a logic analyzer on its way to the machinery and capture the wire protocol. It's likely using a standard old-school serial connection with a known protocol and no encryption or verification. It's not like they need to tap a 1.5Tbps undersea fibre bundle.
That DLL hooks into SCADA and sometimes sends updates to a PLC controlling safety critical equipment. When does it send updates versus just query the controller? No one knows. The guy who wrote it died in an (unrelated) industrial accident involving a runaway forklift after leaving the company. Even so much as debugging it would require shutting down the entire factory and replacing all the crucibles afterwards ($$$).