Indeed, the safer thing is to actually build a few toy compilers on the side so you can get a sense for what they are good for, and what level of effort is required to build and maintain it.
Keeping them locked up in the "scary CS" closet only ends up stunting your growth.
I like to write toy compilers or interpreters as an exercise when learning a new language. Usually for a Forth or Lisp or one of Turing Tarpit languages. It requires some of the most common bits of programming: I/O, lexing, parsing (both of source and of arguments to the compiler), file handling, and some common algorithms & data structures (can't have an AST without a tree).
I'd say the opposite. Building a compiler incrementally, driven by clear needs at each step, as described in the article, tends to work out better than trying to build a compiler from day 1.
I've written 7 compilers. If you don't know you're writing a compiler from the beginning and you've never written before, it is basically guaranteed to be a hot mess in the end, unless you accidentally make exactly the right call at every one of a dozen critical steps. It's extremely unlikely to just stumble on all the good architectural ideas from a field that is 70+ years old without any planning.
Granted, if you've got a really simple language then maybe there's an upper bound on how bad it can be. But in practice people tend to also want to grow the language as they go.
Don't get me wrong; there's a lot than can wrong if you over-engineer the compiler from the beginning, including ending up with a hot mess anyway.
BTW if you can't write down, even in three sentences, the requirements for said compiler from the beginning, congratulations, you have no idea what you are doing.
Oh no. What the article describes ends up with a real mess. If you set out to support a certain language you'll have a global view of how things should fit together, and what the implementation should do.
Is there a meta-library someone somewhere has already written for when I just want to write 20% of a compiler, and it more or less takes care for me 80% of the common compiler-building-related things I’m likely to need to do?
LLVM doesn't prevent you from shooting yourself in the foot via a dozen different language details completely divorced from backend implementation. It's only a small, but very important and difficult, aspect of compiler writing. I think languages like go have demonstrated well that this is of limited benefit to the success of the language.
Racket at least provides most of the tooling required (eg dealing with details of namespacing, syntax, runtime/compile-time/evaluate-time distinction, continuations and garbage collection), but if you've ever tried to introduce it into an organization of significant size this very same result of yak-shaving will be considered a liability.
You're criticism is an apples vs. oranges strawman. LLVM IR and libraries provide powerful plumbing. It's not meant to be a turn-key or cookie-cutter solution, or inherently memory-safe VM or compiler generator. It's also not formally verified and written in C++, which adds project configuration complexity and C++ pitfalls. It's possible to plug in a custom GC using their strategy model their way, or manage GC yourself like Pony does because it's built around ORCA.
And you can step up a level above in the same ecosystem as well, my current project is written in Scala and directly shares its data structures with the Graal Python interprer that I've embedded inside it, so other people can write the stuff they want in Python.
I've found the same thing - a little syntax example, some counter examples and generative AI does well generating syntactically correct code for PCB design.
A lot of the netlists are electrically nonsense when it's doing synthesis for me. Have you found otherwise?
Netlists, footprint diagrams, constraint diagrams etc. are mostly nonsense. I’m working on finetuning Phi3 and I’m hopeful it’ll get better. I’m also working on synthesized datasets and mini-DSLs to make that tuning possible eg https://text-to-footprint.tscircuit.com
My impression is that synthetic datasets and finetuning will basically completely solve the problem, but eventually it’ll be available in general purpose models- so it’s not clear if its worth it to build a dedicated model.
Overall the article’s analysis is great. I’m very optimistic that this will be solved in the next 2 years.
I think Gemini could definitely do that microphone study. Good test case! I remember spending 8 hours on DigiKey in the bad old times, looking for an audio jack that was 0.5mm shorter.
Hah, you're not kidding. Literally my comment was inspired by a recent realization that it is not possible to search for a RF connector by footprint size.
That's absurd to me, it took so long to figure out which random sequence of letters was the smallest in overall PCB footprint.
Maybe we found it, we think it's the AYU2T-1B-GA-GA-ETY(HF) but sure would be nice if Digikey had a search by footprint dimensions.
Yet strangely the physical ability of a device to fit into a location you need it is not in the list of things I can search. Takes ten seconds to find the numbers -- after I download and open the PDF file.
Just so strange, but so common. And digikey is heads and shoulders above average, McMaster might be the only better one I know of at it and they're very curated.
As I understand it, PDF digestion/manipulation (and particularly translation) has long been a top request from businesses, based on conversations I've had with people selling the technology, so it doesn't surprise me that Gemini excels at this task.
Anyone looking for an idea for something potentially valuable to make: ingest PDF datasheets and let us search/compare etc, across them. The PDF datasheet is possibly one of the biggest and most unecessary hurdles to electronics design efficiency.
My gut agrees with you that LLMs shouldn't do this well on a specialty domain.
But I think there's also the bitter lesson to be learned here: many times people say LLMs won't do well on a task, they are often surprised either immediately or a few months later.
Overall not sure what to expect, but fine tuning experiments would be interesting regardless.
I doubt it'd work any better. Most of EE time I have spent is swearing at stuff that looked like it'd work on paper but didn't due to various nuances.
I have my own library of nuances but how would you even fine tune anything to understand the black box abstraction of an IC to work out if a nuance applies or not between it and a load or what a transmission line or edge would look like between the IC and the load?
This is where understanding trumps generative AI instantly.
Make two separate signals arrive at exactly the same time on two 50 ohm transmission lines that start and end next to each other and go around a right hand bend. At 3.8GHz.
Edit: no VSWR constraint. Can add that later :)
Edit 2: oh or design a board for a simple 100Mohm input instrumentation amplifier which knows what a guard ring is and how badly the solder mask will screw it up :)
Right - LLMs would be a bit silly for these cases. Both overkill and underkill. Current approach for length matching is throw it off to a domain specific solver. Example test-circuit: https://x.com/DuncanHaldane/status/1803210498009342191
How exact is exactly the same time? Current solver matches to under 10fs, and I think at that level you'd have to fab it to see how close you get with fiber weave skew and all that.
Do you have a test case for a schematic design task?
It would seem to me that the majority of boards would be a lot more forgiving. Are you saying you wouldn't be impressed if it could do only say 70% of board designs completely?
Not the GP, but as an EE I can tell you that the majority of boards are not forgiving. One bad connection or one wrong component often means the circuit just doesn't work. One bad footprint often means the board is worthless.
On top of that, making an AI that can regurgitate simple textbook circuits and connect them together in reasonable ways is only the first step towards a much more difficult goal. More subtle problems in electronics design are all about context-dependent interactions between systems.
I hate that this is true. I think ML itself could be applied to the problem to help you catch mistakes in realtime, like language servers in software eng.
I have experience building boards in Altium and found it rather enjoyable; my own knowledge was often a constraint as I started out, but once I got proficient it just seemed to flow out onto the canvas.
There are some design considerations that would be awesome to farm out to genai, but I think we are far from that. Like stable-diffusion is to images, the source data for text-to-PCB would need to be well-labeled in addition to being correllated with the physical PCB features themselves.
The part where I think we lose a lot of data in pursuit of something like this, is all of the research and integration work that went on behind everything that eventually got put into the schematic and then laid out on a board. I think it would be really difficult to "diffuse" a finished PCB from an RFQ-level description.
I find I spend an enormous amount of time on boring stuff like connecting VCC and ground with appropriate decoupling caps, tying output pins from one IC to the input pins on the other, creating library parts from data sheets, etc.
There's a handful of interesting problems in any good project where the abstraction breaks down and you have to prove your worth. But a ton of time gets spent on the equivalent of boilerplate code.
If I could tell an AI to generate a 100x100 prototype with such-and-such a microcontroller, this sensor and that sensor with those off-board connectors, with USB power, a regulator, a tag-connect header, a couple debug LEDs, and break out unused IO to a header...that would have huge value to my workflow, even if it gave up on anything analog or high-speed. Presumably you'd just take the first pass schematic/board file from the AI and begin work on anything with nuance.
If generative AI can do equivalent work for PCBs as it can do for text programming languages, people wouldn't use it for transmission line design. They'd use it for the equivalent of parsing some JSON or making a new class with some imports, fields, and method templates.
> But I think there's also the bitter lesson to be learned here: many times people say LLMs won't do well on a task, they are often surprised either immediately or a few months later.
Heh. This is very true. I think perhaps the thing I'm most amazed by is that simple next-token prediction seems to work unreasonably well for a great many tasks.
I just don't know how well that will scale into more complex tasks. With simple next-token prediction there is little mechanism for the model to iterate or to revise or refine as it goes.
There have been some experiments with things like speculative generation (where multiple branches are evaluated in parallel) to give a bit of a lookahead effect and help avoid the LLM locking itself into dead-ends, but they don't seem super popular overall -- people just prefer to increase the power and accuracy of the base model and keep chugging forward.
I can't help feeling like a fundamental shift something more akin to a diffusion-based approach would be helpful for such things. I just want some sort of mechanism where the model can "think" longer about harder problems. If you present a simple chess board to an LLM or a complex board to an LLM and ask it to generate the next move, it always responds in the same amount of time. That alone should tell us that LLMs are not intelligent, and they are not "thinking", and they will be insufficient for this going forward.
I believe Yann LeCun is right -- simply scaling LLMs is not going to get us to AGI. We need a fundamental structural shift to something new, but until we stop seeing such insane advancements in the quality of generation with LLMs (looking at you, Claude!!), I don't think we will move beyond. We have to get bored with LLMs first.
> If you present a simple chess board to an LLM or a complex board to an LLM and ask it to generate the next move, it always responds in the same amount of time.
Is that true, especially if you ask it to think step-by-step?
I would think the model has certain associations for simple/common board states and different ones for complex/uncommon states, and when you ask it to think step-by-step it will explain the associations with a particular state. That "chattiness" may lead it to using more computation for complex boards.
> > If you present a simple chess board to an LLM or a complex board to an LLM and ask it to generate the next move, it always responds in the same amount of time.
> Is that true, especially if you ask it to think step-by-step?
That's fair -- there's a lot of room to grow in this area.
If the LLM has been trained to operate with running internal-monologue, then I believe they will operate better. I think this definitely needs to be explored more -- from what little I understand of this research, the results are sporadically promising, but getting something like ReAct (or other, similar structures) to work consistently is something I don't think I've seen yet.
Some research to the contrary [1] - tldr is that they didn't find evidence that generative models really do zero shot well at all yet, if you show it something it literally hasn't seen before, it isn't "generally intelligent" enough to do it well. This isn't an issue for a lot of use-cases, but does seem to add some weight to the "giga-scale memorization" hypothesis.
Update: Sonnet 3.5 is better than any other model for the circuit design and part finding tasks. Going to iterate a bit on the prompts to see how much I can push the new model on performance.
Figures that any article written on LLM limits is immediately out of date. I'll write an update piece to summarize new findings.
I'll note that the bandwidth doubled from PCIe 4.0 to PCIe 5.0 (it's now 30GHz).
Dealing with that speed increase is causing a whole lot of problems for all the designers in the ecosystem. It's a bit of a nightmare, but people are for sure using it (in your laptops, in the data center).
We ended up adding some specific features/strategies to the autorouter we're building for to handle those speed. It was pretty hard to get right.
Without knowing exactly what you are working on, here are a few ideas:
1. Figure out if you can do fewer slow iterations. What's driving the need for a full PCBA run for each iteration? Might be able to split out to rigid assembly and flex for example.
2. Run more experiments in parallel. If you have multiple ideas or variations to test, design them all and fabricate them all. Flex antenna? Make like 30 parametric variations.
If design then becomes the bottleneck, then automate that next.
I would like to give this response a signal boost, based on the fact that the comment author knows a lot about PCB design, prototyping and manufacturing.
Gotcha - thanks for the advice. The IC performance on the flex circuit is a key part of the product so will figure out how to test that in a lofi manner.
Maybe you only need a small bit on the flex (main CPU and the DDR to prove signal integrity?), and the rest can just go on an ugly quick PCB for software validation and bringup. Stuff it with test points, current shunt points, larger parts for easy probing, add some mounting holes too fasten it to a test rig, whatever.
Then just glomp the flex down to the test PCB with some nice Samtec connectors (you can enough free samples for a small batch of test units even) or an edge connector on the stiffener.
reply