The prompt the user enters is actually not the prompt. Most agents will have an additional background step to use the user's prompt to generate the actual, detailed instructions, which is then used as the actual prompt for code generation. That's how the ability to build a website from "create a website that looks like twitter" is achieved.
This is a very cool UI. Unfortunately I'm getting a lot of static and crackling during playback. Are you using WebAudio? I had similar issues before with https://dj.t-tunes.com/ but was able to mostly fix it, as long as gain is not manipulated.
I agree that assisted engineering means that software development goes from artisanal work modes to industrialized modes of work. Automotive engineers don't go to the factory and build the cars after they've designed them, but software engineers have been doing so. However, I have to say that the post doesn't really explain how you are industrialising code, and I don't really know what differentiates your product.
What we want as developers: To be able to implement functionality that utilizes a model for tasks like OCR, visual input and analysis, search or re-ranking etc, without having to implement an LLM API and pay for it. Instead we'd like to offer the functionality to users, possibly at no cost, and use their edge computing capacity to achieve it, by calling local protocols and models.
What we want as users: To have advanced functionality without having to pay for a model or API and having to auth it with every app we're using. We also want to keep data on our devices.
What trainers of small models want: A way for users to get their models on their devices, and potentially pay for advanced, specialized and highly performant on-device models, instead of APIs.
What seems to be delivered by NPUs at this point: filtering background noise from our microphone and blurring our camera using a watt or two less than before.
If you do video calls for 7 hours a day and then ran out it means you could have maybe ~7.5 hours. Not nothing, but differences in things like screen backlight and other component efficiency still dominate battery interests over whether there is an NPU or not. If you don't spend your day on video calls it's more like a 0% increase (mic noise processing is much lower load).
Regardless if it does zilch or some minor good for you in the battery respect, the point was more NPUs don't deliver on the above reasons everyone was supposed to want AI for. Most likely, IMO, because they are far too weak to do so and making them powerful takes too much power+cost.
I believe iOS and macOS make good use for NPUs, actually, with automatic OCRing on any photo displayed, be it in browser or Photos app. You can select any text easily and quickly.
You cannot buy these. You can only buy companies listed in Shanghai and Shenzhen that are enrolled in Stock Connect, which has certain requirements that means these small IPOd companies cannot be enrolled. It is possible that there mutual funds or ETFs in HK that are available, and hold these types of companies.
Probably the opposite. It might save Starbucks in China. McDonald's China was once sold off to Chinese operators, which expanded the chain significantly, and then sold it back. The problem is probably that Starbucks HQ does not understand the China market, yet wants to control China strategy. Having outside operators implement their strategy is probably good learning for Starbucks.
Since I'm in China, I took a look at Taobao hoping to find cheaper ddr5 sticks made with CXMT chips. Unfortunately, they are the same price or only marginally cheaper than Hynix sticks (1900 - 2300 rmb for 32g ddr5). Perhaps capacity at CXMT is not very high yet.
They failed to achieve quality and decent yield. And that is at least 10 years from DDR3 to DDR5. On the other hand YMTC from NAND space are moving to DRAM.
Or maybe CXMT is not dumb? If the worldwide price of some commodity is $100 and you can produce an item that is nearly as good as everyone else’s, would you sell it for $98 or for $50?
Or the supply of non-CXMT DRAM is sufficient poor right now that Chinese buyers, of which there are plenty, are willing to pay approximately as much for CXMT’s product as for anyone else’s?
CXMT’s real ability to reduce prices will come (if it does) by adding enough supply to drive down prices everywhere, or at least everywhere that is willing to import CXMT’s products without absurd tariffs.
If they have the runway, they can try the tried and true method of undercutting competitors until they fold, and then capture the market for themselves.
Investors have the stomach for this tactic, surely a company with the state's backing can remain solvent even longer than those funded by private investors.
I think the economics are wrong for this, at least so far. CXMT seems to be scaling up as fast as they can, and they have maybe 5% market share. They can’t flood the market with cheap DRAM because they don’t have enough DRAM yet — I’m not convinced they could materially impact anyone else’s profits even if they literally gave away their entire output. At best selling their product cheaply might help them gain some mindshare as integrators test it and hopefully discover that it meets the spec and works fine. What they need is capital and time with which to scale, and selling at a profit-maximizing price will get them the most capital, and this benefit may well exceed the actual profit on goods sold: the stock market might appreciate that they are profitable, thus making it easier for them to obtain funding under favorable terms.
If they actually want to try to destroy their competitors by undercutting them, they need to be able supply enough DRAM to actually drive down the price. At the rate that the big buyers are buying DRAM, that will take several more years at least.
I’m curious whether their presumed inability to buy ASML machines might actually help them. If they can meet the target DDR5 and HBM specs without EUV while maintaining decent yield and acceptable power consumption of the finished product, they may well be able to out-scale their competitors. I imagine it’s a lot easier to procure the equipment for additional DUV lines than EUV lines, especially with other Chinese vendors doing their best to supply semiconductor manufacturing equipment.
Undercutting the competitors is done when a business is working close to cost and a slight anomaly (eg. competitor that doesn't have to make profit, is subsidized or is cheaper for some artificial reason) makes them go out of business.
A stick of ram, made for $10, that used to cost $20 is now $100 due to the AI bubble (numbers pulled out of my ass)... you'd have to bring the prices down to $15 or even less to make those companies fold, but you can earn much more if you sell your ram at $80.
Research and development in conventional computing are already suffering. Investment in efficient CPUs, affordable networking equipment, edge computing, and quantum-adjacent technologies has slowed as capital and talent are pulled into AI accelerators. This is precisely backward. Narrow AI — focused on real-world tasks like logistics, agriculture, port management, and manufacturing — is where genuine productivity gains lie. China understands this and is investing accordingly.
Mini PCs seems to be the perfect vector, since the only serious manufacturers are Chinese brands. International brands only seem to dabble in this sector.
If (SO)DIMM memory prices rise to the stratosphere while integrated memory in Chinese mini PCs remain relatively affordable, from 16GB Intel N150 to 128GB AMD Strix Halo for Edge AI, there will be industry-altering consequences that persist long after DRAM pseudo-shortages end. Oligopolists can relearn old lessons, https://en.wikipedia.org/wiki/The_Innovator%27s_Dilemma
New companies that serve low-value customers with poorly developed technology can improve that technology incrementally until it is good enough to quickly take market share from established business.
Indeed. I am waiting to buy a Strix Halo on Taobao when supply has been replenished. I asked the manufacturer if price would remain the same after restocking, they said they weren't sure. They re-stock in the middle of January, so it will be interesting to see what the price ends up being. I wonder if there is any advantage of having integrated memory in this regard?
Another opportunity is low-latency storage with millions of IOPS. Nvidia is rebooting Intel's cancelled (see eBay) Optane for "AI Storage". Future Mini PCs and LLM accelerators for Narrow/Edge AI industrial use could benefit from high-IOPS storage, https://www.tomshardware.com/tech-industry/nvidia-and-kioxia...
I know there's more variety and typically lower prices (and sketchier windows licenses) from the Chinese brands, but last I looked, there's lots of options if you want an established brand. You can even get them from the likes of HP, Dell, Lenovo (still chinese, but), etc.
I think these have pretty high prices (as I acknowledged in my first message), but they exist. And I see similar things all over the place in retail and medical settings.
1 litre micro PCs are about 4X (35W+) TDP and price of Intel N150 with 6W TDP. They do offer more performance and features (e.g. security) than N150, but exist in a separate and stable (15+ year) category of business desktop.
Mini-PCs have a range of experimental form factors, including mini-desktop, NAS, router, tablet, SBCs, industrial with GPIO, thanks to competition between Chinese and Korean (e.g. ODROID) OEMs.
HP has a $220 4GB Windows laptop and $400+ 8GB Chromebook based on Intel N150.
Why would I buy CXMT if it's virtually the same price as Hynix?
Edit: I see you made an edit without noting that you did so.
Only this sentence was in the pre-edit comment: "Or maybe CXMT is not dumb? If the worldwide price of some commodity is $100 and you can produce an item that is nearly as good as everyone else’s, would you sell it for $98 or for $50?"
For your added comments: There is no supply issue, there are hundreds of sellers of Hynix, Crucial, Samsung ddr5 on Taobao. For your second comment about them driving down prices later on by increasing supply, well, I first of all noted that perhaps capacity is not great in my comment, and you are also contradicting your first point where you said CXMT would be stupid to compete on price.
In reality, the history of every Chinese market entrant, even those leading technologically, shows that they always cut margins to compete by driving down costs. I have a decade worth of experience related to this.
I believe CXMT simply has not gotten to the stage where they really try to win market share and crank up supply - perhaps due to lack of capital, or as I said, due to capacity limitations (which may be a result of capital).
I'm watching the price refresh since last year, hoping the domestic manufacturers in China could rebalance the price a bit. Nope, it turned out that did not happen. It's a free money grab right now of course they raised their prices.
But I guess Americans will not be buying Chinese RAMs/SSDs for critical applications (such as AI), and not a lot of user outside China will use Chinese AI due to restrictions such as censorship etc being enforced on it, limiting the growth and scale of the tech. So, there is still hope that some of the manufacturers will eventually pivot towards tried and true consumer products.
Like Nanya in Taiwan, DDR5 production at CXMT is in its early stages and a small fraction of their overall shipments. Give it a year and that will likely change.
reply