I don't buy the "any constraints cause lower performance via being out of distribution" idea. Sure if you ask the model to output 'reasoning' in JSON steps, that is a completely different 'channel' to its trained 'reasoning' output. For real tasks though, I think it's more about picking the _right_ context free grammar to enforce format correctness. You can enforce an in-distribution format and get the best of both worlds. I don't think the industry should settle so hard on JSON-for-everything.
I was working on a speculative decoding optimisation and its accompanying blog post. Explaining the more basic concepts filled so much of the post I decided to pull them out, forming this article.
I had a bit too much fun with the tokenisation diagrams / animations. The raw text is provided to an Astro component, which tokenises it, and forms the individual DOM elements of the tokens. I find it really hard to read 'tokenised' text, I figured some consistent colouring would help. The 'Probabilities' component is a trivial grid, but all the other components support 'word wrap'.
I ended up writing a 'responsive design aware graph colouring solver'.
Multiple screen widths, 'desktop' and 'mobile' are 'simulated', forming an adjacency graph of tokens that touch. Colours are then greedily allocated, then optimised per page over a few hundred iterations, swapping allocations to enforce minimum hue distance between touching tokens at those common screen sizes. The optimising value function prioritises even distribution of colours, because it looks nicer than maximal hue difference.
Originally I naively outputted the palette styles per component, but found the css post processing optimisers didn't handle that as well as I'd have thought. So then I wrote a little 'CSS compiler' that takes the high level palette and timing concepts of the animations, and optimally merges rule declarations.
The start of the post really relies on the animation occurring while fully in view, so I set up some IntersectionObservers that do the 'please scroll' text.
I tried my best to have it all work when JS is disabled on the client. I tried to get the 'hovering' to be CSS-only, but found the JS solution much more performant.
The DAG diagrams are formed with this neat Needleman-Wunsch algorithm from the bioinformatics field. The Astro component accepts several 'examples' then aligns common subsequences, producing the CSS grid and the 'basic SVG' on the server. The responsive nature meant I had to move the final 'allow' generation to the client.
Some browsers seem to throttle the token animations sometimes but I haven't figured out what causes that. This is my first time leaning hard on CSS variables.
In South Australia an algal bloom started in ~mid-March of this year, it's a pretty big ecological disaster, probably the worst non-bushfire disaster in living memory. Probably 30% of SA's coastline is affected. It's a pretty big deal affecting many people's livelihoods.
The joint state and federal government relief and cleanup package is worth AUD $102.5 million dollars.
I hope the public receives that comparison at every opportunity.
The old website was frankly excellent, the only problem was it didn't have HTTPS support. I would have happily upgraded that part of the system for the cost of a cup of coffee if I'd had an opportunity to submit for the tender!
The new website is significantly more difficult to navigate (for me, a seasoned tech user). The primary thing Dad's everywhere use it for (the weather radar) now requires scrolling to the _bottom_ of the page, and zooming in from the 'map of Australia' to the region you live in. It used to be like, a click to go from home page -> state weather radar with all the info you needed.
> [BOM] said the cost breakdown included $4.1 million for the redesign, $79.8 million for the website build, and the site's launch and security testing cost $12.6 million.
Absolutely stupid, even those numbers are outrageous. They say it's part of some 'larger upgrade package', prompted by a cyber attack in 2015.
But politicians over here love to blame cyber attacks when technical blunders happen. We had a census a couple years ago and the website fell over due to 'unprecedented load' or maybe it was a 'DDOS attack'? The news at the time couldn't decide who to blame!
Welp, I hope this gets as much world-wide attention as possible so they can be embarrassed and do better.
The painpoint for me has been the loss of information density. 99% of my use of the old BoM was the 7 day forecast showing rain and cloud: former for working outside, latter for photography jobs. Now, at about 800px or narrower the 7 day forecast loses the rain estimate, and all they manage to fit in is day, icon, min and max. The day name could be abbreviated, and the other elements are typically 30px wide. Having to expand each or all days to look for the rain estimate is thoroughly tedious.
Among the highlights of vertical space wastage are 130px for a cookie warning, 50px for "No warnings for this location" and then another 110px for heading a table with "7 day forecast" and "expand all". On a large phone screen, it leaves only about a third of the vertical spacing for starting content; the rest is site header and browser chrome!
I don't understand how those kinds of numbers get accepted, approved and paid! We built a fairly complex web application for a customer. The total cost including design, development, QA, data migration from a legacy platform + independent 3rd party security audit/pentest was less than $0.5M!
Even if I accounted for the additional capacity to serve a nation of users, I can't imagine the cost being more than $5M.
I would have settled for https to redirect to http. Instead, it redirected to a generic page telling you they don't support https, with no way to get to the actual content.
In some ways, poor project management is like an algal bloom or wildfire: costs expand, feeding on other costs, unless a huge active effort to keep them under control is made.
It would be nice if users of the codex-cli that are just using API keys as a way to handle rate limits and billing could receive these new models at the same time. I appreciate the reasoning behind delayed 'actual API' release, but I've found the rate limiting to be quite annoying, and my own API keys don't have this limitation.
Re: rate limits, I'm not sure they can, yet, on capacity. See Jensen's comment today about their cloud GPUs being sold out. So capacity increased await the ongoing data center build out.
Apart from the actual model release, this is the second set of models from OpenAI that uses the Harmony response format. I don't suppose anyone knows if OpenAI uses the Harmony format internally for GPT-5 as well?
Aren't these just IPCs disguised as normal function calls though? IIRC only the main node process does anything node, renderers can call "node functions" that really happen in the main process.
Not at all, in a renderer the Node and Chromium event loops are bound together, they’re part of the same v8 isolate, no IPC shenanigans.
The main process really shouldn’t be used for anything except setup. Since it controls gpu paints amongst other things, blocking on it will cause visible stuttering and a bad user experience.
Separately the IPC lets you do zero copy in some circumstances via Transferable objects such as ArrayBuffers. Structured cloning is efficient but not zero copy, and json serialisation shouldn’t be used (since structured cloning is easily available).
Thanks for adding this context! Guess I was mislead by the Electron documentation talking about multiple processes and IPC, appreciate the clarification!
Within a renderer you can access NodeJS APIs directly. The main process shouldn’t be used for any significant computation, as it will block GPU paints and cross-process synchronisation.
The other main difference is Electron bundles a known set of APIs, given the known Chromium version. There’s such a huge variance of supported features across the embedded web views.
Yes, this is the best benefit of elecrron: you dont have to trouble shoot 10s of OS webview versions and their ixremental suppory, especially with MacOS.
But it is right that the ui for elwctron has to use a IPC layer to get a node backend running. However, chrome is moving a lot of things like FilesystemAPI into browsers so there may be a day were nodejs is dropped in favor of a sandboxed chromium.
You don’t need IPC, you can either use a preload script to expose particular Node APIs in a secure manner or set ‘nodeIntegration‘ to ‘true’ to expose everything.
Conversely, the last blog post we wrote was 8,000+ words and took months of testing, yet the average 'read' time is under 2 minutes. I'm convinced there's a correlation between interested technical users and the blocking of analytics scripts - but if I were to naively look at the data, I'd also come to the conclusion that "lower effort" was better return on investment. I wonder if these tech journalism establishments are following their analytics and A/B testing themselves into oblivion.
It's like meat and potatoes, though. Yes you can fill a website up with low effort filler content that keeps your viewers engaged and visiting, but in the long run you also need some solid meaty stuff.
A lot of that sorta stuff moved over to youtube because it was easier to monetize. I think a hybrid of the two is the nicest (reading charts from youtube videos sucks)
It's a weird trap. With no analytics, it'd be difficult to attribute any conversions to a particular user type, so I'd wager that, if the hypothesis that lower tech users don't block ads/analytics holds up, the metrics skew that way. We can't make any realistic assertions without the data for that user group. Shrug.
