Yep, typechecking performance will be drastically improved. Using the trace generator [0], I discovered tsc spent more than three seconds [1] to locate and parse all source files googleapis pulls in during type checking. Replacing it with the individual packages almost completely eliminated that overhead.
This is a neat summary of how they shaved some source and developer time!
A trick I've used to shave bundle sizes: re-mapping `babel-runtime/*` to `@babe/runtime` and proper core-js imports and `core-js` imports from V2 to V3 latest imports ones. This shaved tons off of my bundles (unfortunately have some libraries we rely on that are both substantial but old)
Another one is library deduping. I've re-mapped all of the `lodash.*` to be direct default imports, e.g. `lodash.merge` to `lodash/merge`. Also shaved a ton off my bundle sizes.
Tests can cover this really well. I also find that lodash and Babel runtime packages were extremely stable APIs. If an API changes a lot it’s not the best strategy. You have to know your dependency chain
You will rarely, if ever, deploy an application running CDK, so the bundle size is more or less irrelevant. It’s basically a glorified YAML compiler used to bundle infrastructure.
I think my confusion was the use of MB. If the headline had been "I shaved 200ms from my build" it would be very clear it was about the build process. But when I see MB I immediately think about output.
That interpretation doesn’t even make sense in TypeScript though, does it? You don’t build a “TypeScript app bundle” and send it off to be installed on your TypeScript executor. The only interpretations I can think of for “TypeScript build” would be 1) the typechecking process, which just outputs a list of type errors and warnings or 2) the transpilation process which strips out all the TypeScript syntax from your code leaving you with valid JavaScript.
This was my question as well. The article does answer the question, but off the bat I'd assumed the author was talking about output/dist. The web treemap cli is a great tip. If you are using webpack, webpack-bundle-analyzer is a helpful tool for quickly finding bloated packages. It's definitely helped me cut down my build times: https://github.com/webpack-contrib/webpack-bundle-analyzer
The googleapis package is absolutely ridiculously large (and the design is such that you can't import only part of it which is how it should work). I was able to remove it by depending on `google-auth-library` (an official package that googleapis uses under the hood) instead, but YMMV.
After years of testing different JS frameworks and build systems, I became convinced that heavy Javascript frontends are generally a bad model for the web.
One thing is to generate a build and ship it once over the wire in the form of an Installer or bytecode executable. Another thing is to ship the entire package and/or parts of it with every launch event, plus the complexity of shipping transpiled code to different interpreters (different browser in this case) which adds more complexity to the model until it finally explodes.
The server side with HTML generated at the backend is a more predictable, faster and simpler approach. Bandwidth is not the issue anymore, but latency. Heavy API consumption from the client, leaves data exposed and increases the latter.
Hotwire/Turboframes/StimulusJS removed the need of generating HTML code at the client, leaving a single source of truth while still having a dynamic/friendly frontend. I consider it to be a better model for the web.
Plus, new Page Transition API and Navigation API are possibly game changers for a lot of use cases out here.
What I realized a while back is web development has split into two "routes": building websites and building webapps and I think part of the problem is those two often get conflated.
With a website, the idea is a visitor might only visit a single page. Under this model you want to ship as little JS as possible, and ensure that if the user just tries to read a blog post, the entire site isn't getting downloaded.
Contrast that with a webapp. I'm currently building a real estate investment calculator/dashboard and the mental model is diametrically different than building a website. Unlike a site, I want the entire app to be loaded upfront because the user is using the "entire" app, not just viewing a page.
The problem I see is when one is trying to design a "site" but uses the "app" model, or vice versa. Attempt to use an SSR framework like NextJS for an "app" and you are going to have a bad time, likewise try setting up a site like you would a webapp and you are going to have some pretty serious limitations.
I don't think this model actually works in practice, because mobile traffic vastly eclipses web traffic for almost every kind of web application.
Tricks that work on desktop - where a web application might stay open in a browser tab for weeks at a time - don't apply on mobile. On mobile, I'm much more likely to drop into your app for a few minutes to achieve a single goal, then navigate away and un-load it again.
So assuming that I'm going to pay that loading price once and then keep interacting with your application doesn't actually work - most of my interactions with your app will be a cold start, often with an empty cache too since browser caches on mobile devices (especially cheaper Android phones) are much smaller.
There's an entire class of professionals out there these days who don't use laptops. It's possible to run quite a sophisticated business entirely from a phone these days, and a lot of people who are doing exactly that.
Sure, there are professions for which this doesn't work - but even as a software engineer I find myself increasingly frustrated when I can't do things like review a simple pull request on my phone if I'm away from my desk rather than pulling out my laptop.
> In particular I think most applications for professionals assume you are on a normal computer, not a phone. Consumers, OK.
In 2012 sure, but with so much professional work being done asynchronously and mobile phones/browsers being so capable now mobile availability is table stakes.
A few features may be disabled, but if I log in and see a messed up layout or a "please log in from your desktop" screen it's a big red flag for quality/execution shortcomings that will bite me on desktop soon enough.
Many corporations make it prohibitively difficult to get a phone or iPad on the VPN to access on prem services or federated authentication to external services.
So I only have to build for laptop sizes and up. For what I do, anyway.
I would love a [control]+click combo on opening a link from my desktop into the mobile viewer of my choice...
I should be able to go to HN and "View as safari on ios 15. | View as Chrome/brave/ as Device Type [X]" and assign it to ctrl+middle+shift+click... or some such...
I should, be able to see the view, on my machine between the three: Desktop, iOS, Android for the same URL view and see the interactions on desktop....
Is there a reason why one cant just cycle through these views on a single tab of a single browser?
Click the button and it just cycles through the view
The chrome inspector makes it easy to switch between device sizes/user agents.
Rendering with a specific OS/browser combo is not something you can do accurately without actually running that combo. There are services like Browserstack that will do this for you, but I don’t think it’s something that could realistically be built into a browser.
This is it. Another thing with a webapp that someone might use regularly is that the JavaScript bundle that's sent down is cached and you likely won't have that same load time every single time.
However, if you have a landing page that someone might only visit once, then you want to optimize for as little JavaScript as possible, and in that case, just a pure server-side render might be best for that job.
Your example describe two different extremes, in that case is easy to chose, most of us have to work somewhere in the middle and that's were it gets hairy.
Even in that model, it's the developer deciding what is an app and what is a site.
There's no inherent need for a real estate investment dashboard to act like an app instead of a plain old website. It might serve many of the developers needs, and maybe some of the customers, too - but such a feature could just as easily be served as a plain old webpage as it could a heavy front loaded application.
> There's no inherent need for a real estate investment dashboard to act like an app instead of a plain old website.
How so? This assertion might be conflating your idea of what an investment dashboard should be like and how it should behave with what some team or company has concluded a customer wants. It is inherently a product decision. Actually, claiming server side rendered app suffices for any real estate investment dashboard sounds like the ”developer deciding” angle.
Unless you truly are claiming that there never is need for anything except server side rendered payloads in the internet. In which case it’s hard to explain why companies don’t do that. It’s almost as if there are product oriented reasons to have webapps. I think the original comment’s logic stands correct
The distinction between app and site is one that is done by the business for the business. Customers don't ask for web apps instead of web sites. You can absolutely serve a real estate investment dashboard as a webpage that serves customer needs, depending on what those needs are.
Maybe customers truly are served better by an app experience based on their needs, but all the times I've been involved in that discussion the impetus for moving to an SPA has more to do with internal business needs such cleaner division of code, improved click/ad tracking, third party javascript integrations, and often much more hand-wavey ones that never seem to play out as expected like the myth that SPA's are inherently faster than page loads.
This post isn't about a Javascript frontend, though. The author works very clearly on a Node.js server, and nothing from that gets shipped to a client/browser.
The more I work with advanced JS frameworks, the more I think I could also have done it with just vanilla javascript (with some jquery). Webapps I made 10 years ago where no less responsive than the webapps I make now. Nor do they take more time to create. The only difference is the size of the applications. They significantly increase with all overhead of frameworks. I wouldn't be surprised when webapp development will return partly to what it was a decade ago, where small (stand-alone) libraries are preferred to these huge, batteries-included, frameworks.
edit: With some exceptions. Some webapps are huge. They are near impossible to program without a solid framework. But this is no more than 5% of all the webapps I create.
React and react-dom are < 100kb compressed, which is pretty much how big jquery is. Preact is even smaller. Which framework are you talking about that’s huge and batteries-included? if you want to drop react in a script tag and write JS that’s compatible with all browsers, you can do that too.
If you are running a client side app generally you’ll need many other libraries as well. Just to name a few: GraphQL, i18n, Google Firebase, Date parser/formatter, Numbers, etc.
What libraries does this require? I'm just issuing fetch requests and getting back JSON. Apollo is absolutely NOT EVEN CLOSE TO required to do GQL...
Honestly, both GraphQL and everything else listed seems pretty orthogonal to the original discussion of frameworks... Whether you used VanillaJS or React or a server-rendering platform, your choices of internationalization & datastores & auth & functionality require code...
Ah, I see. I don't see why you need tons of libraries, or any libraries at all, for that though. (And you don't even need to render any intermediary HTML if you use the DOM API.)
I don't really follow your overall point. The discussion was around how "frameworks are huge", and now you're saying... You need other libraries than the framework for HTML rendering?
> There are several GraphQL libraries
And some of them are tiny, which is antithetical to your previous point...
I agree. Every time I try out a framework, it always turns out to be easier to create vanilla Js for the few functions that I actually need. The one exception being JQuery. That was useful.
I feel comments like this must come from an individual who should actually be using SSR (Server Side Rendering) and just spitting out HTML. There is no world that composition of components - the #1 reason to use frameworks - can be solved by a "few functions", and anyone who is building serious UIs will tell you you're pretty boned without composition.
My UI is almost certainly more “serious” than yours. But it was started in 2004, long before frameworks were invented….. Just saying vanilla is not nearly as difficult as people think.
> After years of testing different JS frameworks and build systems, I became convinced that heavy Javascript frontends are generally a bad model for the web
Funny, after only 5 minutes on any website i come to the same conclusion as a user :P
It should all be down the applications need for client side interactions. Even with hotwire/turboframes, phoenix liveview or other websocket / real time server driven update patterns, it can be suboptimal for many catagories of applications as these rely on (albeit smart) whole replacement of DOM nodes. In heavy applications, or applications that have a lot of represented state, its not the panacea it might seem at first glance.
I always tell people, "mileage may vary, there's no silver bullets in this industry". I think it holds true especially when talking about stuff like this.
One thing is to have JS for client-side interactivity, and another thing is to generate the entire HTML using javascript by sending both the data and javascript bundles to the client.
Incremental rehydration techniques make this much much less of an issue (React 18 supports this now, IIRC, for instance). The tl;dr is that if you employ SSG or SSR you won't endup up with immediate re-hydrating once the client loads up, but only when something triggers the need for re-hydration, like a button click.
That to me seems like the sweet spot for highly interactive applications that are very stateful, however like I said previously, your mileage may vary.
> Bandwidth is not the issue anymore, but latency.
Bandwidth is also the issue, but most web devs or website owners don't seem to realise or care that most of the world is probably struggling with their page weight.
Internet speeds are more disparate now than in the 90s, but this issue tends to be made opaque by available bandwidth statistics... There are no distribution statistics available at all. Instead bandwidth stats are always aggregated as a regional average, and worse those samples tend to be from voluntary subset (at least in my country). Even when the median is used, it conceals a significant bandwidth divide due to communication technology gaps.
The bandwidth distribution could be roughly inferred more accurately by grouping the averages by communication technology rather than region - this way a large number of very poor bandwidth users aren't concealed by a handful of super fast gigabit fiber connections or a marginal majority of fiber connections. There's a huge number of households still stuck with ADSL, 100% fiber is a very long way off (will probably never have full coverage, expecting LTE to fill this gap) and there is a huge gap between these technologies. To compound this issue households have to share connections between multiple people and increasingly hungry devices that don't respect users data usage (looking at Apple's massive inefficient update images in particular).
Streaming section by section of a frontend sounds like a troubleshooting nightmare. Users expect an interactive experience with a website. That means JavaScript. Anything else is going to be more complex.
"users expect an interactive experience with a website" - I think this is a generalization that is getting a lot of web developers into needlessly complex toolchains and frameworks.
When I'm in a browser, 99% of the time I expect the page to have what I'm looking for. I rarely care if its interactive. In fact, the more interactive it is, the less enjoyable the experience is. This holds even firmer when I'm on a desktop.
I tend to agree. It’s frustrating whenever whatever nugget of information I’m seeking is buried under multiple clicks (navigation, collapsible sections, modals, popovers, etc) and even worse when one or more of those clicks results in a loading spinner that takes longer to play its fade in transition than it would’ve taken to load an entire static page.
No, regular users definitely prefer web pages over interactive websites.
With webpages, any delays make intuitive sense and are accepted by the user. With an interactive web app, the web app gets blamed for any delays or latency because there is no full-page refresh.
if you think React is big, you'll be sad by the 2 MB mandatory runtime you have to download just to run your Blazor WASM code, which can add megabytes on top.
That said, if your application is actually complex enough to warrant this upfront download cost, might make sense.
YMMV as always
EDIT: in case it wasn't obvious, I was talking about Blazor WASM not Blazor server
This is why I am convinced part of the reason Google and Facebook create JS frameworks is so they can move computation out their data centers and onto your computer.
This whole thread is full of people saying things like, "When _I'm_ building a webapp," and "_My_ projects end up loading slowly," etc. Google is saying things like, "When 400 people are working on a web app, how can we avoid it turning into a shit show."
I think that is in the case of Facebook they were trying to solve their own use case, and in the case of Google, they saw it as a way of strengthening the strategic position of the web in general vs apps.
Shifting to a combination of rolling our own and using headless component libraries, which focus strictly on functionality and let you handle all styling. So much of the original bloat comes from the libraries' built-in styles that you then have to override creating a mess of styles; this is totally avoided with headless libraries.
That matches my experience working with component libraries. Great to throw something together that looks like Bootstrap or MaterialUI, horrible when you have a designer on staff who wants the site to look like something :-)
How've you found the experience so far? IME, making good components that handle everything you'd expect (keyboard shortcuts, a11y, responsiveness, various CSS contexts) is more complicated than you'd expect, so the codebase gets filled with half-assed and duplicated components that don't work that well. Do headless components (never used them) help that significantly?
Yup, headless components/libraries are perfect for keyboard, a11y, focus handling, state handling, etc. Usually they don't render anything at all, they just give you html attributes, event handlers, etc that you apply to your own markup. This way you can let these battle-tested libraries handle the hard stuff and you can focus on what's actually unique to your project, usually just styling.
Not to shamelessly plug, but if you check what I’m working on in my profile it should be relevant. Mostly headless, dramatically more modern, and focused on performance.
There is a new “official” one in the works here [1] that uses lots of the “latest and greatest stuff” and should be very fast when it’s finished (later this year). As far as I know it’s set to become the new default company wide implementation of Material on web.
I think when javascript is used to build html styles you end up with lots of files full of strings and boilerplate. I noticed a similar thing when I was using pixi.js for a small 3d scene. The build output was dominated by massive strings containing all of the shader code. I think once you decide on a style for your project its a good time to fork these libraries and tear out everything you aren't using.
There's a lot of "gotchas" when it comes to tree shaking in the JS ecosystem (not using the correct export style, having to do commonjs vs esm, what browser target you want like es3 vs es5 vs es2018 vs esnext), it also depends on the libraries you are pulling and if they're following practices to minimize outputs as well.
This is a decent article that discusses the basic strategies:
But basically tree shaking and dead code elimination can only be effective as the code you write. If you make it hard to parse with ASTs, it's going to hard to tree shake.
This is the size of the type declarations files (.d.ts files) that `tsc` pulls in when compiling your code. This is distinct from bundle size.
Tree shaking is a big win for bundle size. One of my big takeaways from looking at these visualizations is that we badly need an equivalent for type declaration files.
In projects where the node_modules folder starts to get out of hand, I set `"skipLibCheck": true,` in the compilerOptions of my tsconfig file (at least for dev builds).
In my understanding, it skips type-checking any d.ts files (which most packages include by now) which dramatically lowers compile time. In the authors case it looks like their project is about 400kb of their own code and then 30mb of node_modules libraries.
I think it also skips your own d.ts files (it isn't limited to node_modules), so this might just be a way to dramatically speed up your footgun. I have literally never written my own d.ts file though so it works for my purposes
My understanding is that skipLibCheck skips _type checking_ .d.ts files, i.e. looking for type errors in them. But tsc still has to read those files since you might refer to their types in your own code, which TypeScript will type check.
Incidentally, I also ran into this exact problem with the same package. I briefly thought, "they should let us download individual packages", but since I was already about to bundle everything with esbuild, I didn't look into it.
So that's what I ended up doing: using esbuild. It should only be including what is in use, and the result was fine, even though this service was the largest of the services. But I'm still going to install the separate package and see how much that brings the bundle size down.
I'm curious about the results, but remember that this article is talking about the type declarations files that tsc has to read (.d.ts files), not the JS that winds up in your bundle.
This is a wildly incorrect approach. Ideally, they would separate the core of what's necessary for any one thing into one composer package, and then everything else into their own (e.g. a youtube package, a drive package, etc.)
This way.. I can see you having a code dependency that needs Drive, but b/c you've cleaned out everything except YouTube it's going to fail - and that's sort of breaking the way Composer is supposed to work.
All the Google Python libs are pretty brutal. They use lots of code generation techniques IIRC and have their own home rolled future implementation that is annoying to work with from standard python async.
So we are building a hub for task specific scripts for the needs of Windmill and hitting exactly this. We ended up not being able to use googleapis because it was too hard to import reliably with Deno. So in the end, we are ending up reimplementing a collection of per-task importable collection of googleapis. Example: https://hub.windmill.dev/scripts/gdrive/1279/delete-shared-d... that have permalink so you can use them in your own scripts: https://hub.windmill.dev/raw/101/delete_shared_drive_gdrive....
yup. i had a project consisting of 3 node/ts services where each tsc instance would balloon to 4gb+ memory usage due to googleapis, ooming my 12gb thinkpad
I used to work on Pagespeed. People were always so puzzled that adding Google Analytics or Maps to their page would lower their score. "This is your own company's tools! They shouldn't count! Make them fast!"
We always responded that adding any extra "weight" to a page made it slower. We didn't give our own tools any special treatment. It was up to the site owners to decided what costs were worth their corresponding benefit.
Put another way: the fastest page is an empty one, but that's not terribly useful. Optimize for not just for speed but for utility.
(Disclosure: I still work for Google, but haven't worked on Pagespeed - or anything remotely related - in many years).
> We didn't give our own tools any special treatment.
but you also didn't "Make them fast!" in response to the criticism. i know because i've profiled and purged plenty of google's JS from sites in the past. in most cases, this analytics/tag-manager JS far outweighed the site's own JS, to the tune of 5:1.
always found it odd how a company that makes V8, Blink, Chrome's DevTools, LightHouse, Closure Compiler, then evangelizes and blogs about performance but in the end doesnt take its own medicine.
The analytics team - get the best data possible
The page speed team - get an objective analysis of the speed of a page
Even if it's in Google's interest to give their own stuff preference that would probably require some director or VP to coordinate that collusion and it's hard to say whether it would be worth the liability of having done that.
I don't think it's so much wanting Google scripts to have special treatment, but pushing the script teams to prioritize performance too, just like the Chrome and page speed teams.
Make it fast and make it small are not the same thing. The JS ecosystem has a huge culture of thousands of dependencies. “I don’t want to maintain the string trim function so I will import a library just for that” is routine. This is combined with hyperspecialized libraries with odd/unique names which makes discoverability tricky.
An empty VueJS app starts at several MBs of dependencies. That’s before you write any actual code. This is an example of bloat because “I don’t want the responsibility”.
The solution to this is to have more batteries included libraries or frameworks.
It's the same speed/utility tradeoff that mankyd is describing. There was a whole team staffed primarily to improve speed here, but you run into issues where functionality that is very important to publishers (or that makes money for them and so is indirectly important) just needs a lot of client-side javascript.
(I used to work in this area when I was at Google)
when i looked through the JS, it was not written in a way that took size or performance seriously. perhaps it was transpiled from Dart or something. additionally, i think tag manager includes an entire [likely complex] visual DOM tagging UI that gets downloaded even when not used, etc. (correct me if i'm wrong!)
embedding the youtube player downloads 576KB of compressed JS (2MB decompressed) [1], instead of 0 bytes for a cdn-hosted mp4 video; i'm quite certain there is not 2MB of value delivered to embedded youtube watchers.
i think there were/are a lot of low hanging fruit to give technical users / devs the ability to opt out of the whole kitchen sink.
It scores better know, but previously if you even ran Google's own pages about Lighthouse through Lighthouse itself it would score terribly. I don't know if that ever helped my feelings towards the tool or made it worse.
On the other hand, it's a miracle that there's still any Google product that's taking its medicine at all, considering the sheer size of the company and the forces at play to squeeze capitalism out of it.
If the trick to sensible page sizes is letting the site owners decide which features to enable, couldn't we go a step further and let users decide which features they need from each site?
Turn off ads? Disable third-party scripts site-wide? Don't send over any audio? How about webfonts? Tracking? Webgl? Webgl2?
Would it not make more sense to ask the user (or their agent) what features they want to use, before trying to figure out which dependencies are needed to make them work?
Could we optimize not just for speed and utility but also user feedback?
[] Do you want a Million Dollars?
[] Free Massage?
.
.
or when they are booking an airline ticket
[] Gigabit Internet
[] Sleeper Couch
The amount of entitlement HNers have is mind-boggling.
Here's economics 101.
Users pay with attention/money for a list of goods/services from the provider.
If you don't like it, switch your provider. Unlike some physical monopolies, you aren't bound to any provider.
If you think there is a market for your utopian content provider, how about you start your own company (with your fellow entitled HNers and provide all these features)?
Interesting plot twist! Economists do study these trends and you seem aware that the fact of having a right to something is trending among hackers, however I'm struggling to understand how any of what you just said relates to pruning code.
That would require site owners to build sites that actually work across a large range of feature sets. Given most sites barely work in their default version, godspeed on that quest. (I mean, restaurants now have QR codes to download PDF menus. Clearly, site owner incentives aren't at all aligned with customer needs)
Getting hammered to reduce page load time during development only to have people slather a bunch of 3rd party APIs on at deployment time may not take the gold medal for soul crushing experiences, but it ranks at least an honorable mention, possibly a bronze.
You just undid 3 months worth of work that will take 6 to replicate. Congratulations. We're all really impressed down here.
people slather a bunch of 3rd party APIs on at deployment time
I can't imagine what you're environment is like, I can't believe it's acceptable anywhere to slather anything on at deployment time. I'd think an increase in build size would be the least of my worries.
Schizophrenia from the business side who want metrics on TTFB and TTI but also want to have a firehose of information about users and don't connect that there are real performance costs to gathering this information.
Giant dependencies and time to interactive have been battling each other for decades. It's not just how much code you run on first paint, that's a discipline in its own right. But until the javascript and CSS load you don't really control much of anything in the browser lifecycle.
So you work and work on those, and you've gotten through a bunch of performance-as-feature gates, but then someone throws three analytics packages on that together are proportional to the code you sweated to carve out of the initial payloads. You can usually defer those, but how easy that was depends on which generation of web browser you're talking about.
What you can't control is how much work those libraries due on the page while they bootstrap. As recently as the last project I worked on in my current job, I discovered that something like 60% of the event loop blocking was coming from some analytics library that was stuck in a reflow loop that was taking longer than we had saved on code that a couple of my poor coworkers had been working on for ages, to the point you could hear the exhaustion in their voices in standup.
That one was mostly tripping up due to a performance bug in an old version of jquery (mutation on read), but other examples in this problem domain are often not so simple to fix.
Ultimately this is a social problem, but I've seen it way too many times.Someone high in the org chart didn't insist that we run the application exactly as it would run in production. In a Saas situation a customer may say "I want to be able to add stuff to my pages, can you do that?" and nobody catches on that they want to run analytics, and not just for one dashboard, but for the old one they can't let go of and the new one that is cooler but only answers 80% of their questions. And then they want a third party live chat which is somehow as slow as the two analytics libraries combined.
I have ran into similar experiences at a prior job. It was wonderful when I finally convinced our C-level that we should rip out all 3rd party tracking and it materialized in a 1-2 second Time to Interactive boost. It really does make a huge difference in user experience when you are able to prevent this stuff from bloating your websites.
[0] https://github.com/microsoft/TypeScript/wiki/Performance#per... [1] https://user-images.githubusercontent.com/2109932/176479729-...