Hacker News new | comments | show | ask | jobs | submit login
Reducing Slack’s memory footprint (slack.engineering)
48 points by cpeterso on Mar 3, 2017 | hide | past | web | favorite | 44 comments



Slack has to be one of the worst pieces of software I have seen based on what it does and how it behaves. It gobbles CPU and memory to do something that should require neither. It is offensively wasteful.


it reminds me how lean common IRC clients are (written in C like mIRC). 15MB RAM and less than 1% CPU is enough for 2 networks and several channels (even with hundreds persons in a channel). Incidently it was also mIRC that showed that Intel removed some CPU features from the first Intel Atom CPU gen. Processing lot's of text isn't within the capability of Atom CPU, something even a Pentium 3 had no problem.


I used ircII under various UNIXen and the resource footprint was even smaller.

That being said it had a negligible UI footprint since it was a curses-based text UI, doesn't embed media content or render text in a GUI, and it doesn't really do much.

Slack does a lot more, but it shouldn't need to burn CPU while doing nothing.

But it is an illustration of how little resources you actually need for IO, simple state management and dealing with the protocol layers. You'd accept bloat in all of these areas to get more functionality, but not by orders of magnitude.

Out of curiosity I downloaded ircII-latest, ran `configure` and `make`, and then started it up (about 30 seconds from clicking the link to starting the compiled binary btw). Initial memory footprint on startup was 465kB. Now it is up to a whopping 604kB. As it is mostly idling (very little chat traffic on the channel) the CPU use is at 0.0%.


Indeed - my frequent complaint is how opening slack and trello will use up half the memory on my small laptop.


And yet Slack runs just fine on my 2016 12" MacBook, iPhone 7 Plus, and iPad Mini 4.


"...with a single team of 5 and 2 channels".

Seriously, it's not about "running just fine", it's about how many resources it uses. Even the startup time is awful for what it does.

And the Slack team itself recognizes that there are issues -- as seen in TFA:

"However, these capabilities come at a cost: the desktop client can use a lot of memory. This memory footprint increases as the user signs into more teams, as each team runs in its own webview. More memory usage means worse performance, degrading our customer’s experience of Slack and their other applications."

and:

"At the time of writing, about 36% of active Slack users are signed into more than one team in the desktop app. 17% are signed into 3 or more teams, and 5% are signed into 5 or more teams. Each of these teams is running the full JavaScript web application and maintaining and updating the DOM tree as the state of the team changes. This application can consume between ~130MB (p10) and ~960MB (p99) of memory depending on team activity and UI state."


I suppose you may not know any better. And I am not saying that to be mean, I'm merely pointing out that your frame of reference may not be the best to judge how well the desktop Slack client was done with regard to resource use. You may not understand just how few resources the desktop client could reasonably be expected to consume.

And you would not be alone. Over the past 7-8 years I'd had to teach people how to benchmark things. I see people measure things and then use their first result as the performance reference. Sure, they can see relative changes, but if you are orders of magnitude off a reasonable result, a relative reduction of a few tens of percent is still orders of magnitude off.

As a developer you should have an idea what a _reasonable_ number ought to be. For CPU, for memory, for latency, for anything you care to measure.

"It runs just fine" isn't data. It isn't a useful metric for performance.


I see a lot of it lately. For example I looked for some kind of status bar GitHub notification tool. The most popular open source one includes its own electron environment... to display one icon and a list. That's a >100MB app for a completely trivial task.


> "It runs just fine" isn't data. It isn't a useful metric for performance.

I don't see any claims from "mordant" that their anecdote is a useful performance metric. If anything it'd be a data point that could be used as a user experience metric. ie. X% of users in the target demographic agree with this statement, "works fine for me."

The point was a counter to your claim of Slack being, "offensively wasteful". Further, isn't "offensively wasteful" equally useful as a performance metric? If you're experienced in performance perhaps you could share some of your "reason numbers" and contrast that to measurements your made against the Slack client.


By "offensively wasteful" I mean off by orders of magnitude. At which point the problem isn't one of mere programming but of actually having some idea of what you are doing.

Recovery from this state of affairs doesn't happen by telling people how to fix concrete, individual things, but teaching them to think about what they are doing and to analyze things. In order to know when you are off by orders of magnitude you need to have some idea of what is reasonable before you measure. If the reality you achieve is way off what is reasonable, you either need to change your perception of what "reasonable" is or you have to face the possibility that you may not have done a good job.

A chat client that deals with perhaps 3-4 teams and perhaps 200-300 people should not require 1Gb of memory and it should consume a negligible amount of CPU. Any way you slice it that's a LOT of resources per user, per team or per communication volume. So Slack miserably fails even cursory glance at workload/resource usage.

You can start off with the lower layers: speaking a chat protocol. How much resources should that eat up? For a chat application: too little to measure on a modern machine. (I spent most of the early 2000s working on web-scale web crawlers on much, much weaker server machines than even cheap desktop machine is today). The IO should have trivial cost, the protocol parsing should likewise have trivial cost or you have messed up badly.

Then move on to the internal state. Draw it on paper. What are the main structures you will be needing. How much data do you need to keep in memory, how much space does it need to take, how do access it and manipulate state? What is a reasonable CPU expenditure?

Next is the UI. How do you make it responsive. How do you make it not gobble up tons of memory. Are there tradeoffs you can make? How do other applications do things? Where are the resources wasted?

One doesn't produce a resource hog like Slack by doing one or two things in a wasteful and sloppy manner. One does it by having the wrong attitude and systematically doing a bad job.


Thanks for your response. I generally agree with the technical approach, but not with your assessment of Slack's products.

> By "offensively wasteful" I mean off by orders of magnitude. At which point the problem isn't one of mere programming but of actually having some idea of what you are doing.

That would imply that Slack, doesn't know what they're doing. Slack is an extremely successful company. I myself use it for several teams simultaneously. From a user and business perspective it generates a lot of value and that ultimately trumps criticisms of their implementation.

> One doesn't produce a resource hog like Slack by doing one or two things in a wasteful and sloppy manner. One does it by having the wrong attitude and systematically doing a bad job.

Perhaps what you're taking into consideration isn't broad enough. Going back to the original comment, if they did such a bad job, then why am I able to get a lot of value from it? Their implementation seems "good enough".

What's the negative consequences of their "orders of magnitude" performance problems? Does it destroy it's business value? What would you do if you were the CEO? Cease all active development and re-build everything in scratch in c++?


I'm not denying their success. But I'm also not attributing it to the quality of their product.


And yet I can do internet chat just fine on my Names three outrageously expensive and highly powered devices...


Cool technique, but you hint at the best solution: "running all teams in a shared context to eliminate the overhead of one webview per team."


Wouldn't the best solution be write it in C and have the entire app take 20MB of ram?


I think Slack is focusing on code reuse, eg. using the same HTML/CSS/JS codebase across all platforms - hence it would be preferrable to avoid having to compile for every platform.


I don't think compliation is the issue, it's supporting the diverse platform APIs and UI toolkits.


They use like 5 widgets (styled text, textfield, buttons, radios, etc). They could create their own UI toolkit with 1/10 the resources they have.


If this is true, then why are even bigger players with more resources like Facebook and Microsoft building on top of Electron instead of building natively?

I surmise that you are not alone in believing that Slack would be better built as a native app. But if it's such a bad direction why are so many going down this road and what are they missing?


>If this is true, then why are even bigger players with more resources like Facebook and Microsoft building on top of Electron instead of building natively?

Because software is a pop culture, and the latest fad always prevails at most shops. Also companies care more about cutting corners and shipping sooner than about the long haul.

That said, it's not like Facebook particularly enjoyed web based apps: "Mark Zuckerberg: Our Biggest Mistake Was Betting Too Much On HTML5"

https://techcrunch.com/2012/09/11/mark-zuckerberg-our-bigges...


The mistake there was one of timing and approach and they learned from it. The promise of cross-platform, web-based technology is still there, that's why they created React Native.


React Native is not web-based. It just uses Javascript.


The technology originated from web. It uses javascript, React, babel, flexbox, etc.


> ....Facebook and Microsoft building on top of Electron instead of building natively?

JavaScript devs are usually cheaper to hire than C++, Java, Objective-C, Swift, .NET ones.


I think VSCode by Microsoft is a good example of Electron done well.

In their case, I don't think it's because their web technology devs are cheaper. I wouldn't be surprised if that team knew multiple platforms and languages. I think it's all about finding the balance between dev costs, making a great product, and getting it out fast on multiple platforms.


On their case, they want to get a few notches up in the SV popchart culture, top of the pops.


Being cool in SV is only so valuable.


Apparently for the web dev generation, writing modular native code across OSes is a lost art, attained by us old druids not worth listening to.


Yeah, and it's especially puzzling since they've given up one of the major hassles of going natively cross-platform anyway: having to support a native look and feel.

Which makes creating a similar (uniform looking across platforms) native UI lib even easier than catering to the intricacies of each platform.


Yes, it is incredible the amount of gimmicks web devs do instead of using the right tools for native development.


Could you elaborate on what you think the elaborate on what the "right" tools would be?


Write in native languages, common to all platforms, using the OS specific APIs in each platform with the help on an abstraction layer.

Lets pick C++ as an example of native language.

Design the application architecture with the abstract ideas of what should be available across all platforms, but keeping in mind the native idioms.

For each of such concepts, provide an interface (pure abstract classes in C++ speak) with an implementation for each OS.

The actual cpp files are selected during the build process according to the OS being compiled, without any kind of #ifdefs.

Then showing a notification to the user using this architecture is as simple as.

    auto notification = NotificationService.instance();
    notification->show("New email available!");
Or opening a file, using the platform specific file dialogs:

    auto filePicker = Pickers.file_instance()
    auto filename = filePicker->open_file();
This are just two basic examples, the idea is not to replicate Qt, wxWidgets, JUCE,... rather pick just the concepts required for the application, not to build a full blown framework.

This is the approach taken by quite a few companies.

Of course, there is also the shortcut of using something like Qt, but then you need to live with the problem that the widgets aren't native, rather emulated.

On the other hand, still outperforms any browser based application.

The craziness with Electron based apps isn't nothing new to us oldies.

Since the early days of the web, OSes have supported web widgets like MSHTML on Windows, Windows packaged Web Applications introduced on Windows Active Desktop, Mozilla's XUL and many others.

They were never a match to native UI/UX.


why not use the right tool for the job instead of doing Olympic-level gymnastics to use a hammer as a screwdriver?


Because it is faster and cheaper to get a junior JavaScript developer to bang out a cross platform GUI in electron than it is to get one or more developers together to write a proper cross-platform native application.

They cared more about cost and time to market than performance and technical prowess, and in a business context, that is often a correct trade-off.

I don't like it because I prefer small fast clean applications, but I know I'm in the minority and I accept that that's how the business world works.


They cared more about cost and time to market than performance and technical prowess, and in a business context, that is often a correct trade-off.

Cool when you are starting up and all, but surely they must have a bit more money now, and would be able to do it right?


When Facebook was hitting performance issues with PHP, they decided to rewrite the entire runtime instead of rewriting in a different language.

If Facebook with its billions decided it was cheaper to rewrite PHP, the cost/benefit analysis must favor optimizing current code than doing a rewrite


Facebook is Facebook. Slack isn't Facebook, and neither is every other business besides Facebook Facebook decided to rewrite PHP because the CBA worked out for Facebook - that doesn't mean the argument holds true by default for any other situation.


But why spend the money to do it from scratch when you can spend less focusing on the pain points?

If your users pay you either way, the cheaper option (from a business point of view) is better.


Because Electron (or whatever they use) is web-scale.


Because Electron (or whatever they use) is web-scale.

Web-scale? What's web scale? why does my desktop client need to be "web-scale"?


Everything needs to be web-scale:

https://www.youtube.com/watch?v=b2F-DItXtZs



That's hilarious


This.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: