It's very interesting that Dan discusses the issues of cloud latency vs on-prem with on-prem being faster; something that some on HN have pooh-poohed as not being something that mattered.
And that bandwidth costs they were paying suprised GCP , when it's been pointed out that bandwidth costs can be quite low vs the cloud providers. Granted Twitter is operating at a very large scale...
SLOs for most SaaS are really loose (if they exist at all). Almost the whole industry is leaving 10x-100x performance on the table because I guess very few people even notice.
I follow Dan Luu and he often writes very well, but my perspective as a user is that search is rubbish. I know they're solving a complex problem and indexing is x seconds but I genuinely just use Google with Twitter added as a keyword.
For my own tweets and those of my closest friends, I have them all posted to a group Slack we have and I just use Slack search. It's absurd but instantaneous results.
Ultimately, I'm sure they did lots of kernel hacking and ingestion and ETL stuff but Slack search of Twitter mirror is better than Twitter search for my follow list.
This post is more than ten days old. Twitter has only gotten better. Why can't people exercise some discipline while running their mouths? If only to embarrass themselves less.
You could block his account or the keywords 'Elon' and 'Musk'. That's the idea anyway, to provide moderation tools to the end user. You should check back when that idea is put into practice more comprehensively by Twitter engineers.
I don't know what's cooking there either. The point is the end-user experience has been nothing like what these Nostradamuses have been predicting.
There is no cost for making wild predictions that turn out wrong. In real life if a friend did that you'd begin discounting his talk. On social media that doesn't seem to happen, because there's a sucker born every minute, and crowds like to indulge in mood affiliation and emotional venting rather than rational analysis.
Given that all of this tech is already being used inside Twitter, I feel it is unethical to discuss any non-public info (like discussions with cloud vendors on cost etc.) in public. The engineers have been kicked out, the tech still belongs to Twitter. This is not "decimation" of the engineering, just of engineers' egos. Someone else will do the job, the show will go on.
Many of the projects mentioned there are open source and have been known publicly for a while, and talks about them have been presented over the years in many conferences.
I trust the Tesla engineers who thoroughly review the code for a few hours more than this guy who worked at Twitter for years.
After spending time reading Elons first responders on Twitter, I have learned that in the hierarchy of human competence, rocket scientist or FSD engineer > engineer supporting a website. This means naturally they are better at everything even when they stoop to doing a lesser job and would quickly recognize the coding deficiencies at Twitter.
And that bandwidth costs they were paying suprised GCP , when it's been pointed out that bandwidth costs can be quite low vs the cloud providers. Granted Twitter is operating at a very large scale...