> I think the justification Eugen (Mastodon's main developer) gave for not loading a user's past activities is that it would actually harm smaller instances when a user goes viral and the whole network hammers their servers for their content.
Maybe we need some kind of protocol that can spread the load where popular messages are duplicated across the network instead of being stuck on one node that can get overwhelmed.
Since ActivityPub is mostly composed (75%) of immutable json payloads, they are very suitable for caching. The existing infrastructure should be more than enough.
> Maybe we need some kind of protocol that can spread the load where popular messages are duplicated across the network instead of being stuck on one node that can get overwhelmed.
You mean like any proper P2P protocol?
Add to this multi-cast on internet level, and everybody could actually operate something like YouTube on their internet router…
But of course economic interests of some (already ultra rich) minority will prevent that forever, even we could have it just right now form the technical standpoint.
The irony and tragedy of the Internet is BitTorrent solved Internet Scale at the protocol level, but is at odds with how content companies, ISPs, and Silicon Valley want to organize themselves.
It's also at odds with how people consume content.
If you cram every social interaction a network has in a massive common distributed store (which is what using a bittorrent like protocol would entail) you have every node requiring to consume gigabytes over gigabytes of content, despite being interested in a fraction of a fraction of it.
I don't know if nostr or scuttlebutt resolve this issue, but if they don't ... how do you imagine a mobile app for keeping in touch with your grandma, requiring gigabytes of data transfer and storage would work?
At this point people saying bittorrent is a solution for social networks are basically just the guys who know about hammers and seeing nothing but nails.
I'm advocating it as the content distribution layer, not the social interaction layer. You don't even need all of the metadata, as there are BEP extensions to distribute that as an index [0] or an RSS-style feed [1]. Images are particularly painful, like the previews in Mastodon, because they hammer the origin server rather than looking around at what other peers have. This problem will only get worse as the federation grows in size, both in users and in servers.
I say irony because we had all these scaling problems two decades ago during the peak P2P years, including federation and implicit DDoSing, and BitTorrent emerged as the lasting protocol because it could handle the scaling issues.
I say tragedy because for this to work you do need some openness, which puts people at risk of content companies trolling for you, ISPs dropping you, and is completely antithetical to the control Silicon Valley demands over its users and data.
Then the fediverse is already there. The video hosting service PeerTube uses bittorrent to distribute the video among viewers.
There were some plans at one point for another project which would deal with music and would host the media on IPFS. Sadly I've never seen any progress on it, despite the idea sounding very good.
The mobile app isn't a peer in this hypothetical torrent network. Your mastodon instance is and it serves you using the same client protocol we have right now.
That way, you solve for the potential hosting issues of smaller hosts not being able to handle the load in case of viral posts etc, all without the clients having to know anything about the implementation details.
If you still require servers that you need to connect to then you kind of lose the benefit of having a social network over a p2p protocol. Or you're also thinking that the social serving aspect is parallel to the media serving aspect that another sibling mentioned.
Yes, but you were downloading a movie and then you'd delete it to make room for something else on your 40GB HDD. In this case it would be like you'd try to download the whole internet because you're interested in one movie.
I doubt "valuing virality" is the real problem. But when you're exposed to a universal network whose numbers increase every day, there are moments when it happens no matter if you want it or not. The "slashdot effect" has been coined maybe 25 years ago, so it's obviously not a new thing. :P
Maybe we need some kind of protocol that can spread the load where popular messages are duplicated across the network instead of being stuck on one node that can get overwhelmed.