The problems with P2P is not problems of tech, they’re problems of incentives. The tech exists and it’s mostly solved — it’s just that anyone who is capable of building such a thing is not incentivised to do so, because that work could be spent in more productive (i.e. money-making) endeavours than what will by definition be incapable of making money since it explicitly decentralises the concentration of power.
So if Jack wants you to come build it, and you’re a P2P person, you should go work for him in even if you don’t like him, because he has both the willingness to pay for it, and the broadcast power to make it mainstream. And those are the main problems that needs to be solved, not tech.
I know this because I’m going through the same thing myself — I work on Aether (https://getaether.net), which is close to what I think he is talking about.
If we want ourselves to solve this, and not rely on @jacks of the world, we need to find a way to make P2P into a viable business. For me, in the end I decided to create a Pro version for use within companies, so that the P2P version can be fully free of monetisation concerns.
Thanks for the great project, it fixes the most egregious problems with bulletin boards - which reddit and hn are just fancier guis for.
There's a lot to unpack about the issues of p2p monetization. Probably a few books worth.
Before we get into the social issues I think the original sin of the internet was that IP did not negotiate payments as part of the fundamental protocol. Every layer above that suffers from the same.
This made sense in a government/corporate/university environment where the protocols were invented, but not for the wider world. I'm not going to try and give solutions because all of them are flawed in some way, but better than the current system where only surveillance advertising and altruistic services are viable business models.
> Before we get into the social issues I think the original sin of the internet was that IP did not negotiate payments as part of the fundamental protocol. Every layer above that suffers from the same.
That isn't really what causes the problem. Payments don't need to be inside the network layer of the internet.
The problem is that a digital payment system that works like IP routing doesn't exist at all, even independent of the internet. Existing banks could implement it trivially -- you walk into the branch with $100 in cash, give them the money and they give you a secret passphrase. Then the banks talk to each other so anyone anywhere in the world who walks into another bank and has the correct passphrase can get the money.
Except that there are laws against that. Know Your Customer laws that require them to know who you are and know who the person you're sending the money to is, even if what you're paying for is both legal and private. "Consumer protection" laws that require them to allow you to reverse the transaction, which means they can't just give the money to the other party, they have to sit on it for a while.
The effect of these laws is to make small transactions between random individuals incur prohibitively high transaction costs and subject honest people to capricious denial of service attacks by risk-averse financial institutions without effective recourse. It also makes people afraid to pay for things they don't want an official record of them consuming, which specifically applies to political speech when your views don't align with the party currently in office -- or the party who may be in office next year.
In other words, it interferes with people actually paying people for creating content on the internet. That leaves the creators with only advertising to support them, which implies central control by the ad network.
Fixing that requires a way to anonymously, irreversibly send small payments with low transaction costs. Which is currently de facto prohibited by law.
Thanks for bringing up these points as they are some of the biggest barriers to developing a modern payments system. The technical issues are complex but the legal and regulatory ones are even more complex.
Shameless plug, we are working on Interledger[1]. Which is designed to be an open-clearing system based on packetised payments. It works really well for low value, low-friction payments. Our initial use-case is targeting web monetisation which doesn't work on existing payment rails due to the costs.
Part of the work coming up this year will to be going after the retail payments space and I think the challenges you mentioned are definitely going to be the ones we are going to have to solve next.
No there are no systems that can deal with the $1e-10 you will need to pay for the usual packet. You don't care if someone scams you a thousand times, you do care when someone scams you a few billion times, or when you're talking to millions of people.
The cost of getting it wrong is so low on the internet that you can use the most blunt statistics to still come out ahead.
> No there are no systems that can deal with the $1e-10 you will need to pay for the usual packet.
You don't need packet-level accounting. You need, for example, to be able to charge $0.02/view for a video while getting to keep almost all the money, so that a creator with 10,000 views/day can make a decent living. Then you offer viewers a deal -- no charge if you host the video. Only a small fraction of the viewers take you up on it ($0.02 is not a huge incentive), but a few have unlimited data and are happy to get any price for something that costs them nothing, or it makes them feel like good supporters to be distributing your work, and that's all you need in order to have unlimited scalability and negligible hosting costs.
> You don't care if someone scams you a thousand times, you do care when someone scams you a few billion times, or when you're talking to millions of people.
This system isn't supposed to be a savings account, it's for small transactions. If there's significantly more money in your account than you typically spend on small transactions in a month then you're doing it wrong, and the amount in your account would be the upper limit on what you can get scammed out of.
People would still keep the bulk of their money in a traditional financial instituion that knows their name and can reverse transactions. But a payments system that works differently than that should be able to exist.
What you don't need is accurate and global accounting of packet transactions. A simple cryptographic local ledger between each pair of routers will be more than enough. The routers can then decide how the data is charged, or if the balance isn't closed somehow, just start dropping packets coming from that link. We are not talking about people doing videos - a largely vapid and useless form of entertainment only propped up by legacy add platforms like google because video adds are much more effective than text based ones - we are talking about people running routers in the center of the network. Once we have decentralized the lowest levels of the internet then we can start thinking about the higher ones.
After all the choice between lord Google or lord Verizon isn't much of a choice.
> Only a small fraction of the viewers take you up on it ($0.02 is not a huge incentive), but a few have unlimited data and are happy to get any price for something that costs them nothing, or it makes them feel like good supporters to be distributing your work, and that's all you need in order to have unlimited scalability and negligible hosting costs.
So again, the main business model is altruism with no way to actually monetize the transport layer. Replacing the kings of layer 4 with the kings of data is not a victory. It's just a different tyranny. One I don't see being much better than the current one.
Only when you can make money in each layer of the network can the internet truly be free (as in freedom).
>If there's significantly more money in your account than you typically spend on small transactions in a month then you're doing it wrong, and the amount in your account would be the upper limit on what you can get scammed out of.
Yes, that's why you shouldn't do it that way.
You don't spend 'money' you have agreements with your peers over how much data they are willing to receive from you. The clearing process is completely independent. Cash, cat pictures, happy feelings, revolutionary zeal. All are acceptable currency if the owner of the router agrees with you.
> The routers can then decide how the data is charged, or if the balance isn't closed somehow, just start dropping packets coming from that link.
At this point you're essentially arguing that network neutrality is bad and what we really need is the opposite of that.
Realize that in your system, it's Verizon and Comcast who have all the power. They're sitting on millions of customers who have no other network path to them, so they would get to charge monopoly prices to anybody who wants to communicate with them, and the customers themselves have no other choice because the incumbent ISP has a local monopoly (or the sort of totally inadequate competition that DSL is now putting forth against cable/fiber).
And if you're paying for sending data but not for receiving it, it would give ISPs all the more incentive to keep doing and increase doing the things like selling highly asymmetric connections that upload much slower than they download, which make it harder to create competitive P2P networks.
> So again, the main business model is altruism with no way to actually monetize the transport layer.
How is the network not already monetized? Is there some alternative to Verizon and Comcast where you're paying no monthly fees, short of building your own network like Google does? Even then you still have to pay for it, you're just paying for land, labor and equipment instead of service.
You pay your ISP to get you to a peering point. That's monetized. The other endpoint pays their ISP to get to the peering point. Also monetized. What's left? The peering link? The actual cost of that is such a small percentage of the total that you can round it off to zero, and people only really try to charge for it when they're rent seeking because they have a monopoly path to a large number of customers.
> Yes, that's why you shouldn't do it that way.
Because occasionally people who make bad security choices will be encouraged to do better after suffering an uncovered loss of something like fifty or a hundred bucks? That seems like a small price for having a payments system that has low transaction costs and protects privacy.
> You don't spend 'money' you have agreements with your peers over how much data they are willing to receive from you. The clearing process is completely independent. Cash, cat pictures, happy feelings, revolutionary zeal. All are acceptable currency if the owner of the router agrees with you.
How is that any different than it already works? Anybody can configure their network to accept traffic or not from whoever they like. What's missing is an efficient method for clearing small payments -- but that's a general problem, not anything specific to internet links.
Payment is far too complex and abstract to contemplate building in at layer 3 (IP) of the network stack. Even in some idealized alternate history it would have never worked, and probably still couldn't work today.
The internet is happily chugging along on routers and switches who know nothing about the global topology of the network.
When you reduce the payment problem to 'how do we get two routers to keep track of the data passed between them' it is a non-problem that even the first generation of imps could have solved).
Why? I don't care that I will need to pay 1e-10 dollars to have a packet sent 10 times. Likewise I don't care if I lose 1e-10 dollars because someone doesn't pay for their bandwidth.
Networking is just not worth lying about until you get to sums for which lawyers can get involved, at which point you have to deal with lawyers.
P2P is not really solved. Not to the true satisfaction.
For example, there are tens of millions of images uploaded to Twitter daily. They all have to be stored, and then retrieved for nearly arbitrary tweets and arbitrary users.
All this content has to be stored somewhere and retrieved fast enough to be useable.
This comment is underrated. The theory is solid but piped piper doesn't exist in reality. Striping content across the network is the way to go. STORJ apparently is working on it as is the Interplanetary internet idea. It reminds me of FidoNet back in the day.... The internet we have exists because of low latency networks.. the question is how do we maintain that illusion?
Because anyone who tried that got burned with child porn and the threat of real prison time.
p2p tech is fundamentally incompatible with American morals. You will never see a p2p based firm be successful in the US any more than you could have had the industrial revolution in Ancien Regime France.
Each user storing what they post doesn't really scale. If I have a 100kb image in a post that gets popular, then the following technical things are needed:
1) It needs to work reasonably fast when the suddenly popular post is seen by, say, 10000 people in a short time. My home uplink won't handle that.
2) It needs to work when my computer is turned off or disconnected from the internet.
3) (optional, but still important) People on the other side of the world should be able to get popular data from a cached copy that's physically closer, both for latency and bandwidth reasons.
"Store what you post" isn't sufficient, all that I post needs to be stored also by someone else (e.g. a CDN) - but that immediately rises all kinds of issues about liability. If only I am storing what I post, then the system can afford to not care if that image turns out to be child porn, but if someone else is storing what I post, then that's a problem for everyone.
Halfway through reading your comment I was going to mention Aether as a counterpoint. It's a great project that I'm happy to support even if I don't use it as often as I'd like.
I generally try to only charge it on the months I push a new release on, and otherwise keep it paused. I’ve found that’s a good way for me to not stress out when my ‘day job’ gets too hectic.
Incentivisation is a problem that's faced by many decentralized technologies.
For example, one of the problems with Ethereum's Whisper protocol (decentralized messaging) has been that nodes aren't incentivised to process messages. There are other problems too and some efforts are being made to generally improve the tech[+].
thanks for sharing, I just went through your docs trying to understand the architecture and realized that by nature your system has to erase data after a certain period, otherwise individual nodes would end up retaining potentially large amounts of data. Is that correct? Could a flood or gossip based system have full message history without ending up grinding to a slow halt?
Yes, this is in fact a privacy feature - the data is gone 6 months after its last reference. In other words, we garbage collect for you so that you won’t end up dealing with a post you wrote when you were 14 10 years later, out of context. It’s sort of like the right to be forgotten.
That also makes it relatively unsuitable for long-term storage of stale information. Active communities will last, but archived ones with no new content won’t, and eventually be gone.
No. This is about not collecting data by default, so what you mention is certainly possible. That said, if you stop posting, your profile would also be gone in six months, so a third party would no longer have an entity to correlate the history to.
Histories are useful only if they relate to an active entity and you have the option to kill that entity and move over to some other name - and that new name would be impossible to link from the old one.
Well, they can archive the profile too then I'm assuming. So you can't really ever kill the fact that entity existed in such a scenario, but you can stop using it from its original space. It seems to me that companies like PDL and Oxy, (even the smaller players like RapLeaf potentially), could still put together your identity since they use the full body of your internet usage, and even non internet information about you.
I know it's hard to thwart people like Oxy, that are stacking all sides. Providing both VPN and proxy services. Then using those services to sell anonymized scraping as a service. Then collating online data obtained from scraping with offline data to create ridiculously detailed profiles about people. (Then leaking them all by the way. As though collecting them wasn't bad enough. [1]) But my ideal distributed social media would be architected to defeat something like Oxy. Because Oxy, PDL etc are already behemoths and the government has more comprehensive data on you than even Oxy.
I realize though that aether is probably just something to "keep the honest people out" so to speak. But my vote for a "nice to have" would be the ability to thwart some of the data mining and linking baddies that are out there.
Cryptocurrency/blockchain is a great to power incentives and pay for development. I currently an paid this way to work on a self-sufficient project that operates in this manner. (We don't rely on any fundraising funds as with the more common projects you're familiar with)
Submitted title was "Micro.blog founder on Twitter's bluesky project". That's good information! but the place to add it is as a first comment in the thread, not in the submission title.
I'm suspicious of motives here as much as anyone, but I think the issue of policy enforcement (and policy itself) is very important here. It already occurred to me that Mastodon and Gab were, despite being worst enemies, together inadvertently creating a reasonable content meta-policy. Some people want freedom of expression, some people want protection from objectionable material. With ActivityPub, you can choose your gatekeeper, and still more or less exist in the same social universe.
> Some people want freedom of expression, some people want protection from objectionable material.
You make things sound far too benign. Where do death threads fall in this classification, are they "free speech" or "objectionable material"? Where do disinformation bot armies fall? There's a lot more to content moderation than just "that person doesn't want to see boobies or the f-word".
Presumably most people don't want their life threatened so almost nobody will pick an instance that allows that to happen. The few people who are interested in posting death threats can go ahead and make a rogue Gab instance or something, and every other instance will block them. They'll be their own silo where super sketchy things go on until the feds decide it's time to step in (since death treats are legally actionable). Such silo websites are already possible without the fediverse (and by some accounts already exist), so my model doesn't change anything here (assuming blocking is effective). (BTW even Gab dot com would block that; they ban people for death threats and porn and maybe even Doxing).
Disinformation campaigns? Well again, if you don't want to be misinformed, pick your gatekeeper. Probably all the better. If somebody's going to protect me from being misinformed, I want to know what their biases are. I don't trust Twitter or Facebook; it's not a simple and objective task.
I think your point that this stuff is complicated is exactly my point. It is complicated, and people have different preferences. No one company can be the gatekeeper that satisfies the entire planet's preferences (at least, not under the current models).
In this context, I think they meant "free speech" and "objectionable material" referring to the same things, but by different groups. So death threats would fall under both: Gab would allow them, most other Mastodon communities wouldn't. People (or law enforcement agencies) can then choose their gatekeeper.
Gab, per their claims anyway, use US law as their guideline (plus banning porn). I know they've removed at least a couple people who cross a certain line when it came to directly advocating violence.
If they immediately tasked 20 or 50 people to the project, it would be far more absurd than starting with five. Ideally a project like this starts out from a small team and is given more resources only when it's needed.
> Trying to change the paradigm about how the planet connects and share information is an incredibly challenging task.
Exactly the kind of task you want a small team doing the initial research and planning on otherwise it'll be bikeshed central in every meeting, surely.
adding more people to a project will most of the time make things worst, I'd rather work with 2 very competent people than having a team of 20 mediocre ones.
How many people made most of the tech innovations we enjoy now? Depends on who those five people are. Could be the next Bill Gates, Woz, Paul Allen, Steve Jobs, and Bezos on the team.
> Could be the next Bill Gates, Woz, Paul Allen, Steve Jobs, and Bezos on the team.
I certainly hope not. While these were successful it does not mean that their projects were good or that they contributed for the good of mankind rather than for its downfall.
They should just use ActivityPub. Sure it has problems, but federating with their biggest decentralized competitors would signal that they are serious. Improving the protocol can come next.
Open Source has generated $Trillions in value creation over the last few decades.
Twitter is fighting to stay relevant, they dropped to Top 35 website from Top 14 in last several months.
This is an opportunity for them to become a global standard, and that is worth it.
The most important piece is that they can scale when they pull it off. I've been working on decentralized tech for a while (I run https://github.com/amark/gun) and can say it takes a lot of attention that most projects (dare I say "blockchains") have no scaling abilities at all.
> If Twitter is hoping to outsource curation to shared protocols,
I don't think they would ever do that. It would be really dangerous to implement and would possibly expose an unfiltered stream of partially illegal content.
I mean, can you imagine "validate if this is an unsolicited dickpic", or "verify if this is a dox attempt", or "verify this is cp"? This is one area that can never be exposed to non-employees.
It's almost certainly done by non-employees now, through contracting arrangements. While it's possible the unfiltered stream never leaves Twitter's premises or Twitter's computers, and the individual contractors sign an agreement directly with Twitter, it's only one step removed from just sending your stream to the service and getting back an annotation.
Yes, they have contacts/agreements with the parties that get access. Whether that's full time employees or contractors, I don't think that changes the main issue. They will never expose that information to the public.
There's nothing wrong with NNTP. It's exactly what they'd need. Trouble is developer memory: people don't read back on technology, and thus, no trendy dev remembers NNTP in the container era.
The Matrix protocol is more or less an open, replicated database. E-mail is similar in some ways, but:
1: it doesn't define enough metadata ("this is an image", etc)
2: it doesn't handle backtracking well, if one of the recipient servers misses a message (servers are just supposed to retry a given number of times).
3: it has no security/authentication concept.
All of these could be bolted on regular e-mail, but calling it e-mail at that point would be a travesty, and we might as well call that a new protocol. And if we are writing a new protocol, we might as well take a few liberties to address the shortcommings of the first (one could argue against the last part).
It’s an experiment to explore a distributed Twitter. People read way too much into those tweets. It seems clear that they will evaluate existing projects, and technologies and make their own decisions.
More competition in this domain is a very good sign IMHO.
This seems right. Twitter is not engaging in this in pursuit of real openness, and the end result--if anything--is unlikely to be better for twitter users.
People want the cloud so that they can be responsible for things at the right level of abstraction. Nobody wants to maintain a server with a message bus and a bunch of individual applications that communicate on it.
Just set up AWS EventBridge (or Azure EventGrid), or any of a dozen other options that don't require you to administer a server. And then use AWS Lambda, Azure Functions, Azure Logic Apps, Zapier, etc, etc. Don't maintain a server that runs a bunch of apps.
We care about what we're managing, and care about being responsible for it, we just want to manage it at the right level of abstraction. Let the cloud provider manage and be responsible for the hardware.
So if Jack wants you to come build it, and you’re a P2P person, you should go work for him in even if you don’t like him, because he has both the willingness to pay for it, and the broadcast power to make it mainstream. And those are the main problems that needs to be solved, not tech.
I know this because I’m going through the same thing myself — I work on Aether (https://getaether.net), which is close to what I think he is talking about.
If we want ourselves to solve this, and not rely on @jacks of the world, we need to find a way to make P2P into a viable business. For me, in the end I decided to create a Pro version for use within companies, so that the P2P version can be fully free of monetisation concerns.