Since I know the skill gap on HN is pretty wide, I just wanted to chime in to say that if this is some kind of counter-intelligence measure, it's not really going to work considering that the US government is collecting information at layer 1 (i.e. directly gathering information from the wire).
A true "alternative internet" would, at this point, require its own infrastructure due to the routers and switches of the Internet using the TCP/IP stack. (assuming that an alternative internet would be a variation on the IP protocol).
The entire issue with the government surveillance of today is that they're (allegedly) storing all data, even the encrypted data, and holding it to either brute-force crack, waiting until whichever cipher/protocol gets broken, or waiting until computing power is abundant enough to crack the encryption-du-jour in a reasonable amount of time.
This is why ideas like Raspberry Pis on balloons have been proposed as true "alternative" internets. Not to get all tin-foily, but I would assume that any undertaking that ambitious would have government agents working from the inside.
If you're transmitting any data that could be illegal, your safest bet is to use some kind of encryption that would take longer than the statute of limitations for whichever crime you're committing to break (you'd probably also have to factor Moore's law into your calculation and pray that quantum computing doesn't become viable in the next x years).
Additionally, it is worth noting that the weakest target is very often the human element. Although this was noted by Mitnick in his "hacking" (which generally was just clever social engineering), it is equally true when governments are targeting you. The stasi (or other pernicious organizations organized to carry out state censorship) could easily have someone smeared in the press, dismissed from their job, or many other means short of official imprisonment.
To the point of encryption, as noted by the NSA (inclusive of present revelations), it actually makes you a target for state surveillance. This is a bizarre catch 22. If you are using facebook and "rah rah-ing" about Obama or the latest political fever you probably are at very little current risk of long term data retention. The government doesn't care about you. If you slip into politically suspect views and/or start encrypting your messages simply because you don't want the government to read them, you attract government attention, including the likely possibility that your communications will be retained.
Moreover (and this is where it gets scary, even if it wasn't already), the likelihood is that the many people that are part of your social graph (even if you aren't involved in social networks, some friends likely are and have probably shared their email contacts at the very least), governments can very easily figure out your social circles, and how suspect they may be from any standpoint that the government cares about.
Today this may be "terrorism" (itself a vague term, but one that at least nominally includes the threat of violent acts), tomorrow it may simply be having politically incorrect opinions of some variety. As noted by many, even the nominal but legally actionable protections for American citizens have been eroded to the point that the executive branch can hold anyone in infinite detention without trial. A world without borders has become the panopticon, a jail where we are bound by ever-peering eyes of our peers.
"What you say, can and will be used against you in a court of law" -- even if you said it 10 years ago when you were drunk in a private message on Facebook.
If you are using facebook and "rah rah-ing" about Obama or the latest political fever you probably are at very little current risk of long term data retention.
I'd go further and say that at present everyone (rah-rahing Obama supporters included) is currently at risk of long term data retention, given the facilities currently being built and the policies of the NSA/GCHQ/DGSE etc.
In the UK at least the ambition is complete surveillance, and the present reality is the ingestion of ALL data entering and existing the UK, so the vast majority of internet traffic is already monitored, and stored for as long as they can manage. Snowden claimed in another video released today that the same situation pertains to the US, though I don't think we've seen documentary proof of that yet, just that they're collecting all metadata, which is bad enough. They can probably store metadata indefinitely already, and why wouldn't they? In the next few years as long as they can manage will be 1 year, then 10, then soon after lifetimes - that will easily happen within the lifetime of a person alive today, if the current policies persist. And one day as you say all that data will be used.
So to extend Obama's assertion and make it less untruthful - no-one is listening to your phone calls, but someone will, one day.
Fascinating. My preferred choice for transmitting arcane knowledge is poetry (in fact, there is a long tradition of exchanging alchemical poetry in cipher).
These are all really cool and can be very useful, but personally I think decentralization and meshnets are to some extent a "false god" from a political and activism point of view.
The fact is that any nation state can trivially attack the physical Internet. It can order any corporation operating under its jurisdiction to cooperate. It can force any operator to install filtering software or equipment. Come hell or high water, it can physically unplug things, force power companies to turn off power, etc. Given a political worst case scenario the wired Internet (and that includes things like 4G) will not and cannot be any safer from state tampering than any other infrastructure.
Completely anarchistic decentralized meshnets that operate over things like peer to peer wireless have another problem. I am not aware of any existing totally decentralized meshnet protocol that isn't vulnerable to some form of attack against its trust system.
Then there's "long con" attacks where a group of malicious nodes joins the network and pretends to be friendly for a long period of time, adding a lot of resources and gaining a lot of trust, only to finally use that trust to fragment, DOS, snoop, or otherwise harm the network.
Any of these attacks are well within the technical capability of a well-funded nation state.
In addition, any wireless mesh can have its users located through frequency triangulation. A nation state wouldn't even have to arrest everyone. It could simply isolate the most important nodes, locate them, and confiscate the equipment or pressure, arrest, or pay off their operators. For a fee some trusted operators might be transformed into double agents.
The reason I call it a "false god" is that I fear for some it can act as a substitute for real political activism. The fact is that in a political worst case scenario, nothing is safe. Thus the real battle to be fought is not in the technical sphere but in the human political sphere. We must preserve our liberties or they will be taken from us, period, and no amount of clever tech will truly stop that once the enemy is upon us.
> Thus the real battle to be fought is not in the technical sphere but in the human political sphere
Most people only analyze immediate effects - they don't go out of their way to think in abstract models and worst-case scenarios. If they're being told that the panopticon is necessary for catching scary bad guys and see no immediate negative effects for them or their tribe, they're going to believe it. Only by becoming acquainted with privacy-preserving technology will they ever come to expect communications privacy, realize the government is working against them, and organize to preserve their autonomy.
> If you're transmitting any data that could be illegal, your safest bet is to use some kind of encryption that would take longer than the statute of limitations for whichever crime you're committing to break (you'd probably also have to factor Moore's law into your calculation and pray that quantum computing doesn't become viable in the next x years).
Not all crypto algorithms are breakable by quantum computers. Just use one of the ones that is not. (Because the most common algorithms are based on integer factorization, which can be done efficiently with quantum computers, there seems to be a widespread belief that quantum computing makes encryption with normal computers impossible.)
Wired networks are a lot more difficult to attack than wireless. It's way more cost-effective to just use existing infrastructure. Layer 1 doesn't give anyone any information, and Layer 2 is hardly usable across networks, and you can encapsulate layers 2-7 in a tunnel, so really there is no need for new infrastructure. We just need a critical mass of network nodes.
You're misunderstanding the crux of the issue. Think of the NSA as a man in the middle. Every signal you send is allegedly stored for later processing. At that point, it doesn't matter how many layers you have encapsulating your data -- it can still be analyzed and decrypted given enough time and resources.
The crux of the issue is that somebody thinks the NSA is limited to using existing infrastructure. They aren't. You will be more successful in your aim by simply using the existing infrastructure in a way that is impractical to attack.
The most secure method of transmitting a message is a one-time pad. There is no way to analyze or decrypt it without its mated random pad. However, for internet traffic, this is horribly impractical. So you use alternate methods with some tradeoffs to achieve security that is practically impossible to attack.
The only benefit to alternate infrastructure is to work around government-imposed sanctions on traditional internet services, such as Iran, Egypt and other countries shutting off internet access. Project Byzantium is one way to work around this.
I agree with the crux of your message, but references to "one time pads" are an insidious red herring. They aren't actually secure against anything but the simplest passive adversary, and do little to further an understanding of the scope of modern cryptography (public key distribution, message integrity).
It's not a red herring. I was comparing the commenter's idea that "with enough time you can crack anything" to a system which cannot be cracked and how you wouldn't want to use it for this purpose anyway. I was not attempting to further the understanding of the capabilities of modern cryptography as I don't have all day to sit here and rehash whole textbooks on crypto.
Also, what active adversary are you implying would defeat the one-time pad? Lead pipe cryptography? That pretty much works for everything.
A one-time pad XORed with a message doesn't provide integrity, and therefore can't reliably secure an arbitrary protocol. An active attacker can easily flip any bit in the message. An easy example would be flipping a debug bit (presumably it started off 0) so that the server responds in plain text. We could blame this weakness on the protocol, but the point is that without message integrity we do not have a solid abstraction to build on top of. Referencing OTPs as unbreakable obscures this and several other problems that modern public key cryptography actually solves.
One-time pads (and all symmetric crypto) can never be user friendly - every pair of parties looking to communicate has to meet in person and exchange keys. Pop references to OTPs create false intuitions that real encryption takes significant work to exchange large eys, only secures preexisting links, and is only ever done by those with super serious secrets (spies, children in treehouses, spies, etc).
I also didn't mention the hardware random number generator required to use it properly. Getting a hardware RNG for each machine on the internet is probably more difficult than implementing authentication with OTP. But I did say it was a horrible idea for internet traffic, didn't I? =)
Give me a cryptosystem, and i'll give you a reason nobody should ever use it.
"Not to get all tin-foily, but I would assume that any undertaking that ambitious would have government agents working from the inside."
It's my suspicion that people will slowly stop saying "not to get tin-foily, but" as more and more is revealed re:NSA et al. Seems like a natural shift, given current and likely near-future circumstances. On to the point, the part I quoted is why I think revolts are bottom-up and revolutions are guided, even if the former progresses into the latter, at some point things get diverted. What makes all this scary, though, is the most powerful reason behind the existence of currency: the ability to create and (financially) back an army.
Meanwhile, the gen. pop. has lived their entire lives kept in a dialectic, watching shadows dance in a cave, and consuming bread and circuses. I'm not for anarchy, but it seems short of something akin to the final scene in Fight Club, there's not much to be done about it. Unfortunately, the following options are all that's left: accept the status quo, pretend/believe you can change it or move.
I like the idea of using raspberry pis + baloons + cellphones to create an alternative/parallel network. But you'd always face issues when trying to go international. The scenario that comes to my mind is someone (or a group) very rich and somewhat influential building a small scale worldwide private network completely segregated from the current Internet. Any attempt to install backdoors, "Y" devices etc would be detected and mitigated. Infiltrated "agents" would be tracked by their behavior and exposed to public attention.
But then comes the real reason for building such new network: communications, content sharing and services. The current availability of those 3 elements should be present. Are organizations willing to maintain two instances of their existing services, one in the current internet, another in the new one? Quite a challenge.
Maybe having the new network solely for secure P2P communication via voice, chat and email would be a good start.
There are some other methods, some listed in the OP:
- Commotion Wireless uses p2p wireless between mobile devices, which pretty much qualifies as a separate physical infrastructure
- If you're ok with the NSA knowing that you sent a message to someone on the network, as long as they don't know who received it, you can post encrypted messages to Usenet. Lots of people receive them, only one can decrypt. I think Bitmessage works this way too.
- Although not listed at OP, there's some academic work on scaling up Dining Cryptographer networks, which maintain anonymity even assuming an adversary who monitors the entire network. For example: https://www.usenix.org/conference/osdi12/strong-scalable-ano...
Also I don't think it's been established that the NSA is recording all internet transmission, though that's a good working assumption.
I doubt they're waiting to crack all that data. More likely they hold and if/when the owner is implicated in a crime, they will compel him to give up the password. Courts have compelled defendants to decrypt data seized in a legal search before. They could also maybe extract the password surreptitiously with more traditional investigation. Like looking over their shoulder when they type it.
If you want to transmit illegal data the best option is to take the route where the chance of intersection is minimal. And then Internet is not an option.
That's also why I think surveillance programs won't catch the real bad guys. Take a look at how long it took to get Osama.
What if the encrypted data streams that flow over decentralized p2p networks like RetroShare (http://retroshare.sourceforge.net/) would hold up and not be "decryptable" for the time we need to "undo" the mass-surveillance apparatus?
I mean, encryption could still be the one tool needed just to get activism going enough to beat the bastards (I mean, without being sabotaged, disinformed, killed, arrested, etc.).
Snowden's statement regarding encryption was: "Encryption works. Unfortunately, endpoint security is still weak."
Any cryptographic application could be vulnerable and not "hold up". It is wrong and dangerous to suggest that any program or tool is absolute. A court case in Germany not long ago involving RetroShare and its f2f nature did not end well[0].
> for the time we need to "undo" the mass-surveillance apparatus
You mean the apparatus that the vast majority of Americans are probably unaware of, and the vast majority of aware Americans are probably proponents of? What makes you think there will ever be an undoing on any reasonable time scale?
I think one problem with these initiatives is that money is usually not involved. When things like bandwidth are free, the demand tends to be higher than supply. Limited bandwidth then limits the possibilities.
I'm not proposing a system where you would pay to "TOR Ltd" for their services. Instead the "economy" should be network based, with no single entity controlling it. If I wanted to send stuff through the network I could say that I'm willing to pay x coins for this. The network would figure out cost efficient route and the participating nodes would get the payment. If I wanted to earn money (or credits), I could do so by putting my machine as part of the network and offering my network capacity to others.
If thinks would work out well, there would be people earning some money by running nodes on the network. Just like they are people investing in Bitcoin mining rigs.
At least right now, the problem is that the payments themselves might undermine what the system is set up to do, because the payments might be traceable.
There are proposals for truly anonymous and decentralized digital currencies (ZeroCoin comes to mind). However, they are not exactly practical because their overhead is quite high.
Something that seems to be missing from the current world: A P2P noise network.
A sufficient amount of encrypted noise emitting from many nodes makes the task of capturing and decoding everything quite a bit more challenging for an intrusive regime. Effectively, this is steganography: Everything should be encoded in cat pictures with an accompanying text of hot keywords.
Sorry, I don't see where they are generating random noise or keywords from your link. The link seems to be discussing the merits of a relay of existing traffic.
Ive been thinking about this and how to implement it, my conclusion was that NSA has the resources to filter out noise, especially seemingly random communication, as they already possess the social graph and have pretty good idea of who talks to whom and when. If suddenly a browser extension or app started sending "noise", to whom will it send it? A network? Where to get the list of IPs to spam? Just randomly? Just randomly is detectable! Thats easily filtered out - just exclude the network from the important data they tap. What if you randomly send a bunch of cat pictures and random noise to people of your contact list? This is called spam and receiving such, is not nice, as we can filter it on the receiving end, so can NSA filter it from their important calculations. But then you might think, lets send noise randomly and in the random noise let us communicate... meh, basically what a "P2P noise network" would do is increase the hassle and usability of computing systems for everyone involved but wouldnt really offer privacy for us.
When you kind of begin to make it less random, then you basically introduce plausible deniability and onions and begin solving problems that people are already working on and who need your help there more than "p2p noise network", then youre back to "lets make a good usable p2p anonymous/deniable communication system that can withstand NSA as advesary".
Still, it is an interesting idea, how to flood NSAs/FRA/GHCQs systems? Can we make a browser extension thats easy to use that would randomly GET pages from your history while you are actually away from computer - generating false content and increasing bandwidth usage, or a configurable spider to follow links/pages or even friends streams of choice? Would you use spammers techniques of markov chains to post random crap on open systems? Meh, thats still just an inconvenience for NSA.
What seems to be really important is who talks to who then how do you get in contact with strangers on the internet without NSA knowing? How would you know the stranger youre talking to isnt a cover agent?
As I understand, modern cryptography is distinguishable from "random". Onion and garlic routing provide more useful and productive ways of reaching the same goal.
Understood, I wrote random to emphasize no real communication is sent, just cat /dev/urandom and not in fact encrypted data. But of course, an adversary wouldnt be able to tell if it is something there or not.
I believe the time and resources required for us to develop such a tool and the time invested of users to run it is much more than the total inconvenience it would cause to surveillence systems.
But please prove me wrong, Id love to run such a tool quite easily, even if it doesnt work, just for shits and giggles and just perhaps for that extra cent of cost and resources wasted of powerful organizations. (still kind of our taxpayer cents but...)
Would you run a simple browser extension that would GET pages while you are away, pages that your browser has seen in your history, and an advanced mode, pages on pastebin and a selection of forums/sources? Meh, could be done with a simple bash script and curl I guess for the advanced users. Would be nice if it could also crawl most news sources.
Stepping the idea up a bit, random but seemingly valid User-Agents, exchange of tracking cookies with other users of the extension, and click on all the ads! Would you use such a tool?
It's pretty easy to add ignore flags to messages. With good encryption I think it could be hard to distinguish between real and false messages without the key , and that seems like something that can be proven mathematically.
In order to comply with NSA Directive [REDACTED] and FISA Court Order [Classified], anyone using these technologies will be required to install LiterallySpyingOnYou.exe or an equivalent binary for your OS of choice.
Failure to comply will result in <strikeout>your immediate offshore imprisonment</strikeout> a fair and impartial trial by your peers.
I've always been curious about I2P. I2P is more decentralized than TOR, as TOR has the weakness of needing exit nodes, and today it is very hard to set one up yourself (and not be arrested/shutdown). However, I don't know enough about network/protocol security to even be able to tell if I2P is legitimately secure, and I haven't seen any scientific analysis of it. Anyone else aware of such an analysis?
Well the reliance on exit nodes isn't really what separates Tor from I2P; the latter requires exit nodes for accessing publicly-facing servers as well. Neither requires them for accessing servers that are internal to their network. This is sometimes confused because Tor puts more emphasis on outproxying, and I2P on inproxying, but both networks are capable of both activities.
Haven't heard of Drogulus before, sounds interesting.
Anything that implements a Kademlia-like decentralized key-value datastore (basically, a DHT) gets my curiosity. Kademlia is a system/protocol how to implement a DHT and how to perform searches on it. The idea is that all network nodes are identified by a (traditionally, 160bit I think) hash. If you want to implement a P2P filesharing network based on Kademlia (like Gnutella), you hash each file to a bitstring of the same hash length.
Now, when you want to reach a particular node (and you have its hash but not its IP), you can do an efficient search by traversing the nodes in the direction of the target nodes. For direction, a distance metric is needed in this case. The distance between two given hashes is simply the result of [hash1] xor [hash2]. This is rather simple and ingenious: you get a value representing the amount of bits by which the two hashes differ. Hence you have a way to measure distances in the hashspace.
Therefore, when trying to locate a node, you don't need to do an exhaustive exponential "ping all my neighbours and get all their neighbours" search; one could say that you are able to follow a vector in the hashspace. edit: by which I simply mean, the most simplistic way to achieve this would be - pick a node from pool/neighbourhood with id/hash closest to target; get id/hash from its pool closest to target; continue until reaching a critical distance within which nodes will be responsible for holding the desired key->value pair. (I'm sure the actual algo will need to be somewhat more complex, as it's very easy to run into dead-ends this way, and so on.)
When there are files involved, nodes whose hashes are close enough to a given key (which is a hash of a file, more or less) are responsible for storing those key->value pairs ('value' differs on implementation, but basically it's another hash pointing to the node which actually has the file in question, if I'm not mixing things up; the whole thing is more complex, with a way to search using (hashed) keywords, etc.) Hence one can implement an efficient distributed search algorithm, where you focus on the hash-neighbourhood of a particular file hash, and get key->value responses from the nodes responsible.
This kind of a system has its disadvantages and possible points of attack. See the wiki article on Kademlia. [1] Probably the most well known type of attack is Sybil attack: [2]
In a Sybil attack the attacker subverts the reputation system of a peer-to-peer network by creating a large number of pseudonymous identities, using them to gain a disproportionately large influence. A reputation system's vulnerability to a Sybil attack depends on how cheaply identities can be generated, the degree to which the reputation system accepts inputs from entities that do not have a chain of trust linking them to a trusted entity, and whether the reputation system treats all entities identically.
Now, Drogulus seems to propose at least a partial way around this, though I suppose it'd still be possible to actually flood/populate the hashspace with garbage(?): [3]
The drogulus has three core components:
* A distributed hash table (DHT) that provides the data store and replaces DNS.
* Trust and identity enforced via cryptographic signing of digital assets.
* Logos [...] (say "log-oss"), a simple implementation of a Lisp like programming language for asynchronously working with data stored in the DHT.
The drogulus implements a version of the Kademlia distributed hash table. The innovation the drogulus brings is that keys and values (items) are signed in such a way that their provenance can be proven and content shown to be intact. Furthermore, users cannot interfere with each other's items stored within the distributed hash table unless they have access to the same private key. Items are self contained and any that do not pass the cryptographic checks are ignored and nodes on the network that attempt to propagate such values are punished by being blocked by their peers.
I don't have time to actually delve into this now, but the next paragraph piqued my curiosity even more - if anyone is into this, would be interesting to hear some comment :)
Logos programs are themselves values stored within the DHT, run in asynchronous "ensembles" that arrive at a consensus and process other items of data stored within the DHT. They are sandboxed and intentionally limited in terms of time (how long a computation may last) and space (how much memory may be used). Since Logos programs are also data there is a mind-bending side effect that Logos programs can rewrite other Logos programs in order to extend the Logos programming language itself. This is an important property: users have the autonomy to grow the Logos programming language to suite their own needs.
First things first: it's an unfinished work in progress and an experiment on which I've been hacking during my 40 minute train journey in to work. So, yes, "something something - Kademlia-like crypto-signed homoiconic datastore - something" just about sums it up.
I can't spend a lot of time answering right now, but I gave a short presentation about the drogulus at this year's Opentech in London which resulted in the following blog post (giving a high-level overview of my motivations and intentions): http://ntoll.org/article/ppdd I followed up with some clarifications in this post: http://ntoll.org/article/drogulus-questions-and-clarificatio...
I have several more blog posts in draft form about other aspects of the system.
It's very early days and I'm pretty sure there are lots of problems with what I'm doing. But, as I mentioned in the first post referenced above, I'm having too much fun to stop. ;-)
Happy to answer questions and constructive feedback, comment and critique is most welcome.
I've been working on a similar project for about a year now. Interested in exchanging notes? We are basing ours on a Chord DHT for better robustness.
benshoof@cs.gsu.edu
And that is my fault. I left out a letter. Now that you have displayed interest and I feel less awkward about directly approaching you I just went ahead and emailed you.
Instead of designing your own language Logos, have you considered using Clojure, which is also homoiconic, is a modern variant of lisp, and was/is designed by programmers who have industry experience, so it's not an academic experiment?
I did seriously look at Clojure but it doesn't quite fit my requirements. Also, I didn't say the drogulus was an academic experiment... that makes it sound far too impressive. ;-)
Could you please go into more detail on what were your requirements that Clojure did not fit?
What you have so far IS an impressive, audacious, idea. You can probably tell by the amount of attention you have gotten for it already. False humility is a kind of arrogance too, you know. Honest humility takes more work. Any time anyone starts talking about writing a new language, either they have not thought through all the challenges that would have to be met, like a boy trying to build a rocket to the moon using fireworks, or they are trying to do something impressive. Which are you?
Regarding Clojure: If the drogulus comes to anything then I'm building a working reference specification. I chose to write it in Python and comment ubiquitously to make sure the code is easy to re-implement in other languages or on other platforms. Adding a dependency on Clojure would make this less likely to happen; the drogulus would instantly be linked to the JVM (and other platforms upon which Clojure runs).
I've used Clojure before "for fun" and I like it - it's not a problem with the language per-se. Rather it's more a case of wanting Logos to be small, simple and "fit" the drogulus. Does this make sense?
I agree with you about developing new languages. That's why I chose to re-implement a Lisp: I don't want to invent a new language. However, I do want a language that meets my need for "fit", simplicity, portability and compactness (I'd like my kids to be able to understand and program in Logos). To use a musical analogy, Logos is a re-arrangement an old classic (Lisp) for different ensembles. ;-)
This is a work in progress and I need to push ideas in to code and get feedback as soon as possible. Only then will you be able to tell if I'm playing with fireworks or doing something impressive. Furthermore, while I have a very clear idea of what needs doing, I don't think this is a one-person job. That's why the biggest challenge for me right now is to transition the drogulus from a one person playtime project hacked together during a daily 40 minute commute in to London in to a functioning multi-person project with an active community of users.
Now, that would be impressive!
As always, I'm happy to answer questions and welcome constructive comment, critique and feedback.
Perhaps "Learning how to write a lisp-like language" is a requirement? Or perhaps it's low memory usage and fast start-up time for short-running tasks. Does it matter what he chooses for his own project?
I kinda like Clojure, but I don't think it would occur to me to default to it every time someone mentions the word 'lisp'. Not because it's not an adequate language, but there are downsides to using it for some jobs.
Oh wonderful, thanks for the reply! And thanks for the links (the "Politics, Programming, Data and the Drogulus" post had already been placed in my when-have-time-random-reading-list, cool.) I'll be sure to follow up with comments if I have any later on.
I assume that the crypto identity part of the system is indeed motivated by Sybil-like attack vector towards similar P2P systems, or something of the sorts.
(Warning: convoluted phrasing follows --) Logos programs and anonymity: I assume your system is not primarily designed to provide some level of anonymity (in the sense of message-originator decoupling). Some P2P systems try and implement a characteristic by which it is impossible to say whether a particular message seen in the network is originating from node X, or was simply broadcast by some node(s), one of which is X. Of course, if we have node identities and message signing, this is obvious, but - the idea could be to decouple actual pub-private-key identities from the nodes themselves (or perhaps implement an 'anonymous messaging' identity system atop the other, so there'd be identities for nodes, and for persons/people/virtual-identities, the two sets obviously overlapping but not homomorphic to each other (a node could broadcast/introduce a myriad of secondary identities, etc.)
Thing is, if Logos could be used to organically extend the system to implement such functionality, it would be rather impressive indeed. (By 'could' I do not suggest anyone doing it, merely observing my thought that the very possibility would be a curious fact about drogulus.)
/end of coffee-related rant, gtg, thanks for the reply again!
Yes, the crypto identity aspect goes some way to address Sybil like problems.
Regarding anonymity: you're only as anonymous as your association with your public key[s]. Given that items (k/v pairs) stored in the DHT are self-contained in terms of proving provenance it doesn't matter from what node they originate. Put plainly, there is no need for a node id (a SHA512) to be linked to a specific user (identified by their public key). I explain how signing items works here: http://ntoll.org/article/drogulus-questions-and-clarificatio...
For me, the most curious fact about the drogulus is that it is a ubiquitous ecosystem for asynchronously processing Logos code and data. If you see the implication of this then you'll get where I'm trying to go with the drogulus.
This really is a great project. Thanks for providing the links. It looks like you're thinking about the right things in building something like this. I'm excited to look at the code!
> The distance between two given hashes is simply the result of [hash1] xor [hash2].
This is the main innovation of Kademlia compared to other DHTs like Chord which uses a simple difference-modulo-N distance metric. Kademlia was the first DHT with a symmetric distance metric, which has the advantage that peer-to-peer connections can actually be used bidirectionally. That means that Kademlia networks need only half of the active connections to reach the same level of connectivity compared to earlier DHTs.
As far as I know, this is pretty much the only advantage Kademlia has over other DHTs, but of course it is quite a significant difference in practice.
Ah, you are quite right, I should have emphasized that point! (I guess I simply wanted to get to the drogulus part, but ended up expanding a bit.)
Indeed, the main paper introducing Kademlia is basically called, "P2P network based on the XOR metric." That is its distinguishing feature, but it is indeed enough to make all the difference.
Kademlia is also quite fun to implement! It's a great example of a distributed system, and me and a couple of friends actually made our own in golang (https://github.com/maxsnew/kademlia-go). We also decided to build a fault-tolerant, secure file store on top of it (https://github.com/jontonsoup/tin-foil-hat), as an educational experience.
> Therefore, when trying to locate a node, you don't need to do an exhaustive exponential "ping all my neighbours and get all their neighbours" search; one could say that you are able to follow a vector in the hashspace.
How are you able to do this without pinging all neighbors? How do you know you've found the node with closest distance? Or is there a defined XOR result for closest distance and you can stop searching once you've found it?
You keep track of a few folks at different distances from yourself, and route packets to the folks one knows which are closest to the ultimate destination (or ask them whom they know closest to that destination).
So rather than asking everyone you know, who ask everyone they know, who ask everyone they know, you're asking a subset, who ask a subset, who ask a subset.
The latter, as far as I take it. That is, each given Kademlia-based system defines a 'close-enough' / neighbourhood constant. If a given result from xor is <= that constant, you have reached your destination. Because obviously, you won't have node IDs actually matching your target (e.g. file) hash (assuming no hash collisions.) So you need a 'proximity' value.
Granted, the whole picture is more complex, and my knowledge is very limited. A node that wants to search for something sends its search query to the neighbour closest to target, which then, as I understand, propagates / rebroadcasts that query to its neighbour which itself is closest to destination. There seems to be some redundancy involved / multiple paths taken, perhaps to ensure that all nodes within that proximity distance are found. (So for example there could be multiple nodes storing the same key (say, file hash), but with different values (different node IDs (for nodes that actually have the file in question.))) I'm being vague though, that's very true..
But as I understand it, as long as the whole graph of nodes is connected, with no orphan sub-graphs forming / detaching (something which may not be trivial to accomplish/ensure), you should be able to get all relevant key->value pairs / reach all relevant nodes in your search. The way the search is concluded in Gnutella (I think) is that each target node responds by connecting to the source/query-issuing node directly (every query carries with it source UDP IP:port (I think)).
edit / P.S.: also, it may be that a given node stores node IDs in its proximity range (the 'close enough' value), but I'm not sure of this; I'm adding this because it may actually help ensure that all nodes / key->value pairs of interest are accounted for, i.e. it's a worthwhile idea anyway.
Tonika looks really interesting and relevant to multiple interests of mine - it may be something I'd been looking for some time, actually. Will read up on it (I've seen the particular arXiv paper Tonika design rests on before, but didn't look into it; hmhm.) Thanks for the link!
edit interests / interesting features = message passing, privacy, deniability, etc. Good stuff.
Interesting indeed. I came to the same conclusion for my pet project (as of yet unpublished), a distributed content addressable data store is only useful if there are attack resistant trust metrics included.
Makes sense to have good decentralized tools but what about endpoint security ? We have to suppose that average OS does have many 0day vulnerabilities so it's easy to get all important data from your computer.
To have good firewall & IDS, automatic analysis, secured kernel - that's was should concern you people.
And that's not all - we cannot be sure if hardware itself does not send any unique IDs to vendors... or network card can mark some packet by unique signature and agency can capture those packet as they pass through internet. I remember i read that few years ago : http://www.slideshare.net/the_netlocksmith/defcon-2012-hardw... I would bet there is many backdoors hidden in hardware.
This was discussed within the IETF and slowly moving forward. Key point is: too many fragmented single-person projects exist. No structured approach with documentation, unit testing and related work tracking.
I discovered it a few days ago and it sounds promising but some of the cjdns whitepaper was over my head. Apparently there are a few hundred people participating in their alternative internet on a daily basis already.
Oooohh!!
Hey I will have a look at some of these myself.
But what would some of ye with experience recommend for... irc/sharing some files with a network of friends only?
I'd like to be able to have private chats with only one or multiple people aswell. cheers!
Anybody who hasn't taken a look at Namecoin in the last month or so should take another peek.
A GUI client with name registration platform came out recently and it's quite the change from the command-line-only options we have had available for the last few years.
A true "alternative internet" would, at this point, require its own infrastructure due to the routers and switches of the Internet using the TCP/IP stack. (assuming that an alternative internet would be a variation on the IP protocol).
The entire issue with the government surveillance of today is that they're (allegedly) storing all data, even the encrypted data, and holding it to either brute-force crack, waiting until whichever cipher/protocol gets broken, or waiting until computing power is abundant enough to crack the encryption-du-jour in a reasonable amount of time.
This is why ideas like Raspberry Pis on balloons have been proposed as true "alternative" internets. Not to get all tin-foily, but I would assume that any undertaking that ambitious would have government agents working from the inside.
If you're transmitting any data that could be illegal, your safest bet is to use some kind of encryption that would take longer than the statute of limitations for whichever crime you're committing to break (you'd probably also have to factor Moore's law into your calculation and pray that quantum computing doesn't become viable in the next x years).