
LF: Fully Decentralized Fully Replicated Key/Value Store - api
https://github.com/zerotier/lf
======
mattupstate
ZeroTier tech is so promising, but it appears Adam is trying to do too much at
once. For example, I never received the ZeroTier Edge box, or a refund, from
the IndieGoGo campaign. It's frustrating, but I hope the ZeroTier succeeds in
the long run.

~~~
jbverschoor
Yup.. no focus. The product is really good. Could use a better interface.

But he should be working on getting it into people's hands, instead of
creating new tech.

Seeing this leads me to believe that zerotier wants to do this whole global
network thing. This is not what I want from zt.

This probably came from questions like "how can we trust you as a network
controller". And the answer is to create some tech. But in that case, why not
simply offer an AMI on aws for a controller?

All assumptions of course. Upon further checking:

From
[https://news.ycombinator.com/item?id=20329060](https://news.ycombinator.com/item?id=20329060)
and [https://www.zerotier.com/lf-announcement/](https://www.zerotier.com/lf-
announcement/) :

> We are going to use it to fully decentralize ZeroTier and for some specific
> enterprise customer projects, but others can use it for other stuff.

> Our interest in decentralizing the roots wasn’t just philosophical. Over the
> years we had numerous requests by policy-bound or very (justifiably)
> paranoid customers to run their own infrastructure that could be logically
> separated from ours as much as possible

I dunno. I'll have to see what it turns into. But it surely explains the
silence and lack of updates on zt/zt-one

~~~
api
LF is part of a return to focus: global scale network coordination and
decentralization. Making a hardware box was the big defocusing project and
took more time (with less results) than this. Lesson learned: avoid hardware
if you are not a hardware company in your DNA, and hardware will always take
10-20X longer to ship than to prototype.

So yeah, defocus is a legit criticism but not because of this project.

This was also a two birds with one stone project in that it is also useful on
networks with arbitrary connectivity graphs and unreliable connectivity. An
enterprise customer of ours funded some of this R&D for that purpose and we
combined their objectives with our root server decentralization and network
robustness improvement goals. I wish I could talk about the enterprise
customer's project. It's quite cool but we're under NDA. All I can say is that
it's industrial and involves mobile ad-hoc networks.

There is admittedly quite a lot behind this that is not apparent from the
README, and from the comments here its clear that people have trouble
understanding this because it is not yet another cryptocurrency and isn't
solving exactly the same problems. (There is some overlap but this is
different.) We built this to solve a problem nothing else solves, but I figure
it will take the appearance of use cases for people to get what that problem
is. The people bringing up etcd and consul are closer than the people bringing
up coins.

~~~
rootless
> We built this to solve a problem nothing else solves, but I figure it will
> take the appearance of use cases for people to get what that problem is.

Beautiful solution. Adds so much to what ZT can provide for borderless
networking. First thing I'd implement is distributed DNS for usability. Then
probably the config backend for the network layer.

------
DINKDINK
>Proof of work is not foolproof, especially in LF where there is no intrinsic
economic mechanism to incentivize the runaway growth of PoW "mining"
investment. As a result a well financed or determined attacker willing to
throw a lot of compute power at the problem could overcome the PoW "weight" of
the records beneath a target record in the LF DAG and replace it.

This issue -- of creating equivalent proofs of work -- is what I was most
interested to see if there was any novel art (which it appears there isn't).

Bitcoin's difficulty adjustment is the only well-functioning algo that deals
with the economic coordination issue around how to -- without any third party
coordinator -- equate the value of a proof of work made by computer's of
differing computational capacities (a phone, an ASIC). All PoW consumed by
Bitcoin has marginally positive contribution to the system. Sacrificial PoW
(as in LF, hashcash) only has value if reorgs happen. It also results in a
system with tenuous game stability properties ("When will it be uneconomical
for an attacker to reorged these records out of the DaG?" "Can't say, blocks'
PoW requirement is arbitrary") vs bitcoin ("When will this 50 BTC txn be
uneconomical for the most adversarial miner to reorg out?" "well there's about
14 BTC of block reward per block so about 4 blocks ~40-60 minutes"

~~~
synctext
Fascinating project. It is the first time I've seen proof-of-work combined
with DAG and real world engineering details such as PulseToken, oracle nodes,
varient on S/KEY one-time password scheme, etc.

> adamierymenko 478 commits 403,913 ++ 148,611 --

Impressive productivity.. Worried about the Sybil attack. The documentation
uses the "foolproof" wording, but the overlay might not be attack-resilient
and creating a million fake identities would be sufficient to dominate the
system. Even a loose connection to the main DAG might be sufficient to hurt
LF.

> Oracle nodes document low reputation records in commentary records. When a
> node sees a commentary record it extracts this information and caches it
> internally in an indexed table.

That could be an attack vector. Broadcasting negative information can often be
exploited in my experience. Slandering attack with a 100 Mbps VPS might
cripple the network. Could this be altered into positive re-enforcement loop?

> Casual users can choose to trust nodes that are generally trusted.

How will central server "lf.zerotier.com" handle copyright claims by Hollywood
(people will try to upload also large files)? (disclaimer, creator of Tribler,
we're designing systems like this within our university lab for 14 years and 8
months)

~~~
hoseja
> MaxFileSize: This limits the maximum size of locally written files. Files
> larger than this can appear if someone else wrote them. The hard global
> maximum is 4mb. Note that large files can take a very long time to commit
> due to proof of work on public networks!

~~~
synctext
Makes sense. Is there a measure against spamming the network with 1 billion
files of 1 Byte each? (besides PoW)

------
conradludgate
"Doing proof of work for all those is possible but costly, so we stuffed a
certificate into Sol's configuration to let us cheat and be special. Think of
it as our "fee" for creating and maintaining LF and donating it as open source
to the community."

Not sure how I feel about this, and I'm sure someone's going to figure out how
to remove that 'feature' in a fork... Does kinda seem against the nature or
p2p and blockchain

~~~
api
You can create your own database/network easily. There is even a command for
it. This is just the default.

------
eigenvalue
These CS sayings (linked to on the github page for LF) gave me a chuckle:

From
[https://www.martinfowler.com/bliki/TwoHardThings.html](https://www.martinfowler.com/bliki/TwoHardThings.html)
:

"There are only two hard things in Computer Science: cache invalidation and
naming things.

\-- Phil Karlton

Long a favorite saying of mine, one for which I couldn't find a satisfactory
URL.

Like many good phrase, it's had a host of riffs on it. A couple of them I feel
are worth adding to the page

There are 2 hard problems in computer science: cache invalidation, naming
things, and off-by-1 errors. -- Leon Bambrick

There are only two hard problems in distributed systems: 2. Exactly-once
delivery 1. Guaranteed order of messages 2. Exactly-once delivery -- Mathias
Verraes

there's two hard problems in computer science: we only have one joke and it's
not funny. -- Phillip Scott Bowden"

------
viraptor
I'm having an issue seeing why would I use this. I mean, given the high
resource and bandwidth usage, why would I opt for this rather than a
zerotier'd network of consul/etcd/zookeeper/something distributed on a number
of tiny instances between many regions? It also looks like it's write-once
which makes usage non trivial. Also it gives you the usual blockchain fun of:
now you're hosting whatever you're given, have fun defending yourself if you
ever get sued.

So... What's the realistic use case for this? (In both public and private
network variation) Zerotier's network seems like one of the very few projects
where this makes sense.

(From the tech point of view, this looks really cool though!)

~~~
api
This is for systems that cross trust boundaries. Etcd or consul would be a
better fit for systems controlled by one entity. The only exception might be
where the network is unreliable or transient.

Think of it (as the readme says) as consul for decentralized open systems.

~~~
joatmon-snoo
What's the mitigation for OOMs triggered by someone dumping records into the
system?

Not someone trying to overwrite existing records, just constantly inserting
new records.

~~~
api
This is mitigated via a proof of work "hashcash" mechanism, or for signed
owners the ability of the CA to revoke the cert.

BTW "high resource use" is in reference to things like tiny embedded devices.
You won't be putting this in a fridge magnet or a smart light bulb. This will
run fine on a $10/month VPS or a Raspberry Pi 3 or 4, though if the data set
gets huge you might need to attach some storage.

------
viach
Good marketing move - building a distributed ledger not on blockchain and
without tokens, positioning it as a key-value store. Kudos.

~~~
api
It wasn't built to be a cryptocurrency but to serve as a data store for a
decentralized network backplane. You could theoretically build a token on top
of it I suppose.

~~~
viach
We built a similar thing using Tendermint. Not open source yet but already
have several clients. Good to see other people appreciate similar move - a
tokenless DLT.

------
e12e
"Unlike web SSL servers there is no need to send a certificate back to the
owner. LF is a global shared data store and therefore makes use of itself to
publish certificates."

Nice! Thinking a bit about this - this sounds like it has real potential vis-
a-vis traditional CA. With a (many) globally trusted db - it sounds quite
feasible for eg Mozilla or Debian to simply store (or "bless"/trust) any
number of root certs - and record that in lf.

And the same could be done for signing binaries (whether via x509 or some
other scheme - "all" that is needed is some way to "discover" trust of the
signing key after all - be that a gnupg key or a x509 cert etc).

------
marknadal
After just giving scathing remarks to another database system, I'm pleasantly
surprised by LF!

LF looks like a well-reasoned project with a solid direction that is unique
and applicable.

I do disagree with some of its design choices (DAGs aren't very scalable, full
replication versus partial/sharded, etc., and GPL instead of MIT - which my
decentralized DB, [http://gun.eco](http://gun.eco) is MIT/Apache2), but for
the problem it is trying to solve, it seem like you've made the right choices
for LF and its use cases.

Congrats and keep it up! Excited to hear about this over time. :)

~~~
tasubotadas
I think that as it currently stands, the GPL3 licence makes the project
unusable.

There aren't any externally packaged drivers with a different licence, and you
can't include their native code in your commercial projects without having to
share the code due to the inclusion of the GPL3 licenced code.

~~~
api
We are considering a license more like that adopted by CockroachDB for all our
projects in the future. Haven't decided on the particulars yet.

------
panarky
Could LF be used to replace the SKS keyserver network that's hopelessly
borked?

[https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d695...](https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f)

~~~
api
Yes

------
winrid
Briefly reading the code, it seems like a distributed wrapper around sqlite.

~~~
misterdata
See
[https://github.com/pixelspark/catena](https://github.com/pixelspark/catena)
for a SQLite-on-blockchain implementation

~~~
api
We found stuff like this while researching before creating LF. I remember
finding this exact project and a few others like it. This is in the ballpark
but has numerous issues in a federated or fully decentralized use case, such
as who sets database schema and what happens when you need to change it. It
also lacks any privacy protections. All keys and values are visible to
everyone all the time. That opens up a ton of attack surface that has to be
specifically mitigated on a case-by-case basis. Better to have data be blind.

LF isn't a wrapper around SQLite at all. It just uses SQLite for meta-data and
index information because it happens to be a very capable database and because
the ability to do relational queries saves a ton of time vs. hand-rolling such
queries on top of something like LevelDB.

The last issue is something I've never understood about cryptocurrency code
bases. Why do they insist on implementing a slow ad-hoc relational model on
top of a simple KV store when they could just use SQLite? I assume it's
because they assumed SQLite would be slow, but it's really not unless you are
doing _massive_ numbers of writes.

------
pcr910303
Wow, I was waiting for a global ledger/consensus system based on DAGs/non-
blockchains forever... Blockchains have an inevitable scaling problem. I was
only aware of iota & Radixdlt but I’m pleasantly surprised that there’s lf
now! Kudos for the great work!

------
beamatronic
In your readme, could you add a mention of memcached and how you differ?

~~~
thdxr
This is decentralized as in peer-to-peer, not in the same category as
memcached

------
mirekrusin
It would be great if the author (or some good soul) explicitly stated that
this project doesn't violate any hashgraph patents as at first sight it's not
clear.

~~~
sschueller
Wouldn't that fall under software patent which is not something that can be
patented Europe?

~~~
mirekrusin
I don't know, it's one of those things that you read more and know less, ie.
first google result for "software patents in europe" says "yes, but no":

>> "The European Patent Convention states that software is not patentable. But
laws are always interpreted by courts, and in this case interpretations of the
law differ. So the European Patents Office (EPO) grants software patents by
declaring them as "computer implemented inventions"."

...but it's interesting, ie. what would it mean exactly if bitcoin was built
on something patented? It's completely distributed so nobody to sue. Would the
repo be banned on github? Let's say it would and the development would move to
linux kernel like development, then what? US based exchanges would probably
have some premium fee and that's it?

------
godzillabrennus
Distributed computing technologies are fascinating and evolving quickly.

Caught a demo of a blockchain product performing at over 6,000 tps on a single
shard spread over about a dozen geographic regions on a test net last week.
Expected to be more performant than anything out there today when at scale.

~~~
RL_Quine
Whenever "blockchain" is mentioned there's always exactly these sort of
comments popping up. They are always demos because verifying even 6000
signatures in a second on a normal CPU is not a walk in the park, let alone
any other computation, network latency. The concept is as implausible as its
real world applicability.

    
    
        # openssl speed ecdsa
        Doing 256 bits sign ecdsa's for 10s: 494208 256 bits ECDSA signs in 10.00s 
        Doing 256 bits verify ecdsa's for 10s: 161277 256 bits ECDSA verify in 10.00s

~~~
rolltiide
I think it is more interesting that enthusiasts still fall for the "testnet
tps" sales pitch

