A more popular and, I think, carefully analyzed alternative, from which croc was inspired, is Magic Wormhole; a good Golang implementation that compiles down to a single binary is wormhole-william:
I believe croc has some features wormhole doesn't (and some anti-features, like being able to pick curves and hashes). But also just that it's worth knowing that Magic Wormhole is kind of the "default" tool that does this.
I think the main advantage that I like croc over many utilities is that it can resume file transfers from where it left off. I'm not sure if wormhole has this yet. Of course torrents do this, but I find sharing via torrenting still not super easy for the layperson.
Torrents also don't provide encryption between peers, unless you use SSL torrents, but it's clearly designed towards sharing between a group, not one-on-one transfers.
It's just a bit more involved to use for this use case, especially if the data is confidential (having to set the private flag so clients don't publish the infohash to DHT, using a built-in tracker in your client, transport layer encryption not provided by default). Things like Resilio are fine, they build on top of the protocol and provide a different UX.
It's weird croc doesn't mention magic wormhole in their readme, if only to explain the differences with this well known tool, and why one would use croc rather than wormhole, which people are probably more used to using and that they can apt install, etc
"magic-wormhole has most everything (currently its missing capabilities for multiple file transfers and file resuming), but it requires installing lots of the Python ecosystem which is tricky for non-developers (and Windows users)."
For the python part, I guess wormhole-williams works too then.
A wormhole user would just transfer a directory, rather than a globbed list of files, but croc's multiple file transfer thing is a difference between the two tools.
I think Wormhole William is great, also has a mobile app. But when I discovered croc, I switched to that, it has been very reliable. From the developer:
AFAIK, croc is the only CLI file-transfer tool does all of the following:
- allows any two computers to transfer data (using a relay)
I’m probably missing something, new to magic wormhole, but this seems to contradict the docs on magic wormholes linked by the tool you recommended, wormhole william:
“ The wormhole library requires a “Mailbox Server” (also known as the “Rendezvous Server”): a simple WebSocket-based relay that delivers messages from one client to another. This allows the wormhole codes to omit IP addresses and port numbers.“
Is this a particular library choice vs protocol choice? Wormhole william perhaps allows codes with ip addresses etc?
What makes me wary about these tools is the way many of them seem to bake in a default relay server but aren’t up front about it (i want that discussed in the readme.md, it’s a big deal, even if they can’t read the payload - presumably they can see which two IPs are communicating which should be absolutely disclosed). Like, to me it’s not “magic” to route stuff through a relay server. That’s the whole thing I want “magic” to avoid. (Also it doesn’t seem at all like a “wormhole” - the nomenclature strikes me as insanely grandiose)
That's how croc works, too. You can run your own Magic Wormhole relay. These tools all have the same fundamental design.
What's magic-seeming about them is that you can be deep in a prod network, with tightly controlled ACLs and no routable address, and just "wormhole server.log", and then on your dev laptop on your random home wireless type, like, "wormhole receive 32-hazardous-baboon" and poof! you have "server.log". It feels pretty magical, which is why it has the name.
Croc doesn't change any of that. What croc does differently from Wormhole is that it handles resumption, and you can apparently send a globbed list of files rather than just a file or a directory. Resumption is a real feature (I'd like to know more about how the cryptography works, though). The multi-file thing is a UX tweak.
Croc has a bunch of knobs to change cryptography primitives. Those are anti-features.
You can password-authenticate a custom croc relay. I don't know why you'd ever care. Maybe this is an abuse concern? These are all E2EE designs. You don't trust the server in the first place. It's like a STUN/TURN server.
I don't mean that they invented PAKE's, I mean that they invented this application of PAKEs (I'd say: one of the very few genuinely useful PAKE use cases).
An even more popular alternative is Resilio Sync. I still use it because somehow sharing with the tech savvy isn't a big problem, but with the unsavvy is. A GUI and a single string is on the edge of the possible, and I share much more (large, noncode, files) with this group.
It makes sense to compare croc to wormhole because they're the same thing: PAKE-backed file transfer, where nobody has accounts, no central server holds the files, everything works from behind a NAT, all you have to is install the tool and give your counterparty an autogenerated secret password for the file.
This looks very different; it requires having a server to temporarily store the file, while croc and magic-wormhole directly connect two peers (via a relayer) to send the file.
Also if GitHub stars are anything to go off of, croc is several orders of magnitude more popular. Also croc is written in Go and compiles to a single binary, but there are some helper scripts in the repo for a few things. I really don't know why they said all that in their comment.
Wormhole-william is a Go port of Magic Wormhole. Github stars are not anything to go off of. If you're unaware of Magic Wormhole but have strong opinions about croc, we're not speaking the same language and there's little point in us irritating ourselves by trying fruitlessly to hash things out.
Did you mean to reply to me? I didn't say anything about being more popular -> being more secure. If you did want my input, my answer is: no. I was just commenting on the fact that the commenter said croc was less popular, when I cannot find any metric for that.
About relying on the developer to provide the relay server (this was early 2022):
The cost of croc is mostly bandwidth - over 8 terabytes of data is sent every month through croc! That's amazing to me since I started this project just as a way for me to share files with friends. Four years ago, the public relay server only costed $5/month, but now it is costing me $40-50/month. The higher cost is enabling file transfers for thousands of people all around the world.
Thanks! The usage has only gone up - now in 2023 the croc public relay has been using 20 TB of bandwidth every month :| I like to write software that runs forever, but this will get difficult to run in the future if usage keeps climbing.
Sometimes. The VoIP world has been dealing with this for years. The process of finding and connecting the clients is called signaling, and that typically requires a shared relay. The signaling can contain information on how the clients can be reached directly. In SIP, typically each client will use a STUN server to discover their public IPs, and a process call ICE allows the clients to iteratively try those candidates until they establish a connection. Based on the NAT settings, it may not be possible, so clients will also include the IP of a relay TURN server.
The challenges with Croc here are largely the same, but with data as the media instead of voice or video. (Although, “VoIP” can also handle data in this way. See WebRTC data channels.)
This blog post[1] may interest you. As you suggested, the workflow seems to be:
1. Try various techniques that might trick the firewalls on both ends to let the connection through. This requires a relay for the initial negotiation only.
It depends on the topology of the clients network, if they are both behind a specific kind of firewall (don't remember the details) no connection can be established from outside
When connection is possible, is my understanding that croc connects directly
Which public server does it use by default? I'm surprised the docs don't mention it.
More generally, it feels like in 2023 that connecting two computers via a public relay ought to be a solved problem, on the level of, like, DNS. Or maybe it is and I haven't heard?
The default relay can be found in constants.go [1]:
// DEFAULT_RELAY is the default relay used (can be set using --relay)
var (
DEFAULT_RELAY = "croc.schollz.com"
DEFAULT_RELAY6 = "croc6.schollz.com"
DEFAULT_PORT = "9009"
DEFAULT_PASSPHRASE = "pass123"
INTERNAL_DNS = false
)
Well static IPs solve the problem if everyone has them. But what about a 'standard relay' system that could, like, obsolete port forwarding by giving any two machines a way to negotiate a connection?
I'm surprised it's not listed on the README directly but in case you looked for it like me :) here is what their site says about the alternative you're probably thinking of:
"toss cleverly encodes port information in the code phrase, making it simple but it requires using connected computers (no firewalls) and the long random-ish code phrase is hard to “tell” someone. magic-wormhole has most everything (currently its missing capabilities for multiple file transfers and file resuming), but it requires installing lots of the python ecosystem which is tricky for non-developers (and windows users)."
I appreciated the links to the audit, but your quote was misleading to me when taken out of context like you did. I interpreted it as basically saying that the author couldn't or wouldn't address the issues identified. The full quote was:
> The upstream
author doesn't have enough resources to address them on its own and wants to
develop fixes in the open. Therefore I have created GitHub issues in the
upstream project and publish the full report today.
I.e. the "and wants to
develop fixes in the open" part left me with a very different interpretation from when I first read your comment.
These issues are pretty recent. I would greatly appreciate sponsorship to address issues faster: https://github.com/sponsors/schollz or just help with PRs.
Just wanted to say that Croc is one of the most reliable and straightforward file transfer tools I’ve ever used. It worked so well that I used it for Android (via Termux) to Windows transfers regularly. I only wish there was a way to use it on iOS but I imagine that’s challenging.
Thanks for the kindness :) I use it the same way actually! I don't use any Apple products so that's the major roadblock for me to develop against iOS...
Yes, I subscribe to Daniel Stenberg's RSS feed and have seen his many articles bemoaning excessive classification of bugs as vulnerabilities. One of these bugs, however, show serious cryptographic deficiencies. Unfortunately there are a lot of cryptography amateurs making stuff without a proper understanding ond making grandiose claims, so my default stance is one of skepticism unless reputable cryptographers have looked at it.
I use wormhole-william, the Go version of the Python magic wormhole, and age, mostly because of this Latacora endorsement:
Haven't tried this but I recently started using syncthing.net and I highly recommend it.
Like rsync, it only sends the chunks of the file that changed, so it can be extremely fast for small changes in large files. It's able to use a variety of methods to connect including good through relay servers if the machines can't directly talk to each other.
SyncThing is fantastic, but a different purpose. It's great for syncing files between persistent computer relationships, magic wormhole (or croc) is better at ad-hoc sending of random files.
I also use syncthing. It's been working so well that it's magical for sending files and keeping a passwork manager up-to-date between devices. My next step for my homelab, is to run my own announce server and relay on my local network.
I agree that a secure way to transfer files between systems is a must. If two systems are at the same location, a small USB card reader (nowadays easily OTG capable) and an SD card allow me to transfer data.
For systems connected only by a network: maybe I don't see the advantages of these tools (which I have to install first on any client) as all systems I use already provide ssh and rsync to do secure file transfers. Even my Android devices are capable of using ssh/sshd and rsync via the Termux app, so to transfer files between them and a Linux or Mac laptop is easy.
OK, one user has to be proficient in enabling ssh/sshd if needed, but to install software such as the one discussed here, that's no difference.
It would be even better if you didn't have to install any client software at all on the downloading devices, exchange ssh keys, or pass around a password to an account on your system in order to transfer files around your local network.
I too use rsync over SSH for devices I own, but I would avoid using SSH to transfer files with someone else. As the receiving side you either allow them way more access to your system than necessary or do some extra configuration to limit their access.
It's much easier to install a single static binary than to correctly set up and configure ssh in a safe way. If you need "one and done" file transfers without worrying about open port 22, key exchange, etc, this (or tools like it) is great.
I’m currently working on a new version of pcp [0]. Based on croc, magic-wormhole and the likes it doesn’t require a relay. It uses the IPFS DHT as a discovery point to connect two machines. Haven’t touched the currently released code in two years but the new hole punching capabilities of libp2p show promising result so I’m working on a new version.
Here is my feedback about the pcp's main GitHub page, if you don't mind:
- Move "Motivation", "Project Status", "How Does It Work?" somewhere else
- Add "What is It?" and "Encryption" sections and place them just before the "Usage" section. Keep them reasonable small, the smaller the better
I believe that it will help the project to have a much better resonance with the target audience.
Does this run a mini IPFS node on the sender and relayer and rely instead on the IPFS network being up to propagate the DHT changes through the network?
It’s just running a very lightweight libp2p host that speaks the DHT protocol. It’s basically just taking the bare minimum part of Kubo (IPFS) that’s necessary to interact with the DHT.
It’s then relying on the rest of the IPFS network to propagate the record for discovering the sender and receiver.
I like croc, wormhole, snapdrop and whatever else there is I just don't get that there is no universal convenient standard for transferring files from one device to another over a wired/wireless connection. Mindboggling that most people use cloud hosting or email to transfer a file from one device to another when they are just 30cm from each other.
Used to be "put it on a USB stick". Now no-one carries those so email/google drive it is :-)
When sharing files between devices I own (including my mobile phones) I attach them to a draft email to myself, then get them from the drafts on the other device.
The problem with email attachments is that it only supports files up to around 25MB, and if you have a slow uplink to the cloud, it could take a while.
A friend wrote a faster and more sophisticated version of `python -m http.server 8000` in rust which also supports uploads, you can read about here: https://news.ycombinator.com/item?id=37628347
File transfers are over the local network, and therefore should be very fast.
to share a directory over a local network. Then you can point a web browser on another device to port 8000 of the server's IP address to download files. However, this can be slow for larger files.
A friend wrote a faster and more sophisticated version of `python -m http.server 8000` in rust which also supports uploads, you can read about here: https://news.ycombinator.com/item?id=37628347
I am having a hard time understanding. Does this allow you to send / receive anywhere? Across the internet I mean. what external server does it use for 'stapling' the connection or is it also expected that you self host a relay server + set up the necessary ports etc. Do both ends require a relay server?
>I am having a hard time understanding. Does this allow you to send / receive anywhere? Across the internet I mean. what external server does it use for 'stapling' the connection or is it also expected that you self host a relay server + set up the necessary ports etc. Do both ends require a relay server?
Sorry, I have spoke too soon. I see now it uses a public relay by default, and you can self host as needed. Sorry for the unnecessary comment!
I don't see KDE Connect mentioned so I'd like to just say it's great. Send files and clipboard between PC/Mac/Linux/Android/iOS. And it's a native QT app so pretty light. Only works on same network though.
Unfortunately didn't work well for me using the gnome libraries on linux or mobian/phosh. (If you don't want to install KDE just for file transfer, etc.) scp on the other hand is bulletproof when hosts are directly accessible.
Most people here are referring to transfers of their own data. How do people transfer their confidential information to say, their solicitors, who may be on different operating systems and behind NAT / firewalls?
Specifically if cloud services are banned by the solicitor's company.
You'd have thought by 2023 this would have been solved but I don't believe it has. Particularly as PGP email administration (including key management over time / replacement hardware) is non trivial and unlikely to be successfully achieved by the non tech savvy.
Most people here are referring to transfers of their own data. How do people transfer their confidential information to say, their solicitors, who may be on different operating systems and behind NAT / firewalls?
Specifically if cloud services are banned by the solicitor's company.
You'd have thought by 2023 this would have been solved but I don't believe it has. Particularly as PGP email administration (including key management over time / replacement hardware) is non trivial and unlikely to be successfully achieved by the non tech savvy.
Most people here are referring to transfers of their own data.
How do people transfer their confidential information to say, their solicitors, who may be on different operating systems and behind NAT / firewalls?
Specifically if cloud services are banned by the solicitor's company.
You'd have thought by 2023 this would have been solved but I don't believe it has. Particularly as PGP email administration (including key management over time / replacement hardware) is non trivial and unlikely to be successfully achieved by the non tech savvy.
Thanks for mentioning this but would be more useful to provide some context here.
This works via WebRTC, and since the wtrc package for node.js is not well maintained, the CLI for drop.lol has been abandoned. Therefore croc solves a huge problem that I can't solve in drop.lol right now.
For anyone who needs a web application, this should work though.
(Although: I've stumbled upon a really good implementation of WebRTC in pure Rust, so I might end up trying to get it working with that or even trying to port that to node.js somehow)
Most commentards here are referring to transfers of their own data.
How do people transfer their confidential information to say, their solicitors, who may be on different operating systems and behind NAT / firewalls?
Specifically if cloud services are banned by the solicitor's company.
You'd have thought by 2023 this would have been solved but I don't believe it has. Particularly as PGP email administration (including key management over time / replacement hardware) is non trivial and unlikely to be successfully achieved by the non tech savvy.
Can someone EIL5 this for me. I don't understand how this works. If I have my computer on my home network and the person I was to share a file with is on their home network, how does this work out where it's sending the file/folder? The examples seem to suggest it magically knows based just on a passphrase. I have to be missing something. I'm aware this is a stupid question but I can't, based on what I've read so far, work out how this transfers stuff without more details.
The application has a default parameter that points to a relay server, when you want to share something, you tell to the server "hey I want to connect" and the server gives you back a passphrase, now the receiver connects to the same server and has to use the same passphrase to get the files you are sharing
Then the server tries to make your and the receiver clients connect directly to each other, if this doesn't work (mostly because of firewalls) it also Relais the parts of the file, which by the way are encrypted so that only your receiver client can decrypt
Tried it with a friend a few weeks ago and unfortunately it failed. It first began transferring the file, then for seemingly no reason I stopped receiving and then after waiting a bit got EOF and the file was not fully transferred. Seems like it is not that reliable in transferring files, or it has requirements, like open ports or so. But then why would it start and then stop trnasferring?
I have used it many times in different scenarios and networks, including firewall and behind NAT. I never had to do anything manually except copying the passphrase.
I wish there was a gui version for croc. I've used it a couple of times to send files to non-techie friends. While croc worked great, having to spend 15' on the phone guiding them how to run the command felt like an unnecessary hassle.
But here you download the source code every time you use it. It's not end-to-end anymore, as the person who designed the website can deliver malicious code, unless you pin the JS somehow.
I wish magicwormhole or croc came with every major OS preinstalled.
The main thing I use for ad hoc file transfers is Total Commander for Android (https://www.ghisler.com/android.htm). It can copy files to and from a WebDAV inbox on the local network as well as my home directory (although the SFTP plugin I use for that is fiddly to set up and a bit buggy).
For copying the same things on a regular basis, I run rsync scripts in Termux (https://termux.dev/en/). They push photos from my phone to my computer and pull essential backup from my computer to my phone. They also synchronize themselves and Termux configuration. For exchanging files without a direct network connection, I have croc and the Rust version of Magic Wormhole (https://github.com/magic-wormhole/magic-wormhole.rs) installed in Termux.
It may seem arcane and archaic, but I have found this setup better (more predictable and more reliable) than anything "mobile-native". I appreciate that Termux behaves mostly like other Linux machines.
I had a pretty positive experience with Syncthing for Android on my previous phone. It was before I started using Termux. I had a directory called "clipboard" that synchronized between my desktop, laptop, and phone. Sometimes it took a while to notice new files. I would use it if I didn't use Termux or if I needed to keep multiple mobile devices in sync.
If I had to often send files to a mix of Android and iOS devices on the same local network, I would try LocalSend (https://localsend.org/).
The problem with email attachments is that they only support files up to around 25MB, while slack has a 1GB attachment limit, and if you have a slow uplink to the cloud, it could take a while to upload the file.
A friend wrote a faster and more sophisticated version of `python -m http.server 8000` in rust which also supports uploads, you can read about here: https://news.ycombinator.com/item?id=37628347
File transfers are performed over the local network, and therefore should be very fast. If you don't have a network connection, you can just enable a Wifi hotspot on your mobile device and transfer the files over that. There is no need to install any software on the mobile device, you can just use a web browser to browse to a directory that you share over the network.
I made the android† port of Wormhole William[1] specifically to help transfer some encryption keys that I didn't want to ever live unencrypted on a server in the cloud.
USB cables involve about 5 steps and half my cables don't work because they were not built sturdy enough and got chomped by my office chair, standing desk, or something else.
There should be a way to just right click and "send to phone"
I have a hub and an extra cable hidden under my keyboard. When I need to plug in a device for power or thunderbolt transfer it is a two step... 1) grab cable 2) plug in. Konnect is fine for other cases.
and then disable "do not disturb mode", swipe down notifications about 5 times until USB device pops up, open it, approve the hex shit, peck at the tiny button to change its mode from MDI/UTP/PCP/whatever other hell options are out there to a file transfer mode, then hope and pray it shows up on computer, then click, wait a long time, then navigate the phone's messed up file system before you can drop something in there.
I have a few qualms with this app. I've easily and securely sent things from one computer to another using a simple FTP server, with all of the features that croc has listed. As for CLI support, I've written a small helper tool that allows me to transfer via CLI - giving me more security since I don't pull in any dependencies in my own code.
FTP requires one-time config though, setting up credentials. Of course, it also needs a port to be opened. Croc and other apps like this[1] target different use cases.
[1]: I am building a UX-focused app over at payload.app, but there are many others, both CLI and GUI/web apps.
It can be or it can be encrypted. FTPS(i|e) implements encryption over FTP. I prefer SFTP as it just uses one port which I can define. SFTP also works better through firewalls and NATs.
VSFTPD is an example of an FTP server that supports encryption or FTPS.
From one computer to another? More like from one computer to this guy's server, then from the server to the other computer. Sounds pretty suboptimal if you want to send anything but few kilobyte files.
Last time I checked Croc was flagged by a couple of scanners on VirusTotal. There is the possibility that these are false positives and when we have the attention of HN I wanted to use the opportunity to raise awareness. Maybe someone can have a look into it.
EDIT: At the moment it is 100% green on VirusTotal.
https://github.com/psanford/wormhole-william/releases
I believe croc has some features wormhole doesn't (and some anti-features, like being able to pick curves and hashes). But also just that it's worth knowing that Magic Wormhole is kind of the "default" tool that does this.