
Show HN: patchbay.pub – Poor man's ngrok/IFTTT/serverless - anderspitman
https://patchbay.pub
======
anderspitman
Author here. I implemented[0] a stripped-down server that includes the MPMC
functionality. Should be a good starting point for alternative
implementations. It really shows how simple the concept is under the hood. All
it really does is translate HTTP requests into golang channels.

[0] [https://github.com/patchbay-pub/patchbay-simple-
server](https://github.com/patchbay-pub/patchbay-simple-server)

------
zomglings
This is excellent - a big fan of its scriptability via curl. I think you have
the beginnings of a truly amazing developer experience here.

If you're interested in monetizing this:

I would pay a fee for this - say $4/month. The only features I would like at
this price:

1\. Authentication (-H "Authorization: Bearer <token>" would be good enough) -
at the level of the routes. This sounds like what you have done for routes
like '/' anyway.

2\. A CLI to tell which routes I "own".

3\. Persistence of messages on the patchbay server up to some reasonable time
limit T so that consumers can replay messages since time T without the
publishers being blocked on their side.

I would pay $8 per month if:

1\. You hosted consumers (go binaries or docker containers which would accept
the body and headers of the HTTP response on the consumer side and do
something with it), ran them for me in a loop

2\. Guaranteed me some level of isolation for things like API keys, etc. which
would be passed to my consumers in some fashion.

Even if it were open source, at these prices, I would buy. Don't want the
headache of hosting it myself.

~~~
anderspitman
Thanks for the detailed suggestions. A few questions:

1/2\. Why not a private instance for $4/month, accessible only to
authenticated requests (unless of course you choose to open up certain routes
for GET and/or POST)? Not only does this mean you can use semantic URLs
instead of random ones (which helps offset the annoyance of managing API
tokens), but you also get a unique domain that you can do things like put
behind CloudFlare or another CDN.

3\. This is an interesting idea. However, by far the greatest strength of the
current implementation is that it has 0 (well ok, very little) explicit state.
What's your use case for the semi-persistent producers?

1\. It's interesting that you request hosted consumers and not hosted
producers (a la for hosting a website). But I see the appeal. It would be nice
to be able to set up something like a text notification trigger and not have
to worry about it being tied to your own physical machine.

I totally agree about the open source bit. I don't see open source as a threat
to a business model here. With something so simple it's going to come down to
execution and customer service. As a staunch supporter of self-hosting, after
1 or 2 services it simply becomes too much to manage, unless you enjoy it as a
hobby. I just want to get work done without giving my digital self over to the
big guys. Even paying others per service doesn't really work, because at
5USD/service (which I paid for a Mastodon instance for a while), it quickly
becomes too much money. Part of the inspiration for patchbay was trying to
make a dead-simple substrate on top of which I could implement many other
things, hopefully saving me some self-hosting trouble.

~~~
henryfjordan
for 1/2 you could offer the route /<username>/<channel> to the paying user to
help the namespacing issues and keep the shared infrastructure angle. Then
charge "enterprise" pricing for private instances

~~~
anderspitman
That would go a certain distance, but I'd have to do a lot of measuring to see
how scalable it is. Part of the problem with the free instance is I have to
aggressively rate/bandwidth limit, because it's running on a single
DigitalOcean instance with what maybe 1-10Gbps tops? I'd love to be able to
give paying customers much higher performance by spreading across multiple
machines. Though if any sort of a viable business model comes out of this,
I'll probably look to switch to physical hardware anyway.

------
anderspitman
Noticed someone working on a rather interesting exploit in the logs:

POST /.well-known/acme-challenge/huhn2?mime=text%2Fplain

Plugged that one. If anyone thinks of similar holes I should consider, please
shoot me an email at info@patchbay.pub. There are a whole new set of security
issues that crop up with this sort of system. Definitely be aware of that
before using it!

~~~
brianzelip
How did you "plug" that one? And what is "that one"?

I assume "that one" is posting to /.well-known/.

~~~
anderspitman
POSTing or GETing currently

------
whois
I like this. Have you considered releasing the source so people can run it on
their own servers?

~~~
anderspitman
I would love to make the source available. But there's no getting that genie
back in the bottle once you go that route. I wanted to release it and get some
feedback before deciding what to do with it. That said, a server that supports
the examples on this page is pretty trivial to implement (especially if you
skip the pubsub stuff). I had a working prototype in a weekend. Multiple
implementations would probably be a good thing.

EDIT: also, the current state of the code base is not something I'd be excited
about having my name attached to ;)

~~~
henryfjordan
You can think about releasing the code with no license or with some form of
the newer BSL or something.

It won't stop bad actors from stealing your code and using it but you will
stop most corporate actors from touching it (and they are the ones you'd want
to pay in the future anyway).

------
fenwick67
The name (including the .pub part) made me think this was related to the
Secure Scuttlebutt client Patchbay:
[https://github.com/ssbc/patchbay](https://github.com/ssbc/patchbay)

~~~
anderspitman
No confusion was intended; sorry about that. The domains seem to be fairly
orthogonal. Or am I not understanding what ssbc/patchbay does?

~~~
ChristianBundy
Patchbay is an interface for Secure Scuttlebutt.

Secure Scuttlebutt is a P2P pubsub system for sending files, messages
(chat/notifications), and everything else you can implement with pubsub-like
systems.

I think it's just confusing because of the overlap with:

\- Patchbay

\- Pub

\- Channel

If you're curious about SSB, I've posted a link to your site on the network.
You can view that (and other content) here: [https://oasis-
demo.fraction.io/thread/%25IGLa%2F3NDG3cj5gYOY...](https://oasis-
demo.fraction.io/thread/%25IGLa%2F3NDG3cj5gYOYmeBL93m%2FJyxEAKGp65IksEZHfY%3D.sha256#%25IGLa%2F3NDG3cj5gYOYmeBL93m%2FJyxEAKGp65IksEZHfY%3D.sha256)

~~~
anderspitman
Ah gotcha. I've heard about Scuttlebutt, but didn't realize there was so much
terminology overlap.

------
lsferreira42
Well this is amazing, it's simple and functional, solves a wide variety of
problems with a very simple approach.

Congratulations for shipping it!

~~~
anderspitman
Thank you!

------
sairamkunala
I made a similar open source self hosted tool for sharing logs or analysis
from several machines, hosting internal versioned documentation, counters and
time-series graphs etc.,

This was from a few years ago - [https://github.com/sairam/daata-
portal/tree/master/daata-ser...](https://github.com/sairam/daata-
portal/tree/master/daata-server)

When people working in ops or with remote CLIs would have a great need for
such a tool.

------
fiatjaf
Added to [https://github.com/fiatjaf/awesome-
loginless](https://github.com/fiatjaf/awesome-loginless).

~~~
anderspitman
Thanks! That's a sweet list.

------
davidito
Good luck with your project, it sounds interesting.

FYI, your domain is blocked by Cisco Umbrella (OpenDNS) as "This site is
blocked due to a security threat that was discovered by the Cisco Umbrella
security researchers." Not sure why, you might want to contact them.

~~~
anderspitman
Thanks for the heads up. That's fascinating. Has anyone else ran into this
before?

~~~
royragsdale
I ran into it this as well and the response from the folks running our block
list said it triggered a "newly seen across all of OpenDNS" heuristic. This
matches other similar blocks I've seen for newly registered gTLD domains.

Seems that this is the relevant documentation:
[https://support.umbrella.com/hc/en-
us/articles/235911828-New...](https://support.umbrella.com/hc/en-
us/articles/235911828-Newly-Seen-Domains-Security-Category)

------
thojest
How does this technically work? Is this a server which just tunnels through
tcp traffic? So does all data pass through your server or is it a rendezvous
server, which uses ICE to establish a P2P connection?

~~~
anderspitman
Yep everything goes through my server (see the security and privacy sections
at the bottom of the page).

It would be awesome if someone could figure out a peer-to-peer way to
accomplish the same thing. I'm not familiar with it, but I think that might
essentially be what Tor hidden services are.

~~~
thojest
Thank you very much for your answer, and great project :) I am also not very
familiar with P2P, but I know that it is a bit complicated because of NATS.
ICE is a protocol to make it possible for clients to establish a P2P
connection, but it is not always possible because of symmetric NATS. Thus, for
example in WebRTC there exist STUN and TURN servers.

------
jehna1
Ok, the first thing in my mind is how to share passwords with your teammates.

Sending a password:

    
    
       echo 'hunter2' | openssl bf - | curl -X POST --data-binary @- https://patchbay.pub/mysecretpasswordhandle
    

Receiving a password:

    
    
        curl --silent https://patchbay.pub/mysecretpasswordhandle | openssl bf -d -
    

This uses end-to-end encryption (Blowfish cipher) that relies on a shared
password.

~~~
anderspitman
But how do you securely share the patchbay channel? ;)

~~~
g4d
invisible ink and paper

------
timvdalen
This is incredibly interesting and super clever. I love all the little
examples of what you could do with this and am definitely going to play with
this.

~~~
anderspitman
Thanks!

------
phlyingpenguin
adoc formatted mirror from before/while this got destroyed from its chat
getting DOS'd: [https://pastebin.com/tAWNuRL7](https://pastebin.com/tAWNuRL7)

~~~
anderspitman
Hm I'm not sure it was actually a load problem. The server load never spiked.
I'm guessing the hosting got in a weird state, either on the server side or
the producer process.

EDIT: But in case it happens again here's an internet archive link:
[https://web.archive.org/web/20191126170030/https://patchbay....](https://web.archive.org/web/20191126170030/https://patchbay.pub/)

------
qrv3w
This is so cool! I really like your project. Its so simple and powerful and
very succinct. Do you have an example for the "Poor man's file sharing"? I
tried doing the simple thing:

> curl -F 'data=@test.txt'
> [https://patchbay.pub/test.txt](https://patchbay.pub/test.txt)

And on the other computer:

> curl [https://patchbay.pub/test.txt](https://patchbay.pub/test.txt)

But I'm seeing that the output is being wrapped with "Content-Disposition" and
"Content-Type" at the top. I'm no HTTP expert, but is there a way to use your
server (or namely, curl) to transfer the file so that it doesn't wrap this
info?

~~~
codetrotter
Change the command on the sending computer to

    
    
        curl -X POST --data-binary "@test.txt" https://patchbay.pub/test.txt
    

The command on the other computer remains the same as what you had

    
    
        curl https://patchbay.pub/test.txt

~~~
qrv3w
Thanks!

------
billhathaway
The term 'poor man' isn't very informative to me. Does that mean it has all
the functionality but is free or does it mean has basic functionality and is
free, or is cheaper?

~~~
anderspitman
I'm using it in the idiomatic English[0] sense, meaning a cheaper/inferior
alternative. For any one of the examples, there are way better tools out
there. But how many of those individual tools can be used to accomplish many
different things?

[0] [https://www.merriam-
webster.com/dictionary/poor%20man%27s](https://www.merriam-
webster.com/dictionary/poor%20man%27s)

------
jedimastert
I really like this. I think it'd be interesting to implement on my own server
as well (You know, every non-cloud reliance or whatever). It's super simple
and super easy to grok.

I'm guessing you aren't looking to put this out open source if you're looking
to make a product of it, but I'd love to see the implementation.

------
yoru-sulfur
This is great, I've been thinking of building something similar myself for
exactly the use case you mentioned.

------
ee3456
someone DDoS'ed the chat again, site is down :(

------
moose333
This is a fantastic idea. Great useful examples, the little chat client is
such a novel use of a simple concept. Well done!

------
qbaqbaqba
It would be nice to have read-only endpoints.

~~~
anderspitman
I thought about a couple different ways that could be implemented, without
requiring a login. For example, you could make the channel a public key
signature, then only accept POSTs that can prove they own the private key. The
problem is that it makes it trivial for people to anonymously host whatever
they want on my server...

As it stands now, there's a bit of game theory involved. You can shared a
channel publicly, but if even one person doesn't like what you're publishing,
they can bring it down. This essentially forces people to keep their channels
for private use.

This is also one good use case for charging people for private
namespaces/channels.

~~~
qbaqbaqba
Simplest solution would be a different name for broadcasting channel. But you
are right in your concerns about people being assholes.

~~~
anderspitman
Main problem with simply using a different name is that the server has to keep
track of the mapping between the producer port and the consumer port. As it is
currently the server has almost no state.

------
shark0der
HN hug of death?

~~~
anderspitman
Not quite sure. The server load is surprisingly low, but it's gotten locked up
a couple different ways.

~~~
shark0der
Check the amount of traffic you've got: number of packets, number of
connections. If it's under attack you might have LOTS of packets (for example
SYN) coming your way which explains the slow performance.

~~~
anderspitman
Only a few hundred connections, and iftop is showing very low traffic (does
that capture SYN floods?). I get the feeling people might be DOSing as a
proof-of-concept then backing off, which I appreciate.

------
kylek
This is super weird. The "job queue" example made me facepalm, I'd probably
use a one-liner like-

ls -1 *.mp3 | xargs -P4 -n1 -i ffmpeg -i "{}" "{}.ogg"

(yah the filenames would end up .mp3.ogg, but so does the example...should
throw basename in there somewhere)

~~~
Mic92
It makes sense if you have multiple machines to actually distribute the load
(files could be on a NFS).

~~~
emj
You can send the files through the pubsub directly and just convert a stream
of bytes. It is a beautiful pattern, but needs some tools to make it really
easy to use (e.g. how to I metadata and bytestream to a pipe)

~~~
anderspitman
Sending the actual files would probably be really slow on the free server, due
to bandwidth limiting.

As for metadata, you might consider multipart/form-data, or just packing it
into the query string (obviously depends on how much data you're talking
about). Of course you'd probably need to write that in something other than
bash.

EDIT: It's pretty easy to send form-data with curl: [https://ec.haxx.se/http-
multipart.html](https://ec.haxx.se/http-multipart.html)

~~~
emj
Yes but it is the receiving that mess you up, if you want to use standard
shell tools and just stream it. Everything can be solved by using other tools.

# preserve filename for i in a b; do gzip -9 $i; done

# does not preserve filenames while true; done curl $patchbay| mangle | gzip
-9 | curl -d@- $patchbay; done

