
Show HN: Pipe To Me – Stream data over HTTP using curl - pyinthesky
https://github.com/jpschroeder/pipe-to-me
======
finchisko
Awesome. I would use this as shared clipboard. Immediately made bash functions
for myself :)

 _pipefrom() { local key=${1:? "Error: Missing pipe key"}; curl -s
[https://pipeto.me/$key;](https://pipeto.me/$key;) }_

 _pipeto() { local key=${1:? "Error: Missing pipe key"}; curl -T- -s
[https://pipeto.me/$key;](https://pipeto.me/$key;) }_

------
0db532a0
Would there be no way for the server to indicate over HTTP to curl that it is
not ready to receive until the receiver has connected? Then curl might be able
to wait until it reads from the pipe.

~~~
pyinthesky
Hello, Author here. I toyed around with the idea of not allowing the sender to
connect until a receiver was connected, but I could see the use case where you
wrote a stream to nowhere and then had receivers jump in and out.

I also toyed around with using the "-T." curl argument instead of "-T-" to
allow some response in this case, but I came up against a known issue in curl
([https://github.com/curl/curl/issues/932](https://github.com/curl/curl/issues/932)).
I'm sure this wasn't the intended use case though.

It is definitely worth exploring further.

~~~
0db532a0
That is interesting. What made you stay away from returning an error if
someone tries to post without a receiver?

~~~
pyinthesky
I may end up re-thinking this, but one idea was that you could temporarily
`tail -f` a log file into a pipe and then check up on it later via a mobile
browser. It could be writing the whole time, but just dropping everything when
you weren't watching it.

It would probably also make more sense if I were to add some kind of buffering
or storage in the future.

~~~
0db532a0
I don’t know your needs and goals, and maybe it has to be HTTP, but what about
handling the sockets yourself?

People could then use netcat to send and receive, and the server could block
either end until someone connects on the other end. Personally I would really
appreciate a tool I could connect to with netcat or anything else that speaks
TCP. I don’t know how someone would use the service from javascript though.

No buffering needed by the server. If either end has to be asynchronous, just
run in background + nohup if you’re calling this from a shell.

Maybe some inspiration from how blocking FIFOs work on Linux:
[http://man7.org/linux/man-pages/man7/fifo.7.html](http://man7.org/linux/man-
pages/man7/fifo.7.html)

You could even do a non-blocking version which just refuses connection. netcat
can be set to retry: [https://unix.stackexchange.com/questions/432249/nc-to-
retry-...](https://unix.stackexchange.com/questions/432249/nc-to-retry-on-
connection-refuse)

I realise that this doesn’t make it easy to indicate which pipe one wants to
connect to.

~~~
pyinthesky
I think you are right actually. I will experiment with some ways to make
blocking vs non-blocking operations. I need to try out @paulddraper's
suggestion below to see if I can do that over http. I'd like to make a raw
socket version regardless. Like you said, I just need to figure out the best
way to pass the key.

Edit: One nice thing about the https interface is that you get an encrypted
channel. Maybe SSH would be another option for transport. That would also make
passing keys easier.

~~~
0db532a0
How would the SSH work?

------
ElijahLynn
What are some practical applications for Pipe to Me? (should be in README.md)

~~~
pyinthesky
There are a couple (admittedly simplistic) examples in the readme like
transferring or remotely watching files. The basic idea though is to take data
from a unix/linux pipeline from one machine to another with the benefit of
also being able to watch it in real time in a browser.

------
jayar95
how does it update without javascript?

~~~
Alexendoo
The browser will start displaying the contents of a response before it has
entirely downloaded. So in this case the browser sees it as a really slow
server (you can see the loading icon is still there in the demo) that sends a
bit of content every so often

