
Show HN: Pipedream. Develop any workflow, on any trigger, with auth and no infra - todsac
https://pipedream.com
======
dang
[https://medium.com/@todsacerdoti/introducing-pipedream-
bbca9...](https://medium.com/@todsacerdoti/introducing-pipedream-bbca9dde0dc6)
is an accompanying article. Was originally the submitted URL.

------
todsac
Co-founder here. We have been heads down working Pipedream for the past 9
months and are excited to share our beta with you.

Pipedream is an integration platform built for developers. Develop any
workflow, based on any trigger with authentication management built-in and no
server or cloud resources to manage.

Workflows are code, which you can run for free.

The beta version includes the ability to:

\- Run any Node code, or use pre-built actions

\- Trigger workflows via HTTP, Cron, integrated apps or email

\- Create, share, and fork workflows from the community and

\- Send data to S3, Snowflake, email, SSE, and more.

Coming soon — Develop locally and deploy workflows via CLI, SDK, and more.

We think there’s a lot we can improve and are eager for feedback so please
send us your ideas and opinions. Also, if you want a specific app or API fully
integrated, let us know.

~~~
chrismatheson
Forgive me if this is clean and I haven’t seen it I’m just browsing from a
mobile device.

With all workflow type services I’ve encountered I find that the actual
underlying workflow can’t be managed under source control as one would do with
(hopefully) most if not all application & infrastructure code.

Can I “deploy” workflows from source control?

~~~
dylburger
Hi, Dylan, a co-founder here.

Deploying workflows from source via CLI or your CI/CD pipeline is in the
works! We want you to host workflows on Github / Gitlab and be able to deploy
them using your standard process.

Feel free to reach out to me directly if you'd like more information and I can
let you know when we're testing this.

dylan [at] pipedream [dot] com

~~~
hanniabu
Speaking of which, it's always overwhelming learning new tools. I know I can
benefit from this, but at the same time I'm not sure I can risk trying it out
given the time it will take to experiment and learn. I'd be using triggers
based off Github commits and merges, but I don't see any specific examples for
that and it's not exactly clear how the tutorial on the home page would
translate to what I want to do so I currently feel deterred from giving it a
whirl. Are there plans to create more tutorials? What kind of (rough) timeline
can we expect to see more tutorials?

BTW, love the name Pipedream!

~~~
dylburger
Thank you!

I completely empathize with the time involved learning new tools. I'm happy to
create a tutorial specific to your use case. We've built a few workflows to
process Github events and I'd love to show you how this works end-to-end.

Is there anything specific you'd like to do with the commit / merge events,
just so I make sure the tutorial targets your use case?

Feel free to reach out directly if you'd like to talk more at dylan [at]
pipedream [dot] com.

~~~
hanniabu
Thanks for the response and involvement in this thread, it really shows how
committed you guys are to providing a great user experience.

The specific use-case I have in mind is that when a commit made I'd want to
run various checks on configuration files in the repo to make sure specific
conventions are followed and that certain file references exist. Upon a merge
I'd like run a build process and send the build files to Heroku, Digital
Ocean, or AWS.

~~~
dylburger
The easiest way to run code in response to Github events is using Github's
built-in webhooks for a repo. I made a video showing you how to set that up to
forward events from your repo (e.g. every time you merge a branch to master),
then run code on Pipedream in response:

[https://www.youtube.com/watch?v=9ZGNP1-1vyg&feature=youtu.be](https://www.youtube.com/watch?v=9ZGNP1-1vyg&feature=youtu.be)

You can load the workflow I created for the video here:

[https://pipedream.com/@dylburger/template-workflow-to-
proces...](https://pipedream.com/@dylburger/template-workflow-to-process-
github-events-p_6lCOKW/edit)

and press the Fork button in the top right to create a copy of it in your own
account. You can modify that copy and run it for free.

The video isn't perfectly polished and I made a few mistakes, but we've built
Pipedream to make it easy to debug your workflows, too, so I hope the mistakes
help you understand part of its power!

A couple of notes on your specific use case:

* There's no webhook event for new commits. A PR (like I show in the video) or a push might be the best event to listen for. You can then run Node code to list commits associated with that push. See the API docs for commits here: [https://developer.github.com/v3/repos/commits/](https://developer.github.com/v3/repos/commits/) . * I wasn't sure specifically what build process you wanted to run, so I end the video reviewing how to add new steps to the workflow and walk through a few options: you can run any Node.js code or any of our pre-built actions. You have access to the /tmp dir on Pipedream and can use a package like child_process [1] to spawn some commands if you need to run anything on the shell. Unfortunately you don't have access to a full shell but let me know if what we provide works or doesn't — we'd love the feedback.

Let me know if that helps or if you have any questions!

[1] [https://www.freecodecamp.org/news/node-js-child-processes-
ev...](https://www.freecodecamp.org/news/node-js-child-processes-everything-
you-need-to-know-e69498fe970a/)

------
tmikaeld
The usual question, how they monetize the service?

It's answered here[1], along with limitations[2]:

\- 100kb max body-size.

\- Rate-limit at 10 requests per second

\- 10 seconds per execution for HTTP triggers, and 30 seconds for cron
triggers

\- 192 MB of memory for your code and libraries during workflow execution

\- 512 MB of disk in the /tmp directory.

[1][https://docs.pipedream.com/pricing/](https://docs.pipedream.com/pricing/)
[2][https://docs.pipedream.com/limits/#limits](https://docs.pipedream.com/limits/#limits)

~~~
haggy
This isn't an answer to how they might monetize this service and quite frankly
I've seen the OP/Co-Founder post their "answer" to this question on multiple
other comments and it's extremely hand-wavy. I currently have little to no
faith that this product isn't going to leave anyone that uses it stranded in a
year or so when they have to shutdown due to lack of revenue.

~~~
todsac
Haggy - we have no plans to charge individual developers for low volume
workflows that fit within our defined constraints. Our revenue efforts will be
focused on enterprise use cases.

I have no desire to be hand-savvy so please ask whatever questions you have
and I will do my best to clarify.

~~~
haggy
How would you envision supporting 10X the usage that you have today 1 year
from now if all that usage is falling under your "free" plan constraints?

~~~
pas
More throttling?

~~~
haggy
It's funny because of all the crickets. I asked a question after the OP
requested one and I've seen no response in 5 days. :shrug:

~~~
dylburger
Hi, this is Dylan, another co-founder and engineer. I'm sorry we dropped the
ball here. As you can imagine, it's tough to respond to every request and
we've been a bit busy since launch, but I apologize we didn't get to yours.

All of us were early employees of BrightRoll [1], an advertising marketplace
that handled 10 million requests / second at peak. We've scaled large systems
in the past and we've built this system with scale in mind.

Believe it or not, most workflows are extremely low volume. That, in part with
the design of the system, allows us to offer the generous free tier.

We're planning to introduce paid tiers soon. You can see my comment on the
parent thread that addresses some of the general questions around pricing
asked by others in the thread.

This was a very early release of the platform and we hoped offering the
product for free would encourage experimentation. To a great degree, we've
seen that, so we think it was the right choice.

[1]
[https://en.wikipedia.org/wiki/BrightRoll](https://en.wikipedia.org/wiki/BrightRoll)

------
l1n
Possible alternative for self-hosted applications:
[https://github.com/huginn/huginn/](https://github.com/huginn/huginn/)

~~~
sdan
One quick glance on this: this is amazing!

~~~
joshstrange
If you want something that works really well with nodejs there is also Node-
RED [0].

[0] [https://nodered.org/](https://nodered.org/)

------
dylburger
Hi y’all, this is Dylan, a Pipedream engineer and co-founder. We launched
today to get early feedback from developers on what we’ve built and we’ve
received a lot of thoughtful comments. Thank you to everyone who signed up!

I especially appreciate the discussion around how and why Pipedream is free,
and the concerns around the lack of a visible, scalable paid tier. Those
concerns are valid and I want to give you a little more insight into our
thinking on this and what the future holds.

Workflows created by one developer can be forked, run, and modified by others.
We all build a lot of the same integrations across companies and believe if
this code is shared and executed on a common platform that’s purpose-built for
running these workflows, it’ll save us all a lot of time.

We’ve worked with thousands of alpha users to understand how they’re using the
product and considered what features enterprises ultimately will want and need
to pay for. These include: higher workflow limits, private workflows and
actions, SLAs, premium support, and more. We didn’t launch with that today
because we’re focused on getting feedback from individual developers who will
be the majority of users moving forward.

Of course, you can’t run a business solely on free workflows. We’ve set some
limits on these workflows that help us control excessive use [1]. Our team has
a wealth of collective experience scaling software companies on the business
and tech side, and we have confidence that we’ll be able to retain a generous
free tier while building a sustainable business.

I empathize with the skepticism of Pipedream and of hosted platforms in
general, and welcome any more specific questions. We truly love feedback.
We’ve implemented some ideas that we believe facilitate the developer
experience for building workflows but are looking for y’all to test that,
validate or invalidate it, and give us specific thoughts as you have time.

dylan [at] pipedream [dot] com

[1] [https://docs.pipedream.com/limits/](https://docs.pipedream.com/limits/)

------
zimmund
It looks like an interesting alternative to Zapier (or its open source
counterpart n8n [1]).

As a potential user, before even trying the tool I'd like to know how are you
planning to monetize it. Free sounds awesome, but you'll get to a point where
it costs you actual money to keep all this working. I don't want to waste my
time creating pipelines in a tool that won't exist in a year or two -or that
will become paid and break my existing workflows (or make me expend money
unexpectedly)-.

[1]: [https://n8n.io/](https://n8n.io/)

~~~
todsac
We believe anyone should be able to run simple, low-volume workflows at no
cost, sharing their workflows with the public so everyone benefits from the
work of others. We also want to foster a positive community where people feel
good about sharing their work and where everyone can learn from one another.

In the future, we may offer features available on paid tiers which would be
logical enterprise features such as single sign on (SSO), team collaboration
and higher SLAs and throughput.

The constraints are listed in our docs -
[https://docs.pipedream.com/pricing/](https://docs.pipedream.com/pricing/)

~~~
zimmund
Thanks for your reply! But it doesn't answer my question: I'm worried about
how you will sustain your platform in the long run for all those free tier
users.

Learning a new tool -even if it's friendly and has a community around it-
takes time and effort. We want to know it will be there in the future. That's
why I'm interested to know: how are you planning to keep Pipedream alive for,
let's say, the next 5 years?

------
xemdetia
I do not feel that this is the best article to sell me on your product. It
compensates for trying to describe a rock solid use case with a lot of links
to other things with limited context (that are probably brilliant if you knew
that context). If I wanted to watch a 15 minute video I would watch that
instead. I feel like your pipedream.com landing page does a better job
describing why I want to use your product.

I wish this article was a more digestible single case as an introduction that
showed value rather than saying 'you can run anything!!! for free!!!'

~~~
dylburger
Hi, Pipedream founder here. I appreciate the honest feedback.

The post is meant to articulate the depth of our vision for the platform. The
hope was the linked workflows provide some inspiration and examples to make
Pipedream a little more concrete. I'm glad the homepage resonated with you.

Did you get a chance to sign up and use the tool? We're looking for specific
feedback from devs on how we can improve so I'd love if you had any time to
build a workflow and share your thoughts.

~~~
dang
It's more standard for a Show HN to link to the home page and then include a
link to the article in the thread. We'll put that above.

~~~
penagwin
You're an awesome moderator :D

------
linuxdude314
Why would I use this instead of an option built into a major cloud provider
such as AWS, GCP, or Azure?

How do you differentiate yourself. Do you (or will you) support frameworks
like Serverless?

~~~
dylburger
Hi, Dylan, a co-founder here. It's a great question.

We've found that for integrations or common automations, building on a cloud
platform is too low-level. You've got to manage a number of services that
aren't core to your use case (e.g. API Gateway, IAM, Cloudwatch Logs, etc.).
We operate at a higher level and just give you an execution environment to run
Node.js code or stitch together pre-built actions, similar to integration
platforms like Zapier.

Of course, Serverless and other frameworks abstract part of the cloud
services, but you're still running on that platform, and when you need to
troubleshoot or scale the service, it's likely you'll have to peek under the
covers. We tried to build a serverless platform from the ground up and provide
services like the HTTP server, key-value store, logs, and other services right
inside the UI, or accessible in code. There's no other cloud resources to
manage or monitor.

This kind of abstraction isn't great for every application, but we think it
shines when you're building integrations or data pipelines that aren't part of
your core app.

The pre-built actions and built-in OAuth / key-based auth also operate at a
higher level than most cloud services. We've tried to provide these in a
developer-friendly way, so you have full control over how your workflow runs,
but don't have to manage the parts that are tangential to the logic of your
app.

As we expose more APIs for building and running workflows, we'd love to
support a framework like Serverless.

I'd love if you have time to give Pipedream a try and see how it works for
yourself! We're looking for feedback on how we can improve so we'd love more
eyes on it.

~~~
pevezzac
How would you differentiate yourself from Microsoft Flow?

~~~
dylburger
I haven't used Microsoft Flow in depth so I unfortunately can't comment.

If you've used it and like it, I'd honestly love for you to give Pipedream a
try and tell us where we can improve compared to your experience with other
tools. We'd love the feedback!

dylan [at] pipedream [dot] com

------
keithwhor
Congrats on launching!

I'm a founder of a startup in the same space, so just wanted to say good luck.
I love the name. It's exciting to see so many folks working on making
connective developer tooling a better experience.

I had a link originally included in here but -- you guys should enjoy your
launch. I understand and have a great deal of empathy for the effort it takes
to build and launch something new. Enjoy all of it, and keep up the good work.

\- Keith

~~~
dylburger
Keith, this is Dylan, a co-founder. We've been very impressed with what y'all
are building at Standard Library and I agree it's a great time to be working
on developer integration tools! Thanks for the note.

------
zlagen
The example demo looks very slick! it looks like Zapier but a bit more low
level. Also for anyone interested in running functions but not interested in
handling the complexity of aws lambda I'd check Cloudflare workers
[https://developers.cloudflare.com/workers/](https://developers.cloudflare.com/workers/)

------
atarian
What security measures are in place to deal with malicious code from other
users?

~~~
atarian
Since you guys haven't answered, I'm going to assume there is no such security
in place.

~~~
dylburger
Hi, this is Dylan, a co-founder and engineer. I'm sorry we didn't get back to
you sooner. It's tough to respond to every single comment on a Hacker News
thread and we've been quite busy with launch. Hope you can sympathize.

Your code runs in its own Node execution environment and VM, completely
isolated from other users' code.

Happy to answer any other specific questions you have! Feel free to reach me
at dylan [at] pipedream [dot] com .

------
magicalhippo
> Develop any workflow, based on any trigger

As far as I can see the above should come with a footnote saying "for some
value of any".

Looks very cool, despite being entirely web-centric.

------
davidthewatson
I'd have an easier time evaluating pipedream if I got something other than a
blank page when clicking through to the website from my mac running chrome.

~~~
dylburger
Hi David, this is Dylan, a co-founder. I'd love if you had the time to send me
a screenshot of what you're seeing, with your Developer Tools Console open in
Chrome, if you don't mind. It'd be a huge help!

dylan [at] pipedream [dot] com

------
pythonwutang
How does this differ from Amazon Lambda?

Only read the landing page so far but it looks like Lambda with just Node
support. Can someone correct me if I’m off?

~~~
TheSpiciestDev
Your comment has me imagine a more visual, GUI-based editor on top of AWS
Lambda (i.e. hooking up all sorts or internal/external events, tying workflows
or multiple Lambdas together, still including a small editor for writing
JavaScript, etc.)

I mean, you can write code in-line with AWS Lambda but to have an application
help setup multiple Lambdas and/or integrate outside APIs would be pretty
neat.

~~~
dylburger
That's spot on! Take a look at our intro video for an end-to-end overview of
how to use the tool:
[https://www.youtube.com/watch?v=ivkE26ZsyUo&feature=youtu.be](https://www.youtube.com/watch?v=ivkE26ZsyUo&feature=youtu.be)
.

I'd love if you had the chance to sign up. It's free to use and we'd love the
feedback.

------
jcmontx
I like it but I don't fully get it. Why use this service and not Lambda or
Azure Functions?

~~~
dylburger
Hi, this is Dylan, a co-founder. It's a great question. Having built many
applications on Lambda this is something I think about a lot :) .

We've found that for integrations or common automations, building on a cloud
platform is too low-level. Frequently you're working with more than just a
Lambda. You've got to manage a number of services that aren't core to your use
case (e.g. API Gateway, IAM, Cloudwatch Logs, etc.). There's a lot of
boilerplate config, and many of these services are tangential to the core
logic of your app.

We operate at a higher level and just give you an execution environment to run
Node.js code or stitch together pre-built actions, similar to integration
platforms like Zapier. We manage the HTTP endpoint, function config, and give
you built-in observability. We also provide higher-level services, like built-
in app integrations and OAuth / API key-based authorization. Building this
yourself on AWS can take time.

This kind of abstraction isn't great for every application, but we think it
shines when you're building integrations between services.

Let me know if that answers your questions, or if you have any more!

------
lukevp
How's this going to be monetized? What sort of execution limits are there?

~~~
todsac
We believe anyone should be able to run simple, low-volume workflows at no
cost, sharing their workflows with the public so everyone benefits from the
work of others. We also want to foster a positive community where people feel
good about sharing their work and where everyone can learn from one another.
In the future, we may offer features available on paid tiers which would be
logical enterprise features such as single sign on (SSO), team collaboration
and higher SLAs and throughput.

The constraints are listed in our docs -
[https://docs.pipedream.com/pricing/](https://docs.pipedream.com/pricing/)

~~~
zocuments
What framework/template are you using for docs.pipedream? They look great.

PS: On Desktop, the left third of the Pricing content/copy is rendered behind
the menu.

~~~
dylburger
Thanks for letting us know, I just rolled out a fix to the hidden content. Can
you let me know if you're still seeing the issue?

We use Vuepress for the docs and it's awesome!

[https://vuepress.vuejs.org](https://vuepress.vuejs.org)

~~~
zocuments
Looks good now!

------
saarsaar
WOOHOO!!! Congrats on the launch!!!

------
therealmarv
Name reminds me of a big adult novelties brand and company.

------
grafcet_online
free until when :)

