Pipes was the first service I ever owned from an ops perspective (circa 2008). We had two clusters, one on each coast. I thought we had capacity to take one offline to do a full upgrade, turns out we didn't.
By the time I turned the west coast back online, the east coast was failing health checks, and the LB failed everything over to the west coast, which then proceeded to be overloaded, and everything flipped back to the east coast... Ping ponged for about 2 hours until they finally settled.
Fun times.
Another fun story: For the longest time the front page had an example pipe that merged search results from various online sites (amazon/ebay/cl). It was made by a former employee and was easily one of the most popular pipes. One day we found out he had his affiliate id in all of those links. We chuckled and moved on.
Haha, I loved that search results pipe! The product was amazing at the time. It's really too bad that Yahoo didn't try to make something more out of it.
I worked on Pipes briefly, shortly after it launched. It was really two distinct products rolled into one. The visual interface was groundbreaking and stunning-I think half my team was hired on the strength of that demo. No one could believe Javascript could do that.
The backend was an extremely useful tool for munging RSS feeds. With any kind of support, or even benign neglect, the product would have been successful. It took a lot of active mismanagement and folly to keep Pipes from living up to its promise.
Hats off to psadri and the other Pipes creators for a really stellar piece of work.
By the way: many people in the past, including here on HN says that Elop was right, and that the reason Nokia died was 100% apple+google fault.
Here in Brazil Elop effect was interesting: it was VERY obvious, yet noone here knows who the hell Elop is.
To put it simple: When Elop released that stupid memo, Nokia had 67% of Brazil smartphone market share, and the share was still rising, and Nokia utterly dominated in the non-smartphone market too. Right after Elop memo, the first thing that happened is that suddenly the communities died (although Nokia store DID suck, there was at least in Brazil a vibrant freeware app community that shared stuff in forums), then just some months later you could not find phones for sale, even new models, because the import companies DID heard of Elop memo, and decided to not risk importing the phones, even if shopkeepers still wanted it.
The end result was funny: Elop nuking of Symbian was so hard, that in Brazil other smartphones could not keep up for a while, and we temporarily had a decrease in smartphone use, and a increase in feature phone use, and that increase came mostly from truly crap chinese companies, because not even Nokia feature phones could be found in stores anymore.
> Elop was a director of consulting for Lotus Development Corporation before becoming CIO for Boston Chicken in 1992,[3][7] which filed for Chapter 11 bankruptcy in 1998
> In the same year, he joined Macromedia's Web/IT department[7] [...] from January 2005[10] for three months before their acquisition by Adobe Systems
> He was then president of worldwide field operations at Adobe, tendering his resignation in June 2006 and leaving in December,[12] after which he was the COO of Juniper Networks for exactly one year from January 2007 – 2008
This is not the CV of a man that I would hire for anything other than running a going concern into the ground.
Ahh, what a terrible CEO. When the only two possibilities to explain your tenure are 1) a lying co-conspirator or 2) mind-blowingly incompetent--then you've truly done a bang-up job.
I'm still upset by the fact that MeeGo/Maemo was killed, and Nokia sold to Microsoft. The N9 was still the best phone I've ever used, and it's UX ideas are still the simplest and most effective way of using a phone, and yet no one else has implemented it.
Jolla (https://jolla.com) was founded by people involved in MeeGo development. While their phone got a lukewarm response, I've heard many good things about their upcoming tablet and have ordered one. As you used MeeGo, how does Sailfish OS appear to compare with it?
Sailfish unfortunately does not have all of the UI things I'm talking about -- the "three screen with edge gestures", which is a shame :( They couldn't, because the N9's UI was never open sourced despite the core being so.
Is Nokia's example really good ? Nokia remain one of the largest companies in telecom hardware provider and handsets was always a side business for them. They successfully sold it to microsoft.
i think nokia is more about lack of executive vision/market knowledge. The other examples are more about middle management, besides the executive lack of vision and bean counting.
Incorrect. The first rule of mismanagement club is to never know exactly how you got into mismanagement club; but by god you are going to do a bang up job of it now you are in!
I think most/many people could reasonably say this about their company; I currently work at Google and feel quite the same. I'm not convinced that either Google or Twitter[1] qualify as a particularly spectacular example of mismanagement though.
[1] I don't have first-hand experience but one of my best friends work there and we regularly vent to each other about work complaints.
I'll echo those sentiments, the UI for that time certainly had a wow factor.
> With any kind of support, or even benign neglect, the product would have been successful.
Look at IFTTT services now and it's easy to realize that pipes even in its original incarnation could have been so much more given its brain dead UI ease.
It's a shame that Yahoo is shutting this down rather than pivoting it as an IFTTT or similar. I'd wager that it's lack of ownership that makes it hard to pivot. I also don't recall it having a premium tier which has got to weigh down the resources--talent always gravitates toward revenue centers it seems.
> talent always gravitates toward revenue centers it seems.
Ie the 'new management can't fire my entire division for being unprofitable' center. Seems like a smart move for the people who wrote Pipes but never found a business model for it.
I wrote a simple Wordpress plugin that works with RSS feeds. In addition to aggregating fees, Pipes has been a go-to solution when people report feeds that are malformed, have some weird characters or anything that chokes the Wordpress RSS parser (simplepie). People would run them through pipes, which seemed to handle anything and output clean RSS.
I'm sad to see it go and it'll be missed by quite a few people who use my plugin.
pipes creator here: sad to see pipes shutdown. i am surprised it lasted this long -- it was abandoned years ago. on the positive side, it is nice to see it mentioned in the same sentence as y! maps which used to be a big deal back in the day.
Pipes has long been a sort of monument to the faith of a bygone age -- the first flush of Web 2.0 optimism, when standard formats and open APIs were going to let us mix and match and mash up services of all kinds at will.
That faith is dead now, of course, scoured from the earth by walled gardens and VC money. But like Catholics in Elizabethan England, some of us quietly tend our secret shrines and pray for its return.
(Hopefully that will work out better for us than it did for them.)
Thank you for your comment, for a (long) while I thought that I was the only one remembering those times and especially the ideals people were bringing up then. Sometime in 2005 I even copy-pasted this quote on my blog, high in believing that the future will belong to open data and open information:
> Bosworth advocated an open model for data.(...)
Imagine if you can query any data that is available anywhere in the world. Bosworth said that what this requires is a single, simple, open wire format for items. The format needs to be simple for any P programmer to deliver and any JavaScript programmer to consume. He also pointed out that "complex things tend to break and simple things tend to work." Google has the simplest query language in the world. There is no structure and no syntax. (my note: at that time Bosworth was VP of Product Management at Google)
Love your analogy, sure made me laugh (visualising a geeky dev like myself tending a weathered gnarly monument in a secluded English glade... in real life, that is, not in an RPG).
However, why such pessimism? I hardly think that standard formats and open APIs are dead. Sure, there are some nasty blights upon cyberspace in this regard (ahem, Facebook, Apple), but other players are still keeping the dream alive to some extent (e.g. Google, Yahoo).
I'd count google mostly as an example of deprecating or limiting useful interfaces. Yes, you can feed them stuff in "open" formats, but getting stuff out is harder. First example that comes to mind is youtube APIs, which offered easier access to stuff. Now try to get a feed of someones videos or similar...
But it is possible that it's confirmation bias + uncommon sample on my part.
Yeah... for Google / YouTube, I blame OAuth for most of the difficulty in accessing data (while retaining one's sanity) these days. Once you've waded through the quagmire of access tokens, auth libraries (in different languages), account permissions, etc... making the actual API calls isn't so hard.
Let me just tell you and anyone that worked on the team, thanks so much for it. You saved me hours of work and let me do it really damn well. This is a sad, sad day. I used Pipes extensively and ironically enough at Yahoo for some features/prototypes on News and Search. It was such an elegant and powerful app, I really wish Yahoo had given it more love.
It really would be oh so nice to have something that can ingest RSS and Tweets and then apply filters to them while visualizing the logic through a graphical interface. Maybe even some fuzzy machine learning to use in building filters. Some more advanced dupe technology would go a long way. People would be building their own techmemes in no time. You could build ranking systems that award bonus points if something like longform or techmeme mention something.
Maybe im dreaming about a free version of Percolate, which I have heard is really cool.
This is exactly what we are doing at http://www.feedsapi.com , we don't use a graphical UI for the logic but keywords and RegEx to create filters... Many folks are already building their own techmeme with our API, it's however not a free service. I'm not sure if this fully fits your requirements but you should take a look.
One of our open source projects[1], lets you consume RSS, Email, and Tweets, with more "stuff" coming, and apply filters and set triggers, etc. It's still pretty raw, and we don't have the really awesome visual interface for laying things out, but that sort of thing is something we hope to get to eventually. One other neat thing we do is pipe content through Apache Stanbol to do semantic entity extraction, and build up a knowledgebase as content is consumed.
Founder of Enginuity Analytics here. You can ingest RSS and Tweets into our platform where we apply machine learning to provide social data, sentiment analysis, rankings, filters and demographic analysis of all the content for 27 languages. And easily export the data to Excel and more. http://theenginuity.com
Instead of closing it down, releasing the source and giving it to the community could breath live into it.
I remember being blown away by it, the fact that it was neglected and just kept on says a lot about it's good design.
We (Webflow) are actually considering building a Pipes alternative in the near future, and I'd love to sit down with you and brainstorm some ideas. If you'd be willing to make some time to see a very rough demo, I'd love to get in touch - my email is in my profile.
Several reasons I can think of. For one, no one at Yahoo seemed to understand what Pipes was for (part of a wider problem where Yahoo didn't seem to know what the company was for).
For another, Yahoo legal prevented us from doing any kind of long-term storage of RSS feeds. That made the service far less useful than it could have been.
Finally, they didn't give it any resources at all. Management was too preoccupied with the failed Brickhouse project (internal startup incubator) and the host of failed ideas that it spawned.
Filtering and triggering without the annoyance of having to write code in some syntax-of-the-month or dredge through poorly written documentation for half-ass APIs.
If you are redistributing, hosting or using it commercially you have to have the agreement of the author of the content. RSS just gives you access in another way than HTML, but it does not circumvent copyright.
This is an amazing product. Have you tried talking to VCs or anyone about someonr adopting it somehow?
I wonder if it would be possible to build something similar using IPV6/some kind of P2P/STUN/TURN/UDP hole punching/swarm/ethereum / any way to make a distributed system that does the same thing. Or maybe just volunteer running Node on their own VPSs.
There was IFTTT but looking at their page now it seems like maybe they have pivoted, at least I can't tell that it works like it used to: https://ifttt.com/
If you sign in all the previous functionality is still there. Seems their landing page is only advertising their new "Do" apps (which are a one-button extension of the old IFTTT recipes). Installed it, but never used them much.
It's amazing how much a shift of management focus can change things... Google embraces RSS, develops the premiere online RSS reader, so good that even stand alone readers are largely abandoned... then google nukes RSS support from Chrome, and shutters Reader. And the world is worse off for it.
It definitely inspires a "never forget" kind of mindset, such that I don't trust relying on a SaaS that I don't have a personal exit strategy for.
Zapier is the only reliable tool I found so far that can still import RSS to facebook pages, it tried it with pipes and it had issues with non-standard fields.
I liked the concept and probably didn't give it a fair shake, but it flooded me with notifications and I just didn't want to take the time to tweak it to my needs. Its one of those apps you need a good tutorial (from someone other than the creators) to walk you through useful functionality and setup.
The only thing Pipes would need is a .io domain and it could be dressed up as the Next Cool Thing which LargeCorp is desperate to get its hands on to prove it's still cool.
Yahoo pipes was obviously influenced by UNIX pipes and as such is one of the best demonstrations of flow based programming to date. I really loved Yahoo pipes and it influenced the way I think.
Pipes was a basic web "agent". It made basic programming available to the everyman much like HyperCard. Perhaps is just needed a runtime UI that matched the excellence of its design time interface.
Yahoo is focussing on mobile yet here they had this custom agent building tool that could easily be re-purposed to mobile to make a killer platform for Yahoo users.
Hopefully they will opensource the first generation perl version.
Some of my favourite flow based programming links follow.
Surprised nobody has mentioned noflow node.js workflows:
https://github.com/noflo/noflo
Also this (untested) will convert your yahoo pipes into Node.js
https://github.com/neyric/pipes2js
and this one (also untested) will do the same in python:
https://github.com/ggaughan/pipe2py
There is also python pypes which has a yahoo pipes like frontend with a stackless based backend.
ConMan: A Visual Programming Language for Interactive Graphics (1988)
Abstract
Traditionally, interactive applications have been difficult to build, modify and extend. These integrated applications provide bounded functionality, have a single thread of control and a fixed user interface that must anticipate every-thing the user will need. Current workstations allow several processes to share the screen. With proper communication between processes, it is possible to escape previous models for application development and evolution. ConMan is a high-level visual language we use on an IRIS workstation that lets users dynamically build and modify graphics applications. To do this, a system designer disintegrates complex applications into modular components. By interactively connecting simple components, the user constructs a complete graphics application that matches the needs of a task. A connection manager controls the flow of data between individual components. As a result, we replace the usual user-machine dialog with a dynamic live performance that is orchestrated by the user.
Most of what Pipes did can be done locally, in a library. FeedReader for Windows did this.[1] Unfortunately, they recently announced "Get ready for the brand new FeedReader! ... Check our site later this March for a new amazing web service, featuring industry-best RSS search and analysis.", which means it's becoming "cloud based" and probably has ads or worse. "rss.com" already went that way; they charge $6/month to read RSS feeds.[2]
Following RSS feeds on a continuing basis takes a lot of RSS polls. Most RSS feeds do not implement RSS in a way that allows getting only new items reliably.[3] The RSS "etag" mechanism is not reliable. Some sites with multiple servers and a load balancer have different etag values on each server. The "guid" field sometimes changes when the content hasn't changed. My experience is that nothing short of full text comparison eliminates duplicates properly. I wrote an RSS reader which does a MD5 of the text of each incoming message to throw out duplicates. Presumably the "pipes" system did something similar.
If RSS feed servers complied with the standard, there'd be less need for feed aggregation services.
I think your first paragraph speaks to a real tension in the world of software: web applications have enormous discoverability, immediacy, upgradeability, and (often) social collaboration advantages over native applications, but they have ongoing costs proportional to their popularity, so it's harder to run them without ulterior financial motives.
Aw man - that is sad. We at Zapier took great inspiration from Pipes, it was such a cool and powerful product! It was a shame to see it languish.
I'd sure love to pick the brain of anyone formerly involved in the project - it seems like there is a lot we could learn from the trail it blazed. My email in profile, obligatory beers/coffee/etc. offer.
You guys are fantastic. I use Zapier for several of my organizations and companies, and I cannot understate how incredible of a service you guys have created. <3 for days.
My only wish would be for a paid tier between 100 and 3000. If I went hog-wild and did everything I want to do, I'd probably use 200-300 a month. I would love to pay for that. But there's some loss-aversion psychology happening that makes me not want to pay for 3000 if I know I'll only use 300.
I'd give you my cc number right now if you did a 1000 zaps at $10/m.
That being said - you know your business better than I do. If it would hurt conversions of $20/m more than it would help, I get that. :)
I actually had the same query last year when I pushed a place I work at to use a combination of Yahoo! Pipes and Zapier for our Facebook postings. I asked for a smaller plan (we only need about 600-700 zaps/month at the most) and was just instructed by their support to utilize their reward program[0] which offered an extra 50-100 zaps/month based on what you 'complete,' and it included things like (IIRC) liking them on Facebook or following them on Twitter.
It seems like that program has since been limited, however, as it now redirects to a referral page[1] only. A combination of a couple referrals around the office and performing the required tasks gave us about ~650 monthly "zaps" at the time and we haven't needed more than that so far. Still, I would have happily paid $5/month or something to avoid the hassle (and be able to pay them for their wonderful product).
> ChangeTracker watches the White House’s web site so you don’t have to. Whenever a page on whitehouse.gov changes, we’ll let you know — via e-mail, Twitter, or RSS. But ChangeTracker is not a piece of software. It’s the output of a series of powerful and mostly free Web-based tools, lovingly connected over the Internet. Here’s how to do it yourself so you can track changes on any Web site on the Internets.
One pipe would grab data from Craigslist, strip out irrelevant items, and send me a text if anything new and interesting appeared. Ditto for Westside Rentals.
This combination worked great. If a lame apartment appeared in CL, I'd edit my Pipes regex to strip the same thing out of WR. In this way I'd get only texts for awesome places.
Ah, I did this too! I'd forgotten about it. As I was looking, I'd continually update it to block out streets I wasn't interested in, then eventually I was only getting notifications for exactly where I wanted for exactly how much.
I also used it with Freecycle and got a fridge-freezer! (you had to be quick to get them)
I was also using it exactly like this until recently! I sent my roommate the pipe and we'd each watch/browse it for new places. It saved us a ton of effort.
I've used this pipe as my Hacker News RSS feed for the last 5 years, which flips the link structure so the main link for each entry is to the comments.
The main one I made was to switch the actual link for the Reddit RSS feed to be the main link, set the true article title as the item title, use the submitted title as the item body and add a link to the link's domain in the bottom.
I still have 146 subscribers to keep happy, though Feedburner should make any transition transparent. I'm disappointed it's going. Thank you to those who made it!
I used Pipes as the basis of data mashup classes, with non-developers students that were mostly afraid/bored of anything related to programming.
Using Pipes, in just a few hours they would make small,
custom data apps. A typical example would be a student building a pipe gathering the weather from one source and trying to fetch the relevant clothes from a retailer with pictures and links. I had a student using the location entity extraction to geocode his favorite street art blog that cited cities in the text but didn't use rssgeo, and then output the augmented feed to a map view.
It was an eye opener for some: they could get, transform and repurpose data, and it could be fun.
Pipes wasn't without its flaws: V1 was buggy and gave us a lot of trouble when 30 students were trying to use it from the same IP, the interface was plagued by the usual issues of visual interfaces (clutter…). Some things were odd, some a little too hard for what they achieved and other were magically easy!
But at a moment when every tech company is saying that "learning to code" is important, it's sad to see a tool that had real educational value disappear.
It was a really effective tool for non-developers to learn about data markup standards, to think in term of data flows, to get introduced to the idea of a data or web API.
For this particular use in education I'm not sure there is a replacement.
Who is working on a pipes replacement and where can I sign-up? Maybe this will be like when google killed Reader and RSS replacements became a hot area...
I was looking into doing a Pipes 2.0 sort of project. If you live in the bay area i'd love to get coffee and chat about it. There are a number of the pieces to pipes that are opensource, but none of them are strung together.
I also thought that pipes queries were amazing, but slow, so it was hard to use them as part of an application.
We're working on something that covers a lot of the same ground, although it's not exactly the same thing. It's also not very polished right now, but we definitely hope to get to that level eventually. It's all open source, Apache Licensed, and built using Grails. We don't have YQL, but we extract entities from the content and store it in a triplestore that can be queried using SPARQL, and eventually the SPARQL support will be woven more completely through things.
Shameless relevant plug: http://www.feedsapi.com might be an option, it'sit's not exactly a replacementbfor pipes but it does a lot of what pipes did and is a work in progress
IFTTT doesn't let me basically reformat an RSS feed and then see it in my RSS reader. Most/all of my use of pipes was "go scrape this thing and turn it into RSS, filtering out stuff I don't want to see."
It was great for that - I'm unaware of anything that is even trying to replace it.
Just a shout out to the awesome Pipes community and some people that really helped it along the way. All the original creators (pasha, JT, ed, daniel), community members and team: hapdaniel, dawnfoster, kentbrewster, mirek, spullara, janluehe, dspark, kevink, brettp, lolo, laurencecoates, earth2marsh, psychemedia, davglass, ssaine, sadaf, ido, nagesh, ameya. I know i'm missing tons more, but just top of my head. You are all awesome.
Oh man, I use 3 or 4 pipes to filter various RSS feeds. Ugh. Good suggestions for replacements (especially those that don't require my own server) would be much appreciated.
YQL queries can do everything pipes can, you just have to write little scripts to do it instead of the UI
Argh! The UI was the innovative part. The whole point of something like this is that you don't have to learn a bunch of syntax - when you add a module, it tells you what sort of data it needs to know (such as a URL or...) and when you connect modules together it tells you what sort of inputs and outputs are available. This allows you to focus on the actual problem domain instead of the morphology of the query language, which is not important to the user.
I keep encountering this attitude among programmers - 'I type code all day, so why don't you want to do the same thing?' If people wanted to type stuff all the time the command line would still rule, but the majority of people prefer graphic user interfaces because they don't want to write code, they want to make selections from available options and use the computer to take care of the plumbing. This is a Good Thing. It's easier, more fun, and yields greater productivity, for the same reason that handing someone a box of Lego is better than handing them a few blocks of wood and a set of carving tools. The existence of Lego hasn't led to a collapse of sculpture or mechanical engineering. On the contrary, the simplicity, consistency, and interoperability of the different components has made a massive creative, educational, and commercial success.
I don't mean this as a hit on you personally, but I find comments like the above frustrating. Imagine you went into a police station to report the disappearance of your bicycle and the person behind the desk shrugged and said 'you're still able to run around on your legs, what's the problem? Your bicycle may have allowed you to go faster down the street but it's not like you could ride it up and down stairs so you should be happy with walking on your feet.' The problem is the loss of a tool that offered substantial efficiencies on some common tasks even though it wasn't ideal for every situation. You'd probably be equally pissed off if your favorite high level language went away and you were told you could just go back to writing assembler. I personally love doing things in assembler because it makes me feel smart but that doesn't mean it's better than high level languages.
Yeah, I wasn't suggesting that YQL was equivalent to Pipes, just that it might be able to replace the functionality. Which is why I made sure to mention the significant difference...
My understanding is that YQL is being used as the middleware between services internally. Will probably not be shut down, but they could shut off the publicly accessible one.
One thing not mentioned in this thread is that Pipes 2.0 was a rewrite from the original Pipes which was written in Perl. We ported it to the same engine that powers YQL.
As for why shut down Pipes if it shares infrastructure with YQL? Probably because usage is mostly RSS crawlers that no one looks at anymore.
I get the impression that YQL is suffering from the same kind of neglect Pipes did. It was down multiple times a week for me, and I had to move to other solutions.
While working for YDN I was the one who spilled the beans about the top-secret Pipes _callback parameter, which turned your JSON reply into JavaScript. The service fell over instantly, of course, but far from being upset with me, the team was thrilled with the exposure. Viva Pipes!
I use Yahoo Pipes to get around Netflix's walled garden.
Netflix does not believe in RSS, even though their Influencer content is not behind a subwall. I use Pipes to parse Netflix's json content and then turn it into a feed. I even created a Pipe that took any influencer ID and automatically created an RSS feed.
The idea was started because Daniel Tunkelang of Netflix was very anti RSS. I showed him, but alas, no more.
I never figured out how to use IFTTT. How can you grab any content from the web, like Netflix's json content? There is no HTTP input as far as I can tell.
I used a combination of Yahoo! Pipes and IFTTT personally. I'm not sure how people are using IFTTT on its own.
For me, I would do the data parsing on Yahoo! Pipes and prepare everything to be triggered, and then the actions would be performed on a service like IFTTT (where the instruction was to just post whatever new item came in the feed).
I remember well using and playing with Pipes when it came out early 2007. The service worked. I mostly played around with my twitter and flicker feeds. Here's some screenshots of what it looked like.
I have been a fan of Pipes for a while. Kudos to @psadri and others who made it possible with the state of web tech back in the days. It was a very well made tool both frontend and backend wise!
I have a emotional connect with Pipes as a tool. Right out of college, and before the whole API thing was pervasive, I had leveraged pipes for so many small projects and hacks. Infact I used it in a Yahoo Hackday hack and it landed me a gig at Y!
More relevant to the present, It has been something I have mentioned to so many people who have come to me discussing UIs for automation/control systems/or to manage workflows, processes. In a way it was what noflow's UI looks like, but years in advance. In a different parallel universe, it may be the way people use APIs, or right big Haddop/Spark/Storm jobs/topologies, and the front end was open-sourced back in the day, with full integration with hadoop for job management.
This a disappointment if not an unexpected one. I used pipes again just a few weeks ago to do some simple modification of feed but thankfully ended up taking another approach (not a pipes replacement I just wrote some code for it). Pipes was a really cool tool and it will be missed. Does anyone know of/use any alternatives?
In the enterprise space this concept is thriving and doing well. For example, SnapLogic has been building enterprise integration technology that can easily tie hundreds of different data sources together using pre-built or custom components using a GUI or programmatically. In order to become a critical part of business infrastructure it needs to be a professionally supported and maintained product. Yahoo Pipes was just an experiment and not a serious business so people never built anything serious on top of it that would pay for the underlying maintenance and infrastructure.
So sad to see Pipes go! I absolutely love Pipes and was just raving about it to a coworker this week. Terrific UI and a fantastic idea. When development fizzled out, I tried to find another service as a replacement, but everything I've tried has fallen short. I'm really disappointed as Pipes is definitely one of the best ideas that never caught on line it should. Popfly, while short lived, had the advantage of being able to modify the code behind, but it didn't come close to the fantastic ease of use and tremendous potential for reuse.
I always knew this day would eventually come. It's kind of sad.
I had created this years ago to display Twitter data using Pipes, YQL and Google Charts. I haven't had to touch it in years but it's always just quietly worked, and some people seem to like it:
Sorry to see Pipes being sunset. It had a major design and UX influence on a number of us during the early Web 2.0 days. Kudos to the Pipes folks for a seminal creation.
On a related note, a couple of us have been hacking on a Pipes like user experience around Data Integration and Analytics with Web feeds and APIs -- we had been planning a separate announcement but thought this might be a useful place to post if there are others who are looking for alternatives. Please message me directly - my contact info is in my profile.
If I was looking for a replacement, maybe I'd look for an RSS aggregation plugin for WordPress? Personally, I have my own PHP code that will slice and dice feeds however I want.
I got about halfway through building a "news blend" app where you tap just the categories you want and it would blend all the latest articles from just your categories into one news feed and email that to you daily. But we never finished it.
I'm really motived to create an opensource alternative as I was a big pipes user. I set up a repo (obviously empty for now) ,I'll try to come up with a minimal working app before yahoo pipes gets frozen:
github.com/interactiv/pipes
didn't decide yet what serverside tech i'll be using but quick deployement is a priority for me. If anyone's interested.
EDIT: a lot of useful information here about existing projects, thanks.
I loved yahoo pipes. I used it to filter high-traffic RSS to do a trivial word-match on content. I am now in the process of moving this to Newsblur, which allows filtering RSS content on tags, authors, or any word in the title. It's actually quite good, although we can't use the same filters for multiple feeds; at least it does the job.
I can understand why Pipes would be shut down by Yahoo, it was a impressive technical feat but it definitely appealed to very small developer focused crowd.
I feel like IFTTT is our best bet, while it doesn't do the "Unix pipe" that pipes offered it at least provides some usability to the process.
back when pipes came on the scene it was so out there but it was amazing. We couldn't fathom what kind of magic was making it all work and there seemed to be so many possiblities. Future bright etc. Of course, landscape changed etc...but it was / remains a display of what is possible with the web and kind of the 'idea' of an open web/apis/all that jazz.
I am still using it to repurpose an XKCD feed into a fully formatted feed with images that then is picked up by IFTTT and tweeted out as a tweet with images when new comics go up. I used Pipes just because it was point'n'click and seemed to have all the key components I needed to do the task without worrying about the code-route.
Sad but not expected. It was really powerful but, like most people, I stopped using it years ago. It kept breaking either due to the sites I was using it with changing things or the service being neglected (and my limited ability to understand how it worked!)
Yahoo Pipes was probably one of the best examples of letting non-programmers be developers. I created a lot of useful pipes back in the day for friends and family. Used a particular craigslist one for years.
Aw damn, that kinda sucks, but the project has been dead for a while now. I used it for content aggregation, but moved on to custom scripts, which were way more powerful...
Meetup.com has calendar feeds. You can get a calendar feed for all the events you've RSVPed "yes" to. You can get individual calendar feeds for each group's events. But there's no way to get a calendar feed of all the events happening in all of your groups.
I'm a member of something like 40 groups. The thought of manually adding each feed as a separate calendar in Thunderbird - and the thought of Thunderbird then trying to cope with the result - was... unpleasant.
So I set up a pipe to combine all the feeds into one. Very cursory googling suggested that Yahoo Pipes was the easiest way to accomplish this.
Fairly slick UI. Unfortunately it turns out that meetup.com apparently serves 503s to requests coming from Pipes.
Unfortunately it turns out that meetup.com apparently serves 503s to requests coming from Pipes.
That's really interesting. I think it exemplifies a problem that has been bothering me for a while: for many sites, scripting and design are antipatterns designed to corral rather than enable the user and leverage branding.
I used a pipe that created RSS feeds from Google+ pages in conjunction with some other stuff to get all the articles I'd want to read in one document (with a table of contents) delivered to my kindle.
I'm now working on something that doesn't use Y! Pipes and simplifies the process.
We all knew it was coming, but the end of Pipes still sucks.
By the time I turned the west coast back online, the east coast was failing health checks, and the LB failed everything over to the west coast, which then proceeded to be overloaded, and everything flipped back to the east coast... Ping ponged for about 2 hours until they finally settled.
Fun times.
Another fun story: For the longest time the front page had an example pipe that merged search results from various online sites (amazon/ebay/cl). It was made by a former employee and was easily one of the most popular pipes. One day we found out he had his affiliate id in all of those links. We chuckled and moved on.