Any Yahoo Pipes true substitute out there ?
Tools for RSS feed merging and filtering
Mercredi 30 novembre 2016, par Logiciels, Internet, moteurs de recherche//
The loss of Yahoo Pipes in August 2015  was a major blow to the monitoring/research community and also web site authors looking for selected content. Its ability to combine and filter multiple RSS feeds into one feed was unique among free hosted services.
So I went looking for substitutes which could merge and filter feeds. I mean, real ones : as simple as Pipes. For instance, they shall have a graphic user interface (GUI) ; coding shouldn’t be necessary. The only difficulty allowed should be the use of regular expressions (Regex).
Free hosted substitutes
- FeedCombine, RSSMix, FeedRinse or ChimpFeedr (to merge feeds into one feed) then FeedSifter or FeedRinse (to filter the mega-feed) can answer simple needs.
FeedRinse can do more, as it accepts Regex but it can’t merge more than 5 feeds, is slow and bugs according to La Bibliothèque du CHUM’s test.
FeedCombine cannot combine more than 5 feeds, say Anik Dumont-Bissonnette.
ChimpFeedr is very simple at "chomping" (as their web site says) feeds.
RSSMix, according to the same test, works perfectly, merging up to 100 feeds
- Feed Informer combines and filters feeds, but filtering can only be done using AND or OR operators
- IFTTT doesn’t seem to do exactly that : merging and filtering RSS feeds into one feed. Which is not to say it does not have its (numerous) advantages in the field of monitoring services.
Paid hosted substitutes
- FeedsAPI does not seem to merge RSS feeds but can filter them. Afterwards you can use an IFTTT recipe such as Feedly Category to Twitter. FeedsAPI and Inoreader seem the easiest to use solutions among the paid ones
- Tadaweb from Luxembourg, according to some blog posts, is able to create from web pages, filter and organize RSS feeds. The ouput is in RSS format only if the input is RSS too. NB : following a recent new policy, Tadaweb does no longer give access to its Creator module, its team is doing the work in lieu and place of the information professional and its pricing has accordingly evolved
- the paid version of dlvr.it can filter and mash up RSS feeds
- We Wired Web offers an IFTTT-like interface, RSS in and out and is able to filter it. But it does not merge different RSS feeds
- the VIP and Premium versions of the Netvibes RSS portal are able to merge and filter RSS feeds by keywords, using Universal Search (which allows to query only certain groups of feeds — they’re called "tabs") and then Tracked Topics with RSS export
- the Plus and Pro version of the RSS reader Inoreader  allow merging and filter features throught the use of "active searches" : create a folder of feeds then an "active search" (an automated keyword search) on it and finally right click on the active search and click "Get RSS feed" . It’s a pity Inoreader filter rules does not seem to offer RSS feeds as an outcome ; it would be interesting because those filter rules are much more precise than actives searches since they accept complex boolean queries and Regex  .
Self-hosted solutions (need coding skills)
In the sources listed below, I found other free or paid total or partial replacements — but you have to host them on your own server and/or they’re far too complicated for the average information professional :
- Tiny Tiny RSS (TT-RSS) is an RSS reader with very interesting RSS feeds filtering features (open source)
- Huginn by cantino (open source on GitHub). According to a MakeUseOf article, « this is the most popular of all the GitHub solutions mentioned, and has a huge, active support community behind it. It’s well documented and easy to use »
- WebHookit, based on Node.js and mongoDB (open source)
- Pypes (open source on GitHub), a Python application
- Rss Percolator (open source on GitHub). Downloads, aggregates and filters RSS feeds
- ClickScripts (open source on GitHub)
- Bipio (open source). See the Getting started page (developer documentation) for a quick presentation. Bipio is a graph pipelining API talking RESTful JSON
- Neddick (open source on GitHub).
- pipe2py (open source on GitHub). You need to backup your YP pipes before. It uses JSON and the Google App Engine
- Pipes2js (open source on GitHub)
- Superpipes (open source), to be hosted on a Debian server
- open source edition of MuleSoft’s Anypoint Studio
- Quadrigram seems too graphics-oriented to me (apparently free).
RSS is important to information professionals and, as we’ve seen, much used by people whose job is to monitor web resources. But this does not prevent RSS implementation from receding. If you look into the list of applications which can be connected by IFTTT, you will find that RSS is just one among more than a hundred of mostly proprietary ones ... Walled gardens again.
Did I miss something ? Do you know about any simple tool that could replace the "RSS mashup" function of Yahoo Pipes ? I’m interested in free as well as paid for services.
law librarian researcher, open law specialist and monitoring solutions specialist
Sources for this article
Here are the links I found to some articles/blog posts and a number of forum discussions about Yahoo Pipes substitutes :
- Par quoi remplacer Yahoo Pipes ? : Grille d’évaluation d’outils pouvant fusionner et filtrer des fils RSS (in French) / Anik Dumont-Bissonnette, La Bibliothèque du CHUM 22 June 2015
- Modèle d’une plateforme de veille visant à automatiser la rediffusion de l’information (in French) / Serge Courrier, November 2016. See especially pp. 20-21 and 35-37
- The 13 Best Yahoo Pipes Alternatives You Should Look at Today / Angela Alcom, MakeUseOf 8 June 2015
- Yahoo Pipes s’arrête… Par quoi le remplacer ? (in French) / Korben, 5 June 2015
- TaDaweb, un outil de veille innovant et aux fonctionnalités impressionnantes (in French) / Christophe Deschamps, Outils Froids 28 February 2013
- Need a Yahoo ! Pipes Alternative ? Your Search Is Over / Bill Flitter, dlvr.it blog 11 June 2015
- Does Netvibes provide exportable RSS ?, Netvibes Knowledge base
- How-To : Universal Search for VIP, Netvibes blog 3 April 2014
- Inoreader, enfin le digne successeur de Google Reader ? (in French) / Corinne Habarou, UrfistInfo 24 November 2014. Do read Serge Courrier’s comment
- Yahoo Pipes End-Of-life Announcement (forum thread), Hacker News 6 June 2015
- Yahoo Pipes : Any good alternatives out there ? (forum thread), Yahoo Developer network 4-7 June 2015
- Is there a decent alternative to Yahoo Pipes (forum thread), Stack Overflow 13 April 2010-7 July 2011
- Is there any good alternative to Yahoo ! Pipes ? (forum thread), Quora 6 May 2012-...
- What paid/commercial alternatives to Yahoo Pipes exist in the market today ? (forum thread), Quora February 2011
- Alternatives to Yahoo Pipes ? (forum thread), Buzztouch 6 April 2014
- Alternatives to Yahoo ! Pipes, Meta-Guide.com (bot created directory) (regularly updated)
- What are some open-source alternatives to Zapier ? (forum thread), Quora 8 October 2013-9 August 2014
- 5 Intuitive Solutions for Integrating Cloud Apps, 5 August 2014
- Bringing ETL to the Masses with APIs, API Evangelist 10 February 2013.
This article was written with the the help of Serge Courrier.
Notes de bas de page
 Starting end of August 2015, one was no longer able to create new pipes. At the end of September 2015, the service closed.
 Yahoo Pipes could do much more but we will skip that here.
 To easily scrape web pages that do not have an RSS feed and so create their RSS feed, use paid services such as the simple, hosted and often recommended Feedity and FeedsAPI, the hosted but complex Feed43 — you need to master regular expresssions (Regex) —, or the self-hosted Feed Creator by FiveFilters.org.
 In the Plus version, active searches are limited to 20. The Pro version makes them illimited. Thanks to Marjolein Hoekstra (CleverClogs) for pointing me to this Inoreader feature. See her comments here under.
 Les filtres RSS dans Inoreader : détail de la syntaxe à utiliser (in French) / Serge Courrier, 30 November 2016.
 CloudWork, Zapier or elastic.io (paid for, IFTTT-likes) are not able to do the job — I had a close look at the list of their standard "actions" with RSS. The thread on Hacker News says the same with different words. They know how to transform one RSS feed into something else or sometimes into another RSS feed but they don’t know how to merge multiple RSS feeds into one feed :
- Zapier has a "zap" named "Curate RSS Feed into Your Own RSS Feed" but it does not take in multiple feeds : "Create a customized, curated RSS feed from an already existing RSS feed" says the recipe
- Cloudwork has no RSS to RSS "integration" and seems to be dead (see this post’s comments)
- elastic.io has no RSS to RSS "connector" either.
 See page 20 of these slides.