A lot of name dropping and unprovable statements. I'm really interested by the domain and progress, but Airflow needs to be more generous in real information and less in marketing bs. Can someone share a more introductory article about what makes Airflow different from the current state of the art?
Here is, a slightly outdated, article that compares several ETL workflow tools http://bytepawn.com/luigi-airflow-pinball.html . Why we choose Airflow was because of the following reasons:
* Scheduler that knows how to handle retries, skipped tasks, failing tasks
* Great UI
* Horizontal scaleable
* Great community
* Extensible; we could make it work in an enterprise context (kerberos, ldap etc)
[author] sorry you feel that way. I understand that the section on other similar tools is controversial, especially if you work on one of those. I'm repeating what I hear being very active in this community, and answering the question that was asked to the best of my knowledge. I'm open to editing the article with better information if anyone wants to share more insight.
As mentioned about Luigi, I do not have the whole story, one fact I know is that someone from Spotify gave a talk I did not attend in NYC at an Airflow meetup, and I've heard the original author had left the company. Those are provable statements, happy to debunk on the article if needed.
