- Unlike all existing ETL tools, we're code-first and Git centric. So we're perhaps we are the first Data Engineering tool.
- We want to give Apple like high quality design, and it-just-works experience.
We've built a unique CODE=VISUAL IDE where you can toggle between visual and code editors instantaneously. You can edit code, and the changes show up in visual graph, and vice-versa. So even if you use visual drag-and-drop view to edit a workflow, you're doing Git commits. You can just go to the underlying Git folder and do maven build. Same for unit tests. We've added one-click spin up of Databricks cluster from IDE and step-by-step execution.
We want to enable you to:
- Develop 10x better code for Spark
- Develop 10x faster on Spark
- Deploy to production 10x faster with CI/CD on Spark/Airflow.
For the beta, when you submit your e-mail, we create an account and send you login details. You can create Spark workflows, spin up clusters (in-built, on Databricks, free - please be kind with usage) and execute these. Please try out and give feedback, help us make data engineering better!
However, for our Enterprise version, we install within your network and this supports EMR, Dataproc, Databricks.
This is only relevant where we are spinning up (and down) Spark clusters from within the IDE, you can always connect to an existing cluster from any distribution.