Cool! I'm working on something on the other side of the same pipeline. Real time data ingestion and identity resolution, delivering into a data lake.
Have you considered adding source connectors for S3 based data lakes? For example Parquet files or Delta Lake? Maybe via AWS Athena to make it similar to the Red Shift connector?
Yes! S3 as source connector is in our roadmap. Should be out soon. By connecting to S3 as a source & using Athena, data models can be created & synced to various business apps. Files in Parquet format is supported through Athena.
Have you considered adding source connectors for S3 based data lakes? For example Parquet files or Delta Lake? Maybe via AWS Athena to make it similar to the Red Shift connector?