To me this sounds like a way to avoid the hassle of creating a lambda to buffer streaming data into chunks before HTTP POSTing them to the S3 API.
But to me it wouldn't make much sense to use Kinesis Firehose unless the fees were cheaper than what it would cost to utilize AWS Lamdba for the same work. I mean it can't be all that many lines of nodejs code to pop events off a stream, flush them into a tempfile in batches, HTTP POST those batches to S3 with appropriate error handling/retry logic.
Admittedly I haven't crunched the math but just by eyeballing their pricing I suspect it may be more expensive than using your own lamdba. I wonder if Kinesis Firehose is implemented internally as an AWS Lambda, it wouldn't surprise me.
I suppose that eventually the pricing on this service will drop once someone open sources such a lamdba, especially since installing a lambda is so remarkably simple.
The only complication with this approach right now is that Lambda scripts can't communicate with anything inside a VPC (such as most Redshift instances). I imagine they will fix this issue in the future though.
But to me it wouldn't make much sense to use Kinesis Firehose unless the fees were cheaper than what it would cost to utilize AWS Lamdba for the same work. I mean it can't be all that many lines of nodejs code to pop events off a stream, flush them into a tempfile in batches, HTTP POST those batches to S3 with appropriate error handling/retry logic.
Admittedly I haven't crunched the math but just by eyeballing their pricing I suspect it may be more expensive than using your own lamdba. I wonder if Kinesis Firehose is implemented internally as an AWS Lambda, it wouldn't surprise me.
I suppose that eventually the pricing on this service will drop once someone open sources such a lamdba, especially since installing a lambda is so remarkably simple.