Wow, another neat Cloudflare release. Is there some sort of "innovation week" type thing going on right now?
One question that the example given raises for me, is how to manage that function I see there in a robust, "proper software engineering" sort of way. This has been a challenge for me on AWS when trying out Lambdas. The individual pieces are cool, but the system that connects them all together is a little opaque. How do you version control the code? How do you keep track of which pieces work with which other pieces?
We use terraform to manage the AWS resources, but it's kind of clunky, and not official. Is there some Cloudflare-supported way to maybe declaratively define your different resources?
Someone who has built a somewhat complex app with a lot of workers, and durable objects, and different pieces of the ecosystem: how do you manage that? How do people review changes?
Since it sounds like from another comment that Cloudflare dogfoods its own services internally, I'm curious if there's a write-up about how they actually do so? The little function code snippets give a nice taste of the code a worker might run, but that's still kind of a long way clearing up for me how you would actually use this in practice in a standard production environment.
At least on my team at Cloudflare, great Terraform support is a basically required part of any API change. We have an official Terraform provider (https://registry.terraform.io/providers/cloudflare/cloudflar...), and nearly every API feature is provisionable using that provider.
Cloudflare is really fast putting the pieces together to become a modern simpler AWS. By focusing on edge networks and programmability from the start they've been able to outmaneuver AWS in product development and pricing.
AWS today is stable, reliable but expensive and rigid. A lot of their products feel like they've been taken out of the oven 15 minutes too early. AppSync, API Gateway, CloudFormation and others all feel lethargic, stagnant or not fit for purpose in the current world.
I don't think AWS is in a position to compete with Cloudflare's strategy without massively slashing bandwidth prices and improving their product development. They also aren't really a credible edge platform. Lambda@Edge is really bad, CloudFront is slow and terrible compared to Akamai, Fastly and Cloudflare (and Amazon's retail business doesn't even use it -- because it sucks).
I guess from my perspective that's also why I want it in my cloud. I'm willing to pay for managed services, but don't want to be using services I can't easily migrate away from.
If Akamai were smart they'd buy Fastly yesterday, because they're rapidly falling behind. Cloudflare is harder to see because of antitrust concerns. With Fastly's current position, they should sell, and someone should buy them for the network and progress on edge compute alone.
Pub/Sub, Workers, KV, Durable Objects, R2... yes, these all sounds promising. Awesome tool for developers. But there is one huge obstacle in the middle that prevents most to actually try this stack.
Logs and Account Access control.
Cloudflare is a very powerful platform, you can't let everyone be Admin.
If you run anything, you need to be able to see logs it to ship it somewhere.
For some reason, Cloudflare thinks that these are enterprise feature. I got news for you, these are basic features for anyone who wants to build more than just a simple script for individual use.
You work so hard to build these features but let only a small subset of users really do anything meaningful with it
Yes, fully agree, just to give Fred access to develop a worker in dev environment we have to grant them access to the whole account, it's batshit insane. We created acmecorpx.com for dev and put this on a seperate CF account but it still sucks
Honest question, not a snark attack. But would it be possible to build those things on top of Cloudflare itself, esp the logging? For access control, you would have to have a separate process controlling the CF api for configuration.
Not so simple when a lot of the tooling that facilitates easy development is tied to CF API, are you meant to build your own cloudflared and wrangler just so you dont have to grant everyone in the org super admin?
This is where AWS wins, AWS gives you everything from $0/month, IAM in the above case, but you don't stay at $0 for long, where as CF gives you partial bits but you remain at $0-low/pm for longer
Its actually really ironic the same company that says SAML for all for free, can't even get basic RAC on their own admin controls
Just provide an Aws IAM type interface and everyone would be happy. Also don't charge for it, because that just states your systems literally were not designed with authorization in mind.
FWIW with AWS, despite the naming, the IoT Core MQTT broker is an excellent product for exactly this - it's just sold as IoT specific. It's really just an MQTT broker at it's core and could probably be split off into it's own product without the IoT branding.
Then you have to go through the whole process of setting up, routing, and registering an HTTP endpoint, rather than you would as a normal pub/sub simply have the program connect to the publisher and start receiving messages/events.
In the MQTT world, in which Pub/Sub is based, every publisher is a client, and every subscriber is a client. You can be both, too!
Assuming your existing backend is in $SOME_CLOUD_PROVIDER, publishing messages to Pub/Sub is just a few lines of code in your favorite language - we have a couple of examples up in JS + Rust - e.g. https://developers.cloudflare.com/pub-sub/examples/connect-j...
Subscribers receiving those messages can be anywhere you can run an MQTT client, which turns out to be... a lot of places :-)
(I'm the PM for this, and we'll be adding more examples to our docs in the coming weeks)
I like that Cloudflare is steadily becoming a serious alternative to the existing cloud providers. If you folks can build a decent IAM tool for access control to Cloudflare resources, much more people will start using your services.
(I'm the PM for this at Cloudflare & the blog author)
We looked at a number of existing options — trying to strike a balance between languages we run at scale (tooling, ops, security), memory safety, and long-term maintainability as a growing company.
We ended up writing an MQTT broker from scratch in Rust, with our own Durable Objects[1] for consistency and co-ordination across our hundreds of edge locations (and tens of thousands of machines). The MQTT spec itself[2] is very approachable, which made this undertaking easier still.
One of the increasingly cool parts of Cloudflare is that we use more and more of what our customers use: our Broker service sits behind Spectrum[3], and we use Workers + DO for all of the metadata/session handling. The next set of new products will invariably start to build on Pub/Sub itself, too :-)
Some time ago I was at a talk about verifying different MQTT brokers so they fulfill the spec.
They presented several inconsistencies they found (in OSS brokers) and submitted fixes for.
>>> And, since we're building on the redundant storage of Durable Objects, your database can physically move locations as needed, resulting in self-healing from even the most catastrophic problems in seconds.
One question that the example given raises for me, is how to manage that function I see there in a robust, "proper software engineering" sort of way. This has been a challenge for me on AWS when trying out Lambdas. The individual pieces are cool, but the system that connects them all together is a little opaque. How do you version control the code? How do you keep track of which pieces work with which other pieces?
We use terraform to manage the AWS resources, but it's kind of clunky, and not official. Is there some Cloudflare-supported way to maybe declaratively define your different resources?
Someone who has built a somewhat complex app with a lot of workers, and durable objects, and different pieces of the ecosystem: how do you manage that? How do people review changes?
Since it sounds like from another comment that Cloudflare dogfoods its own services internally, I'm curious if there's a write-up about how they actually do so? The little function code snippets give a nice taste of the code a worker might run, but that's still kind of a long way clearing up for me how you would actually use this in practice in a standard production environment.