
Cloudflare launches Workers Unbound, next evolution of its serverless platform - skolos
https://blog.cloudflare.com/introducing-workers-unbound/
======
redm
Note: If I come across as a hater, in my opinion, I am not. I'm a fan, a fan
who is frustrated!

We moved to Cloudflare from a larger CDN and used workers to duplicate the
functionality otherwise not provided by Cloudflare. The initial results were
great. As our workers became more complicated, especially when interacting
with CloudFlares internal stack, the cracks started to show. Specifically,
when you run into a problem, the documentation being too light to resolve
corner case issues yourself, and support draws a big _zero_ when it comes to
Workers. Even as an Enterprise customer, it can take months to get a clear
answer/resolution, and that's if you nag them.

As a platform, workers have great promise, and Cloudflare is making strides.
As with most things CloudFlare, they could spend more time improving
documentation, and exposing details on their internal stacks so you can self-
service and resolve your issues. What I find is that Cloudflare is on to the
next thing before fully completing, polishing, documenting, the previous three
things they started.

For Workers, I'd like to see a trace tool built for Workers where you can
debug what happened in a Live environment. Currently, they recommend
outputting debug as JSON in a header.

The linked article is worthless. See the blog post, as this poster mentioned:

[https://news.ycombinator.com/item?id=23965764](https://news.ycombinator.com/item?id=23965764)

~~~
eis
> For Workers, I'd like to see a trace tool built for Workers where you can
> debug what happened in a Live environment. Currently, they recommend
> outputting debug as JSON in a header.

That is a general problem I have had with Cloudflare over the years. It's hard
to know exactly what happened when things fail or even the fact that things
failed!

They always branded themselves as that super scalable and stable CDN or DDoS
shield that will keep your service up when it would normally break down. And
those absolutely horrible "You <-> Cloudflare OK, host down" error pages that
suggest it's the host that has the problem and at the same time advertise
Cloudflare to your visitors. Well turns out the vast majority of issues that
clients reported (automated or not) were caused by Cloudflare themselves. And
what really makes it painful is that it's 1. sometimes hard to debug 2. they
wont make this visible to you unless I guess you pay for the highest
Enterprise plan which offers raw logs. They just don't tell you even how many
requests failed. There's no simple error.log. An example: at some point
Cloudflare started blocking a lot of requests from Turkey to an API. The only
way to solve it was to whitelist Turkey as a country.

We've moved off of Cloudflare and had a lot fewer issues.

PS: every feature since the simple CDN stuff seems to be now about lock-in.
It's all Cloudflare specific. This is a dangerous path that moves away from
the commoditization of "hosting" where people can easy move from one hoster to
another. I hope it does not succeed. For the sake of the open web.

~~~
rattray
Who did you move to from Cloudflare?

~~~
eis
For CDN we just put servers in different geological locations, some using
cloud service and some dedicated. In fact the system itself was made as
distributed wherever possible.

For the anti DDoS: it was not a big/common issue and in the rare cases we can
work with providers to mitigate.

~~~
snissn
how do you distribute between your servers? DNS? what DNS service?

~~~
eis
We started with GeoDNS (e.g. AWS Route53) and later on added Anycast into the
mix. The combination of these two is very powerful but I wouldn't recommend
Anycast for projects without good network staff.

For most, I think GeoDNS is good enough and very simple. The number of users
using nameservers that break it (like Cloudflare's 1.1.1.1)* is still small.

*: The reason why it breaks is because they don't send along the EDNS client subnet to the upstream nameserver for "privacy reasons". I disagree with that notion because 1. it's just the subnet 2. the website/service will see the full client IP anyways when it gets the http request. I mean how many times does one resolve a hostname and then not connect to it? And we are talking about the authorative nameserver, not the ISP nameserver.

------
colinclerk
Nice to see more and more pushed to the edge, and good that it's only the
beginning of the week.

I'm a little surprised they stuck with the isolates model when moving to
include generalized compute. We found isolates to be ideal for a cache layer,
but it doesn't have first class support for many languages:
[https://community.cloudflare.com/t/native-golang-support-
for...](https://community.cloudflare.com/t/native-golang-support-for-
workers/65896)

Very interested to see if they expand on their storage options throughout the
week. Additional consistency guarantees in their KV offering would make it
more competitive with Dynamo. I'd love to see a managed relational offering
like Cockroach, or perhaps the addition of CockroachCloud to their bandwidth
alliance.

One point I'm disappointed about is that they list API Gateway and DNS Queries
(per MM requests) at $0.

This may be true if you're deploying on a single domain with Cloudflare, but
if you're a SaaS and need SSL for custom domains, Cloudflare bumps you into
their enterprise offering that costs a minimum of thousands per month, and (we
found) netted out to about $5-10 per month per hostname:
[https://www.cloudflare.com/ssl-for-saas-
providers/](https://www.cloudflare.com/ssl-for-saas-providers/)

(I recognize this is a different way of metering, but listing both at $0 feels
like they're arguing DNS & TLS termination cost nothing at Cloudflare, and
that was far from our experience)

------
insomniacity
Surprised nobody is commenting on the zero millisecond cold-start time:

> The way the team engineered this is by queuing up the process while the two
> servers are still negotiating their TLS handshake.

That's pretty cool. I'm trying to see a downside, given that presumably it
won't spin up a function for traffic from IPs it knows are abusive.

~~~
Ericson2314
Hehe and socket activation reaches the cloud.

------
ignoramous
The private beta sign up form is here for ones interested in Workers Unbound:
[https://www.cloudflare.com/workers-unbound-
beta/](https://www.cloudflare.com/workers-unbound-beta/) (disable adblockers
to see the form). One key thing that sticks out is Cloudflare would charge for
outgoing bandwidth at $0.09 per GB, a stark departure from unlimited bandwidth
stance they're known for, especially wrt Workers Bundled [0].

A few questions, if anyone from Cloudflare is here:

0\. Can we expect in Workers Unbound support for Websockets, WebRTC, HTTP
Connect, and Server Sent Events?

1\. Would there be support for protocols other than HTTP? Say, a raw TCP / UDP
connection that is handed off by Spectrum to a Worker [1] (Spectrum here is
redundant?)?

2\. Inability to rate limit Workers in a cost effective way gives me sleepless
nights [2]. The current rate limiting plans are too expensive when compared to
Workers, almost 10x if expected _good requests_ are to the tune of 10 million
[3] (in my case good requests have potential to be much much higher). Any
announcements due in this space?

3\. Will the same isolate (Unbounded Worker), if kept running for long, be
able to handle multiple connections/requests concurrently, or would each HTTP
connection create a new isolate (that is, a new Unbounded Worker instance)?

Thanks.

[0]
[https://news.ycombinator.com/item?id=20791660](https://news.ycombinator.com/item?id=20791660)

[1]
[https://news.ycombinator.com/item?id=21921821](https://news.ycombinator.com/item?id=21921821)

[2] [https://community.cloudflare.com/t/how-to-protect-
cloudflare...](https://community.cloudflare.com/t/how-to-protect-cloudflare-
worker-from-ddos/179017)

[3] [https://support.cloudflare.com/hc/en-
us/articles/11500027224...](https://support.cloudflare.com/hc/en-
us/articles/115000272247)

~~~
kentonv
> Can we expect in Workers Unbound support for Websockets, WebRTC, HTTP
> Connect, and Server Sent Events?

Definitely yes on WebSocket. In fact, the only reason we haven't rolled out
WebSocket support already is because it's not very useful without long-running
CPU. Workers Unbound fixes that so I'll be dusting off my old WebSocket
implementation PR soon.

I think Server Sent Events are just streaming HTTP? That should work today
(but with the same caveat that CPU limits may cause trouble).

WebRTC is a peer-to-peer (browser-to-browser) protocol. We don't currently
have plans to implement it directly in Workers, though that's an interesting
idea.

By "HTTP connect", I assume you mean the CONNECT HTTP method, usually used to
tunnel TLS connections through forward proxies? That's a request I haven't
heard before, but it's interesting. I guess what you'd get, basically, is the
ability to establish TCP-like connections that are addressed to HTTP URLs and
connect over port 80/443? Is this commonly used for things other than forward
proxies?

> Would there be support for protocols other than HTTP? Say, a raw TCP / UDP
> connection that is handed off by Spectrum to a Worker [1]?

Not part of the current plan but definitely something we'd like to do
eventually.

> Inability to rate limit Workers in a cost effective way gives me sleepless
> nights

Yeah, our rate limiting product is not really intended to be used to rate-
limit the whole site to control costs. It is more intended to, say, rate-limit
your login form to prevent brute-force password guessing.

That said, we do have layer 7 DDoS protections, and those run in front of your
worker. If an attack gets past them and hits your worker, I'd encourage you to
raise a support ticket asking for compensation for the attack traffic that we
failed to block. I am not personally in a position to promise that we'd refund
you but I believe it has happened in the past.

> Will the same isolate (Unbounded Worker), if kept running for long, be able
> to handle multiple connections/requests concurrently,

JavaScript is inherently single-threaded, so an isolate can only be executing
code on behalf of one request at a time. That said, it is already the case
that multiple concurrent requests may be handled by the same isolate (one
request may be executing while another is e.g. waiting for a response from a
remote server). But today you can't really take advantage of that, because you
have no control over exactly which isolate receives any particular request, no
any way to communicate between neighboring isolates. This is definitely
something we're working on but nothing to announce at this time.

~~~
jbergstroem
> JavaScript is inherently single-threaded, so an isolate can only be
> executing code on behalf of one request at a time. That said, it is already
> the case that multiple concurrent requests may be handled by the same
> isolate (one request may be executing while another is e.g. waiting for a
> response from a remote server). But today you can't really take advantage of
> that, because you have no control over exactly which isolate receives any
> particular request, no any way to communicate between neighboring isolates.
> This is definitely something we're working on but nothing to announce at
> this time.

I've seen this used in a few contexts, for instance batching requests to a
origin (logflare: [https://github.com/Logflare/cloudflare-
app/blob/master/worke...](https://github.com/Logflare/cloudflare-
app/blob/master/workers/worker.js)).

Is this generally seen as ok? I'm planning on building similar things for
sending page metrics in parallel to serving traffic.

> > Inability to rate limit Workers in a cost effective way gives me sleepless
> nights

Agreed, there is something very challenging with the current design of how
requests flow through cloudflare.

I have played around with dynamic rules in firewall by manually handling
"firewall rules" in a worker as part of a request flow and using KV to share
this state. I would then update the cloudflare firewall with api. I had to
create a separate service to poll both API and KV to maintain them though.

~~~
kentonv
> Is this generally seen as ok? I'm planning on building similar things for
> sending page metrics in parallel to serving traffic.

It's OK, but you won't see a ton of benefit unless you have a very large
amount of traffic -- enough that you're commonly seeing single isolates
handling concurrent requests.

We're working on something that should be a lot better for this.

~~~
ignoramous
> _It 's OK, but you won't see a ton of benefit unless you have a very large
> amount of traffic_

Our traffic patterns are such that per active-user we would add 100K requests
per month when we go live. We send metrics out batched every 10s or so, but
stumbled upon this post that warns of internal limit on total number of
requests originating from Workers per second set at 2000, account-wide [0].

Is there a way to discuss our usecase and get whitelisted for limits as our
traffic isn't malicious, hopefully without having to subscribe to the
enterprise plan? I raised a support-ticket but a bot replied that since we are
"free customers" it hasn't be been looked at.

[0] [https://community.cloudflare.com/t/workers-and-sub-
requests/...](https://community.cloudflare.com/t/workers-and-sub-
requests/151829)

~~~
kentonv
> limit on total number of requests originating from Workers per second set at
> 2000, account-wide [0].

Oh, there is no such limit.

The post you link to is quoting another poster who is quoting something they
heard from someone else. It sounds like something that may have originated
with someone hearing about a particular implementation detail of our anti-
abuse systems but the game of telephone has transformed it into something
that's just not true at all.

As described in my comment on that thread, we do have certain anti-abuse
heuristics which might result in a 1015 error. It's very unusual for people to
hit these in legitimate use. Have you seen such an error? 100k requests per
user per month doesn't sound like it would come anywhere near being an issue
(even with 10M users).

FWIW the heuristics are designed such that simply adding more users (who
behave the same as existing users) shouldn't ever cause a problem.

~~~
ignoramous
> _100k requests per user per month doesn 't sound like it would come anywhere
> near being an issue (even with 10M users)._

Nice. Thanks! That's re-assuring.

> _It sounds like something that may have originated with someone hearing
> about a particular implementation detail of our anti-abuse systems but the
> game of telephone has transformed it into something that 's just not true at
> all._

A Cloudflare employee engaged in this thread [0] confirmed existence of this
internal limit: "...basically 2k subrequests _per minute_ per colo / zone /
IP. Once you hit that you start serving 429s.", which another customer hit
with ~5M requests per day (per a screenshot shared later in the thread) and
had to get whitelisted for.

[0] [https://community.cloudflare.com/t/massive-rate-limiting-
iss...](https://community.cloudflare.com/t/massive-rate-limiting-issues-with-
worker-in-production/128287/10)

------
pier25
I've run into 3 big limitations when using Workers:

1) The CPU limit.

2) Not being able to use Node modules.

3) Not being able to add domains other than adding those to our CF account.
SSL for SaaS solves this but, so far, it's only available to enterprise
customers.

The CPU limit was not so bad when considering Workers were not intended as a
general use-case for serverless functions. Now that this has been lifted I
feel the Workers environment limitations are going to be much more important.

You can't, for example, use many Node packages in Workers since these are
running in a custom V8 environment which resembles browser service workers. I
say "resembles" because the API is not 100% identical. For example, you can
use streams, but you don't get the full API.

~~~
jFriedensreich
A different runtime from nodejs is more a feature than an issue. The worker
runtime is much closer to deno than to nodejs from a philosophy standpoint and
i expect a lot of synergies between cf workers and deno in the future. the cf
workers being as much compatible with the service worker api as possible makes
them the only faas solution close to a web standard, so vendor lockin will
only be an issue until compatible offerings appear. A killer argument for this
approach is also that you can be sure you can run the cf worker code also in a
browsers service worker (with minimal modification). This allows more flexible
architectures and also more unified development between backend and frontend.

~~~
jFriedensreich
Also regarding the domains: cloudflare routes the requests to the edge centers
based on dns anycast, so cloudflare workers cannot really be seperated from
the dns functionality of cloudflare, see it as a faas offering inside your
worldwide distributed dns. If that is not what you want, it is still possible
to use cf workers with worker domains and use your own domain and dns setup

~~~
pier25
My point is not so much about the tech but actually that SSL for SaaS is out
of reach for most startups or small tech companies.

~~~
jFriedensreich
ahhh, sorry i did not know "SSL for SaaS" was a product name for a specific
feature. I totally agree, this would have been really nice to be able to offer
my clients, but is currently not really accessible. There are a few things
like this such as custom error page etc. Lets hope more enterprise offerings
will be more accessible for smaller projects in the future...

------
sudhirj
One alternative that's available right now is Fly.io — Fly actually runs
Docker containers at the edge, so you get a lot of the benefits including
autoscaling up and down, your application following the sun across the world,
etc.

I'm using them for a couple of small production projects and wrote up a gRPC
example, quite happy with the service so far.

~~~
jeffbee
"At the edge" is going to mean different things on different clouds.
Cloudflare has hundreds of points of presence. What's more edgy about fly.io
that you don't get from Cloudflare, App Engine, or AWS Lambda?

~~~
mrkurt
Oh hello! (disclosure, I work on Fly.io).

We're more like Fargate than Lambda, but Fly apps scale and balance across
regions. I wouldn't really call it "more edgy", it's just built specifically
for running app servers close to users (vs emulating a traditional single
region datacenter).

We actually started with a JS runtime like CloudFlare has, but it was wrong
for our customers. They're better off just running Docker images.

There's a lot more in our HN launch thread:
[https://news.ycombinator.com/item?id=22616857](https://news.ycombinator.com/item?id=22616857)

~~~
rbflx
Sorry if this is somewhat unrelated to the main thread, but considering that
someone working on Fly.io is here, I wanted to ask some questions that I
couldn't find the answers to in your documentation.

1\. What's the average cold start time when spinning up a new instance? For
comparison, AWS Fargate often has cold starts of around 1 minute.

2\. Is there a limit on the container size created by the Dockerfile?

3\. Is there a limit on the number of concurrent instances a single user can
run?

~~~
mrkurt
1\. Cold start times are wildly variable. They can be <1s for optimized, fast
booting docker images. We actually mask cold start time as much as possible,
though, so you'll almost never see one.

2\. There's no hard limit. We've had 8GB images running (that I know of). Cold
starts can be pretty bad on those just because pulling down an 8gb filesystem
is slow.

3\. Also no hard limit. I'm curious what you're doing though! Feel free to
email me: kurt at fly.io

------
colinclerk
(Crosspost from the original thread that didn't pick up traction:
[https://news.ycombinator.com/item?id=23963877](https://news.ycombinator.com/item?id=23963877)
)

Nice to see more and more pushed to the edge, and good that it's only the
beginning of the week.

I'm a little surprised they stuck with the isolates model when moving to
include generalized compute. We found isolates to be ideal for a cache layer,
but it doesn't have first class support for many languages:
[https://community.cloudflare.com/t/native-golang-support-
for...](https://community.cloudflare.com/t/native-golang-support-for..).

Very interested to see if they expand on their storage options throughout the
week. Additional consistency guarantees in their KV offering would make it
more competitive with Dynamo. I'd love to see a managed relational offering
like Cockroach, or perhaps the addition of CockroachCloud to their bandwidth
alliance.

One point I'm disappointed about is that they list API Gateway and DNS Queries
(per MM requests) at $0.

This may be true if you're deploying on a single domain with Cloudflare, but
if you're a SaaS and need SSL for custom domains, Cloudflare bumps you into
their enterprise offering that costs a minimum of thousands per month, and (we
found) netted out to about $5-10 per month per hostname:
[https://www.cloudflare.com/ssl-for-saas-
providers/](https://www.cloudflare.com/ssl-for-saas-providers/)

(I recognize this is a different way of metering, but listing both at $0 feels
like they're arguing DNS & TLS termination cost nothing at Cloudflare, and
that was far from our experience)

~~~
ignoramous
From [https://cloudflare.com/plans](https://cloudflare.com/plans), it looks
like one could use custom SSL/TLS certs with their business plan which is $200
a month?

~~~
colinclerk
We looked into this and spoke to their sales team about it. We could bring our
own certs for a single domain, but Cloudflare wouldn't accept traffic from our
customer domains unless we purchased SSL for SaaS.

------
eastdakota
Some more on how we think about edge computing and the serverless market:
[https://blog.cloudflare.com/cloudflare-workers-serverless-
we...](https://blog.cloudflare.com/cloudflare-workers-serverless-week/)

~~~
rewq4321
Any insight into why bandwidth is so expensive?

~~~
eastdakota
Bandwidth is one of the things that gets significantly cheaper at scale. For
most users, if they call a wholesale bandwidth provider the rates they get
will be more than what a public cloud provider will offer. So it looks like a
good deal. That's much more true than with something like hardware/compute
where the differences between what you can buy a server for and what a big
cloud provider can is less extreme. That's not to say Amazon or Google can't
get better deals on servers than someone who just phones up Dell, but it's
nothing like the savings you can get at scale on bandwidth. What's been
interesting is how little competition there has been between the different
public clouds on bandwidth pricing. That's starting to change and I'm proud of
how we've pushed that at Cloudflare with things like the Bandwidth Alliance
([https://www.cloudflare.com/bandwidth-
alliance/](https://www.cloudflare.com/bandwidth-alliance/)). Oracle's cloud is
also pushing on this (see Zoom's comments on why they chose the Oracle cloud).
If you saw the margins that AWS gets on bandwidth you'd throw up in your
mouth.

~~~
jorams
I find this comment rather confusing under an announcement for Workers
Unbound, which charges $0.09 per GB. That's AWS/GCP/Azure-level pricing, and
definitely does not push competition on that front.

------
nnx
>Data Transfer (per egress GB) $0.09

Surprising that Workers Unbound charges egress (first time egress is directly
charged by Cloudflare?) and at this quite expensive AWS-like price.

------
crisscrosscrash
I see CPU mentioned a bit, but nothing about memory. It seems like 128 MB
isn't enough to run Nuxt.js (Vue server-side rendering), but it would be
awesome to be able to run that (or Next.js for React) at the edge. I haven't
actually tried yet on Workers, but they seem to recommend 512 MB memory for
App Engine for example.

Are Cloudflare's machine sizes unsuitable for being able to offer more memory
as an option?

~~~
eastdakota
We will allow in the future the ability to use larger memory allocations.

~~~
crisscrosscrash
Any rough guesses on when that could happen? Weeks, months, or next year?
Depending on the plans, I'd consider waiting for a Workers-based solution for
an upcoming project instead of working on a Cloud Run based deployment.

~~~
alyson-cabral
Will you sign up for the beta at [https://www.cloudflare.com/workers-unbound-
beta/](https://www.cloudflare.com/workers-unbound-beta/) linking to this post?
Me or my team will follow-up directly to understand your use case and see if
it's a good fit.

------
manigandham
The data transfer pricing is a big letdown considering Cloudflare has always
has free transfer.

When transfer is so expensive, it makes it harder to switch from other clouds
that have more than just compute capacity, especially since you might be
paying double to transfer between cloud->CF->user

------
hashamali
I see Go listed as a supported language. Is that through WASM build targets or
something else?

------
rosywoozlechan
Cloudflare should focus on getting their existing services working. I tried
their 1.1.1.1 Warp VPN on Android as a paying subscriber and it just broke my
internet connection most of the time.

~~~
kyleCF
Hey rosywoozlechan I am the Product Manager for this app, can you file an in
app bug with this name so we can look in to your issue? (look for little bug
icon on main screen to file issue)

------
rbflx
Looks very interesting!

Does anybody know whether this will remove the current maximum runtime
limitation? As far as I can tell, workers have a maximum duration of 10ms on
the free plan and 50ms on the paid one.

I would assume that it does, considering that it's being compared to AWS
Lambda, which has a timeout of 15 minutes.

~~~
kentonv
Yes, this is all about removing those limits.

Note that the current 10ms/50ms limit is on CPU execution time; it doesn't
count time waiting for remote servers to respond.

------
minxomat
LuaJIT support, please. Other than that some tighter consistency guarantees
around KV would be nice. I’m PoCing a small game server on workers and the KV
is working great for this so far (turn by turn, but still).

~~~
majke
Awesome! I love Luajit! What specifically you want to do? HTTP game?

------
feniv
Does this also remove the 30 scripts per account limit? I looked into using
Cloudflare workers to power a cloud runtime for WASM (think thousands of
dynamically generated WASM binaries serving traffic for different sub-
domains), but the script limit and CPU limits were a major blocker. I've had
great experiences with Cloudflare and would love to use workers if it opens up
these limits.

[https://developers.cloudflare.com/workers/about/limits](https://developers.cloudflare.com/workers/about/limits)

~~~
kentonv
The 30-script limit exists because, currently, we copy every script to every
server on our edge upfront, hence the storage is rather expensive and we don't
want people storing a bunch of scripts they aren't actually using. We'll
likely improve this in the future by allowing you to have more scripts as long
as you're OK with some of them being fetched from a central location on-demand
(which might make them a little slower to start up). I don't have a timeline
for when we'll implement this, but it is a common request.

~~~
yjftsjthsd-h
Any chance you could just charge per script or byte after 30 scripts?

------
saurik
At one point, Cloudflare suggested they might open source some of their v8
work; does anyone know if that still might happen? (Even just the bindings for
WebCrypto would be interesting.)

~~~
kentonv
I'd love to open source our V8 API binding layer, which I think is a lot
easier to use than the other solutions out there. But, to be honest, the main
blocker has been finding time to work on it -- there's just way too much going
on.

Our WebCrypto implementation just a thin wrapper around BoringSSL, FWIW.

On the topic of open source, I'd say about 10% of the coding we do for the
Workers Runtime actually lands in Cap'n Proto / KJ, which has been open source
all along:
[https://github.com/capnproto/capnproto](https://github.com/capnproto/capnproto)

------
blntechie
I learned about Cloudflare Workers early this year and they definitely seem
simpler than AWS Lambda + Gateway combo to deploy and rollout.

Unless one is invested in AWS for other needs or need one of the language
which Workers doesn’t support (Java?, Go?), it might be a fair option to
consider.

~~~
pier25
It Depends. You cannot run many Node libraries on Workers for example.

------
saurik
Part of their defense against cpu side channel attacks was the narrow
execution limits; did they decide those are unnecessary? (That was also how
they prevented v8 from allocating tons of memory.)

~~~
kentonv
> Part of their defense against cpu side channel attacks was the narrow
> execution limits;

Some people have claimed narrow execution limits provide a defense, but we on
the Workers team don't believe that's true and I don't think we've made that
claim. Specifically, it's easy for an attacker to store state between requests
and so continue an attack across many requests. In our current implementation,
you can store that state in global variables, but even if we wiped the
worker's state after every request, it would still be easy for an attacker to
store their state remotely.

We'll be posting more about Spectre later this week.

> That was also how they prevented v8 from allocating tons of memory.

No, we explicitly limit V8 memory usage independent of CPU time.

~~~
saurik
> No, we explicitly limit V8 memory usage independent of CPU time.

Ok! I hadn't just gotten that from one your talks on Workers: you had said v8
doesn't really offer a useful hook to limit memory usage as it aborted the
process rather than just stopping the isolate, and that the CPU resource limit
was thereby an important part of the memory limitation (as with the exception
of typed arrays, using memory in JavaScript requires code to actively set
memory). If you did change this, it makes me all the more interested in the
changes you are making in v8 ;P (though it is also possible these changes were
actually just part of upstream and I failed to notice).

[https://www.infoq.com/presentations/cloudflare-v8/](https://www.infoq.com/presentations/cloudflare-v8/)
(at 20m45s)

------
a2tech
They should work on reliability before new niche features. I work with another
developer on his clients sites and he’s a Cloudflare fan. Almost every
complaint and failure we’ve had reported has come from Cloudflare errors. Not
issues with our stuff, issues with Cloudflare. I use cloudfront with a
different set of clients with semi-high visability sites and have never had an
issue stemming from it.

------
jeffbee
> Isolates are far more lightweight than containers, a central tenet of most
> other serverless providers’ architecture. Containers effectively run a
> virtual machine, and there’s a lot of overhead associated with them.

Kill me, now. Containers are in no respect virtual machines. Cloudflare
Workers, by contrast, literally are virtual machines: your code runs on V8.

~~~
james412
An isolate is a v8 "virtual machine" running in-process with many other such
machines, virtualization is provided by a userspace program

A process is a "virtual machine" running on-host with many other such
machines, virtualization is provided by the kernel

A host is a "virtual machine" running on metal with many other such machines,
virtualization is provided by the CPU

Metal is a "virtual machine" running on microcode with up to 3 sets of sibling
architectural state (did any vendor ever implement 8-way SMT?), virtualization
is provided by the silicon

It's turtles all the way down

Even better if the isolate is itself running an emulator, in which case it's
also turtles all the way up

~~~
namibj
Yes, 8-way SMT is available in scale-up POWER 9. But it just merges pairs of
4-way SMT cores instead of adding more threads to each core. The overall
number of threads is not affected.

------
abrookewood
Isn't this statement incorrect? "Isolates are far more lightweight than
containers ... Containers effectively run a virtual machine, and there’s a lot
of overhead associated with them"

Everything I have read suggests containers (being based on CGroups) add
virtually no overhead at all (in the vicinity of 1% at worse from memory).

~~~
throwawaythekey
Containers are very lightweight compared to a vm, but each FAAS container is
required to run its own instance of v8. This implies some start up time (tens
of ms?) and some memory use (10's of mb?).

My understanding of isolates is they are more or less a container, but at the
v8 level instead of OS. This makes spinning up a function more akin to opening
a new tab vs opening a new browser instance.

~~~
abrookewood
Thanks for the explanation. I get that this approach is faster than
containers, but was mainly nitpicking over the statement that containers ARE
VMs.

------
tapirl
It looks Cloudflare Workers and AWS Lambda are just re-implementations of
Google App Engine. It is a sad that Google half gave up App Engine, by
switching to a bad pricing scheme and not putting enough energy in developing
GAE and further innovating in this area.

------
devwastaken
Has the status changed on time synchronization on endpoints? Using workers to
deliver time over http would work really well if the servers they're hosted on
are guaranteed to be through standard NTP.

~~~
ignoramous
Workers don't expose accurate time afaik to mitigate Meltdown/Spectre [0], but
wouldn't their free-to-use NTP server work for your usecase?
[https://www.cloudflare.com/time/](https://www.cloudflare.com/time/)

[0]
[https://developers.cloudflare.com/workers/about/security/](https://developers.cloudflare.com/workers/about/security/)

~~~
kentonv
Workers can read the clock, but the clock doesn't advance while the worker is
executing, which prevents it from using the clock to measure its own execution
time. For a service that just returns the current time, that shouldn't make a
difference.

------
yadco
If they make an aws efs like method of storage, someone might build something
very interesting on it.

------
marcrosoft
Do workers support Go natively yet?

~~~
sudhirj
No, the blog posts talks about using V8 isolates (V8 is a JavaScript engine).
What CF does is support Rust via Web Assembly, and I think Go WASM might be
supported. But they're pretty clear that they're not doing a process model,
you'll have to submit code that runs inside V8.

~~~
dstaley
The sign-up page[1] mentions several languages that don't (as far as I know)
compile to Web Assembly (at least not natively).

[1] [https://www.cloudflare.com/workers-unbound-
beta/](https://www.cloudflare.com/workers-unbound-beta/)

~~~
iampims
[https://github.com/lsds/faasm](https://github.com/lsds/faasm)

Looks like Python there is an existing python <-> wasm interop

------
erikw
The linked Techcrunch article is pretty light on actual content. The actual
Cloudflare blogpost describes what this product is: "We are extending our CPU
limits to allow customers to bring all of their workloads onto Workers, no
matter how intensive."

[https://blog.cloudflare.com/introducing-workers-
unbound/](https://blog.cloudflare.com/introducing-workers-unbound/)

~~~
nnx
Indeed... better link at
[https://news.ycombinator.com/item?id=23963877](https://news.ycombinator.com/item?id=23963877)

------
AnonC
I know that Cloudflare people comment here. So can I go on a tangent and ask
what’s holding up registering new domains through Cloudflare (posting on the
Cloudflare blogs or community forums doesn’t get a response)?

Cloudflare registrar was announced (and released) nearly two years ago,
allowing people to transfer their domains to Cloudflare. It was declared then
that registering new domains would be coming soon. But here we are nearly two
years later and there’s no sign that this is coming this year either.
Meanwhile, the company even went public through an IPO. What’s the holdup and
why the radio silence? I’d actually prefer a blog post about this on the
official site than a “what should we tell and what shouldn’t we tell” response
here.

~~~
eastdakota
We’re thinking about this. What we don’t want are domain speculators buying
domains cheap and then doing nothing with them. That’s all cost and hassle for
us. We want to let you register domains that you’re serious about doing
something with. We’re running a bunch of experiments to see if we can allow
new registrations of domains people are serious about without opening the
domain speculation floodgates.

~~~
eis
I'm sorry but I don't buy it. Domain speculators who buy thousands and
thousands of domains already get very cheap bulk pricing. Thinking that you
would contribute to that problem sounds ridicolous. Also there's plenty of
simple ways to prevent this if you wanted to. Like limiting the amount of
domains per account, credit card etc. Put it in your TOS and kick people out
who do it anyways. Come on...

~~~
eastdakota
You may not buy it, but that's the rationale. I'm the one vetoing us turning
it on so you're literally talking to the decision maker. We've thought about
everything you've suggested and it all feels artificially constrained (e.g.,
what if you have 11 domains that you really care about?). We're running a
bunch of different experiments for different options. We'll open up domain
registration at some point, but only when we're confident it lives up to our
original promise and still accomplishes what's right for our business. We
haven't figured that out yet.

~~~
zimbatm
Isn't speculation affected deterred by higher pricing? What is preventing you
to add a "new domain" cost on top of the renewal?

As a customer I would be ready to pay a bit extra rather than go through the
purchase at NameCheap. wait 60 days, and then transfer dance.

~~~
eastdakota
We could, but we promised we’d never charge more than the wholesale price. One
experiment we’re playing with is bundling a highly discounted Cloudflare plan
with new registrations. So you get registration at cost and a paid Cloudflare
services at a significant discount.

~~~
0xy
Dual pricing for new vs transfers is industry standard. I'm happy to pay a
one-off $1 fee for new registrations (and additional years/renewals are cost
price).

I don't even think you've misled anyone by doing this, since it's essentially
a new product (new domain registrations).

