
Nuclio: Serverless for Real-Time and Data-Driven Applications - dominotw
https://nuclio.io/
======
fest
Slightly OT, but I find it interesting how the definition of real-time depends
on the industry:

* Embedded systems: nanoseconds to milliseconds (depends on the system, but at the very least needs to be predictable within some bounds);

* Application software (including web): tens to hundreds of milliseconds;

* TV broadcasting: seconds to minutes;

I'd love to hear more examples from different industries!

~~~
jacques_chester
Traditionally, "realtime" means you can tell, in terms of actual wall clock
time, how long different operations will take. Tends to show up in embedded
systems because they tend to control devices in which fixed deadlines matter.

Somehow it morphed into a shorthand for "fast".

~~~
vvanders
Yup, realtime was always predictable(ex. 16.666ms for 60fps). Time unit
largely doesn't matter.

Generally precludes GC(yes there are edge cases where you can shoehorn GC in,
I don't care about them since malloc is usually out of bounds as well).

------
rburhum
> nuclio is an open source serverless platform which is faster than bare-metal
> code

WTF? Can somebody explain to me what they are trying to convey here?

~~~
yaronhaviv
nuclio is using a real-time arch (no context switch, zero copy, no Ser/Des,
parallelism, .. ) Go function serves 400K events/sec/proc, Py does 70K
events/sec/proc, this is faster than running your own function with standard
HTTP serving on a bare-metal server.

for comparison most serverless frameworks do 1-5k events/sec and up to 100x
higher latency.

disclosure: i work on nuclio

~~~
rburhum
My point is that - as described - it is a dumb comparison.

> open source serverless platform which is faster than bare-metal code

does't make sense. It eventually has to run on top of bare metal (via
hypervisor or directly), so claiming that something that eventually runs on
top of bare metal is "faster than bare metal" doesn't make sense _at all_.

------
jmspring
Python and "real time"? Tensorflow as an example in the image/example?

Needs more buzzwords, for sure. Developers need to spend more time
understanding "real-time" as well.

------
chx
Clearly this needs blockchain in the title, it's not meaningless buzzword
dense enough otherwise.

------
keyle
I had to look up "Serverless", especially after it says "here is how to start
our docker app".

I might be old by now, but serverless feels like renting a sublet place per
hour. I guess it was only going to happen, after we went from 1 server to many
instances to unmanageable mess: "aaeaeh let it be someone else's mess! Take my
money!"

~~~
ralusek
"Let it be someone else's mess, take my money," is the basis of most
businesses. That's why I don't grow my own food, and I often don't make it.

------
IanCal
I think I'm following along with this and it looks pretty cool.

Is there management for handling things with a slow startup time?

I've got a few things that have startup times in the region of 30s+ but then
each thing to process takes very little time.

I found this, which I think is relevant:
[https://github.com/nuclio/nuclio/blob/master/hack/examples/p...](https://github.com/nuclio/nuclio/blob/master/hack/examples/python/tensorflow/tensor.py)

Are there some pitfalls about this to be wary about / known bad edge cases?

------
mrtesthah
What is the median latency in milliseconds?

~~~
pwaai
this is what I'm curious about, serverless automatically makes me think of
cold startup time.

~~~
jacques_chester
Some poking around suggests that currently they scale to a single copy, so for
sporadic workloads latency will be good (relative to normal latency for warm
code).

Based on the roadmap in the repo[0] (under "In Design").

Disclosure: I also work on a FaaS project, Project Riff.

[0]
[https://github.com/nuclio/nuclio/blob/master/ROADMAP.md](https://github.com/nuclio/nuclio/blob/master/ROADMAP.md)

~~~
yaronhaviv
nuclio auto-scale today (use k8s HPA), the roadmap item is to scale-down to
zero (today holds one instance when idle and will soon be able to shut it down
when idle).

nuclio latency is as low as 30 microsec in Py & Go vs 10s of milisec in Lambda

~~~
jacques_chester
Warm vs cold start is not a very meaningful comparison, though. Lambda
typically scales from zero. Any warm system is going to greatly outperform a
cold one. Java on Lambda can vary from nanoseconds to seconds, depending on
whether you hit a warm copy or a cold start.

Disclosure: I work on a different FaaS, Project Riff.

~~~
yaronhaviv
I benchmarked all the frameworks in cold & warm, Warm is also very slow, no
concurrency etc'. how many events/sec can u do with a warm Riff instance e.g.
in Go & Py? whats the latency?

~~~
jacques_chester
Let me put it another way: are you saying you have no cold-start penalty?

Edit: and to answer your question -- no, I have not personally benchmarked it.
Can you point me to your suite? I've been thinking it would be useful to work
on a benchmark that different implementations can share to catch regressions
(and earn bragging rights, of course).

~~~
yaronhaviv
Great idea, we suggested to create an open benchmark at CNCF, a bunch of
members opposed, we can come with a joint proposal. You can use our test tool
[https://github.com/v3io/http_blaster](https://github.com/v3io/http_blaster),
it can generate a workload of > 600K http req/sec with one client and is
programable.

~~~
jacques_chester
Do you have a link to the benchmarking discussion? I'd be interested to see
what the objections were.

I raised the topic today in a team meeting and I'll probably get to tinker
over the next few weeks. I'm interested in exploring a few different axes of
stress. RPS is one, but also things like "how does it behave with ten thousand
functions installed?", "how does it behave with traffic yo-yoing?" etc.

------
RHSman2
Well I think this is frickin awesome. Very Well Done!

