
Network.framework: A Modern Alternative to Sockets (2018) - cztomsik
https://developer.apple.com/videos/play/wwdc2018/715/
======
jzl
This is really cool. The telltale sign of a modern, robust phone app is its
ability to gracefully handle constantly changing network connectivity. A
number of big-name apps are still not very good at this. This framework will
make it much easier to offload most of that logic to a well-designed API. I
wonder if Android has something like this as well.

------
pedasmith
For a similar take on the problems, but with some different solutions, take a
look at the UWP [StreamSocket]([https://docs.microsoft.com/en-
us/uwp/api/windows.networking....](https://docs.microsoft.com/en-
us/uwp/api/windows.networking.sockets.streamsocket)) (etc) classes.

Specifically, StreamSocket will happily go through proxies for you, will
handle TLS / SSL (either at start up or after the connection is established),
does IPv4 and IPv6 transparently. We also do service discovery using DNS-SD,
although that's technically in a different class, as is the other network
information. The SteamSocket class is designed to be very similar to the other
socket-style classes like StreamWebSocket, so your code will (mostly) move
from one to the other.

I've found from my own experience that the real key to a smooth network
experience is the magic technology called "caring". It's easy to test if your
code handles network disconnects; just run your code and yank the ethernet
cable (or switch off Wi-Fi).

Source: am PM for the Network Developer Experience team at Microsoft and
helped design the socket APIs.

~~~
nishotnew
Is the DNS-SD API in windns.h under your purview? I have some gripes with it
but don't know how to provide feedback.

~~~
pedasmith
No, but I know the team and will happily forward the feedback to them.

------
saagarjha
Documentation:
[https://developer.apple.com/documentation/network](https://developer.apple.com/documentation/network)

------
joshbaptiste
Prev (2018):
[https://news.ycombinator.com/item?id=17265288](https://news.ycombinator.com/item?id=17265288)

------
dang
A transcript is at
[https://asciiwwdc.com/2018/sessions/715](https://asciiwwdc.com/2018/sessions/715).
Should we change the URL above?

Edit: never mind; you can click to get a transcript via the submitted URL. I
didn't see a way to link directly to it though.

------
bradhe
Much of this actually seems a bit _more_ complicated than Berkeley sockets
:\\. The async features look easier to use...

------
based2
MIME type not found on firefox...

~~~
lol768
Might be this
[https://bugzilla.mozilla.org/show_bug.cgi?id=1580895#c2](https://bugzilla.mozilla.org/show_bug.cgi?id=1580895#c2)

~~~
svnpenn
Yep. Which depends on this:

[https://bugzilla.mozilla.org/show_bug.cgi?id=1367105](https://bugzilla.mozilla.org/show_bug.cgi?id=1367105)

which has been open for... 3 years.

------
exikyut
Offtopic genuine question about self-improvement

This presenter shares a problem I find I have myself: I don't instinctively
know how to make sure I'm getting enough air as I talk.

I've historically had a terrible time talking while walking (really hard),
talking and standing (uncomfortable, like this guy), and standing and singing
(gave up a very long time ago).

I'm usually okay, but on some days it's so bad I just don't have the oxygen
and I find I'm predominantly yawns and hot air. Thing is, _I don 't know why_,
so while things have improved over the years I have no idea how to shove the
needle over to where it needs to be.

Insight appreciated, thanks!

~~~
carapace
There's breathing and then there's breathing.

 _Where_ are you breathing from, in your body? Are you taking small breaths
just using your upper chest? Or do you breath from the diaphragm, filling the
lower (and larger) parts of your lungs? Aka breath "from the belly". Lungs are
sort-of bell-shaped: Image search of lungs:
[https://duckduckgo.com/?q=lungs&atb=v60-1&iax=images&ia=imag...](https://duckduckgo.com/?q=lungs&atb=v60-1&iax=images&ia=images)

If you are hunching over and/or tensing up your torso that might be your
problem right there?

------
loktarogar
can anyone tl;dr? what is this, why is it better than sockets etc

~~~
js2
From the introduction:

Network.framework is a modern alternative to sockets.

There are three primary areas in which it's very difficult to use sockets
well. The first one is Connection Establishment. There's a whole host of
reasons that establishing connections can be really difficult with sockets.
For starters, sockets connect to addresses, so you have to, most of the time
you have a host name, so you're going to have to resolve that host name to an
address. When you do that, you often end up with way more than one address.
You'll have some IPv4 addresses, some IPv6 addresses, and now you've got this
challenge, which address should you try and connect to, in what order? How
long do you wait before you try the next one? You can spend years trying to
perfect this. I know because we have.

Once you get past the dual stack host problems, you run into a whole bunch of
other issues.

There are some networks that use something called Proxy Automatic
Configuration or PAC.

On these networks, there's a JavaScript that you get, and you have to pass a
URL into the JavaScript, and the JavaScript runs and spits out an answer that
says either you can go direct or you have to use this SOCKS proxy over here or
that HTTP connect proxy over there.

And now your app has to support SOCKS proxies and HTTP connect proxies, and
this can be really difficult to do well. And the most difficult thing is that
you may not have one of these networks to test on, so you may get a bug report
from one of your customers, and they may complain that it's not working well
on their environment.

And you may want to add code to fix the problem, but that once you've got it
in there, you really don't have a good way to test it. You have to end up
building the whole environment to reproduce the same environment they have. It
can be a real challenge. So connecting with sockets is really hard. The second
thing that becomes challenges with sockets is data transfers. There's a lot of
reasons that transferring data with sockets can be really difficult. The
primary problem is the read and write model itself. If you're using blocking
sockets, it pretty simple, but you're tying up a thread, and it's really not a
great idea to be tying up a thread while you're waiting to read or write data.
You can switch to nonblocking, but then you end up with a whole lot of other
challenges that you run into.

When you're using nonblocking, you may tell the kernel I'd like 100 bytes, and
the kernel will come back and say, I've got 10 bytes for you, why don't you
come back later.

And now you have to build a state machine to keep track of how many bytes you
read versus how many bytes you want to read. This can be a lot of work, and
getting it to perform well can be a real challenge. On top of all of that, you
really shouldn't be reading and writing to sockets directly because you should
be using something like transport layer security or TLS. Sockets don't support
TLS, so you're probably using some other library that is handling TLS for you
and reading and writing to the sockets on your behalf, or you're writing the
glue code between that library and sockets, and you have to figure out how to
get all this to work with all the crazy connection logic that you put in ahead
of time. There's a lot here that can be really difficult. Finally, mobility
can be really challenges with sockets for a variety of reasons.

Today, we have these incredibly powerful devices in our pocket with multiple
radios that may be on at the same time, and some of them are moving from
network to network, and your application has to handle all these transitions
well to provide a seamless experience to your customers. Sockets does nothing
to help you with this.

You can use routing sockets, but it's really, really difficult. We think a
transport API should do better. Fortunately, on our platform as an application
developer you have a great API in URLSession. URLSession handles all of these
problems for you.

It's really focused on HTTP, but it also has stream task that gives you raw
access to TCP and TLS connections. You might think that URLSession is built on
the same primitives that you would use yourself. But it turns out that's not
the case. URLSession is built on top of something we call Network.framework.
URLSession really focuses on all of the HTTP bits, and it offloads a lot of
the transport functionality to Network.framework.

Network.framework is something we've been working on for a number of years,
and in supporting URLSession, we've learned a lot, and we've taken a lot of
those lessons to the IETF. A number of our engineers regularly participate in
the IETF and meet with engineers from other companies, and they've been
discussing a lot of what we've learned in the transport services working
group.

And in those discussions, we've got some great feedback, and we've brought
that back in and improved Network.framework based on that. We are really
excited to announce this year that your applications can take advantage of
this same library directly now. Now we know that one of the things people love
about sockets is that it gives them very fine-grain control over just about
everything, and they're really loathe to give that up. So as we developed
Network.framework, we wanted to make sure that it did the right thing by
default in the way that sockets doesn't, but it gave you all the knobs that
sockets does. And it's kind of got this gradient, so the more knobs you turn,
the more complex it becomes.

It gives you all the power you need, but you don't have to pay for the
complexity unless you actually need some of it. Network.framework has
incredibly smart connection establishment.

It handles the dual stack cases. It handles IPv6 only networks. It handles
PAC. It handles proxies. It will help you connect on networks that are
otherwise very difficult to deal with. It has an incredibly optimized data
transfer path that lets you go way beyond the performance of anything you can
do with sockets, and Tommy will cover that in a little bit. It has support for
built-in security. It supports TLS and DTLS by default. It's really simple to
use.

It has great support for mobility. It provides notifications about network
changes that are relevant to the connections that your application is
establishing. It's available on iOS, macOS, and tvOS as a CAPI with automatic
reference counting, so it's easy to use from Objective C, and it has an
incredible Swift API.

~~~
swiley
The dual stack thing is fixed with getaddrinfo. You’re probably speaking a
protocol, I personally like to use regular languages with a reset byte (or
octet or whatever.) at the beginning of every message. Since the data just
goes into a state machine you can feed it partial messages and it will
reconstruct them. This takes very little code, very little thought (just use
Kleene’s algorithm) is _extremely_ portable (unlike this which is exclusively
apple) and very easy to mock and test.

Yes that still doesn’t take care of SOCKS but that (IMO) is heading into the
“give the user an option to connect using pipes and netcat” territory (and
that’s probably the best way to fix it.)

~~~
aseipp
It's always hilarious to read the ever-puzzling reply that you see here, which
is basically "Why don't you just manually re-build this design into every
single application you've ever written, repeating yourself and scope for bugs
50x over, while I also ignore all other possible design constraints", and it
never ceases to amaze me. The idea that a better abstraction for a component
could exist or should be explored is strictly verboten.

~~~
swiley
Abstraction is great but this one locks your app into a single platform (and
not only that, one that you often have to pay for the privilege of developing
for.)

Furthermore what I said really isn’t difficult, I recently wrote a protocol
that way in two languages (C for an MCU and python for Windows) in a week, and
as far as I can tell neither program has any serious bugs.

------
layoutIfNeeded
An API like this sorely needs async/await.

~~~
onefuncman
What kind, specifically?

async/await in C# is a flaming tire fire of hot garbage, it's more a virus
than a help.

And why isn't an event driven API preferable?

~~~
kyberias
Care to elaborate? I've been happily using async/await in C# without realizing
how bad it is. It seems great, but maybe I don't just know enough. What should
I know?

~~~
lukevp
They’re probably referring to the warning in VS that async methods should be
called from other async methods and aren’t a wait able otherwise, which causes
async to propagate outward. This is imho not a big deal because you can decide
at any point up the chain to make something synchronous by using .Result or
.Wait() instead of awaiting it. It still makes everything downstream async.
It’s a misunderstanding of the await keyword mostly. Await is essentially a
yield you can place in a method that’s marked async but it’s not required for
other called methods to have async/await patterns. Ultimately though the
better way is to just make everything you can async.

~~~
onefuncman
There's terrible dangers that await one who casually throws .Result or .Wait()
into an existing ASP.NET application.

The intersection between synchronous and asynchronous code is the ugliest.

The most dangerous is how deadlocks can arise because of the execution context
of a running application vs the execution context of your unit tests.

If you're already in .NET Core, you're probably fine.

For more, try [https://devblogs.microsoft.com/pfxteam/should-i-expose-
synch...](https://devblogs.microsoft.com/pfxteam/should-i-expose-synchronous-
wrappers-for-asynchronous-methods/) or any of Stephen Cleary's blog posts
[https://blog.stephencleary.com/2012/07/dont-block-on-
async-c...](https://blog.stephencleary.com/2012/07/dont-block-on-async-
code.html)

~~~
DSMan195276
> If you're already in .NET Core, you're probably fine.

Note it's not quite that simple. It's true that ASP.Net Core uses the default
synchronization context, and thus will not generally deadlock if a thread
`await`s a `Task` that was started on the same thread (Because it can get
resumed on a different one), but that doesn't mean there are no problems with
this approach. An ASP.Net Core app is not going to start up as many threads as
a ASP.Net one by default, and will do so much slower (And I believe there is
also a lower limit on the max number of threads, though I'm not 100% sure. The
article you linked says it was 25 for 1.0, but I'm not sure about the limit
for 2.X or 3.X versions). Point being, if you're blocking threads by waiting
on other tasks, it is still possible you encounter deadlocks or big
performance drops because there are no threads currently available to run the
task (And none are getting spun up to handle it).

ASP.Net Core is somewhat "worse" in this regard because of what I put above -
Even if you're not hitting the deadlocks you would get in ASP.Net, if you have
a lot of blocking actions that you haven't converted to be `async` then you're
going to potentially hit bad performance issues due to the fact that the
starting thread-pool is much smaller, resulting in a lot less requests getting
processed (Because a completely `async` pipeline shouldn't require a large
thread-pool to process). _And_ , if for some reason the thread pool is
exhausted and no new threads are getting created, you can still get deadlocked
like before. If you're going to ASP.Net Core, you should really make your
pipeline completely `async` to avoid running into issues.

