That's really awesome. I'd like to know, though, whether the MPTCP is integrated into the mainstream API, or a special case for Siri. It's normal that only Siri generates MPTCP traffic since it requires both endpoint to support and enable multi-path. However, multi-path TCP is compatible with original TCP stack so I don't see a reason it can't be built into the general purpose API.
Could somebody who has developer account that can deploy to real device verify it by making another MPTCP-compliant[0] endpoint?
I suspect that this is done primarily to enhance Apple services like Siri, iMessage, and the like (anything that touches the iCloud), since Apple controls both end points.
For now it's used only for Siri which I guess makes sense: other services can easily retry the connection after switching data source, it's much more of a pain with Siri as one would expect greater responsiveness.
Awesome to have. I know Glass can't even understand my speech when it loses internet connectivity so devices are becoming really sensitive to connection loss. Meanwhile on my phone my LTE is often better than my WiFi so it would be nice to have this and not have to turn off my WiFi to use it.
ProTip: when whiting out your wireshark packet captures, always remember your hexdump:
dst=17.130.16.4
$ whois 17.130.16.4
...
NetRange: 17.0.0.0 - 17.255.255.255
CIDR: 17.0.0.0/8
OriginAS:
NetName: APPLE-WWNET
NetHandle: NET-17-0-0-0-1
Parent:
NetType: Direct Assignment
RegDate: 1990-04-16
Updated: 2012-04-02
Ref: http://whois.arin.net/rest/net/NET-17-0-0-0-1
OrgName: Apple Inc.
OrgId: APPLEC-1-Z
Address: 20400 Stevens Creek Blvd., City Center Bldg 3
City: Cupertino
StateProv: CA
PostalCode: 95014
Country: US
RegDate: 2009-12-14
Updated: 2011-03-08
Ref: http://whois.arin.net/rest/org/APPLEC-1-Z
OrgTechHandle: ZA42-ARIN
OrgTechName: Apple Computer Inc
OrgTechPhone: +1-408-974-7777
OrgTechEmail: droot@apple.com
OrgTechRef: http://whois.arin.net/rest/poc/ZA42-ARIN
OrgAbuseHandle: APPLE11-ARIN
OrgAbuseName: Apple Abuse
OrgAbusePhone: +1-408-974-7777
OrgAbuseEmail: abuse@apple.com
OrgAbuseRef: http://whois.arin.net/rest/poc/APPLE11-ARIN
RTechHandle: ZA42-ARIN
RTechName: Apple Computer Inc
RTechPhone: +1-408-974-7777
RTechEmail: droot@apple.com
RTechRef: http://whois.arin.net/rest/poc/ZA42-ARIN
...
EDIT:
also
subject=/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=California/businessCategory=Private Organization/serialNumber=C0806592/C=US/postalCode=95014/ST=California/L=Cupertino/street=1 Infinite Loop/O=Apple Inc./OU=Siri/CN=guzzoni.apple.com
issuer=/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)06/CN=VeriSign Class 3 Extended Validation SSL SGC CA
I don't understand. What does dst being apple's IP imply? It's connecting to Siri server which is supposed to be an Apple's server. Am I missing something?
I'm not sure if MPTCP is great or if it's an over-engineering that stuffs brand new abstraction into the socket layer.
See, previously when open a connection with an unbound socket, it would always bind it to the local IP that points at the default router. If you had multiple interfaces and you wanted a specific local IP, you'd bind to it explicitly before connecting. But it has always been that a connected socket is bound to exactly one IP. If you want to use two IPs for connecting out, you will have two sockets.
With MPTCP the socket can now end up being bound to two or more local IPs. So effectively it changes the abstraction of the socket model to be more of a collection of (conventional) sockets. Doesn't it sound like something that could be done in the userspace? Sure, it does :)
Though on the other hand, networking code in the kernel has access to a lot more information about the state of the network, so it can make better decisions and quicker. For example, if there's an interface with no default route going through it, the userspace solution may still try and use its IP to try and connect out. The userspace will also need to monitor the routing table for changes and re-try connecting out in response ... so MPTCP makes more sense if you think about it, but a userspace solution is much easier to implement and deploy.
Basically if you bind an address, MP is disabled. If not, then MP will happen by default. If you had code that was doing MP with multiple connections before, it was binding interfaces on multiple sockets and will still work. The idea is that MPTCP is desirable and should be used if possible. Also you get new sockopts for handling the MPTCP stuff like TCP_MULTIPATH_ENABLE and TCP_MULTIPATH_ADD if you want to control the MP stuff in MPTCP.
Yes, both endpoints will need to support it.
Support for MPTCP is advertised through a TCP option during the setup phase of the connection.
One of three things can happen:
- Both endpoints support MPTCP and announce it through the MPTCP option : MPTCP will be used. Subflows will be set-up if possible (i.e. if additional addresses are present on at least one of the endpoint)
- One of the endpoints (say an ios7 device) supports it, but the other (say a webserver) doesn't : the server will ignore the MPTCP option (which it doesn't understand) and a normal, single path TCP connection will be used.
- None of the endpoints support it, thus no MPTCP option will be sent, normal TCP will be used.
From what I read one of the main design decisions was to make sure the nodes along the route do not drop the packets which means it will be a TCP packet for all intents and purposes. So why should the other end change in this case?
It doesn't make sure the nodes along the route do not drop the packets. Nodes along the route would see those packets as normal TCP packets (which is by design because in this way routers in between don't have to change). But both end are aware of MPTCP and they may use multiple path.
In this case, iOS is connecting to Siri server using both 4G and WiFi. They apparently go through different routers. This helps improve reliability when the device is at border of a WiFi coverage and is switching between 4G and WiFi back-and-forth. Using two path simultaneously would allow Siri to switch to the other route when one of them is showing significant packet loss.
It's less about routing and more about connections. Typically, your client makes a single TCP connection for HTTP communication, all data requested on that connection is communicated on that connection. With a supporting server, multiple TCP connections can be used for the same HTTP connection. This doesn't help with transfer speed, but it improves TCP's ability to route around failure. One of your multiple TCP connections goes out, and another one (presumably on a different route) continues handling data transfer. With regular old TCP, the connection tends to fail and has to be re-established.
Pointless Pontification: since TCP was designed to route around failure, multi-connection TCP sounds to me like an adaptation of technology to the "demands of business." What I mean is network operators seem to want to centralize their networks to reduce costs instead of distributing them to make them robust, thus the technologists are attempting to work around this "failure."
If iOS is exposing the API, I believe it should at least provide a parameter whether it should be a MPTCP or a classic TCP. Sometimes you don't need that reliable trasmission; using MPTCP would cause unnecessary cellular data usage.
Multipath TCP could be used for smoothly handing over from the slower, more expensive, less-reliable cellular system, to the faster, cheaper, more reliable WiFi.
Without multipath TCP, you have to drop and restart each connection, which in some applications can cause hiccups, latency, or even complete loss of state.
Of course, all of this depends on server-side MPTCP support as well, which most systems don't ship with out of the box.
Right. What I'm saying is that, by keeping alive both path, you need to use some traffic. Although when there's WiFi available, you are primarily using WiFi, there are traffic going through cellular network in order to keep the path alive.
would be really cool if slower / more expensive connections could ultimately be drained, and the whole data transfer just switches over to a single more faster connection.
edit-1: note that this is not the same as subflow 'deletion' that would happen if an access mechanism was not available at all.
edit-2: section-3.4 of the mptcp paper indicates that FIN has a more limited 'no more data on this subflow' semantics. so, in theory it should be possible to do just that. would be so much better to not go the ANDSF route at all ;)
the shorter answer first: i don't think there is any possibility of resurrecting a subflow once it is gone.
longer one: with mptcp, there needs to be some way to distinguish connection-teardown vs subflow-teardown. with, RST the subflow is terminated. FIN is more subtle (since it occupies the sequence space in normal tcp) however. FIN is handled via an explicit DATA-FIN within TCP option to indicate the end of data-sequence-space (which maps subflow sequence numbers to data sequence numbers in TCP options). gracefully terminating a connection, thus involves sending DATA-FIN on all subflows together with a FIN.
This is far from universally true and a wonderful illustration of why multipath is great: I frequently see wifi where the back link is down or behind a payment gateway, requiring me to turn off wifi to get a reliable connection. This is sadly common at conferences where the venue skimped on AP placement and left entire rooms at max range or with inadequate backhaul.
The true beauty of MPTCP is that you don't need to make assumptions and hope — instead we can let the kernel sort things out based on the current environment.
Cellular mobile units are likely to have a short dwell time on any particular WiFi if they are in a moving vehicle. Can the underlying connections be initialized quickly enough for this case?
apllications use just the socket api for their interactions with the n/w stack. there is no change there, and hence the "unmodified applications" part.
from the mptcp paper, we have the following :
In brief, here is how MPTCP works. MPTCP is nego- tiated via new TCP options in SYN packets, and the end- points exchange connection identifiers; these are used later to add new paths—subflows—to an existing con- nection. Subflows resemble TCP flows on the wire, but they all share a single send and receive buffer at the end- points. MPTCP uses per subflow sequence numbers to detect losses and drive retransmissions, and connection- level sequence numbers to allow reordering at the re- ceiver. Connection-level acknowledgements are used to implement proper flow control.
so yes, both endpoints need to support mptcp for this to work.
So does this mean if I'm connected an adhoc eyefi wifi network and want to share whatever pic I took over the internet, I don't have to turn off wifi to do it?
If so, this is very awesome. It makes things like the Sony QX camera probably x10 more seamless.
That's basically how AirDrop works on recent Macs and recent iOS devices and why they require more recent wifi hardware to make it happen than the OS supports generally.
Does the phone actually send the TCP packets multiple times? Then it would certainly put more stress on the network. Does anybody know how fairness compared to non-multipath TCP stacks is achieved?
Multipath TCP sends packets over multiple paths; multiple different connections, such as cellular and WiFi. It doesn't repeat the packets, unless one of them is dropped. It can be used for several purposes; aggregating bandwidth between the paths, taking advantage of the bandwidth of both connections. It can be used for reliability. And it can be used for transparent handover of connections, where you may walk out of your house and switch over from WiFi to cellular seamlessly.
Am I correct in assuming that MPTCP could potentially make snooping on in-transit data somewhat more troublesome? All potential pathways would need to be monitored simultaneously in order to reconstruct the application-layer stream.
On the other hand, all cross-Atlantic traffic is likely to pass through the same few undersea cables, regardless of the "first 50 miles" are covered (3G/WiFi), so the net effect on snoop-ability is probably null.
> On the other hand, all cross-Atlantic traffic is likely to pass through the same few undersea cables, regardless of the "first 50 miles" are covered (3G/WiFi), so the net effect on snoop-ability is probably null.
Wouldn't the traffic be split over multiple IPs (not all of which might be known), and wouldn't latency sometimes cause them to be out of sequence?
It's definitely not as simple as "monitor this singular connection", though of course it's still feasible.
Indeed, on the surface, the multiple IPs would increase the amount of obscurity. Can anyone familiar with the protocol clarify if there are any other header elements which reveal that the packets belong to the same abstract TCP connection?
That is in theory exactly what this allows you to do. e.g., if you're watching some streaming video while outside, then come home to wifi, the connection will switch to wifi.
> As far as I understand, it uses the least congested path available.
IOW, it uses at least some cellular data even when connected to WiFi, because otherwise it wouldn't be able to determine which path is the least congested.
Ah, that is my misunderstanding. I thought it was some sort of "split traffic between each path".
But I guess for it to work it would still need to check the cellular connection every now and then. I wonder how much traffic that would generate? (hopefully a negligable amount.)
an enabler for ad-hoc, peer-to-peer wireless networks by providing the additional resiliency of tcp apps simultaneously leveraging (n) network interfaces and distributing traffic?
I'd still love ad-hoc better. HotSpot allows you to share 4G through WiFi, but it doesn't scale. The hotspot because AP, and that's the center of the network. On contrast, ad-hoc can scale to thousands of nodes across miles by using multiple hops. Also, ad-hoc is a more distributed model.
I never understood why ad-hoc mode was disabled by driver. AP mode (hotspot) is still available; users can still share their 4G connection without being approved by their carrier.
Ad-hoc networking on wifi generally only supports point-to-point links, not routing, unless you use another routing protocol on top of it like BATMAN Advanced.
I'm not sure why connecting to an ad-hoc network is disabled in Android, but it's certainly never been anything that's caused me problems.
Right. But I supposed by hotspot/ad-hoc, we were talking about the link layer possibilities only. hotspot is also about MAC layer only as well. IP runs on top of that. However, the difference is that, in AP/client mode, all nodes need to associate with and be able to talk to an AP; while in ad-hoc mode, every node is equivalent and they can be as distributed as they want. For ad-hoc routing, there're olsrd, batman, etc. They are really cool :-)
We used to use Android devices to test different scenarios of ad-hoc applications. But none of current mainstream devices support ad-hoc mode any more. On contrast, iOS devices can still join (although not create) an Ad-Hoc network, which is ... ironic.
And what of it? Here's another unrelated whinge: no Apple stuff supports WPS even though it's highly useful for getting your device on the network quickly.
Way to find something negative to say when this MPTCP deployment is overwhelmingly positive no matter what you think of Apple vs Android.
Integrated? I don't know about OpenBSD, but in Linux it is not in mainline. There is an experimental fork of the kernel supporting it, but it hasn't been proposed for merge upstream yet.
Sure, there has been development going on for years, but there are no major distros you can download and use it out of the box.
The point is, this is the first shipping product using it in the wild, as opposed to experimental implementations used in research and development.
Note that this is not multipath routing (which takes place at the IP layer, and requires involvement in routing protocols that end devices don't usually have access to), nor is it channel bonding (which occurs at the ethernet layer).
You can check this for yourself. Just take a look for the MPTCP patches in Linus's kernel. You won't find them. Here's my kernel git repo; origin is Linus's repo, mptcp is the MPTCP development repo:
As you can see, there are 2841 patches in the mptcp_v0.87 branch that are not in the kernel repo, all the way back to the original shim6 patch that was used to start building MPTCP on top of shim6, which was a proposal for multi-homed IPv6 (MPTCP now works on top of IPv4 as well).
Note that when you look at the "installing on Linux" instructions, they mention running it in a virtual machine.
This is not terribly uncommon for an ambitious research effort; work on an out of tree fork for a while, and then when it's ready, propose it for inclusion into the upstream kernel. For a recent example, see Bcache http://bcache.evilpiepirate.org/, which was developed out of tree for several years before being merged a few months ago.
> Please don't say things so scary without looking into it.
Why do you find this scary?
And why do you think that I didn't look into this?
Ahh, the way you said it made it sound like the first bit was you talking out your ass, assuming that they hadn't merged any of their work by a few pages on their site.
I am currently on a very restricted link, so I was not able to check the commit logs.
I think the "first" means being used in a consumer device. 802.11ac was there for a while. But none of big companies was making 802.11ac enabled laptop until Apple. Same thing here. It's not about who at when implemented it first; the point is this time it's being applied to a popular consumer device, even though it's just got approved by IETF (Jan.). This is how new technology gets involved into people's daily lives.
Probably not. A dedicated MITM attacker could simply strip off the MultiPath TCP options from the initial connection, so it would act merely as a regular TCP connection in that case.
Good crypto is a much better solution to minimizing the likelihood of a MITM attack. MultiPath TCP is design to provide speed, reliability, and mobility of TCP connections.
> Apple's iOS 7 is the first large-scale use of a newly-minted Internet protocol, called multipath TCP.
(emphasis mine)
The linux implementation is a third-party module with, as far as I can tell, little penetration so far. And the article linked here seems to have been partially cribbed from said module's implementors (the wireshark screenshot was lifted straight): http://perso.uclouvain.be/olivier.bonaventure/blog/html/2013...
Android runs Linux. Thanks for the unnecessary downvote. Want me to lookup the charts for you? Also there are millions of people running Linux (about 1.3% of all computer users if I remember correctly).
For the second time, since you apparently missed it the first time around, MPTCP is a third-party module, it is not part of the mainline kernel, it requires using the developers's kernel fork. Do you have any evidence that Android is doing so?[0] Then Android running Linux is irrelevant.
> Thanks for the unnecessary downvote.
I didn't downvote you.
> Want me to lookup the charts for you?
No, I'd like you to read what people write.
> Also there are millions of people running Linux (about 1.3% of all computer users if I remember correctly).
That's completely irrelevant to the thread because they are not running an MPTCP-patched kernel.
[0] I'll help you, the MPTCP project has a page pointing out to Android ports of the MPTCP module: http://multipath-tcp.org/pmwiki.php/Users/Android which include a rebuilt kernel to include MPTCP as well as patched connection manager, network daemon and iproute2. Doesn't sound like base Android supports MPTCP to me.
> MPTCP is a third-party module, it is not part of the mainline kernel, it requires using the developers's kernel fork.
Oh I get it now, that explains. I stand corrected. Sorry for being so bitchy about it, I just thought they were saying Linux was not widely used and hyping iOS7 instead.
Could somebody who has developer account that can deploy to real device verify it by making another MPTCP-compliant[0] endpoint?
[0]: http://multipath-tcp.org/pmwiki.php/Users/HowToInstallMPTCP