
Stupidly Simple DDoS Protocol (SSDP) Generates 100 Gbps DDoS - riqbal
https://blog.cloudflare.com/ssdp-100gbps/
======
majke
Author here. Allow me to extend the post a bit. It turns out that about 2.4%
of the IPs that respond to SSDP queries, do so from a weird port number! For
example:

    
    
      IP 192.168.1.75.50950 > 239.255.255.250.1900: UDP, length 95
      IP 192.168.1.71.1026 > 192.168.1.75.50950: UDP, length 249
    

The first packet is SSDP M-SEARCH query. The second is a response from my
printer. Notice - the source port for the response is not 1900 (but the dst
port is okay). I'm not sure what the spec has to say about it, but it's pretty
weird. What's worse - these responses won't be matched against "sport=1900"
DDoS mitigation firewall rule.

I'm not sure what is the moral here. But if you ever see some UDP packets from
a weird port, to a weird port - maybe it's this SSDP case.

~~~
floatboth
Isn't that normal UDP behavior? Source port on responses can always be
anything?

~~~
DSMan195276
You're half right. In most cases, programs have the OS pick their source port,
but that's for the computer _initiating_ the communication. So for example, in
the communication he gave the 50950 was likely picked by the OS (by selecting
a currently open port) and 1900 is the destination port. When the remote
computer responds (his printer), they don't then pick a new random source
port, they just swap the source/dest from the previous message.

If they didn't keep it the same then the OS wouldn't always be able to know
packets are part of which connections, because you can multiple connections
open with the same computer at the same time to the same dest port, and the
only differentiation would be the source port.

~~~
zAy0LfpBZLC8mAC
> If they didn't keep it the same then the OS wouldn't always be able to know
> packets are part of which connections

UDP does not have connections.

> and the only differentiation would be the source port.

... or some request/session/flow id in the application layer protocol. Some
UDP protocols use UDP in this way, some don't, UDP itself doesn't care.

~~~
DSMan195276
> UDP does not have connections.

UDP does not have connections, but the OS does have a concept of UDP
connections to a degree in the form of packet filtering/routing. Point being,
if you send a DNS request (for example), the source IP/port and dest IP/port
is how the OS will decide which packets to route back to you when the DNS
server responds. If the responding DNS server changes the source port, the OS
will not route that packet to the original socket because the source port does
not match. You can still make it work, but you would have to be already
listening for packets from that port (one way or another), so you would have
to know beforehand they are going to be using a different port.

> ... or some request/session/flow id in the application layer protocol. Some
> UDP protocols use UDP in this way, some don't, UDP itself doesn't care.

You still have to get the packets though, and the OS had no idea about any
application layer routing. If you want to get UDP packets from a bunch of
different ports, you have to be listening on those ports.

Edit: It's true I was playing a bit loose with the terminology (UDP is
connectionless), but the behavior of packet routing and how changing the
source port would mess with that is what I was getting at. If you want to be
more correct, replace "connection" with "socket" in my original comment.

~~~
zAy0LfpBZLC8mAC
> UDP does not have connections, but the OS does have a concept of UDP
> connections to a degree in the form of packet filtering/routing.

The filtering is completely optional to use.

> Point being, if you send a DNS request (for example), the source IP/port and
> dest IP/port is how the OS will decide which packets to route back to you
> when the DNS server responds.

That depends on how the requesting resolver has configured the socket.

> If the responding DNS server changes the source port, the OS will not route
> that packet to the original socket because the source port does not match.

That depends on how the requesting resolver has configured the socket.

> You can still make it work, but you would have to be already listening for
> packets from that port (one way or another), so you would have to know
> beforehand they are going to be using a different port.

Yes, obviously you have to know the application protocol you are trying to
speak and how it uses UDP before you try to speak it.

> You still have to get the packets though, and the OS had no idea about any
> application layer routing.

Which is why application layer routing is called application layer routing.

> If you want to get UDP packets from a bunch of different ports, you have to
> be listening on those ports.

No, you listen on local ports, not on remote ports.

> If you want to be more correct, replace "connection" with "socket" in my
> original comment.

Well, technically, some minor details would be more correct - but the
fundamental assumption that you can only receive datagrams from one remote
address/port with a given socket is just completely and utterly wrong, and not
just in the sense that it's a theoretical possibility, but it's a perfectly
normal use case. To take an obvious example, a common configuration for an
OpenVPN server is to accept authenticated packets from any remote address and
automatically switch to changing remote addresses for the sending direction,
so when the client changes addresses, the OpenVPN session just keeps going.

As long as you don't connect() a datagram socket in the BSD sockets API, you
will receive datagrams from any remote address (and you'll have to specify
remote addresses using sendto() when transmitting).

~~~
DSMan195276
It's seems the context of my original comment just went way over your head.
Yes, _obviously_ , if the protocol is defined to allow for varying the source
port then yes it will work because you specifically write your program to
handle that. But the person I was responding too was asking in the context of
protocols like SSDP, which is _not_ defined that way. And he was asking if you
could vary the source port anyway even though the protocol doesn't support it,
and I said no and explained why that wouldn't work.

~~~
zAy0LfpBZLC8mAC
> and I said no and explained why that wouldn't work.

And your explanation is at the very least misleading, bordering on wrong.
Pretty much noone (except where the protocol spec explicitly were to require
such behaviour, maybe) would implement a client that would open a socket per
server/per request, but bind them all to the same local address/port.

Either you use one socket for all requests, in which case you don't connect(),
so you receive all the responses, and thus would also receive datagrams from
addresses/ports that you didn't send to, and instead you would do the matching
of responses to requests in userspace, even if potentially based on the
sender's address/port.

Or you use one socket per server/per request and let the OS assign you a free
port per socket, in which case the local address/port is perfectly sufficient
for the OS to route received datagrams to sockets. In the latter case, it's
common to simplify your code by letting the OS handle the filtering of source
addresses, but that's all it really is, filtering--actual routing based on
remote addresses by the OS is not what normally happens and not why varying
the response source port would not work with many protocols.

------
hueving
More casualties from BCP 38 failures. This article mentions it but then
dilutes the importance of it by suggesting SSDP is a problem. If IP spoofing
did not work on the Internet, _none_ of these UDP reflection attacks would
work.

A scheme to strong arm the adoption of BCP 38 is key to stopping these attacks
from growing. IoT has shown us that expecting device updates to disable these
UDP protocols is a lost battle.

~~~
lightedman
"A scheme to strong arm the adoption of BCP 38 is key to stopping these
attacks from growing. IoT has shown us that expecting device updates to
disable these UDP protocols is a lost battle."

Easily done: "Follow some standards and RFCs or get put on a global blacklist
of companies to not do business with."

------
upofadown
>It's not a novelty that allowing UDP port 1900 traffic from the Internet to
your home printer or such is not a good idea.

How would this even be possible? Home routers have to NAT everything. Normally
you have to set up reverse NAT to get ports forwarded to the LAN.

~~~
dom0
IPv6 (now you really need a real, properly configured firewall instead of
NATting and praying)

~~~
oh_sigh
Of course you can NAT with ipv6. There is a private address space in
ipv6(fc00::/7) like there is in ipv4(192.168.0.0/16,...)

~~~
croon
I don't think the issue was if you can, but if anyone would when they don't
have to for lack of address space.

------
voltagex_
It will be years and years until those vulnerable miniupnpd versions are
updated. Most are in embedded devices which will never see another update.

I'm glad to see miniupnp is still in active development:
[https://github.com/miniupnp/miniupnp](https://github.com/miniupnp/miniupnp)
but I can't work out if it's set to be vulnerable by default.

~~~
Kikawala
Vulnerable by default?

If a device is listening to UPnP on the WAN interface, the fault is not on
UPnP but on whoever configured it to be open on the WAN. IMO, all of these
zeroconf protocols should be limited to responding back only to the local
segment and not allowed to traverse gateways.

~~~
jacquesm
One of the documented use cases for UPnP is IGD which expressly allows UPnP
devices to configure fire wall rules and to set up NAT to map ports to the
outside world. So a UPnP device that wishes to expose itself to the outside
world is able to do so and this is by design, not by accident. Whether you
agree with that or not is another matter.

~~~
CountSessine
Agreed - but can there ever be any legitimate use case for an home router to
speak IGD over its WAN interface? IGD is typically meant to allow your Xbox on
your LAN to set up forwarding rules.

~~~
duskwuff
None. Exposing miniupnpd on a public interface is always a misconfiguration.
I'm disappointed that the application even allows it -- it has to know which
one is public to set up IP forwarding rules, so it has no excuse.

------
saurik
> Internet service providers should never allow IP spoofing to be performed on
> their network. IP spoofing is the true root cause of the issue. See the
> infamous BCP38.

I don't see how it is at all reasonable to shift blame from a protocol that
assumes the world can be trusted to the untraceable goal of "every single
network in the entire world should only generate trusted data: then the
problem would be solved".

> Internet providers should internally collect netflow protocol samples. The
> netflow is needed to identify the true source of the attack. With netflow
> it's trivial to answer questions like: "Which of my customers sent 6.4Mpps
> of traffic to port 1900?". Due to privacy concerns we recommend collecting
> netflow samples with largest possible sampling value: 1 in 64k packets. This
> will be sufficient to track DDoS attacks while preserving decent privacy of
> single customer connections.

OMFG. Do you want deanonymization attacks? Because this is how you get
deanonymization attacks :/. The right form of solution here is not to
encourage ISPs to log even more of our traffic (a practice I wish were
illegal), but to try to kill off UPNP through every form of leverage possible
(even if it breaks things).

I'd say this is "so disappointing", but I guess I shouldn't expect much from
the company that tried its damndest to argue that nothing of importance was
leaked from Cloudbleed even when you could still recover Grindr requests
complete with IP addresses that they had managed to leak well after they tried
to claim that data had been scrubbed :/.

~~~
majke
You commented on two points:

A) ip spoofing B) netflow

On IP spoofing I said plenty already
[https://idea.popcount.org/2016-09-20-strange-loop---ip-
spoof...](https://idea.popcount.org/2016-09-20-strange-loop---ip-spoofing/) .
There are two major points:

\- We will always have DDoS vulnerable UDP protocols. In past we had DNS. Then
we had NTP. Now we have SSDP. The next one is going to be some gaming
protocol. We should fix them as we go, but a more comprehensive solution it to
actually fight the spoofing.

\- Even without using amplification, with IP spoofing it's possible to launch
a direct attack, which will be untraceable. We regularly see 150Mpps+ packet
floods going _direct_ from the attackers to our servers. The ISP's are
clueless. There is no way for anyone to trace the true source of the attack.
(without netflow, that is)

This brings us to second point - netflow. You say - the ISP's are incompetent,
they do not have netflow and this is _good_. No it's not good. The ISP's can
track you / deanonymize anyway, but when I ask them: "hey guys, I see this
150Mpps flood from your network, can you do something about it?" they say -
"no, we can't identify the source because the IP's are spoofed". Yes, I herby
recommend that each of the ISP's should take care of their network. Be able to
answer historical questions about DDoS. That means the netflow collection
point will have statistical metadata about customer connections (1 in 64k
connections will have saved data - source port/ip, dest port/ip, length,
packets, bytes). This might be used to attack your privacy - but the ISP can
do much worse things anyway. Doing netflow right will allow us to finally
trace the IP spoofing.

I really think that DDoS is a threat to the internet as we know it. Think
about centralization that it causes: can your server sustain trivial 100Gbps
SSDP attack? I really think that doing netflow right will allow us to keep the
decentralized internet.

~~~
lima
Question for any network admins here: I enabled IPFIX on our Juniper MX series
routers for that exact purpose, but it contains Layer 3 info only (no MAC
addresses!).

What am I missing? For now, I'm getting the info I need from sFlow but I want
to get rid of that ASAP.

You might be happy to hear that the ISP I work for can definitely identify
where that 150Mpps flood came from :) We're even doing some automated outbound
mitigation in order to be good net citizens. CloudFlare's blog articles
definitely helped us improve our network-level DDoS mitigation, by the way!
Thanks for that.

~~~
jakewilson
We worked extensively with the MX IPFIX export and I never saw mac addresses
in any of the exports Juniper sent us: [https://www.plixer.com/blog/virtual-
netflow/juniper-vmx-ipfi...](https://www.plixer.com/blog/virtual-
netflow/juniper-vmx-ipfix-support/)

------
bsder
Why is IP spoofing _STILL_ an issue? Why?

~~~
NKCSS
Because UDP is fire and forget, you don't have to be able to respond to
packets you send; this is why you can't do the same with TCP packages.

To impose fixes upstream, you'd have to do DPI on all data; which is not
allowed under some laws (i.e. net neutrality).

~~~
sebcat
In this case, you don't have to care about UDP or TCP, only IP.

RFC2827, which should fix the problem where SSDP can be used for DDoS, was
published in 2000:
[https://tools.ietf.org/html/rfc2827](https://tools.ietf.org/html/rfc2827)

Is ingress filtering on layer 3 considered DPI?

~~~
mjevans
I am not this kind of network engineer, HOWEVER. Both IPv4 and IPv6 are
versioned by the first 4 bits of the packet. Depending on that value the
address size and location are fixed.

I would not consider the comparison of the source address of packets crossing
an ingress link to be 'deep'. I consider that check to be very shallow. It
needn't even be every packet from a set, merely picking a random (actually
random) packet and testing for conformity is a good quality control measure
that SHOULD be taken.

What would the comparison be against? Routers are supposed to know which links
are on the other side of all down-stream connections so that they can
effectively route.

------
thomasdereyck
Shameless plug: When I read about SSDP a little while ago I was curious to see
if I'd encounter it on many networks. As I was also trying to learn
Swift/Apple development, I've written two (non-free) little apps for macOS/iOS
to monitor SSDP messages:

[https://itunes.apple.com/us/app/ssdp-
monitor/id1191370425?mt...](https://itunes.apple.com/us/app/ssdp-
monitor/id1191370425?mt=12)

[https://itunes.apple.com/be/app/ssdp-
monitor/id1197048167?mt...](https://itunes.apple.com/be/app/ssdp-
monitor/id1197048167?mt=8&ign-mpt=uo%3D4)

Ever since creating it and just checking on some networks, I'm surprised of
how many devices are actually using it. I probably saw this in Wireshark
before as well, but probably overlooked it because you're never really looking
for it. I wonder if many other such protocols are often used but easily
missed...

------
gbrown_

        More on the SSDP servers
        
        Since we probed the vulnerable SSDP servers, here are the most common Server header values we received:
        
         104833 Linux/2.4.22-1.2115.nptl UPnP/1.0 miniupnpd/1.0
          77329 System/1.0 UPnP/1.0 IGD/1.0
          66639 TBS/R2 UPnP/1.0 MiniUPnPd/1.2
          12863 Ubuntu/7.10 UPnP/1.0 miniupnpd/1.0
          11544 ASUSTeK UPnP/1.0 MiniUPnPd/1.4
    

What an earth is internet facing and running 2.4 Linux kernels?

~~~
falsedan
CEP (home routers)

~~~
ryanlol
CPE surely?

~~~
falsedan
lol yes, not a code enhancement proposal

------
everdayimhustln
Pervasive IoT device deployment without in-the-wild security considerations
and rapid updates is likely to add to DDoS bot farms.

~~~
hellbanner
You're a little late - the Mirai botnet (Larget botnet at the time it became
widely known) is exactly that.

[https://krebsonsecurity.com/2017/01/who-is-anna-senpai-
the-m...](https://krebsonsecurity.com/2017/01/who-is-anna-senpai-the-mirai-
worm-author/#more-37412)

~~~
samstave
Ive always wondered if "anna senpai" is a play on "ono sendai" \-- the
infamous Cyberpunk EvilCorp deck...

I personally feel that it is. (maybe tis was already obvious to others - I
just havent talked about it out oud to anyone prior...)

~~~
krallja
I think the pseudonym "Anna senpai" is in reference to the Japanese media
trope of schoolchildren falling in love with their senpai.

~~~
samstave
< __ _Wipes hands with oily rag, looking at motor..._ __>

Well,now... there's my problem right there... Jus' don' know much 'bout them
japanese now doncha.

< __ _rocks back and forth with thumbs on the straps of my filthy coveralls,
spits..._ __>

Yep yep yep is what I always say...

< __ _heads back into dilapidated datacenter behind squeaky screen door only
holding on by one hinge_ __>

------
ratinacage
I find it fascinating that the packets per second chart resembles an RC
circuit's step response. I wonder if there is a good electrical circuit
analogy for packets, packet size, and bandwidth.

------
walterkobayashi
Is it possible that SSDP Protocol can be run on a non-standard port (10000 -
65535) ?

~~~
dingo_bat
You would have to hack the upnp device's firmware, afaict.

------
IE6
So only slightly faster than GNU yes

------
dsl
It is unfortunate that CloudFlare shared enough PoC code to weaponize this.

Edit: for the downvoters, this isn't just my opinion, please read
[https://en.wikipedia.org/wiki/Responsible_disclosure](https://en.wikipedia.org/wiki/Responsible_disclosure)

~~~
detaro
This is so trivial that it really doesn't matter - it's just sending a
completely normal SSDP request, code for which you could find in any
implementation of the protocol.

