
Update: UniFi Phone Home/Performance Data Collection - bhauer
https://community.ui.com/questions/Update-UniFi-Phone-Home-Performance-Data-Collection/f84a71c9-0b81-4d69-a3b3-45640aba1c8b
======
bhauer
We've seen two episodes of companies who enjoy the goodwill of the
enthusiast/advocate sector committing obvious foot-gun maneuvers in as many
weeks. First it was GitLab, and now Ubiquiti. Both should have known better
than to hoist telemetry on self-hosted software. Doing so betrays a willful
ignorance or disdain (or both) for the very reasons these enthusiasts self-
host.

The good news is that in both cases, the community has been clear in its
reprimand, and the companies have reversed course (or promised to do so, in
the case of Ubiquiti). I'd like to see more companies realize this is an
opportunity rather than a threat: an opportunity to create products and
services that deeply respect privacy and market this advantage over the
competition.

~~~
ohazi
GitLab's CFO was kind enough to leave public evidence of his incompetence:

"I don’t understand. This should not be an opt in or an opt out. It is a
condition of using our product. There is an acceptance of terms and the use of
this data should be included in that." [1]

In the previous thread people were referring to the HiPPO problem (Highest
Paid Person's Opinion). I'm willing to bet that it's the same story at
Ubiquity. But despite occasional backpedaling, there are rarely any real
consequences for the irredeemably stupid managers who push for crap like this.
Unfortunately, that's probably the only way to make it stop.

In my opinion, if the outcry is bad enough for a company to backpedal on a
decision like this, then it's already bad enough that they should fire the
manager responsible for making the decision in the first place. These aren't
line employees. The company doesn't pay them more than everybody else for fun
-- they're paid to understand the business well enough to know better. People
in these positions tend to be natural risk takers. If there are never real
consequences, they're going to continue to bet anything they can get their
hands on for a shot at personal gain, and the company and their customers are
going to foot the bill.

[1] [https://gitlab.com/gitlab-
org/gitlab/merge_requests/14182#no...](https://gitlab.com/gitlab-
org/gitlab/merge_requests/14182#note_203849107)

~~~
ploxiln
Don't penalize GitLab too harshly for having public issues and discussion. The
obvious fix will be to just not have all this public, which is what all other
companies of non-trivial size do. I think the CFO's comment was illuminating,
but not really worse than I would expect from any misc C-level in any
industry.

~~~
ohazi
That's a very valid point, and I don't want to go _too_ overboard in my
criticism. However I don't think my assessment is wrong. If he really was the
one pushing for this decision then I think he should be fired. It's not just
about getting it so wrong when he should have known better, it's about
ignoring all of the internal push-back from employees who were trying to
prevent him from fucking it up the first place. Bullies in the C-suite should
be removed.

> but not really worse than I would expect from any misc C-level in any
> industry.

This is my take as well, unfortunately, but I think it needs to change. There
are loads of competent people who can perform these executive jobs. The bar
should be higher.

~~~
prepend
I don’t think making an executive decision is necessarily “bullying.” The nice
thing about public comments and development is that you would see some non-
hypothetical evidence of such action.

I don’t think it’s unnatural for an OSS product dev team to think this way. I
don’t like that they implemented it, but I think it’s positive they rolled it
back.

Firing a CEO for such a situation will result in worse software, I think. It
won’t make the next CEO less likely to make a dumb mistake. But it will make
them less likely to be public or to roll back changes. If the consequence of a
fairly minor mistake (given GitLab’s history of super smart decisions in the
past 10 years) is firing, then management will make sure nothing is ever
considered a mistake again.

~~~
DoctorOetker
>then management will make sure nothing is ever considered a mistake again.

but it's ultimately the customers who decide if something was a mistake by
switching to different products... hence _company foots the bill_

------
esotericn
Still not enough. Needs to be opt in. As others have stated, crash dumps are
pretty much data leaks.

By pulling stunts like this you force all operators to review the contents of
every update. You actually reduce security across the ecosystem because you
reduce the likelihood individuals will update.

> For any further questions related to this, please review our EULA, Terms of
> Service and Privacy Policy.

May as well be: "For more information, please re-read."

Come on.

~~~
Silhouette
_For any further questions related to this, please review our EULA, Terms of
Service and Privacy Policy._

As someone who was literally about to order a whole set of Ubiquiti gear for
his business this month, having never been a customer before: no, I don't
think I will.

Ubiquiti, you blew it once by trying to push this out in the first place. Now
you've blown it again, by thinking some token opt-out button was enough to fix
the problem, instead of realising that what you're dealing with here is a
major trust issue and you're fundamentally on the wrong side of it. You don't
get a third shot. I need to protect the security of my business data, and I
need to be confident that the professional gear we're buying isn't going to
actively undermine that.

~~~
tw04
While I totally get wanting the option to opt-out of phone home, literally
every enterprise hardware vendor has phone home and expects you to have it
enabled if you want proactive support. Claiming they need to dump the feature
entirely is more than a bit overly dramatic.

The option to disable it was absolutely the right move for them to make and it
should’ve been there from the start. Everyone claiming they should not have
any phone home at all hasn’t spent much time in the enterprise hardware
world...

~~~
Silhouette
_While I totally get wanting the option to opt-out of phone home, literally
every enterprise hardware vendor has phone home and expects you to have it
enabled if you want proactive support._

No, they don't. We work with some clients who are in a very similar area, on
the development side. Anyone who thought this was a good idea would get
bounced out of the room so fast their feet wouldn't touch the floor until they
were in the corridor. It is absolutely not the norm to upload data from within
the customer's network without their knowledge or consent.

~~~
Spooky23
Read more carefully.

Almost all of your enterprise vendors do this today. Some of them even give
Sales Engineers real-time access to performance and utilization metrics. It’s
a great way to get competitive intelligence about other companies.

~~~
Silhouette
You misunderstand. My business develops software, including software that runs
on networked devices, and we have never had a client ask for phoning home like
that. In fact, it has often been a requirement that the device could operate
with no Internet access at all, because a lot of security-conscious customers
such as those in finance or healthcare have strict rules about this.

~~~
tw04
Can you name one?

IBM

Cisco

Dell/EMC

HPe-Aruba

Arista

All phone home. Who is this magical enterprise company that has no phone-home?

This farcical: claiming there are issues with healthcare and finance is
complete rubbish. They aren't phoning home patient data, they're phoning home
telemetry and health status of the hardware. I can tell you with 100%
certainty that both segments have literally hundreds of thousands of hardware
devices phoning home to their respective manufacturers without ANY issue.

~~~
Silhouette
There is nothing magical about it. If you were correct and no-one made
enterprise networking gear without phone-homes, quite a few of our clients'
customers wouldn't have a network, since they are strictly prohibited from
buying any equipment that does. You seem to be extrapolating from limited
experience, possibly of a few specific product ranges or licensing models, and
assuming that the whole industry works the same way. It does not.

~~~
tw04
That’s an awfully long-winded response without actually listing a single
vendor. It would lead me to believe you don’t know what you’re talking about.

I’ve worked with basically every vendor in the space and phone-home is table
stakes which is why your story doesn’t really hold water...

~~~
Silhouette
You've worked with basically every vendor in which space? Surely not
enterprise-level networking, where there are hundreds if not thousands of
sources for numerous different types of device?

~~~
tw04
Let me bring you back since you seem to be trying to change the discussion now
that you're stuck in a corner. We're talking about vendors that provide
wireless networking hardware. Your claim was that any wireless hardware
networking vendor that had phone-home wouldn't fly in hospitals or finance. I
can literally walk into any hospital in the US and prove that wrong, so I
asked you for a list of these magical vendors. There aren't hundreds, there
aren't thousands, there are dozens at best at any given moment. Name one that
is an enterprise wireless networking vendor that doesn't have phone-home.

~~~
Silhouette
_We 're talking about vendors that provide wireless networking hardware._

You might have been, but the comment that I'm replying to is the first one in
this thread to include the word "wireless". Ubiquiti make various other kinds
of network equipment as well, and enterprise networking is a vast industry all
of its own, of which wireless is only a small part.

------
Meph504
I'm confused how they keep saying this is in compliance with Grpd when auto
opt in is not allowed.

And the idea that they force everyone in, and will add an opt out button later
seems like they are gaming the system.

If they don't opt everyone out they didn't ask to opt in as a default, their
actions to me seem hollow.

~~~
eddyg
Anonymous data is _not_ subject to the the GDPR. See recital 26. [0]

IANAL, but as far as I know, companies can collect _anonymous_ data ("what
versions are our customers running?") without involving the GDPR.

> This Regulation does not therefore concern the processing of such anonymous
> information, including for statistical or research purposes.

[0] [https://gdpr-info.eu/recitals/no-26/](https://gdpr-
info.eu/recitals/no-26/)

~~~
tgsovlerkhgsel
I bet they will not actually be GDPR compliant. They will likely get away with
it because enforcement is lacking, but anonymization is hard to get right and
I would be surprised if they did get it right.

~~~
thelittleone
Is there a way to file GDPR complaints? Enforcement might be lacking but with
enough data and support surely it would be possible or be sufficient to make
Ubiquiti move to opt in.

~~~
simpss
contact your local DPA[1] and file a complaint. There's plenty of info on the
original thread[2] on what they're doing wrong.

[1] - [https://ec.europa.eu/info/law/law-topic/data-
protection/refo...](https://ec.europa.eu/info/law/law-topic/data-
protection/reform/what-are-data-protection-authorities-dpas_en)

[2] - [https://community.ui.com/questions/UI-official-urgent-
please...](https://community.ui.com/questions/UI-official-urgent-please-
answer/14259289-e4c3-4c5e-aaa0-02a5baa6cbbe#answer/f0f5de22-396c-4cb3-aad5-9d6038539980)

------
pilif
Contrarian opinion/question: Out of the software developers reading the
comments here: Who is offering an op-out (much less an opt-in into) from their
crash reporting solution they have included in their software?

And of those who do actually offer an op-out: Who is annoyed about not being
able to debug an issue because the crash they know is happening only seems to
be happening on clients who have opted out?

And my second question to all the non-developers out there: Who here is
annoyed about crashes on their machine that "never seem to get fixed" as if
the vendor "didn't care about getting crashes fixed" in software that offered
and explicit opt-in and they didn't opt into "spying" by the vendor? Who
remembers not opting in and remembers to rectify that when the crash happens?

It's very easy to stand on the hill of righteousness and yell at a vendor
implementing crash reporting, but before doing so ask yourself: Are you
collecting crash reports? Do you wish you had crash reports? Are you unhappy
about crashes on your machine not being fixed?

If any of those are true, then you better not complain about a vendor adding
crash reporting to their products.

Do I think this should be opt-outable? Of course. Do I think crash reporting
needs to be opt-in? No. Absolutely not because next to nobody will opt in and
when crashes happen, nobody will remember their decision.

~~~
bchanudet
Make it easy for your customer to get a crash report when one occurs. Make it
easy for them to contact your support, whether it is a Github issues page or a
professional phone support desk.

Make the report readable by a simple text editor. Let the users have a look at
the content inside. If you do need the report, ask them to check it then send
it through trusted and encrypted ways like Firefox Send.

There is no need to make an crash report automatically sent to you. No need to
fix crashes for users that never cared for reporting them to you. Orient your
bandwidth to users that care.

~~~
pilif
_> Make the report readable by a simple text editor. Let the users have a look
at the content inside._

how do you explain the concept of a backtrace to an end-user so they can make
an informed decision whether it's fine to submit the data or not? How do you
do the same with a a full memory dump (if you need one?). How do you expect
users to analyze the memory dump for potentially compromising data?

 _> If you do need the report, ask them to check it then send it through
trusted and encrypted ways like Firefox Send._

this works for you and me. It doesn't work even for my coworker. Also: What is
the advantage of going through the extra hoop of Firefox Send compared to over
plain SSL when you intend the recipient to be able to read the dump anyways?

 _> No need to fix crashes for users that never cared for reporting them to
you_

Judging from metrics available to an app I'm working on I would say that users
are much more likely to stop using my app completely rather than even
bothering to press the "report this issue" bug.

It really depends on your target audience.

But in case of UniFi (as opposed to their AmpliFi line), asking for individual
reports to be reported could possibly be feasible given the product audience.

------
kuon
This has already been added to the excellent host block file of StevenBlack.

[https://github.com/StevenBlack/hosts/issues/1083](https://github.com/StevenBlack/hosts/issues/1083)

------
jlgaddis
I've been saying for several years that, eventually, we will all end up having
to put in "default deny" rules for our _outgoing_ traffic, just as we do today
for incoming traffic.

(Either that, or hosts will have to be configured without a default route and
will need to use a proxy server for all outgoing connections.)

When every piece of hardware and software you use is spying on you, blocking
everything by default and only allowing explicitly whitelisted traffic will be
your only option to stop it.

~~~
daedalus_j
I think we're already there.

Site I manage run separate VLANS for admin (switches, APs, etc) and Internet-
of-Things, and both of those VLANS are default deny on outbound. That rule
blocks a fair amount of traffic.

I wonder if part of the answer is better tools in firewalls for this sort of
thing. Easily tagging "unstrusted device" or something perhaps.

~~~
lostlogin
Pihole, a router that forces anything on port 53 back to the Pihole and
careful device selection seems to be very much required already.

------
notaplumber
For engineers reading this who work at a tech company, or well, any company.
Watch this keynote talk by Patricia Aas.

It's called Embedded Ethics. And it is crystal clear many of you need to watch
it.

[https://www.youtube.com/watch?v=HfNIiitVFtcalk](https://www.youtube.com/watch?v=HfNIiitVFtcalk)

------
alkonaut
Opt out is good. But I just want to weigh in for the statistics that I have
almost zero concerns about anonymous crash reports. I worry a lot more about
whether products requires internet connections to their manufacturers to work
ten years later, than whether they send home crash data.

If it’s clearly indicated and has an opt out I’m fine with it. I’d go so far
as preferring it over opt-in, because I want my product to have the
improvements of as many crash reports as possible.

Obviously, by “crash log” I hope they gather call stacks - not memory content,
since memory content could contain data that I wouldn’t want collected.

------
ErneX
I'm glad they will add the option, but considering their user base background
it was really naive to don't foresee this was going to be detected and that it
was going to be an issue.

------
ericd
Does anyone know if they've announced the same thing for their EdgeMAX gear?

This is really dumb, these are the kind of shenanigans that people buy non-
consumer gear to get away from.

~~~
ocdtrekkie
Safe assumption is they'll add it eventually to all their hardware. But given
the reaction they got here, they'll get the opt out setting done before they
spread it to other devices.

~~~
ericd
Thanks for your thoughts. I’m hoping that the more “serious” bent of the
EdgeMAX stuff (much of the functionality requires dipping into CLI) will cause
them to hold off there. I’m assuming most businesses would find this
unacceptable. But maybe you’re right.

~~~
ocdtrekkie
Well, I think the amount of CLI config required, means most EdgeMAX users
won't think much of having to add a "disable telemetry" command in their
configs: They already have to add like _all the basics_ manually to begin
with.

But the other thing is, since UNMS devices like EdgeMAX and AirMAX are
intended for mass deployment by ISPs, there's a decent chance UNMS will reveal
and aggregate that telemetry data and allow UNMS users to see it themselves
for their own troubleshooting and diagnostic purposes.

~~~
ericd
Neat, thanks again, you clearly have dug more into these devices than I have
:-) I haven’t tried UNMS (mostly for multi-site deployments/massive overkill
for home use?) but hopefully that means that it’ll be fairly easy to disable.

------
newhotelowner
Power outage corrupts Mongodb in the UniFi Cloud key. They won't fix that.

There is no option to remove the dead/unplugged devices from the device list.
The only way to remove is to reset and reconfigure everything from scratch.

~~~
LeoPanthera
> Power outage corrupts Mongodb in the UniFi Cloud key. They won't fix that.

The Cloud Key 2 has a battery inside it and shuts down gracefully if you pull
the power.

So that's a _kind_ of fix. Although it does involve buying a new device.

~~~
lostlogin
Avoiding dedicated hardware and running it on a machine with a UPS is another
way. I can recommend Docker, Synology and a small UPS. And that’s how the
equipment cost got multiplied by 10.

------
purpleidea
Their "protect" camera software doesn't work properly without it being able to
connect to the net. It won't let you login on the localhost!

Now this. What is a good alternative for AP's and cameras?

~~~
rb666
After trying several inferior solutions, including the Unifi cameras, I
switched to Blue Iris in combination with high quality Dahua cameras (IPC-
HDW5231R-Z). It's been very reliable and overall just works great.

------
8fingerlouie
I posted this as a reply to a comment, but I will repeat it here,

If they transmit crash logs, and those contain partial memory contents (core
dumps), chances are actual content from network packets are transmitted to
UBNT as well, and those contents may contain PII (IP addresses, MAC addresses,
usernames, passwords, real names, addresses, credit card information, etc.),
meaning they’re most certainly violating the GDPR by making it opt out.

The GDPR doesn’t care how you came by the PII, it only dictates how you should
behave when you handle it.

Edit: I should clarify before somebody trips over their pitchfork and sets the
house on fire, the GDPR cares a lot about how you come into possession of PII,
and also if you should have that information in the first place, and also
cares about how long you hold on to it.

When it comes to protection of those data though, it doesn’t care if it was
collected through telemetry, emailed to you (yes, also the spam folder), or
it’s an old archive of core dumps on an old server somewhere. The same rules
apply to how those data are handled.

~~~
kasey_junk
This is simply not true. Opt in is only one of many ways to be compliant with
GDPR covered data.

Legitimate interest is the obvious way to not require opt-in and while I’m
sure the regulators are getting tired of seeing ad networks claim that
exception a _router_ company claiming it for crash support seems like
something that will likely be deemed in bounds. Especially if that company has
made a good faith effort to show its not repurposing that data.

~~~
8fingerlouie
>This is simply not true. Opt in is only one of many ways to be compliant with
GDPR covered data.

And yet, Article 32 ([https://eur-lex.europa.eu/legal-
content/EN/TXT/PDF/?uri=CELE...](https://eur-lex.europa.eu/legal-
content/EN/TXT/PDF/?uri=CELEX:32016R0679of) the GDPR states:

    
    
        Silence, pre-ticked  boxes or inactivity should not  therefore constitute consent. 
        Consent should cover all processing activities carried out for the same purpose or purposes. 
        When the processing has multiple purposes, consent should be given for all of them. 
        If the data subject's consent is to be given following a request by electronic means, 
        the request must be clear, concise and not unnecessarily disruptive to the use of the service for which it is provided.
    

It is of course also possible that UBNT is fully GDPR compliant if they can
guarantee that the don't send PII. The problem is we don't know.

~~~
kasey_junk
You are misunderstanding. You only need to collect consent via opt in if
consent is your basis of collection.

There are several other basis as well including ‘legitimate interest’. You can
see all of the basis in article 6(1).

If you use a different basis you don’t need to collect consent _at all_
whether opt in or otherwise.

Hardware data from crashes used just for cause analysis and properly
stored/secured is a fairly straightforward ‘legitimate interest’ argument.

------
shantara
Does anyone else observe hourly connection attempts to "unifi-report.ubnt.com"
from their Unifi Controller? I'm running the controller locally with all cloud
features, error reporting and live support disabled. Not sure what other
options user-facing options I can turn off to stop this from happening.

------
ComodoHacker
The biggest concern for me is not opt-in vs opt-out, and not even data mining.
They create another side channel which can and eventually will be exploited by
the party they can't imagine now. They weaken the security of their otherwise
good products.

------
Youden
"transmitted using end-to-end encryption"

Technically correct but sounds either ignorant or misleading.

~~~
thijsvandien
In other words: over HTTPS.

------
Jonnax
For "enterprise" hardware/software I would expect them detailing exactly what
metrics they collect and what could be in a dump that gets sent.

How do you anonymise a dump with user info especially for a routing device?
Like that requires some sophistication, you'd imagine they'd love to talk
about it to generate blog clicks.

But it seems that some keen user will have to try reverse engineering their
data collection to actually give the users information of what is leaving
their network.

As a user of their products, it has really soured me on their product. When
supposedly if they're so "GDPR compliant" they just need to explain themselves
to win people over.

------
denkmoon
It's unclear from the post, does this impact the firmware on my unifi devices,
or is this part of the controller software? ie. is my raspi phoning home, or
are my wireless APs phoning home?

------
punnerud
Most likely a follow up on this posted 22 hours ago on HN:
[https://news.ycombinator.com/item?id=21430997](https://news.ycombinator.com/item?id=21430997)
(Ubiquiti adds phone-home to the access point firmware)

------
jwr
I do not understand how somebody at UBNT thinks the data is worth the hassle.
They are taking a reputation hit, risking getting PII and getting rolled over
by the GDPR steamroller, angering some of their best customers (corporations
with strict security rules). What for? Is the crash data _really_ so valuable?
Do they intend to pore over every report with dedicated manpower? (they really
should if it matters so much).

I would say they should first get their basic support to work in a top-notch
manner, carefully reading, thinking about, and responding to every bug report.
Then get into automated data collection, and then ASK FOR USER CONSENT. This
isn't difficult. No "opt-out" crap or corporate lingo, TELL ME CLEARLY WHAT
YOU NEED, WHY YOU NEED IT, AND ASK ME IF I AGREE.

------
Godel_unicode
"If you do not wish to participate/provide this data, we will add an opt-out
button in upcoming versions".

