Hacker News new | past | comments | ask | show | jobs | submit login
Garmin Outage Persists After 24 Hours Downtime (twitter.com/garminfitness)
96 points by seanhandley on July 24, 2020 | hide | past | favorite | 70 comments

It’s not just some fitness smartwatch, as the linked Tweet might lead you to think:

> Pilots told the tech website that they had not been able to download new Garmin software with up-to-date versions of the aviation database, which is a legal requirement for flying. The Garmin Pilot app, which is used to schedule and plan flights, was also hit by the attack.

Source: https://news.ycombinator.com/item?id=23937097

PS in case you’re not into aviation - Garmin is used in smaller aircraft, not airliners but this is still a significant disruption - smaller planes and helicopters carry out important roles such as firefighting, heli ambulance, flying doctors services (e.g. in Australia) rescue ops etc.

Garmin has a lot of market share in small piston / hobby flying planes, but they're also big in small to mid size private jets and turboprops. More and more are using the Garmin G1000 and G3000 cockpit or older models upgrading from traditional instruments to the G600. In those cases it's not uncommon to use Garmin Pilot + Jeppesen for the good integration with the onboard systems when loading flightplans etc.

So this outage could be painful for some corporate flight departments and air taxi companies as well. Luckily you can also print the charts without that much trouble and fly like the old days.

But I bet ForeFlight sales just spiked

FlyGarmin is already back up. And it’s not a huge deal in my opinion. In addition to being only smaller operators, dbs don’t expire and roll over on the same day. New releases are available early.

Absolute worst case, if you were flying IFR and it was worth the expense, you could go to Jeppesen and buy a 1-off update for your avionics. Garmin Pilot could be replaced by a ForeFlight trial, etc. It’s not like Garmin goes down, planes are grounded.

The real issue is Garmin's lack of communication.

I'd be willing to bet that their legal and finance teams are fighting to keep them from saying anything (their earnings call is on the 29th).

So much goodwill is lost by companies that don't communicate when problems are affecting customers.

>The real issue is Garmin's lack of communication.

And here we are a full day and a half into the outage and no update and no ETA, just the same generic "Sorry, we have an outage bro" message that they put up a few hours in. I guess if this really is WastedLocker they're just sitting around arguing over whether to pay the ransom.

I'm sure it's a massive fire inside their walls right now. A tweet over 24 hours ago is definitely not going to bring confidence on any progress. Speculation will fuel a lot of negativity until an official update is provided.

Working in transaction consulting I beg to differ: This simply already puts the earnings call on Wednesday in a bad light, much more so than if they e.g. communicated the real issue and, if required and possible, delayed the earnings call... Transparency is valued and priced highly by analysts these days, the worst scenario is that there is no real transparency in the call itself, then "all bets regarding GARMIN's future are off".

Because as a customer, you're still the product.

I'm sure any communication will have to minimize stockholder impact and will be watered down instead of being 'Oh shit, all of our crap is encrypted'

> Because as a customer, you're still the product.

That is simply false. I'm fine with people making that claim about companies like Facebook and Google - they make most of their money selling advertising, so yes, in that sense, their users are a product they are offering to advertisers. But that's not how Garmin works.

Garmin provides products to us, and offers services that they hope will keep us in their ecosystem. Unless you have reasonable evidence that they are lying in their privacy polices, such as at https://www.garmin.com/en-US/privacy/connect/ , then you cannot reasonably make the claim you have made.

I understood the parent to be commenting that Garmin is repackaging customer revenue as a securities product, i.e., their common stock, which they sell to investors. Point being that investors are the key stakeholder, not customers.

It's not a unique POV for a company to have, it's just it's maybe more a subject of laser focus for companies in this era.

That could be extended to all publicly traded companies using that logic.

And a lot of people make that argument. I'm not saying it's right or wrong, just that this isn't some flaw in the theory - the theory is exactly that most, perhaps all, publicly-traded companies are answerable to their shareholders, not their customers. Attempts to reform or mitigate this often fall under the umbrella of "stakeholder capitalism," in which the customers are viewed as one of the stakeholders that needs representation, because they don't currently have any.

I got a feeling customers are more "sentient" than stock holders that can't care less about the whys or whats about the company. It is just another stock in the portfolio that is traded at market equilibrium among other stocks.

The customers has to do a informed choice.

Rumour has it that it's been caused by a ransomware attack (although Garmin hasn't commented publicly) - I can't think of much else that would cause such a long outage.

I thought the same. Especially the fact that their webservices as well as their callcenter and other support systems are down screams ransomware attack.

I'm sure we'll see a long and well funded security department after this incident at other companies /s

My experience with Garmin devices is that the hardware is good but the software is average. GPS device for example are robust and they can function both with batteries and piles BUT the software can crash, searching for names on a map or entering text is almost guarantee to not work. They are years behind Android for example for user friendliness.

Their main distinguishing market at the moment seems to be wearables for athletes as well as dedicated GPS for aviation, etc., not consumer turn-by-turn navigation where smartphones can perform adequately. For example, tracking and providing analytics for training, particularly for running, bicycling, and other distance-based outdoor activities.

that also seems to be a much higher value business - towards the tail end of when dedicated gps navigation devices were reaching full saturation, before everyone switched to just using google maps or waze on their phone, they were down to like $89 per piece... Or the functionality is now built directly into the car. Whereas the higher end fitness watches and their 'tactical' product lines are $350 to $950.

Their software is average compared to maybe other web services, but when compared in the segment - the alternatives are often even worse - I came to Garmin from the Suunto world and it was a big step forward - before, i was facing constant syncing issues, very poor sync software, migration to a newer platform at Spartan watch era was a sad story (multiple different web platforms, missing functionality, etc). Garmin works pretty flawlessly for me, though I'm primarily using Strava for all the analytics (or used till they limited their free tier recently)

Ever since my Oregon 650 refused to boot until I connected it to my computer I kind of lost faith in their hardware. I was hiking so connecting it to a computer was not an option. Needless to say I was not amused.

The best line for outdoor is still the GPSMAP-Series (the 62st for example), very robust but heavy.

Agreed, I've had 2 in that family and never a problem in the field. I just wish it had the larger screen like the Oregon series.

>I just wish it had the larger screen like the Oregon series.

Yes true, the screen bigger and buttons less smush'i :-)

Piles? Is that a typo for wires?

A Pile was the first type of battery [0] and you can still see this in French where la pile is a battery (or more strictly usually used to refer to a cell).

That being said it looks like OP might be German and I've never heard anyone using the term die Säule in this context.

I would take a guess that they are drawing a distinction between rechargeable and disposable batteries.

[0] https://en.wikipedia.org/wiki/Voltaic_pile

May be are referring to thermo-piles, which convert heat to electric power using bimetallic joints.

Better link with more details: https://www.zdnet.com/article/garmin-services-and-production...

A couple important excerpts (there's a lot more detail in the article):

> ... flyGarmin has also been down today. This is Garmin's web service that supports the company's line of aviation navigational equipment.

Other HN commenters have already elaborated on the implications of that.

> ... while we confirmed that this is a ransomware attack, we could not 100% verify claims that this was caused by WastedLocker.

Garmin hasn't officially commented on the cause, but they did tweet that their call centers are down (https://twitter.com/Garmin/status/1286278816302850048):

> This outage also affects our call centers, and we are currently unable to receive any calls, emails or online chats. We are working to resolve this issue as quickly as possible and apologize for this inconvenience. (2/2)

https://status.inreach.garmin.com/ has some status updates and says that SOS communication isn't affected.

More precisely:

"inReach SOS and messaging continue to work."

"inReach SOS and messaging have been fully functional and remain so."

This is reassuring because people who go to remote places and rely on the inReach satellite SOS and messaging have not been affected. If they had been affected, they would've been cut off with no explanation--for example unable to relay status to family, or to request an early pickup. While not in itself life-threatening, it would cause a lot of unnecessary worry and possibly unnecessary rescues.

I had wondered if their satellite messages was affected. Obviously, a message goes from the unit in the field to a ground-station, and then to the recipient's regular messaging or email. The question is whether it touched one of Garmin's servers to do that, and if that server was affected--apparently not.

I'll bet there's a PM somewhere who had to fight like hell to keep those systems separate and is feeling pretty damn vindicated right now.

There must be some regulations that need to be met to run an SOS service? I bet if those didn't exist the SOS service would be down now as well and peoples lives would be in danger.

The rest of their services clearly had as little spent on them as possible to "maximise shareholder return".

Found a pm, ha! Whoever it was, good job. It's astounding that companies don't seem to have real recovery systems, this keeps happening.

That is largely a luck and legacy thing. inReach is an acquisition by Garmin. They haven’t made significant changes to hardware and software besides rebranding things and adding integration to their newer units, e.g. I can trigger SOS from my watch.

In fact the billing is completely separate.

Frustratingly, you still can't sync data with the Garmin Explore app, which means that you can't sync data with the actual inReach device (sign in is broken).

You can send SOS and tracking points, and send/receive messages. But you can't get new courses and/or maps onto the device.

Is this affecting anything important, like aircraft, boat, or car navigation, or just the fitness trackers?

Oh man. I was mildly amused that people couldn’t access their fitness history. I just tried to access flygarmin.com to update my aircraft’s aviation databases, and this shit just got real.

My airplane is grounded for IFR flights — I always fly IFR. I pay Garmin $865/year for subscription. There are thousands of aircraft in the same predicament.

The fact that GARMIN is incapable of offering e.g. aviation updates via an alternative route such as downloading from a website and then manually uploading to a device (needs only to be open in emergencies) is simply ridiculous. This obviously also goes for any other of their services, but obviously with aviation (and potentially also maritime) data there is a much higher and substantial need to offer such services, and I also agree: The fact that no SLA for anything seems to be (offically) in place along with the fact that not a single statement regarding expected downtime is made... I stop there!

Does Garmin offer an SLA on these services? I feel like if I paid that much annually for a service they better well give me an SLA.

I don’t know. I’ve never even thought about it. Until now.

Quick question: Does anyone know who runs/hosts/maintains/secures GARMIN servers? Are they owned and operated by the company itself or is all or parts of it outsourced?

flygarmin.com is back online.

Aviation database updating has apparently been hit by this [1], and pilots are mandated by the FAA to keep them up-to-date. Car navigation won't be significantly affected as long as the roads haven't changed drastically since the maps were last updated - all the plotting and directions are done on the device itself.

1. https://www.zdnet.com/article/garmin-services-and-production...

Their call centres are affected so I suspect any sort of assistance with the above services is affected. Fitness trackers still work, you just can't sync with the app.

For a view of how GARMIN must feel right now: https://www.feltet.dk/octo_cms/files/Feltet.dk/Billeder/2014...

Two weeks ago I posted that I was suspicious of using 'cloud' based fitness data aggregation systems:


In this case I hate to be proven right, but it's not looking good for Garmin. There's lots of road cyclists out there with $750 useless watches now. I can tell you that after this event the odds of me ever purchasing a Garmin device that relies on anything 'cloud' based have even further decreased.

Even if the watches can function offline, how can anybody have any degree of trust that all of their previously uploaded data has not been stolen? Based on the reported use of ransomware and the very lengthy downtime, it really sounds like Garmin's network was owned quite thoroughly. Is there some group out there now in possession of hundreds of thousands of .gpx files with detailed tracking points of peoples' residences, favorite running and cycling routes, and what times of the day they're usually away from home? Nobody knows.

After seeing 20+ years and many dozens of instances of data breaches from this that we would now define as 'the cloud', I find that the only solution is to simply not upload to a third party anything you consider proprietary information.

Meanwhile my fully offline or local-workstation-hosted GPX based tracking method continues to work normally.

> There's lots of road cyclists out there with $750 useless watches now. I can tell you that after this event the odds of me ever purchasing a Garmin device that relies on anything 'cloud' based have even further decreased.

They work offline. At least, as much as I use mine it still functions, there may be more advanced features.

> Meanwhile my fully offline or local-workstation-hosted GPX based tracking method continues to work normally.

You can still get the gpx files right off the watch. Apart from that, this is the classic Hacker News argument of "why do the normies rely on these cloud services it's trivial to <insert giant complicated setup here>".

It's awesome that it works for you. My parents, one of whom in his retirement hacks on code that combines local drone captured data with local government LIDAR data and parses it for more accurate maps of his lifestyle block, don't have time for those shenanagins. The expectation that everyone does is folly

it's not a giant clunky complicated setup - it's a 15 second process once a week of transferring a gpx file from an android phone over to a PC running this:


If I really wanted to automate it, I would use some sort of tool to do the equivalent of a cron job to scp the files from the contents of /sdcard/bike/*.gpx to my desktop PC.

As the other person said, all that you're doing can still be done by these watches if people want to.

On your system though: what happens if your computer crashes, gets a virus or you otherwise lose the data? You should probably have a backup system right? Now you're adding more steps, and either doing a reasonable complicated 321 setup manually, or back to involving the cloud.

I'm not saying what you're doing is bad, but dismissing easy to use cloud systems is just silly. Ironically I push my runs to the cloud (or clouds, as I go watch -> garmin -> strava) because it's an offsite backup that I don't have to manage. I also then export strava data and re-back it up myself, but I'm a tech nerd so that's just icing.

You're also side stepping any social features. I know people who are very encouraged to exercise because they see their friends doing it, or because they get kudos / likes and praise when they post their workouts. This may not be important to you but for many people it is.

>I'm not saying what you're doing is bad, but dismissing easy to use cloud systems is just silly

I think what people are really trying to say is that the Connect app should have some sort of "Store my previous n number of Activities locally on my phone for offline viewing" option. These files are typically just a few MB and my Android phone has like 4X as much storage capacity as my Fenix watch. As it stands, I can't even execute a sync between my watch and my phone right now because their cloud is down and that's just ridiculous.

Why a cloud based service failure should be a barrier to transferring files by Bluetooth or wifi between two devices in the same room is absolutely ridiculous.

I definitely agree with this, and I also agree with others who say that Garmin's software is generally the weakest part of their offering

This always bothered me about the Connect app.

The watch has every run from the past 2 years on it. I want that information on my phone as well.

It'd be nice if they did both. Have an easy to use experience and also a simple path to data access.

(Like the device could share the data over Wifi/Bluetooth)

You can do that offline with Garmin watches right now, along with heart rate data, and can view GPS+HR using a variety of online platforms like Strava. The .FIT files are stored on the watch itself and can be accessed over USB.

The online features that aren't currently working involve syncing wirelessly with your phone and their servers, and more uniquely analytics on training (effectiveness, recovery, records, etc.) that your method doesn't seem to offer.

> can be accessed over USB

True, but some devices (like the Fenix 6 lineup) don't mount as USB mass storage.

For those, Android File Transfer [0] seems to work.

[0] https://www.android.com/filetransfer/

> They work offline. At least, as much as I use mine it still functions, there may be more advanced features.

Not sure if "tracking an activity" only is really considered "working offline". Customization of the devices is a major functionality of GARMIN products and ONLY works with a "Connect connection". Even changing the watch face is not possible, sometimes the watch tries to "verify Connect IQ Apps" which is not possible and you have to restart the watch, to detail just two major nuisances. Not even mentioning people who bought a GARMIN device since the outage started and did not know about it - they are currently proud owners of expensive paperweights and dust collectors. Look out for many pre-owned GARMIN products on eBay in the coming days and weeks!

But when you think about it currently for most users computer = internet. No internet is almost equal to non functional computer. No gmail, no FB, Insta, Twitter, no news sites, no messaging. For some even no document editing if you are using online editors and are not prepared in advance by installing offline editor office 365 or sketchup). And also not many have offline maps ready.

It's scary how much we became dependent on internet connectivity.

> Is there some group out there now in possession of hundreds of thousands of .gpx files with detailed tracking points of peoples' residences, favorite running and cycling routes, and what times of the day they're usually away from home? Nobody knows.

Yeah it’s called anyone who goes on Strava.

The watches are not useless, the only useless thing is the app and the Garmin dashboard. Before, when I had a watch only with USB/ANT client, I used to sync it once per week, the only thing you need to do now is to just get the .fit files and upload them on Strava or whatever platform you want or process the files yourself.

I can also still sync Spotify for my music and podcasts, so that works.

You can still copy data from the Garmin watches manually. For a lot of the watches, this is mainly affecting the ability to use Garmin Connect to view your aggregated data.

It IS frustrating - I have a Fenix 3, so I can relate. But it is still tracking the activities and the service will be back up, so saying that the watches are useless right now is rather hyperbolic.

Those watches work perfectly fine offline, and that's how most people use them. Mine isn't even set up to connect to my phone.

You don't have a need for a 750 dollar watch with dedicated onboard maps and gps unless your use case is specifically being outside of coverage areas to start with.

This is not good but sadly any source of massive amounts of user data will continue to be targeted in this day and age. I'm an avid cyclist and use Garmin Edge GPS cycling computers. Luckily they store activities locally. I'm sure I'll be able to sync my rides soon enough ;)

Asked in a related thread already, but bigger audience here: Does anyone know who runs/hosts/maintains/secures GARMIN servers? Are they owned and operated by the company itself or is all or parts of it outsourced?

Should I block credit cards i have connected with Garmin Pay?

Forensics can sometimes prove "X happened", but not "Y didn't happen".

That's why sometimes companies will make statements like "we have no evidence that the hackers did Y1, Y2 or Y3". It doesn't mean anything really.

That is to say, once Garmin becomes communicative again, they may be prescriptive in answering questions like yours head-on, or due to lack of concrete proof, punt and obfuscate.

Suffice to say, it appears they've been owned through-and-through, so you may want to err on the side of caution.

Welp, here it is - from https://www.garmin.com/en-CA/outage/ :

> Garmin has no indication that this outage has affected your data, including activity, payment or other personal information.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact