
GPS error caused '12 hours of problems' for companies - majke
http://www.bbc.com/news/technology-35491962
======
pjc50
In particular, DAB depends on exact time synchronisation to avoid
interference. Every transmitter in the country has to broadcast the same OFDM
symbol at the same time.

[https://en.wikipedia.org/wiki/Single-
frequency_network](https://en.wikipedia.org/wiki/Single-frequency_network)

The kit is supposed to accomodate loss of GPS, but evidently doesn't handle
error so well: [http://www.microsemi.com/products/timing-synchronization-
sys...](http://www.microsemi.com/products/timing-synchronization-systems/time-
frequency-distribution/gps-instruments/4370a-dvb-sync-source)

~~~
elfchief
It's not that surprising that it has a problem with error, because the reality
is, there's no correct solution for dealing with this kind of error.

The first question, of course, is "how do you determine when there's an
error?" Half the GPS constellation was broadcasting the data, and for almost
all purposes it was 'correct' data, other than being wrong. Perhaps ground
stations could determine that they're getting different data from different
satellites (if they have some good ones in view), or maybe they could just
decide that the new parameters differed enough from the old ones to be
implausible, but neither of these is a cut and dry "I definitely have an
error" ... but let's say you have a foolproof way to determine that A0/A1 are
incorrect. Great. THEN what do you do?

You could just decide to use the previous value for that parameter... which is
great for a single clock, but what about clocks elsewhere that have a
different value (the one being broadcast by the rest of the satellites) or an
out of date value (say, a clock that was powered off while traveling in a
broadcast van)? Barring external communication, you could have an arbitrary
number of different interpretations of the current time -- a different one for
each ground station!

You could also just accept that the new parameters are correct, but how do you
then get your clock to the new correct time, without having a second (or other
time period) that's significantly longer or shorter than a second? There's a
lot of gear that expects a second to be a specific length, which is not gonna
be happy if the length changes. (Think of playing back an audio file that's
exactly 30 minutes long, and needs to start and stop at specific times, down
to the bit. You then configure your system to send one bit every (1/bitrate)
seconds exactly. How does that system recover when 13µs of bits go missing?
How do you coordinate that behavior over multiple disconnected systems around
the world which may have gotten the same information at different times (or
not at all)?

From reading the article, it _sounds_ like what happened was that the units
didn't _fail_ because of the GPS issue, but that they start throwing alarms
(probably while going into a mode where they ran off only their internal
(disciplined) oscillator. This seems like absolutely the right thing to do --
"uh, boss, things look really not-right here, and I can't fix it without
breaking something, you really need to come deal with this yourself." (Note
the comments about 'thousands of system warnings' but none about 'cellphones
broke.') This is, from what I can see, really the only reasonable behavior
these devices could have.

(disclaimer: I don't have any direct data on what broke vs. what complained
loudly and woke people up. I'd love to have some solid, not-parsed-by-the-
tech-media data on this topic, if anyone has any.)

~~~
contingencies
I don't know anything about the specific event.

However, regarding your question of how to programmatically handle partial
failure scenarios, I would reduce the trust in individual components (eg.
satellites, receivers, receiver firmware releases, etc.) via redundancy and a
voting-type mechanism to evaluate individual component outputs and their
changes over time versus probable reality. Such a system would possibly
include multiple offline clocks to match prior signals and thus maintain
increased fault/drift-tolerance.

------
jevinskie
This is the press release that I received:
[https://ghostbin.com/paste/hdj3j](https://ghostbin.com/paste/hdj3j)

------
spencerhakim
It's fascinating that an amount of time small enough to be completely
meaningless to most of us can have such a big impact.

------
skykooler
Interesting. Would the 13-microsecond discrepancy mean that GPS positions were
over two miles off during those 12 hours?

~~~
bjcubsfan
No, the discrepancy was only on a timing message for determining UTC. This is
not used for determining position.

~~~
cnvogel
Specifically, one or few of the sattelites sent a wrong value "A0" in its
"Page 18 of subframe 4" message. Page 37, §2.4.5.5 in
[http://www.gps.gov/technical/ps/1995-SPS-signal-
specificatio...](http://www.gps.gov/technical/ps/1995-SPS-signal-
specification.pdf)

[https://www.febo.com/pipermail/time-
nuts/2016-January/095692...](https://www.febo.com/pipermail/time-
nuts/2016-January/095692.html)

------
madaxe_again
I'm curious as to why they're using atomic clocks in space for a high
resolution time source, rather than an atomic clock on earth... Latency?

~~~
elfchief
The problem with atomic clocks in general is that they have very good short
term stability, but not-so-great long-term stability. So they are very good at
measuring the amount of time that has passed between two events, but not as
great for telling you exactly what time an event happened.

The other problem is that there's not really such thing as a single, true
time. Precision timekeeping is one of those places where relativity matters --
differences in altitude and the geology of the earth under your feet at a
particular point change how fast any particular clock 'ticks'.

So to accommodate these differences, if you want events to be synchronized
across multiple locations over a long period, you both need a single agreed-
upon "current time", and some way to get information about that time
information to the sites that need it.

You could surely do this with directly connected wires that were very
carefully measured, but that gets expensive very quickly. The GPS system
already requires super-precise time just to function, so it makes for a great
distribution mechanism of "what time is it" information, where the people
running the GPS system are bothered with all the details of figuring out and
standardizing a single time, and you 'just' have to listen.

The GPS folks also keep track of things like how fast the GPS standard clock
is drifting from the UTC standard clock (since no two clocks will agree over
the long term, period). And this is actually where the problem happened -- Two
of the parameters broadcast (A0 and A1) were incorrect, and it's these
parameters that are used to specify the current time difference between the
GPS master clock and the UTC master clock (the GPS master clock is disciplined
to be as close to UTC as possible, but is always off by a minute amount, and
that's what A0/A1 represent).

Pretty much it boils down to, much as it is in distributed systems, "there is
no now". Even with maximum care and the highest quality clocks, there's really
not a definition of 'now' that's consistent across more than a single
location. GPS is currently the closest thing we have to a globally-distributed
definition of 'now'.

~~~
cnvogel

        > atomic clocks in general is that they have very good
        > short term stability, but not-so-great long-term
        > stability
    

Well... The normal number to look at is the "Allan Variance". Over a timescale
of days (100'000 seconds) a good rubidium will achieve about 10^-12, which is
10ns. A hydrogen maser will have 10^-13 or 10^-14 over even longer timescales.
This is pretty great long-term stability in my book. A good, but
undisciplined, quartz oscillator will have 10^-12 over timescales of hours.

[http://ivs.nict.go.jp/mirror/publications/gm2002/takahei/img...](http://ivs.nict.go.jp/mirror/publications/gm2002/takahei/img8.gif)
[http://www.ke5fx.com/rb.htm](http://www.ke5fx.com/rb.htm)
[http://www.thinksrs.com/assets/instr/PRS10/PRS10diag2LG.gif](http://www.thinksrs.com/assets/instr/PRS10/PRS10diag2LG.gif)

By definition, anything steered by GPS will have infinite precision for
infinitely long time-scales, as GPS time and UTC time ultimately are steered
such that they are within a certain tolerance of each other.

[http://www.leapsecond.com/pages/tbolt-
tc/](http://www.leapsecond.com/pages/tbolt-tc/)

And UTC time _is_ the pretty elaborate average of ensembles of atomic clocks,
hence atomic clocks produce the international reference time there is.

You can, of course, improve the long-term stability of _any_ clock by locking
it to a GPS (or other means of time-signal distribution). And for a good
Caseium that might only require you to look at the phase, and slightly nudge a
potentiometer (magnetic correction field adjustment) on the front by a tiny
amount every week. But atomic clocks in general are pretty darn impressive.

 _UPDATE_ : This presentaiton has all the interesting plots combined, it
seems:
[http://www.hipster.net/ShadNygren_Atomic_Clocks_for_Amateur_...](http://www.hipster.net/ShadNygren_Atomic_Clocks_for_Amateur_Radio_Astronomy.ppt)
(pages 45 and following).

~~~
elfchief
Nice presentation! Thanks for the link!

The particular nit you pick here is a fair one, if you take my comment
literally, rather than as the (over?)simplification of a complex concept I was
trying to summarize.

Yes, in absolute terms, an atomic clock will have excellent stability over the
long term _in its own frame of reference_. As long as you don't need your
atomic clock to match any other clock in the universe, the long-term stability
is excellent.

The particular topic being discussed here, though, is equipment that is
designed to synchronize events across a large area. Even atomic clocks that
are literally 100% accurate will diverge if installed in different locations.
So if you want your atomic clock to actually be able to tell you _what time it
is_ (in a way that matches up closely with anyone else), you have to
constantly be adjusting it to match a common standard (e.g. UTC)

These adjustments are a form of instability, since to correct for changes, you
have to make your seconds either faster or slower than they actually are
locally, on a constantly-changing basis, or you have to step your time and
have less (or more) than (Y - X) time actually elapse between timestamps X and
Y.

As soon as you move beyond a single frame of reference, you _can 't_ have both
short and long term stability.

I suppose I could have used a different word than 'stability', but that seemed
to get across the underlying effect best. I suppose another way of saying it
(that is more correct) is that atomic clocks are great for telling you how
much time has passed (in your local reference frame), but not so great for
telling you what time it is. GPS is great for telling you what time it is, but
not so great for telling you how much time has passed (in your local reference
frame).

(The best clocks for timekeeping, of course, are atomic clocks disciplined by
a common reference like GPS, which gives you the option of deciding your own
long term vs. short term tradeoff however you'd like. But the original
question was "why do we need GPS at all, and not just atomic clocks?".)

------
fudged71
Funny, I just saw this post yesterday by DARPA about preventing GPS issues:
[http://gazette.com/defense-department-researchers-push-
plan-...](http://gazette.com/defense-department-researchers-push-plan-aimed-
at-avoiding-worldwide-gps-meltdown/article/1569000)

------
Johnny_Brahms
"After the initial jolt, one of the first effects you'd notice would be that
your GPS would stop working. The satellites would stay in roughly the same
orbits, but the delicate timing that the GPS system is based on would be
completely ruined within hours"

[https://what-if.xkcd.com/67/](https://what-if.xkcd.com/67/)

------
basicplus2
more like.. what secret military mission was taking place during this time...

~~~
jrockway
This would have no effect on military uses. It was a static piece of data that
was misconfigured on one satellite, not the actual time code on any satellite.

Disrupting GPS is less useful for military goals these days, since there are
independent positioning constellations, like GLONASS.

