Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
The China GPS shift problem (wikipedia.org)
205 points by ivank on Jan 24, 2016 | hide | past | favorite | 71 comments


What you really get from GPS is a vector in earth-centered, earth fixed (ECEF) coordinates. The origin is the rotational center of the earth, the XY plane is the equator, and the XZ plane goes through the prime meridian at Greenwich. You run this through a simple standard formula which models the Earth's slightly elliptical shape, and get out latitude, longitude, and elevation relative to a defined sea level.

ECEF is based on physical reality. The choice of prime meridian is arbitrary, but everything else has a physical basis. GPS, GLONASS, and Galileo all generate (almost) the same ECEF coordinates. (GLONASS has a position for the center of the earth about 3mm from GPS. Galileo is also about 3mm different.) This reflects when measurements were made.

Latitude and longitude are computed by putting ECEF coordinates into a geoid model. All such formulas are approximations. WGS-84 is used by most of the world. GLONASS uses PZ-90, which is slightly different from WGS-84 and fits the earth's profile better in Russia. The Chinese "encrypted" geoid is WGS-84 with some junk offsets added for coordinates within China's area of interest.[1] There's an actual "out of China" test:

    outofChina <- function(lat, lon){
      if(lon < 72.004 | lon > 137.8347) return(TRUE)
      if(lat < 0.8293 | lat > 55.8271) return(TRUE)
      return(FALSE)
    }
This is a rather expansive view of China; this goes almost to the equator, including all of the South China Sea and the Spratly Islands.

[1] https://github.com/caijun/geoChina/blob/master/R/cst.R


There's some interesting complications to the idea of an underlying Cartesian ECEF coordinate system as physical reality. For one thing, gravitational distortion (Earth tide) changes the shape of the solid part of the Earth by ~50cm twice a day. This would be a huge unwanted effect when doing high precision surveys, but it's removed when producing coordinates in the international terrestrial reference frame. If you have a nearby reference station which shares the common periodic motion this is relatively easy. If not, the software needs to be aware of the periodic motion and remove it explicitly.

I'm constantly amazed that the whole system can be made to work at such high precision - sub 10cm from a moving aerial survey platform with decent gear. There's rather a lot of correction factors to get just right when pinning a consistent coordinate system on a squishy rotating blob like the Earth.


The "encryption" looks like it's just a polynomial function, which would certainly be very easy to "reverse engineer" using the usual mathematical regression techniques. The Baidu one which is also mentioned in your link looks even simpler.


This part of the article explains more:

https://en.wikipedia.org/wiki/Restrictions_on_geographic_dat...

"GCJ-02 (aka Mars Coordinates) is a geodetic datum formulated by the Chinese State Bureau of Surveying and Mapping, and based on WGS-84.[12] It uses an encryption algorithm[13] which adds apparently random offsets to both the latitude and longitude, with the alleged goal of improving national security.[14][15]"

"Despite the secrecy surrounding the GCJ-02 encryption, several open-source projects exist that provide conversions between GCJ-02 and WGS-84, for languages including C#,[20] C, Go, Java, JavaScript, PHP,[21] Python,[22] R,[14] and Ruby.[23][24] They appear to be based on leaked code.[25]"


Interesting that there are actually no encryption at all, just simple math equations.


> just simple math equations

To be fair, you could say the exact same thing about any modern encryption scheme.


"Simple" is... debatable. There's a good reason rolling your own is a bad idea.


The 'how' of encryption is often quite simple - it is 'why' that is difficult


Or the other way around?

Example: We all know why we should protect against timing attacks. How to get it right is the harder part.


No.

Encryption is designed to be irreversible without a specific piece of extra data.

That isn't the same as not knowing the formula or initialization seeds used. Those are both reversible.


I'm going to assume from the context is what they actually did was use a crypto-grade RNG to introduce bumps into the geoid surface. That is, instead of everything being 2 meters to the west (or whatever) things were randomly offset from their reported positions.

You could "map" the whole thing or somehow steal the source and crypto parameters in order to transform between geoids, but you couldn't do something simple like subtraction.


However it was obviously their goal not to have completely random oordinates but just "less precise."


Not random ordinates, unpredictable offsets, apparently on the order of a kilometer.

And it wasn't precision they were after at all. You could just make a global positioning system arbitrarily less precise but that isn't useful. Kilometer precision doesn't have much use to civilians, they need meter-precision at least and repeatable at that. What they were after was unpredictable but static accuracy offsets which presumably could be altered in a crisis.


At work we've standardised on Baidu coordinates in China.

Even though they're "encrypted", the encryption is locally smooth.

Points may vary by 2km from their wgs84 equivalent, but taking two points 2km away, using the haversine formula will still yield a distance ~2km.

This means we can treat wgs84 and baidu-coordinates as equivalent, but not comparable, which makes a lot of things simpler.

Note: we don't do cross border orders, so we don't need to worry about cross border comparison.

Interestingly, Baidu uses it's own additional encryption on top of GCJ-02, so we are very much locked in

https://en.m.wikipedia.org/wiki/Baidu_Maps#Coordinate_system


Why do people call this encryption when in reality obfuscation is a much better word for it?


I'm also wary of calling it encryption. But I don't think "obfuscation" or "hashing" are correct either.

The intent is the same as encryption - to keep the real location a secret, available only to people with the key.

It feels different because the encrypted value is basically an offset, and it's continuous, so it feels almost the same as the unencrypted form.


hashing?


It's not a hash since it's reversible.


China also makes it illegal to make maps of the country without authorisation. Don't get caught working on OpenStreetMap there.


And it's curious to see this in Apple maps. When I travelled to China, I generally found Apple maps to be superior to Google maps, and it works without a VPN. But as soon as I left China, the Apple map data for much of the Chinese regions simply became garbage. Even major cities are shown as empty areas without features.

EDIT: grammar


Apple gets map data for users in China from contractors who have been approved to run GIS businesses, e.g. AutoNavi Software Co., Ltd. BTW, they do not seem to provide world maps so you can only browse and search locations in China when you are in China.


They make the maps doubly illegal if Taiwan is shown with different colors indicating an independent country ;) never mind the Spratleys, Senkaku, etc.


Is this really true? Like all maps you buy in China show Taiwan as part of PRC?


Yes, and China is in the middle of the map.


Oh c'mon, I'm still not sure about parent, but you are obviously trolling.

You need to develop some subtlety to be taken seriously.


Actually, I am completely serious. This is the map on display in Chinese schools: http://www.miniharm.com/wp-content/uploads/2014/08/worldmap....


The map centering is quite okay. After all they'd want to highlight the home country, no big deal.

The troublesome part is coloring land not theirs as theirs in the map as well as labeling India's "hat" as disputed. There ate disputed lands, but not as large as implied.


Crap cultural shock. At least they didn't renumber meridians :)

And it seems that Taiwan is indeed same color as PRC. They really are crazy.


That is the official view of China.


Sure, and the official view of Taiwan is (was?) that PRC are rebels within their territory.

However, it's one thing to have some "official view" and another thing to brainwash citizens into thinking that these countries really are one, as they practically aren't. I was asking about the latter.

For example, some EU regulation classifies snails as fish to make their production eligible for fishing subsidies, but in biology textbooks they still remained molluscs.


Yes. In addition, all Republic of China maps you buy in Taiwan show PRC as part of ROC. lol


Every inch of the earth has been mapped out by satellite and is available with perfect GPS navigation. Too late for that law.


To support you, the USSR had mapped nearly the entire world (presumably with aid of satellites and GPS):

"The Soviets made far more detailed maps of some parts of the world. They mapped all of Europe, nearly all of Asia, as well as large parts of North America and northern Africa at 1:100,000 and 1:50,000 scales, which show even more features and fine-grained topography."

I sincerely doubt they have any of the China GPS offsets.

http://www.wired.com/2015/07/secret-cold-war-maps/


Sounds like their civilian maps did, though:

> Worse, the maps for the masses were deliberately distorted with a special projection that introduced random variations. “The main goal was to crush the contents of maps so it would be impossible to recreate the real geography of a place from the map,” Postnikov tells me. Well-known landmarks like rivers and towns were depicted, but the coordinates, directions, and distances were all off, making them useless for navigation or military planning, should they fall into enemy hands. The cartographer who devised this devious scheme was awarded the State Prize by Stalin.


Yes those Soviet era maps were military and classified.

They only ended up in popular circulation after a quick retreat, fuckup and corruption that happened when the Soviet forced retreated from the Baltic states.

I've heard even US militaryused(uses?) some of those maps for parts of Africa or used then initially in Afghanistan. As nobody else has bothered much to go and cross reference and built detailed maps of those regions in that much detail.


My history teacher claimed she had seen one of these maps and accuracy was unbelievable, there were every minor remote woods path marked only known and used by locals.


Hadn't read that Wired article before - thanks for sharing. Wow! The Soviet maps are beautiful and accurate (not something you can say about many other Soviet legacies), a real historical treasure trove.


Yes! I have one of those maps for my Spanish city, with all the names of features, neighborhoods, etc. in Cyrillic. It looks totally extracted from a dystopian parallel universe!


Satellites have made images, but they haven't made proper street maps, which is what you actually need for navigation. Blurry textures on a globe won't tell you how to get to places unless you already know their GPS coördinates.


TIL, diaeresis is not umlaut.


Huh.

https://en.wikipedia.org/wiki/Diaeresis_%28diacritic%29#Engl...

> The grave accent and the diaeresis are the only diacritics native to Modern English... The use of both, however, is considered to be largely archaic. The diaeresis mark is sometimes used in English personal first and last names to indicate that two adjacent vowels should be pronounced separately, rather than as a diphthong. Examples include the given names Chloë and Zoë... it is far less commonly used in words such as coöperate and reënter except in a very few publications—notably The New Yorker.

https://en.wikipedia.org/wiki/Grave_accent#English

> The grave accent, although not commonly applied to any English words, is sometimes used in poetry and song lyrics to indicate that a vowel usually silent is to be pronounced, in order to fit the rhythm or meter. Most often, it is applied to a word ending with -ed. For instance, the word looked is usually pronounced /ˈlʊkt/ as a single syllable, with the e silent; when written as lookèd, the e is pronounced: /ˈlʊk.ɨd/ look-ed). It can also be used in this capacity to distinguish certain pairs of identically spelled words like the past tense of learn, learned /ˈlɜrnd/, from the adjective learnèd /ˈlɜrn.ɨd/ (for example, "a very learnèd man").


It might be archaic usage, but I agree with The New Yorker that coöperative is much more elegant than co-operative or cooperative.


Aya. Do you think "they" would send the tanks or what?

Many things are forbidden by law, and many people do them anyway, because they know it is not /that/ forbidden. And not only in China, it is the same in France for example.

So, if you are working on OpenStreetMap in China: if your product is successful, be prepared to have to "drink tea" one day with authorities. They may tell you to use another map, or to comply with some rules, or to run your service from outside, etc.


As a foreigner you better follow he law, because the government will make an example out of you. Only Chinese companies can drink tea with chinese officials.


`To drink tea` as a slang in Chinese means to be asked to talk with authority, especially the police, etc., which is generally not regarded as a good thing.

Similarly, `to check the water meter` and `to receive a post package` mean the police to come to your door, as the police are thought to be disguised as staff from tap water company/package delivery company, in both cases, you are about to open the door.


> Many things are forbidden by law, and many people do them anyway, because they know it is not /that/ forbidden. And not only in China, it is the same in France for example.

China is not a place I'd like to test those boundaries.


Just wondering, how does this impact national security? When most of the planet is well-covered by Satellite imagery, isn't finding an installation with 500m difference a matter of searching images?


It's just an excuse to force foreign companies to license the algorithm. There's 0 chance foreign nations don't have a copy of the encryption algorithm.


I mean, there's open source implementations linked in the same Wikipedia article: https://on4wp7.codeplex.com/SourceControl/changeset/view/214...


Which, as mentioned, are technically created via illegal means. And incorporating them would into a commercial product would likely see your product banned from sale in China.


I doubt it's easily enforceable though. This is China, where everything goes (as long as you know the right people and/or have money.)


Quite a few very influential people thought so, as well. They are being executed on disciplinary charges (ie. corruption).


... and there's no chance the space-faring superpowers don't already have their own map data of China, in their own coordinate schemas, with their own annotations for military purposes.

After all, launching all those mapping satellites was expensive. They did it for some reason. ;)


Authoritarians like control just for the sake of control.


One the one hand not every rival nation is a spacefaring superpower. On the other, controlling the information available to it's citizens is at least as high a priority for the Chinese authorities as denying it to foreign nations.


> Just wondering, how does this impact national security?

The goal is to prevent uprisings or riots from having accurate data, not to prevent the US, for example, from having accurate data.

This is about citizenry control for internal threats, not external ones.


I don't think the 'National Security' aspect would result in Uncle Sam's finest B52's using Chinese 'GPS' to then end up dropping their bombs 500m off course in every instance.

That would make a good story though, a full on first strike that missed every target '500m to the East' due to some mapping oversight.

However, as for real use 'National Security', you could hide some area by making it not exist with the surrounding area stretched a bit on the map to hide the secret area. With some natural features, or even a big forest, all kinds of things could be hidden from the general public (who are important and primary 'targets' for 'National Security') if not Uncle Sam.


"That would make a good story though, a full on first strike that missed every target '500m to the East' due to some mapping oversight."

Something similar has happened at least once(0)..

0. https://en.wikipedia.org/wiki/United_States_bombing_of_the_C...


Also reminds me of the less map-related targeting deceptions of WWII

https://en.wikipedia.org/wiki/V-1_flying_bomb#Deception


Wrt hiding an area, that should be labeled the "Lake Woebegone" effect.


I have run into this problem when I was living in China and wanted to map hiking trip with my friends in HangZhou.

Wrote about it on my old blog in 2013: http://michalkow.tumblr.com/post/43214865761/compensate-for-...


I was just in China over the border from Hong Kong and Google maps felt to just be offset to the west and a bit north by about 500m. Once you mentally adjusted to that, the maps still felt very accurate. Check out the border and look how the roads are all cut off at the border, very weird


Oddly, when I checked during the last few days Google Maps suddenly got a lot better, including 3D buildings and accuracy to about 50m for GPS location. Maybe something changed recently? There have been rumours about Google re-engaging in China (to fix their Android woes)


So it wasn't after all, with more testing... I must have been lucky and hit a spot where the obfuscation distance was small.


In Shenzhen? Yeah everything is off by just enough to make google maps no good - it shows several streets in the PRC side accidentally go across the river to HK. I spent several months there and Bing maps was the winner.


I was in China and Google Maps was blocked. I hope you weren't being naughty and bypassing their firewall.


Nope, I had a sim from hong kong (which didn't work literally 200 feet past the border) but had all the maps cached. The GPS obviously still works offline, so it was just running in offline mode.


You can get cross-border sims now which let you use the HK network (going around the GFW) even on the mainland.


I experienced this myself and it was completely confusing!

Back in 2008 before I had a smartphone with GPS, I had a camera and a separate GPS device I would use to "tag" places where I took interesting pictures during a 6-month trip around the world.

When I got back home I ran some code to match picture timestamps with GPS timestamps, correcting for time zone offsets (GPS was GMT, pictures were local). All of the pictures of the rest of the countries matched just fine, except for China :) Since I had relatively few data points, it took me a while to realise there was a systematic error in the data, not something I was doing wrong with the timestamps.


Now I understand why when I was in South East Asia there was sometimes a shift (about 10m), I thought it was a signal reception problem. I was using Symbian Maps (now Nokia Here). So the correction was not perfect.


we use baidu map.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: