Haven't delved deeply into this, but does it let you do everything programatically, including track/catch pokeymon?
I'm blind and would be interested in there being an accessible version, but it's just not ever going to get made. Seems like pokestops or whatever they're called could be represented as a plain native list, with Maps integration for voice guidance to the point if you don't already know where to go.
And it looks like the in-game action is fairly simple. Could probably render it with positional audio, which has been done in accessible games for at least a decade [1]. I wear a bone conduction headset, so I could listen to the audio cues overlaying my actual environment. It may be hard to model the catch mechanics, but I assume they aren't too much more complex than "center one thing on another and throw," which again audio games have pretty much figured out.
Seems like a fun project. Might give it a shot if I can stomach the risk of getting randomly shut down one day.
Edit: Speeling
1. http://audiogames.net Kind of an annoyingly bad site in some respects, but useful info all the same.
I believe you can do pretty much everything, but sadly people are already using it for buildings bots, so there is a chance that Niantic might change it.
On the other hand, this does sound like a fun project.
So let me get this straight -- for these "unnofficial" APIs, someone just scraped a bunch of packets from their phone while letting Pokemon Go run on it? Then investigated to see what the communication from client to server looks like, then implemented an API that mimicks that communication?
If that's all so, could the PoGo devs simply enforce some type of device authentication to 'shut down' these APIs, or otherwise take different steps to make unofficial APIs less compatible/more difficult/effectively impossible?
> If that's all so, could the PoGo devs simply enforce some type of device authentication to 'shut down' these APIs, or otherwise take different steps to make unofficial APIs less compatible/more difficult/effectively impossible?
That's impossible. If you try this you'll either have a bunch of false positives or even more likely a patched api around 12 hours later. Never underestimate the dedication of botters. There are multiple headless WoW Apis around for 10 years now and Blizzard isn't able to close them out.
Agreed. Googling 'pokemon go lvl20' shows a bunch of accounts for sale already, if people are buying them the motivation to keep botting will be there.
The best way to handle this is with account-specific API keys. Even that would just mean creating an account, and the only real benefit to that is that you could track the API key, and if it did "bot-like" things, ban it. That's not really a fix, just a barrier for entry, preventing poorly thought out bots from working.
I'm pretty sure this already exists. These APIs have been in development for a couple of weeks now and as far as I can tell (from watching the /r/pokemongodev subreddit) there haven't been any bans. Niantic, at present, don't appear to have any automated system attempting to catch bot-like behavior.
> If that's all so, could the PoGo devs simply enforce some type of device authentication to 'shut down' these APIs, or otherwise take different steps to make unofficial APIs less compatible/more difficult/effectively impossible?
I sincerely hope that they don't do that, or at least not 'shut it down' completely. The reason is that there are some use cases for this API which don't involve cheating, for example the many different Pokemon Go server status pages including one that I built myself [1].
Majority of the reverse engineering started with packet inspection and reversing the APK's from the Android version. From there a lot of effort was put in to protobuf's and reversing the protobuf messages. Two things learned from the packet inspection was the authentication schemes for Pokemon Trainer Club and Google; and then how to use the JWT's in the API requests. All of the protobuf requests have been figured out, but sometimes it's about implementation details in how things are called (for example S2 cellid's for map object requests, etc).
There's one field in the app's request that's still unknown.
It's a header of seemingly encrypted data, along with a varying number of encrypted blocks (all the same length).
In those blocks could be anything, detailed gps co-ords, device details, there's a fair chance they can ban all these API users at the push of a button based on whatever's in those blocks.
Everything else is unencrypted - sent back and forth using the protobuf format, the formatting of the protobuf's were dropped on pastebin a few weeks ago.
Basically, yes, but as the other guy mentioned it's an ongoing process of fix/break.
It seems that the devs are firefighting scalability issues atm, and it would make sense that unsanctioned 3rd party APIs will be targeted Soon™ but probably heuristically.
They're also in violation of the ToS I believe, but bans in their previous title came in waves, and I've not heard of any pogo banwaves yet.
>They're also in violation of the ToS I believe //
Only if they signed up to it ;o)
If you're reverse engineering an app and someone else is using the device with the app installed there's no need for you to have submitted to a contract of obligations to the app provider. Depending how you orchestrate things the signee might be guilty of giving their credentials away. It would probably come in as unauthorised access of a computer system with respect to USA/England&Wales legislation.
I tried spoofing GPS with Niantic's Ingress game, just about the second time I played, and the spoofing apps relied on the fake location flag in the dev settings - of course Niantic checked to see if that was set and refuse to work if it was [until the system was rebooted, with the flag disabled, AFAICT]. Haven't tried with PoGo as I'm more interested in playing it [with my kids, honest!] than working out how it works.
One of the better ways to find these people using "unofficial" APIs is to look for VPN / hosting IPs. Running all of it on their home connection is very obvious so they'll branch out and let their bots run on servers 24x7. Based on the IP address and their account activity, I think it'll be a lot easier to catch botters or at least catch the big fish.
Aren't bots like this often run on botnets? After the classic DDoS of course, I'd have thought this sort of thing would be what botnets are used for these days.
I'd assume by looking at the usage patterns - has the device teleported instantly across huge areas, does it display any GPS drift and organic movement rather than a series of static locations, is it moving 24/7, are the reaction times inhumanly fast etc.
All that can be spoofed tho, and whether they have the resources to apply that level of analysis to XXX million users is questionable.
Or the motivation, I guess as botted users start claiming gyms it becomes a problem but until they implement trading each user is silo'd by the game mechanics
>I guess as botted users start claiming gyms it becomes a problem //
It might be a nice problem for Niantic if that means non-hacking users pay to improve their own pokemon to try and beat the botted gym users? Provided the numbers of bots is relatively low it probably won't be a problem they feel needs fixing. I guess if someone is claiming more than a few gyms it's going to get flagged and they can probably easily spot if it's genuine or not?!?
Gyms can't be held for very long, in any scenario other than remote destinations that nobody visits (like a reef 2 miles from the coast or a desert monument). 3 Perfect Pokemon of the highest tier will be burned down by 6 Pokemon of various quality (multiple people using 6 at a time), especially with the rock-paper-scissors counters available.
That's been my experience too, taking gyms is considerably easier than successfully holding them for any length of time.
That said, you only really need to hold it long enough to claim your coins, and the XP for fighting rival gyms is probably more valuable in the long run.
That's rife for abuse. It would require a "reporting" function; a moderation team; resolution practices; human intervention; a "scoring" system to judge repeat offenders... on and on and on.
Please don't ask for that. I don't need to see that the gym located at a church being championed by the Pokeman nicknamed "Gaylord."
The notorious Westboro Baptist Church was being held by an apparently LGBT friendly account called LOVEISLOVE recently, which I thought was quite sweet tbh.
The MapCell response actually returns "NearbyPokemon" and "WildPokemon" separately - where that node implementation is using the "WildPokemon" list. I mostly just think just using the "NearbyPokemon" which returns distance in meters only, not latlong, and displaying only the steps would be more in the spirit of the game is all.
The Fort/Gym data from the map request returns only the top cp pokemon in the gym. But the actual gym data returns the actual list of member pokemon, which might also include the nickname (POGOProtos definitions include it, but would need to make an actual request to confirm).
Probably good to regard these first few weeks (months?) of Pokémadness as an "open beta" period, before the security measures get turned on. We can look at Niantic's previous project, Ingress, for a roadmap.
The two major categories of cheatifying in Ingress are falsifying one's location and multi-accounting. There's precious little that can be done about the latter, so Niantic focus on banning players that appear to be "spoofing" their location.
Given the wealth of different devices and playing scenarios, immediate detection of GPS spoofing is infeasible. Things like WiFi router locationing idiocy (or even just dodgy GPS antennae) play havoc with the utopian dream of perfect positioning every time. If a player performs actions seconds apart that are separated by thousands of miles then the game temporarily ignores them, but after some time in the naughty corner they can resume play.
Hardy spoofing detection instead depends on longer-term profiling. Ingress has a similar API to Pokémon Go – JSON chunks (rather than protobuf) over HTTPS, most fields out in the open – but each request from the app includes a monolithic "clientBlob" containing device characterisation. The format of this has been (presumably) reverse-engineered by a few hardy souls but it is certainly closely-protected Niantic knowledge. We could safely assume that it's a proprietary blend of signal strengths, gyroscope readings, touch events and timings, secret herbs and spices etc.
The clientBlobs lend themselves to offline processing. There are conceivably servers continuously trawling through a backlog looking for tell-tale patterns of bad behaviour, but it also provides an audit trail if a particular player is suspected of spoofing. Occasionally Niantic indulges in mass purges, which presumably follow from a new cheat detection heuristic being run on all the collected data for some period. These "ban waves" have a reputation for penalising unusual device configurations (the most recent major wave appeared to target, amongst other things, players with modified Android variants that might mask GPS falsifying code, including cheaper Chinese knock-offs, and Jolla phones running Sailboat).
Occasionally during major Ingress gaming events – so called "XM anomalies" – there is some level of human supervision to quickly identify and remedy clearly-fraudulent player behaviour, but for day-to-day operations it seems that account termination, so-called "hard bans" and shorter-lived "soft bans" are entirely automated, and based on offline player data analysis.
Getting back to the New Cruelty: the clientBlob was not part of Ingress's initial implementation; for a while after it was introduced was ignored, and then it became mandatory. A similar opaque chunk of data is included in the Pokémon Go requests, so we should look forward to its imminent deployment when Niantic scrape together enough Pokécoins to buy a few new servers for batch processing. At that time these convenient APIs won't have long to live.
>If a player performs actions seconds apart that are separated by thousands of miles then the game temporarily ignores them, but after some time in the naughty corner they can resume play. //
I'm curious how the financial side works with the gameplay side - the people doing spoofing might also be those that are motivated enough to spend money on the game; you don't want to ban your whales [best spenders] just because they tried to cheat. Would be really interested to see how much of that weighs in to business decisions on crack-downs on unauthorised "play".
I'd certainly agree that someone who went to the effort of setting up a system for spoofing (even if it was just downloading an extra app) is, in some sense, more motivated than a very casual player.
I don't think though that Niantic have much of a moral hazard to consider here. Looking at what's purchasable in the Pokémon store, there's nothing that would be attractive to anyone who was able to virtually wander the world at all hours from the comfort of their couch, especially since anything that can be bought with cash money could be obtained using coins earned in-game. If a player's motivation in spoofing was to "catch 'em all" by whatever means necessary, it seems unlikely that they'd draw the line at restocking from Pokéstops along the way.
Comparing with the dark side of Ingress, there is a ludicrously well-organised black market economy offering purchases for every in-game commodity – all, of course, completely against the T&Cs, all completely abhorred by legitimate players, but all offered with consummate professionalism (think of the slick ransomware scammers offering a support number). Niantic don't see any of that cash. It is likely to have had a major impact on their design decisions for the PoGo store, and the game in general. If, for instance, there is no way to trade items between players, then it severely limits the options for a parallel economy.
I am just happy to see that the API has "trading" as a concept, looking forward to that feature.
Overall it's sad that most game mechanics of the original games didn't make it into Pokemon Go. Does anyone know how much time they had to implement it?
My understanding is that the beta for the game got out to a larger audience than they intended and they pushed up the release date, though I can't immediately find anything on Google to corroborate.
Most of the important stuff is certainly done server-side, like determining whether a Pokemon appears at coordinates x,y or not (though my guess is the client is tasked with this, and then says "Hey server, I'm showing there should be a Pidgey at x,y, I'm gonna try to catch it." The server confirms by running the exact same deterministic check on x,y, then says "Yep, I see it too," or "No, you're a filthy liar, Joey.")
But for the actual "I caught it" or "I missed it," unless the exact user input is sent to the server, and then the server simulates the projected path of the throw, the client seems like it gets to say that. So perhaps the client could actually say "I hit it perfectly," and the server just says "Well if you say so."
I've seen the app load something (and hang up while doing that) between pokeball throws. That seems like evidence that it's asking the server about something after each throw / once every n throws.
Checking if a throw resulted in a capture server-side would be weird, I don't see why it couldn't be done on the client. It could be querying for the players inventory, to see if they have enough pokeballs? It'd still be better to just sync that once before the fight...
My best guess is, the conditions for a Pokemon running away involve something only the server knows. A weak example: maybe it considers other players in the area - pokemon might be more likely to run away if the area is too crowded? So every once in a while the client asks the server how many players are there nearby, or something.
I've noticed higher CP pokemon tend to run away more quickly. You also have to remember that there's now great balls and razz berries to influence throw chances, so the client probably publishes the events it's doing (gave a razz berry, threw a pokeball, threw a greatball) and the server creates some randomness that says yes/no.
That "randomness generating" still seems like it would be easily done locally, and it'd reduce server load. But maybe there's some more advanced mechanisms we don't know of.
if it's done locally then people will bodge the client so the random is no longer random. The server's always going to have to do the math, even if to just ensure the client's not lying.
>Checking if a throw resulted in a capture server-side would be weird, I don't see why it couldn't be done on the client. //
It locks up [server access icon spins] a lot on the point at which the ball closes and the game is deciding whether to capture or release the pokemon (what's the heuristic there?), often crashes at that point to.
I wouldn't be surprised if this is because it's sending the whole of the swipe data to the server. Surely it doesn't need to check if there are pokeballs available, when your client says "gotcha" then the server will have record of if there were balls available or not.
To answer your first question, yes you can configure everything; type of ball used, whether the throw is curved or not and the "excellence" of the throw. No one gets excellents throws all the time so I think it's easily detectable and hence bannable.
If they are profiting off their work to the extent that it actually makes business sense for the game company to sue them for damages, then that is the action they can expect to get.
This just goes to show that Niantic has no idea what they're doing and completely lucked into the popularity their app has had.
It's terribly buggy and clearly totally insecure. They can't keep up with the server load and nobody had a conversation before the game shipped about protecting against abuse from people reverse engineering their APIs. This is a joke.
Alternatively: the game's success (and profits) show that none of that matters so they were wise not to waste time on it. There's no way to perfectly harden an app like this, anyway.
I think the game's success speaks to the popularity of the franchise and the desire for fans to have this sort of game. I agree, they must have known it was so buggy before launching and went ahead anyway... because profits.
There should be some minimum level of quality regardless of the fact that if you shove crap through the door people will buy it.
that's true on so many levels. even the gameplay is bonkers, both because it is way removed from the actual pokemon lore and themes but also more importantly because level caps and no mid quest but just gyms block every casual player from the only player interaction available
And React news? Python news? Maybe we can just have HN tags :)
I could be reading too much in your comment but I get the feeling you're upset at the overwhelming amount of pokemon-related threads. Don't be.
Reverse engineering is a very interesting field of programming. Video game reverse engineering is one of the most amazing things, actually. It's where many teenagers discover how the games they play work. Where they first start understanding technology, write their first scripts, start being in control of their computer.
I haven't touched Pokemon Go yet, and am not overly interested (mostly because it's not available where I live). But the reverse engineering effort behind it is spectacular and passionating. Why dismiss it like this?
Is it? I've seldom seen it pointed out. Do people keep telling you that?
I respect your comments a lot usually, but you're being incredibly arrogant here. You are taking one subject which people quite clearly enjoy, and dismissing reverse engineering work that goes into it just because you don't like it.
If this is you standing up for what you wrote, I really do hope you didn't read my comment.
> I respect your comments a lot usually, but you're being incredibly arrogant here.
That's another one of those oft repeated patterns. Usually it means "I like you when we agree, but now that we disagree I don't like you any more".
> You are taking one subject which people quite clearly enjoy, and dismissing reverse engineering work that goes into it just because you don't like it.
No, I simply don't like dumb games that cause adults to run around like headless chickens whilst ignoring the world around them and then to see my favorite tech news site flooded with one link after another. And yes, it upsets me because it crowds out other submissions not tied into the hype. Then people start making HN specials in order to ride along on the hype.
> If this is you standing up for what you wrote, I really do hope you didn't read my comment.
I'm not asking if you read the article, I'm asking if you read my comment. But I guess I have my answer now.
> Usually it means ...
Ok, now you're just doing it on purpose. What it means is I respect you, and I'm surprised you're behaving like a run of the mill HN troll.
> I simply don't like dumb games
You're still conflating the game and the work that goes behind the game. I don't like Facebook, but I don't shit on articles about React. I think Twitter is one of the dumbest thing on the web right now, but I still read articles about Twitter's engineering, because it's interesting as fuck.
You're going on your 10th year on HN. How have you not caught on to this pattern?
Note how each of your last three comments contains some kind of personal insult. Is it really that hard to disagree without resorting to personal attacks?
You know, Jacques, at one point a few years ago I remember thinking your articles were on the front page too much. Normally I don't pay attention to things like that, but folks who operate their Wordpress as "Administrator" get my attention, and I landed on your blog a lot from the front page. It's interesting that you're the one making this comment for what you want Hacker News to be, because I was pretty tired of Jacques Matthiej's Link Aggregator for a while. (That seems to have run its course, now that I think about it.)
I didn't comment about this at the time because literally zero value can come from such a thing, though. I think it's just worth keeping the perspective that a lot of people vote up stories that interest them, and your interests are not the end-all for what is welcome or appreciated here. I get absolutely punished occasionally with voting and that reinforces to me that my opinions often diverge from what everybody here wants, so the last thing I'd expect to be is wise on HN.
I don't like SV culture at all, for example. The unabashed sexism and racism, the funneling of money to pointless things that make human beings waste their finite existence building and using them, non-practicing thought leader entities, all of it. This forum can be the epicenter of that at times, along with the other negative traits like a constant desire to prove people wrong. I complain about it while drinking beers and driving my friends away from the bar and carry a passive aggressive attitude a lot when I comment here, but I don't assume based on tenure that I know best for Hacker News nor assert it. And in the rare case that I do, I just email the moderators and annoy them (ask Dan).
Given my reputation here I know this comment is a bit out of character. However, take it from someone who hyperventilates a lot at some of the inane shit here: it's better just to link to your friends and collectively laugh rather than hop on a soapbox to lecture everybody. When Paul implemented non-optional graying out of comments based on voting, he was making clear that outlier opinions — which lecturing a majority are, by definition — are not welcome on Hacker News and will be silenced. I've repeatedly mentioned this and Dan has kept it, so it is what HN wishes; that you can showdead but not showgray is also telling. Just keep in mind that being the lone voice against the tide of something you don't like is unwelcome, will be squelched to invisibility by voting, and is a waste of that finite existence again. Just do what I do: internalize it and be bitter so you can look forward to becoming a cranky, cantankerous old shit in your twilight.
You could use a mai tai. (I could, too, for typing this comment, though it's 10AM here and late enough for you to get away with it.)
> You know, Jacques, at one point a few years ago I remember thinking your articles were on the front page too much.
I actually agree. It got to the point that I asked people not to submit my stuff because there definitely is stuff on my blog that does not overlap with HN in any way. It's the stupid karma, people will race to submit stuff that they believe will give them a karma boost, as if there is any value to it.
> it's better just to link to your friends and collectively laugh rather than hop on a soapbox to lecture everybody.
That I'll agree with. It's just that I spend a lot of time on the 'new' page and the amount of pokemon stuff submitted is insane, and that's besides the amount of stuff that is 'made-for-hn' with some pokemon reference in it just to get some of the attention.
> Just do what I do: internalize it and be bitter so you can look forward to becoming a cranky, cantankerous old shit in your twilight.
I'm blind and would be interested in there being an accessible version, but it's just not ever going to get made. Seems like pokestops or whatever they're called could be represented as a plain native list, with Maps integration for voice guidance to the point if you don't already know where to go.
And it looks like the in-game action is fairly simple. Could probably render it with positional audio, which has been done in accessible games for at least a decade [1]. I wear a bone conduction headset, so I could listen to the audio cues overlaying my actual environment. It may be hard to model the catch mechanics, but I assume they aren't too much more complex than "center one thing on another and throw," which again audio games have pretty much figured out.
Seems like a fun project. Might give it a shot if I can stomach the risk of getting randomly shut down one day.
Edit: Speeling
1. http://audiogames.net Kind of an annoyingly bad site in some respects, but useful info all the same.