
Google refunded $200 because I missed 5 lines of Code - aswinmohanme
https://aswinmohanme.github.io/why-google-refunded-200-because-i-missed-5-lines-of-code.html
======
benjaminwootton
I once worked on a platform where you could buy ringtones, wallpapers, jokes
etc by SMS pre App Store and play store.

Every night the platform would kick off a process to claim money from the
telco using some key which was tied to an SMS the user had sent.

That process was responsible for more than $1 million a month, but it was
buggy as hell.

Half of the billing attempts would fail, the retry logic was buggy, processes
would crash so you would just start them again without proper semantics to
ensure you don’t bill people twice.

After a while there were so many retries outstanding that you couldn’t get
through them overnight, so we would try in random orders and we would have the
process running from 3 days ago whilst try close off today’s bills.

I’m sure people were billed incorrectly and a lot of money was left on the
table from users who should have been billed. We never really heard a lot of
noise about this either.

So I’m sympathetic to this process and glad that Apple and Google have
improved the confidence around mobile commerce.

~~~
tomglynch
That's very interesting. I'm sure there are a lot of stories like this from
back in the day - charging people has never been easier than it is now.

~~~
LeonM
Not just back in the day, I have at least a couple of customers of which I
know they are losing money because of billing issues.

Why? Because most accounting software is horrible and have minimal to no third
party integration options. Many of them don't have an API, so the only option
you're left with is manually (!) importing some XML structure every day. The
logic is usually very limited so stuff like record mutations are often
silently ignored during the import.

The accounting software world is evil. Deliberately engineered vendor lock-in,
very limited third party integrations, ridiculous charges for 'custom'
implementations, etc. I once build a migration tool for a well known
accounting/ERP suite, and my partnership with them was immediately terminated
due to 'breach of terms'.

~~~
benologist
Google themselves stole $75 million from Adwords customers by not coding up a
particular kind of refund and ignoring it for decades. It is likely a similar
thing is happening with banned account balances.

[https://www.businessinsider.com/google-emails-adtrader-
lawsu...](https://www.businessinsider.com/google-emails-adtrader-lawsuit-
refund-ad-click-fraud-2019-5)

[https://www.searchenginejournal.com/adsense-
lawsuit/248135/](https://www.searchenginejournal.com/adsense-lawsuit/248135/)

~~~
ksdale
That’s interesting! I’ve often remarked that it’s a giant competitive
advantage for huge companies like Comcast to be able to net however many
millions of dollars from billing “mistakes” that never get contested. On the
one hand, I’m willing to concede they are mostly probably honestly mistakes
and they only make up some tiny percentage of all their transactions, on the
other hand, it probably adds up to a decent sized company’s total revenue
every year that they basically get just because they’re so giant they can
screw up a ton of transactions and plausibly claim it’s not fraud.

------
iliketosleep
Near the end of the article: _These kinds of incidents make us wonder the real
cost of Code and the more real cost of messing up_

Imagine having an app about there ticking away and making pretty decent
revenue. Little do you know there's a subtle error in business critical code
(e.g. in-app purchases, ads, etc), reducing the revenue by over 50%. This is
something I've seen first-hand and have become quite paranoid about. I suspect
that it happens a lot. The reality is, no matter how many times one looks at
their own code, there is no substitute for continuous testing of typical use
cases in a production environment, ensuring that business critical code
functions as expected. It's a continious process, because even if the code
works properly at the beginng and isn't modified, the underlying OS or API's
it interacts with can change and introduce unexpected behaviour.

~~~
mcv
I've worked for a company that went bankrupt over some accounting system
they'd built and used themselves. It contained a subtle bug that made them
think they had more money than they had.

It's really vital to thoroughly test and verify any financial code. Any code,
but financial doubly so. And do it again in production. Even small errors can
really add up over time.

~~~
raverbashing
Yes, testing is important but also sanity checking and in-flight checks as
well

Why are you getting declined transactions? Why are you getting chargebacks?
Are we getting one transaction per sale?

------
deanCommie
> In order for Google Play to ensure a consistent purchase experience
> regardless of where the user purchases your product, you must acknowledge
> all purchases received through the Google Play Billing Library as soon as
> possible after granting entitlement to the user.

Can someone who understands this domain better explain why this exists?

A superficial reading of the text triggers thoughts of "fraud prevention", but
the code the author uses to fix it is clearly one a fraudulent app can also
easily include.

Other thoughts about preventing poor user experiences such as dark patterns to
trigger accidental purchases, or bait-and-switch functionality with Fremium
apps also don't seem to be prevented by this.

So what's going on?

~~~
tjohns
Play Billing 2.0 added the ability to purchase in-app items from outside of
the application.

This ACK confirms that the application properly processed the notification and
delivered the item to the user. Otherwise, Play assumes that the purchase was
lost in the ether sometime before fulfillment (e.g. bug, the app crashed,
etc.).

~~~
leoc
I assume that part of it is a psychologically-motivated desire to maintain a
tight reward loop for Play purchasers, with no payment-processing delays.

~~~
hoffs
How do you get that out of previous comments

~~~
orbifold
Otherwise the item could only be available once purchase was confirmed.

~~~
izacus
How could it be available if the application didn't ACK that it received the
purchase order and granted the item?

~~~
SenorSourdough
From the post, it sounds like the users receive the purchased item immediately
but the app developer needs to confirm that the charge went through properly.
Otherwise the funds are returned but the user keeps the purchased item.

 _Without acknowledging the purchase the money was refunded back to the people
who purchased it. And the best thing was the dark theme would be activated
even if the user got the money refunded._

------
inlined
This seems like something that static analysis can correct before the new app
is even allowed in the App Store. If you accept a purchase but have no calls
the the acknowledgement APIs then bounce with a message. Hell, it could even
be a compiler warning. This seems squarely Google’s issue and the refund
should be returned.

~~~
ehsankia
Static analysis aside, as soon as it happens even once, you'd think that there
would be a big red warning on your dashboard saying "X purchases were refunded
because they weren't acknowledged". The fact that the author had to figure it
out through user emails and negative growth is strange.

~~~
shearskill
Mobile games get frequent refund requests and devs are more than a few steps
removed from the actual accounting from app stores. The analytics might throw
an event if there’s a hook for a refund request but Apple and Google own the
bank in a black box. Basically devs get a monthly statement with refunds
subtracted from their cut.

~~~
ehsankia
There's a huge difference between the user explicitely requesting a refund,
and Google automatically ordering one because the purchase wasn't
acknowledged.

I'm not familiar with the API, and I could see both of them looking the same
on the dashboard, but Google themselves should be able to distinguish between
the two, and one of the two cases is very likely a bug in the app code. I
could not see a case where not acknowledging the purchase would be intended.

------
wiradikusuma
I was surprised when I read that he moved back from Fluter to Kotlin, down the
line I read it's due to APK size.

I'm happy to hear some developers still care about size esp. for mobile, but
on the other end, I'm sad because I'm currently using Flutter for building
apps and I'm jealous of that 4.5MB difference :D

~~~
aswinmohanme
I love everything about flutter except the size, and I would have stayed on
flutter if it wasn't for the competition's app weighing around 1.6 MB.

~~~
monsieurbanana
Does the size of an app affects something besides the initial download?

Maybe it also lengthens the app loading but everybody here doesn't talk about
it because it's well known of android developers, which I'm not?

~~~
anoncake
Yes, bigger apps take up more space in storage.

~~~
winter_squirrel
I feel like most phones nowadays don't really care about a 5mb difference
considering they have multiple GB's of space. At least on my phone, the space
taken by the actual app binaries is negligible compared to media.

~~~
anoncake
One app being 5MB larger is negligible. 50 apps being 5MB larger isn't a huge
problem but it is significant. Modern phones have gigabytes of storage but not
many gigabytes.

I also didn't see that the difference is just 5MB. Most apps are in the double
or even triple digits.

In addition, caring about anything being 3 times larger than necessary stikes
me as the right mindset. Individual items may insignificant but they add up.
It's certainly better than the electron-ic opposite.

------
mauflows
Shoutout to the people who cared enough to want to pay for it

~~~
teh_klev
I kinda thought that myself.

------
mrb
Wait a minute. 5 lines of code were missing in the app, causing users to be
refunded. So does this mean a malicious user could decompile the app, remove
this code, recompile, and get these auto refunds? This seems a way easier
mechanism to get free in-app purchases rather than reverse-engineering the
rest of the code.

~~~
abedja
Your recompiled version would not have the proper cryptographic signature as
you don't have the keystore used by the dev on your pc

~~~
numerlo
The version without those 5 lines will now be forever out there though.
Malicious peers can download the apk and get the premium version for free.
This feels like a poorly thought out design on Google's part.

~~~
kabwj
Obviously if you decompile the application and modify it you can get the
premium features for free. What’s your point?

~~~
beering
numerlo is saying that users don't even need to decompile the app - they can
continue sharing the buggy APK with each other to get free stuff.

------
whitecrow90
did the same implementation one month ago (and also missed the acknowledgement
step :/) and we are 1 month before release, first thing on Monday morning to
add and test it. Thanks dude cheers :D

------
gibolt
Seems like this shouldn't be the default. As accepting and then refunding a
payment is so uncommon, it should require extra code for that case instead.

~~~
ShinTakuya
The issue is that if you have a bug where the feature isn't activated even if
they pay, the user would lose money. It's natural that as you activate the
feature you should acknowledge the purchase. Otherwise you could have bugs
like double charging and such.

It was an unfortunate mistake by the author, but it makes perfect sense why it
happened. The person who made the mistake should lose out.

------
mesaframe
Little off topic. How was your experience with Flutter? How does it compare
with other two?

------
fjsolwmv
Building a successful business requires taking responsibility for all the
boring details. Complaining that it is someone else's fault wastes precious
time and hurts morale.

------
ddffre
New payment API, it was something they have mentioned.

A lesson for next time always read blog posts about new APIs you are planning
on implementing, especially when it comes to payments.

------
PaulBGD_
Just installed your launcher, exactly what I was looking for :)

One change I'd suggest, dragging to reorder custom apps.

------
2pointsomone
Also off-topic, but why move away from React Native?

------
rognjen
Alternative title: I missed 5 lines of code and lost $200 (and it's all my
fault)

~~~
fencepost
I'd have just added "of sales" or "of my sales" after the dollar amount to
make it more clear.

I actually looked at this because I thought Google had refunded money to the
author and I wanted to see what had actually inspired an act of customer
service from a company known for not having it.

------
anonu
Seems like a PEBCAK

