
Google’s painful Gmail OAuth verification process - kitx
https://aura.app/blog/google-oauth-verification/
======
privateSFacct
As a gmail user - good to hear this. In the long run trust is going to be a
much more important commodity that letting a spam app into your gmail.

If you look at the service that want access to all your gmail data - many
promise something "free" but then mine that data (in the fine print) to send
you offers, alert you to "savings" etc.

I automatically turn down apps that say they need access to my entire google
drive and all email. Why not just ask for permissions for a single app
specific folder? Ie, fax apps -> they should just store inbound faxes into one
folder rather than asking for full drive access.

~~~
Alex3917
> Why not just ask for permissions for a single app specific folder?

Basically because Gmail A) doesn't have folders B) doesn't have permissions
based on tags. Otherwise most Gmail API apps would have this option.

The same thing is a problem when you want to delegate access to an email
account, where there should be a way to delegate access based on tags, but
there just isn't.

What you can now do is create Gmail Add-Ons, which only have access to this
specific thread that's open when you click to activate the add-on. E.g. this
is how we created [https://www.prettyfwd.com](https://www.prettyfwd.com)

~~~
orion138
Looks interesting but it’s unclear to me, do you copy the threads content?

One other comment, there seems to be a spammer on the review page...

~~~
Alex3917
> do you copy the threads content?

Basically we take each message of the thread, normalize the typography so that
it can be consistently styled, and then strip off the signatures and quoted
reply text while leaving any quoted inline replies. There are some open source
libraries that do parts of this, but none of them worked well enough for this
particular use case so we ended up just developing some new techniques
ourselves.

The other thing the tech does is it lets people run much more accurate NLP on
threads. This isn't fully productized yet, so right now there are just a
handful of companies we're working with to clean and normalize their email
data.

In terms of the spammer, yeah I see that but I'm not sure what I can really
do. As the app creator I don't seem to have any special ability to delete
comments or flag things as spam.

------
bengotow
I'm the author of the Mailspring email client and I've been dealing with this
Oauth verification process for the last three months. Mailspring has "pro"
features that leverage a small backend API, but it syncs your mail on your
computer and your mail data, passwords, tokens, etc. never leave your machine.
I care very much about data privacy and I wouldn't use the app myself if it
was sending mail data to the cloud.

I'm a big fan of Google watching out for their users. I know of at least one
very sketchy company that has shut down because of this new policy, which is
great.

But after three months, they basically told me: "Your desktop app makes a
network request to a third party server, you must pay $15,000 for a security
audit." Their process has been vague and I wish they'd make an effort to
understand whether an audit is really necessary. Their security contractors
are going to be laughing all the way to the bank as they review my web service
that never sees Gmail data in the first place.

Thankfully, Mailspring makes a bit of money and I can afford to do this to
keep it alive. But fast-forward a few years and this is going to devastate
innovation and development of third party mail clients. (And I think Google
prefers it this way.) If the app didn't already have critical mass, or if I
was just starting a mail app now, I'd probably throw my hands up and give up
rather than emailing them dozens of times and coughing up $15k.

------
jonbronson
As both a gmail user and developer interested in applications to help me
manage my personal information, this is incredibly depressing to hear.

The idea of a verification process itself is great, and I applaud that effort.
But some of these barriers seems put in place solely to kill competition and
prevent startups from filling the personal data needs before Google comes up
with its own plan.

These exorbitant fees of $15,000 and $75,000 are completely unjustifiable.

~~~
Klathmon
It's the direction that everyone seems to be moving. Wall up the gardens,
remove your own access to your information, and remove the ability to share
and integrate across platforms.

The weird thing is how quickly the sentiment turned from my perspective. I
felt like one day most technical people applauded the ability to have total
real-time access to your data, to be able to write code or use open source
code to plug into these systems and augment them for better. To be able to
have a startup or small company write software that can work with your
google/facebook/apple/whatever account and use the information there in new
ways.

Then all of a sudden (I feel like it was between 1-2 years ago, but I can also
barely remember what I had for lunch yesterday so don't quote me on that!),
technical people started slamming companies for "allowing someone to access
their data", I saw lots of headlines about how it was unethical for Facebook
to allow users to share their information with other companies (don't get me
wrong, facebook does plenty wrong, but to call out the ability to share info
specifically seemed so wrong). Then APIs started shutting down, access is now
only allowed for other big players, and it's getting harder and harder to
integrate outside of a single player's walled garden.

I get why the companies are doing it (someone told them the only ethical thing
was to lock users in!?), but I don't get why HN and other technical circles
are applauding it. Maybe i'm on the wrong side of history here, but I just
feel like it's never a bad thing to allow me to share my information if I
want. I think it should be clearly defined what i'm sharing, I think it should
be obvious that i'm sharing it, and I think that some auditing and controls
are obviously a good thing, but not this almost absolute shutdown of any
ability for me to export or use information from these services on my own.

But maybe I'm really in a bubble, and maybe users really shouldn't be given
the choice to share their personal information, but it just feels so wrong and
so "holier than thou" to make that choice for them.

~~~
jeromegv
You're confusing 2 things.

Facebook shared data on millions of users to Cambridge Analytica. I never gave
my consent, you never did.

It is a different matter than having open APIs that allow you to get _your own
data_ out of Facebook.

I want to be able to get my own data on Facebook through open APIs. What I
don't want is for Facebook to give away my data to other people without my
consent. You can advocate for those 2 things at the same time.

~~~
londons_explore
Both cases are the same.

Alice sends Bob an email and Bob then shares it with an evil 3rd party spell
check extension... Alice's privacy has now been breached without her consent.

Alice posts her contact info to her wall, and her friend Bob (who has read
access) shares it with a third party (Cambridge Analytica). Alice's privacy
has now been breached without her consent.

Cambridge analytica collected the bulk of their private data with the consent
of a friend of the victim.

------
gonyea
This all makes sense to me. If you're not providing enough value to users to
cover the >$15k fee, you're just an attack vector for user data.

Consistency of the process aside, I'm really not sure what people would
expect.

(I work at Google, yadda yadda, but have nothing to do with this.)

~~~
nixpulvis
Fuck this, it's EXACTLY the problem in the valley. Small players should be
empowered, not stifled.

~~~
human20190310
I wouldn't want someone who won't bet $15K-$75K on their own product to have
access to any of my data.

~~~
nixpulvis
Your making a lot of assumptions. For example, who said anything about a
product. Not all "players" small or large are selling something. Sometimes
people just like to make stuff that works well for themselves and others.

~~~
StudentStuff
Things like gsyn.ch will just not be written. Google already throws scary
warnings about unverified apps using OAuth tokens when you use this service,
but Google's use of non-standard CSV entries rather than vCards for sharing
contacts has made for a terrible user experience if you want to use your
phonebook outside an Android phone/Gmail.

------
dktoao
Even for non-Gmail apps, this process is incredibly painful. I have an app
that has been stuck in the process for weeks. Once you have read through the
incredibly confusing and out of date documentation and submit what you think
is the correct set of setting to comply with their policy, you then have to
deal with the reviewer who will email you once every week if you are lucky.
Usually to understand what they are asking you to fix you have to email them
back and forth a few times. I love the platform, but they need to fix this
aspect of it.

~~~
creeble
They emailed you once a week? You are indeed lucky.

I guess I need to bug them more, I haven't heard anything in weeks (busy with
implementation).

One warning: choose the email address for your Google developer account
carefully, there doesn't seem to be a way to change it later. It is forever
tied to your permissions and approvals, afaict.

------
mariusz331
A friend and I have been working on a side project that depends on Google Auth
to send emails on a user's behalf. It's one of the app's two critical
features. We're not necessarily deterred by this story, but we'll start
rethinking our dependence on Google.

A $15-75k fee is something that's hard to stomach at our stage. We have about
10 Gmail users excited to try our product and they might not have an issue
accepting the "Unverified App" screen because we have earned some of their
trust through phone calls and meetings. However, converting people that come
across the app organically will be difficult.

We aren't sure when the right time will be to pay the fee and become verified.
Anyone have ideas on strategy here? It could help us and other developers in
the same position. We'd like to avoid raising money but this might be a good
reason to - investors may see Google verification as a competitive advantage.

------
wjossey
As someone who tried to develop a tool to help pause their box to improve
focus throughout the day, I got bit by this process as well.

Basically google just went dark on me altogether. Has been months since their
last reply and I kept trying to follow up. The feature I needed elevated
permissions on was the ability to add filters, which unfortunately is buried
with a bunch of other more dangerous permissions.

Looks like I’ll never get to launch the product :(

On the plus side, it works fine for just me! So, I just built a tool only I
can use.

------
nixpulvis
Someone needs to make it trivial to host your own email, and sell it as
reliable. I think you could probably sell more than just techies on it, given
how your email is a __critical __system to many people in modern times.

~~~
sn_master
Host it where tho ? Wouldn't you need a 24/7 running server ?

------
jacobsenscott
This terrible, but building a business based on third party api's is always a
tremendous risk. This isn't the first time a bunch of small apps have been
killed off by some company making their api's inaccessible.

Also, for people who are pushing for more government regulation of service
providers - this is the lite version of what you are asking for.

~~~
jonbronson
> this is the lite version of what you are asking for

I'd say this is the heavy version. A new startup can put together and run a
GDPR compliant web app for over a year for far less than $15,000.

~~~
tick_tock_tick
As long as no requests that you delete their data or provide them with a copy.
As soon as that starts happening $15k doesn't go very far in man hours to
comply.

~~~
vertex-four
Just because you’re terrible at documenting where you put users’ information
doesn’t mean everyone is.

------
raxxorrax
It is not too hard to understand them trying to leverage the potential market
of identity providers. Facebooks Libra basically tries to do the same.

It would be a waste to use any services attached to it in my opinion.
Otherwise oauth is a great technology, but interests may make it not
worthwhile.

------
nuclearghost
This is very helpful to see, along with the gmass blog mentioned within. We've
been going through this process for months and it's definitely a moving target
with no clear path to resolution. The whole process feels a bit Kafkaesque.

------
hexo
Stop using it and show google you don't care about their "services" as market
dictates.

------
PaulHoule
To the contrary it will be very good for innovation because it means people
will build extensions for other email providers such as Fastmail.

~~~
icebraining
Does Fastmail even provide an API like this?

~~~
PaulHoule
Sure, IMAP, POP, CALDav, same as all the other mail providers...

And the optimized JMAP protocol too.

The fact that I am getting voted down proves how absolute the groupthink is...
If you work in Google's ecosystem, however, the only possible exit is being
bought by Google... Get bought by a competitor and they'll just turn you off.

~~~
joshuamorton
So the answer is no then, because Gmail still provides imap and pop, which are
unaffected by this.

~~~
PaulHoule
But how does the credential management work?

~~~
joshuamorton
I don't recall since the last time I set it up was a while ago, but I think
just your password + an app specific code if you use 2fa normally. But yeah
I'm pretty confident this doesn't apply to outlook.

~~~
PaulHoule
So why don't we see more startups use open protocols for access to email
rather than make things that are GMail specific?

Why do you get voted down for just suggesting that they do so?

~~~
joshuamorton
I'm not sure, my expectation would be because oauth is more secure and feature
rich than either imap or pop. Jmap I'm less familiar with.

~~~
kitx
IMAP can be awkward with sync, pagination and has never played great with
labels vs folders. JMAP looks promising.

