
Why we aren’t using the Gmail API (yet) - saryant
http://blog.conspire.com/post/100016691078/why-we-arent-using-the-gmail-api-yet
======
amhoab
I wonder if they use IMAP's EXAMINE or SELECT commands when iterating through
folders. The former is read-only, although the end user has no control or
guarantee that this is how they'll go through folders.

------
esquid2
Did you ever try to contact someone at Google for higher quota? (There's a
form linked from the developer site.) I'm not surprised the default quotas
were low, it is still Beta...

~~~
saryant
Our primary issue isn't the quota limits, it's the actual speed of the API.
The best we were able to achieve with the Gmail API was 3000 messages/minute
whereas IMAP easily lets us get 20k messages/minute.

~~~
esquid2
Are you limiting yourself to one thread or something? If you're not getting
limited by quota then why not use batching + more threads until you do get to
those limits?

~~~
saryant
We are getting limited by the quota. As the documentation says, quotas can be
temporarily exceeded and in our tests we could fetch about 3000 messages in
those bursts. After that, we would be 403'd for one to two minutes before
requests would succeed again with quotas strictly enforced for an unknown
amount of time past that.

------
sportanova
How did you make javamail fast? My impression was that you could batch request
meta data info, but not the actual body.

Gmail's been way faster for me

~~~
saryant
We don't request message bodies, just headers.

~~~
esquid2
When you do your tests are you using the new-ish message.get(format=METADATA)?

~~~
saryant
No, I wasn't aware of that option.

In our tests we did list requests for 100 messages and then bulk requests for
each messages' headers (messages.get("me",
message.getId).setFields("historyId,payload/headers")).

I'll look into that option and re-run the tests.

------
jessaustin
Would they have preferred to just have someone at Google to contact about
this?

~~~
toomuchtodo
You can speak to a human at Google regarding their API limitations? I'd be
shocked if that was the case.

~~~
jessaustin
Perhaps my comment was a bit opaque... b^)

~~~
saryant
Not sure if it was you, but someone from Google did reach out to us. If it was
you, I'll be replying this morning once I get to the office! If not, thanks
anyways!

~~~
jessaustin
No I'm not at Google. Congratulations on finding a contact, and good luck with
a resolution for the issue you describe. It might be interesting if you did a
followup after the whole process plays out.

~~~
esquid2
I work at Google. We don't typically monitor HN, reddit, twitter, etc but we
have plenty of people watching Stack Overflow (the one we direct people to
from our docs). The default quotas are conservative during this Beta period
but we're happy to increase when needed--the service is certainly not
provisioned as high as IMAP at this point. Submitting the quota increase
request (from the developers console) will send us an email and we'll either
approve it or if we need to, we'll start an email thread to ask for more
info/suggest improvements. Hope that helps!

