
From side project to 250M daily requests - coderholic
https://medium.com/get-put-post/from-side-project-to-250-million-daily-requests-909b9e373d94
======
pilif
This is information I would never accept an external dependency on. There are
not too many IP ranges around, so this information can easily be stored
somewhere and used without the need to query an external service.

Having the data locally allows for potentially synchronous lookups, or at
least for lookups with an availability guarantee which makes your code much
simpler to reason about.

Plus, you are guaranteed to get the information, independent of the
availability of a third party.

And finally for $100 per month I get access to a weekly updated locally
available database (that's what Maxmind charges for the city-level database)
no matter how many queries I'm going to issue locally.

Flat-fee access to a local database would even allow to bulk post-process web
server log files (yeah - they still exist and don't come with the usual
privacy issues surrounding third-party analytics providers) within a
reasonable time- and cost frame.

Not everything that can be an external service has to be an external service
and for geolocation I definitely cannot see any advantage to not having this
data stored locally.

This isn't big-data and will easily fit any amount of RAM (if it even needs
to), this doesn't require a costly sys-admin team, this doesn't require any
hardware knowledge. This is about fetching a file and putting it somewhere on
the server. You are doing this daily with your daily web-browsing.

Now, to OP, I'm very happy for you and I appreciate the service you are
offering and I'm very happy that you are solving an issue some people are
having. I don't want to belittle this at all.

I'm just saying that while I might personally err a bit too much on "doing it
on my own", I absolutely cannot see any justification to do IP geolocation
with an external dependency.

~~~
toomuchtodo
I could see the value of having this as a docker container to run internally
as a microservice, but agree I would never rely on an external service for
this.

~~~
azylman
We've done this at Clever, if you're interested:
[https://github.com/Clever/resolve-ip](https://github.com/Clever/resolve-ip)

We use the same dataset that IP info started with (from MaxMind), although it
sounds like they have many different sources now. It's worked fine for us,
matches over 99% of the IP addresses we look up, and generally responds in
less than 5 milliseconds - basically only limited by network latency.

Right now this is published to a private Docker hub repository, for no good
reason, but if people would find it useful we can make it public.

------
rokhayakebe
Sometimes the reason things succeed is simply because they are simple. Simple
to understand. Simple to use. Simple to integrate. Simple to pay for. Just
simple. One Feature, maybe. Three max.

------
psuter
Good insight into the engineering that went into this. Functionally, is this
"MaxMind-GeoIP-as-a-service"? Any other data sources?

[http://dev.maxmind.com/geoip/](http://dev.maxmind.com/geoip/)

~~~
coderholic
Yeah, lots of custom data. See
[https://ipinfo.io/data](https://ipinfo.io/data) for an overview

~~~
dougb5
Since you are building off of GeoLite which has a ShareAlike license
([http://creativecommons.org/licenses/by-
sa/3.0/](http://creativecommons.org/licenses/by-sa/3.0/)), does that mean you
are distributing your modified database under the same terms?

~~~
coderholic
It's a good question. I've been in touch with Maxmind's legal team before and
they were happy with what we were doing, but that was before we had started
building out our custom data.

My thoughts are:

1) We're not modifying the maxmind DB. We download their MMDB file and leave
it completely untouched. Our code basically does something like this:

    
    
        if ip in custom_db
            return custom_db[ip]
        else
            return maxmind_db[ip]
    

2) Our database is openly available via our API. Anyone can access it without
needing to signup etc.

They're the principles I've been operating under, but I'm not a lawyer, and
I'll definitely go consult with one to get a definitive answer on where I
stand with this. If it turns out that we're not meeting the license terms then
we'll certainly make whatever changes necessary to ensure we do.

------
psuter
One service you could consider offering is "resolve IP at past date". I don't
know that the standard providers do it and it useful when, for instance,
batch-processing a backlog of session data.

------
mmriis
I'm curious... are you seeing 99% using your free plan or are you actually
selling the $400 plans? What is the conversion ratio?

------
swah
This just placed me 1000km from where I actually am - the porn ads offering to
meet local girls do much better :(

------
merb
In Germany the only useful thing this site would give, would be the region,
not quite good for dealer finders..

