
Show HN: Ghuser.io – Complete GitHub profiles - lourot
https://github.com/AurelienLourot/ghuser.io
======
lourot
OP here. The idea came because on your GitHub profile:

\- I can't see all your code contributions since the beginning of time at a
glance.

\- For each of your pinned repos, I can't tell at the first glance whether you
just fixed a typo or you wrote 90% of it.

ghuser.io tries to close this gap. You can show off with your entire GitHub
"portfolio" and it gets refreshed every day.

And hopefully this is temporary. The best for the community would be to have
all this directly on GitHub. The sooner ghuser.io can be switched off, the
better ;)

~~~
rmetzler
I'm the guy fixing the typos in everybody's README :)

~~~
lourot
Hehe :) And this is really useful, I hope my message didn't make it sound like
the opposite. (Unfortunately our readme is probably full of typos!)

------
fabricexpert
Could you make this work for gitlab, bitbucket and maybe even just regular old
git repos as well?

It would also be really cool if we could pull out basic private repo info as
well (with the user's permission). E.g. Worked on X private repos covering 60%
Go, 20% Javascript etc.

I am also a bit confused as to why you would build this if you don't work at
github?

~~~
brillout
> Could you make this work for gitlab, bitbucket and maybe even just regular
> old git repos as well?"

Yes totally. Although we currently think that the best for the community is
GitHub improving their GitHub profiles making ghuser.io obsolete. Right now
we'd rather do this with GitHub and not against GitHub. But we don't know how
much GitHub is willing to help us. (We need changes to the GitHub APIs for
ghuser.io to be scalable.)

> private repo

Yea we could do that as well. Give permission to our public key and we're good
to go.

> why you would build this if you don't work at github?

Because Aurelien (OP) wanted/needed that so he just built it.

To the both of us it's basically a fun experiment.

Also for me it's cool because we use the web framework Reframe
([https://github.com/reframejs/reframe](https://github.com/reframejs/reframe))
which I'm currently building.

~~~
jdormit
Just so you know, you have a name collision with a popular ClojureScript SPA
library, re-frame: [https://github.com/Day8/re-
frame](https://github.com/Day8/re-frame)

------
lourot
OP again, we're way overloaded. We got 200 profile requests already in an hour
and we didn't see it coming. We're extremely thankful and sorry to disappoint
you at the same time. We're closing the "Get your profile" feature for now.
Spinning up many EC2 instances to process the 200 profile requests faster in
parallel as we speak. And we'll come back soon with a system that can handle
this load.

Many many thanks!

~~~
KenanSulayman
Isn't 200 requests in an hour ... just three requests per minute? What is this
service doing?

~~~
brillout
GitHub's API does't provide the exhaustive list of all your contributions.
Instead we crawl GitHub's website which is slow.

Details at [https://github.com/AurelienLourot/github-contribs#how-
does-i...](https://github.com/AurelienLourot/github-contribs#how-does-it-work)

We are in talks with GitHub and they know that we are crawling GitHub.

~~~
ChristianBundy
> Instead we crawl GitHub's website which is computationally expensive

I'm surprised to hear that you're bottlenecking on CPU time. Could you verify
that my understanding is correct? I would've thought your bottleneck would be
networking and connectivity as you have to wait for GitHub to process all of
the requests.

~~~
brillout
You're right. I edited my answer.

The GitHub's unofficial API we are using is slow and per IP rate limited. We
spin up several servers to have several IPs to circumvent the rate limit.

(GitHub knows that we do that and we are in contact with them.)

~~~
exikyut
If GitHub are okay with you using multiple IPs to get that data then it's not
inherently expensive on their side for you to be using this.

Surely a rate-limit exception could be in order, then?

And perhaps you could help them alpha-test a new API endpoint that just so
happens to include all the info rolled up as one URL :D

(Hmmmmm.... GraphQL....)

------
sergey4096
Check out [https://sourcerer.io](https://sourcerer.io) \- GitHub and GitLab
profiles with extensive timeline, language, library, repo analysis.
Customizable.

~~~
kelsolaar
Just signed up, love it! Your [https://github.com/sourcerer-io/hall-of-
fame](https://github.com/sourcerer-io/hall-of-fame) is great too!

~~~
sergey4096
thank you!

------
toyg
Eh, you were too honest in your FAQs - chances are that Github will now close
the hole. I expect there is a reason your use-case was not enabled in the API
despite being already supported by the official frontend...

~~~
brillout
ghuser.io contributor here.

We talked with GitHub already. They think that what we are doing is cool and
we agreed to tell them our roadmap and they tell us theirs.

We would be glad if GitHub copies us.

~~~
ghoshbishakh
GitHub copying you! Now that's cool! _achievement unlocked_

------
ocdtrekkie
Pretty bummed I missed a chance to try this! I had manually gone through my
merged PRs list before to try and assemble a count of my contributions to each
public project to come up with some sort of profile of stuff I've participated
in/contributed to, as I really didn't see any of these type of pages doing
that. This looks much more like what I'd like to see.

~~~
lourot
hopefully we'll get back soon with a scalable system and you'll be able to try
it out :)

------
ohnoesjmr
Given you disabled onboarding, is my onboarding request queued somewhere or
will I have to comeback and re-request once you got over the hump?

~~~
lourot
We're working on serving all the requests that came in before we disabled
onboarding (about 200 requests). If you give me your github username I can
tell you if you're part of them :)

~~~
ohnoesjmr
I am pretty sure I am not, as I received the message that onboarding is
disabled.

If you have user requests in the logs, you should queue them up.

------
delinka
Am I understanding correctly that this is a manual process and not currently
automated?

(And can I cancel my request so you can move on to others for now?)

~~~
lourot
It's cheeply automated to handle 10 profile requests per day, which is more
than what we got in the past few months. So now we're giving it some human
help and we'll have to rethink the system.

What is your username? I'll cancel your request, thanks!

~~~
social_quotient
Seems like the perfect match for aws lambdas. I’d consider setting the tasks
for crawling in to SQS and then trigger the lambdas to go do single crawl
functions.

For better control over the throttle and concurrency you can leverage
dynamodb... I love it for controlling lambdas but not for storage.

If you need more power than a lambda then you can do a similar process with
EC2. Populate the SQS Trigger the aws lambda to turn on EC2 machine. Consider
spot instances to save a ton of money.

If you need ideas I’m sure HN readers would be glad to help solution for afar.

------
7oorih

          grafana/.github/CONTRIBUTING.md

