

GitHub major issues - repos have "lost" commits and site is erroring  - andrewljohnson
http://support.github.com/discussions/repos/5401-something-went-wrong-error-on-walkerledge-zeppelin-private-repo

======
antirez
Suggestion for creating a startup:

For private repositories gitolite is really awesome, but I think there should
be a company about it. What I mean is that github or some other company should
offer hosted gitolite instances, in the following form:

You pay something like price_of_small_linode_instace+price_of_service, to get:

A virtual machine with gitolite up and running, with a good web interface and
online documentation. I think that the web interface part is the piece this
company should _not_ opensource. With this web interface you can create
repositories and users, and modify the gitolite settings in general.

A backup service, in some other different VM and network, so that if something
goes wrong with your gitolite VM you can ask a free (and automated) restore,
possibly with hour granularity for the latest 24 hours, and less granularity
for bigger periods and so forth.

Note that this gives the user virtually infinite repositories... if they don't
upload movies to their instance, as there is a lot of space even in a small
linode instance for git repositories.

This way this is a "pure product" as there is no need to repair or install
manually virtual machines, and the company can focus into evolving the
offering, developing a better version of gitolite, of the web interface, and
so forth.

p.s. I mentioned linode but can well be ec2 or whatever is better suited for
this stuff.

~~~
icefox
What do people _really_ think of this idea? Would _you_ buy it? I have written
a web interface for managing git repos called GitHaven (my personal home
install is here: <http://git.meyerhome.net:8080/>). It has the usual goodies,
private/public repos etc. I have some friend's companies that run it in the
cloud already. Do you think it would actually make money i.e. Would you be
willing to join me in a startup on it (in boston)? Would it pass the
y-combinator test?

~~~
tptacek
Probably not. Github is on a path to become the market leader, has a critical
mass of customers, and has a business model that harnesses those customers to
benefit from network effects. This idea is akin to building a business
offering private white-label Stack Overflow clones. Key point: the actual
software is not what makes Stack Overflow and Github valuable.

Simpler reaction: Github:FI is already priced at a rounding error for the
kinds of companies from which most of the revenue in this space will
originate.

~~~
icefox
From the research I have done there are three major types of git server
buyers:

1) We are so small that we don't even think twice and buy some private repos
on GitHub. (i.e. no market there)

2) We are so big that we can't possibly even fathom the idea of having our
code on someone else's server so GitHub.com is out of the question. Sadly
GitHub:Fi is per/user and because we are really big that means we have lots of
devs so it is too expensive in most cases. (i.e. enterprise sale yuck)

3) For medium sized companies that either have some devs (more then 5) or a
ton of repos or large repos they want an alternative to github. Many are
slapping up gitolite which while good is a far cry from github.

Like I mentioned I already have found companies that want this solution and
they are using GitHaven. The real question is if this market is too small or
should I just move on to another project, open source GitHaven and let
everyone (who wants to) slap it up as a frontend to their existing gitolite
installs?

~~~
tptacek
Most products targeted at developers are per-seat priced (or have pricing
derived from per-seat pricing); it's how you do value pricing when value
scales with the number of developers. Look at how Atlassian does it; that's
still effectively per-seat licensing (it requires prospects to think about how
many devs they'll have).

You can try to avoid these schemes, but you have to do it _and_ not go out of
business.

~~~
sdesol
Yeah the per-seat point is really a moot point when it comes to enterprise.
ClearCase is $4000 a license and a lot and I mean a lot of large companies use
it. I know of a company that is currently spending $2.5 million a year on
ClearCase licenses and they really want to get off it but ClearCase is so
entrenched/integrated into their work flow that this is not a trivial task.
Github is not going to solve this problem for them.

The problem with Github:Fi is the value proposition is extremely weak when it
comes to meeting enterprise needs.

\- They have an issue tracker that doesn't support custom fields. This is
critical for companies with 500+ developers and/or very diverse product lines.

\- They have a wiki but you can get confluence for $12,000 for the entire
company which is more feature rich. It has decent searching.

\- They have blogging but this is of no value in the enterprise world. The
majority of employees really don't have the time or care about blogging
internally.

\- They have Gist which offers no real value for the same reason why blogging
isn't that big of a deal.

When it's all over, their only strong value proposition for the enterprise
world is their intuitive push and pull interface. But the problem with this is
they are competing directly with free solutions like:

\- gitorious

\- gitolite + internal web interface

\- gerrit

I'm obviously speculating here, but based on how their pricing model changed:

old: 1000 per user (initial purchase) + 200 something a year per user
subscription cost

new: 250 per user if less than 25 users and 200 per user if more than 25 users

I think they realized they over estimated their value proposition initially.
If Github wants to seriously break into the enterprise world, they are either
going to have to lower their per seat cost and/or strengthen their value
proposition.

------
wizard_2
Just posted to the support page.

"Sorry guys, there was a Javascript error displaying an error when there was
none. We've deployed a fix for this and everything should be good to go."

~~~
andrewljohnson
This seems to be fixed now - for my repo at least.

The GitHub guy saying it was a "JavaScript display error" is incorrect though.
I was pulling a repo and getting an old version.

EDIT: I'm not getting the javaScript display error, but I'm still missing my
commit, so I guess these errors are unrelated :/

~~~
mojombo
Did you file a support ticket for this? It sounds like an unrelated problem.
We can investigate if you let us know what repo is causing the problem. Visit
<http://support.github.com> or email support@github.com.

~~~
andrewljohnson
I posted the name of the repo to the thread, and someone else on the thread is
confirming the same problem.

~~~
andrewljohnson
It turns out that GitHub didn't lose any of my commits. All future readers
should ignore my comments on this.

------
rabidsnail
Is it just me or has guthub's performance and reliability been getting
progressively worse over the past month or two? Even putting this issue aside,
and the outages, pushing to a private repo takes _minutes_ now, and file lists
in the web interface often don't load without a page refresh.

~~~
windsurfer
At the price and size of Linode instances, it's cheaper and I feel better
knowing I'm in control when I run Redmine+Gitosis. When I had my github
account and something went wrong, I couldn't just ssh in and fix it. All I
could do is wait.

Now, for some people, I can see how waiting is an easier solution. I
personally just don't like it.

(of course, this is reffering to private repos. public repos are still king on
github)

~~~
erikpukinskis
It's amazing to me that you feel more confident in your own abilities to keep
a server running than GitHub... you must be a really great sysadmin! The
GitHub folks are pretty hardcore!

I, for one, am a positively mediocre sysadmin. And the amount of time I have
to spend waiting for GitHub to fix bugs is almost certainly far, far, FAR less
than the amount of time I'd spend scrambling (but not waiting!) to fix my own.

This is also how I feel about Heroku.

For whatever reason, I prefer the helplessness I feel while waiting for them
to fix stuff to the helplessness I feel when I'm in "OH SHIT MY SERVER IS DOWN
AND I DIDN'T EVEN DO ANYTHING AND WHY IS THE DATABASE NOT RESPONDING IT
RESPONDS WHEN I QUERY IT MANUALLY AAAUUGGHHH!!!" mode.

Although I do try to keep myself sharp by doing things by hand in other areas
of my engineering activities.

~~~
jshen
The beauty os a dvcs is that I have a full copy on my machine as well.

But seriously, if you use git over ssh there is no sys admin work other than
running apt-get upgrade

~~~
ekidd
There's _always_ sysadmin work.

1) You need to keep current on your security patches.

2) You need to upgrade your OS when it reaches end-of-life.

3) You need to make backups, and more importantly, verify that those backups
will actually restore. (For git, this is most critical if you have a large
team and dozens of repos.)

You can ignore a server for years, but eventually you'll get compromised or
loose a hard drive. (If you use RAID, you'll eventually lose or an entire RAID
_array_. Not fun at all, nor cheap.)

~~~
saurik
I'd say "most developers" already have a box running some kind of website
where they are doing this anyway, but even if you disagree I think you have to
be willing to cede "many", especially given how many people who use GitHub are
currently doing Ruby on Rails or node.js work. I'd even go so far as to say
that those that currently aren't /should/, as the experience being a sysadmin
is important when understanding how other sysadmins will react when they see
how your software is deployed, which I guess is another topic of discussion
that comes up often here (oft filed under the "Debian vs. Ruby" banner).

(Part of me is wondering if many of the more controversial discussions on this
site are between people who have sysadmin experience (and considered it
valuable) and people who don't.)

~~~
ekidd
_Part of me is wondering if many of the more controversial discussions on this
site are between people who have sysadmin experience (and considered it
valuable) and people who don't._

I've done a fair bit of sysadmin work over the years, mostly in self-defense,
because I want fewer crises when a critical development server eats itself.

Lots of people can figure out how to install Ubuntu, or rent a Linode. But if
they don't master upgrades, patches and backups, they'll eventually end up
paying a real sysadmin a lot of money at the worst possible moment.

------
hsuma
I'm still on the fence about that whole cloud thing, personally.

~~~
__david__
Well, here's the beauty of Github--if they lose all my repos I just don't
care. Since I've got a copy of my repos locally, there's absolutely nothing
they _can_ lose... Worse case if they go completely down for a couple days I
can still collaborate using ssh or "git send-email".

This is the only reason I decided to use Github after avoiding Sourceforge and
their ilk for the past decade and a half.

~~~
jamesbritt
"This is the only reason I decided to use Github after avoiding Sourceforge
and their ilk for the past decade and a half."

But why github? Why not gitorious.org?

If github goes away, I still have my code, but I lose any issues people have
filed on the site, and I lose being able to easily check on forks of my
projects where people might be doing interesting stuff.

I had been using gitorius.org, but moved to github for all the things other
than git that makes one public git host different from another.

~~~
bitserf
There is the Github API, so you could backup the non-repository data as well
by writing a couple of scripts, if you were so inclined.

~~~
jamesbritt
Right, and I should probably set up something to auto-snag that stuff where it
counts, but having to do that is the sort of thing you have to so for any non-
git-based site.

It'd be nice of all those related items were also in a repo, easy to pull.

------
btipling
Well luckily you have the entire repository where ever it exists on your dev
machines. So I don't expect there would be any data loss. In addition all the
commits, tags, trees and files are hashed. The content is hashed, if there is
a problem you'd know about it.

------
mcmatterson
In future, they keep a very up-to-date status page at

<http://status.github.com/>

~~~
andrewljohnson
There is nothing useful on that page. The error is in full effect right now.

~~~
mcmatterson
Oops, my bad. I mistakenly thought the 'db error' notice was for today.

I retract the 'very' in my original parent comment, replacing it with
'usually'.

------
spooneybarger
It isnt just private repos. I'm seeing it on various public repos on github.

~~~
cilantro
Someone had shared a public repo with me on Monday. It started 404ing the next
day and I had assumed he took it down. I realize now that it must have been
this issue. I just checked for it and it is back.

------
ericmsimons
I hope this has nothing to do with the people who hacked sourceforge..

~~~
jrockway
Git has full cryptographic history (and data) authentication, so any changes
to the history would be easily detectable. And you can sign known-good
commits, so that even if you've never pulled before, you can still verify the
part of the history that's been signed.

SVN and CVS are missing this key feature, which is why the sf.net hacking is
scary.

~~~
ericmsimons
I'm a Git n00b, so thanks for pointing that out! No wonder git has killed off
SVN

~~~
oomkiller
It hasn't yet, but it is making great headway, and I for one can't wait!

------
jasonervin
Not good.

------
ciaogiorno
Why can't we have a web interface for GitHub?

I should be push a new release right from my browser.

