
Repositories held for ransom by using valid credentials - linuxbuzz
https://about.gitlab.com/2019/05/03/suspicious-git-activity-security-update/
======
avar
> We believe that no data has been lost, unless the [...] GitLab copy was the
> only one.

One difference between how GitLab and GitHub run their infrastructure is that
GitLab doesn't keep reflogs, and uses git's default "gc" settings.

As a result they won't have the data in question anymore in many cases[1].
Well, I don't 100% know that for sure, but it's the default configuration of
their software, and I'm assuming they use like that themselves.

Whereas GitHub does keep reflogs, and runs "git repack" with the "\--keep-
unreachable" option. They don't usually delete git data unless someone bothers
to manually do it, and they usually have data to reconstruct repositories as
they were at any given point in time.

GitHub doesn't expose that to users in any way, although perhaps they'd take
pity on some of their users after such an incident.

This isn't a critique of GitLab, just trivia about the storage trade-offs
different major Git hosting sites have made, which might be informative to
some other people.

I'm surprised no major Git hosting site has opted to provide such a "we have a
snapshot of every version ever" feature. People would probably pay for it, you
could even make them opt to pay for access to backups you kept already if they
screwed things up :)

1\. Well, maybe as disaster backups or something. But those are harder to
access...

~~~
1f60c
This tendency of Hacker News users to want to monetize everything is
sickening.

~~~
antoineMoPa
I wonder if one could monetize this tendency.

~~~
soulofmischief
Monetyzer: The world is your oyster, it's about time you start collecting
pearls.

~~~
edbaskerville
Maybe this comment was inspired by WellDeserved, a truly underappreciated but
still important app:

[https://www.youtube.com/watch?v=WoK4_dQbfuU](https://www.youtube.com/watch?v=WoK4_dQbfuU)

~~~
soulofmischief
No, but that was incredible. Thanks for introducing me to Cultivated Wit.

------
blumomo
Also GitHub users are affected. By the time of writing 379 public GitHub repos
have been compromised:

[https://github.com/search?o=desc&q=1ES14c7qLb5CYhLMUekctxLgc...](https://github.com/search?o=desc&q=1ES14c7qLb5CYhLMUekctxLgc1FV2Ti9DA&s=indexed&type=Code)

~~~
cdubzzz
I like this threat for open source software published on GH --

 _If we dont receive your payment in the next 10 Days, we will make your code
public or use them otherwise._

~~~
akoster
(At least for public repos) Isn’t this almost a compliment? (Having another
person host your code and use it?) xD

------
nerdy
Based upon other comments the title has already been changed several times.

It still suggests Gitlab's infrastructure (internally) was compromised:
"Suspicious git activity detected on Gitlab"

Something like "Gitlab users' repos held for ransom" seems more appropriate.

------
edwinbalani
"Gitlab.com was compromised" is a bad title. _Accounts on_ GitLab, and the
credentials to access them, were compromised, but the title suggests that the
whole platform was affected, which doesn't seem to be the case.

~~~
jjeaff
Looks like they changed the title. But I would have to say that the ability to
delete a full repo with the credentials is a bit of a vulnerability.

To me, it seems like a good measure would be to mark deleted repos as "delete
requested" then notify the users involved and give them a week or two to undo
a total delete. Especially if it is an older repo with lots of commits.

------
paxys
Gitlab was NOT compromised. Someone found passwords/tokens for Gitlab
repositories exposed on the internet and held them for ransom.

~~~
mortehu
Purging user data is one of the most common action attackers take when
compromising an account. This makes it prudent for storage service providers
to silently delay mass deletions to the extent allowed by their data deletion
policy/GDPR to allow time to discover any breaches, or perhaps require second
factor verification like a link sent via email.

------
Ayesh
There was a Docker Hub breach a few days ago, that's probably related.

I took a good look at how my personal tokens were used in Github and Gitlab.

\- Enable 2FA.

\- Enable Commit signing with GPG. for the past 2-3 years, I have slowly moved
to sign commits and tags. GPG keys take a log of hygiene to work with (sub
keys, revocation, etc), but they definitely can help in a situation like.

Git is a distributed VCS. If you have a repo cloned in a secure location (your
server, Dev machine, etc), that is just as good as your Gitlab/hub hosted
copy.

~~~
jcims
The ‘play with docker’ site used to make it pretty easy to see what others
were up to and snag git creds if they left them around.

------
vasco
The current title "Gitlab.com Was Compromised" doesn't seem accurate. There's
someone (or a group) currently attacking online repositories (gitlab is not
the only affected provider) using passwords found in scans for files like
.gitconfig's and the like. Unless new information comes to light about gitlab
specifically being compromised, I'd say this is more about individual private
repos being on the sights of a targetted attack.

~~~
commandersaki
So git doesn't let you add the `.git` to the index. Most reports I've seen
mention that SourceTree was used as a git client. Is it possible that
SourceTree committed .git and pushed it to remotes which were then scraped?

~~~
plttn
[https://twitter.com/bad_packets/status/1124429828680085504](https://twitter.com/bad_packets/status/1124429828680085504)

Looks like someone was scraping for `.git/config`

------
a3_nm
I don't get the ransom thing: users of a git repository have a clone of the
repo that contains the whole history, no? So isn't it trivial to recreate the
repository?

~~~
ph4
The attacker is also threatening to make these private repos public, or misuse
their access to the repos in other ways (likely additional types of breaches).

------
2T1Qka0rEiPr
Weird aside question: I notice the article says "at approximately 10:00pm
GMT". Can someone explain why GMT might be chosen as a reference point here?
Is there something I'm missing about the usage of GMT (and not UTC). It just
seems particularly odd given that GMT is not (to my knowledge) actually being
used as a concrete time-zone at the minute (BST is in effect for daylight
savings).

~~~
s_ngularity
In the UK, GMT is often used to refer to "the current British time" both
GMT/BST. I've seen the same in the US where people say EST but mean EDT.

~~~
2T1Qka0rEiPr
Is it? I mean, I'm British and I'm not aware of this.

~~~
stordoff
I've seen a few people do it, but it's also usually corrected as been wrong.

------
chiefalchemist
\- Sidebar

For those who might not be aware: It's possible to configure your .git config
to push to different remotes.

------
snowwolf
While the fault lies with the users for not following security best practices,
including enabling 2FA there are things gitlab/any site can do to help defend
against these sorts of attacks. Some suggestions: Treat logins from
datacenters as suspicious. (In this case the IP block identified belongs to
World Hosting Farm Limited). Treat logins from a new/different ISP as
suspicious. Limit access to the account and verify the login via email. It’s
not foolproof but as part of a defense in depth strategy it can be quite
effective.

~~~
gitlab-security
Thank you for your feedback and suggestions. Unfortunately, for each of these
proposals, we're likely to have users asking us why we are restricting and/or
blocking access.

A better defense-in-depth strategy would be to scan each public repo for
credentials, and act accordingly when credentials are discovered in repos. We
are working on this strategy, currently.

~~~
snowwolf
That doesn’t help stop the attacks using breach lists that are even more
prevalent.

You could start with email warnings of suspicious activity and fine tune the
model parameters based on feedback from false positives. But generally a login
from a device that has no previous cookie, from an ASN the account has never
used before, especially if that ASN is a known data center, that then
immediately attempts a destructive action, should be a pretty big warning
flag.

------
sylens
A bit of a misleading headline. Somebody had passwords/tokens for certain
repos, either from previous breaches at other services or those passwords
being stored in plaintext as part of a deployment.

------
kzrdude
Since the threat is to make the code public, there is nothing more gitlab can
do to shut down the attempted blackmail. It seems unlikely to be a real threat
to most?

~~~
tyingq
Odd threat in that paying the ransom doesn't assure they wouldn't make it
public anyway.

Also, someone else noted the ransom email domain has no MX or A records, so
the instructions to email them won't work. They seem to be hoping someone will
blindly pay the ransom.

------
_bxg1
"...to wipe their Git repositories and hold them for ransom."

What an idiotic strategy to take with git repositories. Every local copy is a
complete and fully-functioning copy of not just the code, but all history,
etc. It's a non-centralized protocol.

------
exabrial
U2F token, which is supported by gitlab, would have stopped this

------
theWheez
A client of mine was hit by this. Makes me think I aught to go into security..
I brought up many concerns, and specifically about git access (using _very_
weak mechanisms), a few months ago, to which I was told to "clean things up
when I can" but that it wasn't a priority.

------
kissgyorgy
This is why I'm hosting GitLab myself even if I'm the sole user of the
instance. For one thing I'm less of a target, the other is that this is not
the first major problem with the hosted GitLab. I have better uptime for my
instance than the hosted version.

------
agildehaus
Mandated 2FA should really be a thing, especially on tech-oriented sites with
such importance.

~~~
smudgymcscmudge
Both GitLab and GitHub allow organizations to require members to use 2fa. So
the option is there.

------
1023bytes
Some other info:
[https://www.theregister.co.uk/2019/05/03/git_ransomware_bitc...](https://www.theregister.co.uk/2019/05/03/git_ransomware_bitcoin/)

~~~
yoz-y
Out of all things to hold for ransom git repos seem like a bad idea. Most of
the time there are multiple clones lying around anyways. I agree that having
the source code leak can be bad news, but the code itself being secret should
not be a critical part of the business.

~~~
ctennis1
Probably not, but I suspect some people will still pay the ransom anyway. Keep
in mind, many people also do keep "secrets" used for other authenticating to
other services in their git repos - even though it's a terrible idea.

~~~
jobigoud
These secrets are now compromised anyway.

------
based2
[https://security.stackexchange.com/questions/209448/gitlab-a...](https://security.stackexchange.com/questions/209448/gitlab-
account-hacked-and-repo-wiped)

------
localhostdotdev
kinda hoping "testing weak passwords on your existing user's passwords"
becomes standard practice at some point.

~~~
tyingq
GitHub does that...it warns you with a banner.

------
nicolaslem
The title on HN is clickbait, the article mentions Gitlab users storing their
own Gitlab password/tokens insecurely. It doesn't look like "Gitlab was
compromised" to me.

The original title is "Critical security announcement: Suspicious git activity
detected".

~~~
sytse
Correct, users across GitLab and GitHub as been affected and in all cases
valid credentials were used. Also see
[https://www.bleepingcomputer.com/news/security/attackers-
wip...](https://www.bleepingcomputer.com/news/security/attackers-wiping-
github-and-gitlab-repos-leave-ransom-notes/)

We are updating our title to better reflect what happened.

~~~
fillskills
Sorry but the title still doesn't reflect the actual issue which per your link
is : "Attackers Wiping GitHub and GitLab Repos, Leave Ransom Notes"

~~~
sytse
Did you see our new title? [https://gitlab.com/gitlab-com/www-gitlab-
com/commit/447f2330...](https://gitlab.com/gitlab-com/www-gitlab-
com/commit/447f2330391992a16c54427d2252b5e3995d207b)

------
fmhul
>The breaches seem to rely on the attacker having knowledge of the affected
users passwords in order to wipe their Git repositories and hold them for
ransom.

Yeah, until I go to my computer and use "git push" again. No?

Also gitsbackup.com is registered but has no A/MX records so...

~~~
tyingq
_" Also gitsbackup.com is registered but has no A/MX records so..."_

Gitlab should really note that in their blog posts and emails to users. Just
in case someone is thinking of paying the ransom.

~~~
sytse
We agree that paying a ransom doesn't guarantee any further actions on the
part of the attackers. But in our blog post we want to stick to what we know
and can influence and not talk about an external DNS record that can be added
at any time.

------
rmkrmk
> We believe that no data has been lost, unless the owner/maintainer of the
> repository did not have a local copy and the GitLab copy was the only one.

Too bad they don't make backups of users repositories?

~~~
phoe-krk
GitLab is not a repository backup service. They are only required to hold
multiple copies of what is the current version of the repository for their own
hard drive fault tolerance purposes; they are not required to hold copies of
the repository from a day, week, month, or year ago.

~~~
rmkrmk
Was just a question, don't get me wrong here. Thought they would do this for
their managed service.

~~~
zoidb
You can see how GitLab handles repository backups on the following page
[https://about.gitlab.com/handbook/engineering/infrastructure...](https://about.gitlab.com/handbook/engineering/infrastructure/design/disaster-
recovery/#repository-data)

