Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
The Rust community's crate host (crates.io)
260 points by steveklabnik on Nov 20, 2014 | hide | past | favorite | 139 comments



Here is the most important part to me:

"Take care when publishing a crate, because a publish is permanent. The version can never be overwritten, and the code cannot be deleted. There is no limit to the number of versions which can be published, however."

That. Is. Awesome.


This has been true of CPAN for a long time, and comes as a result of it being a network of FTP mirrors. I don't know when people decided it was a good idea to be able to yank a package (for example, from rubygems). It just complicates the whole setup and makes it less flexible.


I can speak to the Rubygems case: you'd be surprised at how often people publish proprietary/private information, and then need to pull it down. It happens often enough to actually be a burden on the Rubygems team (at least, when I used to co-work with one of them) that they left 'yank' in.


> you'd be surprised at how often people publish proprietary/private information, and then need to pull it down.

But once published, isn't such information already compromised for all practical purposes ?

I can understand the legal reason behind "pulling it down" (using someone else's IP, etc), but practically speaking the private information once published cannot really be "undone" (at best you can make it hard to find, and disassociate from yourself).


Let's say you accidentally published your name, address, and social security number.

If it is only up for 1 hour, the chances that anyone malicious sees it is low.

If it is there forever, the chances that someone malicious eventually sees it approaches 100%.


What you are saying makes sense, but you are taking on the entire copyright argument with this logic.


Licensing problems or other legal hassles are also foreseeable enough that you'd want to be ready for them.


I think description should be mandatory


homepage, too


For what it's worth this is also true of NuGet:

"NuGet.org does not support permanent deletion of packages, because that would break anyone who is depending on it remaining available. This is particularly true when using the workflow that restores packages on build.

Instead, NuGet.org supports a way to 'unlist' a package, which can be done in the package management page on the web site. When a package is unlisted, it no longer shows up in search and in any package listing, both on NuGet.org and from the NuGet Visual Studio extension (or nuget.exe). However, it remains downloadable by specifying its exact version, which is what allows the Restore workflow to continue working.

If you run into an exceptional situation where you think one of your packages must be deleted, this can be handled manually by the NuGet team. e.g. if there is a copyright infringement issue, or potentially harmful content, that could be a valid reason to delete it."

(Source: http://docs.nuget.org/docs/Creating-Packages/Creating-and-Pu...)


That's great ! Definitely a argument in favor of Perl. Being more used to NPM (for Javascript) and knowing that you can (or at least you could) unpublish a package, I'm glad to see that with Rust, I won't have to worry about dependencies disappearing for no reason.


CPAN as an argument in favor of Perl? That's a new one on me...


Isn't CPAN for Perl ? I don't actually know CPAN. All I'm saying is that having a package manager that doesn't allow removing packages is an advantage IMO.


Yes, CPAN is for Perl. I'm just saying I've never heard of a particularly good experience with CPAN.

CPAN packages can't really be uninstalled (http://stackoverflow.com/questions/2626449/how-can-i-de-inst...) and seem to be unnaturally prone to dependency hell.


I've had particularly good experiences with CPAN. Long enough ago I don't remember the details, and I wasn't doing anything exceptionally complicated, but I remember being favorably impressed and able to get my work done without hiccups.


Same here. Its infrequent that I go to CPAN and I am unable to find a package that will help along.

The concern about running the entire test suite on module installation really isnt a problem if you think about. Additionally you are able to force the install if it fails on account of a test.


CPAN dependency resolution seems worse than with other package managers for two reasons. First, CPAN prefers to install the most recent version of a dependency (rather than pinning to a specific version). Second, the default CPAN client behavior is to run the entire automated test suite for every distribution it installs.

While that behavior feeds back into improved quality for responsive maintainers, it takes additional time during installation. When combined with point #1, it can make dependency chains seem more fragile.


Huh, I can think of a lot of reasons to take a package down. Though I agree that "sorry you fucked up, too bad" seems a whole lot easier to implement.


Can you clarify this?

Generally, if you've published a package, it's possible someone has downloaded it. If you take it down, that's going to be strange and unexpected behaviour for those people.


Someone has maliciously posted a package with illegal content.

Someone has accidentally posted a package that contains a 2TB file, and now all mirrors have to sync it.

Someone accidentally puts their personal information in a package.


Illegal: There are legal ways to make that initial statement ("Cannot be deleted") false. If you're uploading a crate that offers nude pictures of random celebrities the guys at cargo.io will find a way to make this go away.

2TB file: That's .. nonsense. I assume guards are in place to prevent the oldest form of DOS attacks. If not, the guys at cargo.io will learn and .. make that go away?

Personal information: That looks like the only case where I sympathize with the guy uploading stuff. That said, this is how the net works? Publishing sensitive stuff to Github means that it might be out there forever (force pushing a new history doesn't mean that no one cloned the stuff before or just grabbed a zip of the current head).

For me its a win. I certainly can imagine some scenarios that might be painful, but .. that usually boils down to your third example, a developer error. The usual issue with 'removing packages' is that the user suffers. My gut feeling is that there are far more users that get 404s than developers that share their API keys.


There's currently a 10MB limit on uploaded packages.


there are a few possible cases of users suffering:

* you push a revision which introduces a bug

* you push a release which introduces involuntary API breakage

* the new release has a glaring security issue

* release X relies on a third party which has changed (think: some web service) and therefore doesn't work anymore

Sure, you can push a newer release but you don't want _anyone_ to be using the old one.

I'm not saying yanking is good, but maybe a notification system "this package should not be used, upgrade to XXX" would be useful.


Vendor guidance would be great.


If you accidentally published sensitive info in that package you may want to contain the damage.


Then contact the people that run cargo.io and see if they are sympathetic and what options exist in the circumstances. Setting a sane standard of "no, you can't fuck up everybody that has depended on your public package" is a good thing.


same with git rebase on master branches and yet people do that sometimes. there'll always be use cases for deleting a package from anywhere no matter what repo we're talking about.


I would like to contrast the underlying philosophy here to that of Go.

As you may be aware, there is limited support for versioning APIs using "go get" (Go's nearest equivalent of this). The developers of Go argue that the maintenance of your dependencies is your responsibility, and you should clone everything you need to your own environment. That gives you ultimate control, allowing you to pull and update them as you require, modify and bugfix as you see fit, and makes it hard, if not impossible, to end up relying on two packages with different versions.

If a project B depends on project A, then they must collaborate in some way to make sure that there are no breaking changes introduced between (minor) versions. If another project C also depends on A, then they must both use the same version of A if they are to be used together. The maintainers of A, B and C all have a duty to ensure that their code remains up-to-date and backwards compatible.

It seems that Rust values a different freedom. By specifying the exact version you require, there is no need to keep separate projects in sync. The flexibility comes at a cost; you may find you have different versions of many dependencies. Because they are not required to, maintainers may neglect the tedious task of keeping their projects' dependencies up-to-date, requiring bugfixes in multiple places in your own codebase.

Clearly, some of the smartest minds in our industry disagree what the best tradeoff is. It is interesting to see the range of opinion though.


Small nits about the way this works:

> By specifying the exact version you require,

This is only half true. It comes in two steps: you specify a _kind_ of version that you require, like

    foo = "^1.2.0"
and then, during the build step, it resolves all of the dependencies and their dependencies (etc) and writes the exact version out to a file. So you do end up with specific ones, but you won't neccesarily end up with multiple versions.

This also matters because...

> maintainers may neglect the tedious task of keeping their projects' dependencies up-to-date,

You just run 'cargo update' and it will re-calculate that dependency graph, and re-write out the lock file. You're as up to date as you can be. Of course, if that fix is only available in a range outside you've specified, that's different.


? As you may be aware, there is limited support for versioning APIs using "go get" (Go's nearest equivalent of this). The developers of Go argue that the maintenance of your dependencies is your responsibility, and you should clone everything you need to your own environment. That gives you ultimate control, allowing you to pull and update them as you require, modify and bugfix as you see fit, and makes it hard, if not impossible, to end up relying on two packages with different versions.

That's the last time where I heard about this wonderful idea that vendoring everything is a solution, because versioning is hard: https://www.kickstarter.com/projects/2066438441/haunts-the-m...

I'll take the solution where I can send a PR to the maintainer with updated dependencies over this sort of thing any day.


"The developers of Go argue that the maintenance of your dependencies is your responsibility" THIS!

I should also state I fall on the extreme end of the spectrum. My belief is that the moment you depend on upstream package X it is wholly your responsibility to track changes and adapt your code accordingly. I don't believe software versioning does anything but encourage poor security, performance and general fragmentation.

I believe that before pushing your changes to a remote repo, you locally check that everything is stable, meaning there is no dev branch or stable releases just one true master branch!

If you don't want the bleeding edge then don't 'git pull'. simple!


Cargo gives you exactly this, btw: if you don't want to change the version of upstream code on which you depend, don't run `cargo update`. (This applies to both git dependencies and registry dependencies.)


believe me I will 'cargo update' always! the problem is those who don't and still go on to publish libraries, what happens when he needs version 2.1 of X while another library needs version 3.4? And thus anyone who depends on these two libraries now ends up with duplicate libs. What if they also depend on other libs with the same problem...

The world goes boom! exponential overload!

... at least it get's really frustrating after a while, twice the space, twice the compilation time. And you'll know at least one of the libs has bugs lurking in it that have been fixed elsewhere except in that lib:(


Go is always statically linked at the moment so it makes more sense in that scenario.


I don't think that the linking semantics make much of a difference in what you should do with your source. Rust is, in fact, usually statically linked as well. Dynamically linked Rust crates are supported, but are uncommon in practice. (The most common use cases for dynamically linked Rust crates at the moment are compiler plugins and embedding Rust into other languages.)


For a dynamic library you need to ensure your source will work with the version range the package manager allows to be installed. Which may or may not be the most up to date version.

Subtle things in dynamic linking may break applications when dependency versions are changed.

Static linking lets you bundle whatever version is known to work, including the most up to date versions.


the code cannot be deleted

Well, that's a problem waiting to happen. Infringing source? Malicious material? It's a publishing facility and so will be abused.


Exceptions can be treated exceptionally. No need to allow deleting as part of everyday usage.


Can anyone explain the following trend to me? I've been seeing it alot lately: On the install page [0] the installation instructions tell people to run this command: curl https://static.rust-lang.org/rustup.sh | sudo bash This seems to me to be an extremely bad idea. Why would I want to pipe arbitrary commands into my shell? Even worse, the shell has to run with root priveleges? Does anyone actually do this? Am I overthinking it?

Of course, I can verify the contents of the script, but I still don't know that the script I've verified on their website is the script that's being run in my shell. Is it OK because the script is hosted over https, and therefore can't be modified on transmission?

[0]https://crates.io/install


Any time you install a package (an RPM, deb, .exe, etc.), including from something like yum or apt-get, you are trusting somebody with root access to your system, including the ability to run arbitrary shell commands.

This is something to take seriously; but not something to dismiss as entirely unacceptable. It is generally a manageable, and somewhat quantifiable, risk. This is no different than if the Rust developers provided an RPM or deb package for you to install; the same privileges are being granted to them. Though it is the same thing, it doesn't seem to generate the same concern as the "pipe this file into a shell" method of installation. In fact, one of the folks who responded to you explained that this problem will be gone once rust is packaged for distribution...but that merely means the commands are different, not that rust developers aren't being granted root access to your system.

It is possibly even more suspect, in the case of binary packages, since in the case of the shell script, you could download it, look at it, and then run it. A binary package might have modifications that aren't included in the source distribution, and it would be difficult for a non-expert to spot those differences (and an expert would need to go looking for them, and might miss them).

This is why signed packages are a big deal in the free OS world. At least you know who signed off on your package.

In short, you're right: It's kind of crazy that people grant root access to arbitrary people every time they install software. But, it's the way things are done, for now, so we deal with it and only install stuff from people we perceive to be trustworthy. (i.e. be careful what apt and yum repositories you enable on your systems, and what scripts you pipe into a sudo shell.)


All of that is true. But when I pull a package on Ubuntu or Fedora, it's checked against a key that came with the the system at install time as part of the whole package authentication infrastructure. The root of trust is rather stronger than running the output of some random unencrypted HTTP URL. And the distros recognize that and publish hash values for their install media in lots of obvious places.

This is just a bad habit from the OS X world that needs to die. Really there should be some part of the step that allows a typical user to at least try to manually authenticate the root of trust, via a published hash on a bootstrap package maybe (that's what third party RPM/deb archives do, for example).


it is a https url, the same you use to move funds in your bank... If they provided a rpm/deb package it either would not be signed or you would have to download the public key and the package over https.

In the end is the same level of security, this one at least you can first download the script, read it and then pipe it to bash much more easily than a rpm/deb.


TLS encrypts the transfer. It... sort of authenticates the site (modulo multiple holes that exist in that scheme).

But that just pushes the problem on the user: now the user needs to know affirmatively that "rust-lang.org" is the correct source. And they need to know the same thing about every piece of software they install using this insanity.

That's a rather different situation than simply trusting the install-time keys shipped on your system, which you can authenticate manually (via a public key check against a well-published, historically attestable key) with comparative simplicity.


"the install-time keys shipped on your system" -- given Rust's rapid pace of development, I highly doubt that default packages that are provided e.g., by Debian would be relevant.

To get a reasonably fresh version, you would need to use something like Ubuntu's ppa where you need to trust the ppa's author and TLS, to get you signed packages and the corresponding keys.

"keys + signed package" via TLS from a known (via google) site is more secure than "human readable sh script" via TLS but it is not by leaps and bounds.


Oy vey! No! It's not! HTTPS as commonly implemented is well known to be much more compromisable than the approach of using an actual signed system. In particular, the CA system is.... not good (lots of discussion about this exists).

Having the Debian approach is far better here!


I don't see how comparing an install script you get from somebody with a hash you get from that same somebody gains you an appreciable amount of security. If their server is compromised, you can't trust the hash either. Basically, what you need is a source of trust external to the transaction. If that isn't available and you aren't going to manually review everything, I don't see how this is meaningfully different.


It doesn't give you "perfect" authentication (because nothing does, that's the why the whole root of trust issue is so difficult), but to argue that it isn't appreciably different than literally running the output of curl is just silly.

If I don't trust the hash value I got from the Fedora web site I can ask Google. I can check it vs. the mirrors. I can ask my friend who installed it already to see if it matches her file. I can check the signature on the hash to make sure it matches the key I have stored from my existing install (or the one before that, etc...). That's not perfect but at least it's something.


Yes, but I'm talking about the case where you do trust the hash value just like people who use this method trust rust-lang.org. If you don't trust this software, you could download it and get a hash from somebody you trust as well — in neither case are you forced to do it the easier and less secure way. Obviously a quick install process where you trust the source will be less secure than a laborious process where you carefully inspect the code or obtain and verify multiple hashes via sneakernet. But I'm not convinced that the "secure" easy path where you get a file from somebody and a hash/key from the same source, see that the two match and then run the file is actually more secure in any meaningful way than piping curl into sh. The difference seems to be more in the user's level of caution — this method just lays what you're doing bare rather than burying it under layers of indirection.


But... you can't get a hash from somebody you trust with this scheme. The script downloads and runs and throws itself away with no attempt made to authenticate anything (well, I guess there's the TLS cert as it's a https url -- that's something at least).

Or if it's there somewhere I don't see it. Where it it?


The point is that if you're paranoid, you won't pipe curl into sh just like you won't blindly trust an unauthenticated hash. There is nothing stopping you from putting more effort into installing this software just like there's nothing stopping you from putting more effort into installing an apt package of questionable provenance.


you can also read the bash script before running it... and it is infinitely more secure than just trusting a hash as your friend and the mirrors can be compromised without you being able to notice (and it has happen before).


Er, huh? You're mixing two arguments here. You're talking about the difficulty of authenticating the root of trust (very hard) and trying to use it to address the issue of installing authenticated software (a long-solved problem).

I'm aware of no situations where linux distro mirrors have been compromised with code that will install on live systems. Again, the key is set up on the clients at install time, you can't drop compromised software on them via package install without stealing the signing keys.

The whole point here is that the "run the shell script off hte web" skips the whole root-of-trust issue, and suffers for it. It must stop, guys.


It's completely wrong to equate the security of installing software via curl+pipe to installing signed packages. There are certainly security holes in first-party signed package installation but they pale in comparison the attack surface of curl+pipe.

Systems need to be made more secure and curl+pipe is a step in the wrong direction. It's kind of shocking that attacks are becoming more sophisticated and prevalent but security is becoming more complacent.

> only install stuff from people we perceive to be trustworthy.

Except you can't authenticate the script with curl+pipe, so if someone compromised the server or the communication link it's game over.


I don't believe it is "completely wrong", and I discussed the reason you should prefer signed packages from your OS vendor, though that wasn't my focus.

If you install packages from a third party, and if they are signed, you are generally getting the signing key from the same location. i.e. when the rust developers build packages (as has been suggested will be the solution to this problem), you'll also need to install their signing key. Presumably from the same webserver that is serving you the script that is being judged insecure in this thread.

"Except you can't authenticate the script with curl+pipe, so if someone compromised the server or the communication link it's game over."

s/package and signing key/script with curl+pipe/

The two are not dramatically different if an attacker has taken over the website in question, or if the build/distribution infrastructure has otherwise been compromised, and that was my point.

You could do an sha checksum of the script, and you could obtain the package signing key by meeting up with the developer in person and checking their ID, and you could download the sources, check them thoroughly, and build your own packages, etc., etc. Very few people do that.

There are levels of risk at every stage. It is a difficult problem to distribute software in a safe fashion.

All of that said: I install everything from a package. If it is not from the OS standard repository, I download the source package and build it myself (sometimes checking the source tarball for anomalies and checking the patches to be sure nothing is obviously amiss). I do not enable third party repositories unless I trust them a lot. If no package is available, I make one, and sign it with my own key. If I can't make one, I don't use the software in a deployment (the cost of maintaining a piece of software built from source is way too high).

I was not suggesting the curl+pipe is a good practice. Merely that we live in a world full of compromises. And, insuring you are getting what you think you're getting, and that the person who packaged it had no malicious intent, is a lot harder than merely installing from a package (particularly a package from a source that doesn't have stringent identity requirements...Debian and Fedora packages have a pretty clear ownership trail...most third party packages do not).


Signing keys can easily be distributed out of channel and also verified through a chain of trust. Even if you got a forged key you could, at minimum, detect post-facto that the key was fake.

Moreover, there are things you can build on top of signed packages (distributed consensus of signatures and keys) that can't be built on top of curl+pipe.

curl+pipe is mental.


You're right, and I agree with you. Packages that are signed by a trusted party are a reasonable choice; anything less is not reasonable.

But, lots of people add third party software repositories without out-of-channel confirmation of signing keys, and without any awareness of the risks involved in that. I see it more than most (I support server-side software products with a million or so installations), and I'm amazed at how often third party repos are enabled on people's systems, seemingly without any reason other than "more software is better". They often don't even know why/when they enabled those other repos or who operates them.

That's no different than this curl+pipe scenario.


Yeah third party repos are pretty unreasonable. The problem is that software development now moves pretty fast and package maintainers can't or don't want to keep up. That, coupled with the fact that building packages is far too difficult for most distros incentivizes people to take risks that they shouldn't be taking.

Systems need to be safe by default and they should largely service the needs of users. Distros are no longer doing that.


>They often don't even know why/when they enabled those other repos or who operates them.

Most likely because they either blindly followed instructions from a website or blindly ran an install script from a website, giving it root access. In which case it is basically equivalent to curl + pipe.


This is purely because Rust and Cargo both are not yet stable, and so packaging them for distributions is inappropriate. Once there's a stable release, we will be telling people to use their packaging system and/or official installers rather than the script.

There are various issues filed in the repository if you want to track when this changes.


It's bad, but no different than installing a package supplied by a third party, or using a third-party installer, in terms of what authority you're granting, both of which are fairly commonplace. Obviously the eventual goal is to get something vetted and blessed into a package management system, but that's probably not a reasonable expectation for a brand new project.


Do you really think that's a trend because I've been seeing people complain about it on Hacker News for 3 or 4 years now.


Maybe so, but I've only just started seeing it within the last few weeks. I've been on HN for less than a year so maybe thats why.


Does curl return a partial output if a connection dies?

What if it dies and the last thing it got was "rm - fr /" but not the rest after the / ?


Most shell scripts used like this are implemented by defining a function, then calling it - so if it fails downloading, all that happens is a syntax error as the code never actually gets run.


That's a good point. I filed https://github.com/rust-lang/rust/issues/19168 to get it fixed.


Never thought about this, interesting question.

You can try this equivalent, killing the server when it tells you to - https://gist.github.com/aidanhs/e40417381e9aecb87b35. It prints '/', i.e. would remove your root directory if it was an `rm` command.

Won't be doing that again then.


Remember that these days, with GNU rm, you have to pass in `--no-preserve-root` to actually `rm -rf /`. Attempting to remove something in the home directory and getting a `rm -rf /home/username/` would still be pretty bad.


The chance that the connection dies at exactly this moment is negligible compared to the chance that your hard drive dies. Worry about that.


I used to be an http link some time ago...

What alternative do you propose? You can check the script whether it does something malicious, but then it downloads and installs some binaries.


Some good points from Nate Lawson: https://news.ycombinator.com/item?id=2427492



Not only that, but the script is useless.

All it does is bullshit of 100 lines or so, but the main part of what it does is

download a tarball, make some temporary directory, and run install.sh

How difficult would those instructions be? Here take a tarball, and you know, unpack it and run install.sh

I guess, too difficult for the audience that Rust developers are targeting with this curl this | sudo bash bullshit.


I don't disagree that that seems simple enough a process to follow, but bear in mind this command is supposed to be run daily. Convenience is definitely a boon.


Announcement and details are here: http://blog.rust-lang.org/2014/11/20/Cargo.html


I'm a bit disappointed that this whole site works only with JavaScript enabled. No graceful degrading or anything. With disabled JavaScript, the page is just empty.


I was happy to discover that the backend is written in Rust as well. Here's the code for anyone who's curious: https://github.com/rust-lang/crates.io/tree/master/src


At first I saw that ember-cli was being used and wondered if the backend was done in JS. I came to the same realization as you.


Nice to see them using PostgreSQL too.


Great work. Only one improvement I would like to see 1) Switch the web design away from javascript (at least for loading) or, 2) Implement some command line interface in cargo for searching crates and displaying it's description.


Command-line search would be cool, for sure. Submitted! https://github.com/rust-lang/cargo/issues/925


What about requiring JavaScript even for the bare minimum of functionality? (That is, even loading the main page.)


I would agree with adding some <noscript> tags to at least print a message telling you to turn it back on, sure. Filed: https://github.com/rust-lang/cargo/issues/931


Am I correct in assuming that this is focused only on source distributions right now?


That's correct. Rust doesn't have a stable ABI, so you'd have to have the same exact compiler revision as the one it was built with for it to work. And that's just the first issue...


It's very impressive. I am concerned that it requires enabling scripts from google.com to even show any content. Mozilla has shown it actually cares about user privacy so I hope they address it.


I already spoke with someone who is sending a PR to fix this, so hopefully that will not be true soon.


That's great. To be honest I really didn't expect a response for such a pedantic complaint. Most people accept a world where Google touches everything with its JavaScript and font CDNs etc. The site looks great and I'm really enjoying working my way through the Rust Guide at the moment.


I am, uh, very excited for this launch. :) (and glad you're liking the Guide too, please open issues if it can be improved)


For me the ability to use packages made by other people is a draw card for a language.

However, when I first went to the site, all I could think was "Whats a crate in relation to programming or more specifically Rust? Is it Rust's idea of a package (eg like ruby gems)."

I just think it would be good to have the site say Cargo: The Rust language package manager.

The site also says this:

"The Rust community's crate host". Is it not a cargo host for distribution of crates where crates are Rust packages? Is there non-community crate hosts?


Basically, crates are packages, yes. It might be good to add some non-jargon-y copy there too, yeah. :)


If you aren't cool with me using your code, I respect that. I don't like forcing people to give up anything including their rights to their own repo.

I just don't feel comfortable working on someone's code if they aren't willingly consenting to it. just not cool at all! Human consent is just not a permanent thing.

Besides if the file disappears from the remote source, you can still build for the time being. It's not the end of your project, but it may be the end of someone else's career - you never know what reasons people had. Just respect peoples rights over their data.

The constant argument, "it's already on the internet" doesn't absolve any of us, from respecting peoples reservations. And if it's "already on the internet" then you can trust that someone will illegally copy and repost the repo on-line but where does that leave us ethically?

not the way I roll.


That seems like a very extreme view of intellectual property. "Intellectual property rights should be so strong that you can retroactively censor data you made freely available."


The Right To Have My Work Forgotten. Yeah, I don't think Right To Be Forgotten makes any sense either.


Does cargo only support packages with free licenses?


I'm really looking forward to publishing some early crates! A cargo search from the terminal would be ace as well.


You're going to want to subscribe to https://github.com/rust-lang/cargo/issues/925 then :)


Second most downloaded: brainfuck_macros


A compiler plugin that converts brainfuck code into Rust at compile time, letting your BF programs be optimised by LLVM to super-fast native code.


Hooray! This has been eagerly awaited in the rust community for a while. Thanks, Steve!


Basically zero of the thanks should go to me. Alex Crichton has been hard at work on this for the past while, both Cargo itself as well as the site.


Ah, then: Thanks, Alex!

But fortunately "thanks" aren't zero-sum, so I'll leave your ill-gotten gains alone. :p


I'd love to see categories added to crates.io kind of like ruby-toolbox.com. Categories as well as repository activity information is helpful for discovering libraries that will suit your needs. Since there are so many ruby gems, it is now hard to properly categorize gems, and ruby-toolbox doesn't do the best job of this anymore. I found it to be very helpful years ago. Ruby-toolbox is still helpful in that it can show some similar projects and show project activity to help you see which projects have more momentum.


If you look at each project's page, for example, https://crates.io/crates/gl_generator was the first when I loaded up crates.io, on the right there's a "Keywords" section.


Fantastic. Not sure how I missed that.

Edit: There are plenty of crates not categorized, and that must be how I didn't see this feature.


But how does one avoid having crates.io being flooded by bad/useless crates?

And couldn't the same crates end up occupying all the "good" crate names?


In theory, this could happen. In practice, Ruby/Python/JavaScript seem to be doing just fine.


Have you guys given any thought to namespacing packages by developer, like Github. I've had to rename a couple of my own Python packages before uploading them to the package index to avoid naming collisions, so it definitely becomes an issue once the repository hits a certain critical mass.

I assume Rust will be around for a while (I certainly hope so - I've just started learning it!) so it might be wise to plan for growth now before it's too late to implement.


It creates a significant amount of complication. When I'm writing a blog post or documentation or in conversation, how do I differentiate between your foo and my foo? When I 'extern crate foo' in my source, which foo should it use?

While it's true that sometimes collisions happen, in practice, you just choose a new name. Everything is significantly simpler, which in software is very often a Good Thing.


I can appreciate that it adds complication to the implementation. I don't think identifying a particular crate would be an issue though, any more than identifying a particular repository on Github is. It's easy enough to talk about jdoe/foo if there's any danger of ambiguity, and links are definitive by their nature.

Just from my own experience, I know that it makes me less likely to make packages public if I have to give them odd, collision-avoiding names first - especially as I then have to use those names myself. It's particularly annoying when the incumbent package is 10 lines of poorly-written abandonware that hasn't been touched in years, which often seems to be the case.

I know this isn't an issue for Rust now but I'm thinking about ten years down the road when you have 250,000 crates on the site, 99.9% of them junk, clogging up that flat namespace. (Python's repository has more than 50,000 packages and seems to be growing at an accelerating rate now that it finally ships with a decent package installer by default.)


> I don't think identifying a particular crate would be an issue though, any more than identifying a particular repository on Github is.

Nothing is impossible, of course, but the language already has a flat crate namespace. So you could either implement some kind of layer on top of that, or change the language itself to have some sort of crate hierarchy somehow. All just to have two crates named 'http' rather than one named 'http' and one named 'requests' (for example.)

Here's another way of looking at it: shipping a flat namespace is a proven, well-worn solution. Adding in a hierarchical one, on top of the language, which has a flat one, provides dubious benefit, for significant implementation complexity.


Meh, if I wanted a 'proven, well-worn solution' I'd use C++. I want you guys to sprinkle fairy dust on my code and make the experience magical ;)

Seriously, I do get that there are trade-offs. I'm just interested in how you've been weighing up the alternatives as I do often find myself wishing that other projects I've been involved in had taken the hierarchical approach at the beginning when it was an option.


Kudos for choosing simplicity where possible.

Developers who really like a name could always add something such as using "moz-foo" instead of just "foo".

I've been surprised with how few name collision problems I've had on Python's Pypi. If "z-foo" module is more usable than an unmaintained "foo" module, everybody recommends and uses "z-foo" without a second thought.


Node has a process for dealing with name disputes.

https://www.npmjs.org/doc/misc/npm-disputes.html

Not an urgent need, but a similar process will need to be implemented.

Naming conventions could be nice too.

http://guides.rubygems.org/name-your-gem/


Yup, you're absolutely right. There will be processes like these eventually.

Naming conventions are currently "the same as Rust identifiers".


Hmm. If you say so.

I'm somewhat uncomfortable with the idea, but I'll gladly defer to people with more experience on this matter than me.

Besides, the only alternative I can come up with, is some form of procedure for voting out crates which doesn't play nice with the community...


Does cargo support alternative package repositories? For example, at $EMPLOYER we have a DevPI[1] installation, so company-internal packages can be shared between employees as easily as public packages can be shared with the Internet at large.

[1]: http://doc.devpi.net/latest/


Yes, you could grab all this code off of GitHub, spin up your own instance, and then set up your `.cargo/config` file to point at your copy, and it Just Works.


I really wish it would also allow `~/.config/cargo/config` just like `~/.config/git/config`.


I believe there is an open ticket to follow the XDEG specification or whatever. You may want to find and :+1: that.


This will certainly make our lives easier.

BUT.

1) Why store a package, wouldn't an index rather than another repository be more efficient and flexible.

Cargo already has the capability to get packages from git urls...

I had some other nicks but I see this is on github so I'll try and contribute there.

I like the site, and can't wait to test it out. Thanks to all who have been working hard on this!


> 1) Why store a package, wouldn't an index rather than another repository be more efficient and flexible.

I suppose the authors experienced the "guy deleted his Github repo" effect one time too many. It's a very sensible decision. Getting packages for git urls is fine when you're hacking something together, but there is absolutely no way I'd put anything in production relying on something that fragile.


I think it's even worse depending on a repo whose author refuses to or is unwilling to maintain.

Besides git is hardly fragile especially if we are talking github. I happily clone and pull projects all the time without a single hitch. Also if you think about it github/attlassian are all dedicated to task with datacenters around the globe, while for the rust team the repo might be somewhat of a side gig.


I'm not so worried about Github going down as I am about people removing their repos.

And why do you think depending on a repo is worse than depending on Github? If the package is not maintained anymore, license willing, you can always fork it and bump the dependencies yourself, even if, in the worst of cases, you need to keep the fork local. However, if it turns out you have to maintain a lot of third-party libraries yourself, it's a sign either of a bad ecosystem, or that you picked the wrong libraries.


I guess at the end of the day so long as cargo has the git option people like me will always get our code fresh from the source, with the maintainers consent. It's not like I'm being forced to use the crates-repo.

Let's agree on one thing we shouldn't be maintaining other peoples abandoned/illegal libs, however thanks to them being posted on the crates-repo permanently, we'll continuously keep running into them...


> Let's agree on one thing we shouldn't be maintaining other peoples abandoned/illegal libs, however thanks to them being posted on the crates-repo permanently, we'll continuously keep running into them...

Maybe. But at least, if you pick your or somebody else's unmaintained code from five years ago, you have a fighting chance of making it compile and run.


Maybe it's just the very beginning but I would really like to see detailed descriptions for every crate as well as links to the website or repository if available. It's just hard to understand what they do, how they do it and their quality.


Yesterday, we implemented a warning if your description and/or license aren't filled in when you upload a package. And 'homepage' is also possible too, authors just have to put them in.


small nitpick: 'homepage', not 'website'


whoops, edited, thanks


Cool. It would be nice if there was a link to each package's web site or source code.


If authors give us that metadata, it appears on the site.


if the author provides it, it is below the 'stats' portion, though there is an open issue to move this section above the stats: https://github.com/rust-lang/crates.io/issues/35


FWIW, the page is a bit messed up in IE11 on Windows Phone: http://1drv.ms/14VI93Z

Awesome work!



Sweet, let's build an OS based on Rust code now!

Why don't we port something that's already got an eye for security, like Minix?


There's already a few projects doing so. Drop by #rust-osdev on Mozilla's IRC network sometime. :)


Why is cargo not bundled with my rust install?


rustup.sh installs both.

For other install methods see: https://github.com/rust-lang/rust/issues/16456


Congrats on shipping, cargo team!




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: