

Crate.io - an improv(ed|ing) Python Package index - jnoller
http://crate.io/

======
po
Wow, bootstrap really does allow for a much nicer looking front page doesn't
it? This seems like a great step forward. I was recently looking to set up a
local chishop server to act as a mirror.. it seems like the cacheable url's
thing means you're thinking to enable people to set this up locally and have
it pass requests that 404 through to the crate.io server? I would love to do
that. Is that workable now?

My one suggestion would be to make a simpler to type url than
'simple.crate.io' since people may have to type it in on the command line from
time to time. Perhaps '<http://crate.io/s/>?

~~~
donaldstufft
This is a good idea! I used the subdomain because I plan on hosting the
"simple" interface in EC2 (Packages are hosted on S3/CloudFront) in multiple
availability zones while my main site is hosted on Gondor.io.

However I could enable a short url that redirected to simple.crate.io to allow
easier command line access.

Thanks!

~~~
po
Is the expectation that someone should be able to do what I described? Run a
local copy that proxies 404's up to the main crate.io server and then caches
the result locally to disk?

This would be a very nice way to deal with build-time dependencies. You can
get similar behavior today by setting multiple mirrors in pip but this would
allow the grabbing packages on-demand instead of having to mirror everything.

~~~
donaldstufft
pip supports a download cache (off by default) that stores packages as their
urlname. It works similar to how you describe. It will see it needs the
package at url xyz.com/package.tar.gz, it'll check it's download cache and if
it doesn't exist there it will download the file from the url and save it to
the cache.

The cacheable urls stem from the fact that currently on PyPI you can delete an
existing file you've uploaded, and reupload a file of the same name. Pip (and
any other cache like varnish etc) isn't notified of the fact that what used to
be at url xyz.com/packages.tar.gz is gone and now something else is at
xyz.com/packages.tar.gz. This can cause issues with installing packages,
especially on remote deployment infrastructures. With crate I place the sha256
of the file in the url, so if someone deletes + uploads the same name, it
should still (unless you get (un)lucky and get a collision somehow) have a
completely different url, so pip and other caches will see it as a different
file.

As a side effect it would allow a simple caching proxy to be installed at
someones office, or wherever that easily does the behavior you describe as
well. It could function similarly to pip but be shared amongst multiple
people.

------
masklinn
The name bothers me: the Rust team has chosen "crates" to designate Rust
libraries (technically compilation and distribution units for Rust, crates can
yield libraries or binaries): [http://doc.rust-lang.org/doc/rust.html#crates-
and-source-fil...](http://doc.rust-lang.org/doc/rust.html#crates-and-source-
files), and I don't remember ever seeing this kind of lingo around Python
modules and distribution units.

~~~
donaldstufft
Purely coincidence. To be honest I've never seen Rust before, when I was
trying to name it I just hit a thesaurus up until I came up with a word that
had something to do with "packaging" (Shipping Crates!) that was available
that I liked.

~~~
masklinn
> Purely coincidence.

I'm sure it is, and I didn't intend to suggest malfeasance, I'm sorry if it
came out wrong.

