
The New AWS Command-Line Interface - jeffbarr
http://aws.typepad.com/aws/2013/09/new-aws-command-line-interface-cli.html
======
sivers
FWIW, I've been using the aws command-line tool by Tim Kay for over 6 years.
It's only Perl and cURL. A single self-contained script with no other
dependencies. It's been rock solid in production that whole time:

[http://timkay.com/aws/](http://timkay.com/aws/)

The syntax looks quite the same, but Amazon's awscli Python installer has
loads of dependencies. I'll have to see if it's worth switching.

Anyone already know if Amazon's new CLI thing has any big advantages over Tim
Kay's Perl aws?

~~~
gtaylor
> The syntax looks quite the same, but Amazon's awscli Python installer has
> loads of dependencies. I'll have to see if it's worth switching.

Why are the dependencies a problem? By combining a handful of smaller, focused
modules that each do something well, you can end up with something better than
if you were to re-invent the wheel for every need.

AWS and the Python dev team are doing a heck of a job on botocore, and have
cranked up the pace of improvement in the last 6 months. This CLI reaching
"official" status will guarantee (at least until further notice) that it will
see updates and fixes. It's likely to see early or earlier support for new AWS
services.

~~~
sivers
I just like knowing what's on my system.

`pip install awscli` just installed 26 other modules besides awscli. Now I
feel a little obliged to go check out those 26, as well, to see what they are.

I agree about not re-inventing the wheel. But the amount of stuff installed is
definitely a considering factor when choosing between two seemingly identical
scripts.

~~~
the_mitsuhiko
> `pip install awscli` just installed 26 other modules besides awscli. Now I
> feel a little obliged to go check out those 26, as well, to see what they
> are.

So? Use a virtualenv and stop worrying. These 26 dependencies will be
separately updated and maintained, who knows what warts are sitting in the
monolithic perl scripts.

~~~
garnaat
With boto, I battled for years trying to avoid any dependencies. But that has
a lot of negative side effects, too. One of the great things about Python is
the amazing variety and quality of libraries available. We decided to embrace
that with AWS CLI. We have 10 direct dependencies. Four of those are our own
packages that we decided to split to allow maximum reuse. Then there are
fundamental things like requests, six, docutils. The rest are things that, we
think, improve the experience. Virtualenv is an awesome way to manage this. I
highly recommend it.

------
mafro
This latest release marks a milestone in the transition from the old Java
based tools, to the new Python ones.

Mitch Garnaat[1] who built and maintained boto over the years was picked up by
Amazon last year and since has been building out botocore[2] - which the aws-
cli[3] tools use under the hood.

[1] [https://github.com/garnaat](https://github.com/garnaat) [2]
[https://github.com/boto/botocore](https://github.com/boto/botocore) [3]
[https://github.com/aws/aws-cli](https://github.com/aws/aws-cli)

~~~
jevinskie
Thank you for this no-nonsense layout of the situation. =) Can you elaborate
more on how this FOSS Github-er was picked up by Amazon? It sounds like a
story that many here would like to replicate!

~~~
mafro
I'm afraid I don't actually know anything more than the intertubes can tell
you :)

I noticed it had happened sometime last year, and thought I'd share what
little I know.

------
kondro
Whilst this is a much better solution that the existing slow tools Amazon
provided, I don't understand why this is being reported as if command-line
tools for AWS are a new thing.

Command-line tools used to be your only option for managing AWS and Amazon
always create their API and shell tools before the Console.

~~~
dmourati
Yea, not a new thing, even the command names are mostly the same.

$ ec2-describe-instances

$ aws ec2 describe-instances …

Pretty strange decision IMO.

~~~
vacri
It's unifying the commands. Just this morning I've been struggling with ascli
(autoscaling cli), which has its own method for listing AWS keys that don't
match the way other tools do it.

~~~
dmourati
Meh, 90% of what I do is ec2 anyway.

I did have to break out: cli53 rrcreate.

------
vacri
Interesting - I must have had an older version and not played with it. The
version I had did not have 's3 sync', and the help files were a long list of
ec2 commands and some boilerplate headings.

Reinstalled and now they have shorter, saner commands and real help pages...
and they also changed the pager from 'less' (my default pager) to the crappy
'more'... because (apparently) you never want to go back a page in a help
file. More detailed content, but it's harder to review. Odd.

~~~
jevinskie
Amazon doesn't use $PAGER? [0]

[0]:
[https://en.wikibooks.org/wiki/Guide_to_Unix/Environment_Vari...](https://en.wikibooks.org/wiki/Guide_to_Unix/Environment_Variables#PAGER)

~~~
jessaustin
Well, GP isn't just imagining things. The pager for this is at most the
equivalent of crap-ass "more" right out of the box, on systems where all other
cli executables have better pagers.

    
    
        $ PAGER=/usr/bin/less aws help
    

...does seem to do the right thing, but why does this program require it when
no other program does?

------
pkj
Just tried it out and it is excellent ! The integration of various aws
services is really neat. The help and correction-suggestion are nice too.

Great to hear that this is built on the awesome boto library. Will serve as an
useful reference for boto developers.

------
snagage
I think the only thing "new" here is aws-cli hitting version 1.0.0. It's been
the recommended aws command line tool for quite a while now. I've personally
used it for the last 6 months.

------
dencold
Long-time user of boto[1] here. It has been the go to library to hook your
python code into AWS and has a fairly active following on github[2].

One API point that I've found lacking in boto is a "sync" command for S3. Take
a source directory and a target bucket and push up the differences ala rsync,
that's the dream. Boto gives you a the ability to push/get S3 resources, but
I've had to write my own sync logic.

So, the first thing I went digging into is the S3 interface of the new CLI,
and to my surprise, they've put a direct sync command on the interface[3],
huzzah! Their implementation is a little wacky though. Instead of using some
computed hashes, they are relying on a combination of file modtimes and
filesize. Weird.

Anyways, glad to see AWS is investing in a consistent interface to make
managing their services easier.

[1]
[http://boto.readthedocs.org/en/latest/index.html](http://boto.readthedocs.org/en/latest/index.html)

[2] [https://github.com/boto/boto](https://github.com/boto/boto)

[3]
[http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html...](http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html#examples)

~~~
aidos
That is good news. I too wrote a sync layer to sit above boto for a previous
project. My use-case is a little different in that I sync from S3 to
RackspaceCloud as a backup. I just use file name (object name) as the key
because I know that files never change (though are added and removed). I
create a complete object listing of S3 and a complete object listing of CF,
diff and then sync.

One disappointing issue is that the listing process on CF is a magnitude
faster than S3.

    
    
        CF: real	2m7.628s
        S3: real	14m15.680s
    

Keep in mind that this is all being run from an EC2 box, so really, S3 should
win hands down.

------
craigsmitham
Azure has had a node.js CLI for awhile and it's awesome. It's advertised for
Mac/Linux, but I use it on Windows all the time.

[http://www.windowsazure.com/en-us/develop/nodejs/how-to-
guid...](http://www.windowsazure.com/en-us/develop/nodejs/how-to-
guides/command-line-tools/)

------
oblio
Well, a rant:

* Why does each one of them use different parameters for the same stuff? WHATEVER_URL could be REGION (WHATEVER = EC2, ELB, ETC, ...). One uses a config file for ACCESS_KEY_ID, another one wants a environment variable. Plus they use different names for common stuff.

* Why do are the command line arguments named inconsistently across these tools? --whatever, --what-ever

* Why don't they fail on command error? Right now this only happens if there's a configuration problem - if you send a command and there's a problem (like S3 access denied), it still returns 0.

* Why don't they provide synchronous commands? Right now I have to do the polling myself. Super annoying.

Anyway, I've been using the ones included in Amazon Linux - I hope they were
the latest version. If the new version fixes this problem, feel free to
correct me :)

------
unono
About time.

More startups should realize that they could increase developer adoption of
their products if also published shell script interfaces to their product. In
fact, your startup should really start off as command line accessible and add
the gui after.

~~~
jeffbarr
There have been CLI tools for some parts of AWS since the beginning. This
release replaces a set of disparate tools (each with their own installation
instructions and configuration issues) with a unified set.

> In fact, your startup should really start off as command line accessible...

I would argue that startups should always start at the API level and work
upwards. At least that's what we did with AWS.

~~~
MichaelGG
Starting off from the API seems like the only sensible way to approach things.
It exposes core design issues early on, and if done right, eliminates a wide
source of security issues that might be introduced by, say, the front-end
developers.

------
hendry
aws-cli/0.16.0 gripes:

• __slow __

• crap ton of dependencies, making it a pain to install on
Arch[https://aur.archlinux.org/packages/aws-
cli/](https://aur.archlinux.org/packages/aws-cli/) Hint: static build ffs

• broken help. `aws s3 help` instead of `aws help s3` which is more natural to
git users

• no glacier support

• no s3 progress for downloads or uploads

• many jarring UX issues [https://github.com/aws/aws-
cli/issues/305](https://github.com/aws/aws-cli/issues/305)
[https://github.com/aws/aws-cli/issues/304](https://github.com/aws/aws-
cli/issues/304)

~~~
rlpb
> • no glacier support

I'm the author of glacier-cli (github.com/basak/glacier-cli). I'd be happy to
see it move into aws-cli. If anyone wants to do this, please get in touch to
coordinate.

------
sparkzilla
As a semi-technical small business user of AWS I really wonder why so few
resources seem to be put to the AWS GUI. For example, why is autoscaling not
part of the GUI/AWS console? It's one of the most basic functions and it
beggars belief that in this day and age it is done through the command line
only. I'd rather not have to pay extra for Ylastic or some other hack when I'm
sure Amazon can throw together even the most basic UI in a few weeks.

~~~
imperialWicket
Use Asgard from NetflixOSS
([https://github.com/Netflix/asgard](https://github.com/Netflix/asgard)).
Shameless project plug: for anyone using Hubot, easily get Asgard up and
running, then interface with Asgard via Hubot
([https://github.com/imperialwicket/hubot-
asgard](https://github.com/imperialwicket/hubot-asgard)).

* EDIT: that is, use Asgard because it does a great job of managing autoscaling groups.

------
727374
The new CLI looks like a massive improvement over the old one, but ironically
I will probably still prefer boto + ipython because of the very robust
autocompletion.

~~~
xorgar831
I wonder if the fish shell could autocomplete the commands; it has a feature
to generate autocompleation settings from man pages.

------
djrobstep
If anybody is after similar s3 functionality within python, I wrote a small
python wrapper around boto a while back that does similar parallel/multipart
upload to s3, and a bunch of other AWS stuff:
[https://github.com/djrobstep/motorboto](https://github.com/djrobstep/motorboto)

------
kolev
I've been using a combination of aws-cli and the Java-based tools (as some
features are unsupported by aws-cli yet). I've been patiently waiting for
CloudFront support - no luck yet! Also, I'm switching my new projects to use
botocore, but it lacks any documentation and it's been really painful!

------
js-py-pl
Interesting, I've been using CloudSigma's built-in tool for quite some time.
You just hit ' in their webapp :) See
[https://autodetect.cloudsigma.com/ui/#/trynow](https://autodetect.cloudsigma.com/ui/#/trynow)

------
akurilin
I've been using s3cmd for a while now. Am I correctly seeing that the new CLI
tools, or at least the s3 portion, are pretty much a replacement for that?

~~~
crazysim
First-party parallel upload and splitting support immediately trumps s3cmd for
me.

------
throwaway9848
Okay.. glad to see the effort, but I've been using boto and euca2ools for
years. This will have to be good for people to switch.

~~~
nailer
Same author. boto dude (garn something, he's posting here but I'm on my phone)
wrote boto and wrote these.

~~~
throwaway9848
Thanks!

------
zeckalpha
This makes a project I was almost done with much easier. Start over, or finish
with the way I was previously doing it?

------
ausjke
Can I use this for Glacier parallel uploading?

------
simonebrunozzi
Well done!

