Hacker News new | comments | ask | show | jobs | submit login
AWS documentation is now open source and on GitHub (amazon.com)
243 points by sassyboy 10 months ago | hide | past | web | favorite | 98 comments

Brilliant. Let unpaid volunteers fix and update the documentation for your proprietary systems so you don't have to pay someone to do it.

Can someone explain to me why people will inevitably put in countless hours of free labor for Amazon here?

Just think of all the uncompensated value you've given Y Combinator every time you've left a quality comment on their forum.

Then again, Y Combinator makes no direct money off HN. AWS’ docs, if they are quality directly impacts their profit because more people can use their services more easily.

YC does two things that aren't directly making money but are definitely helping their portfolio companies:

1) Special ad positions with front-page placement (currently #22 is "Willing (YC S15) Is Hiring a UI / UX Designer" https://willing.com/designer , a privileged post that can't be voted or commented on)

2) The moderators can and sometimes do intervene by removing certain comments that are critical of YC companies.

Comments make the forum more interesting to the community, which increases eyeballs and makes the front page spots more valuable. And shaping the discourse helps with damage control when necessary.

..and collect your personal data, non-directly-identifiable data or otherwise, from both of which they may earn some money (if i understand the privacy policy correctly.. am no lawyer, though)

Nothing is free. If you aren't paying, you're the product.

Actually, quite a lot is free. I have a free tier in my saas product. I don't monetize it in any way. I simply use it as a gateway to paying customers. I think there is probably a great deal of this going on.

Personally, I'm surprised that's a controversial statement.

Its not on this forum. Its so common as to be a cliche.

It's eye-roll inducing at this point.

That's probably true but it seems so easy for people to forget.

Profit, influence, power. They can be exchanged for each other.

YC doesn't make money directly from the forum. But neither does AWS from their docs.

If you have a problem with a company making money directly from your content, why doesn't it bother you if they are gaining something else off you like influence? Do you think HN confers zero value to YC?

AWS does makes no direct money off documentations either.

Engineers generally have a natural tendancy to fix things, and documentaitons are worth fixing. And it also helps their future work.

I think there’s a reasonable argument that documentation is an integral part of usable software.

Not only is there a reasonable argument, a major chunk of Fred Brooks' "The Mythical Man Month" is about documentation being the entire spec and that software should not do anything that isn't in its actual documentation. Write the documentation, then hand it off to the developers to implement.

Hacker News is the textbook example of a highly successful content marketing platform for software engineers. Second only to Joel on Software.

Indirect money as in brand/capital is still wealth. You have contributed to that creation of wealth.

I think it’s because some people just like to help things get better. And there is no problem about this.

I tend to feel the same way, but would happily throw in a couple of changes (which when you multiply by thousands of people, makes a lot).

In general, I'm happy supporting a company that pushes the bounds rather than fretting over every little detail. If something doesn't work exactly as documented, I probably found a rather unique case. I'd rather struggle with the problem for an hour and suggest a fix, than have X, Y, or Z company pay someone to spend countless hours finding every possible edge case.


On top of that, there is a hugely beneficial conversation that often arises from the opening of issues. I've found that one of the best ways to engage in a meaningful discussion with a company is to open an issue on one of their code repos. Engineers are often more straight to the point than "marketing" or "business" types.

I've noticed documentation bugs and submitted them to amazon before they open sourced this. Overall for documentation I've just fixed things as I find them and I've never seen it as much of a burden and it helps the community

Same here. This just kinda standardizes the process in a way that makes edits super clear...

Microsoft do this and seems to make it a cooperative effort. They do have people working on it full time and would be within their rights to NOT accept contributions from the community. But they do, and those contributions are vetted by the full time staff.

As long as this is in the spirit of collaboration and not a "source code dump" that a lot of commercially-backed projects do to earn an "open source" moniker ... I'm fine with it.

Well as a user of Amazon you have a vested interest in the documentation being correct. It is what other members of your team and organization will be referring to.

Having it on GitHub will also add additional context around changes in the form of PRs, Issues and the commit history.

> Well as a user of Amazon you have a vested interest in the documentation being correct.

As a Walmart customer it's in my best interest that the aisles are clean. But I will doubt the legality to ask customers to clean the aisles so they don't need to pay for a cleaning service. Actually using volunteers to replace employees is kind of illegal.

If there was an aisle full of cardboard boxes blocking you from buying a product would you leave the store or take 5 seconds to move the boxes?

What if 90% of Walmart aisles were consistently blocked by aisle-blocking cardboard boxes and Walmart decides the solution is to use social engineering to encourage more customers more often to take 5 seconds to move the boxes?

Then you should shop elsewhere and accept the cost.

On the one hand, I agree with you. On the other; GOD I hope it works because their documentation is loathsome sometimes.

Literally just ran into this. One AWS doc example says to use "update" whereas another doc says to use "updateItem". Which one works? No way to tell without trying. Their documentation is literally costing me productivity at this point.

They're not too bad, but then again MS has lowered my standards so much it's hard to keep perspective.

Heh that's a very cynical view and it speaks to me. But my first thought was that this will make me choose AWS over alternatives because I've found the docs for GCP very confusing and sometimes outdated.

But regarding why people would bother helping. Well I guess it's the same reason some people help out with wikipedia. Not all of them do it for ideological reasons. Some are just very nitpicky and anal.

I don't want to make the same mistake when doing something. It's more of a service to other people than to Amazon (People probably aren't going to choose a different service based on documentation). It also helps to have different viewpoints from levels of experience to help with wording and such.

As a technical writer working on an open-source docs project for another big-tech company, I can safely say that you’re overestimating the size of external contributions. People rarely want to work on docs, especially if they know that someone else is getting paid to do it.

As someone who codes offline a lot, I really appreciate when I can just clone a repo to have a full copy of the documentation.

Do You do it for productivity reasons or you happen to be offline because of some other conditions?

Mostly column A, a little column B as well (sometimes I travel).

I'm a freelancer and when I am between client projects I try to just code without the internet for a few hours first thing in the day.

Keep in mind that this also makes it easier for any Amazon employee to improve documentation. That's 1,000s of developers who are equally confused about AWS APIs and are incentivized to make it better.

Why is it any different than documenting open-source code for free, or answering a stack overflow question?

> documenting open-source code for free

Foundations that own the open-source projects are allowed to use volunteer work by law. Corporations with a for-profit interest doesn't.

> or answering a stack overflow question?

This one is more tricky. But it depends on who asks the question? It is an individual to get help on a problem? Or is it part of the ways of working of a company to avoid hiring experts? Evaluating intention it's usually hard.

Yeah, it's kind of like wondering why someone would make an open source project better when the person that owns the repo gets all the credit. Or all the competitors they might be inadvertently helping out by improving the project.

Sometimes it's nice to just make things better.

With an open source project, I can fork and use the code for myself and contribute back to the project.

But if I spend a bunch of time fixing AWS docs, it's not like I have any need to fork it and use 8t as documentation for my own AWS-like service.

Improving an open source project's docs aren't likely to help you, either.

Intro-level documentation changes are the most common pull requests I get on any of my projects. And the people making them are not the ones being helped by intro-level docs.

It's a clear case of experts helping beginners.

partially because you're presumably paying for the amazon service, but not necessarily with an open source project. one of the ways some people 'pay' for using an OS project is by helping in forums/docs/etc. presumably, the money you're paying for amazon services is/should be going in to their documentation.

also... with an OS project, I can actually get the code and see how it runs, test patches, etc. I can't actually do that with their services, and any docs I might contribute would be guesstimates as to how things actually work, vs how it actually does work (and what's intended), which would/should come from the company that actually owns the code in question.

> Can someone explain to me why people will inevitably put in countless hours of free labor for Amazon here?

Most people is good a heart and they want to help, even when it's bad for themselves or even illegal.

"Under the federal Fair Labor Standards Act (“FLSA”) and many state and local wage and hour laws, the use of volunteers and interns is strictly regulated." https://www.forbes.com/sites/richardtuschman/2012/08/24/usin...

Amazon strategy is just another example of "if it's online it's legal". Destroying jobs in exchange of "experience", "visibility" or "a better curriculum" is regulated and for good reasons.

Do they want their software to be free? That's good. Everybody can use it and anyone can collaborate. Do they want their documentation to be reviewed and expanded by voluteers? Why not hire Technical Writers?


Amazon evades paying taxes while expecting that volunteers do their job instead of employees. The world economy can't continue running like this for long.

While that's an interesting point, it's not as simple as it being 'illegal' either.

I've told vendors of documentation bugs - and they've even listened. I wasn't paid by the vendor, I was paid by my employer. Who has a business relationship with the vendor. I sort of doubt this is illegal.

It's a great resume padder.

If it's not a huge change I'd rather suggest a fix while I'm in middle of something so that I don't have to update internal company documentation + code comments for special cases. Comes with caveats of course, like I'm not going to spend a lot of time waiting to merge the PR for Amazon's docs. But given how many people use it, it's sort of for greater good anyway.

If it is one/two lines, why not.

> Can someone explain to me why people will inevitably put in countless hours of free labor for Amazon here?

I wish I knew. Reminds me of Google Map Maker, aka "Work for Google producing proprietary mapping data that even you can't get back for free".

I think php.net's documentation with comments was better.

That documentation devolved over time into a tire fire of bad and dangerous advice.

It really needed curation.

At worst, it was a mix of old advice, bad advice, dangerous code snippets, and people treating it like Stack Overflow.

Usually the first N posts were some really shitty solutions and someone would have to scroll way down to see a reply that calls them out and offers a good solution. But someone browsing the docs are most likely to use the first solution that seems good enough, particularly beginners.

which is still better than no advice at all.

It's good to know there are bad practices out there, and the doc team should update the doc and show how to avoid bad practice.

I disagree.

I will happily take docs, such as rust's docs or NaCl's docs, which don't ever mention the possible of md5summing a password, to docs where there are hundreds of comments recommending exactly that terrible practice.

There are a practically infinite number of ways to do things wrong, and very few ways to do things right. Documenting the right way by exhaustively demonstrating the wrong ways is a fool's errand.

But more to the point, I will happily take no docs at all to docs that are more wrong than right.

I would wager that you won't find an obviously bad security practice like md5() the password in the PHP documentation comments that isn't voted way down.

You could still find new, fresh advice about using `mysql_query` as late as like 2013 on there. I don't think it's better to have that advice than none at all.

Expecting a "doc team" for an open-source language to keep on top of what fresh hells people are doing with forever-deprecated things seems like a very big ask.

Ultimately I make docs for myself so it's easier next time. And i forget ... everything.

Because their "OPW strategy" is designed to incentivize people to contribute - https://youtu.be/I3LrtjsE6eg?t=16m17s

Issues. Make their outdated docs known. I think that’s the main benefit.

But how is that different to answering questions on SO?

The payoff in this case is probably getting your name into the official docs and leveraging that reputation as an AWS guru into pay/promotions later

That’s one way of looking at it.

Another is to consider this as Amazon giving the users a standard, simple way of fixing the documentation issues they inevitably run into.

I’ve submitted bug reports about proprietary products because I want them to get better. How is this different? Or is that exploitative too?

Bug reports are a little different than the product itself. I guess my take is that yes, if you needed it fixed because you are using it, then it makes sense.

But in the case of documentation, by the time I figured things out and see that the documentation is wrong, I no longer need the docs and I know what to do.

Documenting something you've just figured out, for other's benefit if not your own, is a pay-it-forward (or golden rule, or reciprocal altruism) approach: behave such that if other people behaved likewise, you would benefit.

Because the existing docs are pretty terrible, I'd be thrilled to help update them to make my life easier

On the other side, what if Amazon gets sued based on something written by a volunteer contributor?

Why do people put in countless hours of free labor for Jimmy Wales?

Because that labor isn't for Jimmy Wales, it's to further the education of the entire world (Wikipedia is free).

I think that's a much better goal than helping Amazon make money.

Spending time to help improve Amazon's documentation helps more than just Amazon, doesn't it?

It helps Amazon and people who pay Amazon. (basically, Amazon)

I know it's been a while, but I still cant get over the azure outage that lasted almost a full day in 2012.

Thats a very long time ago in cloud computing years.

I for one, welcome this change.. assuming I can grep for what I'm looking for in the repos. Or maybe Dash will integrate the docs so I don't have to browse around on metered LTE networks.

This is.. Odd. AWS is notoriously a black box. Their services aren't open source and on github; who would have the insider view necessary to work on the docs outside of their own employees? Is this easier than having an edit suggestion feature? Will PR's cause clashes with their technical writers?

You don't need system insight to improve docs. On one end of the spectrum, docs just need people to improve the language like typos, grammar, and sentence flow.

Also, docs map out the black box but end-users use the black box. They are in a position to find disparities between what the docs say and what the black box does. Like noticing the example response doesn't match the actual response.

Or that if you provide a stream to the API, you must provide a content length, yet this relationship is only marked as optional in the docs.

Maybe an engineer would have to double-check the code to verify the behavior, and that's great.

It looks like at least some of the guides are licensed Creative Commons BY-SA.

For code samples do they do something similar to Mozilla?

Mozilla MDN:

“Code samples and snippets

Code samples added on or after August 20, 2010 are in the public domain (CC0). No licensing notice is necessary, but if you need one, you can use: "Any copyright is dedicated to the Public Domain. http://creativecommons.org/publicdomain/zero/1.0/".”

On a lark, I just forked the repo for the AWS CLI user guide and added some commits that introduced subtle and not-so-subtle errors. Let's hope my fork never shows up in search results. Otherwise, people are going to be making curl requests to r/tacobell to download the AWS CLI Bundled Installer.

If this collaborative-documentation initiative were launched as an Amazon-hosted wiki, editors or mods could shut me down. But since it's open source and under a Creative Commons license, Amazon has essentially no control over how much I vandalize the content.

Fortunately for AWS, I'm not really interested in vandalizing the content or misleading people. But the fact that someone can create a knock-off site riddled with errors, and Amazon can't claim copyright infringement if the site plays by the Creative Commons rules, is concerning, isn't it?

Edit: I know, I know. It's already possible to make unauthorized copy-pasta sites. But at least Amazon would have the ability to threaten the site's owner or hosting provider with copyright infringement claims. That option disappears when you release your official documentation under a Creative Commons license.

Hmmm. Well then any content is open to exploitation, regardeless if it's open source or not. I can basically copy and paste your comment on my own website and change its content...No fork would ever have as much authority as the original repo since that's the one linked and referenced everywhere. This is exactly the problem that Google solved when they created the PageRank algorithm.

> Edit: I know, I know. It's already possible to make unauthorized copy-pasta sites. But at least Amazon would have the ability to threaten the site's owner or hosting provider with copyright infringement claims. That option disappears when you release your official documentation under a Creative Commons license.

You misunderstand trademark law.

Trademark law is meant to prevent confusion, and would kick in here.

Nothing is different with respect to creating a confusing fork; before they wouldn't claim copyright infringement before either, but trademark infringement.

Nominative fair use should cover that. Otherwise, no one could write an "Unofficial Guide to ..." any technology or service with a trademarked name.

I would reject that PR.

Something like that change should have a message explaining why that change is proposed.

Unfortunate that the blogpost itself can't be forked :p

I wrote it -- do I need to make a fix??

well, it depends on what type of comments you guys find acceptable & would want to see when you're deciding whether to merge the changes in.

imagine down the line somebody submitting changes & the message isn't the reason why those changes are there[0] but instead a copy paste of the changes[1]

[0] e.g. "remove fuzzy language"/"update intro to more accurately reflect addition of new services"

[1] not denying that sometimes this makes more sense.

Gah, choke, barf.

Crowd-sourcing is not the right way to document a big complex software suite.

The right way is to use special software to compile or extract the documentation from the underlying source code. The documentation generator should be conceptually similar to a Jupyter Notebook or a Mathematica Computable Document.

Furthermore, the documentation generator should run actual code that throws an error if there's a mistake. For example, in this document:


The CLI operations should be lifted from some other file, and that other file should get run as a test by a CI tool, so that when there's a change to the CLI, it's automatically flagged for correction.

While for docs which are very close to code, you're right, good documentation has a lot of things at much higher level than code, and a lot of times teaching people how to do things unrelated to the actual implementation, ie "how to expand a linux file system after you increased an EBS volume". Your comments are a bit reductionist.

It's interesting that they mix markdown and restructured text. ReST is clearly the superior formatting language, but markdown is so much easier to pick up and use.

Just curious, I've never actually heard of "ReST" as a format before now... I wonder if you might be able to explain what makes it superior to markdown?

ReST is really big in the Python ecosystem, thanks to Sphinx (docs generator), and the read-the-docs website.

The main advantage for ReST is that it is well defined and consistent. It's a bit heavier than .md, but I think it will pay off for large projects.

Here are some articles about it: https://eli.thegreenplace.net/2017/restructuredtext-vs-markd... http://ericholscher.com/blog/2016/mar/15/dont-use-markdown-f...

I've written docs for personal projects in markdown, but now that I work on larger collaborative projects I feel like ReST is the way to go.

PS: If you're using Sphinx, you can actually support both .md and .rst in the same project, and slowly transition from one to the other, see https://github.com/ivanistheone/ricecooker/blob/docs/improve...

rest (restructured text) has been used for python documentation for a very long time. so -- especially for that particular usage -- it's much more functional and clearly documented and consistent and free of edge-cases than (the far-too-numerous flavors of) markdown.

on the negative side, there aren't many quality authoring-tools for rest.

markdown has a very high profile. but it's questionable whether it deserves it. (i wouldn't say that it doesn't. but i wouldn't say that it does, either.)

another option you could look at is asciidoc. it isn't clearly better or worse than those other two systems, but it is different, and you might end up liking it better. (or you might not.)

light-markup systems are much like static blog systems. there's a lot of them, but it's hard to decide among them, and many people end up inventing their own variant that serves their own individual use-cases.

Uh oh, looks like my tech writing career is under threat (so of course I'm posting with a bit of bias here). A wise move on amazon's part though, offload your work on to users who will improve the docs for a couple of reasons:

1. They can add their contributions to their resumes. 2. They use the software and more than likely reference the docs, and so are already stakeholders that desire the content to be correct and so will happily fix it given the chance so that they can save themselves headaches in the future. 3. The general concept of 'open source' is very enticing and is almost always has a positive connotation, so very few people will see that this is a clever way to cut costs and net free labor and not really the virtuous public gesture of transparency and good will a lot will perceive it to be.

This would be different if, like actual open source software others could pull the thing and conceivably make a better version of it and compete, but since this is necessarily bound to a product that's not going anywhere outside the organization, it's de facto always going to be free labor for amazon and never going to promote potential forking out of modified versions of the project, further securing the companies foothold in the market as well--after all, if you no longer have to pay folks for all the extra bits like documentation, support, and maintenance of internal libraries, you can have nothing but a horde of devs working on the user-facing product, and ensure all of your money goes toward nothing other than squashing the competition. Sure you could argue it's still technically possible for someone's fork to gain bigger traction than the official amazon repo, but cmon now, the project is so massive and is so bound to the company that such a scenario is highly unlikely. On the plus side, open sourcing is a boon for archivists were the main content to ever disappear (unlikely).

I realize this sounds a little bleak, and I'm sure amazon also has good intentions with this move, but I can't help but see the potential for future markets where a lot of free labor is snagged under the banner of 'open source' simply because people are shortsightedly seeking resume boosts and not realizing that contributing labor freely will more than likely hurt the job market in the long run.

Not saying you shouldn't contribute to open source, nor is the idea of open source a bad one, but I think the lines get a bit blurred when it comes to open source projects owned by private (massive, in this case) corporations who are then gaining from the work of outsiders without needing to compensate them. Sure, helping out looks good on your resume, just as unpaid internships do--but at some point we need to take a look at these practices that can be, whether intentionally or not, ways of sidestepping the proper dispensation of wages and compensation for labor.

They may still need to pay somebody to green light merges, but that's a lot less costly, I imagine, than paying a team of writing professionals.

I find interesting the assumption every comments make that people are "dumb enough to work for free for Amazon if it's openly accessible". To give the possibility to everyone for proposing changes to their documentation doesn't mean EVERYONE will propose changes. It simply means people who have interest in doing so will be able to do it more easily. We can see this like a reduction of the cost of communication for everyone who have interests at a better Amazon's documentation. To me, it looks like common sense : you won't give money to someone for fixing a comma, because it's not hard and therefore it's not labor. The power of internet is that the work can be split among so many people that the "work" isn't "work" anymore. If it take everyone participating only 5 minutes a day to improve the documentation, then I cannot see how you would remunerate people with money. People can however still point to their github accounts to prove they contributed, and they "earn" reputation instead of money (ala StackOverflow).

I'm curious. What makes contribution to this docs so special that adding it to the resume has been mentioned a few times in this thread? I mean, is it not just another repo you have contributed to in your github profile?

It may just be another repo at the end of the day. Point is, contributions to open source are sometimes considered when selecting job candidates, so it remains an incentive for folks that may be strong enough for it to nudge them to do work for Amazon. I.e. even though it's technically free labor, I potentially benefit in some way thus I'm content. The problem is potential edge in future job searches is a far more ephemeral form of compensation for your time, is not concretely measurable, and is not actually provided by the company the labor was completed for. In the end the company ends up with a tangible product which they can use to compete in a market and the open source contributor ends up with, at best, a potential benefit should she ever need to apply to some job that will actually pay her for her work in the future.

i'd have to think it's a prestige matter. sending some patches to project X vs AWS ... one will be seen by more folks, used by more folks, probably has to be vetted by someone with known authority, etc.

why does going to one university vs another carry more weight with some people?

Cool! How does aws translate their docs?

Finally. Been asking for this for years.

Now if only they would open-source the AWS Console...

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact