> SWF -> Amazon EC2 Queue Use this to
Build a service of "deciders" and "workers" on top of EC2 to accomplish a set task. Unlike SQS - logic is setup inside the service to determine how and what should happen.
I do not find this one to be that helpful, as "deciders" and "workers" doesn't make sense in this context. Logic isn't encapsulated inside of SWF, and the workers do not need to be hosted on EC2. SWF basically tracks workflow and activity versions, and is essentially a state management and work distribution system. It distributes out tasks to polling workers based on what the workers are polling, and what state is currently in SWF.
I think a (possibly) better name/use this to would be Amazon Task Orchestrator/Schedule and run jobs that do a bunch of things in order that can be distributed, where you want to run the workers on either EC2 or your own machines.
Ok, maybe not too much better, but I think it shows the use case a little bit better.
I really like this concept of a product TLDR. It reminds me of https://tldrlegal.com/. I wish these sorts of summaries were available for other software products and services. It would help when trying to get a simple understanding of all the buzzwords that I can't keep up with without having to read pages of documentation or look at examples.
It is a general messaging service and can deliver messages to SQS, AWS Lambda, email and arbitrary HTTP(S) endpoints.
For example, we have some S3 buckets set up to send an SNS message to an SNS topic on new object creation, the topic then in turn notifies our subscribed HTTPS endpoints by sending the SNS message which we can then parse. You can also connect S3 directly to AWS Lambda so that a new event in the bucket sends an SNS message to a lambda function - which is very handy for doing things such as "server-less" thumbnailing, transcoding, etc etc.
I was trying to write this from the standpoint of describing what the services did to another developer in a really high level way.
If you want a high level explanation for someone, I'd describe it as Amazon Callbacks. It lets you create callbacks for any action in and across any Amazon web service, where that callback can be in the form of an HTTP post (e.g. back to your web application), email, push notification, etc.
Sounds like a good fit as an open source documentation project for a dedicated group of platform researchers to knock out collaboratively in a month.
EC2/"Amazon Virtual Servers" - true today, perhaps. But when EC2 first launched, they were trying hard to sell the "elasticity" and "pay for what you use" aspects, which were unique at the time. There were no EBS volumes, the storage was all ephemeral. While it was possible to use EC2 instances as virtual servers, that was really not what they were designed for.
IAM/Users Keys and Certs - I agree that IAM is a bad name, but it's still useful to distinguish between aws API authentication and other kinds of authentication. Say "IAM" in a conversation and people will immediately know what you're talking about. Say "Users, Keys and Certs" and you'll have to keep distinguishing between certs for your websites, OS host keys, and that kind of thing.
S3/"Amazon Unlimited FTP Server" - Sorry but this one would just be very misleading. S3 is not an FTP service it is a distributed object store.
VPC/Virtual Colocated Rack" - Again I think using words like colocation and rack would just be misleading. VPC gives you software-defined networking. I think the real issue here is that amazon missed the mark just a little in the original design. VPC should have been the default mode of operation, where the initial VPC was transparent and got a default name. Not a big mistake in the long run but it did mean they had to deploy and name a new service.
RDS/Amazon SQL - I think "Relational Database Service" should be easily understood by any developer using a relational database. If you don't know that MySQL, Postgres, and Oracle are relational databases, consider this the right time to learn.
SQS/Amazon Queue: "Simple Queue Service" is basically the same as "Amazon Queue" except that it uses the same naming convention as S3 so that it's easily recognized as an AWS service rather than some generic amazon store feature.
Elastic Map Reduce/Hadooper: I think "Map Reduce" is much plainer English than "Hadooper."
CodeCommit/Amazon Github: This would be a blatant trademark violation. Also implies that it's less "plain English" given that Github is a proper noun and not a straightforward description of a service. I don't love "CodeCommit" but can't immediately think of anything better.
Not really, the only "object" you can store, Amazon's language notwithstanding, is effectively a file. It's not like you could give it something that needed to be serialized.
And it's hard to see how it's "distributed" when your bucket price varies "based on the location of your bucket" as Amazon puts it.
It's probably more redundant than FTP, but saying S3 is like FTP is a reasonable shorthand and cuts through a lot of the marketing BS.
> It's probably more redundant than FTP
FTP is a transfer protocol, it says nothing about redundancy of the data on the back end. You could put an FTP server in front of an ext4 filesystem, and NTFS filesystem, an NFS mount, or maybe even S3.
"File Server" or something like that would be better. You're quite right that "Object" is worthless terminology in this case - "Object" is basically nerdspeak for "thing".
This is a common issue in computer science and software development. Whenever new concepts are discovered or developed, they often need new names. You need to pick something that will distinguish this new concept from existing technologies. For laughs (kind of), look at configuration management software (Chef, Puppet, Cfengine, Ansible). They all quickly discovered that there were a whole bunch of new concepts that needed names, and while in some cases this hints that maybe these tools a bit over-engineered, it's hard to deny they've been popular and useful. Regardless, they all encountered these new concepts around the same time and they all made up their own names for them. So in chef you have cookbooks, where ansible has roles. Chef has roles where ansible has playbooks. Ansible has modules where puppet has providers and resource types. Ansible has tasks which are basically components of what chef would call a recipe and puppet would call a manifest.
All in all, I think "object storage" is a pretty fair term for the concept of a data store that neither heirarchical or sequential.
Calling it a "file server" is way more reasonable.
Read the S3 docs: http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingObjects....
That is the technical term they've chosen to describe the key, its value, and all its associated metadata. As far as terms go, it's rather straightforward, in my opinion. They probably didn't use "file" because that usually implies a filesystem, which is specifically what S3 is not.
Also, go ahead and check out the history of the term "Object Store": https://en.wikipedia.org/wiki/Object_storage
If you explicitly requested your browser to create a file, then it could be any kind of data. If the web server triggered the download, your browser probably saw an HTTP header like "Content-type: application/octet-stream"
I will admit that there is an abstract concept of a "file" that exists outside the restricted realm of a "filesystem," but that's not amazon's target audience. Their target audience are programmers using APIs.
To a normal person, MySpreadsheet.xls is considered a file, no matter where it is stored. But S3 is not limited to this definition of file. An S3 object does not have to be a file in the sense of an excel spreadsheet or digital photos from your friend's wedding. An S3 object can just be a sequence of random data, which would be called a file if stored as ".dat" in a filesystem, but is not considered a file in the same abstract sense as a Word Document.
From a programming perspective files tend to come with a number of traits, like options for sequential access, random access, or appending. A file can often be modified without re-writing the entire thing. When you are programming with S3 objects, the only way to "append" something is to GET the object, retrieve the value from the object, append the data to that value, and then use PUT to upload entire object back into S3.
In this way, it is similar to FTP, except that FTP is most definitely about files and filesystems. S3 is not just for downloading files, but for retrieving any kind of data, usually directly into an application.
Given that S3's target audience is software developers, the programming version of the term "file" is more appropriate.
It's not like "Simple Storage Service" is weirdly obscure or anything.
IMNHO "File server" implies it exports an actual file-system, and would also be wrong. In that light, I think S3 is more like FTP than a "file server". Sure, it might technically be more like WebDAV than FTP, perhaps "FTP-like service using HTTP for transport" is more accurate -- but I'm not sure it's better than just calling it FTP...
Well, yes, they aren't OOP objects. But as you say, "files" isn't great either. But it was a theoretical alternative. For example, you could do a search/replace on the S3 docs here: http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingObjects...., and they would still make sense.
But FTP isn't even close. You could not find/replace "Object" with "FTP" without making those docs completely incoherent.
That's the wrong comparison though. You could replace all instances of "S3" with "an FTP server" and it would make a ton of sense, minus the reliability claims.
Amazon Unlimited FTP Server is a simple key, value store designed to store as many FTP Servers as you want. You store these FTP Servers in one or more buckets. An FTP Server consists of the following:
You could use a mix of "FTP Server" and File:
Amazon Unlimited FTP Server is a simple key, value store designed to store as many files as you want. You store these files in one or more buckets. A file consists of the following:
Note that this is essentially redefining the word file to include S3-specific metadata.
"S3 - Simple Storage Service" describes pretty well what the point of the service is without making strong claims around what it may or may not be. It's a service for storing things; it's not a database or a filesystem. The devil is in the details of course but even when I was really new to cloud services I got the gist of what S3 was supposed to do.
It's not "Amazon's language"...your thinking is too programming-language-centric. Object has a much more general meaning. Amazon didn't invent the term blob (binary large object) and S3 is basically a blob store.
Since S3 was none of those things, they needed a different name.
I'd like to help contribute to this if possible.
Disclosure : I am Founder of oauth.io to what Cognito is compare.
Could you add the pricing model and notes? It seems to me that the pricing infos are always hidden somewhere and very hard to find out until you actually used them.
Whenever I look at my AWS console I see these reems of badly named services and I think to myself
"after I've dealt with the current problem for which I've logged in AWS,
I might get to figure out what all that other stuff actually is".
They just seem to be bad at marketing their stuff. Here's an example: https://www.google.de/search?q=source+code+repository+online...
Why isn't CodeCommit here on the first page of the search results?
I'm just 'starting out' with a lot of this, despite a few years building/hosting websites, and these sort of services are the things I'm starting to look at.
I'm completely disinterested in Amazon's offerings chiefly because they're so obtuse. Why should I expend so much cognitive load just figuring out what each service is and what it could do for me?
It's undoubtedly possible to do, AWS has a hell of a lot of users, but I simply am not going to perform the mental gymnastics to work out what they could do for me, let alone work out the price (which seems cleverly designed to keep the final figure a total mystery).
Maybe I'm too early in my progression to really need this stuff yet ( I probably am TBH) and so I'm not motivated enough to really dig into it, but the point stands - why should I need to be?
These AWS services are each solving a separate problem that people encounter, and usually embody a sort of "design pattern" like a distributed queue, or a load balancer, or a cache. Microsoft Azure and to a lesser extent Google Cloud Platform have similar lists of services (many services, each doing a different thing): https://azure.microsoft.com/en-us/services/
It's really just like figuring out any other software ecosystem, though. If you stacked up a list of open source libraries or commercial libraries in a certain area, I'm sure it will feel the same. For example, so you want to get one application to speak to another over RPC. Well, you could use Apache Thrift, or Google Protocol Buffers, or Cap'n Proto, or Swagger.io, or SOAP or ASN.1 or I'm sure countless other things, and that's simply to perform the same task. What if you wanted a message queue? RabbitMQ, ZMQ, IBM MQ, ... I don't know if anyone has tried to put all of these on the same page and product matrix, or even keep track of them.
There's just a lot of stuff out there to learn and take advantage of. Being an effective software engineer in business is just as much about reusing existing stuff as it is about building new stuff from first principles. I'd say it's actually more about reusing stuff, since all other things equal it saves a lot of time. Don't build yourself or manage yourself what you can use for free or buy unless it's your core competency.
I find this is the key to understanding a service, library or technology. It is very difficult for me to wrap me head around something if I haven't yet encountered the problem it solves.
For example, I'm sure Docker is awesome, but I don't really understand it yet because I haven't yet felt the pain Docker takes away. So I enjoy following new technologies, but when I don't really see the point I just assume I haven't experienced the pain the new technology takes away.
I've work with complex and critical linux firewalls, and I love some AWS networking aspects...
Your comment was like a fresh air of sincerity. We all need to deal with domain specific issues. You need to be practical to keep going on.
I do think the page shared here is fantastic though and has changed my attitude somewhat towards AWS.
The services they offer are great. Remember that you don't have to learn them all at once though. I've worked with amazon services for a couple of years and I still don't know/haven't used most of them. Just use what you need, really.
So now my Amazon dashboard looks like:
Is an accurate description of a large number of products that I have been forced to use after various CTOs have played golf with a vendors sales team.
Disclaimer: I work at Linode, but have felt this way since before I started.
To make the analogy accurate:
Running a business on AWS is like taking Uber to work every day. At some point it makes sense to just buy the car, hire a driver, find a good mechanic, lease a parking spot, purchase automobile insurance, remember to buy gas, deal with speeding tickets, schedule regular maintenance ....
At some point it makes sense, but it is way, way down the road for most startups.
Netflix spends millions on AWS -- but it would never make sense for them to do it themselves.
My average was ~$140K per month -- and it would NOT have made sense to do self DC.
Right now, im deploying from DC to AWS and dropping my cost and increasing my performance...
Thing is, unlike the "get driven to work" analogy, programmers/etc are in the same problem domain as the required services.
It would be like a mechanic taking an Uber to his garage every day because he couldn't be bothered personally owning a car. What?!
"Car leases" (rented dedicated servers [possibly even managed]) are a happy medium everyone should consider.
But programmers should be focusing on delivering their startup's unique technology or service, not building and maintaining commodity infrastructure.
But I'm continually embarrassed when companies who raised $100Mns in Series B funding continue to rely on AWS when they could have build out much sooner, and reduced their burn rate measurably.
The highest barrier, quelle surprise, is dealing with interconnect telcos.
I like the services they offer, I just really hate the names they gave it. Funny enough, Jeff Bezos said in an interview that names of a product is important #irony
In fact, if you spend more than a couple of weeks implementing algorithms in your entire career you're probably doing it wrong or part of the 0.1% of programmers working on low level libraries.
Sometimes, it really is easier to just implement something your self than go through the "soul destroying process" of figuring out how to correctly set up and configure a library, framework, or piece of open source infrastructure.
Not an algorithm, but it can be a lot easier to write your own SQL, for example, than setting up and configuring Spring and Hibernate. (Talk about soul destroying. Yes, I am a Java developer, why do you ask?)
But then the next person to look at your code has completely 0 support, rather than the very little support they'd have from the crappy documentation and stack overflow if you used an open source library.
Edge cases will consume months of development instead of spending a day or two wrestling with poorly written error messages or badly documented APIs.
The one I always hit in .Net development is logging, log4net and elmah are both abysmally poor at documentation, but incredibly easy once they're actually set up.
I think AWS has plenty of badly named services, but some of these suggestions are much worse worse (and have a huge American influence - that fascination with using brand/implementation names for a generic/standard item) than the current actual AWS name:
* Amazon Unlimited FTP Server - why? ftp is a transfer protocol. S3 is about storage. It should be called Amazon Storage Server. Which it basically is (Simple Storage Service - S3)
* Amazon Memcached - I don't used the service so I don't know if it's only binary compatible with memcached but if its a generic cache that has multiple interfaces (the page references redis too) then more generic term like cache (and elastic implies it can grow/expand) seems more logical.
* Amazon Beginning Cut Pro - wtf. Transcoding is literally the process of converting a file from one type of encoding to another. That seems to be what this does. Final Cut Pro is not a mere format converter, it's a non-linear editor.
* Device Farm - I'll only concede that maybe this should reference mobile devices, but the name seems pretty clear and concise.
* CodeCommit - how is "code commit" less clear than "github". By this logic, Apple's email client should be called "Apple Outlook" because Outlook was a well known email client on the market.
* EC2 Container Service - the reference to EC2 means its slightly non-obvious, but Amazon Container Service would be much better than something referencing Docker. Docker !== Containers.
* WorkDocs, WorkMail - seriously, you're just replacing "Work" with "Company" here.
* Storage Gateway - what are you trying to be obtuse? From your very description, this sounds like exactly what the name describes - a local gateway to a storage service.
* Elastic Map Reduce - the compute/processing part of Hadoop is Map/Reduce. How is "Hadooper" more clear than the current one?
* Machine Learning - you're just being facetious now, right?
* OpsWorks - again, why does their name have to reflect a single specific implementation of a fairly well understood term(s) - Operations, DevOps, etc??
The idea, though, is great - provide some overview of Amazon's zillions of services that doesn't read like it's been generated by a markovbot. Perhaps the authors will improve the implementation.
I have no idea what your business does or what services you need, but there are plenty of other vendors from simple rented virtual machines up to full co-lo of customer owned hardware. The good ones will rely on good customer satisfaction rather than artificial vendor lock-in to remain profitable and competitive.
Considering all AWS ElastiCache is is "hosted Redis", that one confused me the most. Out of all the AWS services, this is the least 'magic' and custom and easiest to come up with a name.
They also support SFTP, and WebDAV too. Are these protocols all the same thing now, just because some apps support multiple protocols?
FTP is a protocol, but people use FTP in a variety of wrong but commonly understood meanings:
FTP Server = Hard drive accessible via FTP, not the server on that machine running the protocol
FTP Program = a file-management interface that uses FTP protocol to send commands
HTTP is also a protocol, though 'web browsers' can browse other protocols, as well as offline files.
Saying that an FTP Client that contains support for S3 is a misnomer is like arguing that accessing a file from your desktop using a 'web browser' isnt correct since you arent browsing any web.
The list is aimed at demystifying the confusing terms for the layman, its not aimed at providing the most technically correct taxonomy (since that would also not communicate much to the layman)
When I use my FTP client for SFTP am I using FTP or SSH?
If you're using sftp you're using ssh. It doesn't matter if your client also talks to ftp servers.
That seems silly, and kind of missing the entire point of this submission. I haven't used any of Amazon's services, so having something to translate back to things I can understand was really helpful, and I feel like I have a much better idea of what Amazon offers.
The post claims to have "better hames" for the AWS services. Many of those names are much worse.
Take the suggested "Amazon Beginning Cut Pro" which was their suggestion for "Elastic Transcoder".
"Beginning Cut Pro" is clearly a reference to Apple's NLE, Final Cut Pro. I would suggest that there are literally zero people on the planet, who would use Amazon's video transcoding service (I imagine it's essentially a distributed ffmpeg queue.) to process video files before importing them into FCP.
This isn't "tongue in cheek", this is "stupid and confusing as an attempt at a laugh". I mean, what if I suggested that a hosted Memcached service be called AppStart because it uses memory and Apps use memory.
> having something to translate back to things I can understand was really helpful
My issue is that several of the services covered, already have a MORE MEANINGFUL name. So if I had no idea what "Elastic Transcoder" did, but it was instead called "Amazon Beginning Cut Pro", I would then assume it's somehow a tool meant to work with FCP, and disregard it (I don't need a NLE, at most maybe I need something akin to ffmpeg, so I can transcode/scale video files between formats automatically and efficiently.
If it's comedy, call it comedy, and make it actually funny.
If it's meant to be a useful guide, make it fucking accurate, or it's worse than what AWS is doing already.
I found it both funny and useful. But I can see where you're coming from. If I wanted something fucking accurate that goes into greater fucking detail than 1-2 mostly tongue-in-cheek sentences I'd go to the fucking source.
"There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors."
~ Jeff Atwood
You can also set up billing alerts, though as has been discussed, they aren't strict caps, so you can still exceed your cap.
You can start with just a few EC2 instances, so conceivably, you could everything on those you'd do with a VPS.
Maybe you decide that managing a database is taking too much of your time, so now you're looking at RDS. It's more expensive, but, it manages just about everything for you. Maybe you make the decision to switch to RDS Now you turn down that EC2 instance running your database.
Maybe now you're thinking that RabbitMQ is too much of a pain to manage, so you do the same thing with SQS. Analyze the time/cost savings of switching, and see if it makes sense.
I think for new applications, estimating is a lot harder, right? You have no traffic until someone is interested in your product, but you can sort of spitball. There's a bunch of free load testing services out there to let you roughly simulate N concurrent users, and open-source projects to simulate those users yourself.
Ultimately, and maybe someone has an idea how to do this: I don't think you can ever plan beforehand. You could be optimistic, and assume your app is going to explode, and that you'll need 8 web servers and redundant load balancers and DB with a read replica... but that's just what it seems like, optimistic. I think the closest you can get is spitballing -- assuming maybe you'll get 100 concurrent users or something -- and making sure you can handle that.
Separately, there's the whole issue of trying to make smart decisions ahead of time with your infrastructure, but similarly, this can be really hard to do.
The good thing is that services and charges that are hard to estimate (SQS, egress network bandwidth, S3 access charges) are mostly drops in a bucket compared to those that are easy to plan for (ec2, RDS, S3 / Dynamo storage).
The estimation part is definitely very hard to do for the things you mentioned at the end, especially things like DynamoDB which want you to specify throughput. Not only do you spend time on that, you then spend time on handling when you exceed your provisioned throughput, etc. There's a lot of time spent thinking about shit there.
Anywho, I agree overall. Some of their services just make sense to use because you'll be hard pressed to paint yourself into a corner with them.
IIRC, Amazon also offers services for a fixed monthly price, you could look into that too.
I can't wrap my head around why Amazon would make names for some very useful services so inscrutable.
It's like the developers were put in charge of naming everything.
I think this comes from considering all those services as separate products. After all, today every product needs a distinguishable name, a logo, etc. /s
* S3 is not FTP. It's more like static web hosting and storage.
* VPC is not a "colocated rack" as it doesn't offer physical placement of hardware. Amazon VLAN would be a better name. It's just private network address space.
I also did not take this particular description to indicate a literal FTP service.
Analogies do not have to be perfect to be helpful. I've never used Amazon web services, perhaps partly because I found their terminology too obfuscated to bother trying to figure out what it did. Similar services provided by other companies were more recognizable to me for what they were.
I found the post here helpful.
Neither did I. But it's flawed even as a rhetorical device. For that kind of thing to work for me, it would have to at least be plausible.
It's fine the way it is.
AWS makes even less sense at scale than it does for small companies. Once you can afford to hire competent Ops staff (and assuming you don't have developers who insist they know everything needed to run a complex system in a high traffic environment, so Ops will 'get in the way') you can get much better results with co-lo or even rented full-hardware, and largely the same Open Source software AWS is built on, but without the confusing naming and vendor lock-in.
Here, that's the catch. Most companies simply can't. Competing against Amazon, Microsoft and Google for talent doesn't sound like a smart strategy.
> Competing against Amazon, Microsoft and Google for talent doesn't sound like a smart strategy.
You don't have to compete. They are trying to make a profit. You just need to be better off, long term. That could be a mixture of lower service fees, more flexibility, improved productivity, less vendor lock in, etc.
Relying on any single company's services, particularly for key/essential systems your business relies on, doesn't sound like a smart strategy.
"You just need to be better off, long term." - This is a good one. Do you really think your homegrown Docker environment, Hadoop cluster and/or state-of-the-art Cassandra ring maintained by your one and only skilled Unicorn is a better long term plan than 100% managed alternatives?
Let's see how well (and how secure) your in-house stuff work 2-3 years down the road and how it will then compare with the always "up-to-date" alternative.
The "vendor lock-in" part is your only valid argument, in my opinion. Costs/benefits will have be weighted properly. How much do you value business agility and pace of innovation vs. increased vendor dependency?
The proper answer will be different for all businesses, but I would argue that as long as a specific IT component/technology is not part of your core business, you might be better off buying it off-the-shelves instead of building it and maintaining it yourself.
I wrote a quick and dirty chrome extension that adds these names to the dashboard:
Pretty spot on...
My clear favorite is Google. Things are generally simpler (in a good, well-designed way). I get the feeling smart hackers are designing things.
With AWS, the general feeling is that is designed by bureaucrats, the type of people who like big, slow and boring companies.
Also, S3 is NOT some ftp service, it is a new concept. And why should they call Cloudfront instead Amazon CDN? Anyone using it knows what a CDN is and what Cloudfront is? Its like insisting Toyota call the Camry the Toyota Mid Size Car.
Amazon invented their own terms out of hubris.
Like most entries on the list, Amazon S3 _is_ a new concept but try explaining it to somebody without mentioning FTP -- you'll see what I mean.
What was that quote about the only hard problems in Computer Science?
Replication, versioning, security, life-cycle management, events to name a few.
There is a category violation in the name since, arguably, AWS Lambda functions are not true anonymous functions, having ARNs to identify them. If only I could save environment state along with the code function (e.g. for injection of runtime secrets); it'd be AWS Closure.
This is very common for small companies without marketing experience, but this happening in companies like Amazon and Microsoft (Azure), I can't understand.
TL;DR version: I use "Amazon's <service name>" instead of a code name.
As the co-founder of a startup doing exactly that (https://emailoctopus.com), I respectfully disagree!
Where's the main danger as you see it
In addition to KV pairs storage, DynamoDB supports querying, documents, indexing and streaming from a feature stand point and infinite provisioning/scaling from a infrastructure stand point. Give it a shot and ping me about any questions!
Also, love the light application of humor.
It's like: Stacking cash on the sidewalk and lighting it on fire
should have been called: Amazon GitScanner
Use this to: search your online source code repos for injudiciously committed access keys
It's like: Slamming the door in bitcoin miners' faces
I'm surprised they don't offer this actually