
Architect – A Serverless Framework - mrbbk
https://blog.reifyworks.com/architect-the-killer-serverless-framework-41565372811b
======
sergiotapia
Don't invent some other file format please, just use whatever most people use
unless you have a real specific reason for it.

~~~
brianleroux
Author here: we did? Pretty well documented issues w JSON and YAML (coming
from CloudFormation perspective). Both have problems (that are unique to each)
so we created a format for solving that.

If you like JSON and YAML and don't mind those issues you are well served by
existing solutions. Cheers!

~~~
optimuspaul
What problems did you solve and how were they solved with your new format? I'm
neutral in this argument at this point, so I'd like to hear your
justifications. Thanks!

~~~
brianleroux
There are many issues with both formats:

\- JSON has no comments \- YAML nesting tends to get deep which makes creating
bugs very easy \- Both are become hard to read / edit in larger files

 __The primary issue is translating the intent into a machine syntax. The
format we landed focused on minimalism to communicate intent. __

The the file format is the least interesting thing about .arc but one of the
most distinctive. An .arc file VERY QUICKLY communicates the project intent.

An example:

``` @app hi

@html get / ```

The equiv CloudFormation YAML is 700 lines. You'd have no idea what that
project does without digging into the code. The .arc by contrast is extremely
easy to both read and author. As the app grows it becomes very clear the
project intent and were to find things.

We certainly could have created a new YAML thing but many projects already do
that … and I'll let their work speak for itself.

~~~
devxpy
> The equiv CloudFormation YAML is 700 lines

Accepted, but that's not the topic of debate.

The issue here is that you can write the example you gave in YAML like this

    
    
        app:
           hi
        html:
           get /
    
    

And it could still generate the same result. But this way people can write the
config in a more familiar syntax.

Hope I explained my point correctly.

This is just a matter of UX/ API, not how it works.

If you somehow felt that I disrespected your work, forgive me.

~~~
sirbrows
I agree. And isn't that exactly the solution the Serverless framework
implemented?

~~~
devxpy
I am not sure, but serverless requires more config and tweaking as compared to
arhitect. In contrast, serverless is a more mature and flexible platform.

~~~
sirbrows
It was more of a rhetorical question, Serverless definitely uses .yml

[https://serverless.com/framework/docs/providers/aws/guide/se...](https://serverless.com/framework/docs/providers/aws/guide/serverless.yml/)

Obviously that's a verbose example and there are defaults for most everything
AFAIK.

Anyway, Architect seems cool too! Always down for more options and
competition!

------
monksy
One of the biggest promises that I can see about the serverless movement is:

The ability to have the infrastructure isolated from the raw code being
written. (Think more of a grid computing and automatic planning being
included)

Unfortunately, I'm not seeing it. I'm seeing that it's heavy on the initial
business idea: "Get people to pay more and utilize our servers better".

~~~
staticassertion
> The ability to have the infrastructure isolated from the raw code being
> written. (Think more of a grid computing and automatic planning being
> included)

Can you elaborate? It sounds like you're saying that if you write code that's
deployed to a serverless infra you end up with 'serverless code'. This hasn't
been my (very limited) experience - where the serverless stuff is just a thin
layer around your business logic.

~~~
derefr
I think they’re saying that you can expose your lower-level atomic APIs as
serverless functions, and then, separately, build a “routing and aggregation
layer” that exposes APIs that people might actually want to use. And that, for
a lot of projects, that routing/aggregation can be entirely specified in a
declarative manifest format, rather than involving any coding.

This neatly cleaves responsibility: your business logic team can just
build+deploy+expose functions; and your frontend and ecosystem teams can
build+deploy+expose the APIs _they_ want on top, perhaps being maintained
within their own codebases.

These can even be done on two entirely-separate platforms, rather than both
being a part of the same “serverless” IaaS. You could use any combination of
commodity FaaS providers (including your own hosted FaaS clusters) to deploy
functions to; and then you could use some “smart” load-balancing system (e.g.
Cloudflare Workers, or the routing component of an FaaS) to do the
routing+aggregation on.

~~~
monksy
That is exactly what I was shooting for. I just didn't have the words at the
moment to explain it.

What I've seen AWS Lambda being used: One-off functions with manual output
management. They aren't really being chained together.

------
niteshade
DISCLAIMER: I've contributed a few times to the Serverless framework.

I like some things about Architect, but there are things that just don't make
sense:

\- Integrations with Slack but no way to define custom CFN resources

\- No authentication support

\- Custom file format (3)

\- Not being able to set CORS in this custom file format (4)

\- Magical vendor folders (5)

I am sorry for the negativity, but brianleroux has posted in this thread
several times claiming his framework saves 700-line CloudFormation templates.
What I don't see is him comparing snippets with say, the Serverless
framework(1), or AWS SAM(2). He seems to portray the choices as either his
pet-framework-of-the-month or plain old CloudFormation, and that's just not
true. That's not to say Serverless or AWS SAM are perfect (they most certainly
have their warts), but they are both arguably more mature and have larger
communities behind them.

(1) [https://serverless.com](https://serverless.com)

(2) [https://github.com/awslabs/serverless-application-
model](https://github.com/awslabs/serverless-application-model)

(3) [https://arc.codes/quickstart/arc-project-
layout](https://arc.codes/quickstart/arc-project-layout)

(4) [https://arc.codes/guides/cors](https://arc.codes/guides/cors)

(5) [https://arc.codes/guides/sharing-common-
code](https://arc.codes/guides/sharing-common-code)

~~~
brianleroux
Hey np! Difference of views. No integration w CF is absolutely true but you're
also totally free to do that. It doesn't _prevent_ you from using it.

No auth support is false. We support session out of the box and you're free to
impl any auth scheme of your choosing.

Custom file format is being well addressed in other comments. If you don't
like it thats fine: enjoy YAML elsewhere.

CORS could be implemented in the file format…I'm personally not a fan for a
variety of reasons but its open source. PR's welcome.

Magic folders! Absolutely. Structure is a good thing.

I don't find this negative. Serverless framework is an abstraction that is
easily as verbose as CF but I'm not interested in comparing to it because we
have different goals. Serverless is an abstraction between clouds. If
mulitcloud is your goal: use Serverless. SAM is a flavor of CF which is def
less verbose but still very vendor specific and …wordy.

This isn't a 'pet framework of the month'. I do take mild offense to that.
Development of .arc started in 2015 for begin.com and got met with some
serious production traffic (from things like Techcrunch and Venturebeat). It
held up beautifully. We decided to pivot from our original producton into
CI/CD for arc with begin.com because it is clear to us that it is a better way
to build serverless apps.

Architect was officially open sourced in July of 2017 and donated to the JS
Foundation to demonstrate the long term commitment to it. Development has been
very active since. I stand behind my work, dedication to open governance and
approach we're taking and, I think, I have the track record to ask for the
benefit of the doubt when I say that.

~~~
niteshade
Hi, thanks for the reply!

> No integration w CF is absolutely true but you're also totally free to do
> that.

It just seems then that there is no real way for me as a developer to transfer
all the accumulated knowledge I already have about AWS to your framework.
Looking through Architect's source, it actually seems as if everything is done
via the AWS Javascript SDK, which seems a bit ... odd, to say the least. I'd
even venture as far as saying it goes against the AWS Well Architected
Framework guidelines (1) (which recommends using CloudFormation).

> No auth support is false. We support session out of the box and you're free
> to impl any auth scheme of your choosing.

Apologies, I couldn't find any mention in the docs. I meant more in terms of
using an AWS service such as Cognito Federated Identities, and in terms of
protecting API routes with either AWS_IAM, API keys or Cognito Authorizers.

> CORS could be implemented in the file format....

An Architect project deployment is not exactly repeatable if I still have to
go to the Web Console to enable CORS now, is it?

> Magic folders!

I personally would prefer either my project's root `node_modules` or a Webpack
bundle be deployed, over which I have some measure of control. Node already
has a hard enough time with its resolution algorithms, must we add to the
madness?

> Serverless framework is an abstraction that is easily as verbose as CF

I can't agree with you on Serverless being as verbose as CloudFormation, and I
think you should know better as someone who has dealt with the complexity of
setting up API Gateway programatically.

> but I'm not interested in comparing to it because we have different goals.

I think the comparisons are fair - both Architect and Serverless claim to be
"Serverless frameworks". I could be in the minority though.

> Serverless is an abstraction between clouds. If mulitcloud is your goal: use
> Serverless.

I mean... you're not wrong, but that's grossly oversimplifying it. I've
personally added support for Serverless AWS projects to allow hooking up
Lambdas to Cognito User Pool triggers. That's a pretty solid AWS-only use-
case, that likely isn't available on other cloud platforms (or at least, I
never implemented it for them).

> SAM is a flavor of CF which is def less verbose but still very vendor
> specific and …wordy.

Well, isn't Architect vendor-specific as well?

> This isn't a 'pet framework of the month'. I do take mild offense to that.

Apologies for that remark. I'll be honest - I'm not really interested in the
story behind the framework, I'm more interested in what it brings to the
table. I'm glad your project worked for you and your company, and I wish it
the best, but personally I don't see why Architect couldn't have just been a
layer of syntax sugar over SAM/Serverless framework. If there was a killer
feature that's specific to Architect, I can't see it (unless you count the
file format as a feature).

(1)
[https://d1.awsstatic.com/whitepapers/architecture/AWS_Well-A...](https://d1.awsstatic.com/whitepapers/architecture/AWS_Well-
Architected_Framework.pdf)

~~~
brianleroux
Long story short is this existed well before CF support did for Lambda. I
don't think you're actually interested in learning and more interested in
appearing to know what you're talking about so I'll leave it at that.

~~~
brianleroux
Also! Killer features galore but folks having trouble seeing past the fmt.
You're just making so many assumptions here it's very clear you didn't try it.
It's fast.. really fast. Faster than CF.

------
catchmeifyoucan
I actually like this. I build serverless apps all the time, mostly manually.

Serverless framework is alright, but I felt it was really bloaty. There was
just so much going on. Too many variables and it confuses the user. This looks
clean. I might give it a try. I didn't find a link though in your blog.

~~~
wolfgang42
There is a link, but it's buried in the middle of the second paragraph. There
should really be another link in the call-to-action section at the bottom.

I've wanted to try building something as a serverless architecture for a
while, but every time I look into it I get confused by all the different
things you have to set up to get an AWS Lambda function working, or I look
into a framework but I can only find pre-canned examples for hooking into
Slack, or just generally get confused by blog posts with lots of buzzwords and
very little substance. In the end I always wind up just writing another daemon
and sticking it on my server to get the job done. All I want to do is write
some code and trigger it with an HTTP request, but for some reason Architect
is the first framework I've seen that puts an example for how to do that on
the homepage.

~~~
keithwhor
Plug that I’m sure Brian won’t mind (he encouraged me to start a company a
couple years ago):

If you want to get started building serverless functions and APIs quickly, do
it in your browser with [https://code.xyz](https://code.xyz) \- we just
released it along with an announcement of our most recent $2M in funding from
Stripe [1] who join Slack and others as investors and customers.

Disclaimer: I’m the founder.

[1] [https://stdlib.com/blog/stripe-code-xyz](https://stdlib.com/blog/stripe-
code-xyz)

~~~
wolfgang42
See, this is exactly the sort of thing I'm complaining about.

code.xyz gives some 'hello world' examples, but it doesn't even link to any
docs--it just goes "look how easy it is! Here, sign in to our service!" I went
hunting for the stdlib docs, and it took me five minutes to figure out how to
call my service by HTTP--apparently I just get a special username.stdlib.io
endpoint that I can call, but when I clicked 'Calling Services' the first
thing I got was a page that explained how to use bearer tokens. I still don't
know how I can POST to this endpoint, but I have learned that apparently
stdlib has invented the concept of a 'source code' because apparently having a
pre-canned Slack bot integration is super-important to any self-respecting
FaaS provider. Also apparently I can find prebuilt 'source codes' in the
'stdlib registry', but the docs don't link to said registry, only the
aforementioned Slack integration. At this point I gave up; I don't have the
patience to learn a brand-new meaning for 'source codes' when all I want to do
is make an API call when I get a webhook payload, which I already know how to
do in a dozen lines of Python.

I realize this complaint sounds kind of harsh; you're getting a lot of my
frustration dumped on you. Really all I want is basically a managed,
autoscaled version of Express.js. Maybe there's some reason it's like this
(I've never worked with anything that could be remotely described as 'at
scale', it's mostly been line-of-business apps with a few dozen users maximum,
or at most a web shop that still runs on a single dedicated server), but it
seems like all of the Serverless providers add a bunch of extra concepts and
confusion on top of what should really be a simple principle. Or maybe I'm
completely misunderstanding the point of Serverless and it's actually intended
to solve a completely different problem I've never had--I'm also open to that
explanation instead!

~~~
keithwhor
When you run your code a link to your API endpoint is provided immediately:
obviously this isn’t clear enough! Thanks for the feedback. We’ll make sure we
fix this ASAP. :)

No worries re: frustration. It happens!

------
sbr464
I’m a big fan of code generation. I can usually adapt to new formats or
concepts without too much issue. I feel that while it’s not difficult to build
up strings dynamically, the harder part can be overwriting and verifying
values, and working with logic. That’s my main issue with YAML over JSON. How
would you recommend working in this area with this format, or benefits over
JSON?

~~~
RangerScience
YAML is far easier for humans, and has a few advanced features (node
references) that are REALLY nice for any kind of configuration, since you can
re-use blocks.

Let's say you have to define a bunch of services, with mostly the same
environment variables. You can make a "defaults" node, and then a) reference
that node in the service descriptions to re-use those env vars, and b)
selectively override those defaults in the service description's node while
leaving the rest of the defaults untouched. It's really nice.

~~~
sbr464
That's good to know, thanks for that tip. I've been using secrets with
Kubernetes for sharing things currently, but only in small cases.

~~~
RangerScience
It's super useful. There's a way to reference a node that just places that
node as a child, whole cloth, and another way that merges the node reference
with the local node.

This is useful, say, if you're actually de/serializing objects and want to
maintain actual references, or if you're just trying to share data between
nodes.

------
TooBrokeToBeg
Lambda and SNS are super expensive. I'll skip it.

~~~
wiremine
It can be very expensive. I priced out two scenarios recently for an IoT
solution. Using Lambda the cost would be $27,000k a month. Using a LB/EC2
solution with reserved instances the price was $60/month. (We also priced out
IoT core, and it was $3,000k a month).

~~~
pedalpete
Can you share more details on this? We've just moved to Lambda and are paying
approx $60/month for lambda where we were paying $260 on heroku.

But then our db on AWS is another $110, where it's included in heroku.

I know it will depend on your memory usage, etc. and as we grow, we expect to
move to ec2, but I'm curious as to where/how you had such a huge difference in
cost.

~~~
wiremine
In our case the customer was interested in using Lambda as a proxy to handle
logging data from a fleet. It involved a billion+ requests a month.

------
omani
hey nice project. I had my Show HN recently. I've built a tool to also use it
in a serverless fashion. Check it out at
[https://triggerfs.io](https://triggerfs.io)

It has been a month since then and the feedback was awesome.

I would love to hear your opinion and how you could possibly use it (use
case), since you can do a lot with serverless/FaaS etc.

cheers

------
jonesnc
Is there a github link I missed?

~~~
brianleroux
[https://arc.codes](https://arc.codes) for docs and src can be found at
[https://github.com/arc-repos](https://github.com/arc-repos)

------
devxpy
Great! Another file format for me to learn. Why can't everyone use one of the
standard file types, like YAML/JSON?

~~~
jacquesc
I really wish people wouldn't use JSON for configuration. The lack of comments
and shortcomings in formatting abilities make it particularly ill-suited. {
"_comment": "this is a comment" } is cringeworthy to. It's great for
transport, but horrible for static file config.

YAML is good, but not to everyone's taste. I've been using it for a long time
(as a ruby developer) but many people I know find it ugly.

I think Arc config looks much better than anything they could have come up
with using JSON/YAML. Optimize configuration files for humans, not machines.
If its simple enough to learn and be readable, more power to em.

~~~
jfoutz
It's interesting. can you have @js more than once? can you have @app more than
once? pretty sure ಠ-ಠ is a valid app name, but ಠ_ಠ isn't. i also can't quite
tell how to say map of maps. i'd guess it's

    
    
        @this-section-has-a-map
        hello-world
          name child-map
            ಠ-ಠ a
            ಠ_ಠ b
    

but it's hard to tell. Also, i'm not exactly a fan of { "_comment": "this is a
comment" }, but i've run into several situations with yaml where i want to
build a tool to rewrite a bunch at once, but since comments have no
representation in the parse tree i have to do goofy vim macros to do my edits.
since _comment is in the tree, i can add more keys without trashing the
existing comments.

~~~
brianleroux
Hey, one of the .arc authors and maintainers here.

Only one of each section is what we designed for though the validations for
that might be broken (and if so we can fix right away). Nesting is limited to
one level. No need for more that we've seen yet.

------
matte_black
My problem with the serverless movement is database connection pools.

Every time a function runs, it has to create a new connection to the Postgres
database so it can run a query. This is super slow, and dead in the water. If
I have to maintain a separate server to have a global connection pool, I might
as well just have a server for my API as well.

~~~
sbr464
I agree with this. I would be comfortable moving more code to serverless if
this weren't as big an issue. I think having a GraphQL server as an API to the
database may be a good solution, since it can handle a variety of
queries/mutations and RPC like behavior. But I still run into this issue a lot
whenever I make a new serverless function/project.

------
andrewmcwatters
I think this whole idea of pushing "serverless" as a concept to engineers is
deceptive at best and damaging at worst when you think about all of the naive
new developers entering the industry having to shovel all of this shit into
their minds alongside proven technologies and conventional formats.

What does "serverless" framework really tell veterans? Who are you selling to?
What are the considerations for adoption?

There's so many questions that pop up before I even look at the software, and
once I did, there's more questions which obscure the point. Why should I use
your bullshit?

Worse, 5 years in after a company has adopted this junk, some poor sap is
going to have to maintain it after being 100k SLOC deep in it, with no vision
of how to port out of it alongside meeting existing technical task
completions, at which point this will have become completely irrelevant.

~~~
CharlesW
> _I think this whole idea of pushing "serverless" as a concept to engineers
> is deceptive at best and damaging at worst…_

To me, serverless is as normal a progression of abstraction as anything in
software development. What about FaaS crosses a line for you, where
(presumably) PaaS doesn't?

