
Yglu: YAML augmented with an expression language - lbovet
https://yglu.io/
======
fredley
What is this for? YAML in my experience is used for writing and maintaining
config at some supposed sweet-spot between human-readable and machine-readable
structure.

In my experience also, YAML is already too feature-laden for it's own good in
this regard, when what people usually want is just a more human-readable JSON.

Using Yglu for config seems like a bad idea (the whole point of config is that
it's not code!) - so what is it for?

~~~
throwaway894345
For large, repetitive configurations (Kubernetes configs, CloudFormation
templates, CI job configurations, etc) you often want ways to abstract and/or
DRY your configurations. Kubernetes folks use mustache/jinja/Go templates via
Helm, which is a real mess. CloudFormation builds its own (shitty) evaluation
language on top of YAML. This presumably aims to improve on those sort of
hacky solutions. I feel strongly that the "right" solution is either (1)
generate a YAML config with Python/JS/etc or (2) use a terminating expression
language (I think that may be what this is). Since (2) implies a much steeper
learning curve for average developers, I opt for (1).

~~~
escardin
That's what I've been finding a lot lately. Kubenetes manifests have very
complete schemas, but there's not really any official concise way to express
them or relations between them.

I've come to the conclusion that helm/templating is awful. Every helm chart
I've ever seen suffers from the kitchen sink problem. Dependencies don't
really address the problem either, because that's not really resolving a whole
lot in the end.

What I've found so far is that Kustomize strikes a solid balance between an
off the shelf working config provided from official sources, and making
changes myself to suit my environment. Instead of the developer maintaining a
kitchen sink approach, they can have something (or a set of somethings) that
are straight forward, and I can maintain a set of tweaks that work for me.
Unfortunately Kustomize's documentation is not that great, and it is
incredibly awkward to work with.

I'm hoping that solid operators come out for most applications instead of helm
charts. I'm not 100% sure that they're going to be substantially better than
helm, but I'm hoping between operators and kustomize I can link applications
together in a way that I get properly maintained applications for minimal
effort (ala AWS), without having to maintain a large code base linking all the
bits together.

I do think I need to look into pulumi as it might be in the direction I want
to go, but I remain skeptical.

Terraform has some good ideas, but is too rigid for kubernetes, and I feel
like I'm giving up the flexibility of kubectl. I like the idea of having a
state and resolving differences between desired and actual state, but not
enough to rewrite everything in HCL.

~~~
cppforlife
> I like the idea of having a state and resolving differences between desired
> and actual state, but not enough to rewrite everything in HCL.

i authored [https://get-kapp.io](https://get-kapp.io) based on my previous
experiences managing iaas resources as cattle. it strongly revolves around
idea of managing chnageset between actual and desired configuration.

it was also built as a tool that solely focuses on deployment to k8s, leaving
configuration building (ie templating) to other tools.

take a look at it, would love to hear what you think.

on a side note, someone recently asked me to also wrap kapp as a terraform
provider: [https://github.com/k14s/terraform-
provider-k14s](https://github.com/k14s/terraform-provider-k14s)

------
dddbbb
Dhall[1] is a similar concept, though instead of augmenting a particular
format, it is an implementation of a simple pure functional programming
language (with a very expressive type system which guarantees termination).
Dhall programs can then be evaluated to several different formats (currently
JSON, YAML and XML).

[1]: [https://dhall-lang.org/](https://dhall-lang.org/)

~~~
lbovet
Dhall is awesome and rooted in the solid ML line. I wish pure functional
programming langs were significantly growing in the industry. It would show
signs of maturity but it is still currently a high-hanging fruit for many and
traction is diluted in the impression of hobbyism or academism. What can we
bring for those dealing with YAML everyday? Hard to tell them "switch all your
stuff to X". Easier to tell them "you can iteratively add functional
expressions to your YAML". That's the point of Yglu. It's not about proposing
a better Dhall or Jsonnet. It is about proposing better YAML-native tooling.

------
iudqnolq
Some competing languages:
[https://news.ycombinator.com/item?id=20357079](https://news.ycombinator.com/item?id=20357079)

I'm especially interested in jsonnet
([https://jsonnet.org/](https://jsonnet.org/)) because of how thoroughly it's
documented.

An example that produces three different config files:

    
    
        local application = 'my-app';
        local module = 'uwsgi_module';
        local dir = '/var/www';
        local permission = 644;
        
        {
          'uwsgi.ini': std.manifestIni({
            sections: {
              uwsgi: {
                module: module,
                pythonpath: dir,
                socket: dir + '/uwsgi.sock',
                'chmod-socket': permission,
                callable: application,
                logto: '/var/log/uwsgi/uwsgi.log',
              },
            },
          }),
        
          'init.sh': |||
            #!/bin/bash
            mkdir -p %(dir)s
            touch %(dir)s/initialized
            chmod %(perm)d %(dir)s/initialized
          ||| % {dir: dir, perm: permission},
        
          'cassandra.conf': std.manifestYamlDoc({
            cluster_name: application,
            seed_provider: [
              {
                class_name: 'SimpleSeedProvider',
                parameters: [{ seeds: '127.0.0.1' }],
              },
            ],
          }),
        }

------
sawmurai
Congratulations, you just overengineered a data format.

~~~
timw4mail
Yaml was overengineered to begin with.

~~~
tonetheman
There are not enough upvotes in the universe for this comment.

------
IshKebab
For when ordinary YAML isn't quite crazy enough.

Worth reading this if you are considering using it:
[https://www.arp242.net/yaml-config.html](https://www.arp242.net/yaml-
config.html)

~~~
ainar-g
That, and the Norwegian Trap[1].

[1]
[https://news.ycombinator.com/item?id=17359376](https://news.ycombinator.com/item?id=17359376)

------
MockObject
Every data format secretly yearns to be XML, even if it loudly denies it when
asked directly.

~~~
KaoruAoiShiho
In features but not in eye cringe.

~~~
baq
There are features in yaml compared to which xml is a thing of ageless beauty.

------
nkaviani
one of the ytt authors here.

looks like a very similar effort to ytt: [https://get-ytt.io/](https://get-
ytt.io/)

it seems like yaql is a different, potentially more limiting, scripting
language, but otherwise similar efforts.

~~~
nkaviani
actually glanced over the blog post. so this is inspired by ytt, which is
pretty cool.

However, I am not totally sure that the use of exclamation mark and question
mark to start expressions makes it any more readable than embedding code
inside comments.

Also not sure what a "single source for template values" means , since in ytt
you can have multiple input sources

~~~
lbovet
Yes, I liked the idea of ytt and after playing a bit, I felt that we could do
YAML processing in some more YAML-native way.

YTT fulfills its mission, but I wanted to explore a more generic approach and
have the loading of libs and values using the same mechanism. Also, allowing
the reusable parts to be written in YAML and not only in the additional
programming language.

I aim at bringing some ideas in this area and if they make sense, I am happy
to see them live, be it in Yglu, YTT or any other tool.

Also, the fact that building Yglu was fast because of YAML parser and YAQL
were already existing made me explore on my own, rather than developing these
ideas in modifying or coding YTT libs, what I first imagined.

About the readability, I also don't see an argument here. !? vs. #@ does not
matter a lot. It only matters that !? is already understood by the YAML parser
as a tag, avoiding me to write a parser (I would not have started this work).
The cool thing is that tags can also be used in mapping keys.

I will rephrase the "single source for template values" into something that
gives better credit, because I really meant that import mechanisms should be
the same for values and other stuff (libs, functions, ...). And aslo not only
as overlay.

Thanks for the inspiration and let's continue to try and relieve the YAML pain
around us.

------
mv1
Hold on. YAML gained popularity as a nice syntax for JSON, which itself was a
subset of JavaScript, a full programming language. Now, we are adding some
computational capabilities back to YAML? Wouldn't it be better to just execute
JavaScript to generate a JSON object and skip building a new language inside
of YAML?

------
dpc_pw
[http://memecrunch.com/meme/2HKTR/no-god-please-
no/image.png](http://memecrunch.com/meme/2HKTR/no-god-please-no/image.png)

~~~
pavon
Compared to Helm and other common kubernetes templating solutions that try to
use blind textual replacement in whitespace-sensitive language, this (or
something like it) looks like a god-send.

------
techntoke
People that hate YAML must also dislike Markdown.

~~~
moderation
Compare the YAML spec [0] to the Markdown spec [1]

0\. [https://yaml.org/spec/1.2/spec.html](https://yaml.org/spec/1.2/spec.html)

1\.
[https://daringfireball.net/projects/markdown/index.text](https://daringfireball.net/projects/markdown/index.text)

~~~
techntoke
The link you listed for Markdown isn't the spec. Here is a more official spec:

[https://spec.commonmark.org/0.29/](https://spec.commonmark.org/0.29/)

