On: YAML: Probably not so great after all

[mjevans]

(suggestion) Drop the 4th requirement.

Having to close contexts is a VERY good 'sanity check' to see if something is malformed or not.

If appending is necessary make the parser handle multiple copies of the namespace and merge them upon output. Unknown keys and sections should also always be copied from input to output (this is how you embed comments).

[ridiculous_fish → mjevans]

I'm interested, can you explain more about the merging idea?

To clarify the requirement, history could be a JSON array of objects:

    [
        {"cmd": "git checkout", "when": 1234 },
        {"cmd": "vagrant up", "when": 4567 }
    ]

To append an entry to this file and keep it valid, one must locate the closing square bracket and overwrite it. That work is what I hope to avoid.

[ainar-g → ridiculous_fish]

Why not use a “stream” of objects?

  {"cmd": "git checkout", "when": 1234}
  {"cmd": "vagrant up", "when": 4567}

Not sure about other languages and libraries, but Go supports this out of the box[1]. And while we're at it, why not CSV? That can be processed with awk.

  #cmd,when
  "git checkout","1234"
  "vagrant up","4567"

[1]: https://play.golang.org/p/sTN9z4Kv3DB

[ridiculous_fish → ainar-g]

CSV is fine for simple cases but has issues with versioning (adding new/optional fields) and nested data like arrays.

The object stream idea is apparently supported widely and seems pretty strong. Thanks for the suggestion!

[zenexer → ridiculous_fish]

I use JSON streams a lot with command line tools. Keep in mind that it’s limited in that each object must consume a single line. This allows you to recover from syntax errors in a single entry; each line is a fresh start.

Have you considered SQLite? I know it’s not a friendly text format, but it alleviates a lot of the issues with the “append to a text file” approach, such as concurrency. It’s great for this sort of thing.

[mjevans → ridiculous_fish]

Loosely running with your example:

A file already exists with this content:

[{"cmd": "git checkout", "when": 1234 }]

Another tool wants to add a setting/element/etc, and simply creates a new config object with only the change in question included and APPENDS the existing config.

[{"cmd": "git checkout", "when": 1234 }][{"cmd": "vagrant up", "when": 4567},{"comment": "Comments, notes, etc are kept even if they don't validate to the recognized configuration options."}]

A configuration validator / etc loads the config and merges these in state-machine order, over-writing existing values with the latest ones from the end of the stream of objects and then determining if the result is a valid configuration. (Maybe file references fail to resolve / don't open / there's some combination of settings that's not supported...)

    [
      {"cmd": "git checkout", "when": 1234 },
      {"cmd": "vagrant up", "when": 4567},
      {"comment": "... actually kept as above"}
    ]

[scintill76 → mjevans]

> Unknown keys and sections should also always be copied from input to output (this is how you embed comments).

Better than nothing I guess, but I'd say just use a syntax that supports comments.