
Show HN: Metalsmith – A pluggable static site generator - ianstormtaylor
http://www.metalsmith.io/
======
sneak
I like that static site generators are basically the Cups And Balls of our
craft.

It's so well understood and constrained of a problem domain that we can now
ignore the practical considerations and go all-out with the art itself. I feel
like this design and api is a great example of that, much like Penn and
Teller's Cups and Balls with clear cups[1] - wonderfully creative innovation
within a completely and totally solved problem domain.

[1]
[https://www.youtube.com/watch?v=O_n3Zb3bW3g](https://www.youtube.com/watch?v=O_n3Zb3bW3g)

------
Touche
One problem I see. On the one hand you say:

> All of the logic in Metalsmith is handled by plugins.

But on the other hand, you say this:

> Each plugin is invoked with the contents of the source directory, with every
> file parsed for optional YAML front-matter, like so…

The YAML parsing should be a plugin as well, some of us have existing JSON
front-matter files.

~~~
icebraining
A JSON file is a YAML file, so what's the problem?

~~~
asb
I'm not sure why this is getting downvoted (well, maybe the tone), it's true
that YAML is a superset of JSON
[http://en.wikipedia.org/wiki/YAML#JSON](http://en.wikipedia.org/wiki/YAML#JSON)

~~~
ianstormtaylor
That's really cool actually.

------
8ig8
Thanks. Looking forward to trying it out this weekend. My current generator,
which I'm generally happy with, is DocPad.

[http://docpad.org](http://docpad.org)

------
cristianpascu
One thing I don't like about jekyll (unless I'm missing something), is that on
site generation, the last modified time stamp of the files gets updated too
even if the file content hasn't changed. This way a FTP program like Transmit
will not be able to synchronize only the modified files.

~~~
sneak
Just about everything that supports ftp supports ssh+rsync. I'm not
apologizing for the bug, but rsync+ssh is a sane default for synchronizing
everything everywhere these days and sidesteps the problem almost entirely.

------
xianshou
If this really works, I would love to see it replace the hellish jumble of
team-editable documentation. I've seen Confluence, PBWiki, Google Sites, and a
smattering of others used to no good ends...can we please switch to this now?

~~~
ianstormtaylor
I'm actually in the process of converting our Segment.io docs to use it right
now :) makes it way nicer for everyone* folks to just be able to edit
Markdown, but still have the power to do lots of custom things to make the
experience better.

* I was going to say for "less-technical" folks but then I realized that even technical people shouldn't have to be subjected to our current tangle of Jade files!

------
dangoor
Could be compared to assemble.io and stylistically reminds me of Gulp.

~~~
justarandomanon
Gulp was the first thing I though of when looking through the examples.

Edit: In fact, could this whole thing just be a gulp plugin?

~~~
andyfleming
I don't see why not. My thought was "why even have this when you could just
build the plugins for gulp?".

~~~
ianstormtaylor
The problem with Gulp is just that it just adds too much extra cruft into the
mix that isn't really necessary, mostly around running tasks from the CLI. Our
general thought for build tasks is that all of that should be in a Makefile
which will nicely handle mtime checks and everything for you, and is available
on pretty much every setup out there.

The simplicity is nice because you can read through Metalsmith's source and
really understand everything that it's doing very quickly:
[https://github.com/segmentio/metalsmith/blob/master/lib/inde...](https://github.com/segmentio/metalsmith/blob/master/lib/index.js)

------
mercurial
It appears to be indeed both extremely simple, and extremely composable. How
well does it handle large collections of files?

~~~
ianstormtaylor
I haven't tried it on _crazy_ amounts of files, but it's just using node's
basic async I/O under the covers, and reading once. If you notice any
sluggishness let me know!

It's also greatly impacted by what plugins choose to do. I had an extra clone
call (literally cloning the buffers for each file) in the templating plugin at
one point that like 50x'd the build time :) Everything I've done so far though
has sub-second build times—quick enough that I've been building on every
request[1] in development which makes things super simple.

[1]:
[https://github.com/segmentio/metalsmith.io/blob/master/serve...](https://github.com/segmentio/metalsmith.io/blob/master/server.js#L15-L18)

------
shortformblog
This is really intriguing. I also recommend HarpJS
([https://www.harp.io/](https://www.harp.io/)) as well, which has some
impressive pre-compile features.

------
sgdesign
This looks very cool. I like the focus on plugins, it would be pretty awesome
to have a flexible static site generator with an active plugin ecosystem.

------
andrewflnr
I like this design. It kind of goes in the same direction as my
github.com/andrewf/filtdir while being significantly more refined. This seems
to make the whole directory structure available to plugins, while my tool only
works one file at a time. It might even convince me to switch.

------
jon49
A functional type approach. It will be interesting to look more into it. It
would be nice to have a .map, .filter functions (if they don't already exist).
So, if you don't want to rebuild everything you could do a .filter(htmlDate <
mdDate) type workflow.

------
ricardobeat
Not just a nice tool, but a great implementation. Simple, lean code, no
promises or anything fancy.

~~~
tobobo
The overall structure of the thing looks pretty promise-y to me.

------
Kiro
How do I get this working on Windows? I've installed it with npm install
metalsmith. Now what?

~~~
roryokane
You’re right, the “Install It” section should make that clearer. Anyway, there
is an explanation of basic usage if you follow the link to “CLI”
([https://github.com/segmentio/metalsmith#cli](https://github.com/segmentio/metalsmith#cli))
in that section. It says you can create a `metalsmith.json` file that lists
source, destination, and plugins in the described format, and then run
`metalsmith` from the command line to build your pages according to the
configuration file. And I think you will probably also have to install any
plugins you use beforehand. Plugin installation instructions are all in the
plugin READMEs – they are basically all just `npm install <some-package-
name>`.

But it’s harder to figure out how to create my own local plugins and make sure
Metalsmith is able to see them. And the documentation should make it clearer
how to _use_ the JavaScript API, in the context of a static file generator,
where most people are not thinking about writing a program. It took me a bit
of thinking to realize that you would have to create a `whatever.js` file
inside the directory containing JavaScript code with
`Metalsmith(".")….build()`, and then just run it with `node whatever.js`.

~~~
ianstormtaylor
Sorry about that! Just updated the Readme and website to hopefully make that
clearer. And I've added Readme's to all of the examples[1] now too, some of
which use the Javascript API and some the CLI.

[1]:
[https://github.com/segmentio/metalsmith/tree/master/examples](https://github.com/segmentio/metalsmith/tree/master/examples)

------
esquivias
Somewhat relevant plug:

I have a similar weekend project that is aimed towards generating static
markup. The syntax looks a bit like haml with simple to use mixins, includes,
and variables.

[http://rubygems.org/gems/aml](http://rubygems.org/gems/aml)

~~~
roryokane
You should have linked to its home page
[https://abstractmarkup.com/](https://abstractmarkup.com/). The RubyGems page
does nothing to sell me on why I should bother installing your gem.

------
aram
Did anyone else manage to install it? I'm getting a "shasum check failed"
error.

The project scaffold generator part sounds pretty interesting because I needed
it pretty often and eventually had to build that for myself.

~~~
ianstormtaylor
Just republished! Can you let me know if you still see it? Sorry about that :$

~~~
BrandonSmith
Got the checksum error, too. Successful after the republish.

------
rainburg
Couldn't choose between Jekyll and Middleman, but now I think I'm going with
Metalsmith. No ruby, understandable plugin structure… I'm sold!

------
rayshan
With sooooo many static site generators (all very well done too), would be
awesome to have a side-by-side by-feature comparison table.

~~~
rayshan
Apparently there are many aggregation efforts, but no by-feature comparison.
[https://github.com/jaspervdj/static-site-generator-
compariso...](https://github.com/jaspervdj/static-site-generator-
comparison/issues/13)

------
caiob
I fail to see the big advantage of this over its competitors.

~~~
ianstormtaylor
Yeah it depends on what your use case is to begin with. If it's just the
simplest blog with a running series of Markdown files, then really any of the
static site generators will do. But once you get into trying to implement some
more advanced features then you run up against the limitations of most (if not
all) of them because they assume way too much up front. A couple real-world
examples from us at Segment.io are:

Documentation - for our docs[1] we want to be able to use the same simple
static site generator without having all of the blogging logic. Basically the
nesting of the files should result in the nesting of the URLs. But we also
want to be able to tie in metadata that we have in our database about all of
our integrations. And we'd also like to be able to write custom handlebars
helpers that turn a simple JSON object into a widget that renders API calls in
any of our supported languages.

Academy - for our academy[2] I really want to get to the point where we can
generate PDFs for each of our articles and being to re-distribute them that
was as eBooks (or potentially for a collection of articles) because that kind
of thing appeals to enterprises who are looking for guidance. And we could
even end up doing the same thing with our docs pages. And then we also want to
have custom handlebars helpers for

Blog - for our blog[3] we want just the most basic implementation, although
maybe with some niceties about author metadata to load in avatars and such.

Whenever you try and get into additional features that weren't considered by
the original "static site" (or worse "static blog") generators, you usually
end up building really cludgey code, if it's even possible.

So with Metalsmith we avoid all of that, because the plugins can do whatever
they want, and it's super trivial to add local plugins to the mix if you're
cooking up something which you know is unique to just you.

And the last thing was that we were sick of having a Ruby dependency (with all
of the associated slowness) just to build our blog with Jekyll. Basically was
increasing build times by an order of magnitude.

[1]: [https://segment.io/docs](https://segment.io/docs) [2]:
[https://segment.io/academy](https://segment.io/academy) [3]:
[https://segment.io/blog](https://segment.io/blog)

------
Touche
This is a nit, but non-constructors should be lowercase.

~~~
ianstormtaylor
It actually is a constructor, just lets you omit the `new` keyword if you
choose since I think it's nice not to have to do that sometimes.

------
sdegutis
So simple, yet so powerful.

------
borplk
Very nice. Well done.

------
fredsters_s
Awesome.

