

Building Building Blocks - gwintrob
https://segment.com/blog/building-building-blocks/

======
tlrobinson
There's some neat tools here, but it also feels like a bit of NIH. It would be
interesting to hear why these are better than, say, Gulp, Yeoman, Hubot,
Casper.js, etc.

~~~
ianstormtaylor
Absolutely. It's a fine line. We haven't always made the right call for sure.

I think Metalsmith and Khaos are pretty good examples of why we feel the need
to build our own.

The crux of it is that the alternatives (Yeoman and Jekyll) aren't designed to
be augmented without opinion. They are designed as "solves all your needs
tools and gives you the preferred workflow".

Metalsmith doesn't claim to solve all your needs. It actually doesn't solve
any of your needs out of the box, unless you need a really complex way to copy
files from a directory to another without doing anything else to them. It
defines an extremely simple core—an API that extendability can be based
around—and then hands over all of the logic to plugins. Which means it's
pretty much never going to get in your way.

Similarly with Khaos—which we haven't really perfected yet—we're looking for
an extremely simple core API design that people can augment to solve their
scaffolding needs. We want it ourselves to scaffold integrations for Segment,
to scaffold new Node and Go projects, we're even thinking about scaffolding
Segment-branded PDFs.

Yeoman doesn't think that way, evidenced by this paragraph from their landing
page:

> Through our official Generators, we promote the "Yeoman workflow". This
> workflow is a robust and opinionated client-side stack, comprising tools and
> frameworks that can help developers quickly build beautiful web
> applications. We take care of providing everything needed to get started
> without any of the normal headaches associated with a manual setup.

Khaos doesn't prescribe any workflows or client-side stacks, it just lets you
scaffold things. What you want to scaffold is up to you, as it should be. You
can use it for eBooks for all we care :)

Khaos also happens to be built on Metalsmith. It's just a series of plugins
that run before the files are written to an output directory. That wouldn't
have been possible with Jekyll, we'd have had to roll our own again inside
Khaos or do some incredibly hacky shit to get it to work.

\---

All that said, some of our projects suffer from NIH. And when they do it's
usually clear over time since we get sick of dealing with them and we use
something that has more widespread support behind it. But for others, we're
super glad we've rolled our own because we get tired of other people's
opinionated software :)

There are definitely lots of existing pieces of software that we like to use
though. I've been thinking recently it would be cool for companies to have a
Chrome Extension that the engineering team could use that specifically marked
GitHub repositories as "liked/disliked" in terms of adhering to the companies
design ethos. That way we could keep track a lot easier of the stuff we like
to default to, so that we don't have to roll our own all the time.

------
covi
Do you guys face issues with the increased mental complexities this approach
brings? I am a bit shocked to know there are over 1000 repos in the account. I
can assume developers typically need to keep at most tens in their working
set, but 1k does sound a lot to manage.

~~~
pkrein
Big monolithic repos actually increase the mental complexity more in our
opinion. Mental complexity really comes with how much of the system you're
holding in your head at once. And with smaller repos you might be dealing with
10-20 repos on any given day, but that's only 1-2% of our codebase/system. The
other 98% is ignorable. So we end up holding less in our heads, assuming we've
abstracted things correctly.

It did get annoying to deal with the mechanics of lots of repos. So we built
tooling to make that easier. For example, CLI commands like "goto
analytics.js" will clone and take us to the local copy of the repo. And
"publish patch" handles all the mechanics of updating History.md from the git
log, incrementing the version appropriately in package.json and
component.json, tagging the commit and releasing to github and npm. Khaos,
also mentioned in Sperandio's article, helps us template out new repos
quickly. With a few pieces of tooling like that you can move pretty fast
across lots of repos.

------
awhitty
Slightly unrelated, but whenever I load a page from Segment for the first
time, it renders without a stylesheet. Does this happen to anyone else?

~~~
Nzen
On all browsers? Old browsers? Mobile?

------
siyegen
With 1000 repos I'm curious how you manage dev environments. Is it just cli
tools to help, or are you using something heavier?

------
TheMakeA
I actually independently discovered Nightmare today while this was open in
another tab... very cool!

