
Introduction to Flat-File CMS - mymmaster
https://buttercms.com/blog/introduction-to-flat-file-cms-deciding-whether-they-are-right-for-you
======
tootie
My company does a lot of CMS work for clients. Everything from static site
generators up to giant enterprise tools. One thing I've determined is that the
giant enterprise tools are nearly worthless even for the enterprises they're
designed to serve. They add giant layers of complexity to service features
that look great in a demo and serve no business purpose. Then they become a
technical ghetto full of proprietary plugin code that require product
expertise and are inaccessible to the rest of the dev team. That and they cost
$100K-$1MM and more for licenses. It's possible to implement them well, but
only by applying the same kind of design principles you would to simpler
systems. Even when I'm working with a client that doesn't blink at the price
tag, I'm 100% onboard with headless or flat files.

~~~
pjc50
I'm reminded of the now-vanished JWZ post about "groupware". Enterprise
software is nearly always awful in lots of ways that the purchaser insists on.
They don't want a cheap, general solution that empowers users. They want an
expensive, high-profile, highly customised solution that railroads users into
workflows. Often it's less important what the specific railroad is, just that
it can be imposed.

Proprietary plugin code serves the need of the purchaser to feel special by
getting something customised for them. They can then pretend it's a secret
sauce for their business, despite it actually impairing employee productivity
compared to, say, stock Wordpress.

~~~
teddyh
> _I 'm reminded of the now-vanished JWZ post about "groupware"._

It’s not vanished; here it is: [redacted]/doc/groupware.html

~~~
tootie
You should delete that link. It looks like whoever owns that site left an, uh,
easter egg, for HN visitors. A rather inappropriate one.

~~~
teddyh
It seems that the moderators have silently edited my post to remove the link.
I have “referer” headers restricted in my web browser, so I did not see the
redirect.

It seems that the so-called “vanished” article would be more appropriately
named the “disappeared” article.

------
neya
I started out re-writing the core of Wordpress to Elixir around last year.
Once of the decisions I had to make was with databases. I started out with
flat files as well and realized they can quickly grow complex when you have
more than a handful articles. Eg. If you want to add some new attributes to
the flat files, they can't be backported to the older articles if the number
of articles is quite large.

Guess what? That's the problem that the SQL databases already solve.

So, to circumvent this, I searched for a database that can be version
controlled but doesn't require a separate database server. And I found one -
SQLITE.

It's been amazing so far. It can be inside my GitHub repo, while giving me
access to normal SQL queries via my ORM. With a static file solution built on
just flat files, complex querying is definitely not possible. With now over
30,000 posts in the SQLITE database, my static file generator is still
absolutely fast.

I now use my solution for a high traffic client with 3-4 million visitors a
month and it works really well for them.

One downside of SQLITE is multiple concurrent writes, but so far it hasn't
been much of a problem yet.

~~~
seanwilson
> With now over 30,000 posts in the SQLITE database, my static file generator
> is still absolutely fast.

Don't you then miss out on human readable Git diffs when posts are changed?
And now you can't edit posts with a standard editor or e.g. the GitHub web UI?

Also, what language are you using? Shouldn't reading in 30,000 posts and
building some stats/relationships/lookups on them be pretty fast?

~~~
PudgePacket
> Don't you then miss out on human readable Git diffs when posts are changed?
> And now you can't edit posts with a standard editor or e.g. the GitHub web
> UI?

You never got those with wordpress either. Seems like they're going for some
kind of hybrid. Trading some of the simplicity of standard static file
generators for some of the power of a tool like wordpress.

I quite like the idea.

Nothing to stop them from storing historical versions in the DB, then it
wouldn't be much more logic for the site generator to spit out the site at X
point in time, or a diff. Whether that's a feature you'd want is another
thing.

> what language are you using?

"re-writing the core of Wordpress to Elixir"

------
subpixel
I’ve been an API-based CMS fan for a while but at the end of the day it’s a
huge bet on the viability of the vendor, because good luck migrating away from
an API-based CMS.

No way all of the myriad vendors in the headless-CMS world survive this
downturn.

I think that needs to inform any CMS decision.

Ultimately I suspect there will be vertical integration at play with the
companies raising money to be the Wordpress of the 21st century. Wordpress is
a CMS after all.

~~~
akie
I company I worked at had 20 or so sites on a headless API based CMS.
Basically you’d enter structured data in their proprietary backend, and the
site would request that data as JSON via the API. The marketing department
hated it with a passion, because there was no way to do “one-off” things
without involving IT, and IT hated it because marketing needed permanent
handholding for the simplest of things.

After about a year of paying €8000 a month for hosting they replaced it with
one really good 100% custom WordPress theme.

~~~
mymmaster
How did this remove the need of using IT to do one-off things?

~~~
bigbassroller
WordPress has a publish button, post scheduling and revision history all out
of the box with no plugins. That right there eliminates most of the need of IT
in day to day publishing.

~~~
andrewingram
I don’t follow. Everything you’ve listed is a feature of most headless CMSs
I’ve investigated.

~~~
bigbassroller
Headless CMS may have that feature but then you have to deal with a more
complicated deployment pipeline that a marketer might not be as comfortable
with as WP.

~~~
andrewingram
There might be a misunderstanding here. Headless CMSes don't mandate deploys,
that's only if they're used in conjunction with static-site generators. A
headless CMS is just a CMS that provides content as structured data over an
API rather then pre-bound to a rendering layer. You can use WP as a headless
CMS if you use its content API.

------
Grumbledour
The thing with flat-file cms is, they are often represented as a category of
cms, but they are actually not all the same.

There is the type that just saves its data in file form, just like you would
in a db. And I never get why this would be desirable. You loose all the
benefits of a db system without gaining much.

The other type, the interesting one, is the one where you can work directly on
those flat files, not needing some kind of admin interface. Now this can be
great!

But it often seems more flat-file cms belong to the former category. I do not
know why one would prefer these if you still need a server with a dynamic
language and a web interface to edit content. Just use a db!

I never heard of headless API CMS Systems before, but it honestly sound really
silly. If you write a front end that consumes JSON, you could just as well
write a front end that made db calls. And if you must have JSON, write a
wrapper around your db. Having that stuff hosted by someone else is just peak
cloud nonsense.

~~~
wolfhumble
> The other type, the interesting one, is the one where you can work directly
> on those flat files, not needing some kind of admin interface. Now this can
> be great!

I am not sure I understand; could you please explain? That sounds like working
directly on flat html files; or are you talking about working directly with a
DB that then writes to flat files?

Thanks!

~~~
Grumbledour
I was thinking about something like grav cms.

You put markdown files in a folder and it creates html for you when the page
is requested. When you use their web based admin tools, they also just change
the markdown files. Easy to use from web and local.

I guess many static site generators work the same and we had good success
teaching our content creators some markdown. For many, it is often easier than
navigating something like drupal and working with the kinks of whatever
WYSIWYG editor.

------
blondin
very interesting how these things come and go. one day we talk about flat-file
frameworks and, for a long period of time, we forget about them. i remember
when joomla, wordpress, and the other bazillion php frameworks were all coming
about and we debated flat-files vs database-backed frameworks.

one that stood out at the time among flat-filers was textpattern -- not sure
if the very yellowish branding was the reason.

databases won over flat-files though. primarily because the web needed to
scale. it is also worth mentioning sqlite which at the time was, and still is,
a perfect middle ground.

~~~
omnimus
I am not so sure the scale is main reason. Most wordpress sites are static
content and run db+site on same server. They are scaled using caching which
sort of makes it like static website.

The same approach is used with big flat file sites. Editing on dynamic portion
and viewers see cashed site.

Many big sites actualy use cmses as static site generators. But cashing is not
that different.

------
techntoke
Hugo is a great flat-file CMS, although it doesn't come with an admin
interface or editor built in. Would love to see something as fast as Hugo that
I can just point at a directory of Markdown files with front-matter/data that
can edit/commit files from a UI using Git that runs locally.

~~~
netsharc
I've been using Hugo, but the template syntax is a pain to learn/get right...

~~~
gnabgib
The template syntax is actually from go[0] which you can find some guides[1]
for. While the Hugo website now mentions this, I'm not sure it always made it
clear the syntax is not its own (it just seems to be frustratingly light on
some use-cases). Once you understand the syntax though, it applies to more
than just Hugo.

[0]:
[https://golang.org/pkg/text/template/](https://golang.org/pkg/text/template/)
[1]:
[https://gowebexamples.com/templates/](https://gowebexamples.com/templates/)

------
ollerac
I'd like to see someone create a CMS that uses HTML files to store data and
let's you query and copy data between the files using CSS selectors. This, to
me, would be the simplest CMS.

You could just do:

    
    
        <div data-copy-content-from="#about-page .profiles"></div>
    

And that would go into your about page, copy the content from there, and
insert it into the current page.

If you built it out well enough, you could do everything using just static
HTML and CSS files and just syncing the content whenever it changes.

I've been thinking about this idea for a while. I even started my own
framework to try to make something like this work. I'd love to know what other
people think.

~~~
raving-richard
Check out Server Side Includes. They used to be all the rage. <!--#include
virtual="../quote.txt" \-->

You don't need to reinvent anything. Just use what used to exist.

(Also, you start doing SSI, and then you want a bit more power, so you switch
to PHP, then before you know it, you go all out with PHP and MySQL. Finally,
you say, "MySQL sucks, SQLite is much better and easier" (see also the other
comment about SQLite).

In the end you realize that nothing matters. We all die alone in the end.

~~~
ollerac
Yes, I've been down this rabbit hole before :)

I'm not just looking for static includes, but dynamic, real-time, editable
includes built in to a framework that uses a single syntax to describe a full
web app. I'm tired of using full programming languages for describing app data
— and how it can be edited — when so much of what a web app can do has already
been implemented a billion times.

PHP is great for that, but, in my opinion, is 500% more powerful in areas that
don't count (low-level language constructs that make it Turing complete, but
not ideal for building real-time, editable web apps) and 10% as powerful as it
should be in areas that do count (i.e. performing higher level tasks
automatically, like structuring data, syncing data between pages, moving data
around, rendering data to the page easily, etc.).

------
adim86
What are good headless CMS engines? There are so many flavors and no real
community or indication of which ones will survive, is there an open-source
equivalent to Wordpress for headless CMS?

~~~
tomduncalf
I've been pretty happy with [https://getcockpit.com/](https://getcockpit.com/)
for smaller sites. It's flexible and reasonably robust. Not perfect but if you
know some PHP and JS the code is fairly easy to follow if you do need to tweak
stuff.

------
rob-olmos
Somewhat related, I've been fond of flat-file or file-based configs as well.
Such as Nagios/Icinga has configuration files, but now seem to suggest a
database & UI/API.

File-based config made scripting the generation of the configs pretty easy and
updating for changes by just overwriting the files and reloading the daemon.

IIRC, similarly TWiki also uses individual files for the pages and configs
with RCS (Revision Control System) per-file to track changes.

~~~
xorcist
Especially since Puppet/Salt/Ansible does that part for you.

Apply the "web server" class to a node and there is nothing to forget as the
relevant config templates are applied both to the monitoring system, the
backup runner, and anything else.

Configurations that live in a database somewhere is much harder to manage as
they have to be set by a config running which needs to be written and
maintained.

------
tomcooks
I've made a PHP flat-file CMS to manage my own website and for a while it was
the very first result when searching for "fuck wordpress"

------
docandrew
Using a flat-file based system appeals to me, but one question I've never had
a good answer for is at what sort of scale does a certain database make sense?

Aside from questions of ACID, I get the idea that a lot of people jump to
databases because they're uncomfortable with file operations, frankly. But
obviously they make sense for other use cases - what is the decision matrix?
Just a gut feeling?

~~~
klodolph
If you’re strictly asking about scale and not features, the breakpoint is just
a question of whether the data fits comfortably in RAM and whether the front-
end fits on the same machine. Once the dataset is too big for RAM or once you
need multiple front-ends, you’re reimplementing ordinary database features in
whatever home-grown system you have.

~~~
docandrew
That's helpful, thank you.

------
severak_cz
I used few "hybrid" CMSs from this point of view:

\- dynamic wiki backed by a folder of markdown files

\- dynamic CMS backed by single SQlite file

\- classic dynamic CMS which dumps pages as files while editing, effectively
making them static on reading

~~~
uxamanda
I’ve been looking for a folder based wiki - was this a custom tool you made?

~~~
omnimus
Dokuwiki is working well but its pretty oldschool. Meaning not fun to develop
for.

~~~
uxamanda
Ah, was hoping for something where I could simply add markdown files to
folders and it would do all the routing for me. I have a setup that works like
this with Flask flatpages[0], but it seems like there has to be a simpler
solution out there somewhere. I've been unable to find it though. :-)

[0] [https://flask-flatpages.readthedocs.io/en/latest/](https://flask-
flatpages.readthedocs.io/en/latest/)

~~~
omnimus
Well by "wiki" it means to me that it saves history of changes and manage user
access etc. That probably requires more complicated file setup than just
markdown files.

If you want just markdown > website with some administration to edit you can
use pretty much any flat file CMS. Kirby CMS is highend but you need buy
license. Grav is ok open-source option but i don't like the admin part too
much but its atleast active. You can find many more
[https://github.com/ahadb/flat-file-cms](https://github.com/ahadb/flat-file-
cms) but flatfile cms often becomes pet project that people stop maintain so
watch out for activity.

------
patchtopic
it's not perfect, but I had a positive experience recently creating a small
web site with Publii:

[https://getpublii.com/](https://getpublii.com/)

------
ChrisArchitect
add (2019)

outdated article/spammy

