I know you can rig up systemd to act like inetd or cron, but perhaps you would be better with an actual service (how hard is quartz?) than baking your jobs into systemd units..
That said, I only use Linux in prod for enterprise stuff that depends on it, everything else is on bsd so you can probably guess my opinion on systemd ;)
I admin a number of ubuntu servers running systemd, and they all shut down/reboot pretty much instantaneously.
Being able to specify date ranges instead of grepping around multiple files. `journalctl -u <unit> -s <start> -e <end>` instead of horrid combinations of grep and gzcat.
"Okay," you say, "but that's all for logs on the host. Shouldn't you be using Splunk or an ELK stack?"
Journalctl can export logs in a json format that can be natively consumed by splunk for free metadata markup. I wrote some container sidecars to do exactly that, and it worked great.
The only thing I don't like about journalctl is that it doesn't line wrap by default.
Not that it is bad, but you don't have to use it.
The problem is of course when you need to do something that isn't supported by SystemD, in which case you're in for a world of hurt as the system constantly fights you.
For example, I had a requirement to make sure some machines randomized their MAC addresses when connecting to untrusted WiFi in Ubuntu 14. Turns out this functionality was broken in NetworkManager and trying to go around its back was a huge huge headache because the system was relentless in checking the MAC of the interface and setting it back to the physical one.
There are other times where it bites you in the ass too, like writing a raw image to a SD card. In the old days you would just dd the image onto the card, but you can't do that anymore because shortly after you write the filesystem header to the card SystemD will notice the new filesystem attempt to mount it, killing off your dd process.
I always try to work with SystemD now because it's so much harder to fight it.
Could you elaborate on this? I develop an embedded OS which regularly flashes itself while running systemd (we support two methods: 1. write partition table to disk, write fs header to partitions, explicitly mount fs and write file content to fs; and 2. dd pre-built image) and I have never seen this happen. We have also flashed (via dd) our images onto cards from within Ubuntu 16.04 and 18.04 and never seen this.
In your specific case, if you were using systemd-networkd you could specify MACAddressPolicy=random in your link file to achieve what you wanted.
Likewise, systemd doesn't prevent you from running ALSA instead of PulseAudio. The flexibility is there. Better documentation/best practices would be helpful.
It can monitor many things including ram usage, and manages dependencies. Also it is everywhere by default - one less moving part I have to manage.
Love it or hate it, the whole idea of systemd is to centralize all task-ish management, whether one-offs, long-running, periodically-running, static resources, logs, etc.
- Small is beautiful.
- Make each program do one thing well.
- Build a prototype as soon as possible.
- Choose portability over efficiency.
- Store data in flat text files.
- Use software leverage to your advantage.
- Use shell scripts to increase leverage and portability.
- Avoid captive user interfaces.
- Make every program a filter.
> - Store data in flat text files.
Storing encryption keys or the like in flat text files is surprisingly hard, at least in C. I don't think it's even possible to use sscanf(3) without unexpected explosions, and otherwise you get to hand-roll your hex parsing code yourself (Base 64? Get yourself a library).
Working with that sort of file in C is trivial, you either open() and read() from that file, or mmap() it.
Storing keys (or other binary) data is _exceptionally_ easy with C. What's hard is _multiplexing_ it with other data in the same file, but you probably don't need to:
But yes, C is very weak on converting data from one encoding into another. You end-up always having to write your own encoder/decoder.
You should be able to convert between binary and base64 on memory more easily than during IO, and for encryption keys the buffers all have known size limits. As a rule sscanf will add security vulnerabilities to your code, so one should better avoid it if you do not trust your inputs.
openssl's libcrypt has Base 64 routines in it.
Today I wanted to get systemctl list-unit-files give me the full path to each file. Couldn’t figure it out.
It also often fails to admit when it is failing, for example, nobody I know uses journald as the main or only logging system, and are still piping to local (r)syslog(ng) and then to elk/splunk, etc, because journald is simply not up to par.
In general, and I'm just being honest here, besides these mostly work-aroundable issues, it is the seeming disingenuousness of those who automatically rail against anybody who criticises systemd that makes me not like it. Call me a contrarian, but I hate being told I must like X and if I don't I'm dumb and behind the times, and my nature is to push back against that.
In that vein, I support the efforts of Devuan and similar projects if only to support a counter-balance.
Could you expand on what's not up to par in journald vs. rsyslog? Watching textfiles for changes vs. receiving data on a non blocking socket is definitely _not_ more efficient (nor "better" in my opinion).
As the tooling catches up this gap will close. For example the json output of journald can be really useful, that said, most of the tools still require some change of the output from the binary format which they can't read to a text format whether that be json or syslog... and the binary format in general being a general pain point if (read: when) things break (such as corruption of the whole journal past corruption point vs one section of text in syslog).
It's worth remembering that journald was designed for local system use. It's just one more of the reasons I had some hesitation with systemd but got shouted down by everyone on the systemd bandwagon.
If you're in ops and you're mostly managing that fleet but you also have to look after some non-docker stuff (say, an old artifactory or some qa tool or other) then probably the time to learn how it works is before it breaks.
My 0.02$ -- If you can avoid it, do so.
Systemd is the best way I know to handle programs that can fail randomly, with complex dependencies. Just tweak the restart limits on the service. You can manage the whole set of dependencies. It can even be instructed to reboot the server.
However, documentation could be better. For example, I have not been able to use yet the new feature allowing services to join existing namespaces not created by systemd (so not JoinsNamespaceOf=, but
NetworkNamespacePath= ; cf otherwise https://cloudnull.io/2019/04/running-services-in-network-nam...)
But systemd is still a bit new. Documentation and best practices will improve.
EDIT: actual technical content get downvoted as usual, while opinion pieces about systemd being the antechrist fly.
Prime example: the "best practice" of not letting usernames start with numbers is invented on the spot by a SystemD dev to cover his ass: https://github.com/systemd/systemd/issues/6237#issuecomment-...
Like all software, if you didn't test it to work, why should you expect it to work? Cron can fail silently as well if don't set MAILTO. Not all servers start out with working outbound mail, which is something you have to setup for both Cron and Systemd as well.
I agree the OnFailure= recipe could be documented better, although I found an example easily enough when I looked up how to have systemd timers send email on failure. I use a script which mails the output of `systemctl status` on the service that fails, so I get logs of what happens.
I've a number of questions about systemd on StackOverflow sites and there are definitely some FAQs about best practices related to getting started with setting up services.
From the cron man page:
> Any output produced by a command is sent to the user specified in the MAILTO environment variable as set in the crontab(5) file or, if no MAILTO variable is set (or if this is an at(1) or batch(1) job), to the job's owner.
I think the author meant that
Without mail, journalctl and even better, systemctl list-timers will tell you about failed crons.
Can you do something as good with at and cron?