
Lennart Poettering wants to remove syslogd, use systemd journal instead - lawl
http://thread.gmane.org/gmane.linux.redhat.fedora.devel.announce/1117/
======
buster
I didn't know that systemd also provides a syslog functionality. But it makes
me wonder:

\- the wiki pages mention that the default syslog should be removed because it
makes no sense to have two solutions

\- the pages also say that for more features you should install a "real"
syslog solution (rsyslog, syslog-ng)

Question: Why does systemd then come with syslog functionality in the first
place?!

This is a honest question, not trolling, what will i gain from systemd
journal? Everything points to a) pain to switch and b) less functionality.
Whatever that is..

After reading a bit, it appears that this journal is some binary data.. Call
me old fashioned but i really like my easily transferable and everywhere
readable plain text logs.. why is it binary?!

* [https://fedoraproject.org/wiki/Changes/NoDefaultSyslog](https://fedoraproject.org/wiki/Changes/NoDefaultSyslog)

* [https://fedoraproject.org/wiki/Features/systemd-journal](https://fedoraproject.org/wiki/Features/systemd-journal)

Edit: Thanks for the explanations on the binary format, much appreciated.

~~~
gioele
The journal has been introduced as the natural way to manage logs now that
systemd takes care of all the output of daemons.

> If you are wondering what the journal is, here's an explanation in a few
> words to get you up to speed: the journal is a component of systemd, that
> captures Syslog messages, Kernel log messages, initial RAM disk and early
> boot messages as well as messages written to STDOUT/STDERR of all services,
> indexes them and makes this available to the user.

While syslogd and Co. just take care of moving the debug statements from RAM
to the disk or to a network device, the journal is meant from the beginning as
an interface for querying the logs, for example

    
    
        $ journalctl -u httpd --since=00:00 --until=9:30
    

[1]
[http://0pointer.de/blog/projects/journalctl.html](http://0pointer.de/blog/projects/journalctl.html)

~~~
TheCraiggers
Call me old fashioned... but what's wrong with grep? If you need more features
like searchable metadata, the other logging solutions are already pretty full-
featured.

I know this goes back to the old holy war of modular versus... not modular(?),
but personally I prefer the modular approach.

------
naquad
Could somebody please remove Lennart Poettering? This systemd thing is a mess
that tries to do way too much. I've had a huge pain rewriting couple dozens of
init.d scripts with custom logic to service files. Friends of mine complained
on that too. I'm really glad that systemd is not adopted Debian so I won't
have this pain on servers. Also journalctl is not a convinient tool, less and
bunch of files were working much better for me.

~~~
the_mitsuhiko
> less and bunch of files were working much better for me

Just because it works for you does not mean it works for everybody else. The
mess of log files below /var/log has been a problem for a long time. The
format is not descriptive enough, there timestamps are second resolution only,
no timezone information, and yet the times are in local timezone.

Please don't hold on /var/log/messages just because it has been like that for
ages. That should not be the justification for it existing.

> I've had a huge pain rewriting couple dozens of init.d scripts with custom
> logic to service files.

That's funny, because my experience has been that writing init.d scripts is
painful and even package maintainers sometimes get it wrong.

~~~
Karunamon
>The mess of log files below /var/log has been a problem for a long time.

How so? It follows the *NIX "everything is a file" philosophy, so they can
easily be processed by other utilities that everybody already knows.

>The format is not descriptive enough

You mean the standard filepath? /var/log/(daemon name).

You mean the actual textual format of a syslog message? Timestamp, daemon,
facility.severity, message. If your messages are useless, isn't that a
function of what's creating them, rather than what's capturing them? What
additional data do you want that you're not seeing?

>no timezone information, and yet the times are in local timezone

What other timezone should they be in? Personally I think it's easier to look
at the system clock (actually, I usually don't even have to do that, since I
have a clock in the corner of my screen session) and make a mental note what
timezone you're in rather than add two bytes to every message.

>Please don't hold on /var/log/messages just because it has been like that for
ages. That should not be the justification for it existing.

It's the thing that wants to change the way it's always been done that has to
justify itself, not the other way around. What does the systemd journal give
us that's worth throwing out decades of knowledge and muscle memory?

Configuring syslog isn't exactly a great feat of skill either.. pick your
favorite daemon, it still follows the general format of (messages from this
thing) (with this characteristic) (go here)

~~~
bitwize
_It 's the thing that wants to change the way it's always been done that has
to justify itself, not the other way around. What does the systemd journal
give us that's worth throwing out decades of knowledge and muscle memory?_

That's how modern systems are developed. That's all the justification you
need, really. Define the use cases you and your CADT buddies considered as
"modern" and everything else as "legacy", then you can say "That worked fine
in the 1970s, but modern systems don't work like that anymore."

------
peterwwillis
Side effects:

\- The journal has limitations, like what files it stores the journal in and
the [default] size of the journal, which is only 10% of the filesystem size.
Which sounds retarded to anyone who has ever created a /var partition for
logs, but anyway. (Configuring multiple syslog files is an efficient hack to
spread log data load across filesystems, which AFAIK systemd does not support)

\- All applications that were developed specifically with syslog support (such
as using the glibc syslog functions, or the command-line tools, or the flat
log files) will break. The fix is to reinstall syslog and configure it to use
a journal device that systemd exports [once you configure it to].

\- Any devices that this host was collecting syslogs for, or any syslog
collection servers this host was sending its logs to, will break. (Syslog is
the only de-facto standard [enterprise or otherwise] for sending or collecting
logs on a network, besides maybe snmp, but that's not really logging)

\- Systemd's programming-oriented control over log journals means you can only
do things it lets you do. For example, there is built-in log rotation based
both time and disk space used, but no function (that I know of) to tell it to
rotate on-demand, which is a common sysadmin task. You are removing all the
previous flexibility of syslog logs by using these binary journals.

\- If you support more than one operating system, or hell, more than one Linux
distro, you _have_ to install syslogd and configure systemd to use it, or
you'll be doing twice the work just to support Fedora's mandatory new logging
system along with every other system's syslogd. (Even if every distro switches
overnight you have legacy systems and code to deal with)

Weird how Fedora already had a project to fix logging. I wonder what happened
to that.
[https://fedorahosted.org/lumberjack/](https://fedorahosted.org/lumberjack/)

The basic proposal seems to be: I have this new logging system, so remove your
old system because mine is better. Which is no better an argument than: I have
this old stable logging system that works great, so your new system needs to
chill the fuck out until it supports all the legacy stuff.

It doesn't seem like there is any _need_ to remove syslog. The guy just wants
it to disappear.

~~~
npsimons
_Which sounds retarded to anyone who has ever created a /var partition for
log_

Oh, but don't you know? You shouldn't need more than one partition; they're
getting rid of separate /var (I only half joke; after they broke separate /usr
by linking boot/system things to it, they declared separate /usr to be
"deprecated").

 _If you support more than one operating system,_

Another one of Poettering's favorite ideas is that nothing besides Linux
matters. I may agree with him for my own personal computing setup, I think
it's flat out unprofessional to eschew portability and cross-platform
standards. Even ignoring the obvious benefits, making things portable can
reveal bugs and unstated assumptions in your design which will trip you up
later.

 _It doesn 't seem like there is any need to remove syslog. The guy just wants
it to disappear._

If there's one thing that strikes me about Lennart (and others, such as those
working on Wayland), it's what appears to be a severe case of myopia, or just
flat out willful ignorance, of systems that have come before. The egotism is
astounding.

Granted, sound under Linux was far from perfect (so I've heard; it has always
just worked for me), and PulseAudio _eventually_ got stable and efficient
enough to be be acceptable. That being said, it does feel a whole lot like
"here's how we're going to do it, shut up and take it." It would be more
palatable if they identified problems, proposed solutions, and then canvassed
the community. Hell, even just proposing solutions as patches or wholly
separate projects and letting the community vote with their feet would be
preferable to this dictatorship mentality.

EDIT: Just to be clear, I distrust Lennart (and Sievers') approach to system
software because they don't seem to think of the bigger picture or be aware of
the past. Do you know _why_ so many people are attached to the "legacy"
"deprecated" ways of doing things? Because they work, across multiple
platforms, multiple OSes and multiple paradigms (embedded, server, desktop,
cluster, mobile, and future ones we can't even imagine). Because other ways
have been tried (eg, binary logging on Windows) and failed. Yes, some things
could be improved (dependency based boot is a good idea). But other things,
such as the core modularity principle in UNIX, are there for a reason, and
should not be lightly tossed aside.

~~~
andrewaylett
My understanding is that it's fine to have a separate /usr, it just needs to
be mounted by your initrd. Trying to ensure that everything that could
possibly be needed to mount /usr is in / turned out to be a lot more difficult
than building an initrd with the relevant tools in it.

~~~
Tobu
There may be ways to do it right now, but it's hard to fight the fact that
Lennart doesn't want it to be supported. He's already thinking of deprecating
initramfs builders that don't have systemd running inside (which causes
trouble for Arch). If systemd continues to clash with initramfs tools it takes
human resources away from working on things Lennart doesn't care about.

------
coolj
It sounds like systemd journal is a good thing, especially for early boot
debugging. That said, I don't get the proposal to remove the standard logger.
Sure, the logs are duplicated...but having important data stored redundantly
seems like a good thing. Plus, you still keep the "everything's a file"
philosophy, so you can continue to access the log file in situations where
accessing a binary is unfeasible or impossible.

The example of a box where /usr is hosed and you only have access to in-memory
programs like the shell, is maybe not the best example. Perhaps a more
practical one is the need to monitor / alert on certain log entries in a
legacy or upstream system that expects to open and watch a file.

As all distros I know of include a log file rotation facility that deletes old
system logs after some set period, having duplicated data in the journal and
in traditional system log on disk seems like it would be a problem only in
special cases.

Cost-to-benefit, it seems to me there should still be a traditional syslog
provider included by default.

~~~
zb
I would wager that 99% of users would fall into one of two categories:

1) The journal is completely adequate for their needs; or

2) No default configuration could ever be adequate for their needs, since they
need to configure rsyslog to ship logs off the box.

I'm curious as to what sort of cost-benefit analysis would lead you to
conclude that everybody in group (1) should be forced to install rsyslog as
well so that the perhaps 1% of people who "need to monitor / alert on certain
log entries in a legacy or upstream system that expects to open and watch a
file" can be spared the horror of having to type "yum install rsyslog".

~~~
coolj
OK, fair enough. Going with your figure (which I think is pretty low), what is
the benefit to 99% of users from breaking tools for %1 of users? As far as I
can tell it just a bit of additional storage space. Do 99% of users actually
benefit from that extra storage?

------
diaz
For the record, as stated in that thread archlinux moved a long time ago. Some
of the mailing list threads and the annoucement:

* Annoucement: [https://www.archlinux.org/news/install-medium-20121006-intro...](https://www.archlinux.org/news/install-medium-20121006-introduces-systemd/)

* Discussion on arch-general: [https://mailman.archlinux.org/pipermail/arch-general/2012-Au...](https://mailman.archlinux.org/pipermail/arch-general/2012-August/029785.html)

* Discussion on arch-dev: [https://mailman.archlinux.org/pipermail/arch-dev-public/2012...](https://mailman.archlinux.org/pipermail/arch-dev-public/2012-August/023389.html)

And there should be older threads but I dont have time to look for them now.

~~~
darklajid
Is that the same thing?

Yes, Arch is using systemd. So is Fedora. The argument that is linked wants to
remove a syslogd implementation from the default installation.

Right now, the default is journal (cannot be removed, if you're using systemd
as explained in that discussion) _plus_ syslogd. The proposal asks for journal
w/o syslogd by default (but available as a package of course).

That said: My Arch systems don't run a syslogd either, so .. maybe they
switched 'completely' in that announcement?

~~~
diaz
When systemd became the default in Arch a syslogd implementation became
optional.

Just after a few weeks after installing systemd and running it side by side
with syslogd I completly switched to a pure systemd + journald without the
syslogd.

That's how I remember it at least, I may be wrong in some point.

\---------

EDIT:

Ok, I went and searched a little more, here is what I found related to systemd
+ journald + syslog discussion:

* systemd's journal and syslog: [https://mailman.archlinux.org/pipermail/arch-general/2012-Fe...](https://mailman.archlinux.org/pipermail/arch-general/2012-February/025067.html)

* Arch wiki, about journald: [https://wiki.archlinux.org/index.php/Systemd#Journald_in_con...](https://wiki.archlinux.org/index.php/Systemd#Journald_in_conjunction_with_syslog)

I think it's interesting to know how Arch did the switch and how it is used,
considering this current discussion in the fedora mailing list.

------
andrewaylett
Can I just point out, for all the Lennart-haters, that he doesn't have the
final word in what gets implemented in a distro. It's not like he's shutting
down any syslog projects either, or threatening to kill the maintainers' cats
if they don't use his software. Distro maintainers move to his software
because they think it's an improvement over what they had before.

Case in point: Lennart works for Red Hat, but Arch is actually further along
than Fedora in adopting the Journal.

Personally, I removed my syslog daemon about two months ago, and haven't
missed it. I also switched to Fedora back in the new year (from Ubuntu)
specifically because I wanted to try out Gnome 3 and Systemd. Loving it so far
:).

------
gnu8
I'm waiting for these jokers to replace /etc with a database that requires an
editor that only runs on Linux x86/x64 to modify anything.

~~~
airencracken
They already store systemd configuration in /usr/lib/systemd and symlink to
/etc where you can override stuff, so you're not that far off.

------
madhouse
While I'm not a Fedora user, I wish them good luck, and hope the change goes
through. It would be of great benefit for other distributions aswell, and
perhaps we could get rid of traditional syslog for good at some point...
(wishful thinking, I know).

~~~
mordae
According to Lennart, Arch have already moved to journal-only default so
Fedora is not really leading there...

~~~
madhouse
Never said it would be leading. But more distributions switching to journal-
only is a good thing, because that makes it that much easier for the rest to
follow suit.

------
the_mitsuhiko
More power to him. The Journal is a much more interesting and useful approach
than syslog.

------
mercurial
He just wants to remove syslogd from the default Fedora install, to be clear.

~~~
qznc
Also, the title is misleading ("instead"), since systemd logging is already in
use for several releases.

------
txutxu
Component A) Role: init the system

Component B) Role: log dispatching/receiving/sending/filtering

And now what?

A component which partially does in a non _standard_ way, incompatible with
all third party tools (in the sense the tools needs to be modified or
adapted).

What is next, systemd will replace cron too, in powerpoint format?

Reading the whole thread, I have a new concept in my mind... "UEFI cookies",
didn't know about them.

~~~
rapala
>What is next, systemd will replace cron too, in powerpoint format?

systemd has some cron-like features:
[http://www.freedesktop.org/software/systemd/man/systemd.time...](http://www.freedesktop.org/software/systemd/man/systemd.timer.html)

~~~
txutxu
Thanks for the information.

Really does not remember to me my first impressions with cron :)

I don't doubt of the quality of this guy and it's ideas and solutions, I
cannot say I'm better. And I've seen many "weird for me" things across the
years coming from upstream to my systems.

I'm tired of fight with abstraction layers to try to make things simple. But
in my work I use to be condemned to use whatever upstream picks across
different versions.

Again, thanks for the information.

------
nodata
Isn't journalctl that thing that auto-pipes? _shudder_

Fewer files, AND auto-piping! Not very unixlike.

------
Tobu
I haven't seen it in the discussion, but systemd's journal has horrible read
performance. Apparently it's poorly indexed, or the disk layout has issues.

------
kps
“Those who do not understand Unix are condemned to reinvent it, poorly.” —
Henry Spencer (ca. 1987)

~~~
makomk
They're not even reinventing Unix poorly - they're reinventing the Windows
event logger badly, and I'm not aware of anyone that ever really liked that in
the first place, especially Windows admins.

