
Console, the vital tool that Apple abandoned - ingve
https://eclecticlight.co/2020/06/14/last-week-on-my-mac-console-the-vital-tool-that-apple-abandoned/
======
saagarjha
> Instead of building on those unique strengths, Apple has provided token
> support which must leave even its own engineers wishing for better, as they
> wade through the logarchives supplied in sysdiagnose dumps.

Lol, no. Unified log was designed first and foremost for Apple engineers and
through some convoluted series of events that I am evidently not privy to
ended up becoming exposed to developers as well. Except it was pretty much
useless to developers because _they couldn’t figure out how to use it_. And
not because the tools to use it effectively weren’t there, no, not at all;
Apple engineers rely on them heavily (there are private, easy to use
frameworks to work with log archives; the new, public OSLog framework that
came out last year has an admittedly useful subset). No, the problem was that
nothing was documented, most of the stuff that Apple was using to parse the
logs and reassemble them remained private, and without these it’s hard enough
to slog through the log stream let alone when you compound it with the many
idiosyncrasies like private logging or trying to figure out which subsystem
things are coming from. Really, Console is just the cherry on top because it
gets even less love that logging guts itself because nobody has to use it
internally.

~~~
barrkel
There's a certain kind of arrogance that comes from assuming that proles - I
mean end users - couldn't possibly understand our technical logs, and if they
have a problem they should come to us, or wait until we've analyzed the
telemetry and pushed a fix.

Quite frequently, logs show up things like missing files and directories,
incorrect permissions, unexpected configurations etc. which can be fixed by a
sufficiently technical end user, without needing to go through support.

There's a similar problem with error dialogs. Product people always push to
keep technical details out of those dialogs, even if technical users would
find the details useful for accomplishing their goals, figuring out what went
wrong and working around it. It's more forgiveable for dialogs, _as long as
there are logs_.

When the logs are similarly indigestible, the software is simply less usable
by technical users. Quality better be damn high, because if it breaks, and
there's no way to look into the hard shiny shell, it can generate an amazing
amount of frustration to people who are used to making stuff work.

(Of course the % of users who are sufficiently technical is decreasing all the
time.)

~~~
saurik
I will also note that leaving technical information in dialog boxes is better
even for non-technical "end" users, as it isn't like the app worked anyway (so
the user isn't satisfied right now) and it allows those users to screenshot
the dialog and show it to their friends or paste it to a forum and have other
people help them (which I have seen people do constantly with my error
messages, into which I always provide the most low-level raw/useful
technically information I can about the error).

~~~
ci5er
I've always been baffled by this. Especially in pop-up dialog boxes, you can
have drop-down arrows hiding the ugly bits (but that said, a core dump is
usually not _that_ useful).

I always thought: "Something Happened" might be the worst, ever, but I
recently came upon this when setting up a Dell Desktop running Windows 10,
trying to use a wireless keyboard. (I learned to always keep a USB keyboard
around for bootstrapping purposes):

    
    
      - "No keyboard detected, press any key to continue…."

------
fit2rule
Console.app always gets in my way whenever I'm trying to watch the status of
some self-reporting processes. I just find it utter junk to use in the
development flow - and even for trying to debug general purpose computer
issues, its nothing but frustration for the average user.

I think there is a deep-seated bias at Apple against TUI's, and this is well
and truly expressed in the flaming pile of junk that is Console.apps' GUI.

For me, __tail __and __grep __are always the better option when it comes time
to look for messages, or try to observe what 's going on with a system.

That said, it is also very shocking to sit and watch an idle MacOS machines'
logfiles. Either most MacOS software is barely held together with bubble gum
and duct tape, or there are a lot of messages being spewed by folks who forgot
to turn their terrible debug logging off ..

~~~
saagarjha
Logging on macOS is extremely lightweight (lots of fancy stuff, much of which
I don’t understand, there’s some in-memory stuff to prevent disk writes,
settings stored in the commpage) if you don’t actually look at what comes out
the other end, so there’s a bias towards “log a bunch and pick it out of a
sysdiagnose”. You’re supposed to filter on stuff except Console is not very
good at that.

------
coldcode
Console is one of those things that as an iOS developer drive me nuts. It's
absolutely a piece of trash. While Xcode drives me nuts as well, its at least
usable (and 12 so far is much improved; although it only took 12 releases to
make code completion work). It's like Apple decided to budget $XXXB to make
things, and then decided that there was no additional money for Console.

One decent programmer could make Console a useful app.

~~~
saagarjha
I don’t think it has a full engineer assigned to it.

------
IfOnlyYouKnew
I'm confused... When I start Console (on Catalina), there is a sidebar with a
section labeled "Reports" which contains several groups of log files. It
changes depending on what exists, but I currently have "Crash Reports",
"Diagnostic Reports", "system.log", "Spin Reports", and a few more.

One of them is "Log Reports", which lists every log file in /private/var/log/
__.

Console.app also makes plenty of use of structured data: right-click on a log
entry (in the live-scrolling main view), and it offers you a dozen different
facets to filter the view: hide/show by
Thread/Application/Subsystem/User/Category/etc,

I've actually thought about using Console for development. It's far better at
least compared to my current go-to "tail -f"

~~~
saagarjha
Console certainly looks pretty but you’ll find very quickly that if you
actually want to _use_ it it’s really not that great. First off, streaming
logs takes so much CPU that in Big Sur when you open the app it just shows a
pane warning you of the performance implications of actually “turning on”
logging (which itself is kind of stupid, because one of the useful features of
unified logs is that they store log messages from the past efficiently). And
then filtering is kinda janky, and actually viewing the content of a message
is a pain, and good parts of it often get censored, and…

(By the way, I have yet to see anyone use the other logs in the sidecar
because it’s easier to just open that folder in Finder.)

~~~
jupp0r
Big Sur is in Beta. The process you described is quite fast on Catalina.

~~~
1123581321
Developers need the developer beta to work on their apps right now. He’s not
using the beta for fun. Apple, realizing this, could have improved the
performance of this developer tool.

~~~
mixedCase
While I like to crap on Apple for all the things they do wrong on their
overpriced walled gardens, I can still excuse a performance regression in a
beta that is trying to add proper support for a new architecture.

~~~
saagarjha
I don’t think it’s significantly worse, it just seems that the warning is now
there to indicate something that was always like that.

------
makecheck
I remember years ago there was a very obvious console message (related to an
invalid date) that was spamming the system log frequently for _at least_ a
month, across multiple minor OS updates, in nearly every application. It was
actually noticeably impacting the CPU because it appeared so often.

Now I’m wondering if this console “rewrite” happened around the same time,
perhaps in response to it. Maybe after seeing one difficult-to-wrangle log
message, they concluded that gaining complete control over all logging was the
only solution?

In any case, they went way too far. Console is effectively useless now; I got
so tired of opening it up and finding _nothing_ of value that I stopped even
looking, and just apply every other possible debugging method first.

Their push for multi-process solutions like XPC have made logging even _more_
useless because by default the new log system refuses to show output from sub-
processes AT ALL. So imagine an app that previously had a nice sequential log
for in-process actions A, B, and C; after migrating to sub-processes, now you
see only the output of action B because the A and C actions are in different
sub-processes! While there is a way to enable the output from all of them,
it’s not the default and it’s not obvious and it’s just another stupid
debugging hurdle.

------
jurip
And of course the API side of logging, i.e. OSLog, is still unusable if you
have exotic needs like, I dunno, sending the logs somewhere where you, the
developer, might take a look at them. Feels like having sessions telling us
about improvements to logging without solving that one first is a bit
pointless.

~~~
pjscott
Unless I'm misunderstanding your post, try something like this:

$ log stream --level debug --process MyProcess > mylog.txt

For lighter-weight logging, replace "debug" with "info" or "default". If your
program has debug symbols, you can also toss in the --source flag to add the
filename and line number for each logging statement, if available.

The new logging is a significant improvement once you get used to it, IMO.

~~~
jshier
GP meant from remote devices, like a phone, to somewhere the developer can see
it, like a logging service. OSLog can't even log to a local file, making it
fairly useless for in app logging even in testing, as there's no way to see
your app's logs in the app, no?

------
thecureforzits
As a non-mac user I am fascinated by blogs like this. Does Apple care? Like,
if they just ignore the issue, will enough people stop buying their products
that they will even notice? When Apple releases a new product, it seems to me
the person who wrote this post will buy it regardless and there is absolutely
no incentive at all for Apple to address the problem.

~~~
ehvatum
The years of Apple laptop keyboard issues indicate profound dysfunction in
Apple’s “caring” methodology.

------
thanpolas
genuine question, is there any kind of logs that console app has access to
that you can't view using unix commands? (i.e. tail -f /var/log/messages)?

~~~
xinsight
Not all log levels are persisted to disk (info and debug, i think?) so yes,
there is info that you cannot get from `tail -f`

~~~
pjscott
However, you _can_ get all this information with the `log stream` command.

------
PaulDavisThe1st
This doesn't even mention that at some point in one of the last 4 or 5
macOS/OSX releases, Apple made it so that processes started by launchd (i.e.
GUI apps) no longer have stdout/stderr file descriptors that point to
anywhere.

They used to "point at" the Console, which provided a handy, non-platform
specific way to get debug output from a GUI app. But oh no, we can't have
that. Got to use whatever the logging API flavor of the month is to get
anything into the Console (that correct logging API (i.e. one that
actuallyworks) has also changed over the last 4-5 releases).

Oh well, dup2 (2) to the rescue.

------
victor106
I have a few apps that expose logs over a http endpoint. I have to download
them every time to view them and I cannot tail them.

Anyone has recommendations for an app to tail log files over http?

~~~
Jakob
`tail` is just loading the last bytes of a file. If the server supports byte
ranges you can do this with `curl` over http as well:

Last 100 bytes of a website:

    
    
      $ curl -sr -100 https://example.com
    

An approximation of `tail -f`:

    
    
      $ watch -n10 curl -sr -100 https://example.com

~~~
victor106
Very useful, will try this out. Thanks

------
lameiam
another example of a) people with too much time on their hands b) the over use
of hyperbole

------
LockAndLol
And despite that, people will continue using Mac. It really doesn't matter
what they do, as long as they aren't worse (or much worse) than Windows.

~~~
saagarjha
Or Linux.

