
Low-level is easy (2008) - kotrunga
http://yosefk.com/blog/low-level-is-easy.html
======
vardump
Low level development is often fun and somewhat easy on mature hardware where
_someone else_ has already discovered the issues.

But on new hardware, you're dealing with gremlins. Physical factors can affect
hardware, and sometimes you just can't be sure if it's your code, Cthulhu or
position of the moon. For example higher temperature can both _decrease or
increase_ crystal frequency. Firmware/software can affect hardware through
power consumption and other routes. It can be really hard to know if it's
hardware or software, except in hindsight.

Catching something that happens frequently is one thing, you have
oscilloscopes and stuff. But good luck hunting down the issue that happens
once per month with 20 test devices. Sometimes these things can take more than
half a year to catch and fix.

When you mess up in kernel driver, result is often something pretty bad. Like
crashing or freezing the system, or worse, silent corruption. Freezes and
crashes are great, if you can use kernel debugger or get the dumps. Not so
great when it happens on the other side of the world without any low level
data to work with. You also have to _really_ understand kernel interfaces, OS
power management, driver life-cycle issues and performance ramifications to
mention some.

I don't mean low level is necessarily hard, but I do mean it really, really
depends!

~~~
ajdlinux
Additionally, when bringing up new hardware, the knowledge you need in order
to understand the state of a malfunctioning device and debug a crash is often
stored in the head of the logic designer who wrote the HDL, not exactly
something you can look up on StackOverflow.

~~~
taneq
It's the R&D equivalent of "contact your network administrator."

"But... but I AM the hardware vendor! _sob_ "

~~~
ajdlinux
It's always great when you need to figure out what a particular register means
or the architecture of a particular error handling process or something like
that, and you open the hardware spec and find "TBD" because it's all still
unreleased...

------
hamstergene
More like, back-end is easy, easier than front-end. He is comparing low-level
backends with web frontends.

On my first serious project at the first job I estimated back-end part of an
app as 75% and front-end as 25%. It ended up exact reverse of that. And in the
next years the pattern repeated over and over.

Everybody seems to think that UI is just about placing buttons but backend is
the heavy lifting. In reality, requirements change more often and in a more
fundamental way, code is harder to test, more third party dependencies
involved, even the dumb KLOC metric ends up bigger.

By any mean, low-level backend will be harder than high-level backend, and
low-level frontend will be harder than high-level frontend. Try implementing
HTTP server in plain C vs. plain Java (no third party frameworks) and you will
see. Being easier is practically the definition of high-level, he is just
comparing things that are too different.

~~~
AnimalMuppet
> He is comparing low-level backends with web frontends.

No, I don't think so. I think he's doing exactly what he said - comparing low-
level with high-level. The low level stuff he was working on doesn't sound
like it was a backend; it sounds like it was an embedded device running bare-
metal (no OS). You wouldn't do a backend (database server, say) that way (or
so I suspect).

~~~
hamstergene
I'm counting bare-metal/kernel into backend category here, because I don't
have better word for "not directly facing end users".

~~~
AnimalMuppet
But that's not what "backend" means. "Backend" means "not directly facing end
users, _but doing part of the work for the parts that are directly facing end
users_ ". Embedded systems are _not_ backend. They are their own category.

------
lacampbell
I miss this guys blog so much. That and 'programming in the 21st century' by
James Hague.

Is there anything remotely that good going currently? Usually stuff is on
medium so I just avoid clicking it.

~~~
allthetime
All the good stuff seems to be happening on personal blogs/programmer's own
websites and tends to be hyper-specific/technical. More general philosophy
seems to be lacking and mostly the domain of writers lacking substance (the
stuff on medium for instance). Unfortunately I don't really have anywhere to
point you right now but if you pay attention to forums (subreddits for
programming languages etc.) and the twitters of writers/programmers you
stumble upon you might start to find more interesting content. There is no
central place, but there are a lot of interesting people doing interesting
work out there.

------
l0b0
High-level software development is basically building Jenga towers with
thousands of (let's be real) mostly not very good parts. Hands up: how many
developers think 100% code coverage is the absolute _minimum?_ Then consider
that you can reach 100% coverage without ever testing a single corner case.
No, not that `if` statement in _your_ code, but rather what happens when this
value is zero, when this time zone name is unknown (at any of frontend,
backend or database levels), when screen size is unavailable, or any of a
bazillion things which can go wrong? Hardware devs _have_ to test these
things, and that's why you don't have to reboot every three milliseconds.

------
panpanna
> I keep what feels like most of ARM946E-S in my brain

ARM64 architecture reference manual is 5242 pages and doesn't even include
things like GPU, CPU (554 pages for A57), interrupt controller (240 pages for
the main GIC version 2 doc), dma controller (100 pages) and tons more you need
to do low-level stuff.

