
The resource leak bug of our civilization - userbinator
http://countercomplex.blogspot.com/2014/08/the-resource-leak-bug-of-our.html
======
meric
Every once in a while a company like Tesla comes along and reconstructs a
particular product gaining incalculable advantages over its competitors.

Every once in a while a doctor is forced to re-break someone's arm which
didn't recover properly.

Every once in a while severe economic recessions comes along and wipes out all
the badly managed companies kept in life support by banks/government.

Every once in a while a programmer writes a piece of software that has almost
the same functionality as an existing one but, is, some how, just better.

And every once in a while there are revolutions that completely destroys the
existing system of governance, when it can no longer fix itself properly.

And that's when resources get deallocated.

Civilization will fall, and it will rise again from its ashes.

Looks like the world uses stop-the-world garbage collection on particular
threads keeping the rest of the world running.

~~~
brc
This happens at the micro level (one employee is swapped for another who is
better) and at the macro level (banks/tesla) and at the country level
(socialism is replaced by more efficient and humane capitalism as in
ussr/GDR).

The problem is when the bad employee cannot be removed, the bank is bailed out
or the people are unable to get themselves a new form of government.

The key to efficient reallocation is not letting too much power concentrate in
one location. And I think that is the thing at risk the most right now.
Governments have become too powerful and are preventing the necessary creative
destruction, like an operating system refusing to run the garbage collector
and preventing new programs from running.

~~~
FireRabbit
The problem with the USSR wasn't its economic system, it was the fact it was a
corrupt dictatorship.

~~~
hvs
Which was required by its economic system.

[http://en.wikipedia.org/wiki/Economic_calculation_problem](http://en.wikipedia.org/wiki/Economic_calculation_problem)

~~~
imanaccount247
Then why was national socialism such a huge success in Germany?

~~~
brc
It's a late reply but national socialism only ran for a short time before
being replaced with a wartime economy. In effect the program ran fine because
it didn't run long enough for garbage collection to be an issue.

------
bonobo3000
There is no mention of all the benefits that encapsulation gives us - to take
the blogs argument to the extreme imagine writing your own web server, caching
layer, load balancer, JSON/HTTP parser, frontend logic for every platform, and
a hundred other layers of abstraction I may not even know about every time you
want to write a blog.

There is a real tradeoff between programmer productivity and resource usage,
because every library that isn't tailored for your exact use case (i.e every
library) is an inefficiency. Given programmers wages vs. cost of
CPU/RAM/storage, and the long term trends (technology makes one cost lower,
the other cost higher!), it is economically insane to reject encapsulation.

I definitely see the appeal viewing programming as craft. In that case, its no
coincidence that "reinventing wheels" is good advice to learn how to program.
Its just not how to run a business.

~~~
userbinator
_to take the blogs argument to the extreme imagine writing your own web
server, caching layer, load balancer, JSON /HTTP parser, frontend logic for
every platform, and a hundred other layers of abstraction I may not even know
about every time you want to write a blog._

I see the argument as saying that all those layers are not really necessary.
Take HTTP for example: it's a large complex protocol that requires plenty of
code to handle correctly (e.g. the size of a request, and the various pieces
inside it, are not specified and the parser has to read through the whole
request sequentially to figure this out), and the majority of applications
using it never need the full functionality. What if it was a simpler, easier-
to-parse format that was as extensible?

One of the points I get from his argument is that encapsulation has encouraged
us to build massively overcomplicated systems by piling layers upon layers
with little understanding of how everything fits together, when we could
instead be building much simpler and more efficient systems that could still
be using abstraction but in a far more cautious manner.

~~~
crpatino
I think you are partially right, but the situation (I'd rather not label it as
a problem just yet) goes on many levels.

By example, Moore Law has allowed us to pack incredibly large amounts to
transistors in reasonably affordable machines, and what we do is...
virtualization, a.k.a. write software that (presumably) simulates being a
(much slower and limited) hardware system so a third party can run other
programs on top of it, for a fee.

That kind of makes economic sense (because customers have a big appetite for
computing power... just as long as they do not have to keep those pesky
sysadmins on the payroll), but from a purely technological point of view is
craziness. Every component you add into your stack is a potential source of
error, and you do not gain any extra features because you are simulating real
hardware... so, why to do it in the first place?

I think the answer is: because nobody would buy all those transistors if we
didn't do so.

~~~
marcosdumay
Another, possible answer is: Because transistors are really cheap, and energy
only a bit expensive. But that wasting saves competent people time, what's a
very expensive and limited resoure.

Why does everybody ignore the obvious explanation, that you'd get from the
mouth of any participant, for a convoluted one that only fits the criterium of
political correctness?

~~~
crpatino
I do not think anything you say here contradicts what I originally said. Yes,
there are economic reasons why this makes sense, probably having to do with
labor compensation.

Having said that, please excuse my ignorance on system administration but, how
does running a virtual machine that pretends to be a Linux/x64 box... on top
of a real Linux/x64 box saves anybody at least some time? As a software
developer who drank the Java kool-aid back in the 90, at least the "write
once, run everywhere" mantra did make some kind of sense in those terms, but
this other example is genuinely beyond me.

To me it has more to do with a tech company being able to put together an
A-class team to serve a large number of customers that would not stand to pay
the salary of a single full-time B-level IT guy if their business model
depended on it.

~~~
ethbro
"[...] how does running a virtual machine that pretends to be a Linux/x64
box... on top of a real Linux/x64 box saves anybody at least some time?"

To borrow someone else's words: [https://blog.engineyard.com/2014/pets-vs-
cattle](https://blog.engineyard.com/2014/pets-vs-cattle)

Or, in summary, it's a heck of a lot easier to instantiate a new instance of a
machine from a snapshot than instantiate a new machine from a set of
directions and some auto-config scripts.

~~~
crpatino
I agree standard images are much easier to set up than a hand configured
system. That does not mean said image should run on top of an emulator.

I see some value in having a running system right now, instead of waiting for
a long latency while the image is downloaded to real hardware... at least in
some situations. However, is taking a permanent performance degradation of an
order of magnitude worth that advantage?

------
sanxiyn
Nobody knows how to make a pencil, and that is not a bug. Division of labour
is how humanity improved its lot so far.

[http://en.wikisource.org/wiki/I,_Pencil](http://en.wikisource.org/wiki/I,_Pencil)

~~~
alexmat
"Improved" is a relative term and says nothing of the associated costs. Could
we maybe reduce costs and still "improve" at a slower, more sustainable pace?
It's worth exploring the notion without turning to the dogma of efficiency at
any cost.

~~~
Retra
Exactly. "At any cost" should only apply to long-term full-scale survival
threats. With every other problem, you should only ever except "at reasonable
cost."

And in order to know what cost is reasonable, you have to do some kind of
rational analysis. There are few things less efficient than an ideology
without an understood purpose.

------
Houshalter
This entire thing is just a rant against specialization and abstraction. Both
in economics and computer programming. This is just ridiculous. Without
specialization the world wouldn't function at all. No single human being can
build a car by himself from raw materials. Programmers shouldn't be expected
to write their web apps in assembly, and can you imagine the mess if everyone
tried that.

The author starts out with a legitimate complaint that things are suboptimal.
But his cure is worse than the disease.

~~~
tjradcliffe
The second comment on his blog says he's just repeating what Marx said, which
is not quite correct but in the right ballpark. People who don't know anything
about economics tend to do that.

The "problem" of exponential growth is the belief that it will result in using
up all available resources rather than finding more efficient ways to use
resources we didn't even know we had. Engines become more efficient, power
generation becomes lower cost, food production moves from farms to
factories... all of these are good things. We feed the world with far fewer
resources today than we did a century ago when many more people were starving,
even while we use more resources building smartphones and sending spacecraft
out to explore the solar system. We solve problems our ancestors didn't have
because they still hadn't solve the problems they faced.

Economically, exponential growth is required to keep the monetary system
stable, but in human terms we live in a world where new problems always arise,
and require new solutions.

Before Watt and Newcomb we didn't have many problems related to steam power.
Before the Wrights getting to the airport was just not an issue for most
people. Human invention creates new problems which human invention solves, and
that is a treadmill we cannot get off.

Capital markets turn out to be a remarkably good method of funding new
solutions, and private enterprise turns out to be a remarkably good way of
exploring the landscape of new solutions. No one has ever been able to figure
out how to incentive socialist managers to explore that landscape with
anything like the same efficiency--particularly given the opportunities they
have for corruption--and when you look at how inefficient capitalism is you'll
realize that's saying something.

So we continue to press on, while a certain type of person who has been more-
or-less loud in the past two hundred years shouts at us that it is all going
to end badly Real Soon Now.

Somehow it never does. It could, certainly, but we're centuries away from
that. Possibly millennia. In the short term, if the naysayers are so sure of
their analysis, I recommend they apply it to the stock market and use their
impressive prescience to get rich. It shouldn't be difficult for anyone who
has the kind of deep insight into the workings of the world that they claim to
have to manage that.

------
gus_massa
Going to the extreme, instead of using blogspot you can write your post in raw
html and host them in your own server. But the blogspot abstraction layer is
much easier (and it can handle the HN load), so you have more time to do your
unique activities.

------
jonnydark
From what I can tell the writer is arguing that abstractions and
encapsulations are wasteful of resources. The author claims that this is an
inherent problem of abstractions and a lack of understanding of their
implementations. The justification for this is that time and mental energy are
resources as well and although we are being wasteful of other resources we use
up less time and mental energy. The author states "However, during recent
years, I have become increasingly convinced that the portion of true tradeoff
is quite marginal" however doesn't seem to give anything quantifiable to
illustrate this marginality. Much of the argument is based on anecdote and
personal ideas on general human nature.

I am inclined to think that the author has come to this conclusion because
time and mental energy are less valuable to them than other resources and
that's not an unreasonable stance to take. However I don't think this view is
widely shared and that thought most people agree that premature optimisation
is the root of all evil.

------
wlievens
Parts of the post are interesting and challenging, and to some degree I agree
that the cost of layering abstractions is often underestimated, but the final
few paragraphs read like a naive manifesto...

------
fit2rule
I maintain a small outfit of old, antiquated, 'un-useful' computers, the
eldest 30+ years old. I have pretty much never thrown a computer out - there
is _always_ a use for it.

Computers do not die; their users do. ("s/Computers/ConsumerToy/g")&etc.

The leak can be plugged, and it starts like this: don't use technology you
don't understand, or .. to put it likewise: use all technology with full
understanding. The only reason we threw away our old computers was because we
didn't think anyone could/should use them; they are nevertheless: still
useful. The 'usefulness or not' is _entirely_ a spiritual decision, however,
because the machine does not care, and it is this fact - and ignorance of such
spiritual components of so many facets of human culture in general - is the
blind spot in the debate, imho. Things are only as they are, if we decide they
are; the usefulness of a computing device is absolutely not dependent on its
physics properties in any way other than: can you turn it on, does it work,
can the user do something with it? It is the User who makes the decision, all
along the way to actual, real, usefulness.

I'm still using my computer to send email, just like I did in the 80's and so
on, albeit it now requires a much more sinister amount of maintenance, and
general abstract stress it seems, emotionally, than the old hazeltine
connected to a Vaxen we used to have to use to 'get things done'.

tl;dr: there is no fault in our machines but that which the user decides and
identifies.

~~~
paulmd
As a working definition, a piece of hardware really isn't _useful_ if another
piece of hardware will do the same task better, at a lower cost.

You can run a low-power PC and check your email like it's 1980 just fine, and
these days they're VERY cheap. A $30 5W Raspberry Pi is more than capable of
running 'mail' on the command line. For a bit more power, quad-core 1.6ghz ARM
SOCs are about $60-70. For fileserver type stuff, a 15W Liva with a Baytrail-M
can be had for $115, or you can build out a full quad-core mini-ITX for
something like $175.

The reduced power usage very quickly amortizes the cost of the hardware. If
you would save 200W by retiring a dinosaur - that's $17.28 per month of 24/7
usage. If you've got a Rpi on for 4 hours per day checking your email - it
amortizes out within 10 months.

This totally ignores the additional headaches associated with old, failing,
unsupported hardware. You yourself described it as a "sinister amount of
maintenance and stress".

You can make a philosophical argument here that a piece of hardware is never
truly dead if it'll still push bits, but be honest here - if you're running up
your power bill to avoid spending $100 on new hardware, if you're putting up
with a PITA system for the heck of it - it's not _useful_ , it's _your hobby_.
Hobbies are fun, I think it's fun to keep old gear going too, but be realistic
about the merits of such activities.

~~~
fit2rule
>> a PITA system

Entirely subject, utterly arbitrary. I can still use an 8-bit machine to write
a good letter.

------
wmt
Buggy civilization? I don't think so, the improvement through evolution favors
those who aim to grow exponentially. The alternative, micromanaging extremely
complex systems, will unfortunately fail.

Trying to force the global economic system into a software analogy just feels
silly and wrong.

~~~
eropple
_> Trying to force the global economic system into a software analogy just
feels silly and wrong._

It's the same thing you see software developers do all the time. "I know this
one thing, let me apply it to everything. No no, your job can't be hard, I'm a
_software developer_ , let me think for you."

Breadth matters.

------
EGreg
I think as a developer I am attuned to this most acutely. I see garbage ok the
street or read about Amazon's new warehouse and think, "yeah, we may have 500
years to store our garbage, but then what?" When gas prices go down I think
"past peak oil, we are just accelerating running out". Or when I read about
overfishing, etc I think how we could irreparably disrupt ecosystems with a
cascading effect.

But then I think how big this planet is and how ingenuous humans are
collectively, and wonder - am I overworrying? I see small waste accumulatung
but in scandinavia they're discovering ways to recycle garbage into resources!

So, I don't know. Thoughts?

------
akkartik
Two links, one agreeing and one disagreeing:

[http://slatestarcodex.com/2014/07/30/meditations-on-
moloch](http://slatestarcodex.com/2014/07/30/meditations-on-moloch)

[http://www.ribbonfarm.com/2012/08/23/waste-creativity-and-
go...](http://www.ribbonfarm.com/2012/08/23/waste-creativity-and-godwins-
corollary-for-technology)

------
api
Meh. Describe an alternative to a growth-oriented society that isn't either
feudalism, some kind of white agrarian fantasy, or an even more imaginary
"sci-fi utopia where everyone wears togas." (Google The Venus Project and the
Zeitgeist Project for an example of this stuff.)

~~~
crpatino
Don't discard feudalism just yet. If nothing is done (and there are reasons to
believe that not enough will be done to count as "more than nothing") this is
what we will be getting by default.

Also, do not conflate the social concept of feudalism (survival by pledging
allegiance to a chain of command that competes with other chains of command
for access to resources) with the particular customs, laws (or lack thereof)
and religious & aesthetic sensitivities that occurred historically the last
time this happened in Western civilization. It can be argued that global
corporations are already modeled on those basis and beyond the control of
outdated national democracies.

------
waps
The bug in this article. Economically speaking (in other words: in the real
world, where you are not the only player), which is optimal :

1) optimizing resource usage

2) maximizing resource usage

(it's easy to see why some others are suboptimal, e.g. minimizing resource
usage)

From a long term perspective (ie. which of these guarantees you'll survive and
doesn't provide another player with an easy way to destroy you). Other way of
putting this question, which is the Nash-equilibrium answer ?

The answer is : 2.

(Is it pareto-efficient ? yes !)

In other words, no matter how good it sounds to conserve resources, it's a
mistake. If a civilization did that, either it would get run over by another
civilization, or, if it came to that extreme, by evolution itself.

So this is not a bug, it's a feature.

~~~
deciplex
Can you expand on this? It seems like you're saying that, for example, taking
all the world's oil and lighting it on fire, or (more dramatically) firing all
the world's resources out of a giant cannon and into the sun, is an optimal
economic strategy. I'm not an economist, but that doesn't sound right.

It seems like the optimal economic strategy, assuming finite resources, is to
optimize your use of resources. Then, there is a complementary defense
strategy, which is to monitor other civilizations so that they don't get the
drop on you. This might also include acquiring resource pools for future use,
or simply to deny them to adversaries, but it doesn't mean you need to burn
through them as quickly as you can.

~~~
mattdw
Maximising deprives others of the resource, optimising doesn't. I think that's
the point being made.

An economy that first maximised, then second optimised, would beat an economy
that first optimised, then second maximised, because the latter would have
nowhere to expand.

An economy that optimised _and_ maximised would beat an economy that only
optimised.

~~~
deciplex
This doesn't take resource exhaustion into account. Which is to say, that by
maximizing first and then optimizing, you may be only ensuring the extinction
of our species, and the fact that your civilization will still be around when
the lights go out will be cold comfort.

I mean, this discussion is absurd, of course, since we seem to be operating
under the assumption that the problem of how to do civilization can be solved
with a half-dozen variables or so. But, even within that simplistic framework,
the approach being laid out by you two is pretty daft IMO.

How about, an economy that achieves hegemony, and then continuously optimizes,
increases resource consumption as appropriate and only to the extent that such
consumption is sustainable (over some reasonable planning horizon anyway), and
seeks out and destroys rival upstart economies that are a threat to it
(especially when they have an unsustainable model of resource consumption),
will tend to be successful and stay successful.

~~~
mattdw
I'm not saying that I wish that it were true, or that I think it's a good
thing. I agree that it's daft. But over the short term, pillaging societies
appear to beat conserving ones. And it's a fact that we do seem to be (as a
species) adopting a short-term-profitable long-term-suicidal approach.

(An interesting model is Norway's oil – they're maximising in the sense of
getting all the oil out from under the sea, but they're stashing most of the
profits away as a rainy-day fund, which puts the country on very secure
footing for about the next hundred years or so. You might call that
'maximising without consuming', or 'camping'.)

I think the future of the human race depends on finding a better answer to
this whole question, but I think accepting that pillagers beat conservers in a
lot of cases is a necessary step to finding something better.

(I don't think we actually disagree; we seem to be arguing different points.)

------
zobzu
tldr: civilization needs some goddamn refactoring.

~~~
jmnicolas
It is my opinion that unfortunately no amount of refactoring will save our
civilization.

We need to restart it from scratch.

~~~
sergiosgc
Young whipersnapers and their throw the old code away syndrome...

You don't restart from scratch and hope for the best. If the systemic
conditions that make our civilization ill persist, they will affect version
2.0.

It is much much better to incrementally improve the current system, even if
the end game is not clear yet. A full rewrite always throws out the good along
with the bad, and it is clear our civilization has many good characteristics
in it.

I, of course, disagree on the need to restart from scratch.I think that the
resource problems we face are slow moving enough that we will avoid them. The
real danger is in fast moving problems that we do not know about yet.

~~~
jmnicolas
Well I think we will be forced to restart from scratch since we can't manage
to make even the easiest incremental changes needed ... imo we're bound to
collapse a la USSR style.

