
Systems, not Programs - Telichkin
https://shalabh.com/programmable-systems/systems-not-programs.html
======
SomeCallMeTim
This felt to me like a roundabout way of saying "we should think more about
design!"

I _agree_ that we should think more about systems design. And I _agree_ that
the individual "units" that we're designing would benefit from standard ways
of communicating with each other.

The entire thing about "let's not call it a program!" feels like a distraction
from the core point, though. I guess I don't really believe that calling the
"unit of software development that I'm currently working on" a program harms
it or limits it in any way.

Even in the article, the author talks about isolated spaces:

>While the Smalltalk model holds a large number of objects in a single ‘space’
- we could imagine a system where each object has internal space which holds a
cluster of inner objects, and so on.

...and so, poof, we now have a unit of software that could easily be called a
"program." A program that contains multiple layers of inner program, but
there's nothing about the word "program" that limits how it can be applied.

So I think 95% of this article is philosophical semantic gymnastics. There's a
seed of an interesting idea -- software would benefit from the ability to
connect its pieces more flexibly -- but without an actionable hint as to how
to _get_ there, this is just a pointless intellectual exercise.

Changing the word to something else doesn't actually make the problem easier
to solve.

~~~
shalabhc
(Author here)

> Changing the word to something else doesn't actually make the problem easier
> to solve.

This is true, but what exactly is 'the problem' in the first place? Framing
makes a difference. Do we assume we are designing 'units' that communicate
with each other or are there other ways to think about the system?

Is the unit of design also the unit of update? What about cross unit
parameters - if we want to change a parameter that affects multiple units, do
we need to rebuild them and reinstall them in our network of units? Or can we
have a projection of our system that shows this parameter as a single update-
able cell? What if this parameter is derived (i.e. not something you fed in as
a constant but something that represents emergent behavior)?

I'm imagining projectional editing of the system above, now can we consider
the original 'units' just as _one possible projection_ of the system - one
lens we can view and update the system through? One view of the system
(communicating units) doesn't have to be the primary view anymore. Can I
project and 'emergent program' \- a composition of various units, but only the
slices that I'm interested in? Can I update this in place and affect system
behavior?

I find thinking of these as systems helps me ask these questions, while
thinking about 'programs' leads to different questions such as - are the
'statically typed', how do they communicate, and so on. It has more to do with
the common connotations of the word 'program'. The concerns here are broader
than 'standard ways of communicating' between individually designed units.

~~~
SomeCallMeTim
Framing is useful in politics, but I think the underlying limitation here
isn't preconceived notions that you're trying to overcome but the inherent
complexity of the system you're suggesting.

Say for instance you make the unit of design the unit of update. Then any
breaking changes means the unit needs to support both old and new methods of
access. This is often appropriate for something like a public API used by
thousands of people. But if used at _every_ resolution, it would add too much
complexity and need for legacy handling of access patterns.

So if the unit of update is instead a self-contained ecosystem of components,
then the components can all be simultaneously updated, reducing their internal
complexity. The external interface should still probably be versioned.

And...well, you've just again wrapped up something in that self-contained
ecosystem that could very well be called a "program." In fact, it could also
be called an _object oriented_ program: Each component is an object that has
an API and hidden data.

I'm a fan of static typing, so I'd answer a firm _yes_ to that question. How
they communicate -- message passing seems the right paradigm in most
situations. So the _interesting_ (to me) solution would be to standardize
static types over a message passing architecture. Something like Google's
Protobuf? Or JSON with JSON Schemas? Who knows. Depends on the nature of the
problem you're trying to solve.

And that gets to one final point: Most programmers aren't capable of thinking
in big systemic terms like this. Probably 90% of developers are going to be
restricted to operating within a component. I think this is why React and
other web client component architectures are so popular today: They are great
enablers of average developers (and, to be fair, can be high leverage for
great developers as well -- at least up until the point where they get in the
way).

------
seltzered_
“The real goal is to design the form and nature of the software ‘entities’ —
the ‘programmable substrate’ that we manipulate and compose when we program
and use the system”

Reminded me of bret victors “toolkits, not apps” comment (
[https://mobile.twitter.com/worrydream/status/881021457593057...](https://mobile.twitter.com/worrydream/status/881021457593057280)
) and an excerpt from the smalltalk era :
[https://www.youtube.com/watch?v=AnrlSqtpOkw&t=4m19s](https://www.youtube.com/watch?v=AnrlSqtpOkw&t=4m19s)

~~~
shalabhc
That ST video is great.

Yes, apps are silos - why? Making non-siloed apps is harder, and integration
across these is harder still. Is it possible to design the underlying
system/substrate such that siloed apps are not the structures grow easily, and
integration is not something that has to be 'added on', but emerges
automatically?

~~~
Jtsummers
Have you examined BeOS, really the BeFS portion, in the past?

It offered some of these kinds of things. A contact card could be a file
system object with metadata, and you could query the file system not just for
them but for all contact cards that had a phone number or AIM handle in the
metadata. So if applications like your mail client and chat clients are aware
of this they can use one common data store and just pull info from it in a
similar manner. This breaks down the silo between applications that might use
the same data and/or files.

~~~
shalabhc
I'm aware of extended attributes and it definitely seems like a step above the
completely opaque byte arrays that are the status quo. I suppose we could call
it db as a file system.

Another interesting idea is the data types in Amiga OS
([http://www.mfischer.com/legacy/amosaic-
paper/datatypes.html](http://www.mfischer.com/legacy/amosaic-
paper/datatypes.html)). I believe it lets applications be generic enough so
that as new media formats are added, the applications automatically work with
them, without rebuilding or restarting.

Lisp machines were mentioned in another thread here. They also had a better
story around sharing data - you shared rich structures (rather than blobs of
bytes that are encoded and decoded at every boundary) - and they would update
live, etc.

In the end it seems today we're entrenched in something that's reasonable, but
many good ideas could be explored more.

------
rkangel
The design of Erlang makes a lot of sense viewed like this. Even when running
on VM on one machine, you have multiple groups (erlang applications) of
multiple erlang processes (actors) all communicating. These processes can be
hot code reloaded so that the 'system' remains running while th behaviour
changes.

------
eksemplar
I think we should go even further and think systemic. I work in the public
sector, where we have more than 500 different IT systems (in this context,
mostly programs).

It’d be a clear advantage if they were all build to be systems, but it would
be even better if they were build to be systems that coexist with each other.

I wish suppliers would ask us what hardware we operate, what services and API
we’d want them to interact with, and, made plans for what we have to do when
their system eventually dies. Because right now, everything is build for
specific purposes, and it’s impossible to make most things work together.

So you’ll have one program that handles finances for sick people. Another that
handles networking to companies and businesses that help get sick people back
on track. Another program which keeps track on the wellbeing on families with
sick members. A while range of medical programs. And none of them work
together, run on the same hardware or has any form of uniform support
solution, so we have to waste thousands of nan hours simply making sure every
program has the data it needs.

It’s silly, but every developmenthouse seem to suck equally at making non silo
sollutuons.

~~~
magicalhippo
Specialized applications aren't too bad if they're made with integration in
mind. That allows them to tie together different datasets to do a limited set
of things really well.

The problem is if it's difficult to get data into or out of them.

At work we make a fairly specialized application, but we're very flexible in
how we can receive data and send data back out. From manually copying and
pasting Excel sheets, parsing emails and PDFs, plain XML over SFTP or web
services/REST APIs, we got solutions depending on where the core data comes
from and where results needs to go (status back to WMS/TMS, financial details
to invoicing system etc).

If needed we have customer-specific "integration shims" to easily
adjust/correct/ignore data as it comes or goes.

This allows us to focus on making our application really good at a few things,
and let other programs handle the tasks our customer also needs to get the job
done (warehouse management, transport management, invoicing etc).

I agree though that support can be a bit of an issue when you have many
systems working together and something is wrong. The place where the user
discovers the error is often not the place where the error occurred.
Monitoring is important, allowing support to be proactive ("hey, the server
where you're hosting the DB to our program has less than 1GB diskspace left
again").

------
phyrex
This article could do with a look at the old lisp machines in this context.
And Emacs, come to think of it!

~~~
p4bl0
I came back here to say exactly this. I couldn't help but think of Emacs the
whole time I was reading the linked post.

------
joubert
I really enjoyed the book, "Thinking in Systems: A Primer", because it
expanded my horizons and gave words to concepts I had for systems behavior.

[https://www.goodreads.com/book/show/3828902-thinking-in-
syst...](https://www.goodreads.com/book/show/3828902-thinking-in-systems)

~~~
solipsism
Can you share some of those words with us? Because in these comments I'm
reading things like, "A system is a system."

~~~
specializeded
_Excerpt from the preface:_

So, what is a system? A system is a set of things—people, cells, molecules, or
whatever—interconnected in such a way that they produce their own pattern of
behavior over time. The system may be buffeted, constricted, triggered, or
driven by outside forces. But the system’s response to these forces is
characteristic of itself, and that response is seldom simple in the real
world.

When it comes to Slinkies, this idea is easy enough to understand. When it
comes to individuals, companies, cities, or economies, it can be heretical.
The system, to a large extent, causes its own behavior! An outside event may
unleash that behavior, but the same outside event applied to a different
system is likely to produce a different result.

Think for a moment about the implications of that idea:

• Political leaders don’t cause recessions or economic booms. Ups and downs
are inherent in the structure of the market economy.

• Competitors rarely cause a company to lose market share. They may be there
to scoop up the advantage, but the losing company creates its losses at least
in part through its own business policies.

• The oil-exporting nations are not solely responsible for oil- price rises.
Their actions alone could not trigger global price rises and economic chaos if
the oil consumption, pricing, and investment policies of the oil-importing
nations had not built economies that are vulnerable to supply interruptions.

• The flu virus does not attack you; you set up the conditions for it to
flourish within you.

• Drug addiction is not the failing of an individual and no one person, no
matter how tough, no matter how loving, can cure a drug addict—not even the
addict. It is only through understanding addiction as part of a larger set of
influences and societal issues that one can begin to address it.

 _And an initial definition of a system on chapter 1 page 1:_

A system is an interconnected set of elements that is coherently organized in
a way that achieves something. If you look at that definition closely for a
minute, you can see that a system must consist of three kinds of things:
elements, interconnections, and a function or purpose.

 _edit_ oops, I misinterpreted what you meant by “some of those words”, didn’t
take it literally... Will leave this up anyways I suppose, sorry!

------
Jtsummers
> While the Smalltalk model holds a large number of objects in a single
> ‘space’ - we could imagine a system where each object has internal space
> which holds a cluster of inner objects, and so on. This introduces a
> recursive aspect in the design.

Kind of reminds me of tuple spaces. Which is a neat idea. From the system vs
program perspective, your system is running on this tuple space. Programs are
individual actors capable of accessing the tuple space and triggered via some
mechanism (like being notified if a certain type of data is posted or
periodically executing a query and running if they receive a response). You
can then grow the system by allowing tuples of different types to accumulate,
and developing or extending actors in how they process this store.

[https://en.wikipedia.org/wiki/Tuple_space](https://en.wikipedia.org/wiki/Tuple_space)

------
armitron
This is, really, elementary cybernetics (or systems theory). I wish the
article talked more about cybernetic modelling, feedback loops, meshing and
second order cybernetics since these are really the most interesting bits in
my opinion at least when it comes to programming languages.

------
brador
The problem with systems is emergent and unexpected behaviour.

Things can go very bad very quickly once a death spiral kicks in.

~~~
loa-in-backup
Since emergent behavior itself, is wanted, the unexpected subset is what you
imply is detrimental, but that's the case for programs as well.

~~~
Jtsummers
Emergent behavior isn't _always_ wanted. Sometimes it's desirable (Oh, this
widget can be copy/pasted into other documents! I didn't expect that to work
yet.) Sometimes it's not (I can't figure out why this bot keeps stacking
boxes. Patrolling the area is supposed to be its highest priority right now.).

Sometimes we can deliberately induce emergent behavior
([https://en.wikipedia.org/wiki/Automated_planning_and_schedul...](https://en.wikipedia.org/wiki/Automated_planning_and_scheduling))
by composing our system correctly. To create behaviors and solutions that we
didn't initially conceive of. But if _unplanned_ emergent behavior shows up,
your system has become unmaintainable until you've identified the causes.

~~~
Shnickelfritz
Why do you reference the article on planning and scheduling?

~~~
Jtsummers
I should've linked directly to the part I thought was relevant:

[https://en.wikipedia.org/wiki/Automated_planning_and_schedul...](https://en.wikipedia.org/wiki/Automated_planning_and_scheduling#Classical_planning)

Also, I think I deleted something extra in an edit of my original post. These
planning algorithms (or some of them) involve creating options for your
agent/AI and seeing what it does when interacting with its environment. You
don't design the agent so that it will do a complex task, rather you give it
simpler abilities (recognize an object type, pick up an object, set down an
obect on another object, etc.) and the complexity arises precisely from
emergent behavior. This was meant as an example of precisely where you _do_
want emergent behavior and actively work to induce it.

------
John_KZ
Systems and programs are not interchangeable terms. The author seems to think
that in order capture states you need to write a "system". That's just wrong.

A program is a set of commands. It can include state information and
adaptability. A system is a system. A system, from a programming perspective,
is something to be modeled and manipulated to a desired state. If you do this
correctly, you're done. You don't need a degree in philosophy for this. On the
other hand, a lot of programming involves creating a stateless function that
looks like a simple set of commands. That's it. Input in, output out,
predictable behavior, zero adaptability, zero mutability.

------
xte
May I add a small change? s/program-centric/product-centric/ s/system-
centric/human-centric/ Today the trend is selling anything from vaporware to
medical product to cure non existent illness, IT it's no different than that.
Ancient systems, born in a pioneering era, in universities labs, in company
labs lead by tech savvy guys not managers, are built to innovate, actually no
one company want innovation, it's bad for a management point of view, better
found well know stuff salable as new thing to make money in a predictable way.

------
paulgrant999
Well since I've limited to 1 post a day here (very irritating!), this is it!

System dynamics is the name of the field. I studied it outside a CS curriculum
as a "high-level" topic, but really I took to it like a duck to water because
any good coder, does this. Its the difference between a "green" coder and an
"experienced" coder. And it shows up in several of the maths/fields. Anywhere
there is complexity, nonlinear feedback, and long delays in feedback, you get
"emergent" system behavior.

I try and tell people, you can't oversimplify things, you can only stuff the
complexity somewhere else. And when you adopt a framework, its a set of
tradeoffs (on where it shoves the complexity). Beware, frameworks that "hide"
where they shove the complexity (because it will always bite you in the ass).
Sort of like poker, if you can't spot the mark (complexity);)

I hate to tell you this, but most people doing hiring for coders, have NO IDEA
about this level of coding. They're just looking for cogs. How many times have
you heard, "the system is so complex, no single person at the company
understands it?"... inevitably followed by a series out catastrophic failures
that are largely, systemic hazards.

Thats why there is such a large "framework adoption" movemen in CS. Not
because the companies can't dev the software inhouse; but because they don't
understand what they are missing. The mismatch between the "coder" and the
"architect" (or worse where they've fetishized the divorce between system
requirements and implementation). The whole CI is about trying to close the
gap using functional testing and small code updates. But it doesn't work; you
can't test for what you don't know (demand-specific instability)... So now you
have to instrument the entire codebase just to make it run (or take out the
particularization even further ala "containers").

More than anything, this makes the difference in the quality of the
coder/codebase. If you have a good grasp of how things change, rather than
static targets; you don't have to go through the entire morass. You just code
it right the first time.

The other thing that really matters, is domain modeling (separate from system
dynamics, which is actually implementation/environment specific). Knowing your
problem well, and being able to correctly pin the context boundaries, is super
critical.

Last is some control theory; that is, how to auto-remediate the problems as
they develop. Believe it or not, you don't need large amounts of
instrumentation, in order to do this. Good solutions, automatically trade-off
in direct sensing of changes in algo. performance/inputs. They are "ROBUST".

