
The problem with programming and how to fix it - ndh2
https://alarmingdevelopment.org/?p=1173
======
dragonwriter
> What reason is there to believe we can improve programming technology to be
> vastly easier and more accessible? To start with, spreadsheets.

So many failed efforts to make programming vastly easier and more accessible
have started with almost this exact line.

I think it's possible that there is a seductive trap there. We see
spreadsheets and we think, “programming that doesn't have the downsides we
associate with spreadsheets could be just as accessible”. But maybe the
problems are intimately tied to what makes it accessible, and wheb you try to
make programming that accessible you either fail or end up with something that
doesn't offer any advantage over the spreadsheets we have.

> Unfortunately application programming got trampled in the internet gold
> rush.

No, it didn't. Oh, sure, it's not getting chased around by venture capital,
but there's plenty of it going on.

> As a result the internet age has seen an exponential increase in the
> complexity of programming, as well as its exclusivity.

It really hasn't; we may often be solving far more complex problems, but we do
it with languages, frameworks, and platforms that allow casual programmers to
do so more easily than at any time in the past. Programming is “more
exclusive” only in that:

(1) in working environments, IT policies restrict programming (other than
spreadsheets) to the high priesthood more completely than ever before, and

(2) lots of simple problems that would be solved by casual programming in the
past now have end-user tools that allow them to be solved without anything we
recognize as programming.

~~~
david927
_> Unfortunately application programming got trampled in the internet gold
rush._

 _No, it didn 't_

I think if you don't rely on memory but examine the history as Jonathan
Edwards has done, you would be surprised to find that the late 1990's was,
indeed, a big cliff in terms of application development innovation. You can
argue the reasons -- maybe it's not about the internet gold rush -- but you
would have to do a small fraction of his research to start to convince me.

 _> As a result the internet age has seen an exponential increase in the
complexity of programming, as well as its exclusivity._

 _It really hasn 't_

Frankly, I'm not sure you know what you're talking about. Before the web was
popularized, designing a layout meant a using a simple layout tool. Now it's a
stack that is at least HTML, CSS and JS, and often more. We're doing more, to
be sure, but you can't honestly claim, without a deep ignorance of the past,
that it's easier or more inclusive.

~~~
munchbunny
> Frankly, I'm not sure you know what you're talking about. Before the web was
> popularized, designing a layout meant a using a simple layout tool. Now it's
> a stack that is at least HTML, CSS and JS, and often more. We're doing more,
> to be sure, but you can't honestly claim, without a deep ignorance of the
> past, that it's easier or more inclusive.

Since we're on the topic of UI, I think this is actually a great place to
discuss what has happened to programming.

You used to be able to use a simple layout tool. But we traded that away for a
substantially increased level of visual sophistication that wasn't practical
with the predecessor technologies. For visual sophistication, you had a couple
other quite complex options: render it yourself, or for about 15 years, use
Flash.

However, we've also smoothed out the learning curve a lot. Even if you have to
learn the HTML/CSS/JS triad, we've figured out how to teach it more
incrementally.

Even UI in 3d games has gotten dramatically easier. These days you can hack
something together in Unity with barely any coding knowledge. When I first
started learning 3-d programming 10 years ago, your options were basically to
learn C++.

Maybe it's not a 100% obvious conclusion, but I think on the balance of things
it really has gotten easier and more inclusive.

------
throwaway648
Lemme use an analogy: Why is building a house not accessible to all? Surely
house building could be more like building with legos.

Building a proper house is so much more that just piling rocks mixed with
mortar. You have to take into account the ground composition, moisture, light,
ventilation, red tape, usability, safety, etc. In other words: it takes a huge
set of skill. You can use advanced tools, like a tractor shovel for digging
the base but that requires even more skill.

We just have to accept that doing complex stuff requires lots of training to
do it properly. The more advanced tools you use, the more specific training
you need but the more complex stuff you can build in shorter time.

As for programming. Abstract thinking and clear, explicit communication of
ideas is not trivial to automate. This is problem even between two persons
(just think about trying to decipher what your sales person or customer
actually wants), let alone between a human and a machine. It’s a form of art,
not some mechanical process you automate using an array of boxes.

~~~
TuringTest
Not all programming is akin to building a house. Some of it is similar to
hanging a shelf in the kitchen. Why should it take an architect to drill the
holes and fasten the wall plugs to hang a spice rack?

Sure it might need some training, but it should be the kind that can be
explained in a DIY TV show.

~~~
yuhe00
Most of the work in programming is in describing (in very strict and pendantic
terms) how different systems interact with each other and how the data flows
from one end to another. The actual work of moving the data and doing all the
calculations is handled by machines. In that sense, the work done in
programming is designing and integrating systems, not the execution of it. In
your analogy, installing a shelf would be more like installing Photoshop
(which we can presumably agree is actually simpler than installing a shelf).
Most people do not design their own shelf, select the most appropriate
fasteners, source raw materials and send it to manufacturing to be mass-
produced. A shelf is also a relatively simple piece of furniture. What about a
desk-drawer or a closet? Sure, you can buy the raw materials and power tools
at any hardware store, but is it trivial for anyone to actually design and
build usable (and desirable) furniture from pieces of plywood?

------
0xBA5ED
Simplifying software is a fantastic goal, but blaming the situation on an
"isolated subculture of nerdy young men" is the opposite of helpful. Also, I
would seriously consider the fact that some things are simply complicated and
there's no conspiracy behind it. A lot of very smart people work very hard to
find ways of taming this beast. It's just hard.

~~~
randomsearch
I’m interested as to whether there really is complexity that can’t be
simplified in general.

Here’s an obvious example where it could be: web apps are mostly conceptually
simple: a thin client which is basically a UI framework talks to a process on
a remote server. So you need something to develop the UI and something for the
server. But instead of the “web UI toolkit” (which could mostly be a graphical
tool) we have JavaScript, CSS, HTML, jquery, a gazillion hacked front end
frameworks trying to (imperfectly) fix the mistakes of everything below it, a
whole system for preprocessing, packaging, running tasks, deploying - just for
a UI. And all those tools need customising, rely on knowledge of a bunch of
random tools and languages and file editing.

So we could definitely improve 10x in making “web apps” or whatever you would
replace them with something much much simpler (even if for a subset of use
cases). And that’s just one example.

~~~
quuxman
conceptually simple, perhaps, but in practice they are far from it. HTML, CSS,
JS, and web server tools are complex because web pages are extremely
parameterized over different content, client sizes and types, user
preferences, and data. From being a long time web developer, the graphical
layout tools are basically only useful for graphic design, because they output
code that only works for the exact layout and content initially created, which
is typically useless for anything but static content.

I think the difficulty of making tools for abstracting web content are related
to the difficulties of creating more intuitive programming tools.

~~~
randomsearch
Thanks for the thoughtful reply.

I was thinking at a more abstract level:

> because web pages are extremely parameterized over different content, client
> sizes and types, user preferences, and data

You've taken this as a given. Why does it have to be like that? If we're
defining the standards, we can make the web whatever we want it to be. And I
think if your explanation of why it is complicated is the above (and I think
that's only a small part of why it is such a mess), then let's modify some of
those assumptions to make it easier to simplify.

------
Ace17
> Programming today is exactly what you’d expect to get by paying an isolated
> subculture of nerdy young men to entertain themselves for fifty years. You
> get a cross between Dungeons & Dragons and Rubik’s Cube, elaborated a
> thousand-fold.

This is vague and insulting.

Many of us programmers spend a lot of energy to keep systems simple, to a
point that one could mistakenly conclude that they were easy to write.

We don't dream of having to maintain complex machineries reminding us of
Rubik's cubes. We consider maintainance nightmares and hard-to-understand
systems as failures, not goals. We don't love complexity, we hate it (but the
less experienced of us haven't got the chance to fully develop this hatred).

~~~
david927
_This is vague and insulting._

Good. It should be. Get mad about that. Get so mad that you devote time and
effort to fixing it.

Programming compared to other disciplines is what might arise from a _Lord of
the Flies_ situation. Like the end of that novel, we should stop, fall at the
feet of a comparison of programming to other disciplinnes, and weep tears of
shame.

~~~
Vinnl
> Get so mad that you devote time and effort to fixing it.

Interestingly, I feel that insulting people is one of the worst possible ways
to get them to change their behaviour.

~~~
david927
He's not actively insulting anyone; he's calling a spade a spade, he's telling
the truth, and if you find that insulting then change that reality.

~~~
Vinnl
The person you replied to called it insulting. It's not really about what you
think they __should __do; all I 'm saying is what I think it's likely people
__will __do. Formulating criticism in a way that the people it applies to are
not insulted is, I think, the best way to actually get them to fix it.

It's hard, I know. But all __I __' d do when someone refers to Rubik's Cubes
and DnD is shrug it off and think it doesn't apply to me.

------
repsilat
Hah, I was nodding along with the article and then got to this bit:

> _What reason is there to believe we can improve programming technology to be
> vastly easier and more accessible? To start with, spreadsheets. Spreadsheets
> are by far the most popular programming tool, used by far more people than
> “real” programming. Surely it must be possible to carve out other domains
> like that of spreadsheets without taking on the full complexity of
> programming._

As someone writing a spreadsheet app with magical programming powers
([https://6gu.nz](https://6gu.nz)) no wonder the author seemed to be sweet-
talking me :-).

And yeah, I think we can do a lot better. Some low-hanging fruit: backslashes
in strings, "what is Unicode", "what is a hash function and why do I care?".
Etc.

There's a crap-tonne of essential complexity in just about everything we do,
but most tasks for which "programming" should be the right answer are not the
sorts of things programmers actually do.

Part of the problem is, we have all the best tools because we are the tool
makers. There isn't good version control for Excel. Linting is mediocre. And
the language itself sucks, because the programmers who work on it don't think
spreadsheets are programming, and don't think the people using spreadsheets
are programmers.

I do think the conclusions of the article are o ff-base, though. I have no
faith in professional societies, and I don't think the answer is
"institutions". The answer _is_ business -- bringing programming to the masses
is worth _trillions_. Programming is the biggest force-multiplier that exists.
It's what the FAANGs (widely disparate companies in different industries) have
in common -- they (as companies) "know how to program".

Maybe businesses are short-sighted, but the value here is so big it just has
to happen. Magic of capitalism and all that.

~~~
vanderZwan
> _There isn 't good version control for Excel. Linting is mediocre. And the
> language itself sucks, because the programmers who work on it don't think
> spreadsheets are programming, and don't think the people using spreadsheets
> are programmers._

Since you're writing a spreasheet app, I presume you are familiar with
Feliene's work on this topic?

[http://www.felienne.com/publications](http://www.felienne.com/publications)

Also, you probably will find Edwards videos about his own experiments
interesting:

[https://vimeo.com/jonathoda](https://vimeo.com/jonathoda)

~~~
repsilat
> I presume

I'm not! Thank you!

I've been pretty heads-down writing my thing, but I'll be sure to set aside my
evenings to go through both sites exhaustively, and probably try to get in
contact with both to ask for feedback.

~~~
shalabhc
Also check out Flowsheets by Glen Chiacchieri:
[https://tinyletter.com/Flowsheets/letters/flowsheets-v2-and-...](https://tinyletter.com/Flowsheets/letters/flowsheets-v2-and-
welcome-to-tinyletter)

------
cogs
Programming for beginners is just as accessible as it has ever been, there are
good languages (some specifically for beginners: Logo, and game making
languages and so on) and good courses.

Where do I think programming has stalled is in the business environment.
There, as the article correctly points out, it's much harder to automate your
job unless you work in an IT department.

Most business end-users do their work in Microsoft Office, and this is one
area where I think the tools have stalled. Visual Basic and its object models
are a painful hurdle for a smart non-programmer who is trying to get things
done efficiently, or who wants to learn programming for its own sake.

There have been some attempts to wire up Excel with Python, but until
Microsoft decides to bundle an easy tool with every user's MS Office,
programming is going to remain an arcane art.

~~~
deltron3030
>Where do I think programming has stalled is in the business environment.
There, as the article correctly points out, it's much harder to automate your
job unless you work in an IT department.

Depends on your business. Ecommerce can be automated without needing to code.
Many use a central product repository (e.g. Amazon), and build their web
frontend (CMSs, shop engines, landing page & app builders) around that,
interfacing with it through easy installable plugins and other automation
tools. They're already "coding", without code, but "business components".

To have successful funnels, they also need to know UX basics that they can
reproduce with those tools, or pick tools with those basics already baked in.
They're basically full stack developers who don't code.

The downside to that approach is that you're totally dependent on all those
other businesses. But many who start out this way, employ devs as they grow,
or start with programming themselves, and transition to their own
infrastructure.

------
mike_honcho
I disagree. The world we have today couldn't be built on simple tools like
Spreadsheets and Visual Basic. The amount of complexity in programming is
necessary, and the outcome has been tremendous and miraculous if you think
about it.

And we don't want to build a future where the masses should be programming!
You want my mother to be programming? My sister? Why? I work as a software
engineer and most of the time programming is the last thing I want to do when
I get home!

Sure it can be overwhelming at times with all the new technologies coming out,
but in the future technologies are going to be completely different, with AI
and VR becoming more widespread, programming is going to have to change again.
And good, because that is what will allow us to build the future!

~~~
reacweb
In my utopian world, I would like to edit the source code of my microwave so
that it stops its annoying bips as soon as I open its door. I would love to
remove the stupid nag screen that appears each time I start my nissan leaf.

I am not an electrician, but I can fix basic electric issues. Sometimes, I can
fix broken furniture. I am a programmer and I can fix almost nothing.

When we can program, we are in control. I want to be in control of the objects
I own, or at least to know that it is possible if I really want it. When I see
all what already possible in the 90s with so simple computers, I do not think
that the complexity is necessary.

~~~
AstralStorm
Sometimes you can... We did fix a few issues with a washing machine controller
once.

But that took a lot of work on an undocumented software in machine code in
8051 and had the best tools.

The issue is that is not easily traceable from machine code representation.
Even electronics are easier.

------
sgillen
> There is something unique about software that exacerbates these problems.
> Unlike every other technology, software doesn’t wear out. The traditional
> cycle of replacement and renewal is halted; there is no evolution.

I do understand what he's saying here, but saying that software has no
evolution is just wrong. Like yes we still have C code kicking around
everywhere but we also have new technologies languages (rust, go, etc etc),
new philosophies on how to write code (TDD?), and new tools (jupyter notebook
for interactive data munging for example).

Furthermore how many people have been through a rewrite? developers (try) to
replace and renew all the time, although yes this is difficult.

also:

> Unfortunately application programming got trampled in the internet gold rush

It seems to me like application programming is alive and well, although
admittedly I am younger and am probably considered an application programmer
myself (I.E. I don't develop software as a profession, but I write code all
the time to help with my work).

------
lifeisstillgood
I believe that software is not a professional practise but a new form of
literacy (in fact the book I am writing is all about this (Real Soon Now,
thanks for asking))

And when you see software as literacy a lot of the stuff he moans about simply
goes away.

Yes it is seen as exclusive and a priesthood - imagine what illiterate serfs
thought of those who could write. It took years of practise.

No it's not a good idea to make programming more accessible. We don't have
easy literature, we don't think someone who reads and writes solely with
fridge magnets (#) has mastered the language. It is a good idea to invest more
in education and that is being done reasonably well, but as in all things more
is needed.

Yes the results can be a car crash at times - my favourite analogy for this is
our management structures.

Would we ever take a literate organisation like the Washington Post or Harper
Collins and put a layer of senior management in place who were totally
illiterate from birth? If we did do we think they would make sensible
decisions, empower those organisations? No.

Yes there is a lot of shiny new thing, going on. But that's because there are
a lot of people doing software - and many of them select for being good at
marketing too. The real big software projects tend to select for conservatism
married to pragmatism- look at Linus on a mailing list or the PEP process, or
Debian.

So yeah software needs to sort it self out - and professional bodies will
start to solidify (personally I feel a lawyer like body, concerned with
managing the course of OSS code used in government is the most beneficial),
and until then don't try and fix the world or boil the ocean - just focus on
making your coding practises as good as possible, even if your boss is making
crazy calls.

(#) Wanted: Better analogys - can you write pithy phrases that sizzle like ice
cream on a griddle? Contact the author in complete confidence today.

~~~
vanderZwan
> _No it 's not a good idea to make programming more accessible. We don't have
> easy literature, we don't think someone who reads and writes solely with
> fridge magnets (#) has mastered the language._

You're mixing up the _content_ of writing with the _methods_ of writing them.
Writing _" a rose by any other name would smell as sweet"_ is still as poetic
when using fridge magnets.

Because of that, you fail to see that modern writing systems were massively
improved by making it more accessible than what came before it, and that this
has had a major impact on literacy levels as well. We started with Scriptio
Continua[0]. We had to invent spaces and punctuation. Originally we just had
capital letters. And so on.

And if we look beyond the West (which we should), you will encounter Hangul,
which is a phonetic alphabet that points out how arbitrary our letters really
are and replaces it with something more systematic and easy to learn[1][2].

Sure, _mastering_ writing is hard. Basic literacy and writing systems do not
have to be.

[0]
[https://en.wikipedia.org/wiki/Scriptio_continua](https://en.wikipedia.org/wiki/Scriptio_continua)

[1] _How Korea crafted a better alphabet - History of Writing Systems #11_
[https://www.youtube.com/watch?v=j9hzK0K1L4I](https://www.youtube.com/watch?v=j9hzK0K1L4I)

[2] _Learn to read Korean in 15 minutes_ ,
[http://www.ryanestrada.com/learntoreadkoreanin15minutes/](http://www.ryanestrada.com/learntoreadkoreanin15minutes/)

~~~
lifeisstillgood
It's hard to find a suitable analogy (see my advert at end of last post) for
almost-literate. I think i would be better saying something like ::

it does not matter if one is born in china or UK or korea - children are given
huuuuge amounts of training to become literate. We don't dumb down the writing
system, and we don't dumb down the literature written in the system, we spend
more on training the kids.

Similarly with programming - we should not look for some uber click and go
system that makes it easier - we just have to work harder.

Fridge magnets are just a placeholder for "trying to make reading and writing
easier without actually learning to be literate". I mean if someone writes in
korean no one suggests they are not literate because it's _easier_. but if
someone can only communicate by drawing, like some ultra-fast pictionary
competitor, it's hard to say they are literate.

~~~
dragonwriter
> it does not matter if one is born in china or UK or korea - children are
> given huuuuge amounts of training to become literate. We don't dumb down the
> writing system, and we don't dumb down the literature written in the system,
> we spend more on training the kids.

We definitely dumb down literature for kids. In fact, we dumb down literature
in a gradient to provide a gentle on-ramp.

For the writing system, that's a bit more complex of a question, but at a
minimum we judge their use of it more tolerantly (same with spoken
language)—this seems somewhat parallel to using a simplified language or
environment in programming.

------
a_imho
After reading the article I don't get the author's problem or the proposed
fix.

~~~
randomsearch
An ironic answer would be, yes that’s his point.

I think the problem is: Programming is designed for and by a small group of
programmers who enjoy making it difficult.

The solution is to create industrial societies / institutions / guilds etc.
who can encourage a transition away towards simpler and more accessible
programming approaches.

~~~
AstralStorm
Please describe some of those "more accessible" approaches and I'll probably
tell you that they are already here and/or have already failed.

Otherwise the idea is completely not constructive.

~~~
randomsearch
I'd say your comment is not constructive: you're asking me to name all the
solutions, otherwise my observation has no merit?

I'll expand a little. Let's take a single example: developing a single page
web application.

To know how to do that, I need to learn HTML, CSS, JS Frameworks, web pack or
whatever. Linters, preprocessors, package management ....

Contrast that to building flash app where a designer could use a GUI to
produce a beautiful interactive app in no time at all.

Yep, we all hated Flash, but in terms of accessibility to creators, it was
beautiful compared to the mess we've made.

A little off-topic but good for perspective:
[https://www.youtube.com/watch?v=lKXe3HUG2l4](https://www.youtube.com/watch?v=lKXe3HUG2l4)

~~~
lkschubert8
Aren't you then asking for a WYSIWYG editor or a CMS that includes something
like that? Many of those exist and are suitable for a lot of web development.
You step into the lower level for customization that is difficult or just flat
out not implemented in the higher level tooling.

------
PurpleRamen
"Everything Should Be Made as Simple as Possible, But Not Simpler"

Programming is always a tradeoff between ability and knowledge of the
programmer, time he can invest, performance the product must deliver and the
unpredictable future of the product. Nothing can be perfect because nothing is
perfect.

------
kbouck
Enterprises, with the best of intentions, routinely opt for what appear on-
paper to be lowest-cost developers which end-up being the most expensive
option when factoring in the resulting delays, faultiness and complexity of
what is built.

Generally speaking, complexity of a software system is inversely proportional
to the collective skill and experience the development team.

If I want quality products or services, in any field, I shouldn't expect to
get it from the lowest-cost option.

------
teknico
"When do we start throwing obsolete things away?"

We started already: work is afoot to throw away C, C++ and JavaScript, the
main culprits. Rust is a good stab at all three.

