
Pain we forgot - Morgawr
http://www.lighttable.com/2014/05/16/pain-we-forgot/
======
ggchappell
What a wonderful article.

I'm not allowed to forget a lot of this pain. I teach programming, so I see it
anew every semester.

And of course I still experience much of it in my own work. So, yes, let's
deal with these issues better.

One little disagreement. In the "What?" and "Why?" sections the writer present
some ideas for debuggers. While these are good ideas, I prefer to think of the
inadequacies of existing debuggers as motivation for good practices:
modularity, loose coupling, unit testing, etc. Certainly, it would be nice
(say) to be able to examine the entire state of a running program easily. But
I would rather code in such a way that I do not need to.

So to those who would write the World's Greatest Debugger, I say, "Good for
you." But even better would be to turn your efforts to producing languages,
libraries, frameworks, and programming environments that make such a debugger
superfluous.

~~~
jamii
> I would rather program in such a way that I do not need (say) to examine the
> entire state of my code.

But if you could examine the entire state, why wouldn't you? People build
correct intuitions by comparing their mental model to reality. If you don't
ever show see reality, or only peek at it through sprinkled print statements,
then it becomes harder to build an accurate mental model. This applies whether
we are talking about a beginner trying to understand how pointers work or an
expert grappling with a complex system.

Programming is always limited by our ability to comprehend complexity. So
enabling simpler systems is a win, sure, but so is getting better at
understanding things.

~~~
ggchappell
> But if you could examine the entire state, why wouldn't you?

Well, I would, of course. As I said, I think his debugger ideas are good ones.

OTOH, there is a limited amount of effort available for improving programming,
and I would prefer that such effort be directed toward other ends.

Part of the problem is that, in my experience, debuggers tend to encourage
sloppy thinking. And the better the debugger is, the sloppier is the result.
Using a debugger, together with thinking hard about a problem, can be a
powerful combination. But all too often, I see debuggers used as substitutes
for thinking hard; the way debuggers are designed seems to encourage this.

To put it in the terms you used, constructing a mental model and comparing it
with what the debugger shows, is a great idea. But in practice, I'm faced with
a choice: work on my mental model now or fire up the debugger now. The latter
is less taxing, and is often chosen; this is likely harmful in the long run.

~~~
seanmcdirmid
> But all too often, I see debuggers used as substitutes for thinking hard;
> the way debuggers are designed seems to encourage this.

I wish there was a device that would help me think less hard...oh wait, there
is! Its called a computer!

The lack of a decent debugging experience in languages like Haskell mandates
deep thinking upfront along with powerful bondage-style type systems. I can
imagine that a language with a more advanced debugger would push things the
other way.

~~~
coolsunglasses
>bondage-style type systems

For the rest of you, when somebody describes Haskell in this manner you can be
assured of a few things.

One, they don't know Haskell better than extremely superficially. Two, they
don't understand parametricity and haven't read the papers. Three, they don't
know what it's like to do the same kind of work (day-to-day programming) in a
dependent type system.

~~~
vertex-four
> haven't read the papers

I've never needed to read a paper to understand any other language. If I need
one to learn Haskell, that's an immediate downside.

~~~
coolsunglasses
You don't need to read any papers to _use_ Haskell, but if you're going to
explain the type system, you need to understand how it actually works.

------
gedrap
I totally agree. It's true even for us, experienced Web guys (and gals), in
this mess ruled by packages and their managers.

For example, I am trying to make my first Rails app outside a tutorial.

Okay, I want to use bootstrap. Should I use gem? Guess that's the ruby way.
Okay seems like there a few of them. Tried one, another one. Doesn't work.
Don't know why, since I am just copying half-cryptic stuff because I am new to
Rails.

A friend suggests to use Bower. It's easier. Right! I had totally forgotten
about bower, it rocks! Google: bower rails. Okay, there is that thing
sprockets which I apparently need to configure. Googled, a few blog posts
opened, they offer a bit different advice each, let's try. Doesn't work. Nope.
Not really... Google: bower rails bootstrap sprockets. And yay! Works.

Two hours later, I have included Bootstrap. Properly.

~~~
joevandyk
Download bootstrap css, put into app/assets/bootstrap. Should be all that's
needed.

~~~
gedrap
Yeah, it would have worked. But I am too lazy (among real disadvantages) to
download all the libraries manually :)

Real disadvantages such as updating becomes so boring that you will probably
never do that. And 'bower search <something>' followed by 'bower install
<found it!>' is less distracting than googling and unzipping etc.

Being lazy pays off.

~~~
Cacti
If you're a programmer and you can't even bother to download some files and
unzip them, then...

Lazy pays off because you innovate in the hopes of automating something, not
that you just are actually completely lazy.

------
logicallee
I'll delete this if you don't like it, but I think this is why non-engineers
like Steve Jobs, Jony Ives, and a person at HP I won't name, were able to make
remarkably good consumer products without knowing how they work. It may be
better for a non-engineer to design a laptop than for someone who actually
knows how it is put together and exactly how it works.

Of course, in a narrow sense, this means such a person isn't really designing
it at all: the real engineers are, which may cause resentment. There is a very
good chance that a non-programmer can design a programming IDE that is two or
three orders of magnitude better (by whatever standard) than the status quo.
This means such a person can't actually implement any part of it, or even know
exactly what it's doing.

Quite a surprising conclusion.

By the way I have experienced this myself, when designing for a target I
didn't know yet: after/while I was learning it, the resulting design iteration
process was much worse than when I didn't know the implementation details.
It's harder to think from the user's perspective, after you have been forced
to think from the implementation's perspective.

~~~
dreamfactory2
I don't think it's quite ignorance of the implementation domain is the key
here, but rather outside-in vs inside-out thinking. Nearly all engineers solve
problems from the inside in my experience.

~~~
logicallee
It's not just that though. Something may be impossible to do in the
application, except with knowledge of x, but trivial for someone who knows x.
If you know x you might totally forget that without this knowledge the task is
basically impossible for the user.

So if building the application engenders learning x, then you might totally
miss this design problem, unless you step away long enough to forget x again
yourself - and that may never happen.

It is a bit too much to ask for one person to learn all of the details and
totally own the implementation space, and yet be able to instantly forget all
those details and go back to when they didn't know anything, to look at things
from the user's perspective.

------
SolarNet
Personally, I think this is an extremely important topic, we need to change
the way we program. It isn't the 70's anymore, why do we still program like it
is.

Why does it take teams of developers to create and manage applications which
do really simple tasks in the grand scheme of things (and I realize the amount
of complexity in building applications is staggering, but large portions of it
could be better automated). Where are the auto generated GUIs, where is the
ability to ship execution control to arbitrary devices, where is a hypermedia
layer with independent view and presentation code?

I'm approaching this from a different angle than the light table guys appear
to be (I agree with everything they are saying). My angle is an attempt to
build a cross platform module system (where platform includes runtime and
programming language as well operating system and architecture):
[https://github.com/OffByOneStudios/massive-
dangerzone](https://github.com/OffByOneStudios/massive-dangerzone)

My argument is: before we can build the next generation of useful tools, we
need a framework for managing both generations. massive-dangerzone is an
attempt at bringing next generation usage (like that described in the article)
to existing tools. It's still a big work in progress though, and is barely
useful at the moment.

------
fragmede
> I want to just type 'email' and see a list of functions and libraries
> relating to email.

Do people even remember what life was like before Google?

I type 'email python' and get back a link to an email module. Am I being
closed-minded in thinking that can't get much easier?

Yeah, I need to understand a bit as to how email works, smtp and imap/pop and
what not, and how to send vs receive email, but some level of understanding is
just necessary.

~~~
cjslep
I have a similar experience with Go in my spare time projects (very refreshing
coming from a FORTRAN/C++ day job) which is a boon, but falls short of what
the article describes. The way I interpreted it, even having to do the search
in the first place is a problem to be solved. Being able to start typing
"email" and have a context menu with functions available from a list of third-
party libraries relating to email would be the goal. That way when you select
"SendEmail()" in your code, it automatically pulls the library, installs the
binary where it needs to be located (or pulls source and builds) without
interrupting your workflow.

With package management systems standardized within a language (such as Go)
the problem domain is reduced, but extending the solution across multiple
languages sounds like another third-party tool than a language feature.

~~~
_random_
You mean something like Bing Code Search but with NuGet support? That would be
cool.

[http://techcrunch.com/2014/02/17/microsoft-launches-smart-
vi...](http://techcrunch.com/2014/02/17/microsoft-launches-smart-visual-
studio-add-on-for-code-snippet-search)

[http://visualstudiogallery.msdn.microsoft.com/a1166718-a2d9-...](http://visualstudiogallery.msdn.microsoft.com/a1166718-a2d9-4a48-a5fd-504ff4ad1b65)

~~~
jamii
Yes, exactly like that. I can't wait to see how Bing Code Search evolves as it
starts to learn from its users.

------
cclogg
Oh man, I've definitely felt this pain many times lol:

"The samples will be missing lots of implicit information such as how to
install the necessary libraries and how to deal with missing dependencies and
version conflicts. Transcribing and modifying the examples may lead to bugs
that suck up time. It's not terrible, mostly thanks to sites like
stackoverflow, but it's still a lot of unnecessary distractions from the task
at hand."

So many times, the actual programming isn't tough, it's just getting all of
the stuff around it setup that is hard.

~~~
tluyben2
This is what I have mostly with Rails; when I started working with it I was
good at Ruby but didn't use Rails and for a beginner it's horrible. There tons
of options for everything, everything is badly documented, most things scratch
an itch so when you use it, some of the completely logical things you expect
are missing. Samples of gem usage randomly skip steps which can be a hair
pulling exercise for a beginner. Then when you finished your app and try to
update gems a week later, everything breaks completely. And this is not an
attempt at trolling; I now use Rails daily and build rather huge projects in
it and this still upsets me. When things work and you _know_ how they work,
all is easy and great, installing and using. When either or both of those are
not true, you are in absolute hell, especially as a beginner. And nothing of
that has to do with actual programming and it shouldn't be needed; your
environment should solve this for this imho.

------
unwind
I was somewhat surprised, but very happy, to see Verse mentioned in this
context (although he did get Eskil's last name wrong, it's "Steenberg").

I co-developed the initial version of Verse with Eskil; I think it was a bit
before its time perhaps. It was hard to get real traction for it, but at least
the things Eskil has gone on to build on top have gotten some real attention.
Great, and very well deserved!

~~~
jamii
Oh man, I am full of typos today :S

Verse was a huge inspiration for me and it's a shame to see it largely
ignored. We're exploring similar ideas for getting our language tooling away
from plain text and towards structured, networked representations. I would
love to hear your thoughts on how to do that, or any advice you have from
building Verse - jamie@scattered-thoughts.net

EDIT I also thinks Eskil's ideas on 'what we can do' vs 'what we can get done'
applies just as well to the rest of the programming industry. We're so focused
on scaling up to millions of lines of code that we end up with environments
where lunch_app takes a week to build and deploy.

------
mkozlows
Articles like this one annoy me, because it's easy to diagnose big problems --
here, watch me do it: "Why should compilers choke if you forget a semicolon?
If they can diagnose a syntax error, can't they also fix it? Can't functions
be smarter about seeing when they're used improperly and tell you while you're
writing the code instead of when you're running it? Why can't the code be
understandable to anyone who knows English?"

What's hard -- and often impossible -- is fixing those big problems, because a
lot of times they're genuinely intractable; and when they're not, they're
often so difficult that they might as well be.

So just sitting around and complaining about them sounds insightful, but it
doesn't really get anything done. And yeah, I know that they're allegedly
working on "fixes" for these issues, but based on the track record so far
(LightTable promised all sorts of revolutionary views of what an IDE could be;
it's delivered... not a whole lot of revolution), I don't have any faith that
Aurora is going to amount to much either.

And I don't want to be too negative, because sometimes a previously-
intractable problem turns out to now be tractable, and it takes someone who
was willing to question long-accepted pain to find that out. So I'd be
pleasantly surprised if this project delivered something that had even as much
effect as the development of, say, git or xunit. But I'm not holding my
breath.

------
weland
I also wish that programming were a lot different today than it was when I
started learning it. That being said, a lot of this article's points are
things I've heard before. They led to the development of Visual Basic & co.,
mostly by people who had no contact with the Smalltalk and Lisp environment in
the 80s, while people who _did_ were shrugging and throwing tantrums like WHY
THE FUCK DIDN'T YOU FUCKING LIKE IT TEN YEARS AGO?

IMHO, all these things went down to the bottom of history because things like
these:

> Anon the intern needs to be able to open up Programmingâ„¢ and click 'New
> Web Form'

are adequate for people who usually don't program, and extremely inadequate
for people who usually do. Generally, and for good reasons, programmers will
dislike a tool that hides implementation details for ease of operation. Past a
certain level of complexity, the time spent manually doing the _right_ cruft
becomes significantly smaller than the time spent manually cleaning up after a
smart tool.

I sympathize with Anon the intern, but perhaps he should rethink his
expectations about complexity; if discoverability is a problem, perhaps he
could switch to something that's better documented?

And at the risk of sounding like an elitist schmuck who rants about how things
were back in his day, _maybe_ he ought to start with something other than web
programming. The size and complexity of that tech stack is humongous, to the
extent that a large proportion of those who _use_ it don't understand it more
than two layers of abstraction down. Programs are also hard to pack and the
environment that runs them is hard to setup. Because it involves at least two
servers, possibly with several add-ons in order to allow the server-side
languages to run, learning at least three languages (assuming server-side JS
is an option), two of which (HTML and CSS) aren't quite being used for their
original purpose. This is a beginner's nightmare and it has exactly nothing to
do with the development tools.

And then there are things that are far harder to solve than they originally
seem:

> I want to just type 'email' and see a list of functions and libraries
> relating to email.

Related how :-)? Should MIME-related functions, needed to reason about
attachments, also come up here? HTML parsing/converting, in case you need to
deal with HTML email? Information cluttering does nothing to alleviate the
opposite problem of information breadth: if Anon the intern's problem is he
doesn't know how to Google for libraries or how to make efficient use of
documentation, an IDE that presents him with a gazillion of possibly related
things won't help him. _Especially_ when, like all beginning programmers, one
of his main difficulties is correctly defining the problem he's working on
which, in turn, makes it likely for the solutions presented by the IDE to be
nowhere even _close_ to the one he needs, because the IDE (like Anon himself)
thinks Anon is trying to solve another problem.

There is, on the other hand, a lot more truth in this:

> Tightening the feedback loop between writing code and seeing the results
> reduces the damage caused by wrong assumptions, lightens the cognitive load
> of tracking what should be happening and helps build accurate mental models
> of the system.

I do think that the real resolution to this problem is writing simpler
programs whose state is easier to track. On the other hand, programming tools
today suck considerably at presenting program meaning. Things like evaluating
what an expression comprising entirely of constants, or at least evaluating it
based on the default values of the variables involved, are well within reach
for today's tools, and yet programmers' calculators are still employed because
99% of the available IDEs couldn't evaluate ADDR_MASK & IO_SEGMENT if the life
of every kid in Africa depended on it.

This is wicked cool:
[http://repository.cmu.edu/cgi/viewcontent.cgi?article=1165&c...](http://repository.cmu.edu/cgi/viewcontent.cgi?article=1165&context=hcii)
. However, I also find myself thinking that the very fact that _we need_
debuggers that are this smart is proof enough that we don't reason about our
programs well enough. Except for the fringe case of having to quickly debug a
(possibly horrible) codebase I haven't written, I'd much rather prefer being
good enough a programmer to avoid the need for a debugger that can tell me
_why_ foo is not 8 despite the fact that I fucking said foo = 8 ten lines
above, than being a programmer with good enough tools to help me when I'm
stupid.

~~~
jamii
> They led to the development of Visual Basic & co

Visual Basic was phenomenally successful as an end-user programming tool.
There are entire businesses that still run off VB6. It worked because they
nailed the early learning curve. It doesn't matter if your language is faster
or more maintainable or less bug-ridden and insane - the majority of people
will always gravitate towards the tool that is easy to get started with. This
is why VB6 and PHP are so widespread.

The moral of the story is that if you want people to program in a sane and
scalable way you must make it the path of least resistance. No amount of
advertising or education has ever been able to overcome the fact that most
people don't give a shit about programming and are just trying to get stuff
done.

> if Anon the intern's problem is he doesn't know how to Google for libraries

Anon the intern's problem is that the information returned by google is
incomplete and requires manual effort. I don't want to replace stackoverflow,
I want to make it executable.

> I do think that the real resolution to this problem is writing simpler
> programs whose state is easier to track.

Well, that's where the managed state part comes in. We are trying to separate
essential state which is actually part of the program logic from accidental
state which is just an implementation detail (an idea expounded in
[http://shaffner.us/cs/papers/tarpit.pdf](http://shaffner.us/cs/papers/tarpit.pdf)).
That's a whole other post though.

> I'd much rather prefer being good enough a programmer to avoid the need for
> a debugger that can tell me why foo is not 8 despite the fact that I fucking
> said foo = 8 ten lines above, than being a programmer with good enough tools
> to help me when I'm stupid.

There is an implicit assertion here that not having a debugger makes you a
smarter programmer. Would removing other feedback print statements, unit tests
and the repl help too? Why even execute code at all, just print it out and
think hard.

Thinking hard doesn't work if part of your mental model is wrong. Being human
(unfortunately) my mental model is often wrong and a quick glance at a stack
trace does more to enable correct thinking than hours of working the problem
out by hand with pen and paper.

The whole point of having a computer is that it can do some of the thinking
for you.

> maybe he ought to start with something other than web programming

The goal is not for Anon to learn to program, it's to build lunch_app. The web
sucks but it is the only good distribution platform we have. Either we use
that or we build something easier on top of it.

~~~
jrochkind1
> Visual Basic was phenomenally successful as an end-user programming tool.

Was? So what happenened, where did it go, why isn't it still filing the role
you are aiming at?

Another example, held in somewhat better regard by programmers, of a
phenomenally successful tool that let end-users build working programs, is
Hypercard. Which definitely isn't around anymore.

Looking at the prior art of tools that have succeeded at letting end-users
build programs is probably a good bet. What they did right, what they did
wrong, why they don't exist anymore even though they were succesful, if they
don't exist anymore.

~~~
wvenable
> Was? So what happenened, where did it go, why isn't it still filing the role
> you are aiming at?

The last proper version of Visual Basic was released in 1998 -- Microsoft
discontinued it in favor of VB.NET. If they continued to develop VB, I think
it would still be a heavily used platform.

Hypercard is not the same thing. People didn't build real applications in
Hypercard -- it was a toy. I spent a lot of time playing with Hypercard and it
was very VB-like when programming for it but the UI was more like Flash. It's
best use was making simple games.

The real problem with simple tools is that they solve simple problems. Once
you have to start doing something outside of the tools range, you're stuck.
This happens with every tool but more complex tools have a bigger range.

~~~
alecdbrooks
> People didn't build real applications in Hypercard.

Remarkably, they did! Wikipedia:

>A number of commercial software products were created in HyperCard, most
notably the original version of the interactive game narrative Myst, the
Voyager Company's Expanded Books, and multimedia CD-ROMs of Beethoven's Ninth
Symphony CD-ROM, the Beatles' A Hard Day's Night, and the Voyager MacBeth.

It's true that Hypercard was not an ideal platform for development.
Apparently, Myst developers had trouble making the high resolution images load
quickly on machines of the time.

~~~
wvenable
As I said, Hypercard was like Flash -- so it's not surprising that all these
commercial products are sort of point-n-click multimedia projects. But you
couldn't, for example, build an email application in Hypercard. You could,
however, build one in Visual Basic.

~~~
robterrell
Sure, you could build an email application in HyperCard. I'm pretty sure there
actually were several. TCP/IP was available via HyperCard plugins or
extensions or whatever they were called. HyperCard's card/stack data model
would work fine for email. Making the list view would kind of suck, as lists
always did in HyperCard, but you could use a plugin like WindowScript.

I wrote several commercial software products in HyperCard and SuperCard back
then. It provided awesome tooling leverage. I worked on an EDI application
that was written in SuperCard (with the EDI processing running in a C plugin)
which was working, but later the company's president decided we needed to re-
write the whole thing in C++ with the MacApp framework
([http://en.wikipedia.org/wiki/MacApp](http://en.wikipedia.org/wiki/MacApp))
because C++ was a "real" programming language. This was the cfront era --
build times were long, debugging meant manually de-mangling names, etc. Crazy
slow. We spent a long time on it but the schedule slipped long past the end of
my time there.

------
shortstuffsushi
The analogy to me seems similar to that of programmers and compilers. I want
to write programs to do xyz, but I don't particularly care to know the lower-
level to machine level implementation. Instead, I know a roughly-english set
of instructions to tell the compiler what I want to do, and it takes care of
the rest for me.

Could it not be possible to take this even further, to the point where I say
"I want to do ..." and all of the "code," be it high or low level is generated
for me.

I realize making something capable of this level of abstraction would be
incredibly difficult, but it would certainly be fun to try.

~~~
GFischer
There are companies trying.

In Uruguay, a tool called Genexus enjoys a high level of success promising
something like that :) .

For simple CRUD apps, it's pretty much magical - and it looks awesome in
demos, heck, it can generate code for every mobile phone out there just
throwing around some controls, specifying your data fields and writing a few
rules, but if you want to make it do something that isn't explicitly
considered, it's very annoying (and it has some UI and UX problems). And don't
ever try to touch the autogenerated code :P .

That said, it's probably the easiest way to get from 0 to a complete CRUD app,
once you learn how it works.

[http://www.genexus.com/products/genexus?en](http://www.genexus.com/products/genexus?en)

------
rafaelferreira
Not often I can read an essay that leaves me at the same time so excited and
so disappointed.

Excited because I found myself agreeing with every point: isn't it obvious to
everyone that the programming experience could be so much better? We created
wonderful tools for graphical expression and number crunching, that keep
getting better [1] [2], while our own day-to-day tools remain basically in a
rut.

Disappointed because it seems LightTable is foregoing an incremental path to
reach the goal, choosing the boil-the-ocean approach. We do need more long
term start-from-scratch rethink everything kind of projects; like what VPRI's
STEPS project aims to achieve [3]. But there is a lot that can be done to
improve the programming experience today, and I don't see enough work in this
area. IMO, the latest meaningful improvement in software development tooling
was Intellij Idea around 2001 (arguably the functional programming renaissance
represents another meaningful improvement, but the real breakthroughs there
happened in the 70s). LightTable moving to the Blue plane leaves the PinkPlane
unattended.

[1] [http://www.adobe.com/technology/projects/content-aware-
fill....](http://www.adobe.com/technology/projects/content-aware-fill.html)
[2]
[https://www.youtube.com/watch?v=UccfqwwOCoY](https://www.youtube.com/watch?v=UccfqwwOCoY)
[3]
[http://www.viewpointsresearch.org/html/writings.php](http://www.viewpointsresearch.org/html/writings.php)

~~~
jamii
We aim to have some demoable in the next few months and actually usable this
year. We're purplish, at worst :)

------
tim333
Regarding Anon the intern wanting to build lunch_app I've been in a similar
position learning programming from the basics and wanting build some apps.
I'll put in a vote for web2py as being a good tool that gets around a few of
the gripes in the article. It's largely one click to install server, editor,
db, debugger etc. and was specifically designed for teaching beginners (it has
versioning as well using mercurial but I wasn't able to get that going with
one click). It also does most stuff you'd need and is all in Python so you can
use the libraries and rewrite bits if you can't get the framework to do
something. Even so it would take Anon a while to make his app but I think it's
some sort of progress. The creator talks about it here
youtube.com/watch?v=iMUX9NdN8YE

~~~
jamii
I'll check it out, thanks.

------
lemming
I'm glad to see IntelliJ getting some love there for making much of this
easier - few companies have worked so hard at it, and the results are amazing.
Looks like Microsoft is making some great steps too, Bing Code Search gets a
bad rap for pandering to blub programmers but I'd love something like that in
my editor.

------
perrygeo
Or the pain we remember with fond memories. Like anything good in life, the
struggle to do it well, to fully understand it, to master it, is what makes it
worthwhile. It's also what makes programmers effective - we revel in solving
complex but tractable problems.

There is an inherent tradeoff in creating any system: you can make the system
very simple and it becomes too simplistic for the advanced users who want more
control and detail. You allow too much control and detail and you alienate
users who want simplicity.

A programming language/IDE is a system that affords advanced users the ability
to solve an immense set of problems. That unrestrained ability comes at the
expense of simplicity. The more constrained and specific that ability becomes,
the simpler it can be (i.e. Excel is more constrained but far simpler than a
Hadoop cluster)

------
Goopplesoft
> We still program like it's 1960 because there are powerful path dependencies
> that incentivise pretending your space age computing machine is actually an
> 80 character tty. We are trapped in a local maximum.

Love this (although I don't like extra long lines). He's talking to you Pep8.

------
webmaven
While I am watching the development of LightTable with interest, it seems to
me that the example used in the OP is missing the forest for the trees. It
isn't hard to imagine Anon Intern managing to cobble together a solution for
lunch_app that isn't an 'app' at all, just a form created with a builder like
WuFoo or Google Forms, and some 'recipes' in a tool like IFTTT that has built
in integration points for email, the accounting system, etc.

Arguably this wouldn't much of an improvement over the VB6 + Access status quo
(or Visual FoxPro, or FileMaker Pro, etc), except that the individual
components can be much more robust and scalable, monitored, auditable, and so
on, without Anon Intern having to worry about any of that.

~~~
jamii
I mentioned that near the beginning:

> There are probably dedicated apps that cover this particular example but we
> are more concerned with how an end-user would solve this kind of problem in
> general...

VB6, Access and Excel are exactly the kind of tools we seek to emulate on the
usability front. Half of the world runs on Excel precisely because it is easy
to get started, easy to understand and yet is capable of handling fairly
complex simulations. If we can make something equally approachable but backed
by a sane, maintainable and powerful language then we can give programming
powers to a lot of people who need them.

~~~
webmaven
Except that I am not talking about a dedicated app at all, but general purpose
tools/services for parts of the problem that can be combined arbitrarily, just
like libraries can. IFTTT is exactly that sort of service.

------
politician
"The samples will be missing lots of implicit information such as how to
install the necessary libraries and how to deal with missing dependencies and
version conflicts. "

Docker pretty much solves this problem. Writing samples for your new project?
Start with step 1 `docker pull yourproj/your_tutorial_image`.

------
SSLy
Edit: I've mismatched deps and imports, indeed that could be useful. Disregard
my post.

 _If I select a function from autocomplete, its dependencies should be
automatically added to the project without any fuss._

For the point, that that post makes, this one is at least available if you use
IntelliJ with Java or Scala.

~~~
edanm
You know, I've recently started thikning - why do we even need explicit
imports? Isn't the compiler/etc smart enough to figure out the right import
99% of the time?

Every file in a large project starts with dozens of redundant import lines.
Why don't we have imports go the way of the inferred type?

~~~
riquito
What if foo() is present in two different libraries? (is this the 1% you're
talking about?)

What if tomorrow I add another library which implement his own foo()? (this is
trickier)

When I read the code, what is foo()? (here the IDE may be smart and show me
the right documentation).

All that said, I feel that the pythonic "Explicit is better than implicit" is
better.

It's also true that when I see a file whose first 30 lines are used for
including dependencies it doesn't sound right.

The problem lies with the fact that we use text to store structured data. If
we were to save source code as binaries we could have the source code in a
"file|object|item" and the dependencies in another item, linked to the source
code (I'm not advocating this, source code in text files has many advantages,
but also his drawbacks).

~~~
jamii
We give each name a unique id under the hood. At edit time the user chooses
which one they want from a list. The actual AST only ever refers to ids and
the names are effectively just documentation.

This has the neat side-effect that ids are globally unique so you can figure
out what libraries you need to pull down just by looking up ids in your
package repository.

------
zem
solving this problem was part of REBOL's dream.i really wish it had been
properly open-sourced from day one and built up a good community and momentum
around it; it was a very promising language.

------
ilaksh
Can't be sure but it seems like the surveying has resulted in most of the
right conclusions and the right direction for the project.

HOWEVER, I still think most everyone is missing the REAL issue here. As
evidenced by the powerful tools cited in this article that address various
aspects of the "programming" problem, there have been numerous efforts to move
the state-of-the-art in software development forward. And to a great degree
those efforts have _proven_ quite a few superior paradigms.

And yet, we haven't seen those new paradigms become truly mainstream for most
programmers. Why? I do NOT believe it is because the approaches haven't
integrated the right new concepts, because there are so many existing useful
tools combining many different new ideas effectively which have failed to
become mainstream among programmers.

I think many of those new approaches could and should have become normal
operating mode for programmers.

I think the reason they did not is this: __the core definition of
"programming" (and by extension "software engineering" etc.) is an
antagonistic and largely manual process of creating complex textual source
code that can be used to describe system behavior. Period. __

For example, if I want to call myself a web "developer" I DARE NOT use an
interactive GUI tool to generate and maintain the source code for my web site.
Web developers will always say, for example, they avoid this because the
source code generated that way is less maintainable by hand. In many cases
that may be true. In some cases with advanced code generation it is not.
Regardless, I don't believe that is the actual reason. Its just a
rationalization.

What do we call someone who's entire job entails creating a web page using a
graphical user interface? In other words, this is a hypothetical person who
has found a hypothetical GUI tool that can accommodate all of his web site
design and implementation needs without any manual edits to source code. If he
builds a web site or web application this way, and writes zero lines of code,
do we refer to him as a very smart and advanced web "developer"? Or do we call
him a web designer or simply a WordPress user (for example)?

We do NOT refer to him as a web "developer". He has no right to refer to
himself as a "developer" or "programmer" because __he has not wrestled through
an antagonistic manual process to create complex textual source code __. And
that 's what people are not understanding. The reason we don't call him a
programmer is NOT because he didn't create an effective program or website.
Its because the way he did it wasn't hard enough and doesn't match our
outdated definition of what "programming" is.

To create a POPULAR system (among "real" "programmers") that makes programming
more practical or easier using new paradigms, you must either redefine
programming to include the possibility of new paradigms and a non-antagonistic
process, or perhaps somehow trick programmers into thinking what they are
doing is actually harder than it is. Maybe if there are a few places to type a
command to generate source code, that will be sufficiently complex to still be
considered "programming".

If you are too successful without doing those things then you will just have
another tool that only "users" or "beginners" would ever admit to using.

~~~
adrusi
Programming can get easier, and it has, many times. The introduction of
structured programming, the introduction of various forms of polymorphism, the
popularization of automatic memory management, and several other developments
have made the process of programming easier.

But while the process of programming became easier, the role of the programmer
did not. Instead, the programmer was expected to build bigger, more complex
systems, and do it faster than before.

I would hesitate to call someone a programmer for just making a web form in
HTML and sending it to a database, and a GUI tools to do this already exist,
most notably Microsoft Access. But suppose a GUI tool was invented that
allowed its user to create any website, from Google search to eBay and
anything in between, including both the front- and back-end. Anyone well-
versed in using that tool would be a programmer for sure.

------
zan2434
Saved. Can't wait to look back at this in a few years.

------
iammyIP
Maybe every tool that we invent and use is two-faced: it makes life both
easier and harder, and therefore the wish to make programming easy could
result in one of the most difficult journeys so far, comparable with inventing
an AI indistiguishable from a human. If i imagine a success in 75 years, then
i would probably sit infront and talk to a human-like android: "play me some
music based on Bach with the voice of Eddie Murphy, mellow, with a hint of
tragedy, but not too sad" \- then it would in the blink of an eye play me some
music and sing along with it. Chances are i would change Eddie Murphy to some
other singer that is readily available at the huge human voice archive this
machine has access to. But i am bored with archived and old fashioned music, i
want something new and exciting. So i try to teach the machine a new voice. As
it turns out it is pretty good at mixing characteristics of multiple voices
and synthesizing a new one out of it. I let it do some hundred random
variations with hints on what parameters i prefer. "that rough whiskey vibe -
keep that and mix it with Presleys guttural hiccups"... "no , not like this,
listen". I try to sing what i have in mind, but i am no good singer, so the
next test comes out worse than before. "go back ... one more". We spend the
rest of the day analyzing Presleys archive and filtering out the guttural
characteristic that i meant. I heard i am an expert programmer, my grandma
called me that, and i sit down with this android for a whole month and explain
in all details the music i want to hear, constantly listening and tweaking my
instructions in a flawless instant feedback cycle. The most time spent, or
what i sometimes feel - wasted, is actually searching for the right words in
my own head to describe what i mean. After 2 weeks i found out that we have
broken down the characteristic of a singers voice into twohundredandeighty
relevant parameters. Relevant not for everyone of course, but for me and this
music i am working on. Maybe i am taking this too far. "delete
MelodramaticChorusAccentuationTongueModulationMode two to four". I spend the
next week with condensing these parameters down to thirtythree. At some days
in the next week i am really without any inspiration and just let the android
play thousands of random variations based on the current version that i judge
on a 1 to 10 basis while doing some gardenwork. Sometimes i would only change
a little note, or the single expression of a syllable. "uah insiide meeee...
you see the "siide" must sound more desperate because this guy is on the verge
of losing his lifelong dream at that moment." \- we spend the rest of the day
tweaking that "desperate" thing. This android is amazing. After a month and
three weeks the music is ready. I save it and do some other things, mostly
gardenwork and watching the drones fly at the evening - they really got some
nice new formation techniques that i enjoy greatly. The next day i speak with
Andreas (another real human) and tell how good the android has learned my
personal preferences for singing voices and how fractalising Bachs harmonic
structures to the fifth degree is worth it but no further without structural
simplification at the base while keeping the dimensionalitys denominator
intact (that was really a complicated talk, i cant get into all the details
here). We agree to exchange our Androids preference patterns - my bach-
presley11786 for Andreas painting-images9904, and our androids get
automatically updated. Next day i try it out. This may take a while, Andreas
has warned me. But over the next days i can watch the android painting a
really amazing photorealistic but nonetheless astonishingly dreamlike image
with Andreas pattern, although the process is, as he said quite slow by
design, and one image at a size of four square meters takes about 3 weeks to
finish since it includes a complicated technique of iterated de- and
reconstruction with additional time for the oil paint to dry up inbetween. But
the result is definately worth it. After the image is finished, i call Andreas
and congratulate him for his exceptional good taste on imagery (afterall he
worked two and half a years on it). He also thanks me for the musical pattern
and asks if he can use the guttural Elvis thing in one of his next Crazy
Donkey Singalong Performance. That asking just being a polite convention, i
naturally give him full permission. We agree to hold another meeting to talk
in detail about his structural approach to synthesize dreams without crossing
over to kitsch in two weeks. The rest of this day i am back in my garden, the
drones fly really low this week... maybe a thunderstorm is coming... but this
is great! Every human is a programmer now, which essentially means that he
tells other beings what to do, in more or less detail, and of course the fact,
that these to be told are not humans anymore, but human-like machines. So, if
one of these humans does not tell a machine what to do, he usually enjoys not
telling other humans what to do (besides the endless debates about how to tell
another machine best what to do (and not to forget - the debates about how it
could be better to directly think to a machine rather than speak to it and why
it has not been implemented yet)).

------
dreamfactory2
Great article but isn't a lot of this more about using different levels of
abstraction i.e. frameworks, DSLs and specific to those? That could either
take the form of being sufficiently focused that a simple text editor is
adequate (e.g. high level commands, no cruft), or a full-featured managed
environment (state, docs etc) specific to the framework. I'm not seeing how
this is a task for a generic IDE unless I missed something about light table
and they are creating a new language.

~~~
jamii
> unless I missed something about light table and they are creating a new
> language.

[http://www.lighttable.com/2014/03/27/toward-a-better-
program...](http://www.lighttable.com/2014/03/27/toward-a-better-programming/)

The short version is yes, we are making a new language. The kind of workflow I
discussed in the OP was the original goal for Light Table and it turns out
that it can't just be hacked onto existing languages.

~~~
rwallace
Fair enough. I won't say your decision to make a new language is right or
wrong - I don't know how it will turn out - but given that you have decided to
do so, and given your objectives for it, my one big recommendation would be to
make relational databases central to the language, the way Perl makes string
processing central and PHP makes web pages central. Some possible ideas are
outlined at
[http://geocities.com/tablizer/top.htm](http://geocities.com/tablizer/top.htm)
\- you don't have to go with those particular ideas of course, but I think
it's important to make relational data central one way or the other.

~~~
jamii
We are heavily inspired by
[http://shaffner.us/cs/papers/tarpit.pdf](http://shaffner.us/cs/papers/tarpit.pdf)

------
naturalethic
Pretty sweet article. Needs a once over from the author for some edits though.
"if you don't have something useful to say, don't say nothing at all" made me
giggle.

~~~
bridger
I can't figure out what that one is supposed to mean!

------
dclowd9901
Are these guys ever going to actually finish the IDE, or postulate on what it
means to program, ad infinitum. I'm glad they're being considerate in their
design, but this whole project is starting to reek of over-aggrandized
vaporware.

~~~
jamii
[https://github.com/LightTable/LightTable/releases](https://github.com/LightTable/LightTable/releases)

There will be another Light Table release in the next few weeks, featuring
CodeMirror 4 (better performance, multiple cursors).

We've only been working on Aurora since February. Designing a new language
takes time. We may show some early demos in the next few months, depending on
how the current prototype works out.

~~~
nazgob
I think that some people (including me) are a bit disappointed that
some(most?) of your effort goes to Aurora instead of LightTable. Don't get me
wrong, Aurora sounds cool but for professional programmers LightTable is much
more appealing.

~~~
jamii
I can sympathise with that. The reason for our decision is that there is a low
ceiling on how much we can improve matters with Light Table. The problems we
are trying to solve turn out to be pretty baked in to existing languages and
toolchains and we can't hack around them in the editor. Aurora is a riskier
project but it has much more potential to help people.

Also, the fact that Aurora is intended to be simple and easy does not mean
that we are building a toy. Quite the opposite - the intention is to remove
some of the incidental complexity in programming to leave you free to focus on
the actual problems. We intend to bootstrap the compiler and editor by way of
dogfooding. I look forward to doing most of my programming in Aurora in the
future.

------
rch
I thought this project was about improving the programming experience for
programmers, not students (or interns) who will in all likelihood never excell
as programmers.

Edit: after downvotes I decided to read the whole thing afterall, but remain
unswayed. This guy has an important opportunity and I sincerely dislike seeing
it squandered.

~~~
alecdbrooks
I've been programming long enough that I know the magic incantations to get
started in a few languages. But I am still in Anon's shoes any time I learn a
new language or library. As the article says, "Even for experts, programming
is an exploratory process." So, I'd say everything in this article applies to
programmers, too.

Categorizing people into programmers and non-programmers neglects the people
who need programming but not at an expert level. Some interns and students
won't excel as programmers, but they still can bring programming into what
they do instead, especially if efforts like Light Table succeed.

I suppose I shouldn't be surprised by this kind of pushback against
reimagining programming — I've seen it as a response to some of Bret Victor's
projects. Skepticism is great: Maybe the ideas behind Light Table are wrong or
badly implemented. But we gain nothing from glib dismissals that seem to
regard any attempt at improving or simplifying programming as beneath them.

~~~
rch
I disagree with language like 'reimagining programming' in this context almost
as much as I regret seeing Bret Victor's name being dragged in for no reason
-- to the best of my knowledge, Bret Victor didn't design Light Table, and
doesn't inform its progress.

There is progress to be made, and I'll be excited again when I see some.

~~~
alecdbrooks
I've seen the same "oh, this is only for beginners" reaction to some of
Victor's ideas on making data visible and making libraries "discoverable" that
seems needlessly dismissive. That's why I mentioned him. Also, Light Table's
initial concept was inspired by his work[0].

I'm not always impressed by Light Table, but I appreciate the work the team
has put into the problem and think they have good ideas.

[0]: [http://www.chris-granger.com/2012/04/12/light-table---
a-new-...](http://www.chris-granger.com/2012/04/12/light-table---a-new-ide-
concept/)

~~~
lomnakkus
The problem with a program-as-the-mental-model is that you will invariably hit
a barrier (whether that be the halting problem or some simpler manifestation).
I'm much more sympathetic to the Don't Kill Math[0] view. You need (abstract)
analysis and no amount of IDE bling is going to eliminate that necessity.

[0] [http://www.evanmiller.org/dont-kill-
math.html](http://www.evanmiller.org/dont-kill-math.html)

~~~
jamii
[https://news.ycombinator.com/item?id=7763663](https://news.ycombinator.com/item?id=7763663)

