
Generating a Useful Theory of Software Engineering [pdf] - jwdunne
http://users.ece.utexas.edu/~perry/education/382c/Adolph-Kruchten.pdf
======
alephnil
In my experience, the two things that to the largest degree has improved
software development productivity the last 20 years is revision control and
continuous integration. Back in the nineties, the many did not use version
control at all, or used cvs or commercial alternatives that all made it a pain
to merge the changes from different developers. Later svn came along, and then
finally git and other distrubuted version control systems came along and made
it very much easier for a large number of programmers to work on one codebase.

Continuous integration, and in the tools for assisting doing that has made it
much easier to make sure you able to build the software you are developing at
any time. At less organized places it could be weeks or even months where
there did not exist a workable version of the software, and integration was a
pain. Sure this still happens some places, but now it is totally avoidable.

Unfortunately, almost none of the academic studies seems to take the tools
seriously unless you are talking about compilers. I think that tools still is
the area where most gains in software development productivity can be found.

~~~
whoisthemachine
I agree. I think the next steps towards improved software is TDD and
functional programming. Possibly along with higher levels of test automation
as well.

When a system can be automatically proven to meet the requirements a business
has of it, then much of the need for "reconciling perspectives" is removed.
The perspectives that matter is the _business_ perspective of what they need,
and the technical perspective of how to build it. Automated tests in between
those two will help solidify the matching of those perspectives.

------
milkytron
Although I haven't been in the industry for what others may call a long time,
I feel that after working at a few large enterprises, this is something that
upper management should read and understand in order to better relate to those
actually performing the development of projects.

Agile is the newest and most widespread buzzword in my current role and its
practices are being shoved down every team's throat. Those who adopt look like
better performing teams to management and become the favorite and most
flaunted teams so much to be idolized by others.

Meanwhile some of the smartest and proven teams are clearly producing immense
value, and ignore a lot of new methodology mumbo jumbo to continue operating
the way they have been for years.

It's an interesting dynamic, and I believe this paper captured a lot of what
I've been experiencing, but was unable to convey into words.

~~~
blatherard
Are you saying that the high-performing teams are continuing to work as they
were, and the other, presumably lower-performing, teams being made to act
differently? That doesn't sound too bad, at least on its face.

~~~
LoSboccacc
Flashy != Productive

working on stories instead of layers you push out finished ux and features
early on, and that's what both manager loves but ultimately kills the speed
after six month to a year, because those need to be maintained forever but
it's very rare to actually squeeze teme to refactor and even if you do, speed
slows down as you refactor

The team spending less on the ui and more on making sure they get the
datamodel right and the abstraction thight can quickly built an ui over the
stable platform and overall will have better speed past a year/year and an
half depending on the project

------
paulajohnson
A similar paper: [http://alistair.cockburn.us/Characterizing+people+as+non-
lin...](http://alistair.cockburn.us/Characterizing+people+as+non-
linear,+first-order+components+in+software+development)

The book "Peopleware" is also worth reading.

Unfortunately it seems that nobody has done any serious anthropology on
software production with a view to improving it.

------
k__
Is there more anywhere?

I think the ideas are right, but after telling me so, the paper enda without a
theory.

~~~
nickpsecurity
Here's you two on a theory of programming:

[http://www.cs.toronto.edu/~hehner/aPToP/](http://www.cs.toronto.edu/~hehner/aPToP/)

[https://bertrandmeyer.com/2015/07/06/new-paper-theory-of-
pro...](https://bertrandmeyer.com/2015/07/06/new-paper-theory-of-programs/)

~~~
jwdunne
Ooh a free book. Thank you, this looks promising :) have you read it?

~~~
nickpsecurity
Theory of Programs was a nice link. Other one was above my head.

~~~
jwdunne
I agree. Given the author, it makes more sense with some sense of DbC but it's
a slog.

------
craig0990
Reference [21] is written by the same author and is a much more detailed
thesis about the theory of "Reconciling Perspectives".

I haven't read all of it, but it's available on the University of British
Columbia website:

[https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/...](https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0073582)

------
Chiba-City
These are mostly solved problems (minus omniscience or scale) with just
handfuls of books that represent most parts of the solution.

Workaday environments suffer impediments to SE application not solvable by
tooling. Many large organizations run de jure or de facto Need-to-Know (NtK)
departments. I have witnessed Functional or Class Decomposition and Module
Hiding repurposed for NtK atomized development teams not knowing shared
OUTCOME semantics. Organizations and people turn SE abstraction boundaries
into NtK semantic ASYMMETRIES, i.e., SECRECIES.

That does not work well, efficiently, or predictably. Anyone can rightfully
detect motivations for both FSF/OSS and Skunkworks structured "startup" teams
where all wood is behind the same arrow.

Other problems are lack of SE training. There are big tugs in finite training
days between reading/writing skills, problem domain maths, tool churn, machine
models, scheduling techniques and so on. Many HN Skunkworks styles teams are
in VERY good shape compared to 20 years ago or contemporary organizations
trying to turn SE into contract management.

~~~
stenecdote
> These are mostly solved problems (minus omniscience or scale) with just
> handfuls of books that represent most parts of the solution.

Could you list some of those books? Would love to add them to my reading
queue.

------
irfanka
Is it just me, or does this paper lack any real "meat"?!

~~~
proksoup
I believe the title is misleading out of context, I think it is meant to lack
conclusions or clear statements and simply outline some areas for discussion
... a proposition paper?

I'm not familiar with what that really means, just jumping to an assumption
about the intent of the paper that I think was not made clear in the title as
is.

~~~
jackfoxy
In that case I won't waste my time reading this, but submit my outline of a
Useful Theory of Software Engineering:

It comes down to making a good choice in each of the following areas. (My
choice is in parens, along with my comments.)

Cloud Platform (For historical reasons I am on Azure. I have some experience
with Google. It does not hurt of course to work in several, but the cognitive
overhead of gaining a real in depth devops knowledge on very many platforms
will be overwhelming for many people, especially as the vendors invent stuff
on the fly.)

Relational Database Management System (Strive to learn SQL and 1 or 2
platforms, mine is SQL Server, as best as possible. There is only one useful
data model, and it is the relational model. You will store data with other
means over your career, but learning them is mostly trivial, and if your are
intentionally using a different data model than relational, you should have a
good reason. Also learn to data model using natural keys. This is a huge topic
unto itself. The main justification is it will simplify the queries you end up
using. A friend of mine is working on a book on nat keys. There is a real
shortage of good literature on this topic.)

An application programming language (Mine is F#. Learn one or 2 languages
well, chosen because the compiler and canonical workflows do so much to keep
you out of trouble. You will have to know 1 or 2 scripting languages our of
necessity. Given the other areas you need real expertise in, I am not a fan of
spending a lot of time with the language of the month club. Become an expert
in the language(s) you use. It's a career-long pursuit.)

Project Management (Recently saw a good overview video of the "no estimates"
approach. I was sceptical before I saw it. Not really impressed by Agile, but
I don't want to wade into that swamp.)

Specification Language (A new area for me. I am interested in TLA+, but I'm
still learning it. The idea of a specification language that delivers an
unambiguous spec at an arbitrary level of granularity that you can manipulate
and refactor is very appealing to my sense of propriety. We spend too much
time going straight to prototype in our favorite language, refactoring it,
etc. TLA+ forces you to focus on actual spec and not implementation details.
The prototype approach leads us all to confuse implementation and spec.)

~~~
nxtrafalgar
This does not describe software engineering, as that term is academically
defined.

~~~
jackfoxy
and that is?

~~~
nxtrafalgar
The systematic study of methods for producing high quality software. Your
comment is equivalent to calling a flexural test machine 'materials
engineering'.

~~~
jackfoxy
Granted. I'll throw out the cloud platform, but keep the others.

1) There is no substitute for the relational model. Clearly it is a paradigm
for building mathematical objects.

2) I didn't strongly advocate my choice of F#. Strongly-typed, functional
first, ML syntax language is the sweet spot today. In theory this predicts
faster development and fewer bugs. All software assemblies are mathematical
objects, those built with strongly-typed, functional languages tend to be more
humanly comprehendible.

3) No estimates allows practitioners to make useful projections of the course
of a project. (Of the 4, this one has the weakest claim to belong to theory,
yet I will argue useful projections are a kind of prediction.)

4) TLA+, or something like it, separates spec from implementation detail and
makes "predictions" (at arbitrary level of granularity) of what a completed
software component will do.

~~~
verall
Lots of systems software has no place for any kind of RDBMS, I'm not really
sure I understand what you mean by "no substitute".

~~~
jackfoxy
Yep. Didn't think through writing that, did I?

------
nn3
The abstract reads like something that could be generated by an automated
paper generator.

~~~
galori
Even the body...as I was reading it, when I kept seeing new acronyms defined
and used it sounded almost comedic. It reminded me of some talk or something
I've seen somewhere where the made up acronyms end up being really funny
description words like ("and this we call a PAININTHEASS process"). Can't
remember what that was though.

------
blatherard
(2013)

