
Domain Knowledge or a Lack Thereof - ajhai
http://jacquesmattheij.com/domain-knowledge-or-a-lack-thereof
======
gcv
Ah yes, the tragedy of studying computing to the exclusion of everything else.
Graduates with computer science degrees have an in-depth understanding of...
text editors. And maybe compilers, databases, and operating systems — all
items of zero value until they are applied to solving some real problems in
some other field. Problems which the computer science graduate knows nothing
about.

Then these graduates go into the working world, and they typically can't help
solve problems in economics, physics, applied mathematics, or medicine except
by coding up some system to someone else's specs. The more dedicated ones make
an effort to understand what they're doing. They exceedingly rare, gifted, and
motivated ones might learn enough about an industry to make real
contributions.

I hope that our field eventually evolves to the point that programming skill
becomes an accessory. People will then learn to program just like they learn
to write, and use the craft to help them solve their actual problems. The
profession of programmers who only know how to program should disappear like
the profession of scribes who only know how to write.

~~~
martinced
And yet another cheap stab at many people reading HN. HN is filled with
negativity --if not downright hatred-- towards nerds lately.

Items of zero value? A certain sci-fi author would beg to differ with you.
Even someone that would have zero knowledge of anything else but would be
mastering programming is still a technopriest. Even with zero knowledge about
anything else but computers one still understand more about today's world than
most people out there.

I'm not advocating refusing to learn other domains (I'm the kind of person
who, when writing a book, decides to learn typography and typesetting software
so I can typeset the book myself) but your cheap stab at young programmers
(I'm 40) is uncalled for.

The world is going to shift to even more 0's and 1's. Many things are going to
disappear and be replaced by 0's and 1's.

If there's one thing common to _every single profession out there_ is that
people need to read, write and communicate. And you know what? The tools
allowing to read, write and communicate are written by the technopriests.

It's even more "scary" than that: the very infrastructure in place allowing
people to communicate (like electricity and Internet) are, themselves, run
using software. Not to mention very basic need like the entire infrastructure
providing clean water to fullfill one of the most basic Maslow need.

There's no need for someone taking care of a network to understand what's
inside the packets of information running through the network. All they need
to know is that if there's no more Internet packets travelling worldwide, then
there's no western civilization anymore.

The very foundation the entire society now relies on is software. If a
worlwide EMP blast would render all computers useless tomorrow the world would
probably turn to a mad max horror scenario in a few hours. No more water. No
more electricity. No more food. Our logistics would be so screwed that
"advanced" countries would fall into chaos. It's probable that in some places
people would die after a few days because of the lack of drinkable water and
it's very probable that in a few weeks many would die due to lack of food. Not
to mention the nuclear facilities which would leak radioactivity left and
right and break havoc.

Doctors aren't forced to understand how computers works to do their job. Most
doctors are utterly clueless when it comes to technology: they're bound to
live in fear of being hacked and not understanding what's going on when they
connect to their online banking account and not understand why this or that
mainstream article tell them to disable Java in their Mac. I've got doctor
friends: they simply don't understand the world they're living in. We're
_already_ in the technopriest age. And they're asking me questions, lots of
them.

The most computer-wise knowledgable doctor I know was thinking that the latest
state of the art way to diagnose using computers was still to use an "expert
system", filled by doctors. That's not how it works. These "experts" aren't
able to process the amount of information a computer can process and can't
find the correlation a computer software can find. Which is why now machines
are trained with gigantic amount of information and suggesting exams and
deceases to look for that are way more relevant than what doctors would
suggest.

I think you got it totally backwards: it's a lot of the _other_ professions
that are going to disappear. Replaced by robots, machine learning algorithms,
software and... The technopriests.

The ones still able to somehow understand the world they're living in and
still able to _write_ the world they're living in.

P.S: regarding mainstream economics I'd like to point out that Krugman, Nobel
Prize winner, recently said that it wasn't a bad idea to mint a one trillion
coin to deal with the U.S. debt SNAFU. Sure. Why not mint 26 of them and be
done with the U.S. public debt!? _That_ is the kind of economics most
economics and most universities do teach worldwide: Keynesian economics. I
happen to think Keynes was all wrong and keynesians have never been able to
predict the consequences of their interventions more than a few months/years
forward. There are economists from other schools who predicted 15 years before
it happened, before the first euro was even circulating, that the euro would
lead to: too many houses in spain, too many public servants in France, too
many industries in Germany and who explained by which mechanism Greece and
Spain would eventually default. Not a single Keynesian has ever been able to
do a prediction that precise. That's all I need to know about "economics":
that the mainstream is all wrong.

I do also think you put way too much faith in doctors: these people really
aren't nearly half as expert as you think in their field. And _they_ are the
ones getting replaced _today_ by sofware and robots. There's simply no way a
surgeon can be as precise and have as many axis of rotation compared to
_today_ robotic arms (explained to me by a surgeon friend). There's also
simply no way a doctor can read blood samples and other exams results and be
able to correlate that with hundreds of thousands of other results and hence
make more correct deduction as to what's going on.

If we eventually end up one day with true AI then _everybody_ becomes
meaningless because that day there ain't going to be doctors nor economists
anymore.

Until that day programmers (even the one knowing nothing besides 0s and 1s and
how 0s and 1s move around) are going to be more and more like the
technopriests.

~~~
hef19898
I didn't read it as a stab at nerds or young programmers but more like an
advice that domain knowledge still is important beyond pure programming
skills. Maybe the comment was formulated a littel bit over tthe top, but well,
I'm doing that quite regularly.

I completely agree that the world is running on software right now, but the
world isn't software ITSELF (well, not yet).

On the other hand, people being cracks in there field claiming that there
field is the only one that counts are just arrogant. No mater if they a
programmers, accountants, mech. engineers, doctors, car mechanics or what not.
And yes, software written by people without the slightest idea of the purpose
of said software tends to be not so good.

~~~
irahul
> without the slightest idea of the purpose of said software

In the OPs specific example, the purpose of the network is to carry packets,
and your job is to make sure packets are delivered. Whether they carry orders
to launch nukes, or donations to charity isn't the purpose.

~~~
Sandman
The point is that you need to be well acquainted with the problem that you're
trying to solve. If you're developing a new network protocol then yes, you
don't need to know what kind of information is going to flow through the
network, you need to know about network protocols. On the other hand, if
you're developing a business application then you better know how that
business works.

------
jedbrown
In science and engineering, it's common to hear of "Computational X" where X
is "Plasma Physics", "Geodynamics", "Ocean Science", "Electrical Engineering",
"Aerodynamics", or any of a myriad of other disciplines. In these fields, the
primary task is formulating Science/Engineering questions that can be
effectively answered using computation, then writing the software to do so.

There is also the more generic field "Computational Science and Engineering"
(CS&E) [1], which is primarily the development of methods and software that is
applied in many specific disciplines. I work primarily on the library
development side of CS&E, working with many applications to understand their
needs, often finding ways to satisfy those needs with better software reuse
and better ultimate flexibility, then implementing any components that belong
in libraries. Additionally, we do basic research by recognizing unexploited
common themes and designing new algorithms that will benefit existing and
potential users.

There is currently a great deal of demand for Computational X, for CS&E
through the postdoc level, and for CS&E in industry and national labs. It is
recognized that a _versatile_ practitioner of CS&E can effectively assist
Computational X for many values of X, since there really is a lot in common.
Once you've put in 10000 hours of CS&E, plus 1000 hours in a handful of
disciplines, you should be able to get orientedy quickly in new applications.

At the faculty level, support for CS&E is highly variable. At some
universities, X departments have the view that all their faculty should "X
first, computation second", while the mathematics department wants to see
theorems and isn't overly concerned with following through to see the method
applied to obtain a new science or engineering result. When hiring committees
say "we need CS&E, but some other department should have the CS&E faculty", it
drives a lot of talent away from teaching roles (e.g., into industry),
perpetuating the relative lack of early education in CS&E fundamentals. On the
other hand, the trend is for universities to have more direct interest in CS&E
(sometimes recognizing it with a dedicated department or inistitute). It helps
that computation stimulates many interdisciplinary collaborations, leading to
more diverse funding possibilities.

[1] We have a big meeting next week: <http://www.siam.org/meetings/cse13/>

~~~
pmahoney
What sort of backgrounds do people working in this field have? More
specifically, I have a bachelor's degree in materials science and currently
work doing run-of-the-mill web development (were the typical background of my
peers is a bachelor's in CS). What steps might I take to move into
computational engineering?

~~~
jedbrown
There are a couple journals dedicated to computational materials science. Two
active fields are phase field models and molecular dynamics.

Phase field models are mesoscale continuum (PDE) models based on the Cahn-
Hilliard and Allen-Cahn equations, and are useful for things like predicting
crystal grain growth and reorganization under strain, as well as fracture and
many other uses. In phase-field modeling, the interesting physics is a
consequence of the free energy functional, which is phenomenological and
different for each material, so software should make it as easy as possible to
turn specification of the free energy into an accurate and efficient massively
parallel simulation. One such approach is outlined in this paper [1] from some
colleagues. The resulting discrete systems are stiff and have positivity
properties that must be preserved, so scalable implicit solvers (like
multigrid) as well as methods for variational inequalities are necessary.

In molecular dynamics, there are many parallel scalability and expressivity
challenges. Fast algorithms for long-range interactions (such as the fast
multipole method) lead to less regular computational, but huge asymptotic
(O(n) versus O(n^2)). Since better algorithms become more important as our
hardware allows us to keep scaling up the problem sizes, Fast methods need to
be applied in more places.

There is also a lot of activity in multiscale modeling and numerical
homogenization, where explicit microscale models (often kinetic) are used to
provide missing components of meso/macro-scale continuum models.

[1] <http://dx.doi.org/10.1016/j.commatsci.2011.07.028>

------
kevingadd
The lack of this kind of domain knowledge - either from being unwilling to go
find it, or not being given the time and the resources to do so - is a big
problem for tools development in games. People doing the scheduling and
planning tend to assume that if you put enough smart programmers in a room and
give them detailed specifications, they'll be able to solve all the problems
faced by some artists and writers in another room, even though they don't know
anything about art or writing. Sometimes this results in the kind of obvious
mistakes that you might otherwise assume only happen when you're writing
software for the government.

In a sense it's kind of sad: In building traditional software, understanding
the customer is one of the hardest steps because you can't simply walk over
and observe them going about their daily work, or ask them questions on a
regular basis to understand their issues - you're separated from them by a
sales process and probably some automated support tickets and a PR guy who
won't let you have open discussion in public. People building tools for game
development can have their customers sitting a few desks away!

~~~
calpaterson
> In building traditional software, understanding the customer is one of the
> hardest steps because you can't simply walk over and observe them going
> about their daily work

One of the nicest things about the Agile fashion at the moment is that my
customer (or a reasonable representative of him) is a couple of desks away and
I really can go and ask him about what he wants. It's incredible how much time
is saved by avoiding doing things he doesn't care about and how much happier
he is with the end result.

------
kenkam
This particularly resonates with me. I have been working in a bank as a
graduate developer for the last year and a half, getting by without much care
about how instruments are priced, traded, recorded, audited, etc. To me, I was
moving a button around and colouring things using WPF. To me, the "Yield"
column was just another column with numbers that were supposed to be 2 decimal
places and turns red if negative. All the while I was dreaming I wished I
worked at Google on 'technology'.

Which, in hindsight, was a little misguided. There is so much value in
learning and understanding the domain in which we are working in because it
makes us so much more effective. The realisation came when I became frustrated
in meetings where we would discuss ideas on financial models and I could not
add any value to the discussion.

I then imagined working at a company like Google, let's say, doing stuff I
wouldn't normally be interested in. Would I bother learning the domain
knowledge? I couldn't say yes with 100% conviction.

I love programming but I also want to be effective in what I do. To me now,
understanding the problems we are solving is probably just as important as
writing maintainable code.

------
alemhnan
My take is that one of the main improvements in programming languages will be
to reduce the gap between domain experts and programming experts.

Now we should face three cases: 1) the programmer needs to learn the domain;
2) the domain expert needs to know how to develop; 3) something in between.

Most of developing is done in 1). There are some brilliant cases of 2). I once
talked with a biologist doing some multi sequence alignment that told me that
some routine in C was quite slow and was far better rewrite that routine from
scratch in assembly. I was quite speechless.

In order to create better programming languages we could: 1) express domains
in a better way. Actually, just be able to express domains would be really
cool (and not so easy). 2) provide higher levels of abstraction.

~~~
jacquesm
Every programming language from the dawn of time up to the present has made
that claim at some point. This time it would be different.

I think it is just a people problem. Programmers tend to seclude themselves
from their customers. Whole IT departments tend to seclude themselves from
their users.

When you are looking for underwater welders for the off-shore industry, what
is better, teaching a welder to dive or teaching a diver to weld?

(that's a serious question with a real answer).

It is much easier for a programmer to acquire the required domain knowledge
than it is (usually) for a domain specialist to learn how to program. But you
can only do so if you're prepared to 'dive in'.

~~~
unwind
My guess would be the first, teaching a welder to dive.

I base that on the fact that the core value, the welds, will be higher-quality
and that's more important than the diving execution.

The diving might be off, but it shouldn't be _that hard_ to learn well enough
to survive and be effective. Of course, I'm neither a diver nor a welder.

Did I win?

~~~
jacquesm
> Did I win?

Yep. But a drowned welder is also not much ;)

------
qxf2
The line I found most insightful: >>"Software is much more intertwined with
the institution commissioning it and the people using it ..."

Given that, I'm surprised at the emphasis on 'reading' as the primary source
of gaining domain knowledge. There are just 2-3 mentions of
speaking/collaborating with the domain experts that are driving you to build
the software itself. If you are lucky enough to work with acknowledged domain
experts, spend time talking to them. Pick their brain. Two techniques I use:
a) forward interesting articles to them with a question or an analogous
concept in my domain b) resist the urge to show them a 'better' way. E.g.: In
the HealthCare domain, I learnt people don't really want quicker or lesser
clicks. They would prefer a set, stable workflow. If you cater to senior-
citizens, keyboard shortcuts are less important than instructions in large
font.

I still suck at engaging domain experts. So, I'd love to learn more techniques
to hold an interesting conversation with domain experts. Ideas?

~~~
jacques_chester
I can read a book faster than I can speak or hear words.

In addition, books can usually be found, on almost any topic, which are
intended to bring a novice up to a basic level of conversancy. Reading one (or
two) such books will make discussions with the experts much more productive.

Plus: it's fun to learn about new things.

~~~
qxf2
Oh, no argument on the fact that reading is extremely useful. I read too. In
expressing the balance between reading and collaborating, I felt that your
article emphasized too much on reading - but I fully recognize that you do
collaborate.

>>"I can read a book faster than I can speak or hear words."

I disagree that this is an argument in favor of reading. Reading is easier but
I think its not always as high in quality as collaborating.

>> "Reading one (or two) such books will make discussions with the experts
much more productive."

Totally agree. I try my best to read as much with the end goal being picking
the brains of my colleagues.

~~~
jacques_chester
> _I felt that your article emphasized too much on reading_

You've mistaken me for Jacques Mattheij.

~~~
qxf2
Ooops. Mea culpa. Sorry.

~~~
jacques_chester
No worries. It's kinda flattering.

Plus "Jacques" is an uncommon name in the anglosphere, apart from South
Africa.

------
euroclydon
I've held programming jobs in: Water Treatment Process Control, Workflow
Management, Clinical Trials Management, Electrical Power Monitoring and
Simulation, and Medical Imaging. In each case my natural curiosity led me to
learn as much as I could about the field. I always enjoyed talking to the
domain experts at each company and asking them as many questions as they could
handle. There are usually one or two people who relish answering these
questions and I especially like the ones who talk fast, mock astonishment at
my ignorance, and proceed to try to fix it.

In water treatment, I learned about oxic/anoxic bacteria and how they're grown
and recycled in the process to treat waste, how to write chemical metering
logic in a PLC because I was the only one left on-site to do it, how motors
are controlled with variable frequency drives, how proportional-integral-
derivatives are used to prevent oscillation in feedback based controls.

In clinical trials management, I learned how double-blind studies are made,
the process of randomization and dispensing drugs, medical coding and the life
cycle of the trial.

In power systems simulation, I learned about three-phase power transmission,
resistive and capacitive loads, active and reactive power, the challenges of
hooking up a lot of solar panels our existing grids, how data centers are
build and cooled and powered, microgrids, power meters, intelligent breakers
and more.

I feel privileged that my skills at software development have exposed me to a
variety of fields and that I know how several important processes in our word
work. And I got to take home a nice salary while gaining that exposure. I do
lament that, unless I pick a field and stick with it, I'll never get to work
on the difficult programming problems that require a decade or more of
experience and knowledge.

------
east2west
HN is understandably filled with programmers who value programming above all
else and believe domain knowledge can be picked up quickly. It would be true
for someone hired as a programmer who implement a published algorithm or port
existing software to newer systems or different platform; in other words,
improving existing systems. I believe it requires equal depth if not not more
for domain specific knowledge to innovate in non-computational fields. For my
field of bioinformatics/computational biology, many open questions cannot be
answered because of computational hurdles. Reading textbooks won't lead anyone
to them, nor would reading survey papers. It requires expert knowledge in both
biology and computer science to even know what is feasible and interesting. We
want to find the driver mutations for cancer, discover mechanism of gene
regulation, uncover subtle interaction of heredity and environment.
Computation is intimately involved because observation is indirect, noisy, and
massive. Only with knowledge in both fields can answer questions that domain
experts care about but cannot do because of computational/quantitative
obstacles.

I find researchers (I am in academia) in both fields cross-talk too much.
Computational/quantitative experts try to shoe-horn biological questions into
their small corner of research area, while biologists don't care about
computational efficiency, statistical robustness/efficiency, and the
mathematical implication of their simple methods; no one cares about software
quality. This is unfortunate. It is also unfortunate hiring committees do not
look favorably towards multi-disciplinary researchers because one is not a
trained geneticist/cancer researcher. I definitively do not encourage computer
scientists go into support roles in academia; they are poorly paid and get
little respect.

------
kayoone
I am currently working on a web software project in the medical/health space.
Of course its really good to know how the people that use the software work
and what they expect and i am eager to learn those things, but i will not dive
into a sector i am not really interested and absorb any and all information i
can find, since i probably wont need it for the next project and it personally
does not interest me a lot.

For technical domain knowledge i agree, but it also happens that this is often
of personal interest to me anyway.

------
jiggy2011
I agree about the importance of gathering as much domain knowledge as
possible.

One thing that I have always had issues with when gathering requirements
around something for an unfamiliar industry is that people who are in that
industry will often neglect to mention things that are both important and non-
obvious/counter-intuitive to someone unfamiliar with the industry.

This is probably because they are so familiar with the quirks of the way that
the particular industry works that such things seem so obvious as to not need
mentioning (and they probably don't see how it could possibly be any other
way). Basically similar to how we are often amazed by assumptions some people
have about the way that computers/software works.

The paradox of this though is that to deliver truly good software for a
particular industry one must really know that industry as well as somebody who
has years of experience in that area. At which point you start to feel that
you could do their job too.

Another alternative is to have a very tight feedback loop with someone who has
tons of domain knowledge and is the target user. Basically sit next to them as
you develop the software and have them review every single change you make.

~~~
vicbrooker
I think in order to create good software that fits the industry's current best
practices then you do need a really in-depth domain-specific knowledge. If
that's the goal then learn everything possible about that industry.

To create good solutions that are also innovative, I think that programmers
only really need enough knowledge in that domain to be dangerous - to know
more or less what happens but with enough gaps to not really know what's
doable and what isn't. But, this knowledge is only useful if you can combine
it with bits and pieces from several completely separate domains.

If innovation is the goal then I think there is a point where domain knowledge
actually hurts you.

Of course there still needs to be at least one specialist to keep everything
on track though - and I completely agree with you about having someone with
deep knowledge on board. I think it should be someone who has been in that
industry and will stay in that industry.

I can't see the point in programmers trying to become experts in areas that
they're not interested in working in for the long haul. It's easier to learn
(and actually apply) a little about a lot than a lot about a little.

------
Nursie
Actually I think domain knowledge is overrated. A good hacker can pick up the
domain knowledge needed to contribute to a project very quickly.

Now, if you're designing a product to solve a problem in a particular domain
you need the knowledge ahead of time, sure. But if you're a shit-hot coder and
a fast learner, brought in to make things work, who cares?

~~~
rcfox
Because, as the article clearly mentions, there's a difference between making
something work and making something work right.

~~~
Nursie
I disagree that it's important unless you're the person designing the
solution. If you're not (and in many/most teams you won't be) it's far less
important.

~~~
majc2
I guess the key is how often are we just handed a spec and asked to code it
up. In my roles, I've never worked like that, I've nearly always had some
impact on design - but hey, that's just my experiences.

I would agree with your original point though, that domain knowledge can be
learned quickly by a good hacker. Which leads to a more interesting question;
what techniques can we use to learn a new domain?

~~~
hef19898
I agree that domain knowledge can be learned by decent hacker quickly up the
degree necessary to write good code for that domain.

------
mathattack
This is a very true post, but I don't think the solution is at the university,
at least for undergrads. Schools barely have enough time in 4 years to create
entry level programmers. Lopping off half the coursework in exchange for
domain knowledge leaves grads unprepared.

My observation is if you can learn to program well, you can learn any field of
knowledge. A programmer can learn consumer products easier than a brand
manager can learn Python. Similarly a CS major can learn bond math easier than
a Finance major can learn database design. (Yes, these are generalizations)

What's this mean? That bridging the Tech/Client divide is on the shoulder of
the technologist. In addition to learning the latest frameworks, we have to
learn our customers business. In addition to tech credentials, we should add
theirs too.

~~~
jacques_chester
The degree gives you a tour of the _foundations_ of the solution domain.

Mastery of the solution domain and knowledge of various problem domains cannot
be taught in a 3 or 4 year degree.

~~~
mathattack
Agree completely.

------
adrianbg
Programmers should learn more about other domains and domain experts should
learn how to program. Programming is becoming nearly as basic a skill as
writing.

------
keithpeter
Does anyone here find the work of Lave and Wenger ('situated learning') of any
use in their attempts to understand a problem area for a customer?

<http://www.infed.org/biblio/communities_of_practice.htm>

Basically, some organisations make their processes easily visible to most
employees, and some don't. The ones that do can train people more quickly and
improve their processes more quickly. Organisations that make 'peripheral
participation' easy might be better customers for developers than those that
don't. You might not have to make many sails or use a height gauge too often
to 'see' what the issues are, just interview knowledgeable people _carefully_
and cross reference the results with observation.

As a concrete example, the original author invested a considerable amount of
(quite expensive) time in learning about sail making and lathe operating. In
some of the 'business applications' I have to use daily, it is pretty obvious
that the customer was a manager higher up and that the interface we have to
use to populate the database that generates the wonderful reports has not
really had much time spent on it!

------
bwooceli
So I don't work in a scientific field, and I'm not a "programmer" or "software
engineer". I am a business analyst. I am a business analyst who CAN code. My
code isn't _always_ pretty, and I recycle a crap ton of boilerplate and hack
the hell out of django apps. And I read the heck out of HN and have spent 5
years slowly refining my design chops. But most importantly, I know top-to-
bottom the ins and outs of my business. So when our call center shopped around
for a software solution to provide a call center full of agents realtime
access to their stats, we were given a 3 month time frame and quoted $100k+
for a solution. So I wrote and deployed a web based results dashboard of
realtime stats over two weekends. And we were just quoted a price on a quality
management tool for $225k. Guess what's next for me? I've been an agent on the
phones. I've sold in a retail environment. I've been the corporate trainer
dealing with the LMS. Domain knowledge plus tech chops has proven a powerful
combination for me.

------
smoyer
I believe this is actually the major problem faced by outsourcing projects.
When I was in the cable industry, I saw several projects fail when the
"Systems Engineers" (who were in charge of writing the specifications) were
unable to convey enough information to the developers.

There's a real limit to the amount of "truth" you can put into a
specification, and at some point you have to trust the developer to "do the
right thing". If they don't have domain knowledge in the industry they're
writing for, they don't stand a chance ... and so your project fails even when
everyone works hard and has good intentions.

------
DirtyMonkey
I agree, but this is something I've found resistance to in the past. For
example, last year I was building an automated shipping system for a client
and I wanted to spend a few days in the shipping department - this was flat
out refused as the management didn't see it how it would be useful, since the
process was fully documented. Turns out, much had to be rewritten because the
shipping staff were (predictably) doing 'undocumented' things to speed up
their workflow, the new shipping system messed this up and made them slower.

------
mark_l_watson
Jacques is right-on. For a lot of the work in my life, most of the effort was
coming up to speed on a domain rather than just technology. Much of my work
has been in 'artificial intelligence' (for some definition; never liked that
term because it is too general) so domain knowledge is central if you are
trying to get human level or better performance in some narrow domain.

Having to spin up on new domains also keeps work and life interesting.

------
markbao
Turning the tables around: perhaps the software (or even physical products)
that we use where we think, "this is stupid! why would anyone build it this
way?" were similarly built by people with no domain expertise in the product.

Introduce a software engineer with an interest in the subject matter into that
environment, and provided it isn't too big (like government), it might be
self-correcting.

------
hawleyal
OP might be right if they choose to write software without collaboration with
clients or subject matter experts.

IMHO, with collaboration, it's more a matter of providing a hyper-rational
process by which the purpose and priorities of the target audience(s) are
focused and honed, and then producing the software as feedback, automation,
and a usable compliment to their needs.

------
danboarder
This article is talking about what I learned as classic Systems Analysis and
Design. It is an interdisciplinary intentional practice of seeking to fully
understand the business processes involved.

See <http://en.m.wikipedia.org/wiki/Systems_analysis_and_design> as a starting
point.

~~~
treerock
So have you ever tried what the articles suggest? Ever spent a week working on
the 'shop floor'?

In my experience of public sector software projects, the programmers seem to
be kept as far away from the end users as possible.

~~~
danboarder
As a independent contractor / systems analyst, yes I have. However I agree,
most solutions providers don't bother.

------
trvd1707
I think domain knowledge is important, but sometimes it can be a trap that
hinders inovation. As Pierre Bayard put very nicely in "How to talk about
books you haven't read", when we don't have domain knowledge we open ourselves
to questions that we wouldn't make otherwise.

------
joshwa
Reminds me of the old "what the customer actually wanted" comic:

<http://imgur.com/NXHbZ>

------
marpalmin
Perhaps we should all work in compilers ;)

------
dschiptsov
Another metaphor for necessity of Domain Knowledge is translation. Give an
average translator a technological text full of idioms from this technology
domain and you will laugh your ass off.

I remember reading reports about some SAP installation process (written by
idiots) full of stupid special terms such as "System Landscape" and "User
Privileges" translated by an English teacher.)

Why we have this as a top news? Isn't it obvious that you must know the slang
before you're trying to speak, let alone translate?

Well, most of coders use a slightly different strategy - jump right into IDE
and pile it up, using all that containers and design patterns they have heard
of (without understanding, why bother?) until it satisfies the manager
(compiles) who know absolutely nothing (this is why everything _must_ be
statically typed). Then test it like a black-box - correct input produces
correct output (never try incorrect or unexpected) and then ship and forget.)

------
martinced
Sadly in some domain it's not possible. I've had cases where to understand a
particularly complex business logic case it turned out _nobody_ had any idea
how that case should be resolved. Lawyers / jurists, employees, managers, etc.

We tried to go up the chain up to someone, anyone, who could explain how that
case should be resolved but nobody could tell.

Why? Legalese. And just like most legalese documents: there's more than one
interpretation possible because the very concept of legalese (based on
thousands years old tradition) is inherently flawed and open to
interpretation.

So, somewhere, some politicians, probably out of bikeshedding fashion, decides
to vote an amendment exceedingly badly written and exceedingly complex. Then
it becomes law. And law has to be followed. So that case has to be taken into
account by the software but isn't (yet).

Everything is all and well until that specific case appear. The day it appears
all bets are of. It's not working and nobody knows why. Nobody knows what is
not working and why it's not working, nor how it should work. The law is bogus
and the business case cannot be resolved without contradicting with the
law(s).

Trying to determine this ended up with the programming team receiving a PDF
with tens and tens of pages of legalese text that nobody up the chain
(including lawyers) could explain to us. Probably that the politicians who
voted these changes couldn't even themselves explain it.

We simply couldn't make the changes.

How did it end up? Someone very high up saying: _"That's what should be done
in this particular case"_ and then everybody covering its arse.

And there came the technopriests: fixing the production DB...

~~~
raverbashing
> How did it end up? Someone very high up saying: "That's what should be done
> in this particular case" and then everybody covering its arse.

This is usually how it ends up.

I've been close to similar situations (but while creating something, not
changing something that already exists), but usually, if nobody understands
how this is to be done, the fact that you can't code it is second in
importance.

