
Ask HN: How much of your programming work is mapping this data to that? - hoodoof
When I boil it down, it seems a great deal of the stuff I do is taking data from &quot;here&quot;, adding and removing and sometimes transforming fields, so that it can be put &quot;there&quot;.<p>&quot;Here&quot; and &quot;there&quot; are anything that expects or provides data.<p>Just seems to me that a substantial portion of the work of the programmer is &quot;here to there&quot; data mapping work.
======
whitten
This is "interface" programming, and is one of the most common programming
tasks. Creating reports can be seen as doing this (convert from machine info
to human info) Entry/Editing programs go from human to machine, inter-
operability is machine1 to machine2. The list goes on and on.

The more you tie meta-data to your data the more this can be automated. And
the more you use "standardized" formats the more this can be automated as
well. (Unless the standard format can be read in more than one way, then you
have a mess.)

------
paulrpotts
Absolutely.

In real-world projects, it is very often the case that much of what I do
(maybe 75%?) is basically what I think of as "plumbing."

Not just in web development, where it manifests as marshalling and packing and
unpacking data, but in multi-threaded embedded programs, where it manifests as
packing data into structures and sending it between tasks using queues and
serialization, maybe managing checksums and supporting retries and error-
handling, transmitting between chips via serial, I2C, SPI, etc), and in GUIs
(managing state, moving data between formats and cascading state changes while
trying to keep everything sane and consistent in a multi-threaded environment.
It's often a basic design element I've seen plenty of times before but it
still needs to get written and it has to run flawlessly.

The remaining 25% or so can involve some real algorithmic thinking, or and
state machine thinking, and that's more interesting, but there is almost
always some tedious boilerplate around it unless I'm working on, say, a side
project in Haskell, but even using a nice pure functional approach which makes
gleaming, beautiful functions to express an algorithm, there's the "plumbing"
and the tooling, monadic and weirder.

That's just something that is inherently hard about maintaining a long career
in software development -- keeping yourself motivated to get through the
duller parts so that you can enjoy the more interesting and challenging parts,
and not just doing the minimal, but writing quality code and comments, not
skipping steps, documenting your design, writing whatever documentation
someone will need later to understand it later, etc. Even simplifying and
revising it just for the sake of reducing complexity of working code, when
something can be simplified.

------
flukus
> When I boil it down, it seems a great deal of the stuff I do is taking data
> from "here", adding and removing and sometimes transforming fields, so that
> it can be put "there".

Try looking at it differently. When you take data for a database and present
it to the user, you aren't moving data from here to there but you are creating
information. This is why we call it IT not DT.

Creating information for people is a much more interesting problem and you
have to know a lot more about their workflow and requirements.

------
willstepp
You can distill it even more than this. All we ever do as programmers is take
some kind of data as input, process that data, and return new data as output.
This is true for all software: websites, mobile apps, video games, AI,
operating systems, embedded systems...ad infinitum.

------
tchaffee
A lot. Feels like there should be a way to automate it away or give it to a
framework to do, but the best way I've seen so far is to give that work to the
junior devs while I get to work on the more challenging aspects of a project.

~~~
hoodoof
I feel like in the future the Internet, at some lower layer, will become very
much boiled down to primarily a system that has a very standard way of
flexibly and automatically connecting, mapping and transforming data.

Alot of the programming work done today to achieve this will go away.

We are on the way there now clearly.

~~~
tchaffee
One would hope. But I've been doing this for a few decades and I haven't seen
much progress on taking the grunt work out of moving data around and
"reformatting" it. A lot of other things in the programming world have
improved dramatically though. I wish test driven development had been around
when I started.

------
98Windows
I'd say it's like 50% for me, I wish it was less.

------
perfmode
see:
[https://github.com/jbenet/transformer](https://github.com/jbenet/transformer)

