
Why Is Payroll Hard - Ron Jeffries - chrismealy
http://c2.com/cgi/wiki?WhyIsPayrollHard
======
edw519
I love stuff like this!

Instead of complaining about the structure or motivation of business rules,
excellent developers ask different questions:

\- How can I design my architecture to accommodate just about anything my
customer throws at me?

\- How can I use all the tools in my tool box (including CS and math concepts)
to clarify the apparently nebulous?

\- How can I use things like Boolean algebra, parameter driven program logic,
and data base normalization to provide structure to that which appears to be
unstructured?

\- How can I use the customer's seemingly unreasonable demands to
differentiate myself from mortal programmers?

\- What do I need to become in order to say "Yes" to my customer whenever I
want?

\- How can I use all of the above to actually _encourage_ my customer to
attempt the "undoable" in order to differentiate himself from his competitors?

~~~
bigsassy
My first job out of college was with a consultancy that specialized in payroll
integration projects. We were a partner with ADP (largest payroll provider in
the country), and typically they sent us the clients with the trickier payroll
processes, so know I'm speaking from experience.

Long story short, I agree with what you're saying, but you make it sound like
it's really simple.

It's not.

Creating a single payroll solution to handle one company is very easy.
Creating a single payroll solution to handle many companies is also quite
doable.

But once you get to the scale of ADP, it's impossible to handle everyone
elegantly. Between union contracts and non-standard practices, there are (what
felt like) an infinite number of things a customer can throw at you, with one
rule wackier than the next.

It's damn hard to write a generic payroll solution that handles most
companies. Damn hard.

~~~
onemoreact
I seems like the general system is something like:

A) A set of tools to talk with external systems B) A set of tools for
manipulating the internal representation of payroll data C) A GUI for
interacting with / managing the system.

When their internal process is simple managing the payroll system should be
simple. When their internal process is complex you get fat consulting checks.
Which brings up a great point, it seems like large companies that simplify
their internal systems could avoid writing a lot of consulting checks, so I
suspect they must be out there and you just don't hear about.

~~~
bigsassy
You make two assumptions:

1) There is "one" way of representing payroll data. There's not. When you get
a none trivial amount of clients, all their edge cases put together destroy
any ability to fit the data into one conceptual model elegantly.

2) Companies have control over payroll rules. Large companies often can't
simplify their systems due to contracts or the subsequent employee mutiny.
Unless you want to get sued, lose the lawsuit, and pay out the nose (which
I've seen happen), you can't just change how you do payroll on a dime.

Not to mention, there's a ton of disfunction too. Payroll managers aren't
always the brightest tools in the shed. One project I worked on lead to the
payroll manager getting fired over "bad stuff" I uncovered. And then as I
uncovered more stuff his replacement was let go too.

...payroll can be a total clusterfuck. Avoid at all cost.

~~~
onemoreact
1) The only conceptual model I think needs to apply is the how things are
taxed. Granted some organizations have edge cases in how they pay taxes (or
how they try to hide things) but I think innate complexity dominates unique
edge cases when it comes to the tax code.

2) I would argue that much of this complexity shows up because many companies
negotiate while ignoring the cost of adding complexity. Company wants to
minimize cost, workers want to maximize benefits, complexity is a dead weight
that benefits nobody. Once things have been negotiated it's hard to change,
but during negotiation they is no need to end up with yet another edge case.

PS: I don't think organizations end up spending 100+ million on their payroll
systems because it's easy. I just think being proactive lets some companies
minimize those costs. EX: Is Google's payroll system ugly?

------
akeefer
Pretty much all "enterprise software" is fundamentally the art of taking
totally irrational, human-defined, fuzzy, incompletely-specified pre-existing
processes that have been built up over decades or centuries, and turning them
into an automated system that functions more or less identically. It's
fundamentally different than doing something like writing a word processor or
e-mail client which, while it has its own complications, doesn't suffer from
having to exactly model some pre-existing, half-specified, inconsistent
process that was developed by humans long before the advent of software.

~~~
gwright
I often wish there was some way to inject some order back into the real world
systems. So many of the techniques and processes that developers take for
granted seem entirely absent from our regulatory and administrative law
systems:

refactoring

version control

DRY applied to laws and contracts

diff tools for laws and contracts

standard interfaces

~~~
m104
There are many ways to bring some order back into the world, as you say, but
it can be a struggle. In my experience, business leaders will support simpler
systems and processes if (and only if!) the benefits _clearly_ outweigh the
costs.

As an IT professional, framing the right perspective is essential to getting
acceptance for better systems. Presenting a more powerful, but simpler, system
as "losing some rarely used and inefficient features" is a sure way to get
that project rejected. Management will only hear the "losing" part as a loss
in total value and therefore worse than the current solution.

Personally, I like to present formalized system upgrades as the price of
growth and future growth potential. Most business leaders can wrap their heads
around the concept of "we need to focus and streamline in order to reach the
next level", regardless of technical experience.

~~~
gwright
I guess I wasn't clear. I realize that businesses need to survive in the eco-
system created by crazy laws and ill-thought out regulations and contracts.

I was trying to suggest that the legal and regulatory framework is what needs
to be fixed, not the businesses.

~~~
m104
Ah, look at that, I totally misread where you were going! ;-) In my
experience, regulations and crazy laws aren't really a competitive advantage
or disadvantage because all of the market players are playing by similar
rules. I'm not saying that there's no negative effect to bad regs and laws,
but the costs are shared fairly equally.

I've never worked at a business that was cut off at the knees by new laws or
regs. I'm sure it's happened, but I'm not convinced it's _common_. What I have
witnessed though, many times at many employers, is pure self-inflicted
complexity. The did-they-even-consider-the-cost kind of complexity that is
nightmarish to implement and hampers future changes.

As an IT pro I can't do much about the laws and regulations, but I can advise
and design business systems in ways to avoid complexity traps and help
accommodate more sane business processes.

~~~
lsc
>As an IT pro I can't do much about the laws and regulations, but I can advise
and design business systems in ways to avoid complexity traps and help
accommodate more sane business processes.

This, I think, is a very healthy attitude; Focus on what you can change rather
than railing against things that won't.

------
mikebike
Obligatory Tao of Programming reference (by Geoffrey James)
<[http://www.canonical.org/~kragen/tao-of-
programming.html>...](http://www.canonical.org/~kragen/tao-of-
programming.html>):

There was once a programmer who was attached to the court of the warlord of
Wu. The warlord asked the programmer: ``Which is easier to design: an
accounting package or an operating system?''

``An operating system,'' replied the programmer.

The warlord uttered an exclamation of disbelief. ``Surely an accounting
package is trivial next to the complexity of an operating system,'' he said.

``Not so,'' said the programmer, ``when designing an accounting package, the
programmer operates as a mediator between people having different ideas: how
it must operate, how its reports must appear, and how it must conform to the
tax laws. By contrast, an operating system is not limited by outside
appearances. When designing an operating system, the programmer seeks the
simplest harmony between machine and ideas. This is why an operating system is
easier to design.''

The warlord of Wu nodded and smiled. ``That is all good and well, but which is
easier to debug?''

The programmer made no reply.

------
jodrellblank
\-- Stories of the Burroughs B5000,
<http://web.me.com/ianjoyner/Files/Waychoff.pdf>

_He devoted an entire session to the French Railroad payroll program. The
French Rail- road had gone on an economy program and reduced their number of
employees from 540,000 to 360,000. So that was the size of the payroll that
was run on the computer in Paris. That payroll must be the most difficult
payroll in the world to run. The size of the payroll is only the tip of the
iceberg. An engineer’s (i.e. a train driver) salary is a function of the
amount of power that he uses. So if he is on an uphill run, he makes more
money than on a downhill run. The employees live in government housing and
their rent is deducted from their pay. People with a lot of dependents get
more money than people with a few dependents. A vet- eran gets more money for
each medal that he won in the war. The amount of money depends on the medal. A
Purple Heart is worth more than a Sharpshooters medal._

------
smackfu
Another one I saw recently: "Oh, you've worked from that alternate location
for too many days this year, so your travel expenses are no longer a business
expense. The company will still pay them, but now they are taxable benefits.
So the company will actually pay you extra in payroll to offset the taxes you
will now have to pay."

Oy.

------
hkarthik
Healthcare software has the same kinds of problems. New regulations, rules,
and ways that each party in the system attempts to skirt the rules bleed into
the code bases of every system.

Thats why a lot of the technology in healthcare is so far behind. One place I
worked at still makes $38 mil/year on a system written in C++ and ASP.
Replacing it with a modern stack has been on going process for more than 3
years with no end in site. In the mean time, new installations are being
brought up with a whole new set of business rules being added that will just
have to get ported later.

~~~
smackfu
For national companies, state level reporting requirements are a huge burden.
For instance, payment made to doctors by healthcare companies. You just hope
that they pick fairly common requirements, instead of each state reinventing
the wheel.

------
matwood
I just built an internal payroll system for my company. It's been a huge pain
because it involves both parts of the business I generally try to avoid -
legal, and dealing directly with money.

Oh, a big item missing from the list is dealing internationally. If you think
navigating all the state and local laws are hard wait till you look at another
country :)

~~~
cpeterso
Don't forget interfacing with third-party systems like banks and insurance
providers! :)

------
jfruh
"The debators of the rules are more interested in satisfying the group than in
definition or algorithmic simplicity or purity of concept."

This sentence is incredibly useful for understanding why modelling a process
hammered out by more than three people will always be more complicated than
you think it is.

~~~
masklinn
> This sentence is incredibly useful for understanding why modelling a process
> hammered out by more than three people

Three? I'd go with one. Hell, I'd go with zero, if the one person is a banker
or a lawmaker.

------
gbvb
Actually, Payroll systems can be structured. In the end, there are only 3
things that you need to provide: \- compensation (money employee makes) \-
deduction (money employee gives for some service) \- taxes (employee owes the
government)

Everything that you do is toward satisfying those requirements. It is hard
because each of these pieces are influenced by human interaction. But, that
does not mean there are no rules of engagement. e.g. If you have union dues to
be deducted, it is a formula to compute it against the total wage you make (or
it is a flat deduction amount). If you are paying towards 401k, it is a
percentage that you put in towards that.

If you imagine the problem as something to solve in a general sense, you can
narrow the problems to fit into models. But, if you treat every requirement as
a modification to an excel macro (:-D), then you are in trouble.

~~~
Duff
They get nasty. You cannot underestimate the complexity that collective
bargaining agreements bring.

New York City, as an example, has something like 15,000 variations for payroll
categories. Plus garnishments, domestic relations orders, etc.

How do you compute the extra boot benefit for a fireman who has worked more
than 526 hours in a 13 month period?

Or how to you compute the salary of a garbageman who is paid by the ton of
garbage hauled, instead of the hour. Except for when they haul ashes, in which
case the tonnage is adjusted, or on Sundays, where they get overtime as well.
What are the business rules for an ash hauling garbage guy earning overtime on
Sunday, which is also a holiday?

The NYC consolidated payroll system, "CityTime"... has cost over $700M since
1996 to build. (That included some serious fraud.) Fraud aside, Accenture was
raping & pillaging too -- charging $400/hr for college grad coders.

~~~
gbvb
I am really not underestimating the effort. But there are multiple pieces to
what you just talked about. There are complexities in time measurement (or
computing how much work was done) and in payroll calculation.

When systems of this sort are built, it is usually a combination of
Timekeeping (how to measure time spent on the job) and payroll (how to pay for
that time). Each section is a big (i.e. millions of line of code and many
engineers) worth of effort. All that said, depending on how systems are built,
you can a) Create a base system that solves core issues b) Accomodate as much
configuration towards that system to allow computation to occur differently
based on what the employees do c) Allow customizations of computation when as
required d) Make most of the data that can be changed, to be changeable
through design (data driven instead of code)

Of course, there will be customers who will want to behave differently because
the moon is in a certain position in the system. But, that is when you need to
know the domain enough to understand whether what they are asking for is
because "that is how they have always done it" or because there is a valid use
case that needs one of the b), c) or d) to be updated.

------
AndrewDucker
Very true. People wonder why enterprise software takes a lot of developers and
long periods of time, and it's because of things like this.

------
teyc
Because of this payroll systems are fraught with implementation problems,
particularly in government, where this type of rules exist.

In the past I kept a log of IT project failures as I come across them. There
are a few bad payroll system rollouts. <http://it-project-
failures.blogspot.com/>

The Queensland Health Payroll system blew up a political storm because of
implementation issues. The cost of the blowout? $219M.
[http://www.couriermail.com.au/news/queensland/health-pay-
bun...](http://www.couriermail.com.au/news/queensland/health-pay-bungle-bill-
soars/story-e6freoof-1226178920932)

In fact there was an inquiry into what went so badly wrong
[http://www.insightcp.com/blog/why-sap-hrpayroll-projects-
fai...](http://www.insightcp.com/blog/why-sap-hrpayroll-projects-
fail#comment-72)

Payroll systems should never be migrated wholesale. They should be scriptable,
for instance pay masters should be able to script the computations to assist
them.

Anybody else interested in IT project morbidity could also look at
<http://delicious.com/search?p=itfailures> and
<http://delicious.com/tag/deathmarch>

~~~
cobrien
Another example of a payroll system implementation gone awry: SAIC's CityTime
in NYC [1]. $700mm+ now ...

[1] [http://www.nytimes.com/2011/11/01/nyregion/bloomberg-
adminis...](http://www.nytimes.com/2011/11/01/nyregion/bloomberg-
administration-admits-mishandling-citytime-and-nycaps-programs.html)

------
chollida1
Sounds like the kind of business logic where Prolog would really shine.

~~~
gaius
It was written in Smalltalk.

~~~
masklinn
C3 was indeed, but payroll systems have been written in pretty much every
language under the sun, I'm guessing chollida1 was talking generalities (and
the mess described in the article is the kind of stuff you find in every
payroll system)

~~~
chollida1
> I'm guessing chollida1 was talking generalities

I was yes.

------
DanielBMarkham
Typical noob mistake #73: trying to make a class out of every rule in a rules-
based system.

I hate to be blunt (or worse, snarky) but seriously, everybody does this at
least once: modeling and factoring out tons of bullshit when the user is
really explaining an ad hoc rules system. Very painful.

At some point during analysis, you have to ask yourself "Do I really want to
make the system more complex for each example of X the user gives me? Or just
acknowledge that the user is free to come up with all kinds of crazy rules
from now until the cows come home, and code that?" (and simplify)

~~~
glassx
Your reply reminds me of Zed Shaw's presentation 'The ACL is dead'. Basically
he said that if the law rules that you have to handle are turing complete,
you're probably better off with a turing complete language:

<http://news.ycombinator.com/item?id=420650>

Of course, procedural code becomes spaghetti code quite easily, but one
probably needs the same discipline when doing object oriented/relational
modeling...

------
zzygan
Payroll is a really weird beast. In some ways business apps can be a
challenge, and in others a real pain. Payroll is definitely one of the harder
business problems to make generic.

I've just come off a payroll project that has been 2 and half years in the
making, and I can say first hand that all of these things happen. And more.

Acquisitions, benefits, and here in AU award wages for hourly employees make
things even more fun. Different sites of the same company can have completely
different set of rules and regulations that they adhere to, purely because the
unions decided to negotiate differently for a different site.

Ahh fun.

------
jameskilton
Thankfully I've never worked with something this complicated, but it doesn't
surprise me (though reading it did give me a good laugh). What I've learned is
that when money is involved, any sense of logic and sanity immediately goes
right out the window.

