
De-risking custom technology projects: 18F technology budgeting guide - kiyanwang
https://github.com/18F/technology-budgeting/blob/master/handbook.md
======
nataz
I just went through this process on the other side as an 18F client.

Overall, I was happy with our experience, but the key (and in my opinion
highest risk/resource investment) of their process was the identification and
participation of the right product owner on the client side.

If you use 18F you are buying talent and a process. Besides money, you are
also commiting a major personnel resource as well.

The key stakeholder is expected to spend a significant amount of time on the
project on a daily basis. They also need to be someone senior enough to have
decision making authority, ideally a historical perspective of how the
organization works, and hopefully someone who had a moderate degree of
technical knowledge.

Thankfully we had that person, and we were able to reassign a portion of his
portfolio to someone else for the duration of the project. If we didn't have a
person that checked those boxes, the project would have been much more
unlikely to succeed.

The checklist itself is a fine starting point, but it's not going to be
universally applicable in all cases. Again, this is why it's good to have that
right key stakeholder.

I understand that agile may be a useful process for software development, but
software is a tool/means to an end, not a goal.

I also think it's useful for folks in the software industry in general to have
an honest understanding of the tradeoffs being made between different
management processes. It's not a simple as waterfall bad, agile good. I'm
curious how many people in this industry have real cross cutting (non
software) engineering management experience. I get the feeling it's not a lot,
which is too bad because those are the people you are designing tools for.

------
thoraway1010
My quick bullet points.

* Show me some software I can click around in (no animated videos about how XXX will solve ABCD).

* Tech does not solve authority / responsibility or other process / org issues.

* Let people solve their own problems - look at what's happening in shadow IT and support vs crush it.

* Limit "requirements" ruthlessly. Even as basic as why do you need your own logins (can you use a google suites or azure / microsoft oauth login)

* Be crystal clear on what you want to have happen for everything (sketch the input screens / reports etc).

* Value future "extensiblity / framework blah blah" near zero

* Value speed and responsiveness of software highly.

Programmers CAN implement reasonably against an absolutely buttoned up spec,
and training / implementation are MUCH easier when something makes folks lives
easier by being focused.

Govt in particular is terrible here - EVERYTHING is a requirement (5 different
race / ethnicity / origin dimensions that overlap but are different? Yes - an
absolute requirement! No - no one but the experts in diversity stats will ever
be able to code anything properly against these so the data will be mostly
junk), every tough decision is punted by including everything as a
requirement, they constantly get sold based on powerpoint pitch decks /
frameworks / platforms etc.

------
rememberlenny
I just came to say, I love 18F. I worked there from 2016-2018 and it's an
incredible organization. If you are looking for a place to contribute to the
federal government and make a difference with your technical skills, take a
serious look at doing a tour of service.

~~~
echelon
Does 18F offer competitive salary or remote work? If not, what about pension?

I've never worked for the public sector, so I'm not entirely clear how it all
works. How do you get promotions or raises?

Are these "civilian" jobs? Do they require security clearance?

Is there a risk of the office being defunded under the Trump administration?

~~~
thephyber
I found the answers to most of your questions within 5 seconds of a Google
query.

18F hiring page[1]

Civilian. Remote possible. No Security Clearance for most of their projects
(not sure about all). 2 year temporary positions (with possible extensions) so
probably doesn't lead to a federal pension, but there is a government version
of a 401k[2].

There is always a non-zero risk of any office being defunded by any
administration. Honestly speaking Obama's admin cut more public sector jobs
than Trump did (but in 2009 due to lack of revenue, and in the 2011-203 budget
sequestration). Trump and Kushner have both said good things about "high tech
modernization projects", but like financial investment companies always warn
"past performance is no guarantee of future results".

There are a number of current/former DSA / 18F / Code For America employees
that rave about it. I think Matt Cutts[3] (of Google Search fame, recent US
Digital Service Administrator) is one of the more recent ones in my mind.

[1] [https://18f.gsa.gov/join/](https://18f.gsa.gov/join/)

[2] [https://join.tts.gsa.gov/compensation-and-
benefits/](https://join.tts.gsa.gov/compensation-and-benefits/)

[3]
[https://en.wikipedia.org/wiki/Matt_Cutts](https://en.wikipedia.org/wiki/Matt_Cutts)

~~~
echelon
> I found the answers to most of your questions within 5 seconds of a Google
> query.

Thanks, but you don't have to tell me to "lmgtfy" because I wasn't actually
looking for that kind of answer. Though I didn't explicitly state it, I wanted
someone that worked at 18F to weigh in.

The value in HN is in getting commentary from people with interesting context.
In that sense, I think you've subtracted something from the conversation.

Please don't do that. The rest of your comment was useful.

------
wslack
Happy to help answer questions about this (or direct them to the authors!)

------
jasonpbecker
I think a lot of this advice does not apply widely outside of the federal
government. For example, I sell COTS software to school districts. There are
15,000 jurisdictions in the US that operate autonomously, but to achieve very
similar goals and with very similar operation. While there are differences in
state law that can be important, there's far more in common across school
districts than not.

To pick on one part of the bureaucracy, if you're the Department of Health and
Human Services in the US, there may only be one of you, with pretty unique
requirements that change when the law or regulations change. If you're a state
agency who largely implements federal programs, there may only be 50 of you.
Yeah, that's a market, but a pretty small one and more likely to have
meaningful quirks. But a lot of the US is not like this-- school districts,
municipal and county government, etc would do pretty poorly to follow some
parts of these guidelines, and quite well served to follow others.

With a document like this, I feel like it's important to state the audience
and some assumptions up front. They try to do this with some basic
definitions, but I've too often seen someone take a document like this to
justify maintaining a custom built municipal accounting system on an AS400
that three people in their sixties know how to maintain, as though there isn't
a very robust COTS accounting system market for municipalities.

~~~
wslack
> But a lot of the US is not like this-- school districts, municipal and
> county government, etc would do pretty poorly to follow some parts of these
> guidelines, and quite well served to follow others.

True - I think the playbook is specifically oriented to state government folks
- not as much municipal or county.

> I've too often seen someone take a document like this to justify maintaining
> a custom built municipal accounting system

Please feel free to submit a PR to help make that clearer! I would only say
that some COTS solutions can also end up in a "no one knows how to maintain
this" after enough custom code has been added.

~~~
projektfu
Worse, nobody knows how to maintain it and the vendor is gone and took the
source with them.

~~~
wslack
This happens. I've also seen cases where the vendor, by contract, does not
permit government staff outside of 1-2 people from viewing the source code.

------
getaclue
Thanks for sharing this; I've only skimmed it but it seems to hit some of the
common issues right on the nail. Definitely a useful subject matter. Now we
just gotta make the management read it ;-) I like the checklists and the
questions to ask sections. The appendix also seems to ask relevant questions
and provide proper guidance. I'd be interested to hear from a non-technical
person to see what they think of it. Cheers

------
motohagiography
So valuable.

Currently working on bringing product management as a concept into public
services (non-US), and using 18F's ideas as a model.

The key aspect is product ownership, as not only is there a notion of product,
but also introducing the notion of an individual with ownership, as management
and ownership are orthogonal concepts.

The idea of metrics as we understand them in the tech/startup world is also
different, as in tech we orient toward revenue and user growth, where public
sector projects are largely funded in phases with a fixed budget. The whole
economics of public sector projects is you have expensive development up
front, and then you amortize that cost over a decade or more by transitioning
it to cheaper/sunk-cost operations staff to manage it over its lifecycle. The
other piece is that if you launch a public service that meets the needs on an
80:20 basis, you are a government who has left out %20 of people, which
becomes untenable quickly.

Adding product management (e.g. something like aha.io, let alone Jira or Azure
Devops) is a cultural change. Government has only just adapted to the notion
of "client/server," which was a fundamental culture change driven by
technology. Agile, devops, microservices, and the fact that most people under
25 today code, is going to be another revolution in public sector culture, and
one not without bumps.

Agile presumes growth, it was designed to create products that grow revenue,
whereas backfitting it to a fixed cost model takes some gymnastics on multiple
levels.

The technology changes of the last decade were almost as large as introduction
of the internet, and they are going to drive cultural change, but it is worth
recognizing that this itself is the key challange. 18F's approach is the
future, and I would love to replicate aspects of it, as how to effect change
in the culture of public services is the greatest challenge of all.

------
jimktrains2
>State IT projects, in particular, are often challenged because states lack
basic knowledge about modern software development, relying on outdated
procurement processes.

Which is part of the outcome of outsourcing everything and being afraid or
otherwise idealogically against doing much of anything in-house

~~~
wslack
In order to outsource effectively, though, you need the ability to judge which
vendors are strongest. I think there's been a pattern of thinking technical
talent isn't needed when outsourcing when the opposite is true.

~~~
nataz
This is why many technical tenders have both a price evaluation criteria and a
technical feasibility assessment. Best practice would be for folks conducting
both assessments to be subject matter expert in those areas.

~~~
wslack
Agreed, but when the talent isn't there, its easy for "helped supervise a
contract to deliver X" to get equated with "expertise in X." Expertise is also
needed to know what are big problems in a RFQ response, vs small problems.
There's a tendency sometimes without that expertise to make a list and
whichever team has the most +'s and least -'s to win out.

~~~
nataz
Agreed. Although, sometimes difficult in practice. Evaluation committee
selection itself is as much of an art as a science.

------
Ididntdothis
The same applies also to big companies. One thing I always notice is that
management wants to sign off on something and then be done with it. They don't
want to engage with the details and understand them. But all the advice in the
doc seems to revolve about constant interaction with the project and deep
understanding of what's supposed to be done. the common trend I see in IT
departments (and I have heard in governments) that more and more gets
outsourced and in-house capability is reduced so they can't really oversee a
project in a meaningful manner.

