
Learning to code kept me sane when I was a diplomat - vikp
http://www.vikparuchuri.com/blog/how-learning-to-code-kept-me-sane/
======
elwin
The article shows an interesting illustration of the disconnect between
programmers and normal computer users. Like the author says at the beginning,
any experienced developer knows that automating interaction with a GUI is a
crazy idea. It's a painful, fragile way to move data around. But to the author
and anyone who's done data entry, it's an improvement to their workday that
demonstrates the power of programming.

So much programming instruction seems designed to teach people who already
understand programming how to solve problems that nonprogrammers wouldn't even
understand. A tutorial that started with automating tasks like this might be
more effective than all the ones that start by demonstrating how to do basic
arithmetic in $LANGUAGE and go on to a detailed explanation of $LANGUAGE's
object system.

~~~
blackbagboys
I'd be interested in seeing an accompanying article by the author or another
developer laying out how you would solve the same problem if you were an
"experienced" developer; I have done a little of the same type of programming
at my job, and a side-by-side comparison of the 'novice' and 'pro' approach
would be extremely helpful.

~~~
elwin
Maybe I was too vague. I meant that a programmer would always use an API
rather than scripted interactions with a GUI. An ordinary user might assume
that programs use interfaces designed for humans, because the concept of an
API doesn't even have a place in his mental model of how computers work.

------
Jezparov
This is a really great post. For me, the recurring theme is knowing what's
possible.

From a class he took in high school, Vik knew that some sort of automation was
possible. That spurred some reading and exploration, which led to some
success.

Success led to thinking about other opportunities for automation and more
exploration of what's possible, which in turn led to more success.

To me, it absolutely doesn't matter that things could have been done in other
ways. Nothing to cringe about here. A developer can only use approaches that
he/she knows or suspects are possible.

In the day-to-day work of a developer, knowing what's possible is perhaps the
most important thing. Anything else can be googled.

~~~
seanmcdirmid
Equally important to knowing what is possible is to not know what is
considered impossible, or at least considered very hard. Automation is often
considered futile, and knowing that can be intimidating. We are often limited
by what we think is impossible, but really isn't.

It is said that most Nobel prizes are earned for research done before 30,
since any older and they would know the breakthrough they are trying to
accomplish is impossible :)

------
SocksCanClose
This is a fantastic article and illustrates a HUGE problem in government
systems, where data gets aggregated, but not associated.

~~~
Retra
I had a government job and my boss loved me for doing automation stuff. All of
the documentation and data requirements seemed to be written as though a
person had to manually do everything -- like they expected a human brain to be
making decisions at every step of the process.

It just reeks of a lack of care not only for the power of automation systems,
but disrespect for the people who have to do that work. Design your processes
in such a way that easy stuff can be done easily and important stuff can be
done by people who have some authority.

This is my biggest (personal) problem with bureaucracy. "You have to do it
this way because it's the only way _I_ know how to do it, and I don't care if
you know a better way, because you don't get paid enough to have good ideas."

~~~
cportela
Yea my university IT job was like that too. Except automating things was part
of my job description and I still wasn't given much lee-way to automate things
much.

------
sinwave
Oof. Had a hilariously similar experience doing data entry at my first
internship, which was at a financial services firm.

I knew nothing about programming, but I had taken an intro stats class that
had a few assignments in R. A super simple R script did half of my day's work
in seconds (plus some debugging because of course I did next to no error
handling).

In retrospect, I've realized that if I knew a thing of two about regular
expressions at the time, I would have been able to automate my whole summer
internship.

So it goes.

------
techtalsky
I haven't seen much discussion of a more ideal way to solve this problem (to
say: as a one-off like the author was doing, not reusable software), so I
thought I'd just think about how I'd do something similar.

Luckily there are libraries that are great at interacting with a web page or
multiple pages at the same time without needing to actually have a browser
instance open. There's selenium and its various scripting environments, which
are meant for browser testing but work just fine for many automation tasks:
[http://www.seleniumhq.org/](http://www.seleniumhq.org/)

And, newer, are headless technologies like phantom.js:
[http://phantomjs.org/](http://phantomjs.org/) and Dalek.js:
[http://dalekjs.com/](http://dalekjs.com/)

This is SO much easier than using Javascript to find purple links on a page,
and so much more reliable.

He could have looked up each link on DB1, then used the data to look up the
DB2 information.

In any case, what he did saved him hours and hours of time and started him on
a lifelong interest in programming.

Also, obligatory relevant XKCD's:
[http://xkcd.com/1319/](http://xkcd.com/1319/) &
[http://xkcd.com/1205/](http://xkcd.com/1205/)

------
yason
This is golden. I've wondered more often than once myself what would it be
like to have a "normal" office job.

For instance, take some city department or some big firm where things
generally flow at a slow pace and require a lot of time spent on work items,
doing possibly repeated tasks each week. What would a programmer do? Exactly
as described in the article:

\- learn what kind of tasks the job consists of

\- identify repeated actions and patterns in work

\- think of a way to automate the easiest parts, still requiring human to
finalise the task

\- gradually learning more and automating the harder and harder parts and
building tools bottom-up based on earlier blocks

\- finally ending up with a software mechanisation puppet that I can instruct
and program to do my work

\- gradually spend first some of my time doing the automation and as it would
slowly take over I could devote more and more of my time automating the rest,
and when all is done, I would feel the job is "completed" and get bored.

This is something that feels so natural to a programmer: I have dozens of
little tools to aid in doing my programming job at work. The goal of a
programmer is to program anything that can be automated and only work himself
on the hard tasks that can't.

It's intriguing to imagine how you could apply those skills outside
programming jobs.

~~~
eru
Often you have to change the tasks slightly to make them properly automatable.
The fancy term is process improvement. See
[https://en.wikipedia.org/wiki/Business_process_improvement](https://en.wikipedia.org/wiki/Business_process_improvement)

------
thebenedict
Love it. I started my career in International Development and one of my team's
first projects was figuring out how many miles our organization's 1000+ staff
flew in a year (why is a separate story). The travel agent could only provide
records in PDF. 1000's of them.

The intern I shared an office with was assigned to go through each record,
type the origin and destination airport codes into an online flight distance
calculator, and enter the results in a spreadsheet. After two days of watching
this I volunteered to take over and spent a few nights learning enough Java to
read/write CSV and make the HTTP request. Saved days of pain for the intern
and was a satisfying accomplishment as a novice.

~~~
eru
And you even learned some Java in the process!

------
flashgordon
Brings up old memories reading this. I remember my first job about 20 years
ago at NEC as a data entry operator. It was 8 hours each day of taking a bunch
of mal-formatted CSV files from some reports the previous day and importing it
into excel and drawing graphs. I was just getting into learning C at the time
and ended up writing an importer (parsing and formatting in C and ODBC to
write back to excel along with excel macros to draw graphs). Very hacky and
ugly but got the job done. It got the 8 hour job done in about 30 minutes -
most of which was detecting patterns in the input and adding rules for those,
which was fun in itself.

------
jdbernard
Before you knock on "writing code that clicks things" remember that on Windows
clicking things is the primary if not solitary interface to most of the
system. Even the administrative interfaces are secondary to the visual
interface. A lot of programs don't offer an API available to outside
processes.

The most important thing about any program is correctness: does it work and do
what it is supposed to do. The only thing cringe-worthy about having to script
GUI interactions is that fact that the operating system and environment
requires it.

~~~
detaro
For the server versions, Windows finally leaves this path, making PowerShell
(that admittedly also takes some getting used to) a first class configuration
system. You even can do something in the GUI and then check in a log what
PowerShell commands were executed under the hood.

~~~
pjc50
From which version of Server? Where does the log go? I have a use for this.

~~~
detaro
I think it's from 2012? Here is a link that describes it:

[http://www.learnmsexchange.com/windows-servers/windows-
serve...](http://www.learnmsexchange.com/windows-servers/windows-
server-2012/593-what-is-windows-powershell-history-viewer-in-windows-
server-2012)

------
mjklin
Why has no one mentioned Autohotkey?
([http://www.autohotkey.com](http://www.autohotkey.com))

It was practically made for this use case, and has an active community behind
it. It even has its own script editor (SciTE4AutoHotkey) and was recommended
as a Kevin Kelly-approved Cool Tool
([http://kk.org/cooltools/archives/4147](http://kk.org/cooltools/archives/4147)).

My go-to program for my desk job.

------
stefs
this reminds me of when i started to code - find the first solution to the
problem that comes to mind and hack it until it works, even if was something
as abstruse as manipulating the UI. of course the outcomes were incredibly
fragile and broke at the first possible moment, but in the end they still did
the job they were supposed to do most of the time.

today i couldn't do that anymore; i've grown too uptight to try any of those
inefficient "solutions". i guess i'd rather give up and admit defeat if there
were no APIs or passable hacks (i'd say using phantom.js would be a passable
hack) i could work with than solve the problems this way.

i'm usually unable to follow the advice given by the xkcd automatisation
timetable described here: [http://xkcd.com/1205/](http://xkcd.com/1205/) "is
it worth the time?" \- if a dirty hack is destined to run more often than
once, i just _have_ to make it passably elegant, readable and efficient. i'm
sure nine out of ten times the effort is pretty much wasted.

i guess i've just grown old.

------
alexbecker
> I decided to write a program that clicked on things.

I cringed a little. I'm glad he's learned enough to know now that this is not
the right approach.

~~~
vikp
I thought this way before, and looked down on what I had made, but I'm slowly
coming around to the idea that its more about how well the approach works and
what it teaches you than about the technical details. I definitely wouldn't
code it the same way today, but it worked well and taught me a lot. I wouldn't
stop anyone who's in a similar position from doing the same.

~~~
bluedino
The worst that could have happened was that you talked yourself out of
automating it because there wasn't an 'easy way'

------
mschuster91
The biggest problem is not the lack of code and automation.

The biggest problem is that apparently Guyana allows the US embassy to search
their criminal records system:

> Search a different database with a separate program (“DB2”) for criminal and
> other history

So either Guyana allows the US wholesale access to its criminal records system
or the US systems gather this data more or less illegally from somewhere.

edit: downvotes? seriously?

~~~
maxlybbert
I have no idea what DB2 was. It's my understanding that every country in the
world wants to know the criminal history of anyone asking for a visa for any
reason (the host country may ignore that history, if it believes the crimes
were politically motivated). And it's my understanding that INTERPOL allows
countries to get at least some criminal records information. And there is
clearly some system in place for countries to make extradition requests --
which would require some criminal history information.

So it's not clear to me that the US must have wholesale access to Guyana's
criminal records system, or that it must be illegally gathering data to
populate DB2.

Even without INTERPOL, I would expect countries to have treaties to share some
criminal records information. If Guyana is trying to capture a criminal,
clearly it's in Guyana's interest to ask countries to help prevent that
criminal from fleeing.

~~~
mschuster91
> It's my understanding that every country in the world wants to know the
> criminal history of anyone asking for a visa for any reason

I'm actually fine with that. And if I ever apply for a visa, I gladly sign a
waiver allowing the respective country to request the data from my country.
But I am _not_ fine with any government in the world being able to research my
criminal record without authorization! Push, not pull.

> If Guyana is trying to capture a criminal, clearly it's in Guyana's interest
> to ask countries to help prevent that criminal from fleeing. There's a
> system for this: take away the criminal's passport and don't let him out of
> the country. That's a job of regular Guyana border police.

edit: oh, and who guarantees that (e.g.) the US respects expiration dates? In
Germany, records have to be entirely wiped after certain dates (5-20 years,
[http://www.gesetze-im-internet.de/bzrg/__46.html](http://www.gesetze-im-
internet.de/bzrg/__46.html)) - but I highly doubt that the US respect this and
just keep the data instead.

~~~
learc83
>But I am not fine with any government in the world being able to research my
criminal record without authorization!

Criminal records are a matter of public record in most countries (including
the US). That means that _anyone_ can search your criminal history, not just
governments.

There may not always be a central database, but the information is out there.

~~~
mschuster91
Back when the rules placing criminal records in public were drafted, these
records were usually kept at a courthouse, thus greatly restricting their use
to those who actually traveled to the courthouse/archive and manually
searched.

These days, everything can be searched online and especially copied and stored
everywhere, far after expiration. And this is not a good development at all.

~~~
learc83
Yes, they used to be stored in a courthouse, but most people didn't leave the
area they were born in either, so for the average person everyone you were
likely to meet knew about your past.

Criminal records were always meant to be as public as possible.

I do have a huge problem with the content of criminal records, because I have
a huge problem with what we consider criminal--drug offenses mostly.

I also worry a lot more about the non-criminal mistakes we make being
available for anyone and for all time.

~~~
maxlybbert
> I do have a huge problem with the content of criminal records, because I
> have a huge problem with what we consider criminal

While the US wants to know immigrants' criminal history, the US isn't required
to care about all of the crimes that show up on the record. For instance,
convictions for performing or having an abortion (say, in Brazil) probably
won't result in a denied visa.

