

Expect – automates programs that expose a text terminal interface - shawndumas
http://en.wikipedia.org/wiki/Expect

======
mtoddh
Interesting that nobody has mentioned yet how useful Expect can be as an
automated testing tool- my first gig out of college was at Ericsson where we
were developing at an edge router (the AXI-540) that was going to be
competitive with Cisco (this was during the first dot-com). One of my first
projects was using Expect to develop a series of smoketests - basically we'd
connect two routers (pc1-rtr1-rtr2-pc2) and then use Expect to automatically
configure RIP/OSPF/ISIS/etc. and then verify that the routes showed up, that
you could ping from pc1 to pc2 using the new route, etc. Simple stuff like
that but it gradually grew more comprehensive..

Expect made it really easy to automate this sort of testing - where the only
way you can configure something is via the CLI and you want to be able to do
this even when the only connection is via a serial console (for router
reboots). We used to run it daily and as an engineer just out of school it was
an eye-opening experience: because of the daily runs you got to see just how
often people's commits could break simple functionality in other parts of the
system.

~~~
Maven911
Ex ericsson also, were u in plano? San jose?

~~~
mtoddh
Rockville, MD - but they axed the office here after the AXI-540 was
discontinued.

------
julius
I use expect all the time. It saves soo much typing.

A while back I wrote a short blog post on it with a more advanced example on a
debugging workflow with Rails: [http://www.chocobrain.com/Terminal-Automation-
expect-Advance...](http://www.chocobrain.com/Terminal-Automation-expect-
Advanced-automation/4NRKAvM2NWZqai6a3CoDQe)

(ssh into production -> open rails-console -> type out initial commands -> let
user enter commands)

------
liotier
Expect can easily save your day, but only consider it as a last resort - when
all better ways of interacting are not available : like web scraping, it is
quite fragile because the stability of the 'API' is usually far from
guaranteed.

~~~
jlgreco
Yeah, I've always considered Expect to be sort of a canary. As soon as you
realize that you want to use it for something, you realize that there is
something rather wrong going on.

Even so, if you've got to get the job done and can worry about architectural
issues at a later time, expect is great.

~~~
hn12
I couldn't agree more.

I'm sufficiently involved with Expect that I've collected enough material for
an entire book on the subject (serious--I do a lot of writing, and am well
aware of what's involved in such a publication); at the same time, I know that
jlgreco is EXACTLY right that Expect use is a symptom of a larger problem.
Much of what I've already published through the years about Expect is how a
particular task that used to be done with Expect now can better be handled
with automation-interface-X.

------
th0ma5
The Python project pexpect is nice to work with. In addition there is some
overlap with Fabric, but anyway, fun stuff all around.

~~~
jmmcd
I think wexpect (a fork, used in Sage) is faster, and fixes some bugs.

[<http://sage.math.washington.edu/home/goreckc/sage/wexpect/>]

------
pron
I remember using Expect some 12 years ago and liking it a lot.

Back then I even released an open-source utility written in Expect, that
connects and controls multiple hosts. It is essentially a shell that sends
each command to all connected hosts and collects the responses. It even
supports automatic translation of commands to supports different shells on
different hosts. It's taken me a while to find it (I even forgot what I named
it), but now I remember: <http://sourceforge.net/projects/sysman/>. It even
had some pretty nifty features, apparantly: <http://sysman.sourceforge.net/>

Thanks for the reminder!

------
borski
Back when I worked at KBC Financial Products, we used to use Expect -all- the
time. There were a ton of archaic systems we had to interface with, and while
many of the developers/engineers were intelligent there were a couple that
were, er, less so...so Expect came in massively useful and saved everyone a
ton of time.

Also, Tcl turned out to be pretty sweet.

------
1qaz2wsx3edc
Ruby also has support for this type of behaviour: <http://www.ruby-
doc.org/stdlib-2.0/libdoc/pty/rdoc/IO.html>

~~~
wuest
Shameless plug: I built a small library to provide more Expect-like features
in Ruby, since I didn't care for the interfaces exposed, for application
development. Presently works on Linux and OS X for 1.9.3, and Linux for 2.0.0
(OS X untested, but assumed good)

<https://github.com/cwuest/expectr>

------
shawndumas
Official website: <http://www.nist.gov/el/msid/expect.cfm>

~~~
deuteronimi
Did anyone else notice the .gov? I love to see this open collaboration from
this tld!

------
coley
I used expect a while back to write a small bash script that logs into my
server via ssh and pulls the latest commit from Github. I have this set to run
on an Alfred command 'deploy site name'.

This is much more secure, in my opinion, than using a webhook to deploy new
commits.

You may have to tweak it a bit to get it working with your specific ssh agent
forwarding and ssh keys by only by deleting a few lines, but it works out of
the box without those features.

<https://gist.github.com/srcoley/3299526>

------
eb0la
There is also a nice expect script called "kibitz" that allows you to share
one terminal between two users. It's great for support and training.

There's also a nice script called autoexpect that writes simple expect scripts
based on your input: Just execute autoexpect, it will spawn a shell and write
a file "script.exp" based on what your type and the responses your get.

------
thangalin
A while back I released an ancient Java-based, simplified version of JExpect:

<https://bitbucket.org/djarvis/jexpect>

------
wslh
Expect is the precursor of web scraping.

~~~
Millennium
Pretty much, yeah. All the same advantages, and all the same drawbacks, but
over a terminal interface instead of the Web.

That said, just like with Web scraping, when you need it, nothing else will
do. It's an invaluable tool for that reason, especially when you have to work
with code you don't control.

------
angdis
Is there anything like this for Java, or .NET ?

~~~
drnewman
There's <http://expectj.sourceforge.net>,
<https://code.google.com/p/expect4j>, and
<https://code.google.com/p/enchanter>.

------
Dewie
GUI-based programs could have an underlying command-line interface that the
GUI is a mediator for. In that way, common tasks can be recorded by simply
walking through the steps in the GUI, and a script is generated without even
having to know any scripting language (at least for the simpler tasks). I know
that there are programs that have a GUI which just translates user-input into
scripts/commands; has this idea been realized in some applications? I know
that some GUI-frontend for R outputs the resulting script of your actions, so
I guess you could at least copy-paste the relevant code there.

------
absconditus
What about it?

~~~
wuest
It's a really useful tool that has a surprisingly small amount of awareness.

~~~
hn12
All true.

One of the difficulties with Expect is that it's hard to explain its
usefulness to newcomers. Expect itself is easy enough to understand; but a
common initial reaction is, "So?"

Part of the difficulty is that Expect particularly shines in occasional
circumstances that no one wants to repeat. It's a _tool_ , more than a product
--like a funny little jig that looks peculiar, but saves HOURS when you are
doing particular operations with a drill press.

Network devops, for instance, might need to monitor and update hundreds of
devices spread across multiple datacenters for a particular task accessed
through a command-line interface (CLI). The CLI is "easy", in that it only
takes thirty seconds to do one instance of the chore--but a royal nuisance to
repeat for hour after hour. A "correct" answer probably involves setting up
authentication key pairs and ... well, you can see where this is going. How
"correct" is that kind of configuration, though, when it might have to be done
only once in the lifetime of the datacenters? This is a typical case where
twenty minutes of Expect scripting can save HOURS, and sometimes days, of
error-prone typing. It would easily be worth hundreds of dollars to have
Expect on hand for this one use. No one will ever pay for Expect, though,
because it helps most for these pesky side-tasks that no one really budgets.

I can give lots of examples of good uses for Expect--but each single example
will only apply to a handful of people around the world. Nearly everyone needs
Expect in _some_ way, but each instance is highly idiosyncratic.

Expect is a little like duct tape or WD-40.

~~~
absconditus
My response was not directed at Expect, but at this submission.

~~~
hn12
I'm lost, absconditus. If you're waiting for a follow-up, please detail what
you need. I think you _might_ be saying something like, "Expect is cool,
indeed, but why would someone post a link to its Wikipedia article in HN?"
That one, I can't answer.

~~~
wuest
I think it's probably marginally better than linking the Expect site proper,
since it provides decent context.

