
Don Libes' Expect: A Surprisingly Underappreciated Unix Automation Tool (2016) - dvfjsdhgfv
https://blog.robertelder.org/don-libes-expect-unix-automation-tool/
======
thesuperbigfrog
I use Perl for a lot of my scripts, so the Expect Perl module is great for
automation without having to switch over to TCL.

[https://metacpan.org/pod/distribution/Expect/Expect.pod](https://metacpan.org/pod/distribution/Expect/Expect.pod)

The regex matching features of the Perl version make it extremely useful to
match and extract interactive results.

------
pmoriarty
I've recently become a fan of expect-lite[1], which has a much simpler syntax.

Here are some examples:

    
    
      Check the IP address of the localhost
    
      >ifconfig
      <192\.168\.1\.1
      >ip -6 addr show
      <2001:db8::f00d
    
      ssh to a remote host
    
      >ssh root@host-021
      <assword:
      >>secret_password
      # issue a command once logged in
      >ls
      >exit
    

[1] - [http://expect-lite.sourceforge.net/](http://expect-
lite.sourceforge.net/)

~~~
ohithereyou
Inputting an SSH password using an expect script seems like an anti-pattern to
me. In general, why can't you use a password-less key on the client and
restrict what commands it can execute in the authorized_keys file on the
server?

~~~
ebcode
... because you don't control the server?

------
Riverheart
Expect is a great idea. Shout-out for the python version
[https://github.com/pexpect/pexpect](https://github.com/pexpect/pexpect)

~~~
jmmcd
`pexpect` is used by Jupyter Notebook, and gets some maintainence from the
Jupyter people. So, like the article argues, it is likely to be stable for the
long term.

------
zaroth
Years ago I was automating control of 3rd party WiFi devices, many of which
presented a console which would be a much more reliable means of controlling
the device than trying to hack the web UIs of the time.

I was using Tcl mostly running on Windows, and the Expect library for some
reason had tremendous lag, or maybe it was a paid extension from ActiveState,
I can’t remember the exact reason for not using it...

So I decided to just reimplement my own ‘expect’ command using native Tcl and
a socket with just enough knowledge of Telnet control codes to get a prompt to
appear.

What amazed me was how few lines of code it took to implement a fully
functional Expect, and how useful it was as a means of structuring the
automation, specifically around timing when to send inputs, parsing outputs,
and confirming an action completed.

I’ll try to dig up the old code and post it on Github...

------
pram
I used expect a ton a decade+ ago to script a bunch of ncurses-esque menus
over a telnet connection. It was an obscure tool even then.

I can’t think of much I’d use it for today. Ansible is far better.

It was mostly helpful when you had to deal with a mainframe terminal program
that was otherwise impossible to automate.

~~~
thyrsus
If all software was designed for scale, there would be no use for expect. It's
2019 and I still have expect in production:

    
    
      * to deploy commercial software whose configuration is only through  an interactive terminal session and stored in an undocumented binary format
      * to drive gdb to inspect a suspect Red Hat 6 "will not fix" GUI component that can go into an "impossible" pure cpu loop, to determine whether to kill it or whether it's busy in a more benign state.
    

Edit: the deploy is invoked by ansible, whose own expect module recommends the
real expect "for complex cases"

~~~
geofft
gdb has a batch processing mode and its own scripting language with
conditionals, loops, etc., so you could do this entirely with gdb scripting,
probably. I've had good luck using gdb scripting for various things, notably
conditional-ish breakpoints that printed some debugging info only under
certain complicated conditions. (I do concede that if you know expect and not
gdb scripting, except is quite likely easier, and it's possible expect is
still easier if you know both, depending on the problem.)

~~~
sigjuice
gdb can also be scripted using Python.
[https://sourceware.org/gdb/onlinedocs/gdb/Python.html](https://sourceware.org/gdb/onlinedocs/gdb/Python.html)

Edit: and Guile

------
theamk
The reason that "expect" is underappreciated is right there in the text:

> most of the things that you could use expect for, are usually done much
> better using another method.

Outside of it's very narrow use case (driving un-scriptable CLIs and automated
tests for command-line tools, and maybe some exotic CLI-centered workflows),
trying to use "expect" will often result in fragile code with surprising
failure mode.

In 2nd example, script will fail if user has non-default top's config. It also
messes up terminal on exit. Use top's "personal configuration file" instead.

In the 4th example, you are not just trapping Ctrl-D in bash, you are trapping
it in all bash-spawned commands (like "cat >file"). I am going to guess that
bash's native IGNOREEOF option was what the author wanted.

In the 9th example (md5sum) , expect'll work fine -- but learning standard
unix commands like "grep -m1" and/or "tee /dev/stderr" will allow many things
that expect cannot, like doing an md5sum on millions of small files.

------
Stratoscope
Fun fact: Don is the son of microcomputing pioneer Sol Libes:

[https://www.google.com/search?q=sol+libes](https://www.google.com/search?q=sol+libes)

~~~
balnaphone
Don Libes is also the author of "Obfuscated C Code and Other Mysteries", which
is the first book I ever read that really explores all the dark corners of
programming in C; it's a really great book if you use the C language.

I used Expectk extensively in 1993, but I can't say I really enjoyed it. It
was a crufty and error-prone undertaking, and in my opinion is best for
testing, or as a last resort for talking to black-box code with an interactive
cli.

------
polm23
I once used expect to brute force a puzzle in an interactive fiction game.

------
Rich_Morin
While running scripted `apt-get -y install ...` commands, I occasionally
encounter packages that require answers to configuration questions. In some
cases, the "-y" option handles the problem, but in others it does not.

I've thought about going back to the developers of the packages, asking them
to provide workarounds based on environment variables, etc. I may still do
this, but it seems like Expect might provide a handy workaround in the
meanwhile.

However, I'm concerned that the mock-GUI dialogs that these packages use
(using terminal graphics) might make it hard for Expect to interact with them.
For example, some of these dialogs want the user to navigate through a menu,
select an item, etc.

Suggestions, anyone?

~~~
jfim
I believe those are using whiptail, so maybe a modified version of it that
allows scripting would do the trick.

------
lilyball
expect is used to run a bunch of interactive tests for the fish shell
([https://github.com/fish-shell/fish-
shell/tree/master/tests](https://github.com/fish-shell/fish-
shell/tree/master/tests), all of the files ending in .expect, where the file
interactive.expect.rc defines a bunch of utilities).

------
amelius
Some things that would be nice to have: support for mouse events, OCR of
graphical applications (so e.g. you can program it to click on a text even if
the location/font of the text isn't fixed), and support for DOM tree
navigation in case you're interacting with a web page.

------
dang
Discussed at the time:
[https://news.ycombinator.com/item?id=13130218](https://news.ycombinator.com/item?id=13130218)

------
davidgerard
+1. expect is awesome and _so_ useful.

~~~
theamk
I am curious, what are you using it for?

