
Most programmers don't know what Expect can do for them - pmoriarty
https://wiki.tcl-lang.org/page/Most+programmers+don%27t+know+what+Expect+can+do+for+them
======
repairisntcheap
One of my most fond programming experiences was using tcl/expect to create a
custom web interface for an IBM Tivoli tape library. The stock web interface
was too expensive for a college with a few thousand users, and didn't provide
adequate access control to allow our users to restore ONLY their own files.

Yes, wrapping a telnet session is a hack... But it was a hell of a hack that
did everything we needed!

~~~
foobar_
I'd still use expect instead of docker/ansible/salt ... if I could get away
with it.

~~~
marmaduke
Those aren't alternatives; you'd create a Docker image with expect installed,
for example.

~~~
foobar_
Yeah ... but I don't want to touch Kubernetes. Kubernetes doesn't spark joy.

------
tarkin2
If, like me, you still weren't sure what Expect does then this is a great
explanation [https://www2.lib.uchicago.edu/keith/tcl-
course/topics/expect...](https://www2.lib.uchicago.edu/keith/tcl-
course/topics/expect.html)

------
doctor_eval
I’ve written so much code in Tcl/Tk/Expect - but it was all pre-Java;
literally before Java 1 was released. Tk was used in lots of Linux apps, most
notably Zircon, an awesome IRC client.

I used to think that knowing Tcl was a kind of super power. It was incredibly
productive. It was far better for quick scripting than either bash or awk -
but large programs could get somewhat unmanageable.

It would be great to see a renaissance of TCL usage, though I’d be pretty much
re-learning it today.

~~~
ghettoimp
TCL is still in use everywhere in hardware development. Practically every EDA
tool has a TCL interface.

I'm afraid I don't share your fondness for it. :)

~~~
tyingq
It's also in other places, like F5 load balancers.

------
marktangotango
Lots of talk about TCL in this thread. Everyone should be familiar with the
TCL/Guile and GNU scripting story. Also Ousterhout’s dichotomy is a useful
concept to understand:

[https://en.m.wikipedia.org/wiki/Ousterhout%27s_dichotomy](https://en.m.wikipedia.org/wiki/Ousterhout%27s_dichotomy)

~~~
tyingq
Ousterhout also co-invented raft distributed consensus.

------
cobbzilla
I used expect long ago, it was fun but a bit cumbersome.

Quite recently I was working on some python that used netmiko, which is
similar but more ssh-oriented.

Having used both, I don’t know when I’d ever use expect. Maybe if there was no
python available for some reason.

The developer ergonomics on netmiko are just better. It’s not anything about
python vs tcl, it’s that netmiko “does more out of the box”, and expect is a
bit lower-level.

------
computerfriend
I ended up rewriting most of my Expect code in Pexpect and found it both nicer
to write and much more robust.

~~~
based2
[https://github.com/pexpect/pexpect](https://github.com/pexpect/pexpect)
Python module for controlling interactive programs in a pseudo-terminal

~~~
tyingq
There's also Perl's Expect.pm. I know Perl isn't popular here, but it excels
in this sort of use case. The compact syntax for regexes on responses makes it
a good match for expect scripting. It can also be paired with IO:Pty for some
advanced stuff.

------
ohazi
Whenever I need to "screen scrape" an interactive console application, I
usually end up running the process under tmux and interacting with it via
Python after wrapping send-keys and capture-pane.

~~~
kragen
Do you know there's an expect clone for Python?

~~~
ohazi
I do now. Thanks!

~~~
kragen
Happy to help!

------
kylek
I winced at this as I've only ever used expect for dirty hacks like wrapping
ssh (or telnet to strange devices without real management). I wouldn't
discount it though, it's a decent tool when heavier machinery isn't viable.

~~~
stevekemp
I used expect for wrapping router-setup, via telnet, back in the day. Hard to
think what I'd use it for these days, thanks to puppet, ansible, docker, etc.

------
rubicks
Expect is nothing short of a blessing for those who need to automate tools
that actively resist automation. When some janky cli script decides to read
the tty and not stdin, I reach for expect.

~~~
lcall
For years I thought "I should learn to use expect...", and finally I used it
for ~ integration tests for a project
([http://onemodel.org](http://onemodel.org) \- my AGPL, highly efficient
desktop- and keyboard-oriented personal knowledge organizer that I use every
day, slow progress lately but lots of plans...), and: expect is very useful,
and has depth.

------
mzs
I wish there was something like expect that was like, "ok I've done what I
needed to init, now get out of the way." I've always had to create a loop that
just reads and writes everything eventually. Maybe I could ldpreload a tcl
interpreter and change some pointers but yikes.

~~~
_ph_
Isn't that what interact is meant to do?

[https://wiki.tcl-lang.org/page/interact](https://wiki.tcl-
lang.org/page/interact)

------
bachmeier
tcl brings out cries of "dead language" in a way that few other languages do.
I'd not hesitate to write a tcl stript today, knowing that it'd still run
correctly when I die.

Just in the past week, I've heard tcl, ruby, perl, and java all declared dead.
Oh well. Internet comments are cheap and meaningless.

~~~
eesmith
TRAC is dead. (Now I hope to hear from people using Mooers' TRAC in production
code. Or CART or SAM76. I _have_ talked to people who used TRAC back in the
1980s.)

------
skybrian
I do know what it can do, but it seems like a specialized tool for screen-
scraping command line apps that don't have a proper API. That seems like a
rather unusual task these days?

~~~
rubicks
I've wrapped expect scripts around prompt-happy config scripts supplied by
third-party vendors. What they intend to be idiot-proof usually ends up as an
unwieldy tool that takes no options and/or assumes bad defaults. The vendor
didn't intend for it to be run more than once per site installation, but such
intentions don't play nice with disposable environments like VMs and/or
containers.

------
emmelaich
The root of the problem is that too many programs require a tty.

~~~
rubicks
This. Why do programs do this when they don't have to? I'm genuinely asking
here; I'm trying to understand the thinking that motivates this kind of
decision.

------
DictumMortuum
Working in a quite popular b2b betting company. TcL is used for much of the
backend and development is still active today. Expect is used , too.

------
based2
[https://docs.ansible.com/ansible/latest/modules/expect_modul...](https://docs.ansible.com/ansible/latest/modules/expect_module.html)

[https://github.com/ronniedong/Expect-for-
Java](https://github.com/ronniedong/Expect-for-Java)

------
jolmg
I used expect once some years ago, but changed to using ruby's #expect
method[1]. I think the reason I changed was because, in TCL, some functions
seem to keep track of stuff by setting variables in the caller's scope. So, if
I took some code I wrote and defined a new function with it, substituting the
code with a call to the new function, stuff would break because calls to
predefined expect functions wouldn't have the same caller's scope anymore.

I wish I had a concrete example to share so someone here could confirm, but
like I said, it was some years ago.

[1] [https://ruby-
doc.org/stdlib-2.6.4/libdoc/pty/rdoc/IO.html#M0...](https://ruby-
doc.org/stdlib-2.6.4/libdoc/pty/rdoc/IO.html#M002150)

------
gnachman
I created coprocesses in iTerm2 because I missed Expect but I also hated
Expect because it had a very expect-y way of doing things that never felt
comfortable (though I shan't say a nasty word about Tcl!) Also, what mzs said
about "now get out of the way".

------
INTPenis
I've used Expect a couple times in the last 8 years. First to make backups of
Cisco configurations into version control. And second by Ansible (indirectly
it's used to read the sudo prompt).

So more people use expect than know it.

The issue I feel is that it's flimsy to wait for some expected output. I mean,
it's even in the name! Especially when working with unknown APIs like Cisco
IOS CLI.

So whenever I've done manual expect coding I've always felt very much out of
my element and out of control. I am a control freak btw.

It works while it works but as soon as something unexpected happens it all
goes to sh*t.

------
denton-scratch
I tried to use BATS (Bash Automated Testing System) with Expect and Tcl to
test a series of interactive scripts. It turned out to be something of a
rabbit-hole. I didn't spend long enough on the task to get really good at it,
and the result was tests that were hard to read (i.e. not what you want).
Perhaps there was a 'wrong tool for the job' factor at play.

------
noisy_boy
I need to connect to various machines throughout the day and as a first step,
need to sudo to a service account. Expect has been a godsend to automate the
interaction/input my credential (I know that it is far from being the most
secure thing but the convenience of typing an alias and being connected
including sudo is significant).

------
falsedan
The test suite for the grody old IRC bot a few jobs ago was written in Expect.
One intern season we set a project to, when announcing a new PR was published,
to format the primary reviewer’s handle slightly. It ended up taking 3 days to
update the tests, understanding expect, then writing a filter to strip out IRC
color codes.

------
elldoubleyew
I use expect to write little "drivers" for serial devices all of the time. I
don't have a lot of experience with hardware and often its the easiest way to
set up an interface. I never really knew how much it could do until one of my
coworkers recommended it to me for this purpose one day.

------
sus_007
TIL, Tcl is pronounced as Tickle, but not Tee-see-el.

~~~
bch
Either is fine and used. Joke is “tickle” among the initiated/familiar, tee
cee ell if you’re talking to management. However, never name a file test.tcl,
regardless of how you pronounce it.

~~~
sus_007
I'm a total stranger to the Tcl universe, but I'm curious what does happen if
a programmer names a file "test" ? Some kind of conflict, bug, error ?

~~~
daseiner1
“testicle”

------
rowanG077
Doing anything with expect is basically only coding for the happy path. It's
fine for some throw away script but as soon as you need any kind of
reliability it's useless.

~~~
GhettoMaestro
Have you ever stopped and asked yourself: What is a conditional statement?

~~~
rowanG077
Have you ever asked yourself if an API can return an infinite amount of
responses how many conditional you would need to handle that?

The output of a terminal is just simply too unstructured to work in any high
reliability environment.

~~~
GhettoMaestro
I would agree that using a pure CLI (Unix Shell) requires significant operator
skill to not accidentally blow things up. But that is also why you write
scripts, test them, etc.

Sitting at a root shell executing untested or uncertain things is pretty high
on the list of don't do these things.

