
Using unix as your IDE - mgrouchy
http://blog.sanctum.geek.nz/series/unix-as-ide/?
======
ajross
On the one hand, I'm pleased that someone is posting this in the new-fangled
bloggy tutorial style popular among the youth of today. It's nice to see my
preferred environment evangelized.

On the other, it's a giant facepalm that the simplest, best documented, most
powerful development tradition of the last 40 years actually needs this
treatment. That there is a whole generation of developers who come out of
school thinking that "building" is the action of pressing a button in an IDE
GUI (and that "writing a program" means launching a big wizard that pukes a
dozen files into a new directory) is just depressing.

Oh, and get off my lawn.

~~~
Goladus
My problem with make and autoconf, as a young student, was the difficulty of
finding quality, intermediate examples. Textbook examples were trivial,
professional examples were too dense to understand. There was nothing to
bridge the gap.

Trying to read the configure scripts and Makefiles for software like Apache
httpd server was overwhelming. There's obviously many man-months of work there
as well as unknown styles, conventions, and tricks.

How (and why) would I go from "gcc -lexample -o program csci201.c" and maybe a
simple Makefile to a 10,000-line ./configure script? I never had any idea.

~~~
joelthelion
Fortunately there are better build systems available nowadays. CMake is a lot
easier on the beginner, and yet it is used by large projects like KDE.

~~~
joe_the_user
Sadly, my experience with CMake makes miss the otherwise annoying make.

This is mostly from trying to compile tar balls that used it. It didn't seem
to give error messages for missing parts as well as simple make.

------
shanemhansen
I don't want to dis IDE makers. There are folks such as jetbrains that make an
amazing product. But at the end of the day, for us linux-type professional
developers, it's all text. You have maximum leverage with unix text editing
tools. Refactoring 50 files at once by doing something like:

    
    
      grep -lR 'old_method' | xargs sed -ie 's/old_method/some+other()+expression/g'

is just plain awesome. Want to find out what libraries a process is using?

    
    
      cat /proc/$pid/maps

Want to find out where that process is logging to?

    
    
      ls -hal /proc/$pid/fd/
    

Basically it's often hard to use a gui ide to accomplish an arbitrary task if
the IDE wasn't built for it, or if no one has taken time time write plugins.
At the risk of gushing, the command line is just one of the most extensible,
composable environments ever invented, and it's success over the past 40 years
proves it.

~~~
gjulianm
I really love console, but I don't think that command is so awesome (don't
tried it so I'm asking just for curiosity): What if you only want to refactor
Class1.Foo() method while not changing Class2.Foo()? That is, could you
refactor taking context into account with the CL?

I use Visual Studio to develop for WP7, and refactoring is even more awesome:
change any name, click the little popup (non-intrusive at all) and select
refactor. Done. Not only in that file, but also in your project and any other
project you have opened.

I'm not trying to bash command line tools (I'd really love to know them
better), but I think that there are things that are far easier with a
graphical IDE.

~~~
ashleyblackmore
"I'm not trying to bash command line tools" hehe

"if you only want to refactor Class1.Foo() method while not changing
Class2.Foo()"

vim has functionality for scope in most languages (see
[http://vimdoc.sourceforge.net/htmldoc/eval.html#internal-
var...](http://vimdoc.sourceforge.net/htmldoc/eval.html#internal-variables)).
If vim doesn't have the functionality, it is a one-liner in your config file.

Additionally, there are language-specific vim refactoring options such as
eclim (eclipse functionality in vim and vice-versa) for java, and rope/bicycle
repair man for python

edit: on a side note, these religious discussions are great for unearthing
great links and cli tricks!

------
einhverfr
This is more or less exactly my approach. gnome-term, vim, and since this is a
web app, a testing environment of Apache, Chrome and/or Firefox, etc. Between
proper usage of vim, shell tools (grep, find, etc) svn, etc. I can put
together the best of breed tools to form a really, really good development
environment.

------
tallpapab
I agree that many underestimate the power of bash for development. Make, in
particular, is much more widely usable. However, there are some things that I
find very convenient in an IDE that are harder to do at the command line.
Finding usages of a name (variable or function); finding the definition of
same. Renaming same throughout the code base. Yes, I know about "find | xargs
grep", but it's a bit more convoluted than IDE usage. Maybe I just don't know
enough. Can anyone point to command line or vim tools to accomplish these
tasks? Code analysis and warnings are another thing IDEs seem to do better.

~~~
babarock
I'm going to be a tad pedantic and note the following:

\- bash and make are completely separate programs. As a matter of fact, I
spend half of my time developing with ksh and Solaris make. The two programs
you mention don't go 'hand in hand'.

\- `find | xargs grep` is a terrible construct. 99% of the time (if not a lot
more) you'd prefer using something like `find -exec`, I have yet to come
across a version of `find` that wouldn't work that way.

My point is not to be an arrogant prick, and criticize the parent post. These
kind of misconceptions usually indicate poor understanding of the larger Unix
philosophy. By her own admission, my parent "just do[es]n't know enough".

I believe that in order to truly use Unix as an IDE, one has to go beyond the
(arguably bad) reflexes taught by common GUI IDEs. It's not simply a matter of
"What command can replace feature X from Netbeans/Eclipse/Visual Studio/...?"
Modern IDEs weren't conceived with sed/awk in mind, aren't (for the most part)
scriptable like most Unix shells are, and aren't sitting on top of a freaking
Lisp machine like emacs is (you know the whole enlightment deal, lisp is
superior, bla bla bla).

I am sounding like an evangelist, Unix tend to do that to me. I am trying to
make a simple point:

It's normal to feel that something is missing if one is simply transposing her
knowledge from point-and-click IDEs to the Unix shell.

(To answer your question, a combination of sed and awk does wonder to rename
across the code base. And SO much more ...)

~~~
slurgfest
There needs to be far better evangelism/PR to tell people the right way to do
all these things so that they understand how to do all the IDE stuff, the
truth is that the number of use cases people need is not unlimited... of
course it is good to know the full range of flexible tools, but it's far from
obvious to most people whether or not find | xargs grep is good or not, or how
to do a method rename

~~~
npsimons
_of course it is good to know the full range of flexible tools, but it's far
from obvious to most people whether or not find | xargs grep is good or not,
or how to do a method rename_

It's also far from obvious whether a for or a while loop best fits the problem
at hand. How does one figure these things out? By learning about the
tools/constructs and using them (experience). Furthermore, it is expected of
software developers to know these things.

It's all well and good to be an IDE jockey, but being a software developer
entails more than clicking through wizards and filling in some logic. Learning
tools (such as UNIX CLI tools) opens up a whole other world of untold power in
accomplishing not just programming tasks, but everyday tasks as well (how many
people know that there is a CLI unit conversion program that not only has more
units than you can shake a stick at, but is scriptable and has a tiny
footprint to boot?). Not to mention that these tools have been around and
ported to just about everything, and will probably continue to be around for
ever, and eat less RAM and CPU while being more flexible and powerful than
their GUI counterparts.

Edit: I didn't mean to come off condescending and scolding, but I forgot to
insert some helpful links in response to your statement that there should be
better evangelism/PR for shell programming; hope these links fit the bill:

<http://www.bashoneliners.com>

<http://www.commandlinefu.com>

<http://www.shell-fu.org>

<http://noone.org/blog/English/Computer/Debian/CoolTools>

<http://noone.org/blog/English/Computer/Shell>

~~~
slurgfest
You don't have to sell me on using the shell at all and I have seen one-liner
databases (which are really only fun once you have a pretty good level of
mastery)

I find that IDEs make everything so needlessly complex and inflexible - I
don't want 2000 little icons in 200 drawers managed by 54 XML files that I
will eventually be expected to edit.

The observation is that they are winning anyway. In fact, people even think
they are easier, AND they think their IDE does things that can't even be done
otherwise! These people are smart enough to develop software, yet they are
opting for what we think are dumb tools. Either we are just wrong, or there is
just a misunderstanding about the relative easiness of IDEs.

I think that commercial platforms and products aimed at consumers (including
developers) tend to have people paying careful attention to marketing and
experience, to add a layer of glitz and wow and accessibility. It isn't that
it could not be done but no one is bothering, once you know the efficient way
then there is no point dressing it up.

I also think we have built up a culture which is somewhat punitive to newbies.
Too many people treat programming and composition of command line tools as
some kind of dick swinging competition rather than the inherently simplest and
most straightforward way of doing things, which is SUPPOSED TO make your life
easier and let you do things you couldn't otherwise do.

------
drcube
Previous submission:

<http://news.ycombinator.com/item?id=3594098>

~~~
DeepDuh
Thanks alot for that link, it lead me to this very interesting talk by Brent
Victor, as kind of a counter argument to that principle:
<http://vimeo.com/36579366>. And holy shit does he have a point. Many of
today's programmers seem to be trapped in the thinking that the unix and vim
principles are the holy grail, forever to be pursued, never to be changed.
Isn't it time we stand up and reassess the tools we have? And I'm not just
talking about the teleprompt generation tools like unix and vim, but also
today's IDEs that have basically evolved as extensions to those tools.

I also recommend the following talk about "subtext":
<http://www.subtextual.org/subtext2.html>.

~~~
lubutu
I would argue that very few of today's programmers really follow the Unix
philosophy, but those I know who do would likely agree that our tools should
be reassessed. But you seem to be conflating the idea of reassessing our
_tools_ and reassessing our _principles_. In my opinion the core principles
[1] are eternal, however transient the programs themselves.

[1]: <http://en.wikipedia.org/wiki/Unix_philosophy>

~~~
DeepDuh
I'd argue that the tools (if they're 'good' in itself) reflect the principles
with which they have been designed. And no, I don't think that these
principles are eternal. We should look at the context under which these
principles have been formulated or evolved, compare it to today's context,
pick out the principles that still apply and throw the rest out. As an
example, building small programs to do well defined, easily testable tasks and
chain them together, is certainly still a good design principle. Strings as
the universal interface, however, is not in my opinion, since it just doesn't
reflect how most of today's UIs are built on objects - thus we have today's
disconnect between the terminal and the GUI.

~~~
lubutu
You believe that text as a universal interface isn't a good design principle
because it doesn't reflect how other UIs, built upon completely different
principles, behave? Is it not possible that it remains a good principle that
WIMP GUIs simply do not adhere to? (Also related, "everything is a file.")

~~~
DeepDuh
Unformatted text (as in text that doesn't follow a machine-understandable
format like xml or json) is not a good interface today, I absolutely believe
that. Imagine if "ls" would return an xml table that you could easily reuse in
other programs, including graphical editors. Imagine that the OS could
understand what every column represents and give it a name. You could do
something like

ls | showtable in.modified in.filename --sort:1.

Of course you can do this today with sed, but in a very unintuitive way not
easily accessible for beginners.

------
goforit
By becoming dependent on "IDE's" you are only making it easier for companies
to control you. As developers you are important and companies want to have
some control over what you do. That was Microsoft's original plan and it has
worked beautifully.

They were quick to provide an OO environment where someone could create
something "impressive" very fast. A little eye candy is all it takes to
impress many developers and users. And that's all she wrote. They beat
everyone else to the punch. To this day, IDE's and OO programming still rule.
And MS's mysterious API's are still very much a competitive advantage, tying
programmers to the MS IDE and platform. They control you.

The unfortunate result of this over a long period is that today's developers
are a lot less knowledgeable about how to build things from the ground up.
Take away their IDE and they can't really do much.

If you want to create something truly "new" and push the envelope (think
systems programming), then you need to start at a lower level than the IDE
where someone else has handled the low level details for you. (Who built those
objects anyway?) A recent post here on HN said that the most talented
programmers are the ones who can move from different levels of abstraction
effortlessly. From very high to very low and back again. The IDE keeps you in
a box. One level of abstraction. Dirty low level details are scary to many
developers.

IDE's are useful. They increase productivity exponentially. But if we take
away your IDE and you are helpless without it, that's a problem from a
progress standpoint (e.g. new systems development). Companies like MS are
controlling you and controlling the rate of progress. And they have an
incentive to maintain status quo. The envelope does not get pushed.

------
swdunlop
I always think of unix as the Disintegrated Development Environment, and I'm
happy to keep it that way. Every time I go down the rabbit hole of trying to
integrate build, debug and edit into a single tool, I eventually go back to
using individual commands.

------
pnathan
My understanding is that Emacs is not really Unix in flavor or mentality.
GNU's not Unix, remember. :-)

To the best of my knowledge - i.e., I've read somewhere - Emacs is a
derivative in style from Lisp Machines at the MIT Lab. It's not really a Unix
program.

~~~
ajross
Unix is an aesthetic. It's squishy. Emacs is a text editor. Editing text is
very much a Unix Thing. Emacs does a lot of other crap too, most of it (email,
terminal emulation, IRC...) intended to replace or augment some other core
Unix Thing. Is putting all those Unix Things into a single box (that isn't
unix) a Unix Thing? Probably not, but most of us forgive emacs that
transgression. Don't use the crap that doesn't fit your world view, and it
remains a unix tool.

Emacs predates the Lisp machines, btw. The original was written (as TECO
macros) on ITS. The Lisp machines came out of the same culture and thus had
their own emacsen. But GNU Emacs that we are all using today was written on
Unix from day one.

~~~
jff
However, Emacs still holds on to all the unique ITS key combos, which is why I
always hit ^W and wonder why that word didn't get deleted.

There's a public access ITS system over in Sweden, or you can grab an emulator
and run it yourself. I recommend trying it just so you can see how Emacs and
Info have remained essentially the same (RMS's sad devotion to that ancient
religion, etc)

------
dsrguru
The author mentions that ack "allows you to use Perl-compatible regular
expressions (PCRE)" in contrast to grep. I want to point out that grep does
too, or at least the versions on Linux and Mac OS X, when the -E flag is used.
I don't know if it actually uses the PCRE library or if it even uses the same
character class shortcuts that Perl does, but it's at least modern in the
sense that it uses (...) instead of \\(...\\) for grouping and {n,m} instead
of \\{n,m\\} for specifying the number or range of occurrences, as well as +
for {1,}, etc.

Edit: Slight fact correction. The author of this article led me to believe
that ack uses Perl Compatible Regular Expressions, which is a C library that
implements a regular expressions system whose pattern syntax is very similar
to what's found in Perl. Now that I've read ack's website, I see that ack is
written in Perl and actually uses Perl's regexps. The author must have just
meant that ack's regular expressions are literally "Perl-compatible."

~~~
tedunangst
grep -E is just extended regex, which is also a million years old. grep -P,
for some versions of gnu grep, will get you the experimental perl compatible
regexes.

------
bitwize
For a time, Unix was specifically developed _as_ an IDE; the PWB/UNIX system
was basically a version/distribution of Unix intended for development, even of
programs for other machines like the IBM System/370.

Time marches on, however; and modern IDEs offer rich tools like refactoring
capability, in-place documentation (IntelliSense) and fully integrated
debugging tools that are a considerable evolution from what was available in
the 1970s, that can only have come from having all the tools be aware of the
language that's being used. From dumb, simple tools that munge streams to text
to smarter, context- and semantics-aware tools that operate on program
representations -- that's progress. Accordingly, Unix itself is no longer an
acceptable IDE.

------
primitur
vim but .. No cscope? I am disappointed. cscope is a power tool when it comes
to editor-integrated development.

<http://cscope.sourceforge.net/cscope_vim_tutorial.html>

I have cscope+vim+make/build+gdb+logging windows on auto-open for my developer
desktop terms, and spend a lot of time navigating large codebases with cscope
in all its incarnations. If you like this article, but haven't heard of
cscope+vim, give yourself 30 minutes to check it out ..

~~~
soofaloofa
cscope is great. It would be even better if they supported more languages.

------
stcredzero
"X as your Y" is a sign of good architecture in X.

You can consider Unix as a collection of hierarchically organized entities
that can act as nouns and verbs. To develop in this system, all you need to do
is to add verbs for text editing and compiling.

BeOS/Haiku has this versatile property as well. In fact, you can use BeOS as
your media player/mail client with no app at all.

Python and Ruby are built with this quality. There is tremendous power in
their REPL. The language is somewhat its own advanced debugger.

Smalltalk is much like Unix, with its structure of nouns and verbs and a REPL
everywhere. This is precisely why the Smalltalk compiler is just another first
class Smalltalk Object.

Any app like an IDE is actually something like a design pattern. It's actually
a symptom of something _lacking_ in your language/substrate. This isn't
something bad or wrong, because kitchen sink architectures have their own
drawbacks. It's also possible for a system to be too generalized.

Rather, the sign of something wrong (or conversely the sign of something
right) is the extent to which you are building everything yourself or
composing pieces that are already there. To what extent are you building
together your own, and to what extent are you exploring powerful tools that
are already there and putting them together to get things done?

Being able to do the latter is the definition of power in most contexts. The
too much need for the former is pathological.

Unwieldy IDEs, app servers, dependency management, etc -- this is all a sign
that something needs to be improved.

------
pjmlp
Powerful as the CLI might be, what are the CLI ways of:

\- replacing Foo just in specific modules/classes

\- finding the call stack of a specific function/method even across closed
source binaries

\- provide a visual overview of modules/classes dependencies

\- GUI design

\- code refactoring

\- debugging with hot code replacement on the fly

\- browsing symbols in object files / binary modules

------
s00pcan
I'm in the process of completely switching over to the development style
described in this article (I read it last weekend). I started typing a
response to this and accidentally ended the first sentence with a semicolon,
if that gives you any indication of where I am at in this process. I'm at a
PHP/.Net shop doing LAMP development and everyone else here simply installed
Ubuntu or Windows XP/7 then Eclipse, checked out the code from SVN and started
working. I installed Ubuntu fairly quickly but then got fed up trying to get
Eclipse and a bunch of PHP plugins working together. That was a complete waste
of time in my opinion; at best I would be learning Eclipse and Ubuntu. I
switched to Vim for all of my editing needs about 16 months ago when I was
taking a useless Perl class, so I started using that daily at work. 8 months
of 40 hour work weeks later and I'd say I'm pretty decent at it now. Two
months ago I got fed up with the awful window management in Ubuntu and
switched to Xmonad. Configuring my window manager by writing Haskell and
recompiling it is preferable to watching some other piece of garbage break or
crash constantly and wasting time moving windows around with a mouse for the
rest of my life. Shortcuts didn't help; working with the default keybindings
in Ubuntu for web development on a single screen hurt my hand enough to buy a
different keyboard, but it was still glitchy enough to be a daily annoyance.

My development environment currently consists of Ubuntu/Arch Linux (haven't
switched at work yet), Tmux, Vim, Xmonad, Urxvt and Zsh. The only sane way to
start moving towards using these tools is to use one new tool at a time, see
which defaults annoy you and then customize it to your workflow. I would
definitely not start with anyone else's default settings (Janus for Vim or Oh-
My-Zsh) unless you just want to see what these tools are capable of. Last week
I completed a full productive workday without touching my mouse except to move
it out of the way and over the weekend I had to work from home at this job for
the first time. I had my ssh tunnels and keys setup already so it was a simple
as attaching to my tmux session from work and continuing where I left off.

The reason I am going through this process is to learn more about Linux/Unix
but more importantly I am sick and tired of setting up programs on computers,
learning settings and modifying configurations through guis that change at the
developers' whims. I don't need or want a system that anyone can use; I want a
system that works the way I think with my exact hardware. The best benefit of
doing all this is that I can actually learn and understand what my tools are
doing and remove as many abstractions as I can handle. Most recently I learned
about strace; I always wondered how people came up with answers to Linux
questions about random dependencies.

------
Feoh
I like the article as I am also a command line junkie, but I have an issue
with it. I might be being pedantic here, but the fundamental premise of the
article is a falsehood.

UNIX is in no way, shape or form an IDE. IDE stands for Integrated Development
Environment and means exactly that. All your tools under one roof.

I do not deny and in fact would also assert that the separate tool chain /
everything is a stream of bytes philosophy can be used in amalgam as an
incredibly powerful methodology for software development, but that doesn't
make it an IDE.

~~~
mkramlich
> UNIX is in no way, shape or form an IDE. IDE stands for Integrated
> Development Environment and means exactly that. All your tools under one
> roof.

agreed. if anything, it's a UDE: Unintegrated Development Environment. which
is it's strength and weakness. the initial learning curve is steeper than with
an IDE, maybe, but you don't run into these painful low ceilings like with an
IDE. so much synergy & integratability in the CLI paradigm. pick and choose.
customizable UX, custom workflows, automation, and typically much less opaque
configuration, lower idle cpu and memory use, etc.

------
sequoia
Surpised tmux wasn't mentioned in the list of core tools; Author has written
about it before it seems. tmux is _the_ key to building an IDE in *nix for me.

~~~
dsrguru
I've been using GNU screen for years and am happy with it. A brief bit of
Googling suggests to me that the main advantages of tmux over screen are that
its code is designed better and that it's actively maintained and, therefore,
less buggy. It also seems like its configuration is more logical. As someone
who has already configured screen to my liking and who hasn't run into
noticeable bugs, I don't have any real reason to make the switch if those are
the only advantages. Are there others? (e.g. better features, etc.)

~~~
petepete
For me, tmuxinator[1] is a huge advantage. I can easily define environments
suited to various tasks and projects, launch them easily and lose none of
tmux's power in the process.

For example, my 'work' setup is very IDE-like; when it starts it mounts my dev
VM (nfs), starts vim, opens windows for logs, database client, SSH to the VM,
source control (tig and git).

[1] <https://github.com/aziz/tmuxinator>

------
JoelMcCracken
The full series: <http://blog.sanctum.geek.nz/series/unix-as-ide/>

------
krollew
Yeah. POSIX shell is my IDE of choice too. So I agree with it. Although there
is one exception - development in C#/.NET which is only environment in which I
wouldn't code with anything else than Visual Studio. :) But maybe it's just a
matter of way I was tought to use it, with all those one click - hundreds LOCs
stuff. So I can't imagine coding in .NET without it.

