

Go Replace - simple and fast search and replace tool for command line - piranha
http://solovyov.net/en/2013/goreplace/

======
JoeAcchino

        find ~/my/docs/ -type f -name '*.txt' -exec sed -i.bak 's/inheritance/composition/g' {} +
    

This will search all files ending in `.txt` and will exec `sed
's/inheritance/composition/g'` on it. Sed modifies files in-place and saves a
backup file with .bak extension (-i.bak) and is called the minimum necessary
times (-exec +).

My best advice to someone using the command line is to learn `find` + `xargs`
or, even better, `find` + `parallel`.

~~~
gladimdim
Compare with "gr what-is-it -r here-you-go". I do not have to remember all
that argument hell.

~~~
anonymouz
Well, one can write an one-line shell alias (or script) instead of reinventing
the wheel by rewriting the whole thing anew in Go. (Which is certainly good as
learning experience for Go, but the end result is easier to achieve with the
shell)

~~~
piranha
When you start adding coloring, nice output, .hg/.gitignore support, it
becomes just a big mess. Been there, done that.

~~~
andreypopp
coloring is already supported by grep, .hg/.gitignore support is just a few
lines

~~~
piranha
Just do it then.

------
artagnon
Okay, so what I learnt from this:

1\. You've imported droundy/goopt, wsxiaoys/terminal really easily. The first
provides option-parsing, and the second provides helpers for emitting ANSI
escape codes.

2\. Practically everything is overloaded. You can == to strcmp(), + to
concatenate strings.

3\. path/filepath gives you some nice goodies for path manipulation. But you
have to deal with the fallouts of making naive assumptions like whether or not
to recurse symlinks (why isn't this a flag in filepath.Walk?). This has
resulted in ugliness in your walkFunc callback.

4\. You have been able to abstract quite a bit without using Java-like
factories, although your inconsistent error passing has me somewhat confused.

5\. Since it's so strongly typed and there are no raw pointers, it's not going
to be hard to debug at all.

Overall, I'd say Go is an effective language. But not beautiful, or novel in
the slightest. It resembles Algol-68 (yes, a 1968 language) quite strongly,
and I don't think it will drive compiler technology. Their syntax is
specifically engineered to be parseable quickly (and hence super-fast AOT
compilation): what the final program looks like is secondary concern. I'm not
sure I see the appeal.

[edit: clarified that the note on compiler technology was an opinion]

~~~
piranha
> your inconsistent error passing has me somewhat confused.

That was my first somewhat big program in Go plus coming from Python made a
bit of hustle here. I still believe exceptions to be much better than this
error-returning. :\ Never learned to do it nicely. :\

> Overall, I'd say Go is an effective language. But not beautiful, or novel in
> the slightest.

Yes, this is more or less my feelings nowadays. It's good for certain cases
and if I ever need something fast, I'll use Go and not C.

> I'm not sure I see the appeal.

There is some. It's still much simpler than C (GC + stricter compiler), plus
it has goroutines (not much in this app, though), plus interfaces are really
wonderful (you don't need to declare that you implement interface, just
implement it and you're done).

~~~
artagnon
On C. C has been around for _much_ longer than Go, and a lot of people
understand C inside out. The C ecosystem is fantastic: there are extremely
good implementations of virtually everything that you can use to build a
production-grade application.

On Go. It'll take time for a programmer to fully understand how to use a new
language, and it'll be a long time before great implementations appear. If the
language isn't that much of an improvement, why will an existing C programmer
take the effort to learn it? Aren't the returns diminishing? Computers will
only get faster, llvm will only improve (tooling + compile time + link time):
in that respect, doesn't Go seem a bit short-sighted?

~~~
genwin
In the respect of existing C programmers, maybe. Having myself never mastered
C because of its complexity, in a few months part-time effort starting from
getting a book on Go I built a sophisticated website. In the respect of non-C
programmers, Go is extremely useful.

~~~
agentultra
C isn't any more difficult to understand, IMO. Obviously Go has the advantage
of hindsight and an opportunity to go back on the trade-offs that were made
for compiler efficiency before C was standardized. However, at it's core I
think C is much more simple than Go and theoretically should be easier to
understand.

I find GC to be a useful tool but have you ever looked into the implementation
of one? Or have you run into a situation where it was necessary to reason
about the performance characteristics of a GC in relation to some algorithm or
tight main loop?

I don't find C's lack of GC any _more_ complex than GC. In fact I find it much
more simple. It requires more discipline on the part of the programmer (and
perhaps that is where the appeal of a GC comes from). Perhaps it's because I
started programming with addressing memory and building up a mental model of
how it works and have only encountered GC as a useful addition to my
programming toolbox (instead of an always-present assumption).

As well, co-routine trampolining (which is what I assume the playful pun _go-
routines_ are referring to... please correct me if I am wrong) isn't a very
"simple" thing to understand either. At least they are not any more simple
than threads so I don't understand how they make learning Go any more easy for
non-C programmers (except perhaps exposure to co-routines in a language that
already has them like Python).

I guess what I am saying is that you could have just as easily mastered as
much C in those few months as you did Go.

~~~
artagnon
I would argue that C is probably the easiest language to understand, if you
know the basics of hardware. C is essentially portable assembly. The C
standard is tiny, and the behaviors are very clearly defined. The challenge is
not in understanding the language, but rather in honing the engineering
discipline required to use it to write real-world programs.

It lacks a garbage collector because there was no concept of garbage collector
when the language was created. A C program does not provide enough information
for a good garbage collector: the best we can do is a conservative garbage
collector, Boehm. The garbage collector adds to the running-cost to the
program for the promise of automatic memory management. While the GC might run
concurrently, any sort of relocation (for compacting) will require it to pause
the entire program: these GC pauses can be fatal if your program is the Linux
kernel or some high-frequency trading application. Even otherwise, it is
important to remember that every GC introduces trade-offs; the "freedom" from
manual memory management is only worth it if your GC is good.

When I'm not writing C, I like writing Ruby. I like that it's a beautiful
evolving language packed with features. And for the little applications I'm
writing, I don't care even if it takes a second longer. I like that I'm not
being verbose about anything, and not manually managing memory. Writing
something like Jekyll in C would be an absolute pain in the arse, and totally
not worth it. Obviously, the lesson is: use the right tool for the right job.

The pthreads API can certainly be very intimidating, and it requires a lot of
practice to master threading in C. I like that Go has taken the concept of
coroutines from Lisp (Scheme's call/cc) and turned it into something called
Gorountines in imperative land. Although it's nothing novel, I won't deny that
it's a nice abstraction to work with while doing multi-threaded programming.

If we were to redo C from scratch today, I'd definitely bake in more safety
features. Probably design a close garbage collected dialect. For better or
worse, that's an entirely theoretical scenario: we have Go today, but I'm not
sure where exactly it fits in:

1\. It obviously can't replace C in linux, git, zlib, ssh, openssl, libcurl,
nginx, varnish, or anything as core.

2\. Since it doesn't have generics or any higher OO features, it can't
displace C++ in chrome or llvm.

3\. When python, ruby, javascript are around, why will anyone want to use a
strongly typed language for writing web applications? Okay, maybe some
intensive web services like search.

Tools? The most popular Go repositories on GitHub are dotcloud/docker
(software deployment tool), burke/zeus (Rails preloader) and ha/doozerd (a
very specific kind of datastore).

Might be useful in Android development, but is everyone's stuck in a Dalvik
swamp there.

~~~
piranha
> Writing something like Jekyll in C would be an absolute pain in the arse,
> and totally not worth it.

Well, here you go: <https://github.com/piranha/gostatic>

Wasn't pain in the ass, wasn't hard, works so much faster than Jekyll (or my
previous engine, cyrax, which was in Python), that it's even funny.

------
lazyjones
Nice! svn support would be useful (ignore .svn directories) and the -r option
is easy to remember, but perhaps not so much in line with the already messy
standard Unix options (-r or -R for recursive operation).

Note that specifically for Go source code, there is "go fix -r" for
syntax/context-aware code replacement (see "godoc fix").

~~~
piranha
In case of go fix `-r` does something absolutely unrelated.

To be honest, I was just lazy inventing a command line key and it's recursive
by design, so it happened this way. Plus, as you say, it's easy to remember
because of mnemonics.

I haven't touched svn repos in years, but yeah, that could be added. I know
they exist, it just that I haven't felt the need to support them.

~~~
piranha
Btw, it ignores .svn and a bit more common stuff if it doesn't find itself in
a git or hg repo.

------
porker
I have to break out the manual every time I use sed. Or awk. If this works
reliably without any edge cases, it'll be fab!

~~~
piranha
It does for me. :) I partially did it because I was too tired of looking up
sed's regexp syntax and was afraid I'm going to fuck up everything.

------
madisp
For x86 binaries:

go get github.com/piranha/goreplace

go install github.com/piranha/goreplace

is another way to get it if you have GO and GOPATH set up.

I personally aliased it to 'gor' in my zshrc as the git plugin already
occupied the 'gr' (short for git remote).

~~~
piranha
Ha-ha, I've got an issue asking to add some notes about conflict with oh-my-
zsh alias. :) I hastily suggested `gp`, but it's already taken by same plugin,
so I changed it to `gor` there, thanks for idea.

~~~
alexchamberlain
Is tab completion not enough?

~~~
piranha
It's often is. But somehow people end up with 'git remote' aliased to 'gr',
and I ask the same question. :) I use gr very often and so not have to tab
complete is a plus.

------
va1en0k
I really like Facebook's codemod.py for stuff like that:
<https://github.com/facebook/codemod>

It has something like automatic and manual modes, so you can check (and fix)
every diff during the replacement

~~~
piranha
Well you see... It's in Python, which breaks the deal for me (I despise slow
startup). But interactive mode sounds interesting, that's something I'll
consider implementing.

~~~
pekk
By slow startup do you mean 0.01s of wall time? This sounds more like a
prejudice than a decision with a strong technical basis.

~~~
piranha
By slow startup I mean that 0.1s is the fastest small Python program using
optparse, os, sys, re will start up on my system (i7 2.0, ssd). Search and
replace tool will be slower. I constantly get 0.02s-0.3s results from 'gr' on
my small-to-medium repositories (up to ~80k sloc). 'ack' takes more than a
second usually.

Also it was written when I had HDD (not SSD) and codebase was on an encrypted
image. It made enormous difference to me.

~~~
cfrss
btw, here is ag, which also looks .xignore files:
<https://github.com/ggreer/the_silver_searcher>

~~~
piranha
Yes, I've mentioned it in my post. :) It's nice, but it does not perform
replaces. I could add .xignore support though.

------
swah
This isn't working for me. I get:

    
    
        C:\Users\swah\docs>gr TEST
        panic: Given path should be anchored at /
    
        goroutine 1 [running]:
        main.NewIgnorer(0xc08005bcf0, 0x29, 0x0, 0x0, 0x140110, ...)
                /Users/piranha/dev/go/src/goreplace/ignore.go:32 +0xa4
        main.main()
                /Users/piranha/dev/go/src/goreplace/goreplace.go:62 +0x1b7
    
        goroutine 2 [runnable]:
    

[edit] New version is working!

~~~
piranha
Should be fixed, can you download new version please?

~~~
swah
Working! Do you use colors? They don't work the same in DOS, I believe. I'm
seeing some escape sequences.

    
    
        ←[0;32;49mRS\terms.tex
        ←[0m←[1;39;49m←[0;33;49m25:←[0m   ←[0;39;43mOEM←[0m  &                                      \\ \hline

~~~
piranha
Eh, it's not very easy to fix that unfortunately. I opened an issue and will
try to work on this a bit, but then in a meanwhile... Maybe I can strip colors
for outputting things on windows.

Edit: argh, outputting colors on windows means making system calls. Which
means cgo and good-bye cross-compiling. I'll probably just strip all colors...

~~~
swah
Hmm, yeah, actually codemod.py had the same issue and I changed it to use
Python's colorama library.

<https://pypi.python.org/pypi/colorama>

------
M4v3R
Wow, it's super fast AND the output looks great. Good job, piranha! Copied to
/usr/bin, I will use this a lot.

~~~
nitrogen
If you're the only user on your box, you can just create ~/bin and some
distributions will automatically add it to your personal PATH. That way you
don't lose track of what you've copied into /usr/bin, leaving it to be
maintained by packages.

~~~
pekk
you can also very easily add ~/bin to PATH in e.g. your .bashrc

------
zokier
what's the rationale behind 'gr foo -r bar' instead of simply 'gr foo bar'?

~~~
piranha
Just happened so, plus if you forgot to put quotes around argument you could
easily end up with a lot of problems. :)

------
gglanzani
Are 32bit binaries coming? For the rest, great job!

~~~
piranha
Here we go, links are in readme. They work for me on 64-bit OS X and 64-bit
Linux and `file` reports they are indeed 32-bit, but I haven't tested them on
real 32-bit OS (I don't have any ATM).

