
Ripgrep 11 Released - burntsushi
https://github.com/BurntSushi/ripgrep/releases/tag/11.0.0
======
sqs
Ripgrep is awesome. Thank you for making it!

In addition to being really fast, it "just works". By that I mean it
automatically excludes the files I want to be excluded, like `.gitignore`d
files and binary files. I know I can configure ack-grep (ag) and other tools
to do that, but not needing to configure it is nice.

BTW, if anyone hasn't read
[https://blog.burntsushi.net/ripgrep/](https://blog.burntsushi.net/ripgrep/),
highly recommended. It's about how ripgrep is so fast. (Edit: Just saw another
comment mentioned this, too. Goes to show that making a single high-quality
blog post has a big impact.)

~~~
loeg
ack and ag (the_silver_searcher) are two distinct tools. The latter
automatically excludes .gitignored and binary files, just like ripgrep. In
fact, it seems ag inspired ripgrep's behavior; in the blog post[0] introducing
Ripgrep, BurntSushi specifically highlights that rg aims for the usability of
ag:

> I will introduce a new command line search tool, ripgrep, that combines the
> usability of The Silver Searcher [ag] …

[0]:
[https://blog.burntsushi.net/ripgrep/](https://blog.burntsushi.net/ripgrep/)

~~~
minusf
both of these are slower than rg. ag has issues with big files, it's
unreliable.

~~~
zwp
I've not used ag but just a small shout out for ack: it's a single Perl file
(easy to install, v1.x will work with even antique Perl), it works on weird
hardware/OS (no compilation required), it contains its own man-page (ack -man)
and is fast enough for typical sysadmin tasks and moderate size codebases.

(Ack also supports the full PCRE syntax but that's less of an issue now rg has
-P).

If your work takes you to minority platforms ack is certainly worth keeping in
your toolbox.

------
nindalf
I’ve learned a lot by reading burntsushis code, so thank you for everything
you’ve written.

Separately, ripgrep is a great advertisement of what Rust is capable of.
First, it shows it’s possible to write highly performant and reliable
applications without resorting to non readable code. Second, it shows how
simple it is to separate applications into a library + binary thanks to Rust’s
package management. Ripgrep is merely a command line interface to libripgrep,
which is reused in other applications like VS Code. The regex and directory
walking code that was once part of the main ripgrep code base are now crates
that are reused to great effect throughout the ecosystem.

~~~
ngrilly
Last time I checked, VS Code didn't use librigpreg directly. It was calling
the ripgrep binary instead.

~~~
burntsushi
This is correct. VS Code uses ripgrep's JSON output format from the binary.

libripgrep is nice for building your own Rust programs for searching stuff.
But there is a not insignificant amount if code that translates your argv into
uses of libripgrep.

------
dfabulich
If you've been holding back on Ripgrep because of its 0.x version numbering,
now's the time to adopt it. Ripgrep is faster than grep, faster than ag (the
Silver Searcher), and even faster than git grep.

The author breaks down all of these details in this 17,000 word blog post,
full of detailed benchmarks.
[https://blog.burntsushi.net/ripgrep/](https://blog.burntsushi.net/ripgrep/)
(It's from 2016, but it's still mostly in good shape; ripgrep has only gotten
faster since then!)

~~~
loeg
The blog post is good, and fair at the time. One caution I would add is that
time has not stopped for the other projects, either. I believe ag (the silver
searcher) has also seen considerable performance improvements since 2016.

~~~
burntsushi
> I believe ag (the silver searcher) has also seen considerable performance
> improvements since 2016.

Do you know what they are? I haven't noticed them, although it has been a
while since I've rigorously benchmarked it. From looking at the source code, I
don't think much has changed. e.g., it appears to still be using memory maps
on Linux when searching large directories.

~~~
BeeOnRope
Out of curiosity, is there a better approach than memory maps?

~~~
shaklee3
The author goes over that here:

[https://blog.burntsushi.net/ripgrep/](https://blog.burntsushi.net/ripgrep/)

------
nojvek
Vscode integrates ripgrep by default in their “search project” feature.

I mostly use Vscode because of its fast search, decent auto complete and
simple editing experience.

Whoever made ripgrep, deserves massive kudos.

~~~
fiddlerwoaroof
The OP is by the author of ripgrep

------
nudpiedo
I cannot understand all that amount of excitement. Are we under heavy fanbot
attack or am I missing something? If the whole thing is that, is faster than
grep and "in Rust we Trust" then please feel free to ignore my comment.

I would use OSX's slow grep every single day to find things and I never had
any of the non-problems problems which ripgrep is trying to solve. About
recursive, case insensitive and search and file number just use grep with the
flags "-Rni", so I don't get in which aspect is superior. I never ever had
half a thought about grep's performance at the systems I use it (already
faster than my cognitive bottle neck). If I ever would need something better
than grep I would try to move to the next level or deal with lots of
unstructured data I would perhaps move to a local search engine using indexes.

And personally I would consider an antipattern to skip the gitignore contents
per default. What kind of machines or use cases are hitting the people in the
comments?

~~~
burntsushi
> I never had any of the non-problems problems which ripgrep is trying to
> solve

That's OK. Not everyone has the same set of problems, and not every tool that
is built must be used by everybody. I document pretty clearly what ripgrep
does in the README, so if it doesn't target your use cases, then obviously you
just don't need to use it. But also, just as obviously, it should be clear
that plenty of other people do fit into the use cases that ripgrep targets. It
turns out that a lot of people work on big code bases, and as a result,
running the default grep command can be quite slow compared to something that
is just a little bit smarter. It gets super annoying to always remember to
type out the right `--include` commands to grep. I know, because I did it for
ten years before I built ripgrep. I mean, this is the same premise that drives
other tools like `git grep`, ack and ag. ripgrep isn't alone here.

See also:
[https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#pos...](https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#posix4ever)

> And personally I would consider an antipattern to skip the gitignore
> contents per default.

Then use

    
    
        alias rg="rg -uuu"
    

and you'll never have to worry about that specific anti-pattern ever again.
;-)

~~~
nudpiedo
Thanks for answering burntsushi I didn't expect you to reply personally. It's
true that there is some good value in having smart sane defaults, and skipping
always looking for 2 or 3 advanced flags which are only used from time to time
so I ended up giving it a chance (installing it via brew). The difference
showed up immediately.

At the moment first impressions are good, seems to be killing it at the most
common grep use case I am having, so I will continue using it and trying for
some weeks until I can have a better formed opinion.

The only unknowns for me are are the edgy use cases such as piping to other
processes where I might end up falling back to system's grep or maybe learn
better the rg possibilities.

~~~
burntsushi
Neat. FYI, ripgrep works fine in pipelines. You should be able to use it just
like grep in those cases.

------
j0e1
I love ripgrep. For me it has done an exceptional job at searching and finding
a large list of search strings over pretty large text files.

Tangentially, it is because of great tools like these written in Rust that I
have major respect for the language.

------
SanchoPanda
The -pre flag is rarely mentioned but was a revelation for me in for dealing
with MS office files and pdf documents. I use it every single day.

alias rgpre='rg -i --max-columns=1500 --pre rgpre'

*edit Adding my script here, and would be grateful for any improvements. Goal was to catch doc,docx,ppt,pptx,xls,xlsx, epub, mobi, and pdf and still work on termux with default packages there only.

[https://gist.github.com/ColonolBuendia/314826e37ec35c616d705...](https://gist.github.com/ColonolBuendia/314826e37ec35c616d70506c38dc65aa)

~~~
burntsushi
That's a lovely script. And yeah, I should try to publicize --pre a bit more,
since it is pretty handy. I don't use it a ton myself, but when it works, I'm
really thankful for it. (That might sound weird since it's in my tool, but I
was initially kind of opposed to adding it.)

One bummer is that it can slow ripgrep down quite a bit: one process per file
is not good. As a tip, if you add

    
    
        --pre-glob '*.{pdf,xlsx,docx,doc,pptx,...}'
    

and whatever else you need, then ripgrep will only invoke an extra process for
those files specifically, which can significantly speed up ripgrep if you're
searching a mixed directory (or just want to have `--pre` on by default).

~~~
SanchoPanda
That addition was an incredible time savings in my average search, I'm very
grateful.

A more prominent mention, and a sample script, would be very useful imho. Just
googling for a way to do this for even just one of these file types yields
inferior commercial solutions at best, often just nothing.

And while I'm not sure how to make it much faster from a search perspective, I
was able to find a 40% improvement in invocation by switching from rgpre to
rgg as my alais.

I ended here:

alias rgg='rg -i --pre-glob
'*.{pdf,xlsx,xls,docx,doc,pptx,ppt,html,epub,mobi}' \--pre rgpre'

~~~
burntsushi
Hah nice! I'll noodle on this and figure out how to publicize it more. Your
script would be very helpful as an example. Do you mind if I use it in the
docs somewhere?

~~~
SanchoPanda
Please feel free.

Even a cursory mention of the ability at the end of the readme would likely
help many. With a verbatim script and verbatim command using it helping many
more. I only stumbled upon this feature with enough context to really get it
in the Ubuntu man pages and I don't know if I ever would have gotten to the
pre-glob feature to be honest.

I do believe that technically oriented non-programmers would be the biggest
winners from more exposure around it.

~~~
burntsushi
Great! I filed an issue for it:
[https://github.com/BurntSushi/ripgrep/issues/1252](https://github.com/BurntSushi/ripgrep/issues/1252)

------
zlynx
I also love ripgrep and use it all the time. Thanks!

I've lately been doing some Windows programming. Ripgrep from a Git Bash
command-line can generally find ALL instances of some word before Visual
Studio's Find In Solution can even return one result!

The only downside is that rg does not understand namespaces when searching, so
a commonly used member variable will show up all over the place. Can still
usually find the right location by visual inspection before VS2017 though!

------
linsomniac
I tried ripgrep 0.10 when it came out. I did variations of "rg foo", "rg -i
foo", "rg -ri foo", "rg -ri foo .". All of them returned 0 results. Where "ack
foo | wc" returned 79 lines. Sooooo. I uninstalled "rg".

I just tried again and same thing. If I run with "-u" I get results.

I've just spent some time hunting and figured out that in my "projects"
directory I had a ".gitignore" that had " _" , which I did a long time ago to
stop "git status" from spamming me if I ran it in the wrong directory (base
projects, a hg project which I have a lot of).

For some reason "ack" recognizes that it is in a hg repo when I run it, and
"rg" also recognizes that, but it _ALSO* searches up the tree looking for a
".gitignore" which ack does not. Good to know.

~~~
burntsushi
If you use the `--debug` flag, it will tell you which files are being ignored
and why.

If ripgrep doesn't search any files at all, then it should print a warning to
stdout.

ripgrep ascends your directory hierarchy for .gitignore files because that's
how git works; .gitignore files in parent directories apply. ripgrep doesn't
know anything about Mercurial, so it doesn't know to stop ascending your
directories.

ack is different since it doesn't look at your .gitignore files, so it doesn't
have the same behavior here.

~~~
linsomniac
Oooh, ack ignores ".git" and ".hg", but ignores the ignore files. That,
actually, seem like pretty ideal functionality. I guess the equivalent would
be "rg --no-ignore-vcs". I had tried "-vvv" to no avail, thanks for the
pointer about "\--debug".

------
cjohansson
I love ripgrep and I’m thankful for the efforts that went into the project.
It’s seems like a delicate programming project. Works great with Emacs btw

------
m0zg
The only reason why I install Rust on all my machines is to install ripgrep
with Cargo. :-)

~~~
mikojan
This.

------
shaklee3
Does anyone know how ripgrep's pcre support compares to python/Perl/awk/sed
for speed? I noticed it can do substitutions, so I'm curious.

~~~
asicsp
I haven't tried it yet, but it would depend on what sort of features you need

Python wouldn't be a good choice for cli usage

Perl is awesome to use from cli, and it is not just simple search and replace,
see my tutorial[1] if you want to see examples

sed and awk are awesome on their own for cli usage, sed is meant for line
oriented tasks and awk for field oriented ones (there is overlap too) - one
main difference compared to perl is that their regex is BRE/ERE which is
generally faster but lacks in many features like lookarounds, non-greedy,
named capture groups, etc

you could check out sd[2] for a Rust implementation of sed like search and
replacement (small subset of sed features)

[1] [https://github.com/learnbyexample/Command-line-text-
processi...](https://github.com/learnbyexample/Command-line-text-
processing/blob/master/perl_the_swiss_knife.md)

[2] [https://github.com/chmln/sd](https://github.com/chmln/sd)

~~~
shaklee3
Thanks! I was mostly asking because I've found Python's regular expressions to
be extremely slow compared to egrep, and Perl is in the middle. It's annoying
to have to call subprocess functions in python just too launch egrep on large
files.

------
xrouge
Is there a way to output the results in Visual Studio format, which is
<filename>(<line number>)<result>? This way when I hook up ripgrep as an
External Tool, the line shown in the Output window can be clicked on to open
the file and go to the specific line. I see in the source code that the
separator is ":"; VS expects () around the line number.

~~~
burntsushi
Not natively, no. You could probably write a simple script that transforms the
output into what you want though. For example, on Unix at least,

    
    
        rg -n pattern | sed 's/:/(/' | sed 's/:/)/'
    

That won't work in every case (e.g., in the case of a file path containining a
`:`), but it might get you pretty far. And it could probably be made more
precise.

Otherwise, you could always use ripgrep's --json output mode to craft its
output into whatever format you like, but that's probably more work. :-) The
JSON format is documented here: [https://docs.rs/grep-
printer/0.1.2/grep_printer/struct.JSON....](https://docs.rs/grep-
printer/0.1.2/grep_printer/struct.JSON.html)

------
VohuMana
I use this program on a daily basis at work, it is so fast and easy to use, I
love it. Thank you for creating it :D

------
stkrzysiak
Here I thought I was hip using ag, the announcement blog post linked in this
thread is fascinating and thorough, looks like it's time for me to try out rg.

------
stevebmark
Ripgrep is so awesome, "some performance improvements" doesn't make any sense
because it's already horrifically fast.

------
1023bytes
What makes it so fast? Apart from skipping gitignored files and binaries.

Parallelism? Some Rust-specific features?

~~~
Shoop
Here's a blog post from the author explaining how rg works: [1]. It's from
2016 so the benchmarks may not be up to date, including for rg's competitors.

[1]
[https://blog.burntsushi.net/ripgrep/](https://blog.burntsushi.net/ripgrep/)

------
keithnz
version 0.10.0 to 11.0.0 is a big version leap!

~~~
ksherlock
emacs did that too - 1.12 to 13.

~~~
cpeterso
Solaris jumped from 2.6 to 7 (the "2." is now implicit). Of course, Windows NT
jumped from 4.0 to 2000. ;)

~~~
LeonidasXIV
But it didn't, really. NT 4.0 was followed by NT 5.0 (branded Windows 2000)
and then by 5.1 (Windows XP) and then by 6.0 (Windows 7).

~~~
signal11
Vista was 6.0. Windows 7 was 6.1 and 8 was 6.2 (8.1 was 6.3). They finally
bumped up the version to 10 with Windows 10.

------
heyoni
goim was insanely good so I guess I’ll have to check this out!

~~~
burntsushi
If you liked goim, then you'll definitely want to check out its replacement:
[https://github.com/BurntSushi/imdb-
rename/](https://github.com/BurntSushi/imdb-rename/)

It's even faster and comes with its own built in evaluation. There's also no
more reliance on an external database.

