Find files containing a string on the whole filesystem

findgrepxargs

I need to find all the instances of a given string in the whole filesystem, because I don't remember in which configuration files, script or any other programs I put it and I need to update that string with a new one.

I tried with the following command

`grep -nr 'needle' / –exclude-dir=.svn | mail myaddress@example.com -s 'References on xxx'

If I run this command on a small directory it gives me the output I need in the form

/path1/:nn:line containing needle
/path2/:nn:line containing needle

where /path1 is the full path of the file, nn is the row containing the needle and last field is the content of the line.

However when I run the command on the root directory the grep process hang after a while. I run this script about 8 hours ago and even on a small filesystem (less than 5GB) it doesn't end and if I run top or ps the process seems sleeping

root 24909 0.0 0.1 3772 1520 pts/1 S+ Feb10 0:15 grep -nr needle / --exclude-dir=.svn

Why it doesn't end? Is there any better way to do this (it's a one time job, I don't need to execute this more than once)

Thanks.

Update: I found a working solution with find and xargs which seems to be working and uses less system resources than the find -exec solution. Here is my final command line:

find /{boot,etc,home,lib,lost+found,opt,root,sbin,usr,var} -type f -print0 | xargs -r0 grep -nr 'needle' | mail myaddress@example.com -s 'References on xxx'

I used the /{dir1,dir2,...,dirn} syntax because I want the output lines with the fullpath including the leading slash, and in this way you can use the command without cd / i.e. from any directory.

Best Answer

There are some files in the file system that aren't real files, but are instead hooks into the kernel. Some of those can be read from forever. Try

grep foo /dev/zero

to see this in action. Get ready to stop it with ctrl-C before it takes over the whole system.

If I wanted to do what you're doing, I'd enumerate the subdirectories of / that I wanted scanned, and make sure I only checked text files, probably with

cd /
find boot etc home lib lost+found media mnt opt root sbin tmp usr var -type f -exec grep needle {} /dev/null \;

Note how the list does not contain /dev, /proc, /sys or /selinux.

Related Solutions

How to master the UNIX find command

Well, as far as the -exec syntax goes, you could do like a lot of people, give up and use xargs:

find . -type f | xargs chown username

(or the files-with-spaces-and-other-nonsense-in-them-safe version)

find . -type f -print0 | xargs -0 chown username

Or, to try to remember the right thing to do with the semicolon, what you need to drill into your head is that you're using a semicolon to terminate the command that -exec is running, and you have to escape the semicolon because it has special meaning to bash. Which is why it's backslash semicolon. You seem to have the {} substitution part okay.

As to killing files and so on, if you're running something big and dangerous like you're talking about, first do this:

find . -type f -exec echo chown username {} \;

and review the results. This is basically a "dry run" where you're seeing the commands it would run if you let it. Definitely a good practice. Won't help with the .* problem, but you know not to do that one now. :)

Sed can’t find a file that obviously exists

If the file is a symlink pointing to a non-existant file, that's the error you'll get.

Best Answer

Related Solutions

How to master the UNIX find command

Sed can’t find a file that obviously exists

Related Topic