I need to hide all permission denied messages from:
find . > files_and_folders
I am experimenting when such message arises. I need to gather all folders and files, to which it does not arise.
Is it possible to direct the permission levels to the files_and_folders
file?
How can I hide the errors at the same time?
find /. -name 'toBeSearched.file' 2>/dev/null
.
/proc
path using the -path
option. It helps to negate the -prune
option to avoid printing pruned items.
Use:
find . 2>/dev/null > files_and_folders
This hides not just the Permission denied
errors, of course, but all error messages.
If you really want to keep other possible errors, such as too many hops on a symlink, but not the permission denied ones, then you'd probably have to take a flying guess that you don't have many files called 'permission denied' and try:
find . 2>&1 | grep -v 'Permission denied' > files_and_folders
If you strictly want to filter just standard error, you can use the more elaborate construction:
find . 2>&1 > files_and_folders | grep -v 'Permission denied' >&2
The I/O redirection on the find
command is: 2>&1 > files_and_folders |
. The pipe redirects standard output to the grep
command and is applied first. The 2>&1
sends standard error to the same place as standard output (the pipe). The > files_and_folders
sends standard output (but not standard error) to a file. The net result is that messages written to standard error are sent down the pipe and the regular output of find
is written to the file. The grep
filters the standard output (you can decide how selective you want it to be, and may have to change the spelling depending on locale and O/S) and the final >&2
means that the surviving error messages (written to standard output) go to standard error once more. The final redirection could be regarded as optional at the terminal, but would be a very good idea to use it in a script so that error messages appear on standard error.
There are endless variations on this theme, depending on what you want to do. This will work on any variant of Unix with any Bourne shell derivative (Bash, Korn, …) and any POSIX-compliant version of find
.
If you wish to adapt to the specific version of find
you have on your system, there may be alternative options available. GNU find
in particular has a myriad options not available in other versions — see the currently accepted answer for one such set of options.
Note:
This answer probably goes deeper than the use case warrants, and find 2>/dev/null may be good enough in many situations. It may still be of interest for a cross-platform perspective and for its discussion of some advanced shell techniques in the interest of finding a solution that is as robust as possible, even though the cases guarded against may be largely hypothetical.
If your shell is bash
or zsh
, there's a solution that is robust while being reasonably simple, using only POSIX-compliant find
features; while bash
itself is not part of POSIX, most modern Unix platforms come with it, making this solution widely portable:
find . > files_and_folders 2> >(grep -v 'Permission denied' >&2)
Note:
If your system is configured to show localized error messages, prefix the find calls below with LC_ALL=C (LC_ALL=C find ...) to ensure that English messages are reported, so that grep -v 'Permission denied' works as intended. Invariably, however, any error messages that do get displayed will then be in English as well.
>(...) is a (rarely used) output process substitution that allows redirecting output (in this case, stderr output (2>) to the stdin of the command inside >(...). In addition to bash and zsh, ksh supports them as well in principle, but trying to combine them with redirection from stderr, as is done here (2> >(...)), appears to be silently ignored (in ksh 93u+). grep -v 'Permission denied' filters out (-v) all lines (from the find command's stderr stream) that contain the phrase Permission denied and outputs the remaining lines to stderr (>&2). Note: There's a small chance that some of grep's output may arrive after find completes, because the overall command doesn't wait for the command inside >(...) to finish. In bash, you can prevent this by appending | cat to the command.
grep -v 'Permission denied' filters out (-v) all lines (from the find command's stderr stream) that contain the phrase Permission denied and outputs the remaining lines to stderr (>&2).
Note: There's a small chance that some of grep's output may arrive after find completes, because the overall command doesn't wait for the command inside >(...) to finish. In bash, you can prevent this by appending | cat to the command.
This approach is:
robust: grep is only applied to error messages (and not to a combination of file paths and error messages, potentially leading to false positives), and error messages other than permission-denied ones are passed through, to stderr.
side-effect free: find's exit code is preserved: the inability to access at least one of the filesystem items encountered results in exit code 1 (although that won't tell you whether errors other than permission-denied ones occurred (too)).
POSIX-compliant solutions:
Fully POSIX-compliant solutions either have limitations or require additional work.
If find
's output is to be captured in a file anyway (or suppressed altogether), then the pipeline-based solution from Jonathan Leffler's answer is simple, robust, and POSIX-compliant:
find . 2>&1 >files_and_folders | grep -v 'Permission denied' >&2
Note that the order of the redirections matters: 2>&1
must come first.
Capturing stdout output in a file up front allows 2>&1
to send only error messages through the pipeline, which grep
can then unambiguously operate on.
The only downside is that the overall exit code will be the grep
command's, not find
's, which in this case means: if there are no errors at all or only permission-denied errors, the exit code will be 1
(signaling failure), otherwise (errors other than permission-denied ones) 0
- which is the opposite of the intent.
That said, find
's exit code is rarely used anyway, as it often conveys little information beyond fundamental failure such as passing a non-existent path.
However, the specific case of even only some of the input paths being inaccessible due to lack of permissions is reflected in find
's exit code (in both GNU and BSD find
): if a permissions-denied error occurs for any of the files processed, the exit code is set to 1
.
The following variation addresses that:
find . 2>&1 >files_and_folders | { grep -v 'Permission denied' >&2; [ $? -eq 1 ]; }
Now, the exit code indicates whether any errors other than Permission denied
occurred: 1
if so, 0
otherwise.
In other words: the exit code now reflects the true intent of the command: success (0
) is reported, if no errors at all or only permission-denied errors occurred.
This is arguably even better than just passing find
's exit code through, as in the solution at the top.
gniourf_gniourf in the comments proposes a (still POSIX-compliant) generalization of this solution using sophisticated redirections, which works even with the default behavior of printing the file paths to stdout:
{ find . 3>&2 2>&1 1>&3 | grep -v 'Permission denied' >&3; } 3>&2 2>&1
In short: Custom file descriptor 3
is used to temporarily swap stdout (1
) and stderr (2
), so that error messages alone can be piped to grep
via stdout.
Without these redirections, both data (file paths) and error messages would be piped to grep
via stdout, and grep
would then not be able to distinguish between error message Permission denied
and a (hypothetical) file whose name happens to contain the phrase Permission denied
.
As in the first solution, however, the the exit code reported will be grep
's, not find
's, but the same fix as above can be applied.
Notes on the existing answers:
There are several points to note about Michael Brux's answer, find . ! -readable -prune -o -print: It requires GNU find; notably, it won't work on macOS. Of course, if you only ever need the command to work with GNU find, this won't be a problem for you. Some Permission denied errors may still surface: find ! -readable -prune reports such errors for the child items of directories for which the current user does have r permission, but lacks x (executable) permission. The reason is that because the directory itself is readable, -prune is not executed, and the attempt to descend into that directory then triggers the error messages. That said, the typical case is for the r permission to be missing. Note: The following point is a matter of philosophy and/or specific use case, and you may decide it is not relevant to you and that the command fits your needs well, especially if simply printing the paths is all you do: If you conceptualize the filtering of the permission-denied error messages a separate task that you want to be able to apply to any find command, then the opposite approach of proactively preventing permission-denied errors requires introducing "noise" into the find command, which also introduces complexity and logical pitfalls. For instance, the most up-voted comment on Michael's answer (as of this writing) attempts to show how to extend the command by including a -name filter, as follows: find . ! -readable -prune -o -name '*.txt' This, however, does not work as intended, because the trailing -print action is required (an explanation can be found in this answer). Such subtleties can introduce bugs.
It requires GNU find; notably, it won't work on macOS. Of course, if you only ever need the command to work with GNU find, this won't be a problem for you.
Some Permission denied errors may still surface: find ! -readable -prune reports such errors for the child items of directories for which the current user does have r permission, but lacks x (executable) permission. The reason is that because the directory itself is readable, -prune is not executed, and the attempt to descend into that directory then triggers the error messages. That said, the typical case is for the r permission to be missing.
Note: The following point is a matter of philosophy and/or specific use case, and you may decide it is not relevant to you and that the command fits your needs well, especially if simply printing the paths is all you do: If you conceptualize the filtering of the permission-denied error messages a separate task that you want to be able to apply to any find command, then the opposite approach of proactively preventing permission-denied errors requires introducing "noise" into the find command, which also introduces complexity and logical pitfalls. For instance, the most up-voted comment on Michael's answer (as of this writing) attempts to show how to extend the command by including a -name filter, as follows: find . ! -readable -prune -o -name '*.txt' This, however, does not work as intended, because the trailing -print action is required (an explanation can be found in this answer). Such subtleties can introduce bugs.
If you conceptualize the filtering of the permission-denied error messages a separate task that you want to be able to apply to any find command, then the opposite approach of proactively preventing permission-denied errors requires introducing "noise" into the find command, which also introduces complexity and logical pitfalls.
For instance, the most up-voted comment on Michael's answer (as of this writing) attempts to show how to extend the command by including a -name filter, as follows: find . ! -readable -prune -o -name '*.txt' This, however, does not work as intended, because the trailing -print action is required (an explanation can be found in this answer). Such subtleties can introduce bugs.
The first solution in Jonathan Leffler's answer, find . 2>/dev/null > files_and_folders, as he himself states, blindly silences all error messages (and the workaround is cumbersome and not fully robust, as he also explains). Pragmatically speaking, however, it is the simplest solution, as you may be content to assume that any and all errors would be permission-related.
mist's answer, sudo find . > files_and_folders, is concise and pragmatic, but ill-advised for anything other than merely printing filenames, for security reasons: because you're running as the root user, "you risk having your whole system being messed up by a bug in find or a malicious version, or an incorrect invocation which writes something unexpectedly, which could not happen if you ran this with normal privileges" (from a comment on mist's answer by tripleee).
The 2nd solution in viraptor's answer, find . 2>&1 | grep -v 'Permission denied' > some_file runs the risk of false positives (due to sending a mix of stdout and stderr through the pipeline), and, potentially, instead of reporting non-permission-denied errors via stderr, captures them alongside the output paths in the output file.
find . 2>&1 > files_and_folders | grep -v 'Permission denied' >&2
?
{ find . 3>&2 2>&1 1>&3 | grep -v 'Permission denied' >&3; } 3>&2 2>&1
>(...)
are Bash-specific.
find
should be emphasized and advertised: find
's exit code is notoriously useless. Here, it's very likely going to be non-zero (and uselessly so).
execute/search
file-mode permission in order to 'search' a directory (retrieve the inodes of contained files). find
does this in order to descend into a subdirectory (in addition to requiring read
permission to list the files in a directory). This is not a 'bug' or 'porting error'.
Use:
find . ! -readable -prune -o -print
or more generally
find <paths> ! -readable -prune -o <other conditions like -name> -print
to avoid "Permission denied"
AND do NOT suppress (other) error messages
AND get exit status 0 ("all files are processed successfully")
Works with: find (GNU findutils) 4.4.2. Background:
The -readable test matches readable files. The ! operator returns true, when test is false. And ! -readable matches not readable directories (&files).
The -prune action does not descend into directory.
! -readable -prune can be translated to: if directory is not readable, do not descend into it.
The -readable test takes into account access control lists and other permissions artefacts which the -perm test ignores.
See also find
(1) manpage for many more details.
-o
: find . ! -readable -prune -o -name '*.txt'
find
does not include -readable
as an option; neither does the find
for BSD and hence Mac OS X (I'm not sure about other systems). So, where you have GNU find
guaranteed, this works great, but it isn't obvious how to adapt this if you can't guarantee that the system has GNU find
installed. (It'll work fine on Linux; it may or may not work elsewhere.)
find . ! -readable -prune -o -name '*.txt'
doen't seem to work on Ubuntu 14.04 using find 4.2.2. It seems to ingore the -name
. For some weird reason I has success with find . \( ! -readable -prune \) -o -name '*.txt' -print
If you want to start search from root "/" , you will probably see output somethings like:
find: /./proc/1731/fdinfo: Permission denied
find: /./proc/2032/task/2032/fd: Permission denied
It's because of permission. To solve this:
You can use sudo command: sudo find /. -name 'toBeSearched.file'
It asks super user's password, when enter the password you will see result what you really want. If you don't have permission to use sudo command which means you don't have super user's password, first ask system admin to add you to the sudoers file.
You can use redirect the Standard Error Output from (Generally Display/Screen) to some file and avoid seeing the error messages on the screen! redirect to a special file /dev/null : find /. -name 'toBeSearched.file' 2>/dev/null You can use redirect the Standard Error Output from (Generally Display/Screen) to Standard output (Generally Display/Screen), then pipe with grep command with -v "invert" parameter to not to see the output lines which has 'Permission denied' word pairs: find /. -name 'toBeSearched.file' 2>&1 | grep -v 'Permission denied'
sudo find...
I had to use:
find / -name expect 2>/dev/null
specifying the name of what I wanted to find and then telling it to redirect all errors to /dev/null
expect being the location of the expect program I was searching for.
expect
. Instead, expect
is simply the name of the file that this command will try to find.
Pipe stderr
to /dev/null
by using 2>/dev/null
find . -name '...' 2>/dev/null
find . -name '...' -print 2>/dev/null
You can also use the -perm
and -prune
predicates to avoid descending into unreadable directories (see also How do I remove "permission denied" printout statements from the find program? - Unix & Linux Stack Exchange):
find . -type d ! -perm -g+r,u+r,o+r -prune -o -print > files_and_folders
-perm -g+r,u+r,o+r
merely matches files that have the r
(read) permission set for all 3 of the file's security principals, which has no direct relationship with whether the current user can read that file or not. It has the potential to both miss files that the current user can read and to match files that they cannot.
find . -type d ! \( -perm -u+r -o -perm -g+r -o -perm -o+r \) -prune -o -print
would be the good solution.
-readable
with -perm
- see my previous comment and consider this example: echo 'hi' > file; sudo chown nobody:nobody file; sudo chmod o-r file; find file -perm -u=r
prints file
, because its user read bit is set, but it relates to the the nobody
user, not the current user. The current user cannot read this file; try cat file
. See also: this answer of mine.
Redirect standard error. For instance, if you're using bash on a unix machine, you can redirect standard error to /dev/null like this:
find . 2>/dev/null >files_and_folders
While above approaches do not address the case for Mac OS X because Mac Os X does not support -readable
switch this is how you can avoid 'Permission denied' errors in your output. This might help someone.
find / -type f -name "your_pattern" 2>/dev/null
.
If you're using some other command with find
, for example, to find the size of files of certain pattern in a directory 2>/dev/null
would still work as shown below.
find . -type f -name "your_pattern" -exec du -ch {} + 2>/dev/null | grep total$
.
This will return the total size of the files of a given pattern. Note the 2>/dev/null
at the end of find command.
2>/dev/null
. Can you please explain the part -exec du -ch {} + 2>/dev/null | grep total$
.
-exec
option to take further action on the files or directories found by find
command. du -ch file_pattern
calculates the size of each file matching file_pattern
and last line of that output is the grand total of all files that matched the file_pattern
. See man page for du
. grep total
just filters the line that extracts the grand total(which is the last line).
Those errors are printed out to the standard error output (fd 2). To filter them out, simply redirect all errors to /dev/null:
find . 2>/dev/null > some_file
or first join stderr and stdout and then grep out those specific errors:
find . 2>&1 | grep -v 'Permission denied' > some_file
Simple answer:
find . > files_and_folders 2>&-
2>&-
closes (-
) the standard error file descriptor (2
) so all error messages are silenced.
Exit code will still be 1 if any 'Permission denied' errors would otherwise be printed
Robust answer for GNU find:
find . -type d \! \( -readable -executable \) -prune -print -o -print > files_and_folders
Pass extra options to find
that -prune
(prevent descending into) but still -print
any directory (-type
d
) that does not (\!
) have both -readable
and -executable
permissions, or (-o
) -print
any other file.
-readable and -executable options are GNU extensions, not part of the POSIX standard
May still return 'Permission denied' on abnormal/corrupt files (e.g., see bug report affecting container-mounted filesystems using lxcfs < v2.0.5)
Robust answer that works with any POSIX-compatible find (GNU, OSX/BSD, etc)
{ LC_ALL=C find . 3>&2 2>&1 1>&3 > files_and_folders | grep -v 'Permission denied'; [ $? = 1 ]; } 3>&2 2>&1
Use a pipeline to pass the standard error stream to grep
, removing all lines containing the 'Permission denied'
string.
LC_ALL=C
sets the POSIX locale using an environment variable, 3>&2 2>&1 1>&3
and 3>&2 2>&1
duplicate file descriptors to pipe the standard-error stream to grep
, and [ $? = 1 ]
uses []
to invert the error code returned by grep
to approximate the original behavior of find
.
Will also filter any 'Permission denied' errors due to output redirection (e.g., if the files_and_folders file itself is not writable)
-perm
-based solution is not worth presenting, because it more fundamentally than the quote suggests does something different than intended: it is a purely file-centric test, relating to the file's owner and group, neither of which have any guaranteed relationship with the user calling the command (see this answer of mine. It looks like your revised GNU solution now doesn't catch permission denied errors stemming from files.
echo 'hi' > file; sudo chown nobody:nobody file; sudo chmod o-r file; find file -perm -u=r
prints file
, because its user read bit is set, but it relates to the the nobody
user, not the current user. The current user cannot read this file; try cat file
.
-perm
doesn't work for determining the user's current permissions. Removed that alternative from this answer.
To avoid just the permission denied warnings, tell find to ignore the unreadable files by pruning them from the search. Add an expression as an OR to your find, such as
find / \! -readable -prune -o -name '*.jbd' -ls
This mostly says to (match an unreadable file and prune it from the list) OR (match a name like *.jbd and display it [with ls]). (Remember that by default the expressions are AND'd together unless you use -or.) You need the -ls in the second expression or else find may add a default action to show either match, which will also show you all the unreadable files.
But if you're looking for real files on your system, there is usually no reason to look in /dev, which has many many files, so you should add an expression that excludes that directory, like:
find / -mount \! -readable -prune -o -path /dev -prune -o -name '*.jbd' -ls
So (match unreadable file and prune from list) OR (match path /dev and prune from list) OR (match file like *.jbd and display it).
use
sudo find / -name file.txt
It's stupid (because you elevate the search) and nonsecure, but far shorter to write.
sudo
. You risk having your whole system being messed up by a bug in find
or a malicious version, or an incorrect invocation which writes something unexpectedly, which could not happen if you ran this with normal privileges.
None of the above answers worked for me. Whatever I find on Internet focuses on: hide errors. None properly handles the process return-code / exit-code. I use command find within bash scripts to locate some directories and then inspect their content. I evaluate command find success using the exit-code: a value zero works, otherwise fails.
The answer provided above by Michael Brux works sometimes. But I have one scenario in which it fails! I discovered the problem and fixed it myself. I need to prune files when:
it is a directory AND has no read access AND/OR has no execute access
See the key issue here is: AND/OR. One good suggested condition sequence I read is:
-type d ! -readable ! -executable -prune
This does not work always. This means a prune is triggered when a match is:
it is directory AND no read access AND no execute access
This sequence of expressions fails when read access is granted but no execute access is.
After some testing I realized about that and changed my shell script solution to:
nice find /home*/ -maxdepth 5 -follow \ \( -type d -a ! \( -readable -a -executable \) \) -prune \ -o \ \( -type d -a -readable -a -executable -a -name "${m_find_name}" \) -print
The key here is to place the "not true" for a combined expression:
has read access AND has execute access
Otherwise it has not full access, which means: prune it. This proved to work for me in one scenario which previous suggested solutions failed.
I provide below technical details for questions in the comments section. I apologize if details are excessive.
¿Why using command nice? I got the idea here. Initially I thought it would be nice to reduce process priority when looking an entire filesystem. I realized it makes no sense to me, as my script is limited to few directories. I reduced -maxdepth to 3.
¿Why search within /home*/? This it not relevant for this thread. I install all applications by hand via source code compile with non privileged users (not root). They are installed within "/home". I can have multiple binaries and versions living together. I need to locate all directories, inspect and backup in a master-slave fashion. I can have more than one "/home" (several disks running within a dedicated server).
¿Why using -follow? Users might create symbolic links to directories. It's usefulness depends, I need to keep record of the absolute paths found.
nice
and find $HOME -maxdepth 5 -follow ...
?
${m_find_name}
), and contains several options not relevant to the question (nice
, /home*
, -maxdepth 5
, -follow
). I've added an answer that addresses the specific issue of 'filtering readable but not executable directories' more concisely, while remaining general-purpose.
You can use the grep -v invert-match
-v, --invert-match select non-matching lines
like this:
find . > files_and_folders
cat files_and_folders | grep -v "permission denied" > files_and_folders
Should to the magic
-=For MacOS=-
Make a new command using alias: just add in ~/.bash_profile line:
alias search='find / -name $file 2>/dev/null'
and in new Terminal window you can call it:
$ file=<filename or mask>; search
for example: $ file=etc; search
If you are using CSH or TCSH, here is a solution:
( find . > files_and_folders ) >& /dev/null
If you want output to the terminal:
( find . > /dev/tty ) >& /dev/null
However, as the "csh-whynot" FAQ describes, you should not use CSH.
Simply use this to search for a file in your system.
find / -name YOUR_SEARCH_TERM 2>&1 | grep YOUR_SEARCH_TERM
Let's not do unnecessary over engineering, you just want to search for your file right? then that is the command which will list the files for you if they are present in an area accessible to you.
Optimized solutions for GNU find†
At least for some system+filesystem combinations, find
doesn't need to stat
a file to get its type. Then you can check if it's a directory before testing readability to speed up the search‡ —I've got some 30% improvement in tests I did. So for long searches or searches that run often enough, use one of these:
Print everything visible
$ find . -print -type d ! -readable -prune
$ find . -type d ! -readable -prune , [expression] -print
Print visible files
$ find . -type d \( ! -readable -prune -o -true \) -o [expression] -print
Print visible directories
$ find . -type d -print ! -readable -prune
$ find . -type d \( ! -readable -prune , [expression] -print \)
Print only readable directories
$ find . -type d ! -readable -prune -o [expression] -print
Notes
† The -readable
and ,
(comma) operators are GNU extensions. This expression
$ find . [expression] , [expression]
is logically equivalent to
$ find . \( [expression] -o -true \) [expression]
‡ This is because find
implementations with this optimization enabled will avoid stat
ing non-directory files at all in the discussed use case.
Edit: shell function
Here is a POSIX shell function I ended up with to prepend this test to any expression. It seems to work fine with the implicit -print
and command-line options:
findr () {
j=$#; done=
while [ $j -gt 0 ]; do
j=$(($j - 1))
arg="$1"; shift
test "$done" || case "$arg" in
-[A-Z]*) ;; # skip options
-*|\(|!) # find start of expression
set -- "$@" \( -type d ! -readable -prune -o -true \)
done=true
;;
esac
set -- "$@" "$arg"
done
find "$@"
}
The other two alternatives listed in the answers caused either a syntax error in POSIX shell (couldn't even source a file containing the function definition) or bad output in ZSH... Running time seems to be equivalent.
To search the entire file system for some file, e.g. hosts
, except the /proc
tree, which causes all kinds of errors, I use the following:
# find / -path /proc ! -prune -o -name hosts -type f
/etc/hosts
Note: Because -prune
is always true, you have to negate it to avoid seeing the line /proc
in the output. I tried the using ! -readable
approach and found it returns all kinds of things under /proc that the current user can read. So the "OR" condition doesn't do what you expect/want there.
I started with the example given by the find man page, See the -prune
option.
Minimal solution is just to add the readable
flag.
find . -name foo -readable
Success story sharing
2>/dev/null
, without space!2>
is a single unit without any spaces; you can have a space between it and the file name. Similarly with other redirections, such as2>&1
(which redirects standard error to the same place as standard output is going), or2>&-
which closes standard error, etc. See Redirections for the remaining gory details. (The code above is generic POSIX-like shell, not specific tobash
.)/dev/null
; the second does not. And the answer mentions the assumption that you don't have any files calledpermission denied
. So, I'm not clear what you're really objecting to.