Is there an invocation of sed
todo in-place editing without backups that works both on Linux and Mac? While the BSD sed
shipped with OS X seems to need sed -i '' …
, the GNU sed
Linux distributions usually come with interprets the quotes as empty input file name (instead of the backup extension), and needs sed -i …
instead.
Is there any command line syntax which works with both flavors, so I can use the same script on both systems?
If you really want to just use sed -i
the 'easy' way, the following DOES work on both GNU and BSD/Mac sed
:
sed -i.bak 's/foo/bar/' filename
Note the lack of space and the dot.
Proof:
# GNU sed
% sed --version | head -1
GNU sed version 4.2.1
% echo 'foo' > file
% sed -i.bak 's/foo/bar/' ./file
% ls
file file.bak
% cat ./file
bar
# BSD sed
% sed --version 2>&1 | head -1
sed: illegal option -- -
% echo 'foo' > file
% sed -i.bak 's/foo/bar/' ./file
% ls
file file.bak
% cat ./file
bar
Obviously you could then just delete the .bak
files.
This works with GNU sed, but not on OS X:
sed -i -e 's/foo/bar/' target.file
sed -i'' -e 's/foo/bar/' target.file
This works on OS X, but not with GNU sed:
sed -i '' -e 's/foo/bar/' target.file
On OS X you
can't use sed -i -e since the extension of the backup file would be set to -e
can't use sed -i'' -e for the same reasons—it needs a space between -i and ''.
-i
and -i''
are identical at shell parsing time; sed
can’t behave differently on those two first invocations, because it gets the exact same arguments.
When on OSX, I always install GNU sed version via Homebrew, to avoid problems in scripts, because most scripts were written for GNU sed versions.
brew install gnu-sed --with-default-names
Then your BSD sed will be replaced by GNU sed.
Alternatively, you can install without default-names, but then:
Change your PATH as instructed after installing gnu-sed
Do check in your scripts to chose between gsed or sed depending on your system
--with-default-names
was removed from homebrew-core, more info in this answer. when installing gnu-sed
now, the installation instructions specify that you need to add gnubin
to your PATH
: PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH"
bashrc
if you want them in your path before Mac tools of the same name: PATH="$(brew --prefix)/opt/coreutils/libexec/gnubin:$PATH"
As Noufal Ibrahim asks, why can't you use Perl? Any Mac will have Perl, and there are very few Linux or BSD distributions that don't include some version of Perl in the base system. One of the only environments that might actually lack Perl would be BusyBox (which works like GNU/Linux for -i
, except that no backup extension can be specified).
As ismail recommends,
Since perl is available everywhere I just do perl -pi -e s,foo,bar,g target.file
and this seems like a better solution in almost any case than scripts, aliases, or other workarounds to deal with the fundamental incompatibility of sed -i
between GNU/Linux and BSD/Mac.
perl
is part of the Linux Standard Base specification since May 1, 2009. refspecs.linuxfoundation.org/LSB_4.0.0/LSB-Languages/…
There is no way to have it working.
One way is to use a temporary file like:
TMP_FILE=`mktemp /tmp/config.XXXXXXXXXX`
sed -e "s/abc/def/" some/file > $TMP_FILE
mv $TMP_FILE some/file
This works on both
Answer: No.
The originally accepted answer actually doesn't do what is requested (as noted in the comments). (I found this answer when looking for the reason a file-e
was appearing "randomly" in my directories.)
There is apparently no way of getting sed -i
to work consistently on both MacOS and Linuces.
My recommendation, for what it is worth, is not to update-in-place with sed
(which has complex failure modes), but to generate new files and rename them afterwards. In other words: avoid -i
.
Here's another version that works on Linux and macOS without using eval
and without having to delete backup files. It uses Bash arrays for storing the sed
parameters, which is cleaner than using eval
:
# Default case for Linux sed, just use "-i"
sedi=(-i)
case "$(uname)" in
# For macOS, use two parameters
Darwin*) sedi=(-i "")
esac
# Expand the parameters in the actual call to "sed"
sed "${sedi[@]}" -e 's/foo/bar/' target.file
This does not create a backup file, neither a file with appended quotes.
sedi=(-i) && [ "$(uname)" == "Darwin" ] && sedi=(-i '')
sed "${sedi[@]}" -e 's/foo/bar/' target.file
The -i
option is not part of POSIX Sed. A more portable method would be to use Vim in Ex mode:
ex -sc '%s/alfa/bravo/|x' file
% select all lines s replace x save and close
Steve Powell's answer is quite correct, consulting the MAN page for sed on OSX and Linux (Ubuntu 12.04) highlights the in-compatibility within 'in-place' sed usage across the two operating systems.
JFYI, there should be no space between the -i and any quotes (which denote an empty file extension) using the Linux version of sed, thus
#Linux
sed -i""
and
#OSX (notice the space after the '-i' argument)
sed -i ""
I got round this in a script by using an alias'd command and the OS-name output of 'uname' within a bash 'if'. Trying to store OS-dependant command strings in variables was hit and miss when interpreting the quotes. The use of 'shopt -s expand_aliases' is necessary in order to expand/use the aliases defined within your script. shopt's usage is dealt with here.
Portable script for both GNU systems and OSX:
if [[ $(uname) == "Darwin" ]]; then
SP=" " # Needed for portability with sed
fi
sed -i${SP}'' -e "s/foo/bar/g" -e "s/ping/pong/g" foobar.txt
"Darwin"
-- it can't parse to anything but itself. Whereas if it weren't for [[ ]]
's special semantics, you would need quotes around $(uname)
to ensure correct behavior with all possible return values.
I ran into this problem. The only quick solution was to replace the sed in mac to the gnu version:
brew install gnu-sed
If you need to do sed
in-place in a bash
script, and you do NOT want the in-place to result with .bkp files, and you have a way to detect the os (say, using ostype.sh), -- then the following hack with the bash
shell built-in eval
should work:
OSTYPE="$(bash ostype.sh)"
cat > myfile.txt <<"EOF"
1111
2222
EOF
if [ "$OSTYPE" == "osx" ]; then
ISED='-i ""'
else # $OSTYPE == linux64
ISED='-i""'
fi
eval sed $ISED 's/2222/bbbb/g' myfile.txt
ls
# GNU and OSX: still only myfile.txt there
cat myfile.txt
# GNU and OSX: both print:
# 1111
# bbbb
# NOTE:
# if you just use `sed $ISED 's/2222/bbbb/g' myfile.txt` without `eval`,
# then you will get a backup file with quotations in the file name,
# - that is, `myfile.txt""`
You can use sponge. Sponge is an old unix program, found in moreutils package (both in ubuntu and probably debian, and in homebrew in mac).
It will buffer all the content from the pipe, wait until the pipe is close (probably meaning that the input file is already close) and then overwrite:
From the man page:
Synopsis sed '...' file | grep '...' | sponge file
The following works for me on Linux and OS X:
sed -i' ' <expr> <file>
e.g. for a file f
containing aaabbaaba
sed -i' ' 's/b/c/g' f
yields aaaccaaca
on both Linux and Mac. Note there is a quoted string containing a space, with no space between the -i
and the string. Single or double quotes both work.
On Linux I am using bash
version 4.3.11 under Ubuntu 14.04.4 and on the Mac version 3.2.57 under OS X 10.11.4 El Capitan (Darwin 15.4.0).
Success story sharing
sed
)sed
does not accept a positionalcommand
argument when invoked with-i
— it must be supplied with another flag,-e
. Thus, the command becomes:sed -i.bak -e 's/foo/bar/' filename
sed -i.bak 's/foo/bar/' file && rm file.bak