ChatGPT解决这个技术问题 Extra ChatGPT

How to swap text based on patterns at once with sed?

Suppose I have 'abbc' string and I want to replace:

ab -> bc

bc -> ab

If I try two replaces the result is not what I want:

echo 'abbc' | sed 's/ab/bc/g;s/bc/ab/g'
abab

So what sed command can I use to replace like below?

echo abbc | sed SED_COMMAND
bcab

EDIT: Actually the text could have more than 2 patterns and I don't know how many replaces I will need. Since there was a answer saying that sed is a stream editor and its replaces are greedily I think that I will need to use some script language for that.

Do you need to make multiple replacements on the same line? If not just drop the g flag from both of those s/// commands and that will work.
You missed the point of my question. I meant do you need to make each replacement more than once on the same line. Is there more than one match for ab or bc in the original input.
Sorry @EtanReisner i have misunderstood, The anwser is yes. the text can have multiple replacements.

o
ooga

Maybe something like this:

sed 's/ab/~~/g; s/bc/ab/g; s/~~/bc/g'

Replace ~ with a character that you know won't be in the string.


GNU sed handles nuls, so you can use \x0 for ~~.
Is g necessary and what does it do?
@Lee g is for global - it replaces all instances of the pattern in each line, instead of just the first (which is the default behaviour).
Please see my answer stackoverflow.com/a/41273117/539149 for a variation of ooga's answer that can replace multiple combinations simultaneously.
that you know won't be in the string For production code, do not never make any assumption about the input. For tests, well, tests never really prove correctness, but a good idea for a test is: Use the script itself as input.
J
Jean-François Corbett

I always use multiple statements with "-e"

$ sed -e 's:AND:\n&:g' -e 's:GROUP BY:\n&:g' -e 's:UNION:\n&:g' -e 's:FROM:\n&:g' file > readable.sql

This will append a '\n' before all AND's, GROUP BY's, UNION's and FROM's, whereas '&' means the matched string and '\n&' means you want to replace the matched string with an '\n' before the 'matched'


it returns sed: -e: No such file or directory
What if I was using sed -i -e?
This doesn't solve the main problem of the order of operations. Each command is run on the whole file only after the previous command has run. So running this: echo 'abbc' | sed -e 's:ab:bc:g' -e 's:bc:ab:g' still results in abab instead of bcab which is what the question is asking.
Yes, ADJenks, you are right! :) Maybe you could cheat this with: echo 'abbc' | sed -e 's:ab:xx:g' -e 's:bc:ab:g' -e 's:xx:bc:g'
@alper, it works. Perhaps there was only single -e specified. In such case, -e option should prefix every statement.
k
kuriouscoder

sed is a stream editor. It searches and replaces greedily. The only way to do what you asked for is using an intermediate substitution pattern and changing it back in the end.

echo 'abcd' | sed -e 's/ab/xy/;s/cd/ab/;s/xy/cd/'


C
Community

Here is a variation on ooga's answer that works for multiple search and replace pairs without having to check how values might be reused:

sed -i '
s/\bAB\b/________BC________/g
s/\bBC\b/________CD________/g
s/________//g
' path_to_your_files/*.txt

Here is an example:

before:

some text AB some more text "BC" and more text.

after:

some text BC some more text "CD" and more text.

Note that \b denotes word boundaries, which is what prevents the ________ from interfering with the search (I'm using GNU sed 4.2.2 on Ubuntu). If you are not using a word boundary search, then this technique may not work.

Also note that this gives the same results as removing the s/________//g and appending && sed -i 's/________//g' path_to_your_files/*.txt to the end of the command, but doesn't require specifying the path twice.

A general variation on this would be to use \x0 or _\x0_ in place of ________ if you know that no nulls appear in your files, as jthill suggested.


I agree with hagello's comment above about not making assumptions of what the input may contain. Therefore, I personally feel that this is the most reliable solution, aside from piping seds on top of each other (sed 's/ab/xy/' | sed 's/cd/ab/' .....)
o
ooga

This might work for you (GNU sed):

sed -r '1{x;s/^/:abbc:bcab/;x};G;s/^/\n/;:a;/\n\n/{P;d};s/\n(ab|bc)(.*\n.*:(\1)([^:]*))/\4\n\2/;ta;s/\n(.)/\1\n/;ta' file

This uses a lookup table which is prepared and held in the hold space (HS) and then appended to each line. An unique marker (in this case \n) is prepended to the start of the line and used as a method to bump-along the search throughout the length of the line. Once the marker reaches the end of the line the process is finished and is printed out the lookup table and markers being discarded.

N.B. The lookup table is prepped at the very start and a second unique marker (in this case :) chosen so as not to clash with the substitution strings.

With some comments:

sed -r '
  # initialize hold with :abbc:bcab
  1 {
    x
    s/^/:abbc:bcab/
    x
  }

  G        # append hold to patt (after a \n)

  s/^/\n/  # prepend a \n

  :a

  /\n\n/ {
    P      # print patt up to first \n
    d      # delete patt & start next cycle
  }

  s/\n(ab|bc)(.*\n.*:(\1)([^:]*))/\4\n\2/
  ta       # goto a if sub occurred

  s/\n(.)/\1\n/  # move one char past the first \n
  ta       # goto a if sub occurred
'

The table works like this:

   **   **   replacement
:abbc:bcab
 **   **     pattern

g
glenn jackman

Tcl has a builtin for this

$ tclsh
% string map {ab bc bc ab} abbc
bcab

This works by walking the string a character at a time doing string comparisons starting at the current position.

In perl:

perl -E '
    sub string_map {
        my ($str, %map) = @_;
        my $i = 0;
        while ($i < length $str) {
          KEYS:
            for my $key (keys %map) {
                if (substr($str, $i, length $key) eq $key) {
                    substr($str, $i, length $key) = $map{$key};
                    $i += length($map{$key}) - 1;
                    last KEYS;
                }
            }
            $i++;
        }
        return $str;
    }
    say string_map("abbc", "ab"=>"bc", "bc"=>"ab");
'
bcab

A
Alec Missine

Here is an excerpt from the SED manual:

-e script --expression=script Add the commands in script to the set of commands to be run while processing the input.

Prepend each substitution with -e option and collect them together. The example that works for me follows:

sed < ../.env-turret.dist \
  -e "s/{{ name }}/turret$TURRETS_COUNT_INIT/g" \
  -e "s/{{ account }}/$CFW_ACCOUNT_ID/g" > ./.env.dist

This example also shows how to use environment variables in your substitutions.


d
dst_91

May be a simpler approach for single pattern occurrence you can try as below: echo 'abbc' | sed 's/ab/bc/;s/bc/ab/2'

My output:

 ~# echo 'abbc' | sed 's/ab/bc/;s/bc/ab/2'
 bcab

For multiple occurrences of pattern:

sed 's/\(ab\)\(bc\)/\2\1/g'

Example

~# cat try.txt
abbc abbc abbc
bcab abbc bcab
abbc abbc bcab

~# sed 's/\(ab\)\(bc\)/\2\1/g' try.txt
bcab bcab bcab
bcab bcab bcab
bcab bcab bcab

Hope this helps !!


M
Martin Brisiak

If replacing the string by Variable, the solution doesn't work. The sed command need to be in double quotes instead on single quote.

#sed -e "s/#replacevarServiceName#/$varServiceName/g" -e "s/#replacevarImageTag#/$varImageTag/g" deployment.yaml

J
Jotne

Here is an awk based on oogas sed

echo 'abbc' | awk '{gsub(/ab/,"xy");gsub(/bc/,"ab");gsub(/xy/,"bc")}1'
bcab

S
Sandeepraj Singh

echo "C:\Users\San.Tan\My Folder\project1" | sed -e 's/C:\\/mnt\/c\//;s/\\/\//g'

replaces

C:\Users\San.Tan\My Folder\project1

to

mnt/c/Users/San.Tan/My Folder/project1

in case someone needs to replace windows paths to Windows Subsystem for Linux(WSL) paths


This has nothing to do with the posted question.
yes not directly. thats why i qualified it "in case" . If people are like me, not everyone is going to have A specific problem answered everytime they come searching on Stack overflow. But to your point, i have put this answer elsewhere. where the question was to change windows to Linux paths using sed. Thnx
You know you can post your own question and answer it as well. Having that specific question "How to change Windows paths to Linux" would make it helpful if people were really searching for that. People really in need of that answer are unlikely to find it here.
J
John P

I believe this should solve your problem. I may be missing a few edge cases, please comment if you notice one.

You need a way to exclude previous substitutions from future patterns, which really means making outputs distinguishable, as well as excluding these outputs from your searches, and finally making outputs indistinguishable again. This is very similar to the quoting/escaping process, so I'll draw from it.

s/\\/\\\\/g escapes all existing backslashes

s/ab/\\b\\c/g substitutes raw ab for escaped bc

s/bc/\\a\\b/g substitutes raw bc for escaped ab

s/\\\(.\)/\1/g substitutes all escaped X for raw X

I have not accounted for backslashes in ab or bc, but intuitively, I would escape the search and replace terms the same way - \ now matches \\, and substituted \\ will appear as \.

Until now I have been using backslashes as the escape character, but it's not necessarily the best choice. Almost any character should work, but be careful with the characters that need escaping in your environment, sed, etc. depending on how you intend to use the results.


S
Sean

Every answer posted thus far seems to agree with the statement by kuriouscoder made in his above post:

The only way to do what you asked for is using an intermediate substitution pattern and changing it back in the end

If you are going to do this, however, and your usage might involve more than some trivial string (maybe you are filtering data, etc.), the best character to use with sed is a newline. This is because since sed is 100% line-based, a newline is the one-and-only character you are guaranteed to never receive when a new line is fetched (forget about GNU multi-line extensions for this discussion).

To start with, here is a very simple approach to solving your problem using newlines as an intermediate delimiter:

echo "abbc" | sed -E $'s/ab|bc/\\\n&/g; s/\\nab/bc/g; s/\\nbc/ab/g'

With simplicity comes some trade-offs... if you had more than a couple variables, like in your original post, you have to type them all twice. Performance might be able to be improved a little bit, too.

It gets pretty nasty to do much beyond this using sed. Even with some of the more advanced features like branching control and the hold buffer (which is really weak IMO), your options are pretty limited.

Just for fun, I came up with this one alternative, but I don't think I would have any particular reason to recommend it over the one from earlier in this post... You have to essentially make your own "convention" for delimiters if you really want to do anything fancy in sed. This is way-overkill for your original post, but it might spark some ideas for people who come across this post and have more complicated situations.

My convention below was: use multiple newlines to "protect" or "unprotect" the part of the line you're working on. One newline denotes a word boundary. Two newlines denote alternatives for a candidate replacement. I don't replace right away, but rather list the candidate replacement on the next line. Three newlines means that a value is "locked-in", like your original post way trying to do with ab and bc. After that point, further replacements will be undone, because they are protected by the newlines. A little complicated if I don't say so myself... ! sed isn't really meant for much more than the basics.

# Newlines
NL=$'\\\n'
NOT_NL=$'[\x01-\x09\x0B-\x7F]'

# Delimiters
PRE="${NL}${NL}&${NL}"
POST="${NL}${NL}"

# Un-doer (if a request was made to modify a locked-in value)
tidy="s/(\\n\\n\\n${NOT_NL}*)\\n\\n(${NOT_NL}*)\\n(${NOT_NL}*)\\n\\n/\\1\\2/g; "

# Locker-inner (three newlines means "do not touch")
tidy+="s/(\\n\\n)${NOT_NL}*\\n(${NOT_NL}*\\n\\n)/\\1${NL}\\2/g;"

# Finalizer (remove newlines)
final="s/\\n//g"

# Input/Commands
input="abbc"
cmd1="s/(ab)/${PRE}bc${POST}/g"
cmd2="s/(bc)/${PRE}ab${POST}/g"

# Execute
echo ${input} | sed -E "${cmd1}; ${tidy}; ${cmd2}; ${tidy}; ${final}"