Home > Linux router > How to do line substitution in files for linux routers?

How to do line substitution in files for linux routers?

In many situations, this is a trivial task: create a simple regex script in Perl (or Python), and execute it. However, with linux routers it becomes a bit tricky. Usually by default these routers do not have high-level scripting languages installed due to limited resources. It is possible to add them by extending the diskspace through external means (like USB or SD card), but generally unless the user needs to use scripts extensively it isn’t necessary. There are several means to do substitution here.

Method #1: sed substitution.

Sed substitution is the easiest method to do line substitution, and the most recommended method. However, the cryptic syntax is not very user-friendly, especially those unfamiliar to sed (yet familiar to generic regex). To do a simple  substitution, use the following syntax:

# Syntax to do substitution in the file.
> sed -i 's/text1/text2/' file

This will take all instances of ‘text1’ and replaces it with ‘text2’ in the file. Note that the ‘-i’ means that it will change the file directly instead of printing the changes to stdout. This only does pure substitution, not line substitution. To do that, we need to use the ‘c\’ option instead of the ‘s’ option at the beginning of the string. See below:

# Syntax to do line substitution in the file, trying to match text1 with text2.
> sed -i '/text1/ c\ text2' file

Note that the ‘s’ in the beginning of the string is no longer there. There are actually several options in this same format. ‘i\’ will insert the line before the matched text1, and ‘a\’ will append the line after the matched text1.

In a practical example, openWRT tends to not setup the firewall (iptables) properly. When using the UI, it tends to comment out the option to include firewall.user. So I used this simple sed statement to uncomment the option whenever the firewall is changed:

# Uncomment out the etc/firewall.user
> sed -i '/option.*\/etc\/firewall.user/ c\ option path /etc/firewall.user' /etc/config/firewall

This will make:

/etc/config/firewall

...
# option path '/etc/firewall.user'
...

look like this:

/etc/config/firewall

...
 option path /etc/firewall.user
...

(A simplier way is to actually go to the source code and fix the issue, but this example is meant to show how this could be useful).

To understand more about how to use sed, see below references.

Method #2: awk substitution.

Another way to do substitution is to do it through awk. If sed seems cryptic, awk is even more so. Though the language is very powerful and there are many methods to accomplish this through awk, here is a simple example that does the line replacement:

# Replace line containing text1 with text2.
> awk '/text1/ {print "text2"; next} /.*/ print ;}' file > tmp.file
> mv tmp.file file

One disadvantage is that you can’t replace the file directly (since it seems to erase the file completely).

Some explanation over the awk syntax: AWK goes through a pattern-action routine. If /pattern/ matches, then {run action}. So if it matches /text1/, then it’ll {print “text2”}. The next means to skip to the next line, or otherwise it’ll continue to try to match against other patterns. The second pattern (/.*/) is there to match all the other lines, and print as normal.

So, using the above openWRT situation, an awk statement will look like this:

# Uncomment out the etc/firewall.user
>
awk '/option.*\/\etc\/firewall.user/ {print "option path /etc/firewall.user"; next} /.*/ {print ;}' /etc/config/firewall > tmp
> mv -f tmp /etc/config/firewall

More information about awk can be found in the references below.

Method #3: Perl substitution.

For those who really need to use Perl or just so happen to have it installed, here is how to do line substitution in Perl:

# Replace line containing text1 with text2.
> perl -pi -e 's/^.*test1.*/test2/' file

Like in sed, the -i means to edit directly into the file. The -p allows to read in file as input, and -e executes the commands the quotes. To run the openWRT example above, it would look something like this:

# Uncomment out the etc/firewall.user
>
perl -pi -e 's/.*option.*\/etc\/firewall.user.*/option path \/etc\/firewall.user/' /etc/config/firewall

Method #4: Python substitution.

This section will be added later once I have a better understanding of python. Unfortunately, the language is elusive to me, so there's room for improvement to understand how to work with this language.

References:

Sed: An Introduction and Tutorial
AWK: Find and Replace text
AWK: Manpage for awk(1)
Perl: Command-Line Options
Perl: Regular Expressions

Advertisements
Categories: Linux router
  1. October 1, 2009 at 5:14 pm

    I don’t know If I said it already but …Hey good stuff…keep up the good work! 🙂 I read a lot of blogs on a daily basis and for the most part, people lack substance but, I just wanted to make a quick comment to say I’m glad I found your blog. Thanks,)

    A definite great read..Jim Bean

    • coder_commenter
      October 4, 2009 at 12:04 am

      Thanks, this means a lot to me.

      It’s a struggle to come up with meaningful posts instead of random nonsense entries, but I will endeavor to continue posting some more content.

      Thanks for commenting.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: