Archive for the ‘Linux router’ Category

How to do line substitution in files for linux routers?

September 23, 2009 2 comments

In many situations, this is a trivial task: create a simple regex script in Perl (or Python), and execute it. However, with linux routers it becomes a bit tricky. Usually by default these routers do not have high-level scripting languages installed due to limited resources. It is possible to add them by extending the diskspace through external means (like USB or SD card), but generally unless the user needs to use scripts extensively it isn’t necessary. There are several means to do substitution here.

Method #1: sed substitution.

Sed substitution is the easiest method to do line substitution, and the most recommended method. However, the cryptic syntax is not very user-friendly, especially those unfamiliar to sed (yet familiar to generic regex). To do a simpleĀ  substitution, use the following syntax:

# Syntax to do substitution in the file.
> sed -i 's/text1/text2/' file

This will take all instances of ‘text1’ and replaces it with ‘text2’ in the file. Note that the ‘-i’ means that it will change the file directly instead of printing the changes to stdout. This only does pure substitution, not line substitution. To do that, we need to use the ‘c\’ option instead of the ‘s’ option at the beginning of the string. See below:

# Syntax to do line substitution in the file, trying to match text1 with text2.
> sed -i '/text1/ c\ text2' file

Note that the ‘s’ in the beginning of the string is no longer there. There are actually several options in this same format. ‘i\’ will insert the line before the matched text1, and ‘a\’ will append the line after the matched text1.

In a practical example, openWRT tends to not setup the firewall (iptables) properly. When using the UI, it tends to comment out the option to include firewall.user. So I used this simple sed statement to uncomment the option whenever the firewall is changed:

# Uncomment out the etc/firewall.user
> sed -i '/option.*\/etc\/firewall.user/ c\ option path /etc/firewall.user' /etc/config/firewall

This will make:


# option path '/etc/firewall.user'

look like this:


 option path /etc/firewall.user

(A simplier way is to actually go to the source code and fix the issue, but this example is meant to show how this could be useful).

To understand more about how to use sed, see below references.

Method #2: awk substitution.

Another way to do substitution is to do it through awk. If sed seems cryptic, awk is even more so. Though the language is very powerful and there are many methods to accomplish this through awk, here is a simple example that does the line replacement:

# Replace line containing text1 with text2.
> awk '/text1/ {print "text2"; next} /.*/ print ;}' file > tmp.file
> mv tmp.file file

One disadvantage is that you can’t replace the file directly (since it seems to erase the file completely).

Some explanation over the awk syntax: AWK goes through a pattern-action routine. If /pattern/ matches, then {run action}. So if it matches /text1/, then it’ll {print “text2”}. The next means to skip to the next line, or otherwise it’ll continue to try to match against other patterns. The second pattern (/.*/) is there to match all the other lines, and print as normal.

So, using the above openWRT situation, an awk statement will look like this:

# Uncomment out the etc/firewall.user
awk '/option.*\/\etc\/firewall.user/ {print "option path /etc/firewall.user"; next} /.*/ {print ;}' /etc/config/firewall > tmp
> mv -f tmp /etc/config/firewall

More information about awk can be found in the references below.

Method #3: Perl substitution.

For those who really need to use Perl or just so happen to have it installed, here is how to do line substitution in Perl:

# Replace line containing text1 with text2.
> perl -pi -e 's/^.*test1.*/test2/' file

Like in sed, the -i means to edit directly into the file. The -p allows to read in file as input, and -e executes the commands the quotes. To run the openWRT example above, it would look something like this:

# Uncomment out the etc/firewall.user
perl -pi -e 's/.*option.*\/etc\/firewall.user.*/option path \/etc\/firewall.user/' /etc/config/firewall

Method #4: Python substitution.

This section will be added later once I have a better understanding of python. Unfortunately, the language is elusive to me, so there's room for improvement to understand how to work with this language.


Sed: An Introduction and Tutorial
AWK: Find and Replace text
AWK: Manpage for awk(1)
Perl: Command-Line Options
Perl: Regular Expressions

Categories: Linux router