File Handling in Linux

Introducing file handling

The exec command is very interesting. Whenever we run any command in a shell, a new subshell or process gets created, and the command runs in this newly created process. When we run any command as an argument to the exec command, exec will replace the current shell with the command to be executed. It does not create or spawn a new process to run the command.

Using exec to assign a file descriptor (fd) to file

In the Bash shell environment, everyprocesshas three files opened by default. These are standard input, display, and error. The file descriptors associated with them are0,1, and2respectively. In the Bash shell, we can assign the file descriptor to any input or output file. These are called file descriptors.

The syntax for declaringoutput.txtas output is as follows:

exec fd > output.txt

This command will declare the numberfdas an output file descriptor.

The syntax for closing the file is as follows:

exec fd<&-

To close fd, which is 5, enter the following:

exec 5<&-

We will try to understand these concepts by writing scripts.

Understanding the opening, writing, and closing of a file

Let’s understandtheopening, closing, andwritingof a file.

Write a Shell scriptfile_01.sh, shown as follows:

file_01.sh

#!/bin/bash 
# We will open file for writing purpose 
# We are assigning descriptor number 3 for file sample_out.txt 
exec 3> sample_out.txt 
 
# We are sending output of command "echo" to sample_out.txt file 
echo "This is a test message for sample_out.txt file" >&3 
 
# Run command date & store output in file sample_out.txt 
date >&3 
 
# Closing file with file descriptor 3 
exec 3<&-

Save the file, give the permission to execute, and run the script as follows:

$ chmod u+x file_01.sh
$ ./file_01.sh
$ cat sample_out.txt

This should produce the following output:

Output:

This is a test message for sample_out.txt fileTue Sep 29 23:19:22 IST 2015

Understanding reading from a file

Let’s write ascriptto read from a file:

Write the scriptfile_02.sh, shown as follows:

file_02.sh

#!/bin/bash 
# We will open file sample_input.txt for reading purpose. 
# We are assigning descriptor 3 to the file. 
exec 3< sample_input.txt 
 
cat <&3 
# Closing file 
exec 3<&-

Save the file, give the permission to execute, and run the script as follows:

$ chmod u+x file_02..sh

We will create the sample_input.txt file as follows:

$ echo "Hello to All" > sample_input.txt

Run the script and check the result:

$ ./file_02.sh

This should produce the following output:

Output:

Hello to All

Understanding reading and writing to a file

In the earlier examples, we opened the file either for reading or writing. Now we will see how to open the file for reading and writing purposes:

exec fd<> fileName

If the file descriptor number is not specified, then0will be used in its place. The file will be created if it does not exist. This procedure is useful for updating files.

Let’s understand the following script.

Write the shell scriptfile_03.shas follows:

file_03.sh

#!/bin/bash 
file_name="sample_out.txt" 
# We are assing fd number 3 to file. 
# We will be doing read and write operations on file 
exec 3<> $file_name 
 
# Writing to file 
echo """ 
 Do not dwell in the past, 
 do not dream of the future, 
 concentrate the mind on the present moment. - Buddha 
""" >&3 
# closing file with fd number 3 
exec 3>&-

Using the read command on a file descriptor (fd)

We can use the read command to get data from a file to store it in variables. The procedure for using the read command to get a text from a file is as follows:

read -u fd variable1 variable2 ... variableN

Reading from one file and writing to another file

Now we will see how to read from one file and write to another. Let’s write the file_04.sh script as follows:

file_04.sh

#!/bin/bash 
# We are assigning descriptor 3 to in_file.txt 
exec 3< in_file.txt 
# We are assigning descriptor 4 to out_file.txt 
exec 4> out_file.txt 
 
# We are reading first line of input.txt 
read -u 3 line 
 
echo $line 
 
echo "Writing content of in_file.txt to out_file.txt" 
echo  "Line 1 - $line " >&4 
 
# Closing both the files 
exec 3<&- 
exec 4<&-

Save the file, give the permission to execute, and run the script as follows:

$ chmod u+x file_04.sh
$ echo "Sun is at the center of Solar System." > in_file.txt
$ cat in_file.txt

It will gives you output similar to below:

Output:

Sun is at the center of Solar System.
$ ./file_04.sh
Sun is at the center of Solar System.Writing content of in_file.txt to out_file.txt
$ cat out_file.txt
Line 1 - Sun is at the center of Solar System.

In this example, we read the complete line in the variablelineand we use the same variable to write it to another file.

Let’s write one more script,file_05.sh, to get the hostname and addresses:

file_05.sh

#!/bin/sh 
 
cd /etc/hosts hosts2 
 
grep -v '^#' hosts2 > hosts3 
 
exec 3< hosts3    # opening hosts3 as input file 
 
exec 4> hostsfinal  # opening hostsfinal as output file 
 
read <& 3 address1 name_1 extra_info 
read <& 3 address2 name_2 extra_info 
 
echo $name_1 $address1 >& 4 
echo $name_2 $address2 >& 4 
 
exec 3<&-      # Closing hosts3 
exec 4<&-      # Closing hostsfinal

In this script, we used the variables address1, name_1, extra_info, address2, and name_2 to store useful information.

Displaying the file descriptor information from the /proc folder

We willwrite thescriptto display the actual file descriptors associated with the file.

Let’s write thefile_06.shscript, shown as follows:

file_06.sh

#!/bin/bash 
# we are assigning file descriptor 3 to input file test.txt 
exec 3< test.txt 
# we are assigning file descriptor 4 to output.txt 
exec 4> output.txt 
# we are using read command to read line from file 
read -u 3 line 
echo "Process id of current process is $$" 
my_pid=$$ 
echo "Currently following files are opened by $0 script :" 
ls -l /proc/$my_pid/fd 
 
# We are closing both files test.txt and output.txt 
exec 3<&- 
exec 4>&-

File handling – reading line by line

You willlearnhow to use thewhileloop and thereadcommand to read a file line by line. You will learn more about thewhileloop in the upcoming chapters.

Let’s write thefile_07.shscript, as follows:

file_07.sh

#!/bin/bash 
echo "Enter the name of file for reading" 
read file_name 
exec<$file_name 
while read var_line 
do 
  echo $var_line 
done

For executing the preceding script, we will need to create a file with some text in it. Then, we can pass this filename for reading purposes.

Executing the command and storing the results in a file

The following is the syntax for storing the output of a command in a file:

Command >& fd./script >& fd

The following is the illustrative example script, file_08.sh:

file_08.sh

#!/bin/bash 
exec 4> output.txt 
cat /proc/cpuinfo  >&4 
exec 3<&-

Save the file, give the permission to execute, and run the script as follows:

$ chmod u+x file_08.sh
$ ./file_08.sh

Here’s the output:

In this example, we have executed the command cat /proc/cpuinfo and we have stored the output in the file, output.txt

Summarizing usage of the exec command

The following is a summary of the exec command for using various file handling-related operations:

Command

What it does

execcommand

This command will replace shell and execute it. Therefore, it will not return to its original shell, which started it.

exec > data.txt

This opensdata.txtfor writing standard output.

exec < data.txt

This opensdata.txtfor reading standard input.

exec 3< data.txt

This opensdata.txtfor reading with descriptor3.

sort <&3

This will sort thedata.txtfile.

exec 4> data.txt

This opensdata.txtfor writing with descriptor 4.

ll >&4

The output ofllis redirected todata.txt.

exec 6<&5

This makesfd 6a copy offd 5.

exec 4<&-

This closesfd 4.