The skill which refuses to die - Shell scripting

Well I agree this is another article among thousands of articles on shell scripting available on the internet, but I thought of presenting it in a way which would attract you to write your first shell script right away. This may be a tall claim but let me try.
2. Find and replace text in thousands of files.
3. Read and output delimited files and it's fields.
4. Send mail with attachments.
5. Make backups of your files and also restore them.
6. Compile programs for you.
7. Download files from the internet.
8. Sync your local files with remote files in the server.
9. Scan your files for viruses.
10. FTP your files to and from your remote server.
11. Run batch jobs for you overnight
12. Monitor, report and block IP addresses which attack your server or your PC automatically.
13. Filter your mails for spam.
14. You name it and it can do it for you exactly as you want.
I have a server installed with GNU/Linux and it occupies only 3GB of disk space and I can do all of the above using simple shell scripts. If I hadn't used shell scripts, I would need atleast 20GB of disk space for various programs which does each of the above jobs separately.
2. Break it into smallest possible steps each of which can be easily done.
3. Sequence your steps identified above one after another.
4. Identify repetitive tasks in your iteration.
5. Identify your failure and success criteria of each of your smaller tasks.
2. Open and read system syslog file, search for text patterns, sort text.
3. Read the server's syslog file, identify logs printed by iptables and separate them out, extract source ip addresses and destination port from those logs, filter out IP addresses where destination port is 22 (SSH), sort the IP addresses, find the count of each distinct IP address, sort again based on count in descending order, report the first 10.
4. No repetitive tasks are present.
5. If no IP addresses attack my SSH port then output a appropriate message or if some error happens then output an error message and quit.
cat /var/log/syslog | grep "BLOCKED_PACKETS" | sed -rn 's/^.*SRC=([^ ]*).*DPT=22 .*/\1/p' | sort | uniq -c | sort -nr | head -10 |
The above one liner will give you your top ten IP addresses who are attacking your SSH port.
Let me explain what every command is doing and what it is passing to the next command.
cat /var/log/syslog - Reads the system log file from location /var/log/syslog and output the contents of the file line by line.
grep "BLOCKED_PACKETS" - Takes the output of the above command and filters lines which have the comment "BLOCKED_PACKETS" which is usually set by the iptables rules when it blocks a packet. The exact comment may vary by your iptables configuration. If your iptables does not append any comments to it's log then you may omit this command altogether.
sed -rn 's/^.*SRC=([^ ]*).*DPT=22 .*/\1/p' - Takes the output of the above command and extracts the IP addresses which tried to connect to the SSH port 22 with the help of regular expressions. Iptables usually writes log in the format SRC=<IP address> and DPT=<port number> to output source IP address and destination port respectively.
sort - Sorts the IP address for next command uniq to report the unique IP addresses correctly.
uniq -c - Takes the output of above command and prints the count and IP address of all unique IP addresses.
sort -nr - Sorts the output of the above command in descending order of the count of IP addresses.
head -10 - Takes the output of the above command and prints only the first 10 lines.
So you can see how a complicated problem can be solved by just passing output of one command to another.
The complete solution to the above stated problem can be solved using the below shell script.
#!/bin/bash
SSH_ATTACK_PRESENT=`cat /var/log/syslog | grep "BLOCKED_PACKETS.* DPT=22" | head -1`;
if [ "$SSH_ATTACK_PRESENT" = "" ]
then echo "No attacks on SSH port."; exit 0; fi; cat /var/log/syslog | grep "BLOCKED_PACKETS" | sed -rn 's/^.*SRC=([^ ]*).*DPT=22 .*/\1/p' | sort | uniq -c | sort -nr | head -10;
if [[ $? -ne 0 ]]
then echo "An error occured while processing your request, quitting"; exit 1; fi; |
A shell script can be used as program to use Linux/UNIX commands with conditional branching and iteration based on output of the commands and their exit criteria.
A shell script provides the below special variables for the user to program accordingly.
Shell variable | Description |
$? | Exit code of last executed command |
$* | All command line parameters passed to the shell script separated by $IFS if parameters were passed inside quotes |
$@ | All command line parameters passed to the shell script as it is. |
$1, $2, $3 .... | Command line parameter 1, 2, 3 ... respectively |
$0 | Invocation command of the shell script itself |
$# | Count of command line parameters |
$$ | Process ID of the running shell script |
$! | Process ID of the last command run in the background |
Shell uses the below control statements like a programming language to take care of branching and iteration.
Control statement | Description |
if, else, elif | Checks for a condition to be true or false, gets into the block if the condition is true |
for | Loops through a list separated by $IFS character for each of the values in the list |
while | Loops till the condition in the while statement is true |
case | Jumps to the statement where the condition matches, behaves similar to switch-case used in various programming languages |
break | Jumps out of the current loop block (for, while) |
continue | Jumps to the next iteration of the current loop block |
Another useful feature of shell scripts is functions. You can write functions in a shell script to execute a block of shell script statements over and over again. A good use case for functions is to perform logging where you need to output the log in a certain format like appending date-time or a suffix. Functions can be invoked similar to invoking another shell script. Also shell script functions utilizes the special variables as described above with respect to it's own invocation like for example I invoke a function with two parameters "param1" and "param2", $1 and $2 variables inside the function will have values param1 and param2 respectively. Below is an example to illustrate this feature.
#!/bin/bash function test_function test_function "param1" "param2"; |
The above shell script will print param1,param2 as output.
Photo credit - https://www.sandervanvugt.com/wp-content/uploads/2015/06/bash-shell-scri...
- Log in to post comments