Bash blog postings

Bash Postings

Earliest: March 1, 2017Latest: March 22, 2017Total: 4

March 22, 2017

Grep

Grep is a great powerful tool that print lines matching certain patterns.

Some example Regular Expressions using Grep:

Remove BOTs from the website access.log file:

grep -Ev '|spider|slurp|yandex|bingbot|majestic12' access.log > access2.log

Show only City/Towns in Massachusetts that have at least 19 charaters

grep -x '.{10}' MassachusettsCityTowns.txt

Highlight the Search Results

export GREP_OPTIONS='--color=auto' GREP_COLOR='100;8'
grep '.{10}' MassachusettsCityTowns.txt

Using Grep in Bash Script

This is a script that I wrote a long time ago, that I used to check to see how many times a certain error would appear in a log file.

grepcode

#!/bin/bash
itoday=$(date +"%d/%b/%Y")
todayerrors=$(cat grep $itoday /server/debug.log | grep $itoday | grep "Exception" | wc -l)
echo "# ------------------------------------------------------------"
echo "# Number of Errors Found on $itoday : $todayerrors"
echo "# ------------------------------------------------------------"
echo "# Raw data"
grep -C 10 "Exception" /mnt/$i/server/debug.log

This script will count out the total number of errors that were generated today. Then display the RAW lines. This is useful to have handy to see if a particular error is occurring quite often.

Quick RegEx Table

This is a quick regular expression table that I found many years ago. It's quick reminder of some of the common grep commands:

#######################################################
# RegExpression Table
#######################################################
Reg-expr   | Description
-----------+---------------------------------------------------
.          | Matches any character except newline
[a-z0-9]   | Matches any single character of the set
[^a-z0-9]  | Matches any single character not in set
d         | Matches a digit,                   i.e., [0-9]
w         | Matches a alpha-numeric character, i.e., [a-zA-Z0-9_]
W         | Matches a non-word character,      i.e., [^a-zA-Z0-9_]
metachar  | Matches the character itself,      i.e., |, *, +
x?         | Matches 0 or 1 x's, where x is any of the above
x*         | Matches 0 or more x's
x+         | Matches 1 or more x's
x{m,n}     | Matches at least m x's but no more than n
foo|bar    | Matches one of foo or bar
(x)        | Brackets a regular expression (this is a bit of a lie :-)
b         | Matches a word boundary

March 15, 2017

Directory Cleanup via Bash Script

I Mac Terminal

This is a simple BASH script that I used to sort a folder that has a lot of images. I wanted to put photos in a folder based on the date it was created.

This script will work in any environment, but I wrote this specifically to handle a large set of photos in a folder on my Macintosh.

#!/bin/bash
for f in *; do
dir=$(stat -f%SB -t%Y-%m-%d "$f")
echo $f '->' $dir
[ -d "$dir" ] || mkdir "$dir"
mv "$f" "$dir"/
done

What this Script Does

When you execute this script in any directory, it will use the UNIX stat command to check on the time stamp on the file. It will then move the file to a folder with the same. If the directory doesn't exist then one will be created.

Script Execution

Create a file with the above content. I call my file sort.sh. Move the file to any directory that you want to clean up. To execute, simply type in:

./sort.sh

In a few minutes you'll see the contents of the folder be replace with a bunch of date folders. (Yes even the sort.sh file will get moved to a folder.)

March 8, 2017

Bashmarks

When you SSH to a remote box do you change to the same set of directories? Wouldn't it be great to bookmark the locations to make it easy to get to? You can with Bashmarks!

Bashmarks gives you the ability to create simple bookmarks to any location on your server. Now you can jump to whatever directory you want wherever you are on the box.

As my former boss would say, "Now your cooking with gas."

Bashmarks

Setting up Bashmarks

Bashmarks is simply a shell script that performs actions.

Download bashmarks.sh from GitHub. Put the file someplace where you won't accidentally delete it. (~/bin is a safe place, Macintosh users may want to keep it in /usr/local/bin)

If it isn't easy to move the file, simply copy and paste the contents of the file into a new file on the server that you want to use. (The file size is small.)

Update the .bash_profile with the following line:

# Make sure the directory and filename is correct.
source ~/bin/bashmarks.sh

Exit out and log back in the server.

Type:
bookmarksshow

If it's installed correctly, you should get back an empty line.

Using BashMarks

It's super easy to use BashMarks. When your at a directory that you want to bookmark simply type in:

bookmark staging

Where "staging" is the name of the bookmark - Tip: keep it simple and short.

When you want to go to that directory simply type in:

go staging

If you want to see all the Bashmarks that you have installed, type in:

bookmarksshow

Additional Notes

Currently Bashmarks only works with directories and not applications. However, you have the bashmarks.sh source file and it should be easy to manipulate it to do what works best in your environment. For example, you could shorten the word 'bookmark' to simply 'book.'

Enjoy playing around with Bashmarks!

March 1, 2017

Emoji on the Command Line

In today's environment, Developers and QA engineers quite often have to log in to a remote AWS server that has virtual servers to perform tasks. Sometimes they are busy and may forget which server they are monitoring. One way to fix this is to use the emoji name in the Macintosh prompt. This way you know that your back on your computer and not on any production AWS server.

You will need to use Apple's Terminal app to add Emoji to the prompt, as iTerm doesn't really have good emoji support. This isn't an issue since your not going to be using emoji all the time.

Enable Emoji at the Command Line

Open Terminal app and use nano to modify the .bash_profile file:

nano .bash_profile

Tech Note: This will not work using BBEdit. This functionality appears to be an exclusive functionality of the Terminal app.

Add a new line like the following:

PS1=" "

PS1 is the Bash variable for Prompt.

Now pull down the "Edit" menu and choose "Special Characters", then select "Emoji" from the special character menu

Find the Emoji you want to use in the shell prompt, I would suggest an Apple or the House. Then drag & drop it into the PS1=" " line so that it's contained within the quotes.

Save the .bash_profile change with Control+O then exit out of nano with Control+X

Open a new Terminal window to see the emoji as the prompt

Bash Login

What is Nano?

nano is a small, free and friendly editor which aims to replace Pico, the default editor included in the non-free Pine package. Rather than just copying Pico's look and feel, nano also implements some missing (or disabled by default) features in Pico, such as "search and replace" and "go to line and column number".

Bash Generator

If you want more out of your Bash Prompt, check out the Bash Generator where you can create a productive Bash Prompt.