15 Feb 10 When grep and sed are not enough…
I found an article over at Linux Journal that lead with the sentence:
When grep and sed aren’t enough, gawk may provide the extra horsepower that you need.
I would have to argue against that. Nothing personal against gawk, awk itself, or any of it’s variants. I personally love awk. Though I would rewrite their first sentence a bit.
When grep and sed aren’t enough, you obviously don’t know enough grep and sed.
At any rate he does provide some gawk examples that aren’t totally worthless but certainly don’t require it over grep and/or sed. I am guessing that there will be some decent examples either way in the comments section as well.
18 Nov 09 Move LVM2 VG From Old Drive To A Drive On A New Machine
My old computer recently died. It was old enough that I didn’t spend a lot of time trying to fix it. I took the opportunity to get a new one. I have my drives set up so that my “home” partitions are in an LVM volume group. I wanted to move that VG to the new drive on the new machine. This is what worked well for me.
Assuming you have the old drive connected to the new machine the simplest way seemed to be adding the physical volume on the new disk to the VG on the old disk then moving all of the extents that exist in the VG on the old disk to the new. Finally I removed the old disk from the VG.
Here are the steps I took, I will explain each after the code.
vgchange -ay pvcreate /dev/sda5 vgextend MyVG /dev/sda5 vgmove -v /dev/sdb3 /dev/sda5 vgreduce MyVG /dev/sdb3
1: That started up the MyVG volume group on the old HDD. Although it’s not shown above I mounted it and looked around to make sure the files were there and everything looked good.
2: I allocated the partition on the physical disk in the new machine (sda5 for me) to LVM.
3: Here I added the LVM physical volume (pv) created above to the MyVG volume group that up to this point resided only on the old HDD. So now instead of a 100G “disk” only on the 1 HDD I now had a 360G “disk” that actually spanned 2 physical disks.
4: This moved all of the physical extents of MyVG that resided on the old HDD (sdb3 for me) to physical extents on the new HDD. -v just gave me a 15 second rolling counter of the progress. It took about 25 minutes to move 100G or so.
5: This removed the old HDD from the MyVG. Everything is now on the same volume group it has always been on but that VG now resides fully on the new HDD now.
In case you are wondering why I didn’t just keep the old HDD I moved from a desktop machine to a laptop.
Tags: configuration, Linux, lvm, lvm2, pvcreate, Shell, vgchange, vgextend, vgmove, vgreduce
18 Nov 09 Simple Vim Macros
I found myself editing a configuration file (multiple actually) at work today and ran into an issue. The config file looked similar to:
Var_1=A Var_2=B Var_3=C ... Var_146=ABC Var_147=ABD
As it happened I needed to remove one of the lines at position 23. The program that reads/uses these config files has an issue if there is a break in the sequence. So after removing the line with “Var_23″ I needed to shift all of the numbers in following lines down by 1. Here is how I accomplished that.
In vim I put my cursor on the 24 in the line that now follows the “Var_22″ line and performed the following commands:
qa CTRL-X j q 125@a
Here is what happened broken down by line:
1: Here I am recording a macro (q) with the name or identifier of ‘a’. Note you will probably see some indication in the last line that you are in “recording” mode.
2: CTRL-X decrements the number under the cursor. Since I am on the 24 that decrements it to 23.
3: I move down 1 line and now the cursor is on the 25 in the next line.
4: I quit recording the macro. I know have a named (a) macro that includes the commands “decrement the number under the cursor and move down one row”.
5: I perform that named (a) macro 125 times. Now realistically I only needed to do it 123 times. It didn’t seem to matter that I used 125 but YMMV. The first line after the last line I wanted to change was a blank line so I assume the macro quit when the CTRL-X failed. You are probably better off using the exact number.
And there you have it. I successfully renumbered all of the variables below the line I deleted to be in sequence.
Tags: automation, configuration, replace, Shell, variable, vi
06 Jun 09 Serving Files Simply With Python
I had found this code for serving up a directory with a simple python command.
python -m SimpleHTTPServer 9914
The above command will start a simple HTTP server on port 9914 serving files out of the current directory and below.
For those looking for something just a bit more fancy the below script should do the trick while still keeping things simple. Again, this will serve files from the current directory down.
#!/usr/bin/python
import BaseHTTPServer, SimpleHTTPServer
import os
import sys
def run(server_class=BaseHTTPServer.HTTPServer,
handler_class=SimpleHTTPServer.SimpleHTTPRequestHandler):
print 'Server version:',handler_class.server_version
port=8000
if len(sys.argv)>1:
if sys.argv[1].isdigit():
port=int(sys.argv[1])
server_address = ('', port)
httpd = server_class(server_address, handler_class)
myurl='http://localhost:'+str(server_address[1])+'/'
print 'Your Server is running on:',myurl
print 'and serving files from:',os.getcwd(),'and below.'
print 'To stop the server, type ^C.'
if 'b' in sys.argv:
print 'Trying to start webbrowser...'
import webbrowser
webbrowser.open(myurl)
httpd.serve_forever()
run()
The longer script was found here (site is in German but the code is clear). It may be advisable to get the script code from the actual site since my code plugin seems to be messing up the formatting which is kind of important in python code.
If you would rather use WebDAV I have used the code from the pywebdav project with some decent results.
NOTE: I believe both the above python code and pywebdav require Python2.4 or greater but I have not tested that myself.
15 Jan 09 Process A File Line By Line In A Script
There are multiple ways to skin this cat. For now one of the simplest.
#!/bin/sh
while read LINE
do
...DO SOME STUFF WITH THE ${LINE}...
done <somefile.txt
Replace “somefile.txt” with the file you want to process.
15 Jan 09 How To Set The Window/XTerm Title
I find it useful to set the window title of my terminals for long running processes so I can see at a glace what is happening in the window or where in the process running in that window is at. In scripts I will use the following:
echo "\033]0;Window for ${USER}\007"
Of course replace “Window for ${USER}” with whatever is appropriate for your use.
18 Dec 08 Remembering The Order Of Time/Date Fields For Cron
If you have a hard time remembering the order of the times/dates fields of a crontab record this header might help you. I use this on all of my crontabs. Each value, if tab seperated should line up under the correct heading.
# #min hr day mon dayofwk cmd # #-------------------------------------------------------------------------------
18 Dec 08 Spawning Sub-Shells
Creating sub-shells in bash is simple: just put the commands to be run in the sub-shell inside parentheses. This causes bash to start the commands as a separate process. This group of commands essentially acts like a separate script file, their input/output can be collectively redirected and/or they can be executed in the background by following the closing parenthesis with an ampersand.
#!/bin/bash server_cmd=server pid_file=$(basename $server_cmd .sh).pid log_file=$(basename $server_cmd .sh).log ( echo "Starting server" echo "Doing some init work" $server_cmd # server becomes a daemon while true do if [[ -f $pid_file ]]; then sleep 15 else break fi done mail -s "Server exitted" joe@example.com <<<CRAP ) 2>&1 >> $log_file & echo "Server started"quote and example via LinuxJournal
18 Dec 08 Remembering The Shell’s Special Variables
I can never seem to remember what all of these default parameter variables are. I found this small shell script that prints them out when I forget.
#!/bin/sh -vx ####################################################### # example_1.1 (c) R.H.Reepe 1996 March 28 Version 1.0 # ####################################################### echo "Script name is [$0]" echo "First Parameter is [$1]" echo "Second Parameter is [$2]" echo "This Process ID is [$$]" echo "This Parameter Count is [$#]" echo "All Parameters [$@]" echo "The FLAGS are [$-]"It produces output like this:
#!/bin/sh -vx ####################################################### # example_1.1 (c) R.H.Reepe 1996 March 28 Version 1.0 # ####################################################### echo "Script name is [$0]" + echo 'Script name is [/home/me/bin/shellhelp]' Script name is [/home/me/bin/shellhelp] echo "First Parameter is [$1]" + echo 'First Parameter is []' First Parameter is [] echo "Second Parameter is [$2]" + echo 'Second Parameter is []' Second Parameter is [] echo "This Process ID is [$$]" + echo 'This Process ID is [15190]' This Process ID is [15190] echo "This Parameter Count is [$#]" + echo 'This Parameter Count is [0]' This Parameter Count is [0] echo "All Parameters [$@]" + echo 'All Parameters []' All Parameters [] echo "The FLAGS are [$-]" + echo 'The FLAGS are [hvxB]' The FLAGS are [hvxB]
