17 Feb 10 Dealing With Recursiveless Grep
If you work at all in Solaris (and maybe just older versions I don’t know) you will eventually find out that the version of “grep” shipped with the OS is missing the “-r” option for doing recursive greps. Over the years I have found myself becoming a huge fan of this ability and I am always amazed to find there are versions that don’t include it. Today I set out to come up with a way to emulate the behavior.
I came up with two solutions. First we have:
find . |xargs grep PATTERN
While this seemed to work fine at first I noticed that it had issues with file that contained spaces. It would treat each “word” of a file name with spaces as a separate filename and I would get errors from grep about not being able to find the file. If you know that you don’t have any filenames with spaces though the above should work just fine.
As I played around further I managed to come up with the following second iteration:
find . -exec grep PATTERN {} /dev/null \;
This one worked just as well as the first one but had the added advantage of treating file names with spaces in the correctly. I don’t claim to be a find master. Unfortunately I can’t tell you why the “/dev/null” is required except that I can tell you without it you will get the lines in files matching the pattern returned but not the filename the line is contained in.
15 Feb 10 When grep and sed are not enough…
I found an article over at Linux Journal that lead with the sentence:
When grep and sed aren’t enough, gawk may provide the extra horsepower that you need.
I would have to argue against that. Nothing personal against gawk, awk itself, or any of it’s variants. I personally love awk. Though I would rewrite their first sentence a bit.
When grep and sed aren’t enough, you obviously don’t know enough grep and sed.
At any rate he does provide some gawk examples that aren’t totally worthless but certainly don’t require it over grep and/or sed. I am guessing that there will be some decent examples either way in the comments section as well.
18 Nov 09 Move LVM2 VG From Old Drive To A Drive On A New Machine
My old computer recently died. It was old enough that I didn’t spend a lot of time trying to fix it. I took the opportunity to get a new one. I have my drives set up so that my “home” partitions are in an LVM volume group. I wanted to move that VG to the new drive on the new machine. This is what worked well for me.
Assuming you have the old drive connected to the new machine the simplest way seemed to be adding the physical volume on the new disk to the VG on the old disk then moving all of the extents that exist in the VG on the old disk to the new. Finally I removed the old disk from the VG.
Here are the steps I took, I will explain each after the code.
vgchange -ay pvcreate /dev/sda5 vgextend MyVG /dev/sda5 vgmove -v /dev/sdb3 /dev/sda5 vgreduce MyVG /dev/sdb3
1: That started up the MyVG volume group on the old HDD. Although it’s not shown above I mounted it and looked around to make sure the files were there and everything looked good.
2: I allocated the partition on the physical disk in the new machine (sda5 for me) to LVM.
3: Here I added the LVM physical volume (pv) created above to the MyVG volume group that up to this point resided only on the old HDD. So now instead of a 100G “disk” only on the 1 HDD I now had a 360G “disk” that actually spanned 2 physical disks.
4: This moved all of the physical extents of MyVG that resided on the old HDD (sdb3 for me) to physical extents on the new HDD. -v just gave me a 15 second rolling counter of the progress. It took about 25 minutes to move 100G or so.
5: This removed the old HDD from the MyVG. Everything is now on the same volume group it has always been on but that VG now resides fully on the new HDD now.
In case you are wondering why I didn’t just keep the old HDD I moved from a desktop machine to a laptop.
Tags: configuration, Linux, lvm, lvm2, pvcreate, Shell, vgchange, vgextend, vgmove, vgreduce
06 Jun 09 Serving Files Simply With Python
I had found this code for serving up a directory with a simple python command.
python -m SimpleHTTPServer 9914
The above command will start a simple HTTP server on port 9914 serving files out of the current directory and below.
For those looking for something just a bit more fancy the below script should do the trick while still keeping things simple. Again, this will serve files from the current directory down.
#!/usr/bin/python
import BaseHTTPServer, SimpleHTTPServer
import os
import sys
def run(server_class=BaseHTTPServer.HTTPServer,
handler_class=SimpleHTTPServer.SimpleHTTPRequestHandler):
print 'Server version:',handler_class.server_version
port=8000
if len(sys.argv)>1:
if sys.argv[1].isdigit():
port=int(sys.argv[1])
server_address = ('', port)
httpd = server_class(server_address, handler_class)
myurl='http://localhost:'+str(server_address[1])+'/'
print 'Your Server is running on:',myurl
print 'and serving files from:',os.getcwd(),'and below.'
print 'To stop the server, type ^C.'
if 'b' in sys.argv:
print 'Trying to start webbrowser...'
import webbrowser
webbrowser.open(myurl)
httpd.serve_forever()
run()
The longer script was found here (site is in German but the code is clear). It may be advisable to get the script code from the actual site since my code plugin seems to be messing up the formatting which is kind of important in python code.
If you would rather use WebDAV I have used the code from the pywebdav project with some decent results.
NOTE: I believe both the above python code and pywebdav require Python2.4 or greater but I have not tested that myself.
23 Jan 09 Add A Comma To The End Of A Line
I work with long lists and WinSQL a lot. I always find myself wishing these long lists had commas at the end for pasting into a SQL where clause using “IN” for the list. This can be accomplished in one of two ways.
From the command line:
sed "s/$/,/g" <FILENAME>
I usually pipe the output there to a new file.
In vi (from command mode):
:%s/$/,/g
The “%” says to perfom the substitution on the whole file. In both cases we are doing a substitution of “$” (the end of the line) with a “,”.
Tags: automation, replace, sed, substitute, vi
15 Jan 09 Process A File Line By Line In A Script
There are multiple ways to skin this cat. For now one of the simplest.
#!/bin/sh
while read LINE
do
...DO SOME STUFF WITH THE ${LINE}...
done <somefile.txt
Replace “somefile.txt” with the file you want to process.
15 Jan 09 How To Set The Window/XTerm Title
I find it useful to set the window title of my terminals for long running processes so I can see at a glace what is happening in the window or where in the process running in that window is at. In scripts I will use the following:
echo "\033]0;Window for ${USER}\007"
Of course replace “Window for ${USER}” with whatever is appropriate for your use.
18 Dec 08 Spawning Sub-Shells
Creating sub-shells in bash is simple: just put the commands to be run in the sub-shell inside parentheses. This causes bash to start the commands as a separate process. This group of commands essentially acts like a separate script file, their input/output can be collectively redirected and/or they can be executed in the background by following the closing parenthesis with an ampersand.
#!/bin/bash server_cmd=server pid_file=$(basename $server_cmd .sh).pid log_file=$(basename $server_cmd .sh).log ( echo "Starting server" echo "Doing some init work" $server_cmd # server becomes a daemon while true do if [[ -f $pid_file ]]; then sleep 15 else break fi done mail -s "Server exitted" joe@example.com <<<CRAP ) 2>&1 >> $log_file & echo "Server started"quote and example via LinuxJournal
18 Dec 08 Remembering The Shell’s Special Variables
I can never seem to remember what all of these default parameter variables are. I found this small shell script that prints them out when I forget.
#!/bin/sh -vx ####################################################### # example_1.1 (c) R.H.Reepe 1996 March 28 Version 1.0 # ####################################################### echo "Script name is [$0]" echo "First Parameter is [$1]" echo "Second Parameter is [$2]" echo "This Process ID is [$$]" echo "This Parameter Count is [$#]" echo "All Parameters [$@]" echo "The FLAGS are [$-]"It produces output like this:
#!/bin/sh -vx ####################################################### # example_1.1 (c) R.H.Reepe 1996 March 28 Version 1.0 # ####################################################### echo "Script name is [$0]" + echo 'Script name is [/home/me/bin/shellhelp]' Script name is [/home/me/bin/shellhelp] echo "First Parameter is [$1]" + echo 'First Parameter is []' First Parameter is [] echo "Second Parameter is [$2]" + echo 'Second Parameter is []' Second Parameter is [] echo "This Process ID is [$$]" + echo 'This Process ID is [15190]' This Process ID is [15190] echo "This Parameter Count is [$#]" + echo 'This Parameter Count is [0]' This Parameter Count is [0] echo "All Parameters [$@]" + echo 'All Parameters []' All Parameters [] echo "The FLAGS are [$-]" + echo 'The FLAGS are [hvxB]' The FLAGS are [hvxB]
06 Dec 08 Increase Loadaverage
Increase the load average on a machine for testing.
while true; do w & w & w & w; done
