Shell Scripting Tips (revisited)

Nick Holland

Target Audience

• Assuming you know what shell scripting is, you've done it, but you could do it better.
• No claim of omnipotence
  • Could be better ways
  • Could be all wrong!
• Targeted at ksh, but much is applicable to bash & sh

Keep it neat and simple

• Comment! Comment! Comment!
  • What the code does.
  • Why you wrote it that way.
  • Problems you ran into along the way.
• Code for readability, not to impress.
  • One liners are hard to read.
  • Hard to read is hard to maintain.
  • If performance matters, shell script is probably not the right tool.

Classes of scripts

• Quick & Dirty
  • Run Once and Forget.
  • Tempting to ignore error handling...DON'T!
• Administrative scripts
  • Run by people who know what they are doing.
  • Probably running in cron or other "controlled" environment.
  • Temping to minimize error handling...what could go wrong?
• User-facing applications
  • Users can manage to enter the craziest things.
  • They assume you controlled the damage they can do.
  • Arrow keys. They found the damn arrow keys.

Deal with errors

• Test your assumptions:

   cd $DIR
   rm -r * 

  what if $DIR is blank? What if it fails?
• Validate input AND results
  • Did that nifty bit of string manipulation and calculation really give you a valid response under all circumstances?
  • are you so sure you are betting your data on it?

-e : Trap all errors!

#!/bin/ksh -e

cd /some/dir
rm -r *   # Could be bad if the 'cd' failed!

Header for Q&D scripts?

#!/bin/ksh
yell() { echo "$0: $*" >&2; }
die()  { yell "$*" ; exit 111; }
try()  { "$@" || die "cannot $*"; }

try cd /some/dir
rm -r *   # Could be bad if the 'cd' failed!

 
$ ./test.ksh   
./test.ksh[7]: cd: /some/dir - No such file or directory
./test.ksh: cannot cd /some/dir

Validate all input

Make friends with set, typeset

• set -f -> Turns off globbing!
• set -- $A -> populate $1, $2, ... with $A
• typeset -l var -> lower case variable var
• typeset -u var -> upper case variable var
• There is much, much more to these commands.
• Good thing to comment, though.

set -f -- Turn off globbing!

$ A="*"
$ echo $A
Desktop Downloads ShellScriptTips.odp ShellScriptTips01.odp
   [...]
$ set -f  
$ A="*"
$ echo $A
*
$ 

set -- $V -- put $V into $*

$ A="     A    b  c D   e    "
$ echo "$A"
     A    b  c D   e    
$ echo "$*"

$ set -- $A
$ echo "$*" 
A b c D e
$ echo "$3"
c

• (this is why normal people hate computer people)
• You may well want to save $0 (or $*) to $THISPROG or similar.
  

Case insentitivity: tr(1)

$ V="AbCdEf"
$ V=$(echo $V | tr "A-Z" "a-z")
$ echo $V
abcdef
$ 

Case insentitivity: the lazy way

$ typeset -l V
$ V="AbCdEf" ; echo $V
abcdef
$ typeset -u V         
$ V="AbCdEf" ; echo $V 
ABCDEF

tput(1)

BOLD=$(tput bold)
NORM=$(tput sgr0)
print -n "Enter your input ->$BOLD"
read response
print -n "$NORM"

General Tips(using KSH!)

• Use [[ ... ]] instead of [ ... ]
• Use &&, || instead of -a, -o in [[ ... ]]
• Much less quoting needed in [[ ... ]]
• X"$var" no longer needed
• use $(program) instead of `program`

• Use $V instead of ${V} when possible. From Ken Westerback

Commonly forgotten tools

• basename(1)

  $ basename /a/long/path
  path
  

• dirname(1)

  $ dirname /a/long/path
  /a/long
  

• logger(1)
• printf(1) / print -n
• test(1) (a.k.a., "[" )
  • test -d bla || mkdir bla
  • test -f bla1 && rm bla1
• mktemp(1) (better than $$, but not so portable. :-/ )

print vs. echo

"Here documents"

## Classic:
cat <<__ENDOFTEXT
this text is to be output.
__ENDOFTEXT

# Starting your "here doc" with "<<-" will 
# strip off leading tab characters. 
    cat <<-__ENDOFTEXT
	this is text to be output.
	__ENDOFTEXT

Running as root (or anyone)

# Check to see who we are running as, if not
#   root, re-invoke with sudo.
if [[ $(whoami) != "root ]]; then
    if ! sudo -n -u root $0 $* 2>/dev/null; then
        echo "Sorry, sudo isn't configured properly
        logger "$0 sudo is broke"
        sleep 2
    fi # sudo failed
    exit 
fi # not yet root
# We are now running as root

Reporting Tips

• Reports should indicate where they came from:

  echo "EOT.  Generated from $0 by $(whoami) on $(hostname) https://wiki.company.com/thisscript"

• Link back to wiki page
• Subject line should inform if further investigation is needed -- i.e., "success!" / "Failure!"
• Alerts sent to pagers: Important stuff in first 140 chars ... or separate message to pager.

Remote administration with SSH

• ssh user@$HOST "command to run"
• You will want to use keys
• You may want agent forwarding
• Script can run as non-root
• Careful with output redirection and think hard about what commands are run where.
  • ssh $HOST "cmd opt|cmd2|cmd3 >output.file"
  • ssh $HOST "cmd opt"|cmd2|cmd3 >output.file
• scp -3 : Copy between two machines through this machine

Optimizing admin with SSH

• On controlling system's user, you REALLY want ~/.ssh/config:

  ControlMaster auto
  ControlPath ~/.ssh/control/%h:%r:$p
  ControlPersist 10s
  

  (for 100 connections, could be the difference between 50 seconds and 2.5 seconds!)
• Some command line options you should know about:
  • -oStrictHostKeyChecking=no
  • -oUserKnownHostsFile=/dev/null
  • -c arcfour

Auto-change control

• Two or more machines replicating the same data/config.
• Change one machine, test, run replication script that:
  • Generate diff against $OTHER system
  • Create file with user, date, time, diff
  • Drop user into an editor to explain the change.
  • Replicate change and diff/explaination file to $OTHER
• Great for firewalls, DNS servers.

ksh "select"

$ PS3="Pick a motorcycle ->"          
$ select MOTORCYCLE in K100 K1200LT Buell Harley; do
> print "Riding the $MOTORCYCLE today"
> done
1) K100
2) K1200LT
3) Buell
4) Harley
Pick a motorcycle ->2
Riding the K1200LT today
Pick a motorcycle ->^C

Prompt maddness

Prompt maddness

• Who says a unix command prompt has to be one line?
  • Blanks between prompts help space things out on the screen
  • \n username @ machine name \n $PWD $"
• Who says a unix prompt has to lead the line at all?
  • remember: a lot of micros used to just give you a blank line to type commands into.
  • lead info lines with a "#" so they can be copied/pasted
  • maybe "shade out" the comment # so it doesn't look like a root prompt
  • export PS1="\n$(tput setf 7)#$(tput setf 0)"'\$\$\$ \u @ \h \w \$\n'
  • export PS2="#>\r"
• Not quite advocating for this, it is weird. But proving to be useful

Shell Scripting Tips

Discussion? `