Shell Scripting Tips (revisited)

Nick Holland

Target Audience

Keep it neat and simple

Classes of scripts

Deal with errors

-e : Trap all errors!

#!/bin/ksh -e

cd /some/dir
rm -r *   # Could be bad if the 'cd' failed!
  • Big hammer. Often too big.
  • Header for Q&D scripts?

    #!/bin/ksh
    yell() { echo "$0: $*" >&2; }
    die()  { yell "$*" ; exit 111; }
    try()  { "$@" || die "cannot $*"; }
    
    try cd /some/dir
    rm -r *   # Could be bad if the 'cd' failed!
    
     
    $ ./test.ksh   
    ./test.ksh[7]: cd: /some/dir - No such file or directory
    ./test.ksh: cannot cd /some/dir
    
  • A more selective hammer, but still...a hammer
  • but possibly what you need for Q&D.
  • Validate all input

  • Don't try to remove undesired things. Only accept things you expect.
  • "tr" is your friend -- "-d" (delete) and "-c" (complement).
  • tr -dc "valid-characters"
  • INPUT=$(echo "$INPUT"|tr -dc "a-zA-Z_0-9-")
  • definitely good for user-facing code
  • Make friends with set, typeset

    • set -f -> Turns off globbing!
    • set -- $A -> populate $1, $2, ... with $A
    • typeset -l var -> lower case variable var
    • typeset -u var -> upper case variable var
    • There is much, much more to these commands.
    • Good thing to comment, though.

    set -f -- Turn off globbing!

    $ A="*"
    $ echo $A
    Desktop Downloads ShellScriptTips.odp ShellScriptTips01.odp
       [...]
    $ set -f  
    $ A="*"
    $ echo $A
    *
    $ 
    
  • don't forget if you are turning globbing off!
  • set -- $V -- put $V into $*

    $ A="     A    b  c D   e    "
    $ echo "$A"
         A    b  c D   e    
    $ echo "$*"
    
    $ set -- $A
    $ echo "$*" 
    A b c D e
    $ echo "$3"
    c
    
    • (this is why normal people hate computer people)
    • You may well want to save $0 (or $*) to $THISPROG or similar.

    Case insentitivity: tr(1)

    $ V="AbCdEf"
    $ V=$(echo $V | tr "A-Z" "a-z")
    $ echo $V
    abcdef
    $ 
    

    Case insentitivity: the lazy way

    $ typeset -l V
    $ V="AbCdEf" ; echo $V
    abcdef
    $ typeset -u V         
    $ V="AbCdEf" ; echo $V 
    ABCDEF
    

    tput(1)

    BOLD=$(tput bold)
    NORM=$(tput sgr0)
    print -n "Enter your input ->$BOLD"
    read response
    print -n "$NORM"
    
    Lots of other options, varying degrees of portability, however.

    General Tips(using KSH!)

    • Use [[ ... ]] instead of [ ... ]
    • Use &&, || instead of -a, -o in [[ ... ]]
    • Much less quoting needed in [[ ... ]]
    • X"$var" no longer needed
    • use $(program) instead of `program`
    From Robert Peichaer (rpe@openbsd)
    • Use $V instead of ${V} when possible. From Ken Westerback

    Commonly forgotten tools

    • basename(1)
      $ basename /a/long/path
      path
      
    • dirname(1)
      $ dirname /a/long/path
      /a/long
      
    • logger(1)
    • printf(1) / print -n
    • test(1) (a.k.a., "[" )
      • test -d bla || mkdir bla
      • test -f bla1 && rm bla1
    • mktemp(1) (better than $$, but not so portable. :-/ )

    print vs. echo

  • echo : display string with newline
    • no options! (originally)
    • Ambiguities in the POSIX standard
  • print : better display string with newline
    • Better definition in POSIX
    • not in bash :-/
  • Nick's suggestion: IF both are available, use print
    • ...but echo makes a great debugging tool.
    • when program works, search for "echo", remove.
  • "Here documents"

    Much less picky about formatting characters than many other ways to store data.
    ## Classic:
    cat <<__ENDOFTEXT
    this text is to be output.
    __ENDOFTEXT
    
  • really messes with indentation
    # Starting your "here doc" with "<<-" will 
    # strip off leading tab characters. 
        cat <<-__ENDOFTEXT
    	this is text to be output.
    	__ENDOFTEXT
    
    
  • Running as root (or anyone)

    # Check to see who we are running as, if not
    #   root, re-invoke with sudo.
    if [[ $(whoami) != "root ]]; then
        if ! sudo -n -u root $0 $* 2>/dev/null; then
            echo "Sorry, sudo isn't configured properly
            logger "$0 sudo is broke"
            sleep 2
        fi # sudo failed
        exit 
    fi # not yet root
    # We are now running as root
    
  • Hopefully, $0 returns 0.
  • Reporting Tips

    • Reports should indicate where they came from:
      echo "EOT.  Generated from $0 by $(whoami) on $(hostname) https://wiki.company.com/thisscript"
    • Link back to wiki page
    • Subject line should inform if further investigation is needed -- i.e., "success!" / "Failure!"
    • Alerts sent to pagers: Important stuff in first 140 chars ... or separate message to pager.

    Remote administration with SSH

    • ssh user@$HOST "command to run"
    • You will want to use keys
    • You may want agent forwarding
    • Script can run as non-root
    • Careful with output redirection and think hard about what commands are run where.
      • ssh $HOST "cmd opt|cmd2|cmd3 >output.file"
      • ssh $HOST "cmd opt"|cmd2|cmd3 >output.file
    • scp -3 : Copy between two machines through this machine

    Optimizing admin with SSH

    • On controlling system's user, you REALLY want ~/.ssh/config:
      ControlMaster auto
      ControlPath ~/.ssh/control/%h:%r:$p
      ControlPersist 10s
      
      (for 100 connections, could be the difference between 50 seconds and 2.5 seconds!)
    • Some command line options you should know about:
      • -oStrictHostKeyChecking=no
      • -oUserKnownHostsFile=/dev/null
      • -c arcfour

    Auto-change control

    • Two or more machines replicating the same data/config.
    • Change one machine, test, run replication script that:
      • Generate diff against $OTHER system
      • Create file with user, date, time, diff
      • Drop user into an editor to explain the change.
      • Replicate change and diff/explaination file to $OTHER
    • Great for firewalls, DNS servers.

    ksh "select"

    $ PS3="Pick a motorcycle ->"          
    $ select MOTORCYCLE in K100 K1200LT Buell Harley; do
    > print "Riding the $MOTORCYCLE today"
    > done
    1) K100
    2) K1200LT
    3) Buell
    4) Harley
    Pick a motorcycle ->2
    Riding the K1200LT today
    Pick a motorcycle ->^C
    
    In modern bash now

    Prompt maddness

  • Traditional Unix Prompt: "#" or "$" (or "%")
  • More info:
    • /usr/src $
    • nick@fluffy3 /usr/src/sys/arch/amd64/compile/GENERIC.MP $
  • Crazy info
    • date time PWD user, machine+domain name, shell name and version, history number ...
  • Prompt maddness

    • Who says a unix command prompt has to be one line?
      • Blanks between prompts help space things out on the screen
      • \n username @ machine name \n $PWD $"
    • Who says a unix prompt has to lead the line at all?
      • remember: a lot of micros used to just give you a blank line to type commands into.
      • lead info lines with a "#" so they can be copied/pasted
      • maybe "shade out" the comment # so it doesn't look like a root prompt
      • export PS1="\n$(tput setf 7)#$(tput setf 0)"'\$\$\$ \u @ \h \w \$\n'
      • export PS2="#>\r"
    • Not quite advocating for this, it is weird. But proving to be useful

    Shell Scripting Tips

  • Discussion? `
  • /