learn.mirda.tk Open in urlscan Pro
94.112.87.11  Public Scan

Form analysis
0 forms found in the DOM

Text Content

ADVANCED BASH-SCRIPTING GUIDE


AN IN-DEPTH EXPLORATION OF THE ART OF SHELL SCRIPTING


MENDEL COOPER




<thegrendel.abs@gmail.com>

10


10 Mar 2014


Revision HistoryRevision 6.505 Apr 2012Revised by: mc'TUNGSTENBERRY'
releaseRevision 6.627 Nov 2012Revised by: mc'YTTERBIUMBERRY' releaseRevision
1010 Mar 2014Revised by: mc'PUBLICDOMAIN' release

This tutorial assumes no previous knowledge of scripting or programming, yet
progresses rapidly toward an intermediate/advanced level of instruction . . .
all the while sneaking in little nuggets of UNIX® wisdom and lore. It serves as
a textbook, a manual for self-study, and as a reference and source of knowledge
on shell scripting techniques. The exercises and heavily-commented examples
invite active reader participation, under the premise that the only way to
really learn scripting is to write scripts.

This book is suitable for classroom use as a general introduction to programming
concepts.

This document is herewith granted to the Public Domain. No copyright!



--------------------------------------------------------------------------------


DEDICATION

For Anita, the source of all the magic

Table of ContentsPart 1. Introduction1. Shell Programming!2. Starting Off With a
Sha-BangPart 2. Basics3. Special Characters4. Introduction to Variables and
Parameters5. Quoting6. Exit and Exit Status7. Tests8. Operations and Related
TopicsPart 3. Beyond the Basics9. Another Look at Variables10. Manipulating
Variables11. Loops and Branches12. Command Substitution13. Arithmetic
Expansion14. Recess TimePart 4. Commands15. Internal Commands and Builtins16.
External Filters, Programs and Commands17. System and Administrative
CommandsPart 5. Advanced Topics18. Regular Expressions19. Here Documents20. I/O
Redirection21. Subshells22. Restricted Shells23. Process Substitution24.
Functions25. Aliases26. List Constructs27. Arrays28. Indirect References29. /dev
and /proc30. Network Programming31. Of Zeros and Nulls32. Debugging33.
Options34. Gotchas35. Scripting With Style36. Miscellany37. Bash, versions 2, 3,
and 438. Endnotes38.1. Author's Note38.2. About the Author38.3. Where to Go For
Help38.4. Tools Used to Produce This Book38.5. Credits38.6.
DisclaimerBibliographyA. Contributed ScriptsB. Reference CardsC. A Sed and Awk
Micro-PrimerC.1. SedC.2. AwkD. Parsing and Managing PathnamesE. Exit Codes With
Special MeaningsF. A Detailed Introduction to I/O and I/O RedirectionG.
Command-Line OptionsG.1. Standard Command-Line OptionsG.2. Bash Command-Line
OptionsH. Important FilesI. Important System DirectoriesJ. An Introduction to
Programmable CompletionK. LocalizationL. History CommandsM. Sample .bashrc and
.bash_profile FilesN. Converting DOS Batch Files to Shell ScriptsO.
ExercisesO.1. Analyzing ScriptsO.2. Writing ScriptsP. Revision HistoryQ.
Download and Mirror SitesR. To Do ListS. CopyrightT. ASCII TableIndex
List of Tables8-1. Operator Precedence15-1. Job identifiers33-1. Bash
options36-1. Numbers representing colors in Escape SequencesB-1. Special Shell
VariablesB-2. TEST Operators: Binary ComparisonB-3. TEST Operators: FilesB-4.
Parameter Substitution and ExpansionB-5. String OperationsB-6. Miscellaneous
ConstructsC-1. Basic sed operatorsC-2. Examples of sed operatorsE-1. Reserved
Exit CodesN-1. Batch file keywords / variables / operators, and their shell
equivalentsN-2. DOS commands and their UNIX equivalentsP-1. Revision History
List of Examples2-1. cleanup: A script to clean up log files in /var/log2-2.
cleanup: An improved clean-up script2-3. cleanup: An enhanced and generalized
version of above scripts.3-1. Code blocks and I/O redirection3-2. Saving the
output of a code block to a file3-3. Running a loop in the background3-4. Backup
of all files changed in last day4-1. Variable assignment and substitution4-2.
Plain Variable Assignment4-3. Variable Assignment, plain and fancy4-4. Integer
or string?4-5. Positional Parameters4-6. wh, whois domain name lookup4-7. Using
shift5-1. Echoing Weird Variables5-2. Escaped Characters5-3. Detecting
key-presses6-1. exit / exit status6-2. Negating a condition using !7-1. What is
truth?7-2. Equivalence of test, /usr/bin/test, [ ], and /usr/bin/[7-3.
Arithmetic Tests using (( ))7-4. Testing for broken links7-5. Arithmetic and
string comparisons7-6. Testing whether a string is null7-7. zmore8-1. Greatest
common divisor8-2. Using Arithmetic Operations8-3. Compound Condition Tests
Using && and ||8-4. Representation of numerical constants8-5. C-style
manipulation of variables9-1. $IFS and whitespace9-2. Timed Input9-3. Once more,
timed input9-4. Timed read9-5. Am I root?9-6. arglist: Listing arguments with $*
and $@9-7. Inconsistent $* and $@ behavior9-8. $* and $@ when $IFS is empty9-9.
Underscore variable9-10. Using declare to type variables9-11. Generating random
numbers9-12. Picking a random card from a deck9-13. Brownian Motion
Simulation9-14. Random between values9-15. Rolling a single die with RANDOM9-16.
Reseeding RANDOM9-17. Pseudorandom numbers, using awk10-1. Inserting a blank
line between paragraphs in a text file10-2. Generating an 8-character "random"
string10-3. Converting graphic file formats, with filename change10-4.
Converting streaming audio files to ogg10-5. Emulating getopt10-6. Alternate
ways of extracting and locating substrings10-7. Using parameter substitution and
error messages10-8. Parameter substitution and "usage" messages10-9. Length of a
variable10-10. Pattern matching in parameter substitution10-11. Renaming file
extensions:10-12. Using pattern matching to parse arbitrary strings10-13.
Matching patterns at prefix or suffix of string11-1. Simple for loops11-2. for
loop with two parameters in each [list] element11-3. Fileinfo: operating on a
file list contained in a variable11-4. Operating on a parameterized file
list11-5. Operating on files with a for loop11-6. Missing in [list] in a for
loop11-7. Generating the [list] in a for loop with command substitution11-8. A
grep replacement for binary files11-9. Listing all users on the system11-10.
Checking all the binaries in a directory for authorship11-11. Listing the
symbolic links in a directory11-12. Symbolic links in a directory, saved to a
file11-13. A C-style for loop11-14. Using efax in batch mode11-15. Simple while
loop11-16. Another while loop11-17. while loop with multiple conditions11-18.
C-style syntax in a while loop11-19. until loop11-20. Nested Loop11-21. Effects
of break and continue in a loop11-22. Breaking out of multiple loop levels11-23.
Continuing at a higher loop level11-24. Using continue N in an actual task11-25.
Using case11-26. Creating menus using case11-27. Using command substitution to
generate the case variable11-28. Simple string matching11-29. Checking for
alphabetic input11-30. Creating menus using select11-31. Creating menus using
select in a function12-1. Stupid script tricks12-2. Generating a variable from a
loop12-3. Finding anagrams15-1. A script that spawns multiple instances of
itself15-2. printf in action15-3. Variable assignment, using read15-4. What
happens when read has no variable15-5. Multi-line input to read15-6. Detecting
the arrow keys15-7. Using read with file redirection15-8. Problems reading from
a pipe15-9. Changing the current working directory15-10. Letting let do
arithmetic.15-11. Showing the effect of eval15-12. Using eval to select among
variables15-13. Echoing the command-line parameters15-14. Forcing a
log-off15-15. A version of rot1315-16. Using set with positional
parameters15-17. Reversing the positional parameters15-18. Reassigning the
positional parameters15-19. "Unsetting" a variable15-20. Using export to pass a
variable to an embedded awk script15-21. Using getopts to read the
options/arguments passed to a script15-22. "Including" a data file15-23. A
(useless) script that sources itself15-24. Effects of exec15-25. A script that
exec's itself15-26. Waiting for a process to finish before proceeding15-27. A
script that kills itself16-1. Using ls to create a table of contents for burning
a CDR disk16-2. Hello or Good-bye16-3. Badname, eliminate file names in current
directory containing bad characters and whitespace.16-4. Deleting a file by its
inode number16-5. Logfile: Using xargs to monitor system log16-6. Copying files
in current directory to another16-7. Killing processes by name16-8. Word
frequency analysis using xargs16-9. Using expr16-10. Using date16-11. Date
calculations16-12. Word Frequency Analysis16-13. Which files are scripts?16-14.
Generating 10-digit random numbers16-15. Using tail to monitor the system
log16-16. Printing out the From lines in stored e-mail messages16-17. Emulating
grep in a script16-18. Crossword puzzle solver16-19. Looking up definitions in
Webster's 1913 Dictionary16-20. Checking words in a list for validity16-21.
toupper: Transforms a file to all uppercase.16-22. lowercase: Changes all
filenames in working directory to lowercase.16-23. du: DOS to UNIX text file
conversion.16-24. rot13: ultra-weak encryption.16-25. Generating "Crypto-Quote"
Puzzles16-26. Formatted file listing.16-27. Using column to format a directory
listing16-28. nl: A self-numbering script.16-29. manview: Viewing formatted
manpages16-30. Using cpio to move a directory tree16-31. Unpacking an rpm
archive16-32. Stripping comments from C program files16-33. Exploring
/usr/X11R6/bin16-34. An "improved" strings command16-35. Using cmp to compare
two files within a script.16-36. basename and dirname16-37. A script that copies
itself in sections16-38. Checking file integrity16-39. Uudecoding encoded
files16-40. Finding out where to report a spammer16-41. Analyzing a spam
domain16-42. Getting a stock quote16-43. Updating FC416-44. Using ssh16-45. A
script that mails itself16-46. Generating prime numbers16-47. Monthly Payment on
a Mortgage16-48. Base Conversion16-49. Invoking bc using a here document16-50.
Calculating PI16-51. Converting a decimal number to hexadecimal16-52.
Factoring16-53. Calculating the hypotenuse of a triangle16-54. Using seq to
generate loop arguments16-55. Letter Count"16-56. Using getopt to parse
command-line options16-57. A script that copies itself16-58. Exercising dd16-59.
Capturing Keystrokes16-60. Preparing a bootable SD card for the Raspberry
Pi16-61. Securely deleting a file16-62. Filename generator16-63. Converting
meters to miles16-64. Using m417-1. Setting a new password17-2. Setting an erase
character17-3. secret password: Turning off terminal echoing17-4. Keypress
detection17-5. Checking a remote server for identd17-6. pidof helps kill a
process17-7. Checking a CD image17-8. Creating a filesystem in a file17-9.
Adding a new hard drive17-10. Using umask to hide an output file from prying
eyes17-11. Backlight: changes the brightness of the (laptop) screen
backlight17-12. killall, from /etc/rc.d/init.d19-1. broadcast: Sends message to
everyone logged in19-2. dummyfile: Creates a 2-line dummy file19-3. Multi-line
message using cat19-4. Multi-line message, with tabs suppressed19-5. Here
document with replaceable parameters19-6. Upload a file pair to Sunsite incoming
directory19-7. Parameter substitution turned off19-8. A script that generates
another script19-9. Here documents and functions19-10. "Anonymous" Here
Document19-11. Commenting out a block of code19-12. A self-documenting
script19-13. Prepending a line to a file19-14. Parsing a mailbox20-1.
Redirecting stdin using exec20-2. Redirecting stdout using exec20-3. Redirecting
both stdin and stdout in the same script with exec20-4. Avoiding a subshell20-5.
Redirected while loop20-6. Alternate form of redirected while loop20-7.
Redirected until loop20-8. Redirected for loop20-9. Redirected for loop (both
stdin and stdout redirected)20-10. Redirected if/then test20-11. Data file
names.data for above examples20-12. Logging events21-1. Variable scope in a
subshell21-2. List User Profiles21-3. Running parallel processes in
subshells22-1. Running a script in restricted mode23-1. Code block redirection
without forking23-2. Redirecting the output of process substitution into a
loop.24-1. Simple functions24-2. Function Taking Parameters24-3. Functions and
command-line args passed to the script24-4. Passing an indirect reference to a
function24-5. Dereferencing a parameter passed to a function24-6. Again,
dereferencing a parameter passed to a function24-7. Maximum of two numbers24-8.
Converting numbers to Roman numerals24-9. Testing large return values in a
function24-10. Comparing two large integers24-11. Real name from username24-12.
Local variable visibility24-13. Demonstration of a simple recursive
function24-14. Another simple demonstration24-15. Recursion, using a local
variable24-16. The Fibonacci Sequence24-17. The Towers of Hanoi25-1. Aliases
within a script25-2. unalias: Setting and unsetting an alias26-1. Using an and
list to test for command-line arguments26-2. Another command-line arg test using
an and list26-3. Using or lists in combination with an and list27-1. Simple
array usage27-2. Formatting a poem27-3. Various array operations27-4. String
operations on arrays27-5. Loading the contents of a script into an array27-6.
Some special properties of arrays27-7. Of empty arrays and empty elements27-8.
Initializing arrays27-9. Copying and concatenating arrays27-10. More on
concatenating arrays27-11. The Bubble Sort27-12. Embedded arrays and indirect
references27-13. The Sieve of Eratosthenes27-14. The Sieve of Eratosthenes,
Optimized27-15. Emulating a push-down stack27-16. Complex array application:
Exploring a weird mathematical series27-17. Simulating a two-dimensional array,
then tilting it28-1. Indirect Variable References28-2. Passing an indirect
reference to awk29-1. Using /dev/tcp for troubleshooting29-2. Playing music29-3.
Finding the process associated with a PID29-4. On-line connect status30-1. Print
the server environment30-2. IP addresses31-1. Hiding the cookie jar31-2. Setting
up a swapfile using /dev/zero31-3. Creating a ramdisk32-1. A buggy script32-2.
Missing keyword32-3. test24: another buggy script32-4. Testing a condition with
an assert32-5. Trapping at exit32-6. Cleaning up after Control-C32-7. A Simple
Implementation of a Progress Bar32-8. Tracing a variable32-9. Running multiple
processes (on an SMP box)34-1. Numerical and string comparison are not
equivalent34-2. Subshell Pitfalls34-3. Piping the output of echo to a read36-1.
shell wrapper36-2. A slightly more complex shell wrapper36-3. A generic shell
wrapper that writes to a logfile36-4. A shell wrapper around an awk script36-5.
A shell wrapper around another awk script36-6. Perl embedded in a Bash
script36-7. Bash and Perl scripts combined36-8. Python embedded in a Bash
script36-9. A script that speaks36-10. A (useless) script that recursively calls
itself36-11. A (useful) script that recursively calls itself36-12. Another
(useful) script that recursively calls itself36-13. A "colorized" address
database36-14. Drawing a box36-15. Echoing colored text36-16. A "horserace"
game36-17. A Progress Bar36-18. Return value trickery36-19. Even more return
value trickery36-20. Passing and returning arrays36-21. Fun with anagrams36-22.
Widgets invoked from a shell script36-23. Test Suite37-1. String expansion37-2.
Indirect variable references - the new way37-3. Simple database application,
using indirect variable referencing37-4. Using arrays and other miscellaneous
trickery to deal four random hands from a deck of cards37-5. A simple address
database37-6. A somewhat more elaborate address database37-7. Testing
characters37-8. Reading N characters37-9. Using a here document to set a
variable37-10. Piping input to a read37-11. Negative array indices37-12.
Negative parameter in string-extraction constructA-1. mailformat: Formatting an
e-mail messageA-2. rn: A simple-minded file renaming utilityA-3. blank-rename:
Renames filenames containing blanksA-4. encryptedpw: Uploading to an ftp site,
using a locally encrypted passwordA-5. copy-cd: Copying a data CDA-6. Collatz
seriesA-7. days-between: Days between two datesA-8. Making a dictionaryA-9.
Soundex conversionA-10. Game of LifeA-11. Data file for Game of LifeA-12.
behead: Removing mail and news message headersA-13. password: Generating random
8-character passwordsA-14. fifo: Making daily backups, using named pipesA-15.
Generating prime numbers using the modulo operatorA-16. tree: Displaying a
directory treeA-17. tree2: Alternate directory tree scriptA-18. string
functions: C-style string functionsA-19. Directory informationA-20. Library of
hash functionsA-21. Colorizing text using hash functionsA-22. More on hash
functionsA-23. Mounting USB keychain storage devicesA-24. Converting to
HTMLA-25. Preserving weblogsA-26. Protecting literal stringsA-27. Unprotecting
literal stringsA-28. Spammer IdentificationA-29. Spammer HuntA-30. Making wget
easier to useA-31. A podcasting scriptA-32. Nightly backup to a firewire HDA-33.
An expanded cd commandA-34. A soundcard setup scriptA-35. Locating split
paragraphs in a text fileA-36. Insertion sortA-37. Standard DeviationA-38. A pad
file generator for shareware authorsA-39. A man page editorA-40. Petals Around
the RoseA-41. Quacky: a Perquackey-type word gameA-42. NimA-43. A command-line
stopwatchA-44. An all-purpose shell scripting homework assignment solutionA-45.
The Knight's TourA-46. Magic SquaresA-47. Fifteen PuzzleA-48. The Towers of
Hanoi, graphic versionA-49. The Towers of Hanoi, alternate graphic versionA-50.
An alternate version of the getopt-simple.sh scriptA-51. The version of the
UseGetOpt.sh example used in the Tab Expansion appendixA-52. Cycling through all
the possible color backgroundsA-53. Morse Code PracticeA-54. Base64
encoding/decodingA-55. Inserting text in a file using sedA-56. The Gronsfeld
CipherA-57. Bingo Number GeneratorA-58. Basics ReviewedA-59. Testing execution
times of various commandsA-60. Associative arrays vs. conventional arrays
(execution times)C-1. Counting Letter OccurrencesJ-1. Completion script for
UseGetOpt.shM-1. Sample .bashrc fileM-2. .bash_profile fileN-1. VIEWDATA.BAT:
DOS Batch FileN-2. viewdata.sh: Shell Script Conversion of VIEWDATA.BATT-1. A
script that generates an ASCII tableT-2. Another ASCII table scriptT-3. A third
ASCII table script, using awk

--------------------------------------------------------------------------------

  Next  Introduction