Shell Scripting Recipes - Springer

Shell Scripting Recipes A Problem-Solution Approach CHRIS F. A. JOHNSON

Shell Scripting Recipes: A Problem-Solution Approach Copyright © 2005 by Chris F. A. Johnson All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN (pbk): 1-59059-471-1 Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1 Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. Lead Editor: Jason Gilmore Technical Reviewer: William Park Editorial Board: Steve Anglin, Dan Appleman, Ewan Buckingham, Gary Cornell, Tony Davis, Jason Gilmore, Jonathan Hassell, Chris Mills, Dominic Shakeshaft, Jim Sumser Assistant Publisher: Grace Wong Project Manager: Kylie Johnston Copy Edit Manager: Nicole LeClerc Copy Editor: Scott Carter Production Manager: Kari Brooks-Copony Compositor and Artist: Kinetic Publishing Services, LLC Proofreader: Linda Seifert Indexer: James Minkin Cover Designer: Kurt Krames Manufacturing Manager: Tom Debolski Distributed to the book trade in the United States by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor, New York, NY 10013, and outside the United States by Springer-Verlag GmbH & Co. KG, Tiergartenstr. 17, 69112 Heidelberg, Germany. In the United States: phone 1-800-SPRINGER, fax 201-348-4505, e-mail [email protected], or visit http://www.springer-ny.com. Outside the United States: fax +49 6221 345229, e-mail [email protected], or visit http://www.springer.de. For information on translations, please contact Apress directly at 2560 Ninth Street, Suite 219, Berkeley, CA 94710. Phone 510-549-5930, fax 510-549-5939, e-mail [email protected], or visit http://www.apress.com. The information in this book is distributed on an “as is” basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work. The source code for this book is available to readers at http://www.apress.com in the Downloads section.

Contents at a Glance About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv About the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi

CHAPTER 1 CHAPTER 2 CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER

3 4 5 6 7 8 9

CHAPTER 10 CHAPTER 11 CHAPTER 12 CHAPTER 13 CHAPTER 14 CHAPTER 15 CHAPTER 16 CHAPTER 17 CHAPTER 18 CHAPTER 19 CHAPTER 20 APPENDIX

The POSIX Shell and Command-Line Utilities . . . . . . . . . . . . . . . . . . . . 1 Playing with Files: Viewing, Manipulating, and Editing Text Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 String Briefs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 What’s in a Word? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Scripting by Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Loose Names Sink Scripts: Bringing Sanity to Filenames . . . . . . . 137 Treading a Righteous PATH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 The Dating Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 Good Housekeeping: Monitoring and Tidying Up File Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 POP Goes the E-Mail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 PostScript: More Than an Afterthought . . . . . . . . . . . . . . . . . . . . . . . . 229 Screenplay: The screen–funcs Library . . . . . . . . . . . . . . . . . . . . . . . . . 251 Backing Up the Drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Aging, Archiving, and Deleting Files . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 Covering All Your Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 Home on the Web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 Taking Care of Business . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 Random Acts of Scripting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 A Smorgasbord of Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 Script Development Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397 Internet Scripting Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407

INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411 iii

Contents About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv About the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi

■CHAPTER 1

The POSIX Shell and Command-Line Utilities . . . . . . . . . . . . . . 1 Shell Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 echo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 printf. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 set. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 shift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 getopts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 eval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 local . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Parameters and Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Positional Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Special Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 standard-vars—A Collection of Useful Variables . . . . . . . . . . . . . . . . . 9 Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Pathname Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Parameter Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 The Bourne Shell Expansions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 POSIX Parameter Expansions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Shell-Specific Expansions, bash2, and ksh93 . . . . . . . . . . . . . . . . . . 15 Shell Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Aliases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Sourcing a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

v

vi

■CONTENTS

Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Functions Are Fast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Command Substitution Is Slow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Using the Functions in This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 standard-funcs: A Collection of Useful Commands . . . . . . . . . . . . . . . . . . . 19 1.1 get_key—Get a Single Keystroke from the User . . . . . . . . . . . . 19 1.2 getline—Prompt User to Enter a Line . . . . . . . . . . . . . . . . . . . . . . 20 1.3 press_any_key—Prompt for a Single Keypress . . . . . . . . . . . . . 21 1.4 menu1—Print a Menu and Execute a Selected Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.5 arg—Prompt for Required Argument If None Supplied . . . . . . . 23 1.6 die—Print Error Message and Exit with Error Status . . . . . . . . . 24 1.7 show_date—Display Date in D[D] MMM YYYY Format . . . . . . . 25 1.8 date_vars—Set Date and Time Variables. . . . . . . . . . . . . . . . . . . 26 1.9 is_num—Is This a Positive Integer? . . . . . . . . . . . . . . . . . . . . . . . 28 1.10 abbrev_num—Abbreviate Large Numbers . . . . . . . . . . . . . . . . 29 1.11 commas—Add Thousands Separators to a Number . . . . . . . . 30 1.12 pr1—Print Arguments, One to a Line . . . . . . . . . . . . . . . . . . . . . 32 1.13 checkdirs—Check for Directories; Create If Necessary . . . . . 33 1.14 checkfiles—Check That a Directory Contains Certain Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 1.15 zpad—Pad a Number with Leading Zeroes . . . . . . . . . . . . . . . . 34 1.16 cleanup—Remove Temporary Files and Reset Terminal on Exit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 The Unix Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 cat: Concatenate Files to the Standard Output . . . . . . . . . . . . . . . . . . 36 sed: A Text Stream Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 awk: Pattern Scanning and Processing Language . . . . . . . . . . . . . . . 37 grep: Print Lines Matching a Regular Expression . . . . . . . . . . . . . . . . 38 date: Show or Set the System Date . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 tr: A Character Translation Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 wc: Count Characters, Words, and Lines in a File . . . . . . . . . . . . . . . 38 file: Determine the File Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 ls: Sort and Provide Details About Files . . . . . . . . . . . . . . . . . . . . . . . . 39 uniq: Remove Consecutive Duplicate Lines. . . . . . . . . . . . . . . . . . . . . 39 sudo: Execute Commands as the Superuser . . . . . . . . . . . . . . . . . . . 40 split: Divide a File into Equal-Sized Pieces . . . . . . . . . . . . . . . . . . . . . 40 which: Show the Full Path to a Command . . . . . . . . . . . . . . . . . . . . . . 40 gs, gv: Render, Convert, or View PostScript and PDF Files . . . . . . . . 40 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

■CONTENTS

■CHAPTER 2

Playing with Files: Viewing, Manipulating, and Editing Text Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 End of the Line for OS Differences: Converting Text Files. . . . . . . . . . . . . . 42 2.1 dos2unix—Convert Windows Text Files to Unix . . . . . . . . . . . . . 42 2.2 unix2dos—Convert a Unix File to Windows . . . . . . . . . . . . . . . . . 43 2.3 mac2unix—Convert Macintosh Files to Unix . . . . . . . . . . . . . . . . 44 2.4 unix2mac—Convert Unix Files to Mac Format . . . . . . . . . . . . . . 44 2.5 dos2mac—Convert Windows Files to Macintosh . . . . . . . . . . . . 45 2.6 mac2dos—Convert Macintosh Files to Windows . . . . . . . . . . . . 45 Displaying Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 2.7 prn—Print File with Line Numbers . . . . . . . . . . . . . . . . . . . . . . . . 46 2.8 prw—Print One Word per Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 2.9 wbl—Sort Words by Length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Formatting File Facts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 2.10 finfo—File Information Display . . . . . . . . . . . . . . . . . . . . . . . . . . 52 2.11 wfreq—Word Frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.12 lfreq—Letter Frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.13 fed—A Simple Batch File Editor . . . . . . . . . . . . . . . . . . . . . . . . . 58 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

■CHAPTER 3

String Briefs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Character Actions: The char-funcs Library . . . . . . . . . . . . . . . . . . . . . . . . . . 61 3.1 chr—Convert a Decimal Number to an ASCII Character . . . . . . 62 3.2 asc—Convert a Character to Its Decimal Equivalent . . . . . . . . . 63 3.3 nxt—Get the Next Character in ASCII Sequence . . . . . . . . . . . . . 64 3.4 upr—Convert Character(s) to Uppercase . . . . . . . . . . . . . . . . . . . 65 3.5 lwr—Convert Character(s) to Lowercase . . . . . . . . . . . . . . . . . . . 67 String Cleaning: The string-funcs Library . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 3.6 sub—Replace First Occurrence of a Pattern . . . . . . . . . . . . . . . . 69 3.7 gsub—Globally Replace a Pattern in a String . . . . . . . . . . . . . . . 70 3.8 repeat—Build a String of a Specified Length . . . . . . . . . . . . . . . 72 3.9 index, rindex—Find Position of One String Within Another . . . . 74 3.10 substr—Extract a Portion of a String . . . . . . . . . . . . . . . . . . . . . 76 3.11 insert_str—Place One String Inside Another . . . . . . . . . . . . . . 78 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

vii

viii

■CONTENTS

■CHAPTER 4

What’s in a Word?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Finding and Massaging Word Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 wf-funcs: WordFinder Function Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 4.1 write_config—Write User’s Information to the Configuration File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 4.2 do_config—Check For and Source Default Configuration File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 4.3 set_sysdict—Select the Dictionary Directory . . . . . . . . . . . . . . . 84 4.4 mkwsig—Sort Letters in a Word . . . . . . . . . . . . . . . . . . . . . . . . . . 85 4.5 wf-clean—Remove Carriage Returns and Accents . . . . . . . . . . 86 4.6 wf-compounds—Squish Compound Words and Save with Lengths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 4.7 wf-setup—Prepare Word and Anagram Lists . . . . . . . . . . . . . . . 89 Playing with Matches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 4.8 wf—Find Words That Match a Pattern . . . . . . . . . . . . . . . . . . . . . 91 4.9 wfb—Find Words That Begin with a Given Pattern . . . . . . . . . . . 93 4.10 wfe—Find Words That End with a Given Pattern . . . . . . . . . . . 94 4.11 wfc—Find Words That Contain a Given Pattern . . . . . . . . . . . . 95 4.12 wfit—Find Words That Fit Together in a Grid . . . . . . . . . . . . . . . 97 4.13 anagram—Find Words That Are Anagrams of a Given Word . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 4.14 aplus—Find Anagrams of a Word with a Letter Added . . . . . 101 4.15 aminus—Remove Each Letter in Turn and Anagram What’s Left . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

■CHAPTER 5

Scripting by Numbers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 The math-funcs Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.1 fpmul—Multiply Decimal Fractions . . . . . . . . . . . . . . . . . . . . . . . 106 5.2 int—Return the Integer Portion of a Decimal Fraction . . . . . . . 110 5.3 round—Round the Argument to the Nearest Integer . . . . . . . . 110 5.4 pow—Raise a Number to Any Given Power . . . . . . . . . . . . . . . . 111 5.5 square—Raise a Number to the Second Power . . . . . . . . . . . . 113 5.6 cube—Raise a Number to the Third Power . . . . . . . . . . . . . . . . 113 5.7 calc—A Simple Command-Line Calculator . . . . . . . . . . . . . . . . 114 Adding and Averaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 5.8 total—Add a List of Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 5.9 mean—Find the Arithmetic Mean of a List of Numbers . . . . . 116 5.10 median—Find the Median of a List of Numbers . . . . . . . . . . . 118 5.11 mode—Find the Number That Appears Most in a List . . . . . . 119

■CONTENTS

5.12 range—Find the Range of a Set of Numbers . . . . . . . . . . . . . 120 5.13 stdev—Finding the Standard Deviation . . . . . . . . . . . . . . . . . . 122 Converting Between Unit Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 5.14 conversion-funcs—Converting Metric Units . . . . . . . . . . . . . . 124 5.15 conversion—A Menu System for Metric Conversion . . . . . . . 131 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

■CHAPTER 6

Loose Names Sink Scripts: Bringing Sanity to Filenames. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 What’s in a Name? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 POSIX Portable Filenames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 OK Filenames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 Functioning Filenames: The filename-funcs Library . . . . . . . . . . . . . . . . . 139 6.1 basename—Extract the Last Element of a Pathname . . . . . . . 139 6.2 dirname—Return All but the Last Element of a Pathname . . . 141 6.3 is_pfname—Check for POSIX Portable Filename . . . . . . . . . . . 142 6.4 is_OKfname—Check Whether a Filename Is Acceptable . . . . 143 6.5 pfname—Convert Nonportable Characters in Filename . . . . . 144 6.6 OKfname—Make a Filename Acceptable . . . . . . . . . . . . . . . . . . 145 6.7 is_whitespc—Does the Filename Contain Whitespace Characters? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 6.8 whitespc—Fix Filenames Containing Whitespace Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 6.9 is_dir—Is This a Directory I See Before Me? . . . . . . . . . . . . . . . 147 6.10 nodoublechar—Remove Duplicate Characters from a String . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 6.11 new_filename—Change Filename to Desired Character Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 6.12 fix_pwd—Fix All the Filenames in the Current Directory . . . 150 6.13 fixfname—Convert Filenames to Sensible Names . . . . . . . . . 152 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

■CHAPTER 7

Treading a Righteous PATH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 The path-funcs Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 7.1 path—Display a User’s PATH . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 7.2 unslash—Remove Extraneous Slashes . . . . . . . . . . . . . . . . . . . 158 7.3 checkpath—Clean Up the PATH Variable . . . . . . . . . . . . . . . . . . 159 7.4 addpath—Add Directories to the PATH Variable . . . . . . . . . . . . 161 7.5 rmpath—Remove One or More Directories from $PATH . . . . . 163 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

ix

x

■CONTENTS

■CHAPTER 8

The Dating Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 The date-funcs Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 8.1 split_date—Divide a Date into Day, Month, and Year . . . . . . . . 166 8.2 is_leap_year—Is There an Extra Day This Year? . . . . . . . . . . . 167 8.3 days_in_month—How Many Days Hath September? . . . . . . . 169 8.4 date2julian—Calculate the Julian Day Number . . . . . . . . . . . . 170 8.5 julian2date—Convert Julian Back to Year, Month, and Day . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 8.6 dateshift—Add or Subtract a Number of Days . . . . . . . . . . . . . 172 8.7 diffdate—Find the Number of Days Between Two Dates . . . . 174 8.8 day_of_week—Find the Day of the Week for Any Date . . . . . . 175 8.9 display_date—Show a Date in Text Format. . . . . . . . . . . . . . . . 176 8.10 parse_date—Decipher Various Forms of Date String . . . . . . 179 8.11 valid_date—Where Was I on November 31st? . . . . . . . . . . . . 183 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

■CHAPTER 9

Good Housekeeping: Monitoring and Tidying Up File Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 9.1 dfcmp—Notify User of Major Changes in Disk Usage . . . . . . . 185 9.2 symfix—Remove Broken Symbolic Links . . . . . . . . . . . . . . . . . 189 9.3 sym2file—Converts Symbolic Links to Regular Files . . . . . . . . 191 9.4 zrm—Remove Empty Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 9.5 undup—Remove Duplicate Files . . . . . . . . . . . . . . . . . . . . . . . . . 194 9.6 lsr—List the Most Recent (or Oldest) Files in a Directory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

■CHAPTER 10 POP Goes the E-Mail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 10.1 popcheck—Display Number of Messages in POP3 Mailbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 10.2 pop3-funcs—Functions for Managing a POP3 Mailbox . . . . 200 10.3 pop3list—Examine a POP3 Mailbox . . . . . . . . . . . . . . . . . . . . . 215 10.4 pop3filter—Delete and Retrieve Messages from a POP3 Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 10.5 pflog—Print a Summary of a Day’s Activity . . . . . . . . . . . . . . . 224 10.6 viewlog—Continuously View the pop3filter Log File . . . . . . . 225 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

■CONTENTS

■CHAPTER 11 PostScript: More Than an Afterthought . . . . . . . . . . . . . . . . . . 229 11.1 ps-labels—Producing Mailing Labels from a List. . . . . . . . . . 231 11.2 ps-envelopes—Print Envelopes from List . . . . . . . . . . . . . . . . 237 11.3 ps-grid—Convert a Crossword Grid to PostScript . . . . . . . . . 242 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

■CHAPTER 12 Screenplay: The screen–funcs Library . . . . . . . . . . . . . . . . . . . 251 12.1 screen-vars—Variables for Screen Manipulation . . . . . . . . . . 252 12.2 set_attr—Set Screen-Printing Attributes . . . . . . . . . . . . . . . . . 254 12.3 set_fg, set_bg, set_fgbg—Set Colors for Printing to the Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 12.4 cls—Clear the Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 12.5 printat—Position Cursor by Row and Column . . . . . . . . . . . . . 257 12.6 put_block_at—Print Lines in a Column Anywhere on the Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 12.7 get_size—Set LINES and COLUMNS Variables . . . . . . . . . . . . 262 12.8 max_length—Find the Length of the Longest Argument . . . 263 12.9 print_block_at—Print a Block of Lines Anywhere on the Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264 12.10 vbar, hbar—Print a Vertical or Horizontal Bar . . . . . . . . . . . . 265 12.11 center—Center a String on N Columns . . . . . . . . . . . . . . . . . 267 12.12 flush_right—Align String with the Right Margin . . . . . . . . . 268 12.13 ruler—Draw a Ruler Across the Width and Height of the Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 12.14 box_block, box_block_at—Print Text Surrounded by a Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 12.15 clear_area, clear_area_at—Clear an Area of the Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 12.16 box_area, box_area_at—Draw a Box Around an Area . . . . 273 12.17 screen-demo—Saving and Redisplaying Areas of the Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

■CHAPTER 13 Backing Up the Drive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 13.1 bu—Backup Directories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 13.2 bin-pack—Fill Fixed-Capacity Directories with Files . . . . . . . 285 13.3 unpack—Extract All Files from One or More Archives . . . . . . 290 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

xi

xii

■CONTENTS

■CHAPTER 14 Aging, Archiving, and Deleting Files . . . . . . . . . . . . . . . . . . . . . . 293 14.1 date-file—Add a Datestamp to a Filename . . . . . . . . . . . . . . . 293 14.2 rmold—Remove Old Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 14.3 keepnewest—Remove All but the Newest or Oldest Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

■CHAPTER 15 Covering All Your Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 15.1 lookup—Find the Corresponding Value for a Key . . . . . . . . . . 304 shdb-funcs: Shell Database Function Library . . . . . . . . . . . . . . . . . . . . . . . 306 15.2 load_db—Import Database into Shell Array . . . . . . . . . . . . . . 306 15.3 split_record—Split a Record into Fields. . . . . . . . . . . . . . . . . . 307 15.4 csv_split—Extract Fields from CSV Records . . . . . . . . . . . . . . 309 15.5 put_record—Assemble a Database Record from an Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310 15.6 put_csv—Assemble Fields into a CSV Record . . . . . . . . . . . . 311 15.7 db-demo—View and Edit a Password File. . . . . . . . . . . . . . . . 312 PhoneBase: A Simple Phone Number Database. . . . . . . . . . . . . . . . . . . . . 317 15.8 ph—Look Up a Phone Number . . . . . . . . . . . . . . . . . . . . . . . . . 318 15.9 phadd—Add an Entry to PhoneBase. . . . . . . . . . . . . . . . . . . . . 319 15.10 phdel—Delete an Entry from PhoneBase . . . . . . . . . . . . . . . 320 15.11 phx—Show ph Search Results in an X Window . . . . . . . . . . 320 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321

■CHAPTER 16 Home on the Web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 Playing with Hypertext: The html-funcs Library . . . . . . . . . . . . . . . . . . . . . 323 16.1 get_element—Extract the First Occurrence of an Element . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 16.2 split_tags—Put Each Tag on Its Own Line . . . . . . . . . . . . . . . . 326 16.3 html-title—Get the Title from an HTML File. . . . . . . . . . . . . . . 327 HTML on the Fly: The cgi-funcs Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 16.4 x2d2—Convert a Two-Digit Hexadecimal Number to Decimal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 16.5 dehex—Convert Hex Strings (%XX) to Characters . . . . . . . . . 330 16.6 filedate—Find and Format the Modification Date of a File. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

■CONTENTS

Creating HTML Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333 16.7 mk-htmlindex—Create an HTML Index . . . . . . . . . . . . . . . . . . 333 16.8 pretext—Create a Wrapper Around a Text File . . . . . . . . . . . . 341 16.9 text2html—Convert a Text File to HTML. . . . . . . . . . . . . . . . . . 342 16.10 demo.cgi—A CGI Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344

■CHAPTER 17 Taking Care of Business . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 17.1 prcalc—A Printing Calculator . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 17.2 gle—Keeping Records Without a Shoebox . . . . . . . . . . . . . . . 350 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360

■CHAPTER 18 Random Acts of Scripting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 The rand-funcs Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 18.1 random—Return One or More Random Integers in a Given Range . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 18.2 toss—Simulate Tossing a Coin . . . . . . . . . . . . . . . . . . . . . . . . . 365 18.3 randstr—Select a String at Random . . . . . . . . . . . . . . . . . . . . . 366 A Random Sampling of Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 18.4 rand-date—Generate Random Dates in ISO Format . . . . . . . 367 18.5 randsort—Print Lines in Random Order . . . . . . . . . . . . . . . . . . 369 18.6 randomword—Generate Random Words According to Format Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370 18.7 dice—Roll a Set of Dice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 18.8 throw—Throw a Pair of Dice . . . . . . . . . . . . . . . . . . . . . . . . . . . 378 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

■CHAPTER 19 A Smorgasbord of Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 19.1 topntail—Remove Top and Bottom Lines from a File . . . . . . . 381 19.2 flocate—Locate Files by Filename Alone . . . . . . . . . . . . . . . . . 383 19.3 sus—Display a POSIX Man Page . . . . . . . . . . . . . . . . . . . . . . . . 384 19.4 cwbw—Count Words Beginning With . . . . . . . . . . . . . . . . . . . . 385 19.5 cci—Configure, Compile, and Install from Tarball . . . . . . . . . 386 19.6 ipaddr—Find a Computer’s Network Address . . . . . . . . . . . . . 387 19.7 ipaddr.cgi—Print the Remote Address of an HTTP Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388 19.8 iprev—Reverse the Order of Digits in an IP Address . . . . . . . 389 19.9 intersperse—Insert a String Between Doubled Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389

xiii

xiv

■CONTENTS

19.10 ll—Use a Pager for a Directory Listing Only If Necessary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390 19.11 name-split—Divide a Person’s Full Name into First, Last, and Middle Names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391 19.12 rot13—Encode or Decode Text . . . . . . . . . . . . . . . . . . . . . . . . 392 19.13 showfstab—Show Information from /etc/fstab . . . . . . . . . . 393 19.14 unique—Remove All Duplicate Lines from a File . . . . . . . . . 395 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

■CHAPTER 20 Script Development Management . . . . . . . . . . . . . . . . . . . . . . . . 397 20.1 script-setup—Prepare the Scripting Environment . . . . . . . . . 397 20.2 cpsh—Install Script and Make Backup Copy . . . . . . . . . . . . . 402 20.3 shgrep—Search Scripts for String or Regular Expression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 20.4 shcat—Display a Shell Script. . . . . . . . . . . . . . . . . . . . . . . . . . . 404 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

■APPENDIX

Internet Scripting Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 Introductions to Shell Scripting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 Intermediate and Advanced Scripting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408 Collections of Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408 Home Pages for Shells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408 Regular Expressions, sed, and awk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408 Miscellaneous Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409 History of the Shell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409

■INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411

About the Author ■CHRIS F. A. JOHNSON was introduced to Unix in 1990 and learned shell scripting because the system he was using had no C compiler. His first major project was a menu-driven, user-extensible database system with a report generator. Chris’s recent shell projects have included a member database, menuing system, and POP3 mail filtering and retrieval. When not pushing shell scripting to the limit, Chris teaches chess and composes cryptic crosswords.

xv

About the Technical Reviewer ■WILLIAM PARK learned Unix by using the original Bourne shell. And, after a long journey through language wilderness, he has come full-circle back to shell. If he’s not busy selling Linux thin clients, he’s busy writing patches for the bash shell, giving other scripting languages a run for their money. He uses Vi so much that even his command line is in Vi mode.

xvii

Acknowledgments I

’d like to thank the Children’s Aid Society Foundation of Metropolitan Toronto, who introduced me to Unix and hired me as their Systems Administrator two weeks later. Their system was a 20MHz 386, with 8MB RAM, 105MB hard drive, 12 serial terminals, 4 serial printers, and a 1200 bps modem. There was no C compiler, so I had to learn shell scripting. And the man pages were in the red binder that now has a permanent place on my bookshelf. Special thanks go to the denizens of the comp.unix.shell newsgroup who showed me how much I didn’t know about shell scripting when I thought I was an accomplished scripter. Special mention goes to Stéphane Chazelas, who catches the most obscure problems; Heiner Stevens, whose SHELLdorado web site is the source of much inspiration; Geoff Clare of the Open Group, who clarifies the POSIX standard; and Laura Fairhead, Bruce Burnett, Barry Margolin, Ed Morton, and many others whose posts have provided many answers and much food for thought. Finally, I’d like to thank Jason Gilmore, Apress’s Open Source editor, who suggested the book, and all the Apress people who guided it to completion.

xix

Introduction W

hen I was introduced to Unix in 1990, it was still the domain of multiuser systems and high-end workstations. Even the i386 system I started with had 12 users logging on concurrently through serial terminals. I had remote access through a blazingly fast 1200 bps modem. Things were changing by the mid-1990s, when systems using the Linux kernel, integrated with GNU utilities and the X Window System, provided a viable alternative to Microsoft Windows. At the same time, computers with the power, memory, and hard drive space to run it came within reach of an individual’s pocketbook. The Internet brought fast and efficient distribution of the new systems and software (and enabled their development in the first place). Unix had arrived on the home computer. The twenty-first century has seen the burgeoning of a new breed of Unix user: the home (or small business) user whose computer experience was previously limited, at most, to Microsoft Windows. Such computers may well be used by only one person. This modern user quite likely has no intention of becoming a system administrator, and just wants to use the computer with as little fuss as possible. If that describes you, I hope this book will be an eye-opener. I want to give you a glimpse of the power of the shell. Shell scripts are not just for system administration; they can automate many tasks and solve many different problems. This book is not a tutorial, but by studying the scripts (after reading Chapter 1), you should gain a good grounding in the techniques of shell programming (and most of them can also be used at the command line). If you are an experienced shell scripter, you may find in these pages the scripts you never got around to writing—perhaps even programs you didn’t think could be implemented as shell scripts. A few here had been on my to-do list for years; writing this book gave me the incentive to develop them. I hope you’ll also find interesting techniques and different ways of doing things.

Why Shell Scripts? The shell is the primary means of communication with the computer. Much can be done with a graphical interface, but many things are difficult or even impossible. Many more tasks are much slower to perform by point-and-click. Shell scripts, on the other hand, use the language that you use to talk to the computer; the same commands you type at the command prompt can be put into a file and used as a script. Shell scripts fall into three basic categories: automation, encapsulation, and computation. Automation scripts are for oft-repeated series of commands. Encapsulation scripts are for storing complex commands under a simple name. Computation scripts are for deriving information from data. The three are not mutually exclusive; indeed, all are forms of automation. In its simplest form, automation is a series of simple commands For example, this script prints some information about the current directory: xxi

xxii

■INTRODUCTION

set -- * files=$# set -- */ dirs=$# echo "Number of files in $PWD: $files (including $dirs directories)" I can put those lines in a file (e.g., $HOME/bin/ff) and turn on execute permissions (chmod +x $HOME/bin/ff). If I execute that script (ff) in the main directory for this book, it produces this output: Number of files in /home/chris/Documents: 156 (including 34 directories) Encapsulation scripts store an obscure, long, or complex command in an easy-toremember name. For instance, I regularly copy a file from my home computer to my web page. For a long time I had to look up the location on my home computer and the destination address every time (Which one had the directory name capitalized?). I finally got smart and put it in a script called ttcp: eval "`date "+DATE=%Y-%m-%d"`" src=$HOME/work/Tuesday-Tips/$DATE.html [email protected]:html/shell/tuesday-tips/ [ -f "$src" ] && scp -p "$src" "$dest" My phone number look-up command, ph, is a very simple computational script that takes a name supplied on the command line and searches for it in a file: egrep -ih "$*" $HOME/.phones Shell scripts range from simple one-liners to complex applications, and they are easily adaptable to different situations. You may very likely find scripts in this book that almost do what you want, and you can modify them to fit the bill exactly.

Which Shells Will Run These Scripts? Among the many shells for Unix operating systems, there are three main varieties: Bourne-type shells, C shells, and specialty shells. This book is limited to the Bourne-type, because these are the most significant for writing scripts. Narrowing the field to a further subset, this book uses POSIXcompliant shells that incorporate features found in the KornShell. This scripting language is backward-compatible with the Bourne shell. Since many shells do not fully conform to the standard, these scripts avoid syntax not available in the majority of otherwise POSIX-compliant shells. Unless otherwise stated, these scripts have been tested with bash version 2 (or version 3), pdksh (the public domain KornShell), KornShell 93, and ash (or /bin/sh on FreeBSD or NetBSD). Some scripts take advantage of features present in bash2+ and KornShell 93. Interactive scripts use the readline library available in Bash. Where feasible, I provide alternative methods to allow implementation in a wider variety of shells; these may be replacement functions or guidelines on how to make the necessary changes.

■INTRODUCTION

Looking Through the Book This collection of scripts goes beyond system administration tools. I provide practical applications and building blocks for many types of programs in these chapters: Chapter 1: The POSIX Shell and Command-Line Utilities presents an overview of the shell and some of the Unix utilities used in the book, and a library of functions used in many scripts throughout the book. Chapter 2: Playing with Files: Viewing, Manipulating, and Editing Text Files puts text files under the microscope and the scalpel. Besides code to convert line endings from one operating system to another, you’ll find scripts for displaying information about files, and an editor for making changes. Chapter 3: String Briefs performs sleight of hand on characters and strings of characters. Chapter 4: What’s in a Word? gives a number of scripts to help in word games: anagrams, words that fit into a crossword grid, and words beginning with, ending with, or containing a given set of characters. Chapter 5: Scripting by Numbers shows how to perform various calculations, including basic arithmetic, averaging, and metric conversion. Chapter 6: Loose Names Sink Scripts: Bringing Sanity to Filenames is a set of scripts for operating on filenames, including removing spaces and other problem characters. Chapter 7: Treading a Righteous PATH presents functions for adding and removing directories from your PATH variable, and for checking that all its components exist and are directories. Chapter 8: The Dating Game has scripts that should be good at least until the year 9999 (now that Y2K behind us), whether for checking the day of the week or the validity of the date. Chapter 9: Good Housekeeping: Monitoring and Tidying Up File Systems provides scripts for monitoring disk usage, cleaning up broken symbolic links, removing empty files, and other housekeeping tasks. Chapter 10: POP Goes the E-Mail has scripts for filtering spam in a remote POP3 mailbox and for viewing the mail. Chapter 11: PostScript: More Than an Afterthought deals with the universal printer language of Unices. In this chapter, you’ll find scripts for creating mailing labels and envelopes, and for printing crossword puzzle grids. Chapter 12: Screenplay: The screen-funcs Library describes methods of using the screen as a canvas rather than a teletypewriter. Chapter 13: Backing Up the Drive presents a script for backing up directories, one for packing them into the fewest possible disks, and one for unpacking any of several archive formats. Chapter 14: Aging, Archiving and Deleting Files contains scripts for adding dates to files and removing old files.

xxiii

xxiv

■INTRODUCTION

Chapter 15: Covering All Your Databases contains a simple but complete phonebook application, functions for working with text databases, and a sample script for editing databases. Chapter 16: Home on the Web where the netizens surf, and the bandwidth’s consumed all day long, is a compendium of tools for creating web sites and extracting information from them. Chapter 17: Taking Care of Business has a printing calculator and a general ledger entry system. Chapter 18: Random Acts of Scripting presents a function for obtaining random numbers in the best way for the shell you are using, even if it doesn’t support the $RANDOM variable. Chapter 19: A Smorgasbord of Scripts that don’t fit anywhere else. Chapter 20: Script Development Management sets up and maintains all your scripts, allowing modification and testing without touching the production versions.

Thinking Outside the Book The scripts in this book are available on the Apress web site (http://www.apress.com) in the download section. The package includes a full installation script. The author has a web site at http://cfaj.freeshell.org and can be reached by e-mail at [email protected]. A list of online scripting resources is provided in the Appendix.