1bfe7c

1bfe7c
Archive-Name: editor-faq/sed
1bfe7c
Posting-Frequency: irregular
1bfe7c
Last-modified: 10 March 2003
1bfe7c
Version: 015
1bfe7c
URL: http://sed.sourceforge.net/sedfaq.html
1bfe7c
Maintainer: Eric Pement (pemente@northpark.edu)
1bfe7c

1bfe7c
                            THE SED FAQ
1bfe7c

1bfe7c
                  Frequently Asked Questions about
1bfe7c
                       sed, the stream editor
1bfe7c

1bfe7c
CONTENTS
1bfe7c

1bfe7c
1. GENERAL INFORMATION
1bfe7c
1.1. Introduction - How this FAQ is organized
1bfe7c
1.2. Latest version of the sed FAQ
1bfe7c
1.3. FAQ revision information
1bfe7c
1.4. How do I add a question/answer to the sed FAQ?
1bfe7c
1.5. FAQ abbreviations
1bfe7c
1.6. Credits and acknowledgements
1bfe7c
1.7. Standard disclaimers
1bfe7c

1bfe7c
2. BASIC SED
1bfe7c
2.1. What is sed?
1bfe7c
2.2. What versions of sed are there, and where can I get them?
1bfe7c

1bfe7c
2.2.1. Free versions
1bfe7c

1bfe7c
2.2.1.1. Unix platforms
1bfe7c
2.2.1.2. OS/2
1bfe7c
2.2.1.3. Microsoft Windows (Win3x, Win9x, WinNT, Win2K)
1bfe7c
2.2.1.4. MS-DOS
1bfe7c
2.2.1.5. CP/M
1bfe7c
2.2.1.6. Macintosh v8 or v9
1bfe7c

1bfe7c
2.2.2. Shareware and Commercial versions
1bfe7c

1bfe7c
2.2.2.1. Unix platforms
1bfe7c
2.2.2.2. OS/2
1bfe7c
2.2.2.3. Windows 95/98, Windows NT, Windows 2000
1bfe7c
2.2.2.4. MS-DOS
1bfe7c

1bfe7c
2.3. Where can I learn to use sed?
1bfe7c

1bfe7c
2.3.1. Books
1bfe7c
2.3.2. Mailing list
1bfe7c
2.3.3. Tutorials, electronic text
1bfe7c
2.3.4. General web and ftp sites
1bfe7c

1bfe7c
3. TECHNICAL
1bfe7c
3.1. More detailed explanation of basic sed
1bfe7c
3.1.1.  Regular expressions on the left side of "s///"
1bfe7c
3.1.2.  Escape characters on the right side of "s///"
1bfe7c
3.1.3.  Substitution switches
1bfe7c
3.2. Common one-line sed scripts. How do I . . . ?
1bfe7c

1bfe7c
      - double/triple-space a file?
1bfe7c
      - convert DOS/Unix newlines?
1bfe7c
      - delete leading/trailing spaces?
1bfe7c
      - do substitutions on all/certain lines?
1bfe7c
      - delete consecutive blank lines?
1bfe7c
      - delete blank lines at the top/end of the file?
1bfe7c

1bfe7c
3.3. Addressing and address ranges
1bfe7c
3.4. Address ranges in GNU sed and HHsed
1bfe7c
3.5. Debugging sed scripts
1bfe7c
3.6. Notes about s2p, the sed-to-perl translator
1bfe7c
3.7. GNU/POSIX extensions to regular expressions
1bfe7c

1bfe7c
4. EXAMPLES
1bfe7c
   ONE-CHARACTER QUESTIONS
1bfe7c
4.1.  How do I insert a newline into the RHS of a substitution?
1bfe7c
4.2.  How do I represent control-codes or non-printable characters?
1bfe7c
4.3.  How do I convert files with toggle characters, like +this+,
1bfe7c
      to look like [i]this[/i]?
1bfe7c

1bfe7c
   CHANGING STRINGS
1bfe7c
4.10. How do I perform a case-insensitive search?
1bfe7c
4.11. How do I match only the first occurrence of a pattern?
1bfe7c
4.12. How do I parse a comma-delimited (CSV) data file?
1bfe7c
4.13. How do I handle fixed-length, columnar data?
1bfe7c
4.14. How do I commify a string of numbers?
1bfe7c
4.15. How do I prevent regex expansion on substitutions?
1bfe7c
4.16. How do I convert a string to all lowercase or capital letters?
1bfe7c

1bfe7c
   CHANGING BLOCKS (consecutive lines)
1bfe7c
4.20. How do I change only one section of a file?
1bfe7c
4.21. How do I delete or change a block of text if the block contains
1bfe7c
      a certain regular expression?
1bfe7c
4.22. How do I locate a paragraph of text if the paragraph contains a
1bfe7c
      certain regular expression?
1bfe7c
4.23. How do I match a block of specific consecutive lines?
1bfe7c
4.23.1.  Try to use a "/range/, /expression/"
1bfe7c
4.23.2.  Try to use a "multi-line\nexpression"
1bfe7c
4.23.3.  Try to use a block of "literal strings"
1bfe7c
4.24. How do I address all the lines between RE1 and RE2, excluding the lines themselves?
1bfe7c
4.25. How do I join two lines if line #1 ends in a [certain string]?
1bfe7c
4.26. How do I join two lines if line #2 begins in a [certain string]?
1bfe7c
4.27. How do I change all paragraphs to long lines?
1bfe7c

1bfe7c
   SHELL AND ENVIRONMENT
1bfe7c
4.30.   How do I read environment variables with sed ...
1bfe7c
4.31.1.   ... on Unix platforms?
1bfe7c
4.31.2.   ... on MS-DOS or 4DOS platforms?
1bfe7c
4.32.   How do I export or pass variables back into the environment ...
1bfe7c
4.32.1.   ... on Unix platforms?
1bfe7c
4.32.2.   ... on MS-DOS or 4DOS platforms?
1bfe7c
4.33.   How do I handle shell quoting in sed?
1bfe7c

1bfe7c
   FILES, DIRECTORIES, AND PATHS
1bfe7c
4.40.  How do I read (insert/add) a file at the top of a textfile?
1bfe7c
4.41.  How do I make substitutions in every file in a directory, or
1bfe7c
        in a complete directory tree?
1bfe7c
4.41.1.   ... ssed solution
1bfe7c
4.41.2.   ... Unix solution
1bfe7c
4.41.3.   ... DOS solution
1bfe7c
4.42.  How do I replace "/some/UNIX/path" in a substitution?
1bfe7c
4.43.  How do I replace "C:\SOME\DOS\PATH" in a substitution?
1bfe7c
4.44.  How do I emulate file-includes, using sed?
1bfe7c

1bfe7c
5. WHY ISN'T THIS WORKING?
1bfe7c
5.1.  Why don't my variables like $var get expanded in my sed script?
1bfe7c
5.2.  I'm using 'p' to print, but I have duplicate lines sometimes.
1bfe7c
5.3.  Why does my DOS version of sed process a file part-way through
1bfe7c
      and then quit?
1bfe7c
5.4.  My RE isn't matching/deleting what I want it to. (Or, "Greedy vs.
1bfe7c
      stingy pattern matching")
1bfe7c
5.5.  What is CSDPMI*B.ZIP and why do I need it?
1bfe7c
5.6.  Where are the man pages for GNU sed?
1bfe7c
5.7.  How do I tell what version of sed I am using?
1bfe7c
5.8.  Does sed issue an exit code?
1bfe7c
5.9.  The 'r' command isn't inserting the file into the text.
1bfe7c
5.10. Why can't I match or delete a newline using the \n escape
1bfe7c
      sequence? Why can't I match 2 or more lines using \n?
1bfe7c
5.11. My script aborts with an error message, "event not found".
1bfe7c

1bfe7c
6. OTHER ISSUES
1bfe7c
6.1.  I have a problem that stumps me. Where can I get help?
1bfe7c
6.2.  How does sed compare with awk, perl, and other utilities?
1bfe7c
6.3.  When should I use sed?
1bfe7c
6.4.  When should I NOT use sed?
1bfe7c
6.5.  When should I ignore sed and use Awk or Perl instead?
1bfe7c
6.6.  Known limitations among sed versions
1bfe7c
6.7.  Known incompatibilities between sed versions
1bfe7c

1bfe7c
6.7.1. Issuing commands from the command line
1bfe7c
6.7.2. Using comments (prefixed by the '#' sign)
1bfe7c
6.7.3. Special syntax in REs
1bfe7c
6.7.4. Word boundaries
1bfe7c
6.7.5. Commands which operate differently
1bfe7c

1bfe7c
7. KNOWN BUGS AMONG SED VERSIONS
1bfe7c
7.1. ssed v3.59
1bfe7c
7.2. GNU sed v4.0 - v4.0.5
1bfe7c
7.3. GNU sed v3.02.80
1bfe7c
7.4. GNU sed v3.02
1bfe7c
7.5. GNU sed v2.05
1bfe7c
7.6. GNU sed v1.18
1bfe7c
7.7. GNU sed v1.03
1bfe7c
7.8. sed v1.6 (Briscoe)
1bfe7c
7.9. sed v1.5 (Helman)
1bfe7c
7.10. sedmod v1.0 (Chen)
1bfe7c
7.11. HP-UX sed
1bfe7c
7.12. SunOS sed v4.1
1bfe7c
7.13. SunOS sed v5.6
1bfe7c
7.14. Ultrix sed v4.3
1bfe7c
7.15. Digital Unix sed
1bfe7c

1bfe7c

1bfe7c
------------------------------
1bfe7c

1bfe7c
1. GENERAL INFORMATION
1bfe7c

1bfe7c
1.1. Introduction - How this FAQ is organized
1bfe7c

1bfe7c
   This FAQ is organized to answer common (and some uncommon)
1bfe7c
   questions about sed, quickly. If you see a term or abbreviation in
1bfe7c
   the examples that seems unclear, see if the term is defined in
1bfe7c
   section 1.5. If not, send your comment to pemente[at]northpark.edu.
1bfe7c

1bfe7c
1.2. Latest version of the sed FAQ
1bfe7c

1bfe7c
   The newest version of the sed FAQ is usually here:
1bfe7c

1bfe7c
       http://sed.sourceforge.net/sedfaq.html (HTML version)
1bfe7c
       http://sed.sourceforge.net/sedfaq.txt  (plain text)
1bfe7c
       http://www.student.northpark.edu/pemente/sed/sedfaq.html
1bfe7c
       http://www.student.northpark.edu/pemente/sed/sedfaq.txt
1bfe7c
       http://www.faqs.org/faqs/editor-faq/sed
1bfe7c
       ftp://rtfm.mit.edu/pub/faqs/editor-faq/sed
1bfe7c

1bfe7c
   Another FAQ file on sed by a different author can be found here:
1bfe7c

1bfe7c
       http://www.dreamwvr.com/sed-info/sed-faq.html
1bfe7c

1bfe7c
1.3. FAQ revision information
1bfe7c

1bfe7c
   In the plaintext version, changes are shown by a vertical bar (|)
1bfe7c
   placed in column 78 of the affected lines. To remove the vertical
1bfe7c
   bars (use double quotes for MS-DOS):
1bfe7c

1bfe7c
     sed 's/  *|$//' sedfaq.txt > sedfaq2.txt
1bfe7c

1bfe7c
   In the HTML version, vertical bars do not appear. New or altered
1bfe7c
   portions of the FAQ are indicated by printing in dark blue type.
1bfe7c

1bfe7c
   In the text version, words needing emphasis may be surrounded by
1bfe7c
   the underscore '_' or the asterisk '*'. In the HTML version, these
1bfe7c
   are changed to italics and boldface, respectively.
1bfe7c

1bfe7c
1.4. How do I add a question/answer to the sed FAQ?
1bfe7c

1bfe7c
   Word your question briefly and send it to pemente[at]northpark.edu,
1bfe7c
   indicating your proposed change. We'll post it on the sed-users
1bfe7c
   mailing list (see section 2.3.2) and discuss it there. If it's
1bfe7c
   good, your contribution will be added to the next edition.
1bfe7c

1bfe7c
1.5. FAQ abbreviations
1bfe7c

1bfe7c
       files = one or more filenames, separated by whitespace
1bfe7c
       gsed  = GNU sed
1bfe7c
       ssed  = super-sed
1bfe7c
       RE    = Regular Expressions supported by sed
1bfe7c
       LHS   = the left-hand side ("find" part) of "s/find/repl/" command
1bfe7c
       RHS   = the right-hand side ("replace" part) of "s/find/repl/" cmd
1bfe7c
       nn+   = version _nn_ or higher (e.g., "15+" = version 1.5 and above)
1bfe7c

1bfe7c
   files: "files" stands for one or more filenames entered on the
1bfe7c
   command line. The names may include any wildcards your shell
1bfe7c
   understands (such as ``zork*'' or ``Aug[4-9].let''). Sed will
1bfe7c
   process each filename passed to it by the shell.
1bfe7c

1bfe7c
   RE: For details on regular expressions, see section 3.1.1., below.
1bfe7c

1bfe7c
1.6. Credits and acknowledgements
1bfe7c

1bfe7c
   Many of the ideas for this FAQ were taken from the Awk FAQ:
1bfe7c
       http://www.faqs.org/faqs/computer-lang/awk/faq/
1bfe7c
       ftp://rtfm.mit.edu/pub/usenet/comp.lang.awk/faq
1bfe7c

1bfe7c
   and from the old Perl FAQ:
1bfe7c
       http://www.perl.com/doc/FAQs/FAQ/oldfaq-html/index.html
1bfe7c

1bfe7c
   The following individuals have contributed significantly to this
1bfe7c
   document, and have provided input and wording suggestions for
1bfe7c
   questions, answers, and script examples. Credit goes to these
1bfe7c
   contributors (in alphabetical order by last name):
1bfe7c

1bfe7c
   Al Aab, Yiorgos Adamopoulos, Paolo Bonzini, Walter Briscoe, Jim
1bfe7c
   Dennis, Carlos Duarte, Otavio Exel, Sven Guckes, Aurelio Jargas,
1bfe7c
   Mark Katz, Toby Kelsey, Eric Pement, Greg Pfeiffer, Ken Pizzini,
1bfe7c
   Niall Smart, Simon Taylor, Peter Tillier, Greg Ubben, Laurent
1bfe7c
   Vogel.
1bfe7c

1bfe7c
1.7. Standard disclaimers
1bfe7c

1bfe7c
   While a serious attempt has been made to ensure the accuracy of the
1bfe7c
   information presented herein, the contributors and maintainers of
1bfe7c
   this document do not claim the absence of errors and make no
1bfe7c
   warranties on the information provided. If you notice any mistakes,
1bfe7c
   please let us know so we can fix it.
1bfe7c

1bfe7c
------------------------------
1bfe7c

1bfe7c
2. BASIC SED
1bfe7c

1bfe7c
2.1. What is sed?
1bfe7c

1bfe7c
   "sed" stands for Stream EDitor. Sed is a non-interactive editor,
1bfe7c
   written by the late Lee E. McMahon in 1973 or 1974. A brief history
1bfe7c
   of sed's origins may be found in an early history of the Unix
1bfe7c
   tools, at <http: www.columbia.edu="" ~rh120="" ch106.x09="">.
1bfe7c

1bfe7c
   Instead of altering a file interactively by moving the cursor on
1bfe7c
   the screen (as with a word processor), the user sends a script of
1bfe7c
   editing instructions to sed, plus the name of the file to edit (or
1bfe7c
   the text to be edited may come as output from a pipe). In this
1bfe7c
   sense, sed works like a filter -- deleting, inserting and changing
1bfe7c
   characters, words, and lines of text. Its range of activity goes
1bfe7c
   from small, simple changes to very complex ones.
1bfe7c

1bfe7c
   Sed reads its input from stdin (Unix shorthand for "standard
1bfe7c
   input," i.e., the console) or from files (or both), and sends the
1bfe7c
   results to stdout ("standard output," normally the console or
1bfe7c
   screen). Most people use sed first for its substitution features.
1bfe7c
   Sed is often used as a find-and-replace tool.
1bfe7c

1bfe7c
     sed 's/Glenn/Harold/g' oldfile >newfile
1bfe7c

1bfe7c
   will replace every occurrence of "Glenn" with the word "Harold",
1bfe7c
   wherever it occurs in the file. The "find" portion is a regular
1bfe7c
   expression ("RE"), which can be a simple word or may contain
1bfe7c
   special characters to allow greater flexibility (for example, to
1bfe7c
   prevent "Glenn" from also matching "Glennon").
1bfe7c

1bfe7c
   My very first use of sed was to add 8 spaces to the left side of a
1bfe7c
   file, so when I printed it, the printing wouldn't begin at the
1bfe7c
   absolute left edge of a piece of paper.
1bfe7c

1bfe7c
     sed 's/^/        /' myfile >newfile   # my first sed script
1bfe7c
     sed 's/^/        /' myfile | lp       # my next sed script
1bfe7c

1bfe7c
   Then I learned that sed could display only one paragraph of a file,
1bfe7c
   beginning at the phrase "and where it came" and ending at the
1bfe7c
   phrase "for all people". My script looked like this:
1bfe7c

1bfe7c
     sed -n '/and where it came/,/for all people/p' myfile
1bfe7c

1bfe7c
   Sed's normal behavior is to print (i.e., display or show on screen)
1bfe7c
   the entire file, including the parts that haven't been altered,
1bfe7c
   unless you use the -n switch. The "-n" stands for "no output". This
1bfe7c
   switch is almost always used in conjunction with a 'p' command
1bfe7c
   somewhere, which says to print only the sections of the file that
1bfe7c
   have been specified. The -n switch with the 'p' command allow for
1bfe7c
   parts of a file to be printed (i.e., sent to the console).
1bfe7c

1bfe7c
   Next, I found that sed could show me only (say) lines 12-18 of a
1bfe7c
   file and not show me the rest. This was very handy when I needed to
1bfe7c
   review only part of a long file and I didn't want to alter it.
1bfe7c

1bfe7c
     # the 'p' stands for print
1bfe7c
     sed -n 12,18p myfile
1bfe7c

1bfe7c
   Likewise, sed could show me everything else BUT those particular
1bfe7c
   lines, without physically changing the file on the disk:
1bfe7c

1bfe7c
     # the 'd' stands for delete
1bfe7c
     sed 12,18d myfile
1bfe7c

1bfe7c
   Sed could also double-space my single-spaced file when it came time
1bfe7c
   to print it:
1bfe7c

1bfe7c
     sed G myfile >newfile
1bfe7c

1bfe7c
   If you have many editing commands (for deleting, adding,
1bfe7c
   substituting, etc.) which might take up several lines, those
1bfe7c
   commands can be put into a separate file and all of the commands in
1bfe7c
   the file applied to file being edited:
1bfe7c

1bfe7c
     #  'script.sed' is the file of commands
1bfe7c
     # 'myfile' is the file being changed
1bfe7c
     sed -f script.sed myfile  # 'script.sed' is the file of commands
1bfe7c

1bfe7c
   It is not our intention to convert this FAQ file into a full-blown
1bfe7c
   sed tutorial (for good tutorials, see section 2.3). Rather, we hope
1bfe7c
   this gives the complete novice a few ideas of how sed can be used.
1bfe7c

1bfe7c
2.2. What versions of sed are there, and where can I get them?
1bfe7c

1bfe7c
2.2.1. Free versions
1bfe7c

1bfe7c
   Note: "Free" does not mean "public domain" nor does it necessarily
1bfe7c
   mean you will never be charged for it. All versions of sed in this
1bfe7c
   section except the CP/M versions are based on the GNU general
1bfe7c
   public license and are "free software" by that standard (for
1bfe7c
   details, see http://www.gnu.org/philosophy/free-sw.html). This
1bfe7c
   means you can get the source code and develop it further.
1bfe7c

1bfe7c
   At the URLs listed in this category, sed binaries or source code
1bfe7c
   can be downloaded and used without fees or license payments.
1bfe7c

1bfe7c
2.2.1.1. Unix platforms
1bfe7c

1bfe7c
   ssed v3.60
1bfe7c
   ssed is the version recommended by the FAQ maintainers, since it
1bfe7c
   shares the same codebase with GNU sed, has the most options, and is
1bfe7c
   free software (you can get the source). Though there were earlier
1bfe7c
   version of ssed distributed, sites for these are not being listed.
1bfe7c

1bfe7c
       http://sed.sourceforge.net/grabbag/ssed
1bfe7c
       http://freshmeat.net/project/sed/
1bfe7c

1bfe7c
   GNU sed v4.0.5
1bfe7c
   This is the latest official version of GNU sed. It offers in-place
1bfe7c
   text replacement as an option switch.
1bfe7c

1bfe7c
       ftp://ftp.gnu.org/pub/gnu/sed/sed-4.0.5.tar.gz
1bfe7c
       http://freshmeat.net/project/sed
1bfe7c

1bfe7c
   BSD multi-byte sed (Japanese)
1bfe7c
   Based on the latest version of GNU sed, which supports multi-byte
1bfe7c
   characters.
1bfe7c

1bfe7c
       ftp://ftp1.freebsd.org/pub/FreeBSD/FreeBSD-stable/packages/Latest/ja-sed.tgz
1bfe7c

1bfe7c
   GNU sed v3.02.80
1bfe7c
   An alpha test release which was the base for the development of
1bfe7c
   ssed and GNU sed v4.0.
1bfe7c

1bfe7c
       ftp://alpha.gnu.org/pub/gnu/sed/sed-3.02.80.tar.gz
1bfe7c

1bfe7c
   GNU sed v3.02a
1bfe7c
   Interim version with most features of GNU sed v3.02.80.
1bfe7c

1bfe7c
   GNU sed v3.02
1bfe7c
       ftp://ftp.gnu.org/pub/gnu/sed/sed-3.02.tar.gz
1bfe7c

1bfe7c
   Precompiled versions:
1bfe7c

1bfe7c
   GNU sed v3.02-8
1bfe7c
   source code and binaries for Debian GNU/Linux
1bfe7c

1bfe7c
       http://www.debian.org/Packages/stable/base/sed.html
1bfe7c

1bfe7c
   For some time, the GNU project <http: www.gnu.org=""> used Eric S.
1bfe7c
   Raymond's version of sed (ESR sed v1.1), but eventually dropped it
1bfe7c
   because it had too many built-in limits. In 1991 Howard Helman
1bfe7c
   modified the GNU/ESR sed and produced a flexible version of sed
1bfe7c
   v1.5 available at several sites (Helman's version permitted things
1bfe7c
   like \<...\> to delimit word boundaries, \xHH to enter hex code and
1bfe7c
   \n to indicate newlines in the replace string). This version did
1bfe7c
   not catch on with the GNU project and their version of sed has
1bfe7c
   moved in a similar but different direction.
1bfe7c

1bfe7c
   sed v1.3, by Eric Steven Raymond (released 4 June 1998)
1bfe7c
       http://catb.org/~esr/sed-1.3.tar.gz
1bfe7c

1bfe7c
   Eric Raymond <esr@snark.thyrsus.com> wrote one of the earliest
1bfe7c
   versions of sed. On his website <http: www.catb.org="" ~esr=""/> which
1bfe7c
   also distributes many freeware utilities he has written or worked
1bfe7c
   on, he describes sed v1.1 this way:
1bfe7c

1bfe7c
   "This is the fast, small sed originally distributed in the GNU
1bfe7c
   toolkit and still distributed with Minix. The GNU people ditched it
1bfe7c
   when they built their own sed around an enhanced regex package --
1bfe7c
   but it's still better for some uses (in particular, faster and less
1bfe7c
   memory-intensive)." (Version 1.3 fixes an unidentified bug and adds
1bfe7c
   the L command to hexdump the current pattern space.)
1bfe7c

1bfe7c
2.2.1.2. OS/2
1bfe7c

1bfe7c
   GNU sed v3.02.80
1bfe7c
       http://www2s.biglobe.ne.jp/~vtgf3mpr/gnu/sed.htm
1bfe7c

1bfe7c
   GNU sed v3.02
1bfe7c
       http://hobbes.nmsu.edu/pub/os2/util/file/sed-3_02-r2-bin.zip # binaries
1bfe7c
       http://hobbes.nmsu.edu/pub/os2/util/file/sed-3_02-r2.zip     # source
1bfe7c

1bfe7c
2.2.1.3. Microsoft Windows (Win3x, Win9x, WinNT, Win2K)
1bfe7c

1bfe7c
   GNU sed v4.0.5
1bfe7c
   32-bit binaries and docs. Precompiled versions not available (yet).
1bfe7c

1bfe7c
   GNU sed v3.02.80
1bfe7c
   32-bit binaries and docs, using DJGPP compiler. For details on new
1bfe7c
   features, see Unix section, above.
1bfe7c

1bfe7c
       http://www.student.northpark.edu/pemente/sed/sed3028a.zip # DOS binaries
1bfe7c
       ftp://alpha.gnu.org/pub/gnu/sed/sed-3.02.80.tar.gz        # source
1bfe7c
       ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed3028b.zip # binaries
1bfe7c
       ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed3028d.zip # docs
1bfe7c
       ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed3028s.zip # source
1bfe7c

1bfe7c
   GNU sed v2.05
1bfe7c
   32-bit binaries, no docs. Requires 80386 DX (SX will not run) and
1bfe7c
   must be run in a DOS window or in a full screen DOS session under
1bfe7c
   Microsoft Windows. Will not run in MS-DOS mode (outside Win/Win95).
1bfe7c
   We recommend using the latest version of GNU sed.
1bfe7c
       http://www.simtel.net/pub/win95/prog/gsed205b.zip
1bfe7c
       ftp://ftp.cdrom.com/pub/simtelnet/win95/prog/gsed205b.zip
1bfe7c

1bfe7c
   GNU sed v1.03
1bfe7c
   modified by Frank Whaley.
1bfe7c

1bfe7c
   This version was part of the "Virtually UN*X" toolset, hosted by
1bfe7c
   itribe.net; that website is now closed. Gsed v1.03 supported Win9x
1bfe7c
   long filenames, as well as hex, decimal, binary, and octal
1bfe7c
   character representations.
1bfe7c

1bfe7c
   The Cygwin toolkit:
1bfe7c
       http://www.cygwin.com
1bfe7c

1bfe7c
   Formerly know as "GNU-Win32 tools." According to their home page,
1bfe7c
   "The Cygwin tools are Win32 ports of the popular GNU development
1bfe7c
   tools for Windows NT, 95 and 98. They function through the use of
1bfe7c
   the Cygwin library which provides a UNIX-like API on top of the
1bfe7c
   Win32 API." The version of sed used is GNU sed v3.02.
1bfe7c

1bfe7c
   Minimalist GNU for Windows (MinGW):
1bfe7c
       http://www.mingw.org
1bfe7c
       http://mingw.sourceforge.net
1bfe7c

1bfe7c
   According to their home page, "MinGW ('Minimalist GNU for Windows')
1bfe7c
   refers to a set of runtime headers, used in building a compiler
1bfe7c
   system based on the GNU GCC and binutils projects. It compiles and
1bfe7c
   links code to be run on Win32 platforms ... MinGW uses Microsoft
1bfe7c
   runtime libraries, distributed with the Windows operating system."
1bfe7c
   The version of sed used is GNU sed v3.02.
1bfe7c

1bfe7c
   sed v1.5 (a/k/a HHsed), by Howard Helman
1bfe7c
   Compiled with Mingw32 for 32-bit environments described above. This
1bfe7c
   version should support Win95 long filenames.
1bfe7c
       http://www.dbnet.ece.ntua.gr/~george/sed/OLD/sed15.exe
1bfe7c
       http://www.student.northpark.edu/pemente/sed/sed15exe.zip
1bfe7c

1bfe7c
2.2.1.4. MS-DOS
1bfe7c

1bfe7c
   sed v1.6 (from HHsed), by Walter Briscoe
1bfe7c

1bfe7c
   This is a forthcoming version, now in beta testing, but with many
1bfe7c
   new features. It corrects all the bugs in sed v1.5, and adds the
1bfe7c
   best features of sedmod v1.0 (below). It is available in 16-bit and
1bfe7c
   32-bit compiled versions for MS-DOS. Sorry, no URLs available yet.
1bfe7c

1bfe7c
   sed v1.5 (a/k/a HHsed), by Howard Helman
1bfe7c
   uncompiled source code (Turbo C)
1bfe7c
       ftp://ftp.simtel.net/pub/simtelnet/msdos/txtutl/sed15.zip
1bfe7c
       ftp://ftp.cdrom.com/pub/simtelnet/msdos/txtutl/sed15.zip
1bfe7c

1bfe7c
   DOS executable and documentation
1bfe7c
       ftp://ftp.simtel.net/pub/simtelnet/msdos/txtutl/sed15x.zip
1bfe7c
       ftp://ftp.cdrom.com/pub/simtelnet/msdos/txtutl/sed15x.zip
1bfe7c

1bfe7c
   sedmod v1.0, by Hern Chen
1bfe7c
       http://www.ptug.org/sed/SEDMOD10.ZIP
1bfe7c
       http://www.student.northpark.edu/pemente/sed/sedmod10.zip
1bfe7c
       ftp://garbo.uwasa.fi/pc/unix/sedmod10.zip
1bfe7c

1bfe7c
   GNU sed v3.02.80
1bfe7c
   See section 2.2.1.3 ("Microsoft Windows"), above.
1bfe7c

1bfe7c
   GNU sed v2.05
1bfe7c
   Does not run under MS-DOS.
1bfe7c

1bfe7c
   GNU sed v1.18
1bfe7c
   32-bit binaries and source, using DJGPP compiler. Requires 80386 SX
1bfe7c
   or better. Also requires 3 CWS*.EXE extenders on the path. See
1bfe7c
   section 5.5 ("What is CSDPMI*B.ZIP and why do I need it?"), below.
1bfe7c
   We recommend using a newer version of GNU sed.
1bfe7c
       http://www.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed118b.zip
1bfe7c
       ftp://ftp.cdrom.com/pub/simtelnet/gnu/djgpp/v2gnu/sed118b.zip
1bfe7c
       http://www.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed118s.zip
1bfe7c
       ftp://ftp.cdrom.com/pub/simtelnet/gnu/djgpp/v2gnu/sed118s.zip
1bfe7c

1bfe7c
   GNU sed v1.06
1bfe7c
   16-bit binaries and source. Should run under any MS-DOS system.
1bfe7c
       http://www.simtel.net/pub/gnu/gnuish/sed106.zip
1bfe7c
       ftp://ftp.cdrom.com/pub/simtelnet/gnu/gnuish/sed106.zip
1bfe7c

1bfe7c
2.2.1.5. CP/M
1bfe7c

1bfe7c
   ssed v2.2, by Chuck A. Forsberg
1bfe7c

1bfe7c
   Written for CP/M, ssed (for "small/stupid stream editor) supports
1bfe7c
   only the a(ppend), c(hange), d(elete) and i(nsert) options, and
1bfe7c
   apparently doesn't support regular expressions. A -u switch will
1bfe7c
   "unsqueeze" compressed files and was used mainly in conjunction
1bfe7c
   with DIF.COM for source code maintenance. (file: ssed22.lbr)
1bfe7c

1bfe7c
   change, by Michael M. Rubenstein
1bfe7c

1bfe7c
   Rubenstein released a version of sed called CHANGE.COM (the
1bfe7c
   TTOOLS.LBR archive member CHANGE.CZM is a "crunched" file).
1bfe7c
   CHANGE.COM supports full RE's except grouping and backreferences,
1bfe7c
   and its only function is global substitution. (file: ttools.lbr)
1bfe7c

1bfe7c
2.2.1.6. Macintosh v8 or v9
1bfe7c

1bfe7c
   Since sed is a command-line utility, it is not customary to think
1bfe7c
   of sed being used on a Mac. Nonetheless, the following instructions
1bfe7c
   from Aurelio Jargas describe the process for running sed on MacOS
1bfe7c
   version version 8 or 9.
1bfe7c

1bfe7c
   (1) Download and install the Apple DiskCopy application
1bfe7c

1bfe7c
       ftp://ftp.apple.com/developer/Development_Kits
1bfe7c

1bfe7c
   (2) Download and install Apple MPW
1bfe7c

1bfe7c
       ftp://ftp.apple.com/developer/Tool_Chest/Core_Mac_OS_Tools/MPW_etc./
1bfe7c

1bfe7c
   (3) Download and expand Matthias Neeracher's GNU sed for MPW. (They
1bfe7c
   seem to have misnumbered the sed filename.)
1bfe7c

1bfe7c
       ftp://sunsite.cnlab-switch.ch/software/platform/macos/src/mpw_c/sed-2.03.sit.bin
1bfe7c

1bfe7c
   (4) Enter the sed-3.02 directory and doubleclick the 'sed' file
1bfe7c

1bfe7c
   (5) MPW Shell will open up. It will be a command window instead of
1bfe7c
   a command line, but sed should work as expected. For example:
1bfe7c

1bfe7c
       echo aa | sed 's/a/Z/g'<enter>
1bfe7c

1bfe7c
   Note that ENTER is different from RETURN on an iMac. Apple *also*
1bfe7c
   has its own version of sed on MPW, called "StreamEdit", with a
1bfe7c
   syntax fairly similar to that of normal sed.
1bfe7c

1bfe7c
2.2.2. Shareware and Commercial versions
1bfe7c

1bfe7c
2.2.2.1. Unix platforms
1bfe7c

1bfe7c
       [ Additional information needed. ]
1bfe7c

1bfe7c
2.2.2.2. OS/2
1bfe7c

1bfe7c
   Hamilton Labs:
1bfe7c
       http://www.hamiltonlabs.com/cshell.htm
1bfe7c

1bfe7c
   A sizable set of Unix/C shell utilities designed for OS/2. Price is
1bfe7c
   $350 in the US, $395 elsewhere, with FedEx shipping, unconditional
1bfe7c
   guarantee, unlimited support and free updates. A demo version of
1bfe7c
   the suite can be downloaded from this site, but a stand-alone copy
1bfe7c
   of sed is not available.
1bfe7c

1bfe7c
2.2.2.3. Windows 95/98, Windows NT, Windows 2000
1bfe7c

1bfe7c
   Hamilton Labs:
1bfe7c
       http://www.hamiltonlabs.com/cshell.htm
1bfe7c

1bfe7c
   A sizable set of Unix/C shell utilities designed for Win9x, WinNT,
1bfe7c
   and Win2K. Price is $350 in the US, $395 elsewhere, with FedEx
1bfe7c
   shipping, unconditional guarantee, unlimited support and free
1bfe7c
   updates. A demo version of the suite can be downloaded from this
1bfe7c
   site, but a stand-alone copy of sed is not available.
1bfe7c

1bfe7c
   Interix:
1bfe7c
       http://www.interix.com
1bfe7c

1bfe7c
   Interix (formerly known as OpenNT) is advertised as "a complete
1bfe7c
   UNIX system environment running natively on Microsoft Windows NT",
1bfe7c
   and is licensed and supported by Softway Systems. It offers over
1bfe7c
   200 Unix utilities, and supports Unix shells, sockets, networking,
1bfe7c
   and more. A single-user edition runs about $200. A free demo or
1bfe7c
   evaluation copy will run for 31 days and then quit; to continue
1bfe7c
   using it, you must purchase the commercial version.
1bfe7c

1bfe7c
   MKS NuTCRACKER Professional
1bfe7c
       http://www.datafocus.com/products/nutc/
1bfe7c

1bfe7c
   A different, yet related product line offered by MKS (Mortice Kern
1bfe7c
   Systems, below); the awkward spelling "NuTCRACKER" is intentional.
1bfe7c
   Various packages offer hundreds of Unix utilities for Win32
1bfe7c
   environments. Sed is not available as a separate product.
1bfe7c

1bfe7c
   UnixDos:
1bfe7c
       http://www.unixdos.com
1bfe7c

1bfe7c
   UnixDos is a suite of 82 Unix utilities ported over to the Windows
1bfe7c
   environments. There are 16-bit versions for Win3.x and 32-bit
1bfe7c
   versions for WinNT/Win95. It is distributed as uncrippled shareware
1bfe7c
   for the first 30 days. After the test period, the utilities will
1bfe7c
   not run and you must pay the registration fee of $50.
1bfe7c

1bfe7c
   Their version of sed supports "\n" in the RHS of expressions, and
1bfe7c
   increases the length of input lines to 10,000 characters. By
1bfe7c
   special arrangement with the owners, persons who want a licensed
1bfe7c
   version of sed *only* (without the other utilities) may pay a
1bfe7c
   license fee of $10.
1bfe7c

1bfe7c
   U/WIN:
1bfe7c
       http://www.research.att.com/sw/tools/uwin/
1bfe7c

1bfe7c
   U/WIN is a suite of Unix utilities created for WinNT and Win95
1bfe7c
   systems. It is owned by AT&T, created by David Korn (author of the
1bfe7c
   Unix korn shell), and is freely distributed only to educational
1bfe7c
   institutions, AT&T employees, or certain researchers; all others
1bfe7c
   must pay a fee after a 90-day evaluation period expires. U/WIN
1bfe7c
   operates best with the NTFS (WinNT file system) but will run in
1bfe7c
   degraded mode with the FAT file system and in further degraded mode
1bfe7c
   under Win95. A minimal installation takes about 25 to 30 megs of
1bfe7c
   disk space. Sed is not available as a separate file for download,
1bfe7c
   but comes with the suite.
1bfe7c

1bfe7c
2.2.2.4. MS-DOS
1bfe7c

1bfe7c
   Mix C/Utilities Toolchest
1bfe7c
       http://www.mixsoftware.com/product/utility.htm
1bfe7c

1bfe7c
   According to their web page, "The C/Utilities Toolchest adds over
1bfe7c
   40 powerful UNIX utilities to your MS-DOS operating system. The
1bfe7c
   result is an environment very similar to UNIX operating systems,
1bfe7c
   yet 100% compatible with MS-DOS programs and commands." The
1bfe7c
   toolchest costs $19.95, with source code available for an
1bfe7c
   additional fee. Mix C's version of sed is not available separately.
1bfe7c

1bfe7c
   MKS (Mortice Kern Systems) Toolkit
1bfe7c
       http://www.mks.com
1bfe7c

1bfe7c
   Sed comes bundled with the MKS Toolkit, which is distributed only
1bfe7c
   as commercial software; it is not available separately.
1bfe7c

1bfe7c
   Thompson Automation Software
1bfe7c
       http://www.tasoft.com
1bfe7c

1bfe7c
   The Thompson Toolkit contains over 100 familiar Unix utilities,
1bfe7c
   including a version of the Unix Korn shell. It runs under MS-DOS,
1bfe7c
   OS/2, Win3.x, Win9x, and WinNT. Sed is one of the utilities, though
1bfe7c
   Thompson is better known for its version of awk for DOS, TAWK. The
1bfe7c
   toolkit runs about $150; sed is not available separately.
1bfe7c

1bfe7c
2.3. Where can I learn to use sed?
1bfe7c

1bfe7c
2.3.1. Books
1bfe7c

1bfe7c
       _Sed & Awk, 2d edition_, by Dale Dougherty & Arnold Robbins
1bfe7c
       (Sebastopol, Calif: O'Reilly and Associates, 1997)
1bfe7c
       ISBN 1-56592-225-5
1bfe7c
       http://www.oreilly.com/catalog/sed2/noframes.html
1bfe7c

1bfe7c
   About 40 percent of this book is devoted to sed, and maybe 50
1bfe7c
   percent is devoted to awk. The other 10 percent covers regexes and
1bfe7c
   concepts common to both tools. If you prefer hard copy, this is
1bfe7c
   definitely the best single place to learn to use sed, including its
1bfe7c
   advanced features.
1bfe7c

1bfe7c
   The first edition is also very useful. Several typos crept into the
1bfe7c
   first printing of the first edition (though if you follow the
1bfe7c
   tutorials closely, you'll recognize them right away). A list of
1bfe7c
   errors from the first printing of _sed & awk_ is available at
1bfe7c
   <http: www.cs.colostate.edu="" ~dzubera="" sedawk.txt="">, and errors in
1bfe7c
   the 2nd are at <http: www.cs.colostate.edu="" ~dzubera="" sedawk2.txt="">,
1bfe7c
   though most of these were corrected in later printings. The second
1bfe7c
   edition tells how POSIX standards have affected these tools and
1bfe7c
   covers the popular GNU versions of sed and awk. Price is about (US)
1bfe7c
   $30.00
1bfe7c

1bfe7c
   -----
1bfe7c

1bfe7c
       _Mastering Regular Expressions, 2d ed.,_ by Jeffrey E. F. Friedl
1bfe7c
       (Sebastopol, Calif: O'Reilly and Associates, 2002)
1bfe7c
       ISBN 0-596-00289-0
1bfe7c
       http://regex.info
1bfe7c
       http://www.oreilly.com/catalog/regex2/
1bfe7c
       http://public.yahoo.com/~jfriedl/regex/ (for the first edition)
1bfe7c

1bfe7c
   Knowing how to use "regular expressions" is essential to effective
1bfe7c
   use of most Unix tools. This book focuses on how regular
1bfe7c
   expressions can be best implemented in utilities such as perl, vi,
1bfe7c
   emacs, and awk, but also touches on sed as well. Friedl's home page
1bfe7c
   (above) gives links to other sites which help students learn to
1bfe7c
   master regular expressions. His site also gives a Perl script for
1bfe7c
   determining a syntactically valid e-mail address, using regexes:
1bfe7c

1bfe7c
       http://public.yahoo.com/~jfriedl/regex/code.html
1bfe7c

1bfe7c
   -----
1bfe7c

1bfe7c
       _Awk und Sed_, by Helmut Herold.
1bfe7c
       (Bonn: Addison-Wesley, 1994; 288 pages)
1bfe7c
       2nd edition to be released in March 2003
1bfe7c
       ISBN 3-8273-2094-1
1bfe7c
       http://www.addison-wesley.de/main/main.asp?page=home/bookdetails&ProductID=37214
1bfe7c

1bfe7c
2.3.2. Mailing list
1bfe7c

1bfe7c
   If you are interested in learning more about sed (its syntax, using
1bfe7c
   regular expressions, etc.) you are welcome to subscribe to a
1bfe7c
   sed-oriented mailing list. In fact, there are two mailing lists
1bfe7c
   about sed: one in English named "sed-users", moderated by Sven
1bfe7c
   Guckes; and one in Portuguese named "sed-BR" (for sed-Brazil),
1bfe7c
   moderated by Aurelio Marinho Jargas. The average volume of mail for
1bfe7c
   "sed-users" is about 35 messages a week; the average volume of mail
1bfe7c
   for "sed-BR" is about 15 messages a week.
1bfe7c

1bfe7c
       sed-BR mailing list:    http://br.groups.yahoo.com/group/sed-br/
1bfe7c
       sed-users mailing list: http://groups.yahoo.com/group/sed-users/
1bfe7c

1bfe7c
   To subscribe to sed-users, send a blank message to:
1bfe7c

1bfe7c
       sed-users-subscribe@yahoogroups.com
1bfe7c

1bfe7c
   To unsubscribe from sed-users, send a blank message to:
1bfe7c

1bfe7c
       sed-users-unsubscribe@yahoogroups.com
1bfe7c

1bfe7c
2.3.3. Tutorials, electronic text
1bfe7c

1bfe7c
   The original users manual for sed, by Lee E. McMahon, from the
1bfe7c
   7th edition UNIX Manual (1978), with the classic "Kubla Khan"
1bfe7c
   example and tutorial, in formatted text format:
1bfe7c
       http://sed.sourceforge.net/grabbag/tutorials/sed_mcmahon.txt
1bfe7c

1bfe7c
   The source code to the preceding manual. Use "troff -ms sed" to
1bfe7c
   print this file properly:
1bfe7c
       http://plan9.bell-labs.com/7thEdMan/vol2/sed
1bfe7c
       http://cm.bell-labs.com/7thEdMan/vol2/sed
1bfe7c

1bfe7c
   "Do It With Sed", by Carlos Duarte
1bfe7c
       http://www.dbnet.ece.ntua.gr/~george/sed/OLD/sedtut_1.html
1bfe7c

1bfe7c
   "Sed: How to use sed, a special editor for modifying files
1bfe7c
   automatically", by Bruce Barnett and General Electric Company
1bfe7c
       http://www.grymoire.com/Unix/Sed.html
1bfe7c

1bfe7c
   U-SEDIT2.ZIP, by Mike Arst (16 June 1990)
1bfe7c
       ftp://ftp.cs.umu.se/pub/pc/u-sedit2.zip
1bfe7c
       ftp://ftp.uni-stuttgart.de/pub/systems/msdos/util/unixlike/u-sedit2.zip
1bfe7c
       ftp://sunsite.icm.edu.pl/vol/wojsyl/garbo/pc/editor/u-sedit2.zip
1bfe7c
       ftp://ftp.sogang.ac.kr/pub/msdos/garbo_pc/editor/u-sedit2.zip
1bfe7c

1bfe7c
   U-SEDIT3.ZIP, by Mike Arst (24 Jan. 1992)
1bfe7c
       http://www.student.northpark.edu/pemente/sed/u-sedit3.zip
1bfe7c
       CompuServe DTPFORUM, "PC DTP Utilities" library, file SEDDOC.ZIP
1bfe7c

1bfe7c
   Another sed FAQ
1bfe7c
       http://www.dreamwvr.com/sed-info/sed-faq.html
1bfe7c

1bfe7c
   sed-tutorial, by Felix von Leitner
1bfe7c
       http://www.math.fu-berlin.de/~leitner/sed/tutorial.html
1bfe7c

1bfe7c
   "Manipulating text with sed," chapter 14 of the SCO OpenServer
1bfe7c
   "Operating System Users Guide"
1bfe7c
       http://ou800doc.caldera.com/SHL_automate/CTOC-Manipulating_text_with_sed.html
1bfe7c

1bfe7c
   "Combining the Bourne-shell, sed and awk in the UNIX environment
1bfe7c
   for language analysis," by Lothar Schmitt and Kiel Christianson.
1bfe7c
   This basic tutorial on the Bourne shell, sed and awk downloads as a
1bfe7c
   71-page PostScript file (compressed to 290K with gzip). You may
1bfe7c
   need to navigate down from the root to get the file.
1bfe7c
       ftp://ftp.u-aizu.ac.jp/u-aizu/doc/Tech-Report/1997/97-2-007.tar.gz
1bfe7c
       available upon request from Lothar Schmitt <lothar@u-aizu.ac.jp>
1bfe7c

1bfe7c
2.3.4. General web and ftp sites
1bfe7c

1bfe7c
       http://sed.sourceforge.net/grabbag             # Collected scripts
1bfe7c
       http://main.rtfiber.com.tw/~changyj/sed/       # Yao-Jen Chang
1bfe7c
       http://www.math.fu-berlin.de/~guckes/sed/      # Sven Guckes
1bfe7c
       http://www.math.fu-berlin.de/~leitner/sed/     # Felix von Leitner
1bfe7c
       http://www.dbnet.ece.ntua.gr/~george/sed/      # Yiorgos Adamopoulos
1bfe7c
       http://www.student.northpark.edu/pemente/sed/  # Eric Pement
1bfe7c

1bfe7c
       http://spacsun.rice.edu/FAQ/sed.html
1bfe7c
       ftp://algos.inesc.pt/pub/users/cdua/scripts.tar.gz (sed and shell scripts)
1bfe7c

1bfe7c
   "Handy One-Liners For Sed", compiled by Eric Pement. A large list
1bfe7c
   of 1-line sed commands which can be executed from the command line.
1bfe7c
       http://sed.sourceforge.net/sed1line.txt
1bfe7c
       http://www.student.northpark.edu/pemente/sed/sed1line.txt
1bfe7c

1bfe7c
   "Handy One-Liners For Sed", translated to Portuguese
1bfe7c
       http://wmaker.lrv.ufsc.br/sed_ptBR.html
1bfe7c

1bfe7c
   The Single UNIX Specification, Version 3 (technical man page)
1bfe7c
       http://www.opengroup.org/onlinepubs/007904975/utilities/sed.html
1bfe7c

1bfe7c
   Getting started with sed
1bfe7c
       http://www.cs.hmc.edu/tech_docs/qref/sed.html
1bfe7c

1bfe7c
   masm to gas converter
1bfe7c
       http://www.delorie.com/djgpp/faq/converting/asm2s-sed.html
1bfe7c

1bfe7c
   mail2html.zip
1bfe7c
       http://www.crispen.org/src/#mail2html
1bfe7c

1bfe7c
   sample uses of sed in batch files and scripts (Benny Pederson)
1bfe7c
       http://users.cybercity.dk/~bse26236/batutil/help/SED.HTM
1bfe7c

1bfe7c
   dc.sed - the most complex and impressive sed script ever written.
1bfe7c
   This sed script by Greg Ubben emulates the Unix dc (desk
1bfe7c
   calculator), including base conversion, exponentiation, square
1bfe7c
   roots, and much more.
1bfe7c
       http://sed.sourceforge.net/grabbag/scripts/dc_overview.htm
1bfe7c

1bfe7c
   If you should find other tutorials or scripts that should be added
1bfe7c
   to this document, please forward the URLs to the FAQ maintainer.
1bfe7c

1bfe7c
------------------------------
1bfe7c

1bfe7c
3. TECHNICAL
1bfe7c

1bfe7c
3.1. More detailed explanation of basic sed
1bfe7c

1bfe7c
   Sed takes a script of editing commands and applies each command, in
1bfe7c
   order, to each line of input. After all the commands have been
1bfe7c
   applied to the first line of input, that line is output. A second
1bfe7c
   input line is taken for processing, and the cycle repeats. Sed
1bfe7c
   scripts can address a single line by line number or by matching a
1bfe7c
   /RE pattern/ on the line. An exclamation mark '!' after a regex
1bfe7c
   ('/RE/!') or line number will select all lines that do NOT match
1bfe7c
   that address. Sed can also address a range of lines in the same
1bfe7c
   manner, using a comma to separate the 2 addresses.
1bfe7c

1bfe7c
     $d               # delete the last line of the file
1bfe7c
     /[0-9]\{3\}/p    # print lines with 3 consecutive digits
1bfe7c
     5!s/ham/cheese/  # except on line 5, replace 'ham' with 'cheese'
1bfe7c
     /awk/!s/aaa/bb/  # unless 'awk' is found, replace 'aaa' with 'bb'
1bfe7c
     17,/foo/d        # delete all lines from line 17 up to 'foo'
1bfe7c

1bfe7c
   Following an address or address range, sed accepts curly braces
1bfe7c
   '{...}' so several commands may be applied to that line or to the
1bfe7c
   lines matched by the address range. On the command line, semicolons
1bfe7c
   ';' separate each instruction and must precede the closing brace.
1bfe7c

1bfe7c
     sed '/Owner:/{s/yours/mine/g;s/your/my/g;s/you/me/g;}' file
1bfe7c

1bfe7c
   Range addresses operate differently depending on which version of
1bfe7c
   sed is used (see section 3.4, below). For further information on
1bfe7c
   using sed, consult the references in section 2.3, above.
1bfe7c

1bfe7c
3.1.1. Regular expressions on the left side of "s///"
1bfe7c

1bfe7c
   All versions of sed support Basic Regular Expressions (BREs). For
1bfe7c
   the syntax of BREs, enter "man ed" at a Unix shell prompt. A
1bfe7c
   technical description of BREs from IEEE POSIX 1003.1-2001 and the
1bfe7c
   Single UNIX Specification Version 3 is available online at:
1bfe7c
   http://www.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap09.html#tag_09_03
1bfe7c

1bfe7c
   Sed normally supports BREs plus '\n' to match a newline in the
1bfe7c
   pattern space, plus '\xREx' as equivalent to '/RE/', where 'x' is any
1bfe7c
   character other than a newline or another backslash.
1bfe7c

1bfe7c
   Some versions of sed support supersets of BREs, or "extended
1bfe7c
   regular expressions", which offer additional metacharacters for
1bfe7c
   increased flexibility. For additional information on extended REs
1bfe7c
   in GNU sed, see sections 3.7 ("GNU/POSIX extensions to regular
1bfe7c
   expressions") and 6.7.3 ("Special syntax in REs"), below.
1bfe7c

1bfe7c
   Though not required by BREs, some versions of sed support \t to
1bfe7c
   represent a TAB, \r for carriage return, \xHH for direct entry of
1bfe7c
   hex codes, and so forth. Other versions of sed do not.
1bfe7c

1bfe7c
   ssed (super-sed) introduced many new features for LHS pattern
1bfe7c
   matching, too many to give here. The complete list is found in
1bfe7c
   section 6.7.3.H ("ssed"), below.
1bfe7c

1bfe7c
3.1.2. Escape characters on the right side of "s///"
1bfe7c

1bfe7c
   The right-hand side (the replacement part) in "s/find/replace/" is
1bfe7c
   almost always a string literal, with no interpolation of these
1bfe7c
   metacharacters:
1bfe7c

1bfe7c
       .   ^   $   [   ]   {   }   (   )  ?   +   *   |
1bfe7c

1bfe7c
   Three things *are* interpolated: ampersand (&), backreferences, and
1bfe7c
   options for special seds. An ampersand on the RHS is replaced by
1bfe7c
   the entire expression matched on the LHS. There is _never_ any
1bfe7c
   reason to use grouping like this:
1bfe7c

1bfe7c
       s/\(some-complex-regex\)/one two \1 three/
1bfe7c

1bfe7c
   since you can do this instead:
1bfe7c

1bfe7c
       s/some-complex-regex/one two & three/
1bfe7c

1bfe7c
   To enter a literal ampersand on the RHS, type '\&'.
1bfe7c

1bfe7c
   Grouping and backreferences: All versions of sed support grouping
1bfe7c
   and backreferences on the LHS and backreferences only on the RHS.
1bfe7c
   Grouping allows a series of characters to be collected in a set,
1bfe7c
   indicating the boundaries of the set with \( and \). Then the set
1bfe7c
   can be designated to be repeated a certain number of times
1bfe7c

1bfe7c
       \(like this\)*   or   \(like this\)\{5,7\}.
1bfe7c

1bfe7c
   Groups can also be nested "\(like \(this\) is here\)" and may
1bfe7c
   contain any valid RE. Backreferences repeat the contents of a
1bfe7c
   particular group, using a backslash and a digit (1-9) for each
1bfe7c
   corresponding group. In other words, "/\(pom\)\1/" is another way
1bfe7c
   of writing "/pompom/". If groups are nested, backreference numbers
1bfe7c
   are counted by matching \( in strict left to right order.  Thus,
1bfe7c
   /..\(the \(word\)\) \("foo"\)../ is matched by the backreference
1bfe7c
   \3. Backreferences can be used in the LHS, the RHS, and in normal
1bfe7c
   RE addressing (see section 3.3).  Thus,
1bfe7c

1bfe7c
       /\(.\)\1\(.\)\2\(.\)\3/;      # matches "bookkeeper"
1bfe7c
       /^\(.\)\(.\)\(.\)\3\2\1$/;    # finds 6-letter palindromes
1bfe7c

1bfe7c
   Seds differ in how they treat invalid backreferences where no
1bfe7c
   corresponding group occurs. To insert a literal ampersand or
1bfe7c
   backslash into the RHS, prefix it with a backslash: \& or \\.
1bfe7c

1bfe7c
   ssed, sed16, and sedmod permit additional options on the RHS. They
1bfe7c
   all support changing part of the replacement string to upper case
1bfe7c
   (\u or \U), lower case (\l or \L), or to end case conversion (\E).
1bfe7c
   Both sed16 and sedmod support awk-style word references ($1, $2,
1bfe7c
   $3, ...) and $0 to insert the entire line before conversion.
1bfe7c

1bfe7c
     echo ab ghi | sed16 "s/.*/$0 - \U$2/"   # prints "ab ghi - GHI"
1bfe7c

1bfe7c
   *Note:* This feature of sed16 and sedmod will break sed scripts which
1bfe7c
   put a dollar sign and digit into the RHS. Though this is an unlikely
1bfe7c
   combination, it's worth remembering if you use other people's scripts.
1bfe7c

1bfe7c
3.1.3.  Substitution switches
1bfe7c

1bfe7c
   Standard versions of sed support 4 main flags or switches which may
1bfe7c
   be added to the end of an "s///" command. They are:
1bfe7c

1bfe7c
       N      - Replace the Nth match of the pattern on the LHS, where
1bfe7c
                N is an integer between 1 and 512. If N is omitted,
1bfe7c
                the default is to replace the first match only.
1bfe7c
       g      - Global replace of all matches to the pattern.
1bfe7c
       p      - Print the results to stdout, even if -n switch is used.
1bfe7c
       w file - Write the pattern space to 'file' if a replacement was
1bfe7c
                done. If the file already exists when the script is
1bfe7c
                executed, it is overwritten. During script execution,
1bfe7c
                w appends to the file for each match.
1bfe7c

1bfe7c
   GNU sed 3.02 and ssed also offer the /I switch for doing a
1bfe7c
   case-insensitive match. For example,
1bfe7c

1bfe7c
     echo ONE TWO | gsed "s/one/unos/I"      # prints "unos TWO"
1bfe7c

1bfe7c
   GNU sed 4.x and ssed add the /M switch, to simplify working with
1bfe7c
   multi-line patterns: when it is used, ^ or $ will match BOL or EOL.
1bfe7c
   \` and \' remain available to match the start and end of pattern
1bfe7c
   space, respectively.
1bfe7c

1bfe7c
   ssed supports two more switches, /S and /X, when its Perl mode is
1bfe7c
   used. They are described in detail in section 6.7.3.H, below.
1bfe7c

1bfe7c
3.1.4. Command-line switches
1bfe7c

1bfe7c
   All versions of sed support two switches, -e and -n. Though sed
1bfe7c
   usually separates multiple commands with semicolons (e.g., "H;d;"),
1bfe7c
   certain commands could not accept a semicolon command separator.
1bfe7c
   These include :labels, 't', and 'b'. These commands had to occur
1bfe7c
   last in a script, separated by -e option switches. For example:
1bfe7c

1bfe7c
     # The 'ta' means jump to label :a if last s/// returns true
1bfe7c
     sed -e :a -e '$!N;s/\n=/ /;ta' -e 'P;D' file
1bfe7c

1bfe7c
   The -n switch turns off sed's default behavior of printing every
1bfe7c
   line. With -n, lines are printed only if explicitly told to. In
1bfe7c
   addition, for certain versions of sed, if an external script begins
1bfe7c
   with "#n" as its first two characters, the output is suppressed
1bfe7c
   (exactly as if -n had been entered on the command line). A list of
1bfe7c
   which versions appears in section 6.7.2., below.
1bfe7c

1bfe7c
   GNU sed 4.x and ssed support additional switches. -l (lowercase L),
1bfe7c
   followed by a number, lets you adjust the default length of the 'l'
1bfe7c
   and 'L' commands (note that these implementations of sed also
1bfe7c
   support an argument to these commands, to tailor the length
1bfe7c
   separately of each occurrence of the command).
1bfe7c

1bfe7c
   -i activates in-place editing (see section 4.41.1, below). -s
1bfe7c
   treats each file as a separate stream: sed by default joins all the
1bfe7c
   files, so $ represents the last line of the last file; 15 means the
1bfe7c
   15th line in the joined stream; and /abc/,/def/ might match across
1bfe7c
   files.
1bfe7c

1bfe7c
   When -s is used, however all addresses refer to single files. For
1bfe7c
   example, $ represents the last line of each input file; 15 means
1bfe7c
   the 15th line of each input file; and /abc/,/def/ will be "reset"
1bfe7c
   (in other words, sed will not execute the commands and start
1bfe7c
   looking for /abc/ again) if a file ends before /def/ has been
1bfe7c
   matched. Note that -i automatically activates this interpretation
1bfe7c
   of addresses.
1bfe7c

1bfe7c
3.2. Common one-line sed scripts
1bfe7c

1bfe7c
   A separate document of over 70 handy "one-line" sed commands is
1bfe7c
   available at
1bfe7c
       http://sed.sourceforge.net/sed1line.txt
1bfe7c

1bfe7c
   Here are several common sed commands for one-line use. MS-DOS users
1bfe7c
   should replace single quotes ('...') with double quotes ("...") in
1bfe7c
   these examples. A specific filename usually follows the script,
1bfe7c
   though the input may also come via piping or redirection.
1bfe7c

1bfe7c
   # Double space a file
1bfe7c
   sed G file
1bfe7c

1bfe7c
   # Triple space a file
1bfe7c
   sed 'G;G' file
1bfe7c

1bfe7c
   # Under UNIX: convert DOS newlines (CR/LF) to Unix format
1bfe7c
   sed 's/.$//' file    # assumes that all lines end with CR/LF
1bfe7c
   sed 's/^M$// file    # in bash/tcsh, press Ctrl-V then Ctrl-M
1bfe7c

1bfe7c
   # Under DOS: convert Unix newlines (LF) to DOS format
1bfe7c
   sed 's/$//' file                     # method 1
1bfe7c
   sed -n p file                        # method 2
1bfe7c

1bfe7c
   # Delete leading whitespace (spaces/tabs) from front of each line
1bfe7c
   # (this aligns all text flush left). '^t' represents a true tab
1bfe7c
   # character. Under bash or tcsh, press Ctrl-V then Ctrl-I.
1bfe7c
   sed 's/^[ ^t]*//' file
1bfe7c

1bfe7c
   # Delete trailing whitespace (spaces/tabs) from end of each line
1bfe7c
   sed 's/[ ^t]*$//' file               # see note on '^t', above
1bfe7c

1bfe7c
   # Delete BOTH leading and trailing whitespace from each line
1bfe7c
   sed 's/^[ ^t]*//;s/[ ^]*$//' file    # see note on '^t', above
1bfe7c

1bfe7c
   # Substitute "foo" with "bar" on each line
1bfe7c
   sed 's/foo/bar/' file        # replaces only 1st instance in a line
1bfe7c
   sed 's/foo/bar/4' file       # replaces only 4th instance in a line
1bfe7c
   sed 's/foo/bar/g' file       # replaces ALL instances within a line
1bfe7c

1bfe7c
   # Substitute "foo" with "bar" ONLY for lines which contain "baz"
1bfe7c
   sed '/baz/s/foo/bar/g' file
1bfe7c

1bfe7c
   # Delete all CONSECUTIVE blank lines from file except the first.
1bfe7c
   # This method also deletes all blank lines from top and end of file.
1bfe7c
   # (emulates "cat -s")
1bfe7c
   sed '/./,/^$/!d' file       # this allows 0 blanks at top, 1 at EOF
1bfe7c
   sed '/^$/N;/\n$/D' file     # this allows 1 blank at top, 0 at EOF
1bfe7c

1bfe7c
   # Delete all leading blank lines at top of file (only).
1bfe7c
   sed '/./,$!d' file
1bfe7c

1bfe7c
   # Delete all trailing blank lines at end of file (only).
1bfe7c
   sed -e :a -e '/^\n*$/{$d;N;};/\n$/ba' file
1bfe7c

1bfe7c
   # If a line ends with a backslash, join the next line to it.
1bfe7c
   sed -e :a -e '/\\$/N; s/\\\n//; ta' file
1bfe7c

1bfe7c
   # If a line begins with an equal sign, append it to the previous
1bfe7c
   # line (and replace the "=" with a single space).
1bfe7c
   sed -e :a -e '$!N;s/\n=/ /;ta' -e 'P;D' file
1bfe7c

1bfe7c
3.3. Addressing and address ranges
1bfe7c

1bfe7c
   Sed commands may have an optional "address" or "address range"
1bfe7c
   prefix. If there is no address or address range given, then the
1bfe7c
   command is applied to all the lines of the input file or text
1bfe7c
   stream. Three commands cannot take an address prefix:
1bfe7c

1bfe7c
      - labels, used to branch or jump within the script
1bfe7c
      - the close brace, '}', which ends the '{' "command"
1bfe7c
      - the '#' comment character, also technically a "command"
1bfe7c

1bfe7c
   An address can be a line number (such as 1, 5, 37, etc.), a regular
1bfe7c
   expression (written in the form /RE/ or \xREx where 'x' is any
1bfe7c
   character other than '\' and RE is the regular expression), or the
1bfe7c
   dollar sign ($), representing the last line of the file. An
1bfe7c
   exclamation mark (!) after an address or address range will apply
1bfe7c
   the command to every line EXCEPT the ones named by the address. A
1bfe7c
   null regex ("//") will be replaced by the last regex which was
1bfe7c
   used. Also, some seds do not support \xREx as regex delimiters.
1bfe7c

1bfe7c
     5d               # delete line 5 only
1bfe7c
     5!d              # delete every line except line 5
1bfe7c
     /RE/s/LHS/RHS/g  # substitute only if RE occurs on the line
1bfe7c
     /^$/b label      # if the line is blank, branch to ':label'
1bfe7c
     /./!b label      # ... another way to write the same command
1bfe7c
     \%.%!b label     # ... yet another way to write this command
1bfe7c
     $!N              # on all lines but the last, get the Next line
1bfe7c

1bfe7c
   Note that an embedded newline can be represented in an address by
1bfe7c
   the symbol \n, but this syntax is needed only if the script puts 2
1bfe7c
   or more lines into the pattern space via the N, G, or other
1bfe7c
   commands. The \n symbol does *not* match the newline at an
1bfe7c
   end-of-line because when sed reads each line into the pattern space
1bfe7c
   for processing, it strips off the trailing newline, processes the
1bfe7c
   line, and adds a newline back when printing the line to standard
1bfe7c
   output. To match the end-of-line, use the '$' metacharacter, as
1bfe7c
   follows:
1bfe7c

1bfe7c
     /tape$/       # matches the word 'tape' at the end of a line
1bfe7c
     /tape$deck/   # matches the word 'tape$deck' with a literal '$'
1bfe7c
     /tape\ndeck/  # matches 'tape' and 'deck' with a newline between
1bfe7c

1bfe7c
   The following sed commands usually accept *only* a single address.
1bfe7c
   All other commands (except labels, '}', and '#') accept both single
1bfe7c
   addresses and address ranges.
1bfe7c

1bfe7c
     =       print to stdout the line number of the current line
1bfe7c
     a       after printing the current line, append "text" to stdout
1bfe7c
     i       before printing the current line, insert "text" to stdout
1bfe7c
     q       quit after the current line is matched
1bfe7c
     r file  prints contents of "file" to stdout after line is matched
1bfe7c

1bfe7c
   Note that we said "usually." If you need to apply the '=', 'a',
1bfe7c
   'i', or 'r' commands to each and every line within an address
1bfe7c
   range, this behavior can be coerced by the use of braces. Thus,
1bfe7c
   "1,9=" is an invalid command, but "1,9{=;}" will print each line
1bfe7c
   number followed by its line for the first 9 lines (and then print
1bfe7c
   the rest of the rest of the file normally).
1bfe7c

1bfe7c
   Address ranges occur in the form
1bfe7c

1bfe7c
       <address1>,<address2>    or    <address1>,<address2>!
1bfe7c

1bfe7c
   where the address can be a line number or a standard /regex/.
1bfe7c
   <address2> can also be a dollar sign, indicating the end of file.
1bfe7c
   Under GNU sed 3.02+, ssed, and sed15+, <address2> may also be a
1bfe7c
   notation of the form +num, indicating the next _num_ lines after
1bfe7c
   <address1> is matched.
1bfe7c

1bfe7c
   Address ranges are:
1bfe7c

1bfe7c
   (1) Inclusive. The range "/From here/,/eternity/" matches all the
1bfe7c
   lines containing "From here" up to and including the line
1bfe7c
   containing "eternity". It will not stop on the line just prior to
1bfe7c
   "eternity". (If you don't like this, see section 4.24.)
1bfe7c

1bfe7c
   (2) Plenary. They always match full lines, not just parts of lines.
1bfe7c
   In other words, a command to change or delete an address range will
1bfe7c
   change or delete whole lines; it won't stop in the middle of a
1bfe7c
   line.
1bfe7c

1bfe7c
   (3) Multi-linear. Address ranges normally match 2 lines or more.
1bfe7c
   The second address will never match the same line the first address
1bfe7c
   did; therefore a valid address range always spans at least two
1bfe7c
   lines, with these exceptions which match only one line:
1bfe7c

1bfe7c
      - if the first address matches the last line of the file
1bfe7c
      - if using the syntax "/RE/,3" and /RE/ occurs only once in the
1bfe7c
        file at line 3 or below
1bfe7c
      - if using HHsed v1.5. See section 3.4.
1bfe7c

1bfe7c
   (4) Minimalist. In address ranges with /regex/ as <address2>, the
1bfe7c
   range "/foo/,/bar/" will stop at the first "bar" it finds, provided
1bfe7c
   that "bar" occurs on a line below "foo". If the word "bar" occurs
1bfe7c
   on several lines below the word "foo", the range will match all the
1bfe7c
   lines from the first "foo" up to the first "bar". It will not
1bfe7c
   continue hopping ahead to find more "bar"s. In other words, address
1bfe7c
   ranges are not "greedy," like regular expressions.
1bfe7c

1bfe7c
   (5) Repeating. An address range will try to match more than one
1bfe7c
   block of lines in a file. However, the blocks cannot nest. In
1bfe7c
   addition, a second match will not "take" the last line of the
1bfe7c
   previous block.  For example, given the following text,
1bfe7c

1bfe7c
       start
1bfe7c
       stop  start
1bfe7c
       stop
1bfe7c

1bfe7c
   the sed command '/start/,/stop/d' will only delete the first two
1bfe7c
   lines. It will not delete all 3 lines.
1bfe7c

1bfe7c
   (6) Relentless. If the address range finds a "start" match but
1bfe7c
   doesn't find a "stop", it will match every line from "start" to the
1bfe7c
   end of the file. Thus, beware of the following behaviors:
1bfe7c

1bfe7c
     /RE1/,/RE2/  # If /RE2/ is not found, matches from /RE1/ to the
1bfe7c
                  # end-of-file.
1bfe7c

1bfe7c
     20,/RE/      # If /RE/ is not found, matches from line 20 to the
1bfe7c
                  # end-of-file.
1bfe7c

1bfe7c
     /RE/,30      # If /RE/ occurs any time after line 30, each
1bfe7c
                  # occurrence will be matched in sed15+, sedmod, and
1bfe7c
                  # GNU sed v3.02+. GNU sed v2.05 and 1.18 will match
1bfe7c
                  # from the 2nd occurrence of /RE/ to the end-of-file.
1bfe7c

1bfe7c
   If these behaviors seem strange, remember that they occur because
1bfe7c
   sed does not look "ahead" in the file. Doing so would stop sed from
1bfe7c
   being a stream editor and have adverse effects on its efficiency.
1bfe7c
   If these behaviors are undesirable, they can be circumvented or
1bfe7c
   corrected by the use of nested testing within braces. The following
1bfe7c
   scripts work under GNU sed 3.02:
1bfe7c

1bfe7c
     # Execute your_commands on range "/RE1/,/RE2/", but if /RE2/ is
1bfe7c
     # not found, do nothing.
1bfe7c
     /RE1/{:a;N;/RE2/!ba;your_commands;}
1bfe7c

1bfe7c
     # Execute your_commands on range "20,/RE/", but if /RE/ is not
1bfe7c
     # found, do nothing.
1bfe7c
     20{:a;N;/RE/!ba;your_commands;}
1bfe7c

1bfe7c
   As a side note, once we've used N to "slurp" lines together to test
1bfe7c
   for the ending expression, the pattern space will have gathered
1bfe7c
   many lines (possibly thousands) together and concatenated them as a
1bfe7c
   single expression, with the \n sequence marking line breaks. The
1bfe7c
   REs *within* the pattern space may have to be modified (e.g., you
1bfe7c
   must write '/\nStart/' instead of '/^Start/' and '/[^\n]*/' instead
1bfe7c
   of '/.*/') and other standard sed commands will be unavailable or
1bfe7c
   difficult to use.
1bfe7c

1bfe7c
     # Execute your_commands on range "/RE/,30", but if /RE/ occurs
1bfe7c
     # on line 31 or later, do not match it.
1bfe7c
     1,30{/RE/,$ your_commands;}
1bfe7c

1bfe7c
   For related suggestions on using address ranges, see sections 4.2,
1bfe7c
   4.15, and 4.19 of this FAQ. Also, note the following section.
1bfe7c

1bfe7c
3.4. Address ranges in GNU sed and HHsed
1bfe7c

1bfe7c
   (1) GNU sed 3.02+, ssed, and sed15+ all support address ranges like:
1bfe7c

1bfe7c
       /regex/,+5
1bfe7c

1bfe7c
   which match /regex/ plus the next 5 lines (or EOF, whichever comes
1bfe7c
   first).
1bfe7c

1bfe7c
   (2) GNU sed v3.02.80 (and above) and ssed support address ranges of:
1bfe7c

1bfe7c
       0,/regex/
1bfe7c

1bfe7c
   as a special case to permit matching /regex/ if it occurs on the
1bfe7c
   first line. This syntax permits a range expression that matches
1bfe7c
   every line from the top of the file to the first instance of
1bfe7c
   /regex/, even if /regex/ is on the first line.
1bfe7c

1bfe7c
   (3) HHsed (sed15) has an exceptional way of implementing
1bfe7c

1bfe7c
       /regex1/,/regex2/
1bfe7c

1bfe7c
   If /RE1/ and /RE2/ both occur on the *same* line, HHsed will match
1bfe7c
   that single line. In other words, an address range block can
1bfe7c
   consist of just one line. HHsed will then look for the next
1bfe7c
   occurrence of /regex1/ to begin the block again.
1bfe7c

1bfe7c
   Every other version of sed (including sed16) requires 2 lines to
1bfe7c
   match an address range, and thus /regex1/ and /regex2/ cannot
1bfe7c
   successfully match just one line. See also the comments at
1bfe7c
   section 7.9.4, below.
1bfe7c

1bfe7c
   (4) BEGIN~STEP selection: ssed and GNU sed (v2.05 and above) offer
1bfe7c
   a form of addressing called "BEGIN~STEP selection". This is *not* a
1bfe7c
   range address, which selects an inclusive block of consecutive
1bfe7c
   lines from /start/ to /finish/. But I think it seems to belong here.
1bfe7c

1bfe7c
   Given an expression of the form "M~N", where M and N are integers,
1bfe7c
   GNU sed and ssed will select every Nth line, beginning at line M.
1bfe7c
   (With gsed v2.05, M had to be less than N, but this restriction is
1bfe7c
   no longer necessary). Both M and N may equal 0 ("0~0" selects every
1bfe7c
   line). These examples illustrate the syntax:
1bfe7c

1bfe7c
     sed '1~3d' file      # delete every 3d line, starting with line 1
1bfe7c
                          # deletes lines 1, 4, 7, 10, 13, 16, ...
1bfe7c

1bfe7c
     sed '0~3d' file      # deletes lines 3, 6, 9, 12, 15, 18, ...
1bfe7c

1bfe7c
     sed -n '2~5p' file   # print every 5th line, starting with line 2
1bfe7c
                          # prints lines 2, 7, 12, 17, 22, 27, ...
1bfe7c

1bfe7c
   (5) Finally, GNU sed v2.05 has a bug in range addressing (see
1bfe7c
   section 7.5), which was fixed in the higher versions.
1bfe7c

1bfe7c

1bfe7c
3.5. Debugging sed scripts
1bfe7c

1bfe7c
   The following two debuggers should make it easier to understand how
1bfe7c
   sed scripts operate. They can save hours of grief when trying to
1bfe7c
   determine the problems with a sed script.
1bfe7c

1bfe7c
   (1) sd (sed debugger), by Brian Hiles
1bfe7c

1bfe7c
   This debugger runs under a Unix shell, is powerful, and is easy to
1bfe7c
   use. sd has conditional breakpoints and spypoints of the pattern
1bfe7c
   space and hold space, on any scope defined by regex match and/or
1bfe7c
   script line number. It can be semi-automated, can save diagnostic
1bfe7c
   reports, and shows potential problems with a sed script before it
1bfe7c
   tries to execute it. The script is robust and requires the Unix
1bfe7c
   shell utilities plus the Bourne shell or Korn shell to execute.
1bfe7c

1bfe7c
       http://sed.sourceforge.net/grabbag/scripts/sd.ksh.txt (2003)
1bfe7c
       http://sed.sourceforge.net/grabbag/scripts/sd.sh.txt  (1998)
1bfe7c

1bfe7c
   (2) sedsed, by Aurelio Jargas
1bfe7c

1bfe7c
   This debugger requires Python to run it, and it uses your own
1bfe7c
   version of sed, whatever that may be. It displays the current input
1bfe7c
   line, the pattern space, and the hold space, before and after each
1bfe7c
   sed command is executed.
1bfe7c

1bfe7c
       http://sedsed.sourceforge.net
1bfe7c

1bfe7c

1bfe7c
3.6. Notes about s2p, the sed-to-perl translator
1bfe7c

1bfe7c
   s2p (sed to perl) is a Perl program to convert sed scripts into the
1bfe7c
   Perl programming language; it is included with many versions of
1bfe7c
   Perl. These problems have been found when using s2p:
1bfe7c

1bfe7c
   (1) Doesn't recognize the semicolon properly after s/// commands.
1bfe7c

1bfe7c
       s/foo/bar/g;
1bfe7c

1bfe7c
   (2) Doesn't trim trailing whitespace after s/// commands. Even lone
1bfe7c
   trailing spaces, without comments, produce an error.
1bfe7c

1bfe7c
   (3) Doesn't handle multiple commands within braces. E.g.,
1bfe7c

1bfe7c
       1,4{=;G;}
1bfe7c

1bfe7c
   will produce perl code with missing braces, and miss the second "G"
1bfe7c
   command as well. In fact, any commands after the first one are
1bfe7c
   missed in the perl output script, and the output perl script will
1bfe7c
   also contain mismatched braces.
1bfe7c

1bfe7c
3.7. GNU/POSIX extensions to regular expressions
1bfe7c

1bfe7c
   GNU sed supports "character classes" in addition to regular
1bfe7c
   character sets, such as [0-9A-F]. Like regular character sets,
1bfe7c
   character classes represent any single character within a set.
1bfe7c

1bfe7c
   "Character classes are a new feature introduced in the POSIX
1bfe7c
   standard. A character class is a special notation for describing
1bfe7c
   lists of characters that have a specific attribute, but where the
1bfe7c
   actual characters themselves can vary from country to country
1bfe7c
   and/or from character set to character set. For example, the notion
1bfe7c
   of what is an alphabetic character differs in the USA and in
1bfe7c
   France." [quoted from the docs for GNU awk v3.1.0.]
1bfe7c

1bfe7c
   Though character classes don't generally conserve space on the
1bfe7c
   line, they help make scripts portable for international use. The
1bfe7c
   equivalent character sets _for U.S. users_ follows:
1bfe7c

1bfe7c
     [[:alnum:]]  - [A-Za-z0-9]     Alphanumeric characters
1bfe7c
     [[:alpha:]]  - [A-Za-z]        Alphabetic characters
1bfe7c
     [[:blank:]]  - [ \x09]         Space or tab characters only
1bfe7c
     [[:cntrl:]]  - [\x00-\x19\x7F] Control characters
1bfe7c
     [[:digit:]]  - [0-9]           Numeric characters
1bfe7c
     [[:graph:]]  - [!-~]           Printable and visible characters
1bfe7c
     [[:lower:]]  - [a-z]           Lower-case alphabetic characters
1bfe7c
     [[:print:]]  - [ -~]           Printable (non-Control) characters
1bfe7c
     [[:punct:]]  - [!-/:-@[-`{-~]  Punctuation characters
1bfe7c
     [[:space:]]  - [ \t\v\f]       All whitespace chars
1bfe7c
     [[:upper:]]  - [A-Z]           Upper-case alphabetic characters
1bfe7c
     [[:xdigit:]] - [0-9a-fA-F]     Hexadecimal digit characters
1bfe7c

1bfe7c
   Note that [[:graph:]] does not match the space " ", but [[:print:]]
1bfe7c
   does. Some character classes may (or may not) match characters in
1bfe7c
   the high ASCII range (ASCII 128-255 or 0x80-0xFF), depending on
1bfe7c
   which C library was used to compile sed. For non-English languages,
1bfe7c
   [[:alpha:]] and other classes may also match high ASCII characters.
1bfe7c

1bfe7c
------------------------------
1bfe7c

1bfe7c
4. EXAMPLES
1bfe7c

1bfe7c
   ONE-CHARACTER QUESTIONS
1bfe7c

1bfe7c
4.1. How do I insert a newline into the RHS of a substitution?
1bfe7c

1bfe7c
   Several versions of sed permit '\n' to be typed directly into the
1bfe7c
   RHS, which is then converted to a newline on output: ssed,
1bfe7c
   gsed302a+, gsed103 (with the -x switch), sed15+, sedmod, and
1bfe7c
   UnixDOS sed. The _easiest_ solution is to use one of these
1bfe7c
   versions.
1bfe7c

1bfe7c
   For other versions of sed, try one of the following:
1bfe7c

1bfe7c
   (a) If typing the sed script from a Bourne shell, use one backslash
1bfe7c
   "\" if the script uses 'single quotes' or two backslashes "\\" if
1bfe7c
   the script requires "double quotes". In the example below, note
1bfe7c
   that the leading '>' on the 2nd line is generated by the shell to
1bfe7c
   prompt the user for more input. The user types in slash,
1bfe7c
   single-quote, and then ENTER to terminate the command:
1bfe7c

1bfe7c
     [sh-prompt]$ echo twolines | sed 's/two/& new\
1bfe7c
     >/'
1bfe7c
     two new
1bfe7c
     lines
1bfe7c
     [bash-prompt]$
1bfe7c

1bfe7c
   (b) Use a script file with one backslash '\' in the script,
1bfe7c
   immediately followed by a newline. This will embed a newline into
1bfe7c
   the "replace" portion. Example:
1bfe7c

1bfe7c
     sed -f newline.sed files
1bfe7c

1bfe7c
     # newline.sed
1bfe7c
     s/twolines/two new\
1bfe7c
     lines/g
1bfe7c

1bfe7c
   Some versions of sed may not need the trailing backslash. If so,
1bfe7c
   remove it.
1bfe7c

1bfe7c
   (c) Insert an unused character and pipe the output through tr:
1bfe7c

1bfe7c
     echo twolines | sed 's/two/& new=/' | tr "=" "\n"   # produces
1bfe7c
     two new
1bfe7c
     lines
1bfe7c

1bfe7c
   (d) Use the "G" command:
1bfe7c

1bfe7c
   G appends a newline, plus the contents of the hold space to the end
1bfe7c
   of the pattern space. If the hold space is empty, a newline is
1bfe7c
   appended anyway. The newline is stored in the pattern space as "\n"
1bfe7c
   where it can be addressed by grouping "\(...\)" and moved in the
1bfe7c
   RHS. Thus, to change the "twolines" example used earlier, the
1bfe7c
   following script will work:
1bfe7c

1bfe7c
     sed '/twolines/{G;s/\(two\)\(lines\)\(\n\)/\1\3\2/;}'
1bfe7c

1bfe7c
   (e) Inserting full lines, not breaking lines up:
1bfe7c

1bfe7c
   If one is not *changing* lines but only inserting complete lines
1bfe7c
   before or after a pattern, the procedure is much easier. Use the
1bfe7c
   "i" (insert) or "a" (append) command, making the alterations by an
1bfe7c
   external script. To insert "This line is new" BEFORE each line
1bfe7c
   matching a regex:
1bfe7c

1bfe7c
     /RE/i This line is new               # HHsed, sedmod, gsed 3.02a
1bfe7c
     /RE/{x;s/$/This line is new/;G;}     # other seds
1bfe7c

1bfe7c
   The two examples above are intended as "one-line" commands entered
1bfe7c
   from the console. If using a sed script, "i\" immediately followed
1bfe7c
   by a literal newline will work on all versions of sed. Furthermore,
1bfe7c
   the command "s/$/This line is new/" will only work if the hold
1bfe7c
   space is already empty (which it is by default).
1bfe7c

1bfe7c
   To append "This line is new" AFTER each line matching a regex:
1bfe7c

1bfe7c
     /RE/a This line is new               # HHsed, sedmod, gsed 3.02a
1bfe7c
     /RE/{G;s/$/This line is new/;}       # other seds
1bfe7c

1bfe7c
   To append 2 blank lines after each line matching a regex:
1bfe7c

1bfe7c
     /RE/{G;G;}                    # assumes the hold space is empty
1bfe7c

1bfe7c
   To replace each line matching a regex with 5 blank lines:
1bfe7c

1bfe7c
     /RE/{s/.*//;G;G;G;G;}         # assumes the hold space is empty
1bfe7c

1bfe7c
   (f) Use the "y///" command if possible:
1bfe7c

1bfe7c
   On some Unix versions of sed (not GNU sed!), though the s///
1bfe7c
   command won't accept '\n' in the RHS, the y/// command does. If
1bfe7c
   your Unix sed supports it, a newline after "aaa" can be inserted
1bfe7c
   this way (which is not portable to GNU sed or other seds):
1bfe7c

1bfe7c
     s/aaa/&~/; y/~/\n/;    # assuming no other '~' is on the line!
1bfe7c

1bfe7c
4.2. How do I represent control-codes or nonprintable characters?
1bfe7c

1bfe7c
   Several versions of sed support the notation \xHH, where "HH" are
1bfe7c
   two hex digits, 00-FF: ssed, GNU sed v3.02.80 and above, GNU sed
1bfe7c
   v1.03, sed16 and sed15 (HHsed). Try to use one of those versions.
1bfe7c

1bfe7c
   Sed is not intended to process binary or object code, and files
1bfe7c
   which contain nulls (0x00) will usually generate errors in most
1bfe7c
   versions of sed. The latest versions of GNU sed and ssed are an
1bfe7c
   exception; they permit nulls in the input files and also in
1bfe7c
   regexes.
1bfe7c

1bfe7c
   On Unix platforms, the 'echo' command may allow insertion of octal
1bfe7c
   or hex values, e.g., `echo "\0nnn"` or `echo -n "\0nnn"`. The echo
1bfe7c
   command may also support syntax like '\\b' or '\\t' for backspace
1bfe7c
   or tab characters. Check the man pages to see what syntax your
1bfe7c
   version of echo supports. Some versions support the following:
1bfe7c

1bfe7c
     # replace 0x1A (32 octal) with ASCII letters
1bfe7c
     sed 's/'`echo "\032"`'/Ctrl-Z/g'
1bfe7c

1bfe7c
     # note the 3 backslashes in the command below
1bfe7c
     sed "s/.`echo \\\b`//g"
1bfe7c

1bfe7c
4.3. How do I convert files with toggle characters, like +this+, to
1bfe7c
look like [i]this[/i]?
1bfe7c

1bfe7c
   Input files, especially message-oriented text files, often contain
1bfe7c
   toggle characters for emphasis, like ~this~, *this*, or =this=. Sed
1bfe7c
   can make the same input pattern produce alternating output each
1bfe7c
   time it is encountered. Typical needs might be to generate HMTL
1bfe7c
   codes or print codes for boldface, italic, or underscore. This
1bfe7c
   script accomodates multiple occurrences of the toggle pattern on
1bfe7c
   the same line, as well as cases where the pattern starts on one
1bfe7c
   line and finishes several lines later, even at the end of the file:
1bfe7c

1bfe7c
     # sed script to convert +this+ to [i]this[/i]
1bfe7c
     :a
1bfe7c
     /+/{ x;        # If "+" is found, switch hold and pattern space
1bfe7c
       /^ON/{       # If "ON" is in the (former) hold space, then ..
1bfe7c
         s///;      # .. delete it
1bfe7c
         x;         # .. switch hold space and pattern space back
1bfe7c
         s|+|[/i]|; # .. turn the next "+" into "[/i]"
1bfe7c
         ba;        # .. jump back to label :a and start over
1bfe7c
       }
1bfe7c
     s/^/ON/;       # Else, "ON" was not in the hold space; create it
1bfe7c
     x;             # Switch hold space and pattern space
1bfe7c
     s|+|[i]|;      # Turn the first "+" into "[i]"
1bfe7c
     ba;            # Branch to label :a to find another pattern
1bfe7c
     }
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
   This script uses the hold space to create a "flag" to indicate
1bfe7c
   whether the toggle is ON or not. We have added remarks to
1bfe7c
   illustrate the script logic, but in most versions of sed remarks
1bfe7c
   are not permitted after 'b'ranch commands or labels.
1bfe7c

1bfe7c
   If you are sure that the +toggle+ characters never cross line
1bfe7c
   boundaries (i.e., never begin on one line and end on another), this
1bfe7c
   script can be reduced to one line:
1bfe7c

1bfe7c
     s|+\([^+][^+]*\)+|[i]\1[/i]|g
1bfe7c

1bfe7c
   If your toggle pattern contains regex metacharacters (such as '*'
1bfe7c
   or perhaps '+' or '?'), remember to quote them with backslashes.
1bfe7c

1bfe7c
   CHANGING STRINGS
1bfe7c

1bfe7c
4.10. How do I perform a case-insensitive search?
1bfe7c

1bfe7c
   Several versions of sed support case-insensitive matching: ssed and
1bfe7c
   GNU sed v3.02+ (with I flag after s/// or /regex/); sedmod with the
1bfe7c
   -i switch; and sed16 (which supports both types of switches).
1bfe7c

1bfe7c
   With other versions of sed, case-insensitive searching is awkward,
1bfe7c
   so people may use awk or perl instead, since these programs have
1bfe7c
   options for case-insensitive searches. In gawk/mawk, use "BEGIN
1bfe7c
   {IGNORECASE=1}" and in perl, "/regex/i". For other seds, here are
1bfe7c
   three solutions:
1bfe7c

1bfe7c
   Solution 1: convert everything to upper case and search normally
1bfe7c

1bfe7c
     # sed script, solution 1
1bfe7c
     h;          # copy the original line to the hold space
1bfe7c
                 # convert the pattern space to solid caps
1bfe7c
     y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/
1bfe7c
                 # now we can search for the word "CARLOS"
1bfe7c
     /CARLOS/ {
1bfe7c
          # add or insert lines. Note: "s/.../.../" will not work
1bfe7c
          # here because we are searching a modified pattern
1bfe7c
          # space and are not printing the pattern space.
1bfe7c
     }
1bfe7c
     x;          # get back the original pattern space
1bfe7c
                 # the original pattern space will be printed
1bfe7c
     #---end of sed script---
1bfe7c

1bfe7c
   Solution 2: search for both cases
1bfe7c

1bfe7c
   Often, proper names will either start with all lower-case ("unix"),
1bfe7c
   with an initial capital letter ("Unix") or occur in solid caps
1bfe7c
   ("UNIX"). There may be no need to search for every possibility.
1bfe7c

1bfe7c
     /UNIX/b match
1bfe7c
     /[Uu]nix/b match
1bfe7c

1bfe7c
   Solution 3: search for all possible cases
1bfe7c

1bfe7c
     # If you must, search for any possible combination
1bfe7c
     /[Ca][Aa][Rr][Ll][Oo][Ss]/ { ... }
1bfe7c

1bfe7c
   Bear in mind that as the pattern length increases, this solution
1bfe7c
   becomes an order of magnitude slower than the one of Solution 1, at
1bfe7c
   least with some implementations of sed.
1bfe7c

1bfe7c
4.11. How do I match only the first occurrence of a pattern?
1bfe7c

1bfe7c
   (1) The general solution is to use GNU sed or ssed, with one of
1bfe7c
   these range expressions. The first script ("print only the first
1bfe7c
   match") works with any version of sed:
1bfe7c

1bfe7c
     sed -n '/RE/{p;q;}' file       # print only the first match
1bfe7c
     sed '0,/RE/{//d;}' file        # delete only the first match
1bfe7c
     sed '0,/RE/s//to_that/' file   # change only the first match
1bfe7c

1bfe7c
   (2) If you cannot use GNU sed and if you *know* the pattern will
1bfe7c
   not occur on the first line, this will work:
1bfe7c

1bfe7c
     sed '1,/RE/{//d;}' file        # delete only the first match
1bfe7c
     sed '1,/RE/s//to_that/' file   # change only the first match
1bfe7c

1bfe7c
   (3) If you cannot use GNU sed and the pattern *might* occur on the
1bfe7c
   first line, use one of the following commands (credit for short GNU
1bfe7c
   script goes to Donald Bruce Stewart):
1bfe7c

1bfe7c
     sed '/RE/{x;/Y/!{s/^/Y/;h;d;};x;}' file       # delete (one way)
1bfe7c
     sed -e '/RE/{d;:a' -e '$!N;$ba' -e '}' file   # delete (another way)
1bfe7c
     sed '/RE/{d;:a;N;$ba;}' file                  # same script, GNU sed
1bfe7c
     sed -e '/RE/{s//to_that/;:a' -e '$!N;$!ba' -e '}' file  # change
1bfe7c

1bfe7c
   Still another solution, using a flag in the hold space. This is
1bfe7c
   portable to all seds and works if the pattern is on the first line:
1bfe7c

1bfe7c
     # sed script to change "foo" to "bar" only on the first occurrence
1bfe7c
     1{x;s/^/first/;x;}
1bfe7c
     1,/foo/{x;/first/s///;x;s/foo/bar/;}
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
4.12. How do I parse a comma-delimited (CSV) data file?
1bfe7c

1bfe7c
   Comma-delimited data files can come in several forms, requiring
1bfe7c
   increasing levels of complexity in parsing and handling. They are
1bfe7c
   often referred to as CSV files (for "comma separated values") and
1bfe7c
   occasionally as SDF files (for "standard data format"). Note that
1bfe7c
   some vendors use "SDF" to refer to variable-length records with
1bfe7c
   comma-separated fields which are "double-quoted" if they contain
1bfe7c
   character values, while other vendors use "SDF" to designate
1bfe7c
   fixed-length records with fixed-length, nonquoted fields! (For help
1bfe7c
   with fixed-length fields, see question 4.23)
1bfe7c

1bfe7c
   The term "CSV" became a de-facto standard when Microsoft Excel used
1bfe7c
   it as an optional output file format.
1bfe7c

1bfe7c
   Here are 4 different forms you may encounter in comma-delimited data:
1bfe7c

1bfe7c
   (a) No quotes, no internal commas
1bfe7c

1bfe7c
       1001,John Smith,PO Box 123,Chicago,IL,60699
1bfe7c
       1002,Mary Jones,320 Main,Denver,CO,84100,
1bfe7c

1bfe7c
   (b) Like (a), with quotes around each field
1bfe7c

1bfe7c
       "1003","John Smith","PO Box 123","Chicago","IL","60699"
1bfe7c
       "1004","Mary Jones","320 Main","Denver","CO","84100"
1bfe7c

1bfe7c
   (c) Like (b), with embedded commas
1bfe7c

1bfe7c
       "1005","Tom Hall, Jr.","61 Ash Ct.","Niles","OH","44446"
1bfe7c
       "1006","Bob Davis","429 Pine, Apt. 5","Boston","MA","02128"
1bfe7c

1bfe7c
   (d) Like (c), with embedded commas and quotes
1bfe7c

1bfe7c
       "1007","Sue "Red" Smith","19 Main","Troy","MI","48055"
1bfe7c
       "1008","Joe "Hey, guy!" Hall","POB 44","Reno","NV","89504"
1bfe7c

1bfe7c
   In each example above, we have 7 fields and 6 commas which function
1bfe7c
   as field separators. Case (c) is a very typical form of these data
1bfe7c
   files, with double quotes used to enclose each field and to protect
1bfe7c
   internal commas (such as "Tom Hall, Jr.") from interpretation as
1bfe7c
   field separators. However, many times the data may include both
1bfe7c
   embedded quotation marks as well as embedded commas, as seen by
1bfe7c
   case (d), above.
1bfe7c

1bfe7c
   Case (d) is the closest to Microsoft CSV format. *However*, the
1bfe7c
   Microsoft CSV format allows embedded newlines within a
1bfe7c
   double-quoted field. If embedded newlines within fields are a
1bfe7c
   possibility for your data, you should consider using something
1bfe7c
   other than sed to work with the data file.
1bfe7c

1bfe7c
   Before handling a comma-delimited data file, make sure that you
1bfe7c
   fully understand its format and check the integrity of the data.
1bfe7c
   Does each line contain the same number of fields? Should certain
1bfe7c
   fields be composed only of numbers or of two-letter state
1bfe7c
   abbreviations in all caps? Sed (or awk or perl) should be used to
1bfe7c
   validate the integrity of the data file before you attempt to alter
1bfe7c
   it or extract particular fields from the file.
1bfe7c

1bfe7c
   After ensuring that each line has a valid number of fields, use sed
1bfe7c
   to locate and modify individual fields, using the \(...\) grouping
1bfe7c
   command where needed.
1bfe7c

1bfe7c
   In case (a):
1bfe7c

1bfe7c
     sed 's/^[^,]*,[^,]*,[^,]*,[^,]*,/.../'
1bfe7c
             ^     ^     ^
1bfe7c
             |     |     |_ 3rd field
1bfe7c
             |     |_______ 2nd field
1bfe7c
             |_____________ 1st field
1bfe7c

1bfe7c
     # Unix script to delete the second field for case (a)
1bfe7c
     sed 's/^\([^,]*\),[^,]*,/\1,,/' file
1bfe7c

1bfe7c
     # Unix script to change field 1 to 9999 for case (a)
1bfe7c
     sed 's/^[^,]*,/9999,/' file
1bfe7c

1bfe7c
   In cases (b) and (c):
1bfe7c

1bfe7c
     sed 's/^"[^"]*","[^"]*","[^"]*","[^"]*",/.../'
1bfe7c
              1st--   2nd--   3rd--   4th--
1bfe7c

1bfe7c
     # Unix script to delete the second field for case (c)
1bfe7c
     sed 's/^\("[^"]*"\),"[^"]*",/\1,"",/' file
1bfe7c

1bfe7c
     # Unix script to change field 1 to 9999 for case (c)
1bfe7c
     sed 's/^"[^"]*",/"9999",/' file
1bfe7c

1bfe7c

1bfe7c
   In case (d):
1bfe7c

1bfe7c
   One way to parse such files is to replace the 3-character field
1bfe7c
   separator "," with an unused character like the tab or vertical
1bfe7c
   bar. (Technically, the field separator is only the comma while the
1bfe7c
   fields are surrounded by "double quotes", but the net _effect_ is
1bfe7c
   that fields are separated by quote-comma-quote, with quote
1bfe7c
   characters added to the beginning and end of each record.) Search
1bfe7c
   your datafile _first_ to make sure that your character appears
1bfe7c
   nowhere in it!
1bfe7c

1bfe7c
     sed -n '/|/p' file        # search for any instance of '|'
1bfe7c
     # if it's not found, we can use the '|' to separate fields
1bfe7c

1bfe7c
   Then replace the 3-character field separator and parse as before:
1bfe7c

1bfe7c
     # sed script to delete the second field for case (d)
1bfe7c
     s/","/|/g;                  # global change of "," to bar
1bfe7c
     s/^\([^|]*\)|[^|]|/\1||/;   # delete 2nd field
1bfe7c
     s/|/","/g;                  # global change of bar back to ","
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
     # sed script to change field 1 to 9999 for case (d)
1bfe7c
     # Remember to accommodate leading and trailing quote marks
1bfe7c
     s/","/|/g;
1bfe7c
     s/^[^|]*|/"9999|/;
1bfe7c
     s/|/","/g;
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
   Note that this technique works only if _each_ and _every_ field is
1bfe7c
   surrounded with double quotes, including empty fields.
1bfe7c

1bfe7c
   The following solution is for more complex examples of (d), such
1bfe7c
   as: not all fields contain "double-quote" marks, or the presence of
1bfe7c
   embedded "double-quote" marks within fields, or extraneous
1bfe7c
   whitespace around field delimiters. (Thanks to Greg Ubben for this
1bfe7c
   script!)
1bfe7c

1bfe7c
     # sed script to convert case (d) to bar-delimited records
1bfe7c
     s/^ *\(.*[^ ]\) *$/|\1|/;
1bfe7c
     s/" *, */"|/g;
1bfe7c
     : loop
1bfe7c
     s/| *\([^",|][^,|]*\) *, */|\1|/g;
1bfe7c
     s/| *, */|\1|/g;
1bfe7c
     t loop
1bfe7c
     s/  *|/|/g;
1bfe7c
     s/|  */|/g;
1bfe7c
     s/^|\(.*\)|$/\1/;
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
   For example, it turns this (which is badly-formed but legal):
1bfe7c

1bfe7c
   first,"",unquoted ,""this" is, quoted " ,, sub "quote" inside, f", lone  " empty:
1bfe7c

1bfe7c
   into this:
1bfe7c

1bfe7c
   first|""|unquoted|""this" is, quoted "||sub "quote" inside|f"|lone  "   empty:
1bfe7c

1bfe7c
   Note that the script preserves the "double-quote" marks, but
1bfe7c
   changes only the commas where they are used as field separators. I
1bfe7c
   have used the vertical bar "|" because it's easier to read, but you
1bfe7c
   may change this to another field separator if you wish.
1bfe7c

1bfe7c
   If your CSV datafile is more complex, it would probably not be
1bfe7c
   worth the effort to write it in sed. For such a case, you should
1bfe7c
   use Perl with a dedicated CSV module (there are at least two recent
1bfe7c
   CSV parsers available from CPAN).
1bfe7c

1bfe7c
4.13. How do I handle fixed-length, columnar data?
1bfe7c

1bfe7c
   Sed handles fixed-length fields via \(grouping\) and backreferences
1bfe7c
   (\1, \2, \3 ...). If we have 3 fields of 10, 25, and 9 characters
1bfe7c
   per field, our sed script might look like so:
1bfe7c

1bfe7c
     s/^\(.\{10\}\)\(.\{25\}\)\(.\{9\}\)/\3\2\1/;  # Change the fields
1bfe7c
        ^^^^^^^^^^^~~~~~~~~~~~==========           #   from 1,2,3 to 3,2,1
1bfe7c
         field #1   field #2   field #3
1bfe7c

1bfe7c
   This is a bit hard to read. By using GNU sed or ssed with the -r
1bfe7c
   switch active, it can look like this:
1bfe7c

1bfe7c
     s/^(.{10})(.{25})(.{9})/\3\2\1/;          # Using the -r switch
1bfe7c

1bfe7c
   To delete a field in sed, use grouping and omit the backreference
1bfe7c
   from the field to be deleted. If the data is long or difficult to
1bfe7c
   work with, use ssed with the -R switch and the /x flag after an s///
1bfe7c
   command, to insert comments and remarks about the fields.
1bfe7c

1bfe7c
   For records with many fields, use GNU awk with the FIELDWIDTHS
1bfe7c
   variable set in the top of the script. For example:
1bfe7c

1bfe7c
     awk 'BEGIN{FIELDWIDTHS = "10 25 9"}; {print $3 $2 $1}' file
1bfe7c

1bfe7c
   This is much easier to read than a similar sed script, especially
1bfe7c
   if there are more than 5 or 6 fields to manipulate.
1bfe7c

1bfe7c
4.14. How do I commify a string of numbers?
1bfe7c

1bfe7c
   Use the simplest script necessary to accomplish your task. As
1bfe7c
   variations of the line increase, the sed script must become more
1bfe7c
   complex to handle additional conditions. Whole numbers are
1bfe7c
   simplest, followed by decimal formats, followed by embedded words.
1bfe7c

1bfe7c
   Case 1: simple strings of whole numbers separated by spaces or
1bfe7c
   commas, with an optional negative sign. To convert this:
1bfe7c

1bfe7c
       4381, -1222333, and 70000: - 44555666 1234567890 words
1bfe7c
       56890  -234567, and 89222  -999777  345888777666 chars
1bfe7c

1bfe7c
   to this:
1bfe7c

1bfe7c
       4,381, -1,222,333, and 70,000: - 44,555,666 1,234,567,890 words
1bfe7c
       56,890  -234,567, and 89,222  -999,777  345,888,777,666 chars
1bfe7c

1bfe7c
   use one of these one-liners:
1bfe7c

1bfe7c
     sed ':a;s/\B[0-9]\{3\}\>/,&/;ta'                      # GNU sed
1bfe7c
     sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'  # other seds
1bfe7c

1bfe7c
   Case 2: strings of numbers which may have an embedded decimal
1bfe7c
   point, separated by spaces or commas, with an optional negative
1bfe7c
   sign. To change this:
1bfe7c

1bfe7c
       4381,  -6555.1212 and 70000,  7.18281828  44906982.071902
1bfe7c
       56890   -2345.7778 and 8.0000:  -49000000 -1234567.89012
1bfe7c

1bfe7c
   to this:
1bfe7c

1bfe7c
       4,381,  -6,555.1212 and 70,000,  7.18281828  44,906,982.071902
1bfe7c
       56,890   -2,345.7778 and 8.0000:  -49,000,000 -1,234,567.89012
1bfe7c

1bfe7c
   use the following command for GNU sed:
1bfe7c

1bfe7c
     sed ':a;s/\(^\|[^0-9.]\)\([0-9]\+\)\([0-9]\{3\}\)/\1\2,\3/g;ta'
1bfe7c

1bfe7c
   and for other versions of sed:
1bfe7c

1bfe7c
     sed -f case2.sed files
1bfe7c

1bfe7c
     # case2.sed
1bfe7c
     s/^/ /;                 # add space to start of line
1bfe7c
     :a
1bfe7c
     s/\( [-0-9]\{1,\}\)\([0-9]\{3\}\)/\1,\2/g
1bfe7c
     ta
1bfe7c
     s/ //;                  # remove space from start of line
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
4.15. How do I prevent regex expansion on substitutions?
1bfe7c

1bfe7c
   Sometimes you want to *match* regular expression metacharacters as
1bfe7c
   literals (e.g., you want to match "[0-9]" or "\n"), to be replaced
1bfe7c
   with something else. The ordinary way to prevent expanding
1bfe7c
   metacharacters is to prefix them with a backslash. Thus, if "\n"
1bfe7c
   matches a newline, "\\n" will match the two-character string of
1bfe7c
   'backslash' followed by 'n'.
1bfe7c

1bfe7c
   But doing this repeatedly can become tedious if there are many
1bfe7c
   regexes. The following script will replace alternating strings of
1bfe7c
   literals, where no character is interpreted as a regex
1bfe7c
   metacharacter:
1bfe7c

1bfe7c
     # filename: sub_quote.sed
1bfe7c
     #   author: Paolo Bonzini
1bfe7c
     # sed script to add backslash to find/replace metacharacters
1bfe7c
     N;                  # add even numbered line to pattern space
1bfe7c
     s,[]/\\$*[],\\&,g;  # quote all of [, ], /, \, $, or *
1bfe7c
     s,^,s/,;            # prepend "s/" to front of pattern space
1bfe7c
     s,$,/,;             # append "/" to end of pattern space
1bfe7c
     s,\n,/,;            # change "\n" to "/", making s/from/to/
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
   Here's a sample of how sub_quote.sed might be used. This example
1bfe7c
   converts typical sed regexes to perl-style regexes. The input file
1bfe7c
   consists of 10 lines:
1bfe7c

1bfe7c
       [0-9]
1bfe7c
       \d
1bfe7c
       [^0-9]
1bfe7c
       \D
1bfe7c
       \+
1bfe7c
       +
1bfe7c
       \?
1bfe7c
       ?
1bfe7c
       \|
1bfe7c
       |
1bfe7c

1bfe7c
   Run the command "sed -f sub_quote.sed input", to transform the
1bfe7c
   input file (above) to 5 lines of output:
1bfe7c

1bfe7c
       s/\[0-9\]/\\d/
1bfe7c
       s/\[^0-9\]/\\D/
1bfe7c
       s/\\+/+/
1bfe7c
       s/\\?/?/
1bfe7c
       s/\\|/|/
1bfe7c

1bfe7c
   The above file is itself a sed script, which can then be used to
1bfe7c
   modify other files.
1bfe7c

1bfe7c
4.16. How do I convert a string to all lowercase or capital letters?
1bfe7c

1bfe7c
   The easiest method is to use a new version of GNU sed, ssed, sedmod
1bfe7c
   or sed16 and employ the \U, \L, or other switches on the right side
1bfe7c
   of an s/// command. For example, to convert any word which begins
1bfe7c
   with "reg" or "exp" into solid capital letters:
1bfe7c

1bfe7c
       sed -r "s/\<(reg|exp)[a-z]+/\U&/g"              # gsed4.+ or ssed
1bfe7c
       sed "s/\<reg[a-z]+ \u&="" g;="" s="" \<exp[a-z]+="" \u&="" g"="" #="" sed16="" and="" sedmod="" <="" code="">
1bfe7c

1bfe7c
   As you can see, sedmod and sed16 do not support alternation (|),
1bfe7c
   but they do support case conversion. If none of these versions of
1bfe7c
   sed are available to you, some sample scripts for this task are
1bfe7c
   available from the Seder's Grab Bag:
1bfe7c

1bfe7c
       http://sed.sourceforge.net/grabbag/scripts
1bfe7c

1bfe7c
   Note that some case conversion scripts are listed under "Filename
1bfe7c
   manipulation" and others are under "Text formatting."
1bfe7c

1bfe7c
   CHANGING BLOCKS (consecutive lines)
1bfe7c

1bfe7c
4.20. How do I change only one section of a file?
1bfe7c

1bfe7c
   You can match a range of lines by line number, by regexes (say, all
1bfe7c
   lines between the words "from" and "until"), or by a combination of
1bfe7c
   the two. For multiple substitutions on the same range, put the
1bfe7c
   command(s) between braces {...}. For example:
1bfe7c

1bfe7c
     # replace only between lines 1 and 20
1bfe7c
     1,20 s/Johnson/White/g
1bfe7c

1bfe7c
     # replace everywhere EXCEPT between lines 1 and 20
1bfe7c
     1,20 !s/Johnson/White/g
1bfe7c

1bfe7c
     # replace only between words "from" and "until". Note the
1bfe7c
     # use of \<....\> as word boundary markers in GNU sed.
1bfe7c
     /from/,/until/ { s/\<red\>/magenta/g; s/\<blue\>/cyan/g; }
1bfe7c

1bfe7c
     # replace only from the words "ENDNOTES:" to the end of file
1bfe7c
     /ENDNOTES:/,$ { s/Schaff/Herzog/g; s/Kraft/Ebbing/g; }
1bfe7c

1bfe7c
   For technical details on using address ranges, see section 3.3
1bfe7c
   ("Addressing and Address ranges").
1bfe7c

1bfe7c
4.21. How do I delete or change a block of text if the block contains
1bfe7c
      a certain regular expression?
1bfe7c

1bfe7c
   The following deletes the block between 'start' and 'end'
1bfe7c
   inclusively, if and only if the block contains the string
1bfe7c
   'regex'. Written by Russell Davies, with additional comments:
1bfe7c

1bfe7c
     # sed script to delete a block if /regex/ matches inside it
1bfe7c
     :t
1bfe7c
     /start/,/end/ {    # For each line between these block markers..
1bfe7c
        /end/!{         #   If we are not at the /end/ marker
1bfe7c
           $!{          #     nor the last line of the file,
1bfe7c
              N;        #     add the Next line to the pattern space
1bfe7c
              bt
1bfe7c
           }            #   and branch (loop back) to the :t label.
1bfe7c
        }               # This line matches the /end/ marker.
1bfe7c
        /regex/d;       # If /regex/ matches, delete the block.
1bfe7c
     }                  # Otherwise, the block will be printed.
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
   Note: When the script above reaches /regex/, the entire multi-line
1bfe7c
   block is in the pattern space. To replace items inside the block,
1bfe7c
   use "s///". To change the entire block, use the 'c' (change)
1bfe7c
   command:
1bfe7c

1bfe7c
     /regex/c\
1bfe7c
     1: This will replace the entire block\
1bfe7c
     2: with these two lines of text.
1bfe7c

1bfe7c
4.22. How do I locate a paragraph of text if the paragraph contains a
1bfe7c
      certain regular expression?
1bfe7c

1bfe7c
   Assume that paragraphs are separated by blank lines. For regexes
1bfe7c
   that are single terms, use one of the following scripts:
1bfe7c

1bfe7c
     sed -e '/./{H;$!d;}' -e 'x;/regex/!d'      # most seds
1bfe7c
     sed '/./{H;$!d;};x;/regex/!d'              # GNU sed
1bfe7c

1bfe7c
   To print paragraphs only if they contain 3 specific regular
1bfe7c
   expressions (RE1, RE2, and RE3), in any order in the paragraph:
1bfe7c

1bfe7c
     sed -e '/./{H;$!d;}' -e 'x;/RE1/!d;/RE2/!d;/RE3/!d'
1bfe7c

1bfe7c
   With this solution and the preceding one, if the paragraphs are
1bfe7c
   excessively long (more than 4k in length), you may overflow sed's
1bfe7c
   internal buffers. If using HHsed, you must add a "G;" command
1bfe7c
   immediately after the "x;" in the scripts above to defeat a bug
1bfe7c
   in HHsed (see section 7.9(5), below, for a description).
1bfe7c

1bfe7c
4.23. How do I match a block of _specific_ consecutive lines?
1bfe7c

1bfe7c
   There are three ways to approach this problem:
1bfe7c

1bfe7c
       (1) Try to use a "/range/, /expression/"
1bfe7c
       (2) Try to use a "/multi-line\nexpression/"
1bfe7c
       (3) Try to use a block of "literal strings"
1bfe7c

1bfe7c
   We describe each approach in the following sections.
1bfe7c

1bfe7c
4.23.1.  Try to use a "/range/, /expression/"
1bfe7c

1bfe7c
   If the block of lines are strings that *never change their order*
1bfe7c
   and if the top line never occurs outside the block, like this:
1bfe7c

1bfe7c
       Abel
1bfe7c
       Baker
1bfe7c
       Charlie
1bfe7c
       Delta
1bfe7c

1bfe7c
   then these solutions will work for deleting the block:
1bfe7c

1bfe7c
     sed 's/^Abel$/{N;N;N;d;}' files    # for blocks with few lines
1bfe7c
     sed '/^Abel$/, /^Zebra$/d' files   # for blocks with many lines
1bfe7c
     sed '/^Abel$/,+25d' files          # HHsed, sedmod, ssed, gsed 3.02.80
1bfe7c

1bfe7c
   To change the block, use the 'c' (change) command instead of 'd'.
1bfe7c
   To print that block only, use the -n switch and 'p' (print) instead
1bfe7c
   of 'd'. To change some things inside the block, try this:
1bfe7c

1bfe7c
     /^Abel$/,/^Delta$/ {
1bfe7c
         :ack
1bfe7c
         N;
1bfe7c
         /\nDelta$/! b ack
1bfe7c
         # At this point, all the lines in the block are collected
1bfe7c
         s/ubstitute /somethin/g;
1bfe7c
     }
1bfe7c

1bfe7c
4.23.2.  Try to use a "multi-line\nexpression"
1bfe7c

1bfe7c
   If the top line of the block sometimes appears alone or is
1bfe7c
   sometimes followed by other lines, or if a partial block may occur
1bfe7c
   somewhere in the file, a multi-line expression may be required.
1bfe7c

1bfe7c
   In these examples, we give solutions for matching an N-line block.
1bfe7c
   The expression "/^RE1\nRE2\nRE3...$/" represents a properly formed
1bfe7c
   regular expression where \n indicates a newline between lines. Note
1bfe7c
   that the 'N' followed by the 'P;D;' commands forms a "sliding
1bfe7c
   window" technique. A window of N lines is formed. If the multi-line
1bfe7c
   pattern matches, the block is handled. If not, the top line is
1bfe7c
   printed and then deleted from the pattern space, and we try to
1bfe7c
   match at the next line.
1bfe7c

1bfe7c
     # sed script to delete 2 consecutive lines: /^RE1\nRE2$/
1bfe7c
     $b
1bfe7c
     /^RE1$/ {
1bfe7c
       $!N
1bfe7c
       /^RE1\nRE2$/d
1bfe7c
       P;D
1bfe7c
     }
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
     # sed script to delete 3 consecutive lines. (This script
1bfe7c
     # fails under GNU sed v2.05 and earlier because of the 't'
1bfe7c
     # bug when s///n is used; see section 7.5(1) of the FAQ.)
1bfe7c
     : more
1bfe7c
     $!N
1bfe7c
     s/\n/&/2;
1bfe7c
     t enough
1bfe7c
     $!b more
1bfe7c
     : enough
1bfe7c
     /^RE1\nRE2\nRE3$/d
1bfe7c
     P;D
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
   For example, to delete a block of 5 consecutive lines, the previous
1bfe7c
   script must be altered in only two places:
1bfe7c

1bfe7c
   (1) Change the 2 in "s/\n/&/2;" to a 4 (the trailing semicolon is
1bfe7c
   needed to work around a bug in HHsed v1.5).
1bfe7c

1bfe7c
   (2) Change the regex line to "/^RE1\nRE2\nRE3\nRE4\nRE5$/d",
1bfe7c
   modifying the expression as needed.
1bfe7c

1bfe7c
   Suppose we want to delete a block of two blank lines followed by
1bfe7c
   the word "foo" followed by another blank line (4 lines in all).
1bfe7c
   Other blank lines and other instances of "foo" should be left
1bfe7c
   alone. After changing the '2' to a '3' (always one number less than
1bfe7c
   the total number of lines), the regex line would look like this:
1bfe7c
   "/^\n\nfoo\n$/d". (Thanks to Greg Ubben for this script.)
1bfe7c

1bfe7c
   As an alternative to work around the 't' bug in older versions of
1bfe7c
   GNU sed, the following script will delete 4 consecutive lines:
1bfe7c

1bfe7c
     # sed script to delete 4 consecutive lines. Use this if you
1bfe7c
     # require GNU sed 2.05 and below.
1bfe7c
     /^RE1$/!b
1bfe7c
     $!N
1bfe7c
     $!N
1bfe7c
     :a
1bfe7c
     $b
1bfe7c
     N
1bfe7c
     /^RE1\nRE2\nRE3\nRE4$/d
1bfe7c
     P
1bfe7c
     s/^.*\n\(.*\n.*\n.*\)$/\1/
1bfe7c
     ba
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
   Its drawback is that it must be modified in 3 places instead of 2
1bfe7c
   to adapt it for more lines, and as additional lines are added, the
1bfe7c
   's' command is forced to work harder to match the regexes. On the
1bfe7c
   other hand, it avoids a bug with gsed-2.05 and illustrates another
1bfe7c
   way to solve the problem of deleting consecutive lines.
1bfe7c

1bfe7c
4.23.3.  Try to use a block of "literal strings"
1bfe7c

1bfe7c
   If you need to match a static block of text (which may occur any
1bfe7c
   number of times throughout a file), where the contents of the block
1bfe7c
   are known in advance, then this script is easy to use. It requires
1bfe7c
   an intermediate file, which we will call "findrep.txt" (below):
1bfe7c

1bfe7c
       A block of several consecutive lines to
1bfe7c
       be matched literally should be placed on
1bfe7c
       top. Regular expressions like .*  or [a-z]
1bfe7c
       will lose their special meaning and be
1bfe7c
       interpreted literally in this block.
1bfe7c
       ----
1bfe7c
       Four hyphens separate the two sections. Put
1bfe7c
       the replacement text in the lower section.
1bfe7c
       As above, sed symbols like &, \n, or \1 will
1bfe7c
       lose their special meaning.
1bfe7c

1bfe7c
   This is a 3-step process. A generic script called "blockrep.sed"
1bfe7c
   will read "findrep.txt" (above) and generate a custom script, which
1bfe7c
   is then used on the actual input file. In other words,
1bfe7c
   "findrep.txt" is a simplified description of the editing that you
1bfe7c
   want to do on the block, and "blockrep.sed" turns it into actual
1bfe7c
   sed commands.
1bfe7c

1bfe7c
   Use this process from a Unix shell or from a DOS prompt:
1bfe7c

1bfe7c
     sed -nf blockrep.sed findrep.txt >custom.sed
1bfe7c
     sed -f custom.sed input.file >output.file
1bfe7c
     erase custom.sed
1bfe7c

1bfe7c
   The generic script "blockrep.sed" follows below. It's fairly long.
1bfe7c
   Examining its output might help you understanding how to use the
1bfe7c
   _sliding window_ technique.
1bfe7c

1bfe7c
     # filename: blockrep.sed
1bfe7c
     #   author: Paolo Bonzini
1bfe7c
     # Requires:
1bfe7c
     #    (1) blocks to find and replace, e.g., findrep.txt
1bfe7c
     #    (2) an input file to be changed, input.file
1bfe7c
     #
1bfe7c
     # blockrep.sed creates a second sed script, custom.sed,
1bfe7c
     # to find the lines above the row of 4 hyphens, globally
1bfe7c
     # replacing them with the lower block of text. GNU sed
1bfe7c
     # is recommended but not required for this script.
1bfe7c
     #
1bfe7c
     # Loop on the first part, accumulating the `from' text
1bfe7c
     # into the hold space.
1bfe7c
     :a
1bfe7c
     /^----$/! {
1bfe7c
        # Escape slashes, backslashes, the final newline and
1bfe7c
        # regular expression metacharacters.
1bfe7c
        s,[/\[.*],\\&,g
1bfe7c
        s/$/\\/
1bfe7c
        H
1bfe7c
        #
1bfe7c
        # Append N cmds needed to maintain the sliding window.
1bfe7c
        x
1bfe7c
        1 s,^.,s/,
1bfe7c
        1! s/^/N\
1bfe7c
     /
1bfe7c
        x
1bfe7c
        n
1bfe7c
        ba
1bfe7c
     }
1bfe7c
     #
1bfe7c
     # Change the final backslash to a slash to separate the
1bfe7c
     # two sides of the s command.
1bfe7c
     x
1bfe7c
     s,\\$,/,
1bfe7c
     x
1bfe7c
     #
1bfe7c
     # Until EOF, gather the substitution into hold space.
1bfe7c
     :b
1bfe7c
     n
1bfe7c
     s,[/\],\\&,g
1bfe7c
     $! s/$/\\/
1bfe7c
     H
1bfe7c
     $! bb
1bfe7c
     #
1bfe7c
     # Start the RHS of the s command without a leading
1bfe7c
     # newline, add the P/D pair for the sliding window, and
1bfe7c
     # print the script.
1bfe7c
     g
1bfe7c
     s,/\n,/,
1bfe7c
     s,$,/\
1bfe7c
     P\
1bfe7c
     D,p
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
4.24. How do I address all the lines between RE1 and RE2, excluding the
1bfe7c
      lines themselves?
1bfe7c

1bfe7c
   Normally, to address the lines between two regular expressions, RE1
1bfe7c
   and RE2, one would do this: '/RE1/,/RE2/{commands;}'. Excluding
1bfe7c
   those lines takes an extra step. To put 2 arrows before each line
1bfe7c
   between RE1 and RE2, except for those lines:
1bfe7c

1bfe7c
     sed '1,/RE1/!{ /RE2/,/RE1/!s/^/>>/; }' input.fil
1bfe7c

1bfe7c
   The preceding script, though short, may be difficult to follow. It
1bfe7c
   also requires that /RE1/ cannot occur on the first line of the
1bfe7c
   input file. The following script, though it's not a one-liner, is
1bfe7c
   easier to read and it permits /RE1/ to appear on the first line:
1bfe7c

1bfe7c
     # sed script to replace all lines between /RE1/ and /RE2/,
1bfe7c
     # without matching /RE1/ or /RE2/
1bfe7c
     /RE1/,/RE2/{
1bfe7c
       /RE1/b
1bfe7c
       /RE2/b
1bfe7c
       s/^/>>/
1bfe7c
     }
1bfe7c
     #---end of script---
1bfe7c

1bfe7c
   Contents of input.fil:         Output of sed script:
1bfe7c
      aaa                           aaa
1bfe7c
      bbb                           bbb
1bfe7c
      RE1                           RE1
1bfe7c
      aaa                           >>aaa
1bfe7c
      bbb                           >>bbb
1bfe7c
      ccc                           >>ccc
1bfe7c
      RE2                           RE2
1bfe7c
      end                           end
1bfe7c

1bfe7c
4.25. How do I join two lines if line #1 ends in a [certain string]?
1bfe7c

1bfe7c
   This question appears in the section on one-line sed scripts, but
1bfe7c
   it comes up so many times that it needs a place here also. Suppose
1bfe7c
   a line ends with a particular string (often, a line ends with a
1bfe7c
   backslash). How do you bring up the second line after it, even in
1bfe7c
   cases where several consecutive lines all end in a backslash?
1bfe7c

1bfe7c
     sed -e :a -e '/\\$/N; s/\\\n//; ta' file   # all seds
1bfe7c
     sed ':a; /\\$/N; s/\\\n//; ta' file        # GNU sed, ssed, HHsed
1bfe7c

1bfe7c
   Note that this replaces the backslash-newline with nothing. You may
1bfe7c
   want to replace the backslash-newline with a single space instead.
1bfe7c

1bfe7c
4.26. How do I join two lines if line #2 begins in a [certain string]?
1bfe7c

1bfe7c
   The inverse situation is another FAQ. Suppose a line begins with a
1bfe7c
   particular string. How do you bring that line up to follow the
1bfe7c
   previous line? In this example, we want to match the string "<<="
1bfe7c
   at the beginning of one line, bring that line up to the end of the
1bfe7c
   line before it, and replace the string with a single space:
1bfe7c

1bfe7c
     sed -e :a -e '$!N;s/\n<<=/ /;ta' -e 'P;D' file   # all seds
1bfe7c
     sed ':a; $!N;s/\n<<=/ /;ta;P;D' file             # GNU, ssed, sed15+
1bfe7c

1bfe7c
4.27. How do I change all paragraphs to long lines?
1bfe7c

1bfe7c
   A frequent request is how to convert DOS-style textfiles, in which
1bfe7c
   each line ends with "paragraph marker", to Microsoft-style
1bfe7c
   textfiles, in which the "paragraph" marker only appears at the end
1bfe7c
   of real paragraphs. Sometimes this question is framed as, "How do I
1bfe7c
   remove the hard returns at the end of each line in a paragraph?"
1bfe7c

1bfe7c
   The problem occurs because newer word processors don't work the
1bfe7c
   same way older text editors did. Older text editors used a newline
1bfe7c
   (CR/LF in DOS; LF alone in Unix) to end each line on screen or on
1bfe7c
   disk, and used two newlines to separate paragraphs. Certain word
1bfe7c
   processors wanted to make paragraph reformatting and reflowing work
1bfe7c
   easily, so they use one newline to end a paragraph and never allow
1bfe7c
   newlines _within_ a paragraph. This means that textfiles created
1bfe7c
   with standard editors (Emacs, vi, Vedit, Boxer, etc.) appear to
1bfe7c
   have "hard returns" at inappropriate places. The following sed
1bfe7c
   script finds blocks of consecutive nonblank lines (i.e., paragraphs
1bfe7c
   of text), and converts each block into one long line with one "hard
1bfe7c
   return" at the end.
1bfe7c

1bfe7c
     # sed script to change all paragraphs to long lines
1bfe7c
     /./{H; $!d;}             # Put each paragraph into hold space
1bfe7c
     x;                       # Swap hold space and pattern space
1bfe7c
     s/^\(\n\)\(..*\)$/\2\1/; # Move leading \n to end of PatSpace
1bfe7c
     s/\n\(.\)/ \1/g;         # Replace all other \n with 1 space
1bfe7c
     # Uncomment the following line to remove excess blank lines:
1bfe7c
     # /./!d;
1bfe7c
     #---end of sed script---
1bfe7c

1bfe7c
   If the input files have formatting or indentation that conveys
1bfe7c
   special meaning (like program source code), this script will remove
1bfe7c
   it. But if the text still needs to be extended, try 'par'
1bfe7c
   (paragraph reformatter) or the 'fmt' utility with the -t or -c
1bfe7c
   switches and the width option (-w) set to a number like 9999.
1bfe7c

1bfe7c
   SHELL AND ENVIRONMENT
1bfe7c

1bfe7c
4.30. How do I read environment variables with sed?
1bfe7c

1bfe7c
4.30.1. - on Unix platforms
1bfe7c

1bfe7c
   In Unix, environment variables begin with a dollar sign, such as
1bfe7c
   $TERM, $PATH, $var or $i. In sed, the dollar sign is used to
1bfe7c
   indicate the last line of the input file, the end of a line (in the
1bfe7c
   LHS), or a literal symbol (in the RHS). Sed cannot access variables
1bfe7c
   directly, so one must pay attention to shell quoting requirements
1bfe7c
   to expand the variables properly.
1bfe7c

1bfe7c
   To ALLOW the Unix shell to interpret the dollar sign, put the
1bfe7c
   script in double quotes:
1bfe7c

1bfe7c
     sed "s/_terminal-type_/$TERM/g" input.file >output.file
1bfe7c

1bfe7c
   To PREVENT the Unix shell from interpreting the dollar sign as a
1bfe7c
   shell variable, put the script in single quotes:
1bfe7c

1bfe7c
     sed 's/.$//' infile >outfile
1bfe7c

1bfe7c
   To use BOTH Unix $environment_vars and sed /end-of-line$/ pattern
1bfe7c
   matching, there are two solutions. (1) The easiest is to enclose
1bfe7c
   the script in "double quotes" so the shell can see the $variables,
1bfe7c
   and to prefix the sed metacharacter ($) with a backslash. Thus, in
1bfe7c

1bfe7c
     sed "s/$user\$/root/" file
1bfe7c

1bfe7c
   the shell interpolates $user and sed interprets \$ as the symbol
1bfe7c
   for end-of-line.
1bfe7c

1bfe7c
   (2) Another method--somewhat less readable--is to concatenate the
1bfe7c
   script with 'single quotes' where the $ should not be interpolated
1bfe7c
   and "double quotes" where variable interpolation should occur. To
1bfe7c
   demonstrate using the preceding script:
1bfe7c

1bfe7c
     sed "s/$user"'$/root/' file
1bfe7c

1bfe7c
   Solution #1 seems easier to remember. In either case, we search for
1bfe7c
   the user's name (stored in a variable called $user) when it occurs
1bfe7c
   at the end of the line ($), and substitute the word "root" in all
1bfe7c
   matches.
1bfe7c

1bfe7c
   For longer shell scripts, it is sometimes useful to begin with
1bfe7c
   single quote marks ('), close them upon encountering the variable,
1bfe7c
   enclose the variable name in double quotes ("), and resume with
1bfe7c
   single quotes, closing them at the end of the sed script.  Example:
1bfe7c

1bfe7c
     #! /bin/sh
1bfe7c
     # sed script to illustrate 'quote'"matching"'usage'
1bfe7c
     FROM='abcdefgh'
1bfe7c
     TO='ABCDEFGH'
1bfe7c
     sed -e '
1bfe7c
     y/'"$FROM"'/'"$TO"'/;    # note the quote pairing
1bfe7c
     # some more commands go here . . .
1bfe7c
     # last line is a single quote mark
1bfe7c
     '
1bfe7c

1bfe7c
   Thus, each variable named $FROM is replaced by $TO, and the single
1bfe7c
   quotes are used to glue the multiple lines together in the script.
1bfe7c
   (See also section 4.10, "How do I handle shell quoting in sed?")
1bfe7c

1bfe7c
4.30.2. - on MS-DOS and 4DOS platforms
1bfe7c

1bfe7c
   Under 4DOS and MS-DOS version 7.0 (Win95) or 7.10 (Win95 OSR2),
1bfe7c
   environment variables can be accessed from the command prompt.
1bfe7c
   Under MS-DOS v6.22 and below, environment variables can only be
1bfe7c
   accessed from within batch files. Environment variables should be
1bfe7c
   enclosed between percent signs and are case-insensitive; i.e.,
1bfe7c
   %USER% or %user% will display the USER variable. To generate a true
1bfe7c
   percent sign, just enter it twice.
1bfe7c

1bfe7c
   DOS versions of sed require that sed scripts be enclosed by double
1bfe7c
   quote marks "..." (not single quotes!) if the script contains
1bfe7c
   embedded tabs, spaces, redirection arrows or the vertical bar. In
1bfe7c
   fact, if the input for sed comes from piping, a sed script should
1bfe7c
   not contain a vertical bar, even if it is protected by double
1bfe7c
   quotes (this seems to be bug in the normal MS-DOS syntax). Thus,
1bfe7c

1bfe7c
       echo blurk | sed "s/^/ |foo /"     # will cause an error
1bfe7c
       sed "s/^/ |foo /" blurk.txt        # will work as expected
1bfe7c

1bfe7c
   Using DOS environment variables which contain DOS path statements
1bfe7c
   (such as a TMP variable set to "C:\TEMP") within sed scripts is
1bfe7c
   discouraged because sed will interpret the backslash '\' as a
1bfe7c
   metacharacter to "quote" the next character, not as a normal
1bfe7c
   symbol. Thus,
1bfe7c

1bfe7c
       sed "s/^/%TMP% /" somefile.txt
1bfe7c

1bfe7c
   will not prefix each line with (say) "C:\TEMP ", but will prefix
1bfe7c
   each line with "C:TEMP "; sed will discard the backslash, which is
1bfe7c
   probably not what you want. Other variables such as %PATH% and
1bfe7c
   %COMSPEC% will also lose the backslash within sed scripts.
1bfe7c

1bfe7c
   Environment variables which do not use backslashes are usually
1bfe7c
   workable. Thus, all the following should work without difficulty,
1bfe7c
   if they are invoked from within DOS batch files:
1bfe7c

1bfe7c
       sed "s/=username=/%USER%/g" somefile.txt
1bfe7c
       echo %FILENAME% | sed "s/\.TXT/.BAK/"
1bfe7c
       grep -Ei "%string%" somefile.txt | sed "s/^/  /"
1bfe7c

1bfe7c
   while from either the DOS prompt or from within a batch file,
1bfe7c

1bfe7c
       sed "s/%%/ percent/g" input.fil >output.fil
1bfe7c

1bfe7c
   will replace each percent symbol in a file with " percent" (adding
1bfe7c
   the leading space for readability).
1bfe7c

1bfe7c
4.31. How do I export or pass variables back into the environment?
1bfe7c

1bfe7c
4.31.1. - on Unix platforms
1bfe7c

1bfe7c
   Suppose that line #1, word #2 of the file 'terminals' contains a
1bfe7c
   value to be put in your TERM environment variable. Sed cannot
1bfe7c
   export variables directly to the shell, but it can pass strings to
1bfe7c
   shell commands. To set a variable in the Bourne shell:
1bfe7c

1bfe7c
       TERM=`sed 's/^[^ ][^ ]* \([^ ][^ ]*\).*/\1/;q' terminals`;
1bfe7c
       export TERM
1bfe7c

1bfe7c
   If the second word were "Wyse50", this would send the shell command
1bfe7c
   "TERM=Wyse50".
1bfe7c

1bfe7c
4.31.2. - on MS-DOS or 4DOS platforms
1bfe7c

1bfe7c
   Sed cannot directly manipulate the environment. Under DOS, only
1bfe7c
   batch files (.BAT) can do this, using the SET instruction, since
1bfe7c
   they are run directly by the command shell. Under 4DOS, special
1bfe7c
   4DOS commands (such as ESET) can also alter the environment.
1bfe7c

1bfe7c
   Under DOS or 4DOS, sed can select a word and pass it to the SET
1bfe7c
   command. Suppose you want the 1st word of the 2nd line of MY.DAT
1bfe7c
   put into an environment variable named %PHONE%. You might do this:
1bfe7c

1bfe7c
       @echo off
1bfe7c
       sed -n "2 s/^\([^ ][^ ]*\) .*/SET PHONE=\1/p;3q" MY.DAT > GO_.BAT
1bfe7c
       call GO_.BAT
1bfe7c
       echo The environment variable for PHONE is %PHONE%
1bfe7c
       :: cleanup
1bfe7c
       del GO_.BAT
1bfe7c

1bfe7c
   The sed script assumes that the first character on the 2nd line is
1bfe7c
   not a space and uses grouping \(...\) to save the first string of
1bfe7c
   non-space characters as \1 for the RHS. In writing any batch files,
1bfe7c
   make sure that output filenames such as GO_.BAT don't overwrite
1bfe7c
   preexisting files of the same name.
1bfe7c

1bfe7c
4.32. How do I handle Unix shell quoting in sed?
1bfe7c

1bfe7c
   To embed a literal single quote (') in a script, use (a) or (b):
1bfe7c

1bfe7c
   (a) If possible, put the script in double quotes:
1bfe7c

1bfe7c
     sed "s/cannot/can't/g" file
1bfe7c

1bfe7c
   (b) If the script must use single quotes, then close-single-quote
1bfe7c
   the script just before the SPECIAL single quote, prefix the single
1bfe7c
   quote with a backslash, and use a 2nd pair of single quotes to
1bfe7c
   finish marking the script. Thus:
1bfe7c

1bfe7c
     sed 's/cannot$/can'\''t/g' file
1bfe7c

1bfe7c
   Though this looks hard to read, it breaks down to 3 parts:
1bfe7c

1bfe7c
      's/cannot$/can'   \'   't/g'
1bfe7c
      ---------------   --   -----
1bfe7c

1bfe7c
   To embed a literal double quote (") in a script, use (a) or (b):
1bfe7c

1bfe7c
   (a) If possible, put the script in single quotes. You don't need to
1bfe7c
   prefix the double quotes with anything. Thus:
1bfe7c

1bfe7c
     sed 's/14"/fourteen inches/g' file
1bfe7c

1bfe7c
   (b) If the script must use double quotes, then prefix the SPECIAL
1bfe7c
   double quote with a backslash (\). Thus,
1bfe7c

1bfe7c
     sed "s/$length\"/$length inches/g" file
1bfe7c

1bfe7c
   To embed a literal backslash (\) into a script, enter it twice:
1bfe7c

1bfe7c
     sed 's/C:\\DOS/D:\\DOS/g' config.sys
1bfe7c

1bfe7c
   FILES, DIRECTORIES, AND PATHS
1bfe7c

1bfe7c
4.40. How do I read (insert/add) a file at the top of a textfile?
1bfe7c

1bfe7c
   Normally, adding a "header" file to the top of a "body" file is
1bfe7c
   done from the command prompt before passing the file on to sed.
1bfe7c
   (MS-DOS below version 6.0 must use COPY and DEL instead of MOVE in
1bfe7c
   the following example.)
1bfe7c

1bfe7c
       copy header.txt+body temp                  # MS-DOS command 1
1bfe7c
       echo Y | move temp body                    # MS-DOS command 2
1bfe7c
                                                    #
1bfe7c
       cat header.txt body >temp; mv temp body    # Unix commands
1bfe7c

1bfe7c
   However, if inserting the file must occur within sed, there is a
1bfe7c
   way. The sed command "1 r header.txt" will not work; it will print
1bfe7c
   line 1 and then insert "header.txt" between lines 1 and 2. The
1bfe7c
   following script solves this problem; however, there must be at
1bfe7c
   least 2 lines in the target file for the script to work properly.
1bfe7c

1bfe7c
     # sed script to insert "header.txt" above the first line
1bfe7c
     1{h; r header.txt
1bfe7c
       D; }
1bfe7c
     2{x; G; }
1bfe7c
     #---end of sed script---
1bfe7c

1bfe7c
4.41. How do I make substitutions in every file in a directory, or in
1bfe7c
      a complete directory tree?
1bfe7c

1bfe7c
4.41.1. - ssed and Perl solution
1bfe7c

1bfe7c
   The best solution for multiple files in a single directory is to
1bfe7c
   use ssed or gsed v4.0 or higher:
1bfe7c

1bfe7c
     sed -i.BAK 's|foo|bar|g' files       # -i does in-place replacement
1bfe7c

1bfe7c
   If you don't have ssed, there is a similar solution in Perl. (Yes,
1bfe7c
   we know this is a FAQ file for sed, not perl, but perl is more
1bfe7c
   common than ssed for many users.)
1bfe7c

1bfe7c
     perl -pi.bak -e 's|foo|bar|g' files                # or
1bfe7c
     perl -pi.bak -e 's|foo|bar|g' `find /pathname -name "filespec"`
1bfe7c

1bfe7c
   For each file in the filelist, sed (or Perl) renames the source
1bfe7c
   file to "filename.bak"; the modified file gets the original
1bfe7c
   filename. Remove '.bak' if you don't need backup copies. (Note the
1bfe7c
   use of "s|||" instead of "s///" here, and in the scripts below. The
1bfe7c
   vertical bars in the 's' command let you replace '/some/path' with
1bfe7c
   '/another/path', accommodating slashes in the LHS and RHS.)
1bfe7c

1bfe7c
   To recurse directories in Unix or GNU/Linux:
1bfe7c

1bfe7c
     # We use xargs to prevent passing too many filenames to sed, but
1bfe7c
     # this command will fail if filenames contain spaces or newlines.
1bfe7c
     find /my/path -name '*.ht' -print | xargs sed -i.BAK 's|foo|bar|g'
1bfe7c

1bfe7c
   To recurse directories under Windows 2000 (CMD.EXE or COMMAND.COM):
1bfe7c

1bfe7c
     # This syntax isn't supported under Windows 9x COMMAND.COM
1bfe7c
     for /R c:\my\path %f in (*.htm) do sed -i.BAK "s|foo|bar|g" %f
1bfe7c

1bfe7c
4.41.2. - Unix solution
1bfe7c

1bfe7c
   For all files in a single directory, assuming they end with *.txt
1bfe7c
   and you have no files named "[anything].txt.bak" already, use a
1bfe7c
   shell script:
1bfe7c

1bfe7c
     #! /bin/sh
1bfe7c
     # Source files are saved as "filename.txt.bak" in case of error
1bfe7c
     # The '&&' after cp is an additional safety feature
1bfe7c
     for file in *.txt
1bfe7c
     do
1bfe7c
        cp $file $file.bak &&
1bfe7c
        sed 's|foo|bar|g' $file.bak >$file
1bfe7c
     done
1bfe7c

1bfe7c
   To do an entire directory tree, use the Unix utility find, like so
1bfe7c
   (thanks to Jim Dennis <jadestar@rahul.net> for this script):
1bfe7c

1bfe7c
     #! /bin/sh
1bfe7c
     # filename: replaceall
1bfe7c
     # Backup files are NOT saved in this script.
1bfe7c
     find . -type f -name '*.txt' -print | while read i
1bfe7c
     do
1bfe7c
        sed 's|foo|bar|g' $i > $i.tmp && mv $i.tmp $i
1bfe7c
     done
1bfe7c

1bfe7c
   This previous shell script recurses through the directory tree,
1bfe7c
   finding only files in the directory (not symbolic links, which will
1bfe7c
   be encountered by the shell command "for file in *.txt", above). To
1bfe7c
   preserve file permissions and make backup copies, use the 2-line cp
1bfe7c
   routine of the earlier script instead of "sed ... && mv ...". By
1bfe7c
   replacing the sed command 's|foo|bar|g' with something like
1bfe7c

1bfe7c
     sed "s|$1|$2|g" ${i}.bak > $i
1bfe7c

1bfe7c
   using double quotes instead of single quotes, the user can also
1bfe7c
   employ positional parameters on the shell script command tail, thus
1bfe7c
   reusing the script from time to time. For example,
1bfe7c

1bfe7c
       replaceall East West
1bfe7c

1bfe7c
   would modify all your *.txt files in the current directory.
1bfe7c

1bfe7c
4.41.3. - DOS solution:
1bfe7c

1bfe7c
   MS-DOS users should use two batch files like this:
1bfe7c

1bfe7c
      @echo off
1bfe7c
      :: MS-DOS filename: REPLACE.BAT
1bfe7c
      ::
1bfe7c
      :: Create a destination directory to put the new files.
1bfe7c
      :: Note: The next command will fail under Novel Netware
1bfe7c
      :: below version 4.10 unless "SHOW DOTS=ON" is active.
1bfe7c
      if not exist .\NEWFILES\NUL mkdir NEWFILES
1bfe7c
      for %%f in (*.txt) do CALL REPL_2.BAT %%f
1bfe7c
      echo Done!!
1bfe7c
      :: ---End of first batch file---
1bfe7c

1bfe7c
      @echo off
1bfe7c
      :: MS-DOS filename: REPL_2.BAT
1bfe7c
      ::
1bfe7c
      sed "s/foo/bar/g" %1 > NEWFILES\%1
1bfe7c
      :: ---End of the second batch file---
1bfe7c

1bfe7c
   When finished, the current directory contains all the original
1bfe7c
   files, and the newly-created NEWFILES subdirectory contains the
1bfe7c
   modified *.TXT files. Do not attempt a command like
1bfe7c

1bfe7c
       for %%f in (*.txt) do sed "s/foo/bar/g" %%f >NEWFILES\%%f
1bfe7c

1bfe7c
   under any version of MS-DOS because the output filename will be
1bfe7c
   created as a literal '%f' in the NEWFILES directory before the
1bfe7c
   %%f is expanded to become each filename in (*.txt). This occurs
1bfe7c
   because MS-DOS creates output filenames via redirection commands
1bfe7c
   before it expands "for..in..do" variables.
1bfe7c

1bfe7c
   To recurse through an entire directory tree in MS-DOS requires a
1bfe7c
   batch file more complex than we have room to describe. Examine the
1bfe7c
   file SWEEP.BAT in Timo Salmi's great archive of batch tricks,
1bfe7c
   located at <ftp: garbo.uwasa.fi="" pc="" link="" tsbat.zip=""> (this file is
1bfe7c
   regularly updated). Another alternative is to get an external
1bfe7c
   program designed for directory recursion. Here are some recommended
1bfe7c
   programs for directory recursion. The first one, FORALL, runs under
1bfe7c
   either OS/2 or DOS. Unfortunately, none of these supports Win9x
1bfe7c
   long filenames.
1bfe7c

1bfe7c
       http://hobbes.nmsu.edu/pub/os2/util/disk/forall72.zip
1bfe7c
       ftp://garbo.uwasa.fi/pc/filefind/target15.zip
1bfe7c

1bfe7c
4.42. How do I replace "/some/UNIX/path" in a substitution?
1bfe7c

1bfe7c
   Technically, the normal meaning of the slash can be disabled by
1bfe7c
   prefixing it with a backslash. Thus,
1bfe7c

1bfe7c
     sed 's/\/some\/UNIX\/path/\/a\/new\/path/g' files
1bfe7c

1bfe7c
   But this is hard to read and write. There is a better solution.
1bfe7c
   The s/// substitution command allows '/' to be replaced by any
1bfe7c
   other character (including spaces or alphanumerics). Thus,
1bfe7c

1bfe7c
     sed 's|/some/UNIX/path|/a/new/path|g' files
1bfe7c

1bfe7c
   and if you are using variable names in a Unix shell script,
1bfe7c

1bfe7c
     sed "s|$OLDPATH|$NEWPATH|g" oldfile >newfile
1bfe7c

1bfe7c
4.43. How do I replace "C:\SOME\DOS\PATH" in a substitution?
1bfe7c

1bfe7c
   For MS-DOS users, every backslash must be doubled. Thus, to replace
1bfe7c
   "C:\SOME\DOS\PATH" with "D:\MY\NEW\PATH":
1bfe7c

1bfe7c
     sed "s|C:\\SOME\\DOS\\PATH|D:\\MY\\NEW\\PATH|g" infile >outfile
1bfe7c

1bfe7c
   Remember that DOS pathnames are not case sensitive and can appear
1bfe7c
   in upper or lower case in the input file. If this concerns you, use
1bfe7c
   a version of sed which can ignore case when matching (gsed, ssed,
1bfe7c
   sedmod, sed16).
1bfe7c

1bfe7c
       @echo off
1bfe7c
       :: sample MS-DOS batch file to alter path statements
1bfe7c
       :: requires GNU sed with the /i flag for s///
1bfe7c
       set old=C:\\SOME\\DOS\\PATH
1bfe7c
       set new=D:\\MY\\NEW\\PATH
1bfe7c
       gsed "s|%old%|%new%|gi" infile >outfile
1bfe7c
       :: or
1bfe7c
       ::     sedmod -i "s|%old%|%new%|g" infile >outfile
1bfe7c
       set old=
1bfe7c
       set new=
1bfe7c

1bfe7c
   Also, remember that under Windows long filenames may be stored in
1bfe7c
   two formats: e.g., as "C:\Program Files" or as "C:\PROGRA~1".
1bfe7c

1bfe7c
4.44.  How do I emulate file-includes, using sed?
1bfe7c

1bfe7c
   Given an input file with file-include statements, similar to
1bfe7c
   C-style includes or "server-side includes" (SSI) of this format:
1bfe7c

1bfe7c
       This is the source file. It's short.
1bfe7c
       Its name is simply 'source'. See the script below.
1bfe7c
       
1bfe7c
              And this is any amount of text between
1bfe7c
       
1bfe7c
       This is the last line of the file.
1bfe7c

1bfe7c
   How do we direct sed to import/insert whichever files are at the
1bfe7c
   point of the 'file="filename"' token? First, use this file:
1bfe7c

1bfe7c
     #n
1bfe7c
     # filename: incl.sed
1bfe7c
     # Comments supported by GNU sed or ssed. Leading '#n' should
1bfe7c
     # be on line 1, columns 1-2 of the line.
1bfe7c
     /