Nikolai Bezroukov. Portraits of Open Source Pioneers

For readers with high sensitivity to grammar errors access to this page is not recommended :-)

Chapter 7: Shell Giants

• Introduction to the Unix shell history
• The Bourne Shell Family
• C-shell family (c-shell and tcsh)
• Zsh and other "post-scripting languages" shells
• Visual shells and Orthodox file managers
• Shells History Webliography

Appendix 1: Shell History by John Mashey

Appendix 2:  An Introduction to the C shell by William Joy (revised for 4.3BSD by Mark Seiden)

Appendix 3: An Introduction to the UNIX Shell by S. R. Bourne.

The book Life with Unix by Don Libes and Sandy Ressler is a fascinating reading for anyone interested in Unix in general and scripting history in particular. It covers a lot of the history, interactions, etc. Much in the present section is summarized from this book. Some interesting info is in UNIX FAQ.

As most of the readers know,  Unix history goes back to 1969 and the famous "little-used PDP-7 in a corner" on which Ken Thompson, Dennis Ritchie (the R in K&R), and others after being exiled from Multics started work on replicating Multics development environment, which later was to become Unix.

But what is less known and appreciated is the fact that Ken Thompson was also the author of the first UNIX shell and that concepts of this shell greatly influenced the subsequent work of John Mashey and Stephen Bourne. Influenced to the extent that John Mashey actually objected to calling his shell Mashey shell considering it to be a direct extension of Thompson shell.

The name "Unix" was intended as a pun on Multics (and was written "Unics" at first, for UNiplexed Information and Computing System). As Ch 9 of the Netizens Netbook  ( On the Early History and Impact of Unix Tools to Build the Tools for a New Millenium) aptly stated:

When AT&T made the decision to pull out of the Multics collaboration, they took the research operating system off their GE 645  computer and put up the GECOS operating system. Though GECOS was adequate  for applications, it was "nowhere near as satisfactory if you were trying  to do things that were technically difficult and imperfectly defined,"  explained Vyssotsky, "which is the main task of research."(23)

 For the pioneering work of Bell Labs research programmers like Ken Thompson and the research purposes of the Labs, an operating system more  like what Multics had promised was needed. Along with the advantages of  immediate feedback which time-sharing provided, the Bell Labs researchers  wanted to continue the ability to work collaboratively which time-sharing  had made possible.

As Dennis  Ritchie aptly stated:

 "What we wanted to preserve, was not just a good programming environment in which to  do programming, but a system around which a fellowship could form. We knew from experience that the essence of communal computing, as supplied by remote-access, time-shared machines, is not just to type programs into a terminal instead of a keypunch, but to encourage close  communication."(24)

Ritchie describes how an informal group led by Thompson had begun  investigating alternatives to Multics before the GE-645 Multics machine had been removed from the Labs.(25) Thompson and Ritchie presented Bell Labs with proposals to buy them a computer so they could build their own interactive, time-sharing operating system. Their proposals weren't acted on. Eventually, Thompson found a little used PDP-7 computer. As Vyssotsky noted, the orphaned PDP-7 computer was extremely small computer, a machine, "more in the class of a Commodore 64 than the class of a PC-AT."(26)

For the first 10 years, Unix development was essentially confined to Bell Labs and most scripting related work was also done in New Jersey.  The initial versions of Unix were labeled "Version n" or "Nth Edition" (of the manuals) and some milestones in shell history are directly related to particular Unix releases. 

The first major Unix implementation was for DEC's PDP-11 (16 bits) which was much better then PDP-7, bu still very tiny by today's hardware standards (typical configuration were limited to 128K memory, 2.4M disc, 64K per-process limit (inc the kernel)) that similar configurations on watches, not even cell phones ;-). The fact that they managed to created pretty powerful shells for such a computer is nothing but simply amazing and attests tremendous ingenuity of Unix development team.

• 1971 V1: 1st Unix version written in assembler on a PDP-11/20. Included file system, fork(), roff, ed. Was used as a text processing tool for preparation of patents.
   
• 1972 V3: Pipe() was implemented in this version !
   
• 1973 V4: Unix was rewritten in C.
   
• 1974 V5 The UNIX Time-Sharing System is published in CACM by Ken Thompson and Dennis Ritchie. It is a revision of the 1973 paper. Fifth edition was made freely available to universities for educational purposes. University of California at Berkeley (UCB) gets Version 4 of UNIX.  In January 1974, a Version 4 tape was delivered and Unix was installed in Berkeley by graduate student Keith Standiford.  Berkeley begins making major enhancements to UNIX and sets the stage for becoming a major distribution center for their version of UNIX. The Elements of Programming Style by Kernighan and Plauger is published.
   
• 1975
  • V6: First version of Unix became available outside Bell Labs. Thompson begins one year sabbatical at Berkeley bringing V6 with him. AT&T officially begins licensing UNIX to universities. Due to its portability and flexibility, UNIX Version 6 (V6) became the minicomputer OS of choice for universities. In the fall of 1975 were already two graduate students, Bill Joy and Chuck Haley working on enhancing this version. J. Lions published his famous "A commentary on the Unix Operating System" based on V6.
     
  • PWB Unix got Mashey shell -- the first powerful shell for Unix that was close to a scripting language.  As John Mashey recollected:

    The "PWB Shell" first appeared in mid-1975. It derived from a set of requirements and suggestions from me in early 1975 in trying to do serious shell programming. In mid-1975, the shell acquired variables, including 3 that were derived from per-process data. This is where the idea of more generalized path- searching came in. Some of the existing support commands were beefed up, including if/else/endif. Most of this work was done by Alan Glasser or Dick Haight. I ended up taking this over, doing performance analysis, moving most of the control structure commands into the shell, adding switch,, while, etc. etc.  By this time there was serious shell programming in massive amounts. This had mostly stabilized by early/mid 1976, although a few more features were added in early 1977. For various reasons this came to be called the "Mashey shell", a term I universally abhorred, since it was fundamentally still a Thompson shell with selected additions and tunings by various of us, constrained to be mostly upward compatible, leading to some things that I found less than esthetic. Even less than esthetic was the title of a later internal Bell Labs course: "Bourne Shell Programming for Mashey Shell Programmers". See [2] and [3].

    Bell Labs Piscataway used the PWB shell quite heavily for its scripting needs in the mid-1970s and that probably was the first time a scripting language was widely used outside research. Rephrasing John R. Mashey  quote "UNIX is a lever for the intellect"  we can say that "Scripting is a level for the intellect".
    
    Thirty years later we are taking scripting for granted, but it was pretty amazing breakthrough made on a PDP 11 of those days with typical configuration 128K memory, 2.4M disc, 64K per-process limit (inc the kernel). In other words this was a computer much less powerful that a typical cellphone.  Unix on PDP 11 was a tight squeeze, and utilities like "sh" had to be extremely tightly coded. Mashey shell was also pretty fast (faster then Bourne shell that replaced it later).
     
• 1976. John R. Mashey publishes the paper "Using a Command Language as a High-Level Programming Language", Proc. 2nd Int. Conf on Software Engineering, IEEE (Oct 13-15, 1976), 169-176. -- the first paper about scripting as a very high level programming language.
   
• 1977. The new scripting language AWL was designed by Alfred V. Aho, Peter J. Weinberger, and Brian W. Kernighan. It was a better designed language then either Mashey shell or Bourne shell.  Much better designed... It's really sad that thier efforts was not understood and extended into shell by managers of the project. I would say that this was a major blunder on Unix managers that AWK it was not extended to a full Unix shell.  
• 1978
  • A Bourne shell by Stephen Bourne, simply known as the 'sh', is released. It became the most popular Unix shell for the next two decades.  While Steven Borne introduced some Algol-68 style syntax into the shell (and it is the only place where Algol-68 style "palindromatic syntax" as in if ... fi survived; actually the implementation has nothing to other feature common with algol-68 and was extremely crude and primitive). The quality of Bourne shell was vastly inferior to AWK implementation. In a way it was a second rate project done by second rate researcher.  Compatibility issues hampered his work to the extent that nobody ever produced a formal syntax of  Borne shell, and also signify absence of real insights in the design of the Bourne shell. The fact that people suffer from consequences of this blunders even today is a kind of  achievement :-)
     
  • Awk was described in the article published in Software Practice and Experience in July, 1978, and first distributed with Seventh Edition UNIX.
     
  • 1978 Bill Joy produces 2BSD, the "Berkeley Software Distribution", that contains C-shell (2BSD was a set of applications that ran on 6th Edition UNIX, and was not a kernel nor a set of kernel patches). It was implemented as a standalone interpreter. Csh was a big improvement over the original sh in many respects; it had more built-in commands, a different and friendlier syntax with better  flow control constructs and an arithmetic facility. It also supported command history which provided much better command interface. Among key ideas here were the ideas of a history list and an in-line editing. Combined they produced a kind of pseudo-GUI environment that saved users a lot of typing. Naturally, csh took off quickly. A full distribution of the 2BSD tape is available at http://www.tuhs.org/Archive/PDP-11/Distributions/ucb/
  • Famous  Steve Bourne paper, "An Introduction to the UNIX Shell"  was published (BSTJ  57, No 6, Part 2 (Jul-Aug 1978), 2797-2822)
  • Ports to Interdata and VAX. Steve Johnson completed the first port of UNIX to an Interdata 8/32, the first non-DEC computer to run UNIX. Note that this happened nearly ten years after being limited only on DEC equipment. Soon after that,  UNIX was ported to a DEC VAX by Bill Joy, not by Thompson and Ritchie, since they had become disenchanted by DEC and its unwillingness to support UNIX. 
  • The C Programming Language by Kernighan and Ritchie is published and served as the standard for the C language for over a decade, until the ANSI C standard appeared in December 1989.
     
• 1979
  • V7: It included full K&R C, uucp, Bourne shell and awk. V7 was ported to the VAX as 32V. The V7 kernel was a mere 40KB! Here (for reference) are the system calls of V7:

      _exit, access, acct, alarm, brk, chdir, chmod, chown,
      chroot, close, creat, dup, dup2, exec*, exit, fork, fstat,
      ftime, getegid, geteuid, getgid, getpid, getuid, gtty,
      indir, ioctl, kill, link, lock, lseek, mknod, mount,
      mpxcall, nice, open, pause, phys, pipe, pkoff, pkon,
      profil, ptrace, read, sbrk, setgid, setuid, signal, stat,
      stime, stty, sync, tell, time, times, umask, umount,
      unlink, utime, wait, write.

  • T. A. Dolotta and  John Mashey presented a talk and published a paper "Using a Command Language as the Primary Programming Tool" at IFIP Working Conference on Command Languages, Sept 10-14, 1979.
     
• 1981 C-shell was extended into tcsh, which for a long time became the most popular interactive shell. It introduced the idea of command completion.
   
• 1982. David Korn created ksh -- a dominant shell in commercial Unixes for the next twenty years.
   
• 1987. The first version of Perl was developed by Larry Wall  at  the end of the year. that means in 2013 Perl is more then 25 years old.  See excellent  PerlTimeline for more information. Perl was an attempt to combine the capabilities of the shell and awk into a single language.  Because Perl is freely available and performs better than combination of  shell, sed and awk, Perl rapidly acquired a large user community, primarily at universities. Here is the description from Perl manpage:

  NAME
  perl | Practical Extraction and Report Language

  SYNOPSIS
      perl [options] filename args

  DESCRIPTION
       Perl is a interpreted language optimized for scanning  arbi
       trary  text  files,  extracting  information from those text
       files, and printing reports based on that information.  It's
       also  a good language for many system management tasks.  The
       language is intended to be practical  (easy  to  use,  effi-
       cient,  complete)  rather  than  beautiful  (tiny,  elegant,
       minimal).  It combines (in  the  author's  opinion,  anyway)
       some  of the best features of C, sed, awk, and sh, so people
       familiar with those languages should have little  difficulty
       with  it.  (Language historians will also note some vestiges
       of csh, Pascal, and  even  BASIC|PLUS.)   Expression  syntax
       corresponds  quite  closely  to C expression syntax.  If you
       have a problem that would ordinarily use sed or awk  or  sh,
       but  it exceeds their capabilities or must run a little fas-
       ter, and you don't want to write the silly thing in C,  then
       perl  may  be  for  you.  There are also translators to turn
       your sed and awk scripts  into  perl  scripts.   OK,  enough
       hype.

• 1988 John K. Ousterhout developed TCL, the first embeddable scripting language for Unix. 

  I began work on Tk in late 1988, but it was a part-time project, so it took about two years before there was enough functionality for Tk to be useful.
  ... By 1989 I was using Tcl in a couple of projects, and I believe that I gave a talk on Tcl during the Berkeley Industrial Liaison Conference, where several hundred people from industry came to Berkeley to hear about research in our department. I believe that this led to invitations to give talks on Tcl at a few interested companies, and that some of the people at those talks asked if they could have copies of Tcl. In any case, I gave away a few copies of Tcl to early adopters in 1989.

  In 1994 John Ousterhout joined Sun. Unfortunately Sun failed to capitalize on the chance to make TCL a standard Unix macro language the way REXX was in OS/2.

• 1989.
  • As part of the GNU project  a retro-shell called the Bourne-Again Shell, or bash was developed.  Bash was originally written by Brian Fox of the Free Software Foundation. Initially was rather weak, but later became a prominent shell due to popularity of Linux. Now it is often as a standard shell on Solaris and other flavors of Unix.   Later version of bash were maintained by Chet Ramey, who copied into it large part of functionality of ksh93.
  • Z Shell was created by Paul Falstad in Princeton University, NJ were he was a student. This was another attempt to create a "super-shell" a shell suitable for interactive use but which can double as a decent scripting language. Many of the useful features of bash, ksh, and tcsh were incorporated into zsh; many original features were added. Later David Korn created ksh93 with a similar agenda.
     
• 1990 
  • Python was created by Guido van Rossum at CWI in Amsterdam.
  • Don Libes of the National Institute of Standards and Technology created Expect.  He made Expect freely available after presenting his paper in the summer of 1990, and it became an instant hit among system administrators. Expect was not only the first widely distributed Tcl application, but for many years it was the most popular scripting application in Unix universe. Many of the early Tcl users learned about it by using Expect.
  • John Ousterhout presented a paper on Tcl at the USENIX Conference n January.
     
• 1991. Perl 4.000 is released and includes an artistic license as well as the GPL.
   
• 1993. David Korn released ksh93, an attempt to backport some Perl features into shell framework. Here is a partial list of ksh93 features:
  • Associative arrays - ksh88 already supports indexed arrays.
  • Floating point arithmetic + math library functions.
  • Arithmetic for command similar to C and awk.
  • Complete ANSI C printf formatting with extensions.
  • Run time linking of libraries and builtins.
  • A number of additional substring operations such as offset length.
  • Full extended regular expressions matching capabilities
  • Compound variables which can be used to represent data aggregates.
  • Name references for passing aggregate variables by name.
  • Active variables. Users can trap variable assignment and references by associating intercept functions of the form name.get and name.set.
  • Ability to make socket connections to servers by name.
  • read with timeouts.
  • Conformance to POSIX 1003.2.
  • Command completion and variable completion, ksh93 only had file name completion.
  • A key binding mechanism that allows users to bind keys to new editing functions.
     
• 1994. Perl 5.000 is unveiled.

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March, 12, 2019