History of VI editor and its derivatives

I would like to reproduce Bill's Joy recollection of how vi was really written (from Linux Magazine November 1999 FEATURES The Joy of Unix):

LM:: What inspired you to write vi?

BJ: What happened is that Ken Thompson came to Berkeley and brought this broken Pascal system, and we got this summer job to fix it. While we were fixing it, we got frustrated with the editor we were using which was named ed. ed is certainly frustrating.

We got this code from a guy named George Coulouris at University College in London called em -- Editor for Mortals -- since only immortals could use ed to do anything. By the way, before that summer, we could only type in uppercase. That summer we got lowercase ROMs for our terminals. It was really exciting to finally use lowercase.

LM: What year was that?

BJ: '76 or '77. It was the summer Carter was president. So we modified em and created en. I don't know if there was an eo or an ep but finally there was ex. [laughter] I remember en but I don't know how it got to ex. So I had a terminal at home and a 300 baud modem so the cursor could move around and I just stayed up all night for a few months and wrote vi.

LM: So you didn't really write vi in one weekend like everybody says?

BJ: No. It took a long time. It was really hard to do because you've got to remember that I was trying to make it usable over a 300 baud modem. That's also the reason you have all these funny commands. It just barely worked to use a screen editor over a modem. It was just barely fast enough. A 1200 baud modem was an upgrade. 1200 baud now is pretty slow.

9600 baud is faster than you can read. 1200 baud is way slower. So the editor was optimized so that you could edit and feel productive when it was painting slower than you could think. Now that computers are so much faster than you can think, nobody understands this anymore.

The people doing Emacs were sitting in labs at MIT with what were essentially fibre-channel links to the host, in contemporary terms. They were working on a PDP-10, which was a huge machine by comparison, with infinitely fast screens.

So they could have funny commands with the screen shimmering and all that, and meanwhile, I'm sitting at home in sort of World War II surplus housing at Berkeley with a modem and a terminal that can just barely get the cursor off the bottom line.

It was a world that is now extinct. People don't know that vi was written for a world that doesn't exist anymore -- unless you decide to get a satellite phone and use it to connect to the Net at 2400 baud, in which case you'll realize that the Net is not usable at 2400 baud. It used to be perfectly usable at 1200 baud. But these days you can't use the Web at 2400 baud because the ads are 24 kilobytes.

Peter Salus in his Open Source Library - Papers gives the following version of events:

The original UNIX editor was ed. It was a line editor of reluctant and recalcitrant style. When UNIX (version 4) got to Queen Mary College, London, in 1973, George Coulouris -- a Professor of Computing -- wasn't happy with it. So he wrote a screen editor, which he called "em," or "ed for mortals."

Coulouris went on sabbatical to Berkeley, where he installed em on "his" machine. A graduate student noticed it one day, and asked about it. Coulouris explained. He then went off to New Jersey to Bell Labs, and when he returned to Berkeley, he found that em had been transmuted into ex, a display editor that is a superset of ed and a number of extensions -- primarily the one that enables display editing.

At the beginning of 1978, the first Berkeley Software Distribution was available. It consisted of a tape of the Berkeley Pascal System and the ex text editor. The graduate student was Bill Joy, and the distribution cost $50. The next year Berkeley got some ADM-3a terminals, and Joy rewrote em to vi -- a truly visual editor.

In sum, ed came out of Bell Labs in New Jersey, went to Queen Mary College in London, from there to the University of California at Berkeley, and from there back to New Jersey, where it was incorporated into the next edition of UNIX.

NEWS CONTENTS

• 20111108 : Two decades of productivity Vims 20th anniversary by Ryan Paul ( November 2, 2011 )
• 20030829 : The Text Editor Sam by Rob Pike. ( The Text Editor Sam, Aug 29, 2003 )
• 20001010 : vimstory ( Oct 10, 2000 )

Old News ;-)

[Nov 08, 2011] Two decades of productivity Vim's 20th anniversary by Ryan Paul

November 2, 2011

The Vim text editor was first released to the public on November 2, 1991-exactly 20 years ago today. Although it was originally designed as a vi clone for the Amiga, it was soon ported to other platforms and eventually grew to become the most popular vi-compatible text editor. It is still actively developed and widely used across several operating systems.

In this article, we will take a brief look back at the history of vi and its descendants, leading up to the creation of Vim. We will also explore some of the compelling technical features that continue to make Vim relevant today.

Prehistory

The vi text editor was originally created in the late '70s by Bill Joy, an early BSD developer who later went on to cofound Sun Microsystems. The original implementation of vi was conceived as an interactive "visual" mode for an ed-like line editor called ex. It was developed at first on an old ADM-3A terminal, about a decade before computer mice became ubiquitous. Users relied on commands and keyboard-based navigation to interact with the editor.

The limitations and idiosyncrasies of the ADM-3A influenced some of vi's most distinctive characteristics. For example, vi was programmed to use the h, j, k, and l keys for directional navigation because the ADM-3A keyboard had arrows printed on the keys with those letters. Although it was an accident of history, the hjkl navigation model in vi proved to be enormously efficient. The combination of those letters has become practically iconic among vi users. The navigation scheme is used today in a variety of other applications, including Gmail and the official Twitter client for Mac OS X.

The vi editor became an inseparable part of the UNIX landscape. Joy bundled it with BSD and ATT incorporated it into System V. The core functionality and behavior of vi was later specified in the POSIX standard, which led to the inclusion of the text editor in many of the major UNIX systems.

The vi clones, which began to emerge in the late '80s and early '90s, were widely adopted due to more permissive licensing. Joy's vi implementation was based on the original ATT version of ed, which meant that the code was not freely redistributable. It could only be used by parties that had a UNIX license from ATT.

The first two prominent vi clones were Stevie and Elvis. Stevie, the ST Editor for vi Enthusiasts, was originally developed for the Atari ST in 1987 and ported to UNIX the next year. It was somewhat primitive but attracted a modest following. Elvis, which was first released in 1990, was more sophisticated and was designed to offer more functionality. Elvis was the first vi clone to introduce support for syntax highlighting.

Although Elvis still has some users and remains popular in the Slackware community, it hasn't seen a major update since 2003. Elvis replaced Joy's vi in the original 386 port of BSD, but the BSD developers later produced a new clone called nvi that was intended to more closely match the behavior of Joy's implementation. The nvi editor is still shipped today by the BSD family of operating systems.

History

The earliest version of Vim was developed on the Amiga by Bram Moolenaar in 1988. Moolenaar was dissatisfied with the vi clones that were available for the Amiga platform and set out to make one that came closer to matching vi's feature set. He based his new editor on Stevie, which he has said was the best Amiga-compatible vi clone at the time.

The first version of Vim that was released to the general public was 1.14, which was published on November 2, 1991. It was distributed on Fish Disk #591, one of the disks in Fred Fish's Amiga freeware collection. Following its public debut, users began contributing patches.

"A long time ago I got myself an Amiga computer. Since I was used to editing with Vi, I looked around for a program like Vi for the Amiga. I did find a few so-called 'clones', but none of them was good enough; so I took the best one, and started improving it. At first the main goal was to be able to do all that Vi could do. Gradually I added some additional features, like multi-level undo," Moolenaar wrote in the first issue of Free Software Magazine, back in 2001. "When I started working on Vim it was just for my own use. After some time I got the impression it was useful for others, and sent it out into the world. Since then I'm working more and more on making the program work well for a large audience. It's fun to create something useful. Also, there is a nice group of co-authors and power users, which is very inspiring."

Moolenaar drafted his own loose copyleft license for the software. The license grants the user broad freedom to use, distribute, and repurpose the code, but gives the maintainer the right to ask for improvements to be contributed back to the project. Some clarifications were added to Vim's license in the version 6.0 release to ensure compatibility with GNU's General Public License (GPL).

Vim is an open source software project, but it's also charityware. Moolenaar helped establish a foundation called ICCF Holland that works to support to a children's center in Uganda. He encourages users to consider making a donation to ICCF Holland or the Kibaale Children's Fund. He serves as treasurer of the foundation and visits the site in Uganda nearly every year to monitor the center's progress.

The name "Vim" originally stood for Vi IMitation, but it later became Vi IMproved. The name was changed in 1992 when version 1.22 was released with compelling new features and a UNIX port. It gained support for multiple buffers in version 3.0, which was released in 1994. Version 4.0 in 1996 introduced a graphical user interface, version 5.0 in 1998 added syntax highlighting capabilities, and version 6.0 in 2001 added support for vertical window splits and a plugin system that simplified script loading.

Many, many other features were added along the way. Vim has been ported to numerous other platforms, including Linux, BeOS, Windows, Mac OS X, and QNX. Although Vim was originally designed to be used in a terminal, there are several graphical frontends built with various user interface toolkits that add conventional menus, toolbars, and scrollbars.

The latest major release of Vim was version 7, which was released in 2006. It introduced some particularly significant features, including native support for spellchecking, an autocompletion system, a tab interface, and undo branches. The most recent minor version update was version 7.3, released last year, which introduced a persistent undo feature and support for Python 3.

... ... ...

The full scope of Vim's advantages are difficult to articulate to users who are unfamiliar with the editor. Although a full explanation of how Vim works is beyond the scope of this article, the following is a small assortment of useful features:

• Flexible multiple document interface: In Vim, your files and unsaved documents are referred to as buffers. The editor gives you a tremendous amount of control over how your buffers are displayed on the screen. You can horizontally and vertically split the window as many times as you want so that you can view many buffers at the same time. You can even have the same buffer displayed in multiple splits, which lets you view separate parts of the same document simultaneously. You can also optionally organize sets of split layouts into tabs. Layouts and state can be preserved by saving a "session" and restoring it later.
• Modal editing with sophisticated keyboard shortcuts: Vim has separate interaction modes for text input and text editing. Insert mode behaves largely as you would expect a regular text editor to work-commands are performed with conventional keyboard shortcuts and characters are appended to the buffer as you press the associated keys. In the "normal" mode, however, sequences of key presses perform commands. The most useful commands allow you to quickly navigate and manipulate the text in the buffer. You can extensively customize the behavior of the bindings to create custom shortcuts and commands.
• Multiple clipboards: Instead of a conventional clipboard, Vim stores copied text with a mechanism that it calls registers. There is a default register that is used to store text that is copied by the standard yank and delete operations, but the user can also indicate a specific register in which they want to store text when they cut or copy. This effectively acts like a clipboard multiplexer. The contents of the registers persist between uses of Vim, which means that they are preserved when you quit and will still be there when you open the editor again.
• Macros: Vim has a macro system that allows you to record keypresses for later playback. Macros are trivially easy to create from the keyboard and can consist of operations across multiple Vim modes. Macros are stored in registers, just like the clipboard items. As such, they can also persist across uses of the application.
• Extremely powerful search capabilities: Vim has some very sophisticated tools for automated search and replace, including extensive support for regular expressions. It also has a built-in version of the grep command, which integrates with Vim's enormously convenient quickfix feature-a special buffer that shows you a list of results and allows you to conveniently jump between them.
• Extremely rich extensibility: Vim is prodigiously scriptable and highly conducive to automation. It has its own native scripting language with container types, a unique variable scoping model, and a bunch of useful Vim-specific functionality. It also has built-in scripting engines and bindings that allow it to be customized via numerous mainstream programming languages, including Perl, Python, Ruby, Tcl, and Lua. Vim can also be extended to add syntax highlighting for additional languages or create custom color schemes. Users widely share their scripts through various online repositories and package them into plugins. As we have previously demonstrated, installing a few simple plugins and scripts can give Vim many of the advanced capabilities of an integrated development environment.
• Portability: Vim will work almost everywhere that you do. Vim is widely used on Windows, Linux, and Mac OS X and is available for many other platform. Users can run it from the terminal or operate it with a native graphical interface on all three major operating systems. Many system administrators value Vim because it gives them a productive text editing environment on practically any remote Linux or Mac OS X system that they access through the terminal via ssh.

[Aug 29, 2003] The Text Editor Sam by Rob Pike.

An interesting attempt to modernize vi. This paper is reprinted from Software­Practice and Experience, Vol 17, number 11, pp. 813-845, November 1987. If you want to play with it there is a UNIX version at:
ftp://netlib.bell-labs.com/netlib/research/sam.shar.gz
and a Windows version (currently distributed in binary form only) at:
ftp://netlib.bell-labs.com/netlib/research/sam.exe

the editor has some interesting features, for example more flexible piping constructs:

There are also three commands that apply programs to text:

< UNIX program

replaces dot by the output of the UNIX program. Similarly, the > command runs the program with dot as its standard input, and | does both. For example,

| sort

replaces dot by the result of applying the standard sorting utility to it. Again, newlines have no special significance for these sam commands. The text acted upon and resulting from these commands is not necessarily bounded by newlines, although for connection with UNIX programs, newlines may be necessary to obey conventions.

[Oct 10, 2000] vimstory

The paper was reformatted from the text original --NNB...

Oct 10, 2000

Paper submitted for the Linux200.nl conference, 9-10 Oct 2000 in Ede

The continuing story of Vim
by Bram Moolenaar

The development of Vim (Vi IMproved) started in 1988 as a small program for the Amiga, used by one person. It is now included with every Linux distribution and has been given an award for the best open-source text editor. This article will discuss the current status of Vim, placing it in the context of the past and the future.

Vi compatible

Vim started as a replacement for Vi on the Amiga. Being used to Vi on Unix systems, the author wanted to use this powerful editor on his newly obtained Amiga too. There was a program called "Stevie", which lacked many commands and contained bugs; but since the source code was available, it was possible to enhance the program. Gradually more Vi commands were added and problems fixed. Then new useful commands were added that Vi didn't have: multi-level undo, text formatting, multiple windows, etc. At that point it was renamed from "Vi IMitation" to "Vi IMproved".

But Vim still tries to be very Vi compatible, if that is what you want. For most commands you will not notice any difference between Vi and Vim. But some Vi commands work in a clumsy way and some may be considered a leftover from the old days of slow computers and teletypes. Here Vim gives the user a choice of doing it the old Vi way, or doing it in an improved Vim way.

For example, in Vi the "u" command toggles the text between the situation before and after a change. Vim offers multi-level undo. What commands to use to get to the multiple levels? One way would be to use the "." command to repeat the "u" command. Nvi follows this approach. But this is not Vi compatible: Typing "xxu." in Vi deletes two characters: The "u" undoes one "x" and the "." repeats one "x" again. In Nvi the "." repeats the undo, thus both "x" commands are undone and you end up with no change.

The author of Vim doesn't like these unexpected and obscure incompatibilities. Another solution would be to use another command to repeat the undo or redo. In Vim this is CTRL-R, "R" for "repeat". Thus "xxu^R" is used to undo both "x" commands. This is both Vi compatible and offers the multi-level undo feature. Still, typing CTRL-R requires using two fingers. Since undo is an often used function, it should be easy to type.

Many people prefer to repeat the "u" command to undo more. Then CTRL-R is used to redo the undone commands. Thus "u" goes backwards in time and CTRL-R forward again. Since this is not compatible with Vi, it has to be switched on with an option.

What a user prefers often depends on his previous experience. If he has used Vi for many years, his fingers are trained to hit "u" and expect the last change to be toggled. But people who start with Vim find it strange that "u" toggles and prefer it to always perform an undo. For these matters of user preference Vim offers an option.

In Vim you can set "nocompatible", to make Vim behave more nicely, but in a not fully Vi compatible way. If you want, you can carefully tune each compatibility aspect by adding flags to the 'cpoptions' option. This is a typical Vim choice: offer a good default to make most people happy, and add options to allow tuning the behavior for personal preferences.

The current version of Vim is very much Vi compatible. One of the last things that has been added was the Ex mode and the "Q" command. Not many people use Ex mode, it was added to be able to execute old Vi scripts. One thing that's still missing is the open mode. Since this is really only useful when you are using a very primitive terminal, hardly anyone will miss this - most Vi users don't even know what it is.

There are still a number of small incompatibilities to be solved - you could call these bugs. Work on these continues, but it's very likely that Vim already contains less bugs for Vi commands than Vi itself.

Programmers aide

Many of the features that have been added to Vim over time are for programmers. That's not unexpected, since Vim is often used to edit programming languages and similar structured text, and the author himself does a lot of programming.

One of the first programming aids to be added was the "quickfix" feature. This was actually present in the Vi-like editor "Z" that came with the Amiga C compiler from Manx. Since it was very useful, it was added to Vim, using the "Z" editor as an example.

The "quickfix" feature allows the programmer to compile his code from within Vim and quickly fix the reported errors. Instead of making the user write down the line numbers and messages from the compiler, Vim parses the compiler output and takes the user directly to the location of the error, putting the cursor at the position where the error was reported. You can fix the problem and compile again with just a few commands. There are commands to jump to the next error, list all errors, etc. An option is used to specify how to parse the messages from the compiler, so that it can work with many different compilers.

"quickfix" also works with a command like "grep", which outputs lines with a file name and line number. This can be used to search for a variable and all places where it's used in the code, comments and documentation. Not only does this reduce the time needed to make changes, it also minimizes the risk of missing a location.

When editing programs you will type text once and read it many times. Therefore it is very important to easily recognize the structure of the code. Since everybody uses his own style of coding, and not everyone pays enough attention to the layout, the code may take time to understand. Highlighting items in the text can be of help here. For example, by making all comments blue it is easy to spot a short statement in between long comments. It's also easier to recognize the structure of the file when quickly paging through it.

Highlighting keywords can help spotting errors. The author has a tendency to type "#defined" instead of "#define". With highlighting switched on, the first one stays black, while the second one becomes brown. That makes it easy to see this mistake when it happens. Unmatched closing parentheses can be marked as an error, with a red background. That is a great help when changing a complicated "if" statement or Lisp code.

Highlighting is also useful for the last used search pattern. For example, when searching for the "idx" variable in a function, all its occurrences in the code will get a yellow background. That makes it very easy to see where it is used and check where its value is changed.

The syntax highlighting is completely adjustable. Everybody can add his own language if it's not already included with the Vim distribution. But since there are syntax files for about 200 languages now, mostly you just have to switch the syntax highlighting on and your files are coloured. Thanks to all the people who have submitted and are maintaining syntax files for everybody to use.

Folding

In 1998 Vim 5.0 was released. The question was: What next? A survey was held to ask Vim users which features they would like to see added to Vim. This is the resulting top ten:

1. add folding (display only a selected part of the text) (*)
2. vertically split windows (side-by-side) (*)
3. add configurable auto-indenting for many languages (like 'cindent') (*)
4. fix all problems, big and small; make Vim more robust (+)
5. add Perl compatible search pattern
6. search patterns that cross line boundaries (*)
7. improve syntax highlighting speed (*)
8. improve syntax highlighting functionality (+)
9. add a menu that lists all buffers (*)
10. improve the overall performance (+)

The goal for Vim 6.0 was to implement a number of these items - at least the top three - and this has actually taken place. The items marked with (*) have been implemented in Vim 6.0. The items marked with (+) are on-going activities. The number one requested feature deserves more explanation.

Folding means that a range of lines is displayed as one line, with a text like "34 lines folded: get_arguments()", but the actual text is still there and can be seen by opening the fold. It is as if the text were on a long roll of paper, which can be folded to hide the contents of each chapter, so that you only see the chapter titles. This gives a very good overview of what a file contains. A number of large functions, occupying thousands of lines, can be viewed as a list of function names, one per line. You can move to the function you want to see and open the fold.

Adding folding was a lot of work. It has impact on all parts of Vim. The displaying of text is different, and many commands work differently when the cursor is in a fold, but it mostly works now. And there are actually several ways to use folding:

1. Manually: use commands to create and delete folds. This can also be used in a function to fold your text in any way you like.
2. On syntax: use the syntax highlighting mechanism to recognize different items in the text and define folds with it.
3. On indent: the further a line is indented the deeper it will be folded.
4. By expression: define an expression that tells how deep a line is folded.
5. By markers: put markers in the text to specify where a fold starts and ends.

Why so many different ways? Well, because the preferred way of folding depends on both the file you are editing and the desires of the user. Folding with markers is very nice to precisely define what a fold contains. For example, when defining a fold for each function, a comment in between functions could belong to the previous or the next function. But if you edit files in a version-controlled project, you are probably not allowed to add markers. Then you can use syntax folding, because it works for any file in a certain language, but doesn't allow you to change where a fold starts or ends. A compromise is to first define folds from the syntax and then manually adjust them. But that's more work.

Thus the way folding was implemented in Vim is very flexible, to be able to adjust to the desires of the user. Some related items have not been implemented yet, like storing the folding state of a file. This is planned to be added soon, before version 6.0 goes into beta testing.

Indenting

Giving a line the right indent can be a lot of work if you do it manually. The first step in doing this automatically is by setting the 'autoindent' option. Vi already had it. It simply indents a line by the same amount as the previous line. This still requires that you add space below an "if" statement and reduce space for the "else" statement.

Vim's 'cindent' option does a lot more. For C programs it indents just as you expect. And it can be tuned to follow many different indenting styles. It works so well that you can select a whole file and have Vim reindent it. The only place where manual correction is sometimes desired is for continuation lines and large "if" statements. But this only works for C code and similar languages like C++ and Java.

Only recently a new, flexible way of indenting has been added. It works by calling a user defined function that returns the preferred indent. The function can be as simple or as complex as the language requires. Since this feature is still new, only indenting for Vim scripts is currently included in the distribution. Hopefully people will write indent functions for many languages and submit them to be included in the distribution, so that you can indent your file without having to write an indent function yourself.

A disadvantage of using a user defined function is that it is interpreted, which can be a bit slow. A next step would be to compile the function in some way to be able to execute it faster. But since computers keep getting faster, and indenting manually is slow too, the current interpreted functions are very useful already.

Recommended Links

vimstory