LINFO

The Unix Philosophy: A Brief Introduction



The Unix philosophy lies at the core of not only the original UNIX as developed by Ken Thompson at Bell Labs from 1969 but also of its numerous direct descendants and clones, including Solaris, the BSDs1 and Linux. It has made them, collectively referred to as Unix-like operating systems, into what are widely considered to be the best2 operating systems to have been developed to date, despite the fact that they are by far the oldest operating systems in widespread use, and it has been a major factor in the rapid growth and increasing success of Linux.

There is no single, standardized statement of the philosophy. But if it had to be described with only a single word, that word would be modularity, which refers to a system that is composed of components (i.e., modules) that can be fitted together or arranged in a variety of ways.

Modularity is common in nature, and its application to man-made products (both goods and services) has been a key factor in the development and advance of industrial societies. Yet, it was relatively little used for computer software prior to the development of UNIX, and even today its great benefits fail to be fully exploited by other operating systems, most notably the Microsoft Windows systems.

A slightly longer, and more conventional, statement of the Unix philosophy, but one which says essentially the same thing, would be: Design programs to do only a single thing, but to do it well, and to work together well with other programs3.

The philosophy is commonly described with a series of brief rules or tenets4. Although they all generally follow logically from the concept of modularity (e.g., small size, efficiency, simple interfaces and ease of understanding), it can be useful to state and discuss them individually.

Among these tenets is the the so-called rule of composition, which states that programs (of which a modular operating system is composed) should be designed to be easily connected to other programs. Another is the rule of silence, which states that programs should, by default, say nothing (i.e., have no output) other than that which is interesting, unusual or surprising.

A serious attempt is made to apply the principle of modularity to everything on a Unix-like system, not only programs but also parts of programs, such as algorithms, and even the kernel (i.e., the core of the operating system). Thus, a Unix-like operating system generally (or at least ideally) consists of a small kernel together with a large number of small, specialized programs that can interact with each other through a variety of well-defined interfaces.

The most familiar of these interfaces, and one of the most important innovations of UNIX, is the pipe. Represented by the vertical bar character in commands typed in by the user, pipes allow the combining of programs so that the output of one becomes the input of another. Such pipelines of commands make it possible to easily perform highly specialized operations that would be difficult or virtually impossible using a non-modular system.

Another major tenet of the philosophy is to use plain text (i.e., human readable alphanumeric characters) rather than binary files (which are not fully human readable) to the extent possible for the inputs and outputs of programs and for configuration files. This is because plain text is a universal interface; that is, it can allow programs to easily interact with each other in the form of text outputs and inputs, in contrast to the difficulty that they would have if each used mutually incompatible binary formats and because such files can be easily interfaced with humans. The latter means that it is easy for humans to study, correct, improve and extend such files as well as to port (i.e., modify) them to new platforms (i.e., other combinations of operating systems and hardware).

In addition to making Unix-like operating systems more efficient and easier to use, the Unix philosophy also facilitates their development and improvement. This is because the use of small, specialized (i.e., modularized) programs makes it far easier for developers to improve them than would be the case with large, multifunctional programs. One reason is that smaller programs can be sufficiently small and simple so that they can be comprehended by a single human mind, whereas large and complex programs generally cannot. It is also because such specialization makes it practical for development (including improvements) to be widely scattered rather than being concentrated at one or a few central locations.


Historical Context

The Unix philosophy grew out of the original design goal of UNIX, which was to create an operating system that was as simple and efficient as possible. This goal was a reaction to what Thompson correctly viewed as the unnecessary complexity of the operating systems that were in use at that time. Such complexity was related to the fact that there was no standard operating system that could be used on a wide variety of computers; rather, each computer manufacturer developed a separate operating system for its own hardware.

Another important factor in Thompson's desire for maximum simplicity and efficiency was the fact that he initially wrote his operating system for a computer, the PDP-7, that had an extremely small main memory of only 4000 18-bit words. (However, this was considered quite adequate when the computer was built in 1965, especially given its low cost, of only U.S.$72,000 for a basic model.)

Although the size and complexity of Unix-like operating systems has increased greatly over subsequent decades in response to the continuous drop in memory costs and increases in CPU (central processing unit) performance, the legacy of the original simplicity lives on, and Unix-like operating systems are still far more modular than most other systems, most notably the Microsoft Windows systems.

The decline in UNIX market share that began in the 1990s relative to the Microsoft Windows systems was sometimes attributed to the perception that UNIX was an old, clunky system that had outlived its usefulness. However, it turned out that this decline was, in fact, not due to any inherent defect in either the philosophy or the operating system itself, nor was it due to any inherent superiority of the Microsoft systems. Rather, it was the result of the fact that the philosophy was designed for a different era, one in which software was treated like mathematics or scientific knowledge. That is, software was something to be shared and improved for everyone's benefit rather than something to be hoarded for the sake of increasing corporate profits.

Thus, rather than discarding the philosophy, something additional was needed so that operating systems based on it could again prosper and grow in this new environment. This addition came in the form of the concept of free software licensing, which keeps software free not only in a monetary sense but also with regard to use (including copying, modifying, extending and redistributing). Particularly important in this context was the development of the concept of copyleft, which precludes the emergence of dominant versions that require the use of expensive hardware and that also promotes compatibility among the various versions without inhibiting innovation. The result has been a returning ownership of Unix-like operating systems and application programs for use with them to the public as a whole and a consequent reinvigoration of technological development.


Contrast With Proprietary Systems

The Unix philosophy is clearly very different from the philosophy behind the Microsoft Windows operating systems. In fact, those systems are characterized by some of the opposite qualities. Most fundamentally, they are monolithic (i.e., lack modularity). Specific characteristics include extremely large program sizes, great complexity of the source code, an apparent lack of craftsmanship on many aspects (such as poor security), a lack of transparency (the source code is a closely guarded secret) and the absence of portability (i.e., can only run on one basic type of processor).

Source code (also referred to as source or code) is the version of software (usually an application program or an operating system) as it is originally written (i.e., typed into a computer) by a human in plain text. It can be written in any of numerous programming languages, some of the most popular of which are C, C++, Java, Perl, PHP, Python and Tcl/Tk.

This sharp contrast is only natural, as the Microsoft Windows operating systems, like much other proprietary (i.e., commercial) software, originated in a different era and are governed by a very different set of priorities. That is, the emphasis is on gaining or maintaining market share and maximizing profits, and little attention is paid to the priorities of the original UNIX developers. The very different development model (e.g., centralized versus dispersed development) also likely plays a role.

Of course, there is nothing necessarily wrong with these market-oriented goals. In fact, they are, in general, beneficial, as they are what has been responsible for much of the tremendous economic growth that has occurred around the world during the past millennium. Yet, as economists would be quick to point out, these goals work for the maximum benefits to society only where free competition (i.e., many sellers and buyers competing on the basis of price and quality) exists, and not where markets have (for whatever reasons) become monopolistic (i.e., characterized by only a single supplier or seller of a product with no good substitutes).

In all fairness, it should be mentioned that the various Unix-like operating systems have not always completely conformed to their idealized philosophy either. In a way, these goals are a bit like democracy -- there is a tendency to stray, but the inherent virtue of the concept keeps pulling it back.


Beyond Proprietary UNIX

The great success of UNIX and its philosophy has had effects far beyond the original operating system as it was developed at Bell Labs and subsequently at the University of California at Berkeley (UCB). Perhaps most significantly, they form the basis for Linux, which is now the most rapidly growing operating system and which many computer experts expect could become the dominant system for many applications in the future.

Linux has been able to bring the advantages of the Unix philosophy back into prominence and restore Unix-like operating systems taken as a whole to their former leadership role largely through the use of free software licensing. Also fundamental to its success, and the growing success of other free software as well, has been the availability of the Internet for allowing collaboration of programmers around the world. This has done a great deal to level the playing field and allow free software to compete effectively with proprietary software (and possibly even endanger the proprietary software development model) despite the massive financial resources of the latter.

In fact, the use of free software licensing together with the ability for programmers (as well as testers and general users) to participate in the development process irrespective of geographical location or organizational membership has further extended the basic concept of the Unix philosophy (i.e., modularity) and has resulted in a large increase in benefits. This is what could be termed the Linux philosophy.

In addition to operating systems, the Unix-philosophy has also played a major role in the swift development and astonishing success of the Internet. It has accomplished this not only by providing an outstanding operating system on which to base much of that network, but also by demonstrating the value of modularization for both the hardware and software aspects of it, which are major factors in its outstanding performance, including its high degree of robustness and scalability.


________
1In addition to FreeBSD, NetBSD and OpenBSD, this also includes Darwin, which is at the core of the increasingly successful Mac OS X.

2Unix-like operating systems are generally regarded as the most secure, the most robust (i.e., the most resistant to malfunctioning under unusual or stressful circumstances), the most portable (i.e., i.e., the easiest to adapt for use on other types of hardware) and the most configurable.

3Doug McIlroy, the inventor of Unix pipes, summarized the philosophy in Peter H. Salus's 1994 book A Quarter Century of Unix as: "Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface." This is usually severely abridged to "Do one thing, do it well." Of the three parts, only the third is specific to Unix-like operating systems.

4The philosophy was well expressed by Mike Gancarz in his 1995 book The UNIX Philosophy as a set of major and minor tenets. The former include: small is beautiful make each program do one thing well, build a prototype as soon as possible, choose portability over efficiency, store numerical data in flat files, use software leverage to your advantage, use shell scripts to increase leverage and portability, avoid captive user interfaces and make every program a filter. The latter include: allow the user to tailor the environment, make operating system kernels small and lightweight, use lower case and keep it short, save trees, silence is golden, think parallel, the sum of the parts if greater than the whole, look for the ninety percent solution, worse is better and think hierarchically.






Created August 6, 2006.
Copyright © 2006 The Linux Information Project. All Rights Reserved.