Why the postulate about high quality of OSS products in comparison with commercial is partially a myth ?

Raymondism postulates the higher quality of open source. Proprietary software is considered as a sub-optimal solution or, usually sub-optimal. Here is a relevant ESR quote (Quoted in Slashdot Thus Spake Stallman with reference to Salon Sept- Oct 1998.):

"Either open source is a net win for both producers and consumers on pure self-interest grounds or it is not. If it is, you cannot lose; if it is not, you cannot (and *should* not) win."

Actually in my second paper I discussed this problem. I would like to add just one interesting example from Linux Kernel Mailing List:

Re: Update: Subtle data corruption of TCP streams

From: Wietse Venema ([email protected])
Date: Sat Mar 25 2000 - 00:20:12 EST

Next message: Jeff Dike: "Re: VM modules in kernel?"

• Previous message: Brad Whitehead: "Source Routing"
• In reply to: Austin Schutz: "Re: Update: Subtle data corruption of TCP streams"
• Next in thread: David S. Miller: "Re: Update: Subtle data corruption of TCP streams"
• Reply: David S. Miller: "Re: Update: Subtle data corruption of TCP streams"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Seems that routers don't do the packet rewriting that I'm observing here. It's the domain of bandwith management systems. There are at least four players. I got reactions from several.

Disabling TCP options in the Linux kernel does help, especially when you're talking to non-Linux systems.

        Wietse

Austin Schutz:
> [bugtraq removed from cc:]
>
> I've noticed this problem on my network. I'm behind an Ascend pipeline 50. I had thought that this >might be the root of the problem. Supposedly there's something similar reported as a problem with
> older ascend software.
>
> Any idea what the other people experiencing the problem are using as gateway routers?
> Austin
> On Fri, Mar 24, 2000 at 04:26:13PM -0800, Blu3Viper wrote:
> > How about taking this to linux-kernel for discussion since it appears to be
> > a development kernel bug possibly?
> >
> > -d
> >
> > On Fri, 24 Mar 2000, Wietse Venema wrote:
> >
> > > Date: Fri, 24 Mar 2000 10:38:36 -0500
> > > From: Wietse Venema <[email protected]>
> > > To: [email protected]
> > > Subject: Update: Subtle data corruption of TCP streams
> > >
> > > Apparently, once instance of this data corruption problem is caused
> > > by an unnamed bandwidth management system. It runs as a bridge,
> > > and does not show up in traceroute etc. output. We were able to
> > > estimate its location (at 5 ms round-trip time from one endpoint)
> > > by analyzing packet arrival times.
> > >
> > > Until now, this TCP data corruption problem has been observed only
> > > when one of the connection endpoints runs a recent LINUX version.
> > > Sightings have been reported by sites in Germany and in France.
> > >
> > > Only recent LINUX versions request the use of timestamp options
> > > that cause the tell-tale patterns of "01 01 08 0a" in TCP packets,
> > > and that end up being regurgitated as ^A^A^H^J data.
> > >
> > > I have updated the analysis at ftp://ftp.porcupine.org/pub/debugging/
> > >
> > > Wietse
> > >
> > > Wietse Venema:
> > > > This note is about a subtle data corruption problem with TCP data
> > > > streams that may bite people as more and more (LINUX) systems are
> > > > sending network traffic with TCP-level options turned on.
> > > >
> > > > Last week, several Postfix users reported mail delivery failures
> > > > because sequences of control characters (for example, ^A^A^H) were
> > > > being inserted into their SMTP connections, resulting in SMTP
> > > > protocol errors and non-delivery of email.
> > > >
> > > > These data corruption problems are not host specific: they are
> > > > observed with both Linux and BSD/OS systems, and with mail sent to
> > > > and/or received from systems running Postfix, Sendmail and qmail.
> > > >
> > > > Over the weekend of March 18, 2000, a few people left tcpdump
> > > > running on their machines, in order to record some of these corrupted
> > > > SMTP sessions. This note is based on an analysis of that data.

There is also one more paradox here. According to ESR, Open Source is a superior engineering model than Closed Source and hence Open Source Software should be more reliable and stable than most Closed Source Software. Paradoxically that also means that Open Source Software should need less support than Closed Source software. But here there is one painful for ESR logical consequence: Open Source companies that are relying on support as a source of revenues will never be as attractive to investors because closed source companies will always have larger profit margins ;-).  Therefore, paradoxically, by ESR's own logic there are very little faith in any Linux company or other OSS pure-play ever becoming a very dominant company or even surviving in a long run.

But things are not that bad :-). Quality of open source products varies greatly, much like quality of closed source products:

Actually quality of some parts of the Linux kernel, for example the process scheduler, is relatively lame. In no way they are the best available solutions for the Unix kernel. Free/OpenBSD still has  a better engineered kernel. As another Slashdot reader put:

"I use OpenBSD not because I necessarily like or agree with everything Theo has done that may be controversial over the years. I use OpenBSD because, all things considered, it's a damn good OS. The developers work hard with a primary goal of producing the best code, not just code-that-works-and-supports-latest-doohickey."

Probably the same reasoning can be applied to Sendmail (which as a software product is a mess that produced some of the most famous security holes in the history of Unix) vs. Postfix. And the list can be continued indefinitely.

Some Linux companies try to find ways to improve the quality of open source applications.  For example regexp.com proposed  to restore a commercial model of paying for a close access to developers and early releases Quality Costs Money regexps.com Pioneers R&D Strategy for Open Source:

The commercial success of Linux is not surprising. Linux is good stuff. But is it ready for the end-user application market?

A lot of Linux projects, though they produce tantalizing applications, miss the boat on producing quality products: documentation is shoddy or missing, testing is inadequate, developers work without feedback from users outside of the community of hackers, release engineering is uneven, and the applications have bugs, bugs, bugs.

Proprietary software companies, like Microsoft, spend many millions on testing and user studies and use that spending to steer the R&D process. That feedback from market to developers, more than any other single factor, earns customers for the Microsofts of the world.

At the same time, the Linux community, and the Free Software community in general, produces a lot of software with very little tangible compensation. Yet every major computer manufacturer now promotes Linux. Several companies with large revenues sell Linux distributions.

What's wrong with this picture?

regexps.com thinks it has the solution: a market-driven development process in which paying customers, those who sell to end-users, become subscribers to R&D efforts. Subscribers pay for an intimate relationship with development teams: helping to prioritize development with feedback from the demands of the market; calling developers attention to where better QA is needed; using funding incentives to organize separate teams of developers into a more coherent whole.

We already discussed this issue and to summarize the discussion I would like to reiterate that quality is not automatic, it depends on the personality of the leader, underling algorithms, architectural integrity and the quality of code (including right implementation language).  Minimalist approach is usually more secure. That's why Linux will never be as secure as OpenBSD. See also RedHatIsNotLinux.Org

Maturity is also important for software tools because of the complexity. It's impossible to write a perfect OS or a perfect compiler in six month no matter what. And what is the most important: like there is no replacement for displacement, there is no replacement for the talent in software engineering.  Open source is not a panacea, it can help, but you still need a talent, right tools and a lot of time. As for development tools the quality is really a myth and the recent study revealed severe dissatisfaction among OSS developers with the tools currently available for the Linux platform. Nine of the eleven 'tools' categories received satisfaction ratings under 50%, including OSS debuggers, profilers, modeling tools, error detection tools, GUI Frameworks, testing tools and code management tools:

Monday, March 27, 2000 - Evans Marketing Services released what is being referred to as the most comprehensive research study of Linux developers ever. Compiled throughout March, the study consists of more than 300 in-depth interviews with Linux developers on a variety of topics ranging from what applications they are working in, to which languages and distributions they are using. One of the most significant findings of the study revealed severe dissatisfaction among developers with the tools currently available for the Linux platform. Nine of the eleven 'tools' categories received satisfaction ratings under 50%, including debuggers, profilers, modeling tools, error detection tools, GUI Frameworks, testing tools and code management tools. When asked, 87% of developers didn't care if the tools were proprietary or open source. This means that the opportunity is there for a vendor to step up and fill the niche in any or all of those categories. For more information on the study, visit www.evansmarketing.com

Other interesting fact is that "When asked, 87% of developers didn't care if the tools were proprietary or open source." Of course one can attack the validity of the study, but I believe that the life is too short and developer need to use the best tools available. For example very few developers has time to modify the internals of the editor to their liking; at most they write macros. Most are using it as a closed source product even if source is available. So much for the CatB idealistic blah-blah-blah on the subject. The last but not least is that CatB mythology cannot change the stubborn fact that  probably 99% of Open Source projects never have more than one maintainer. More often then not the latter is the same as the initial developer.

Actually this is a question of choice. I believe that RMS was right when he said:

The Open Source Movement seems to think of proprietary software as a suboptimal solution (at least, usually suboptimal). For the Free Software Movement, proprietary software is the problem, and free software is the solution. Free software is often very powerful and reliable, and I'm glad that adds to its appeal; but I would choose a bare-bones unreliable free program rather than a featureful and reliable proprietary program that doesn't respect my freedom.

But it's very important to understand (and RMS here does not point out this issue, although I think he understands it too)  that paradoxically bare-bone open source program can be more productive that complex open source program or supercomplex commercial solution. But again source code in not enough. In his article Be An Engineer, Not An Artist  Monty Manley wrote:

We can refine software quality into the following points:

1. Efficiency. All other things being equal, a smaller program is better than a big one because it achieves the same result in a smaller space. Efficient programs use fewer resources and run faster. ("Bloat" is a curse in the developer community, and for good reason.)
2. Design. A good program exhibits a structure and flow, even when they are very complicated. Too many programmers just sit down and start banging out code without designing the program first; this leads to a program that is obfuscated, inflexible, and hard to hand off to other developers.
3. Modularity. Good programs can be extended and enhanced without requiring a major rewrite. (Modularity is hard to achieve without a good design -- see previous point.)
4. Robustness. Good programs anticipate the unexpected and gracefully handle problems. (It's worth noting that a lack of QA is something that devils Linux badly.)
5. Documentation. Good programs are *documented*, both internally as comments and externally as developer/user documentation. No program can be considered finished until the documentation is completed. And yet the vast majority of Open Source projects suffer from incomplete, inaccurate, or missing documentation.

There is a lot to be said for art, but there is still more to be said for *craft*. The hard fact is that the important parts of writing a good program are the boring parts -- the "sexy" stuff is usually less than 30% of the project. Unfortunately, Open Source projects tend to stall when the "sexy" stuff is done. The developers lose interest and move on to other "sexy" stuff. If you consider the high-profile projects (Mozilla, AbiWord, Samba, etc.), you discover that about 10% of the developers do 90% of the work -- in other words, all those legions of programmers do essentially nothing for the project.

I used to be amazed that there was so much duplication in the Linux software space. How many FTP clients or newsreaders does one platform need? How many XClocks? CD players? Window managers? And the hell of it is, few of these programs are actual *refinements*; they are simply the same thing done a different way. This tendency also underscores the relative immaturity of many Linux developers: they have a hard time thinking up new concepts, and so keep on reimplementing old ones.

I'd like to see Open Source software submitted to some kind of formal code-review and auditing process. Not only would it improve the quality of the code, it would reduce the endless security exploits that have deviled Unix code for three decades. OpenBSD should be a model here -- that OS has not had a remote root exploit in more than two years! However, I doubt this will happen; there is too much ego and testosterone floating around the Open Source community to make such a thing work.

I believe in Open Source, but the promise of "better software" isn't materializing, despite all the noise to the contrary. The *process* won't do it; the *developers* have to do it. We need to be engineers first and artists second.

Unix was designed as a development OS. That's why historically it was extremely popular at universities. And because Linux belong to the Unix tradition or school of thought if you wish, we need to understand that most Open Source software was written by developers who will use their products, which means Open Source software is written by developers for developers or at least system administrators. Developers typically love to configure everything via a configuration file, love to do everything with their keyboard without repeating the same task twice, etc. Not all categories of users share this love ;-).