Dan Kaminsky's Blog

Executive Summary

Open Source Software has proven itself to be inordinately successful in providing high quality software in a method that, on its face, defies standard business reasoning. This article clarifies OSS in the language of economics such that the true reasons behind its success may be understood by everyone, not just the coders. The following summary, written by Jack Carrol, well explains the contents of this essay:

Open source software development is a barter economy with some very unusual economic properties. The original author of a piece of OSS gets paid, but not in money. The author gets paid in software. So does everybody else who contributes a feature or a bug fix. This form of payback is very real, for all of its intangibility. The return is huge — everybody who contributes to the growing program ends up with the use of a much more capable and higher-quality product than they could afford to develop on their own, and they get a product which has been carefully enhanced to meet their exact needs. The actual developers aren’t the only contributors and beneficiaries, either. Everyone who contributes documentation, distribution services, mutual tech support, or publicity participates in this barter economy.

Some other highlights:

• Source Licenses, where businesses pay to receive access to the codebase, are a far different creature than Open Source Software, as they end up placing fewer eyeballs against more bugs–in essence, it devotes less resources to more weaknesses.
• OSS means outside engineers see your code before it crashes, while it remains in the development cycle–and these engineers are likely to assist in the fixing of your bugs, saving you substantial amounts of money.
• Closed source code development forces sites that require custom features to choose between software that is too expensive to produce or code that is too unstable to trust, with significant diversion from core competencies unavoidable due to the massive redudant effort that is necessary for any complex project.
• Closed Source software providers are actually less likely to provide timely code fixes due to their organizational structure, and the “insurance premium” that managers justify paying out large amounts of money to a single supplier with actually ends up creating a support monopoly, in the sense of the supplier knows that it´d be too expensive for you to leave and absolutely impossible to find anyone else who could patch the code. LinuxCare can be founded, MicrosoftCare never can be.
• OSS merges the advantages of custom code with the advantages of Commercial Off-The-Shelf(COTS) products, essentially creating Modifiable COTS, minimizing the amount of inefficient labor that must be expended to create a product.
• OSS allows hiring managers to truly understand the skills of an individual who has coded under an Open Source license, thus increasing both his hirability and his salary range–OSS developers, quite frankly, are more bankable, especially since they have the ability to prove themselves worthy of any position, not just one that fits the position they´ve taken at a given company.
• Specialization, and Ricardo´s Theory of Comparative Advantage provide significant backing behind a free and open code architecture.
• When one examines the rest of our society, it quickly becomes clear that when stability matters, openness is the only way to achieve it. Stability matters for business. Period.

---

Introduction

Open Source Software works. A simple statement, but finally understood–code developed openly, without significant restrictions on redistribution, works better than code developed under almost any other software methodology. Business is finally beginning to accept us on our technical merits, although it still doesn´t yet understand why we´d work so hard on something we just don´t seem to be getting paid for. They know what we´ve done, just not why we´ve done it or how we´ve managed to do it so effectively. We have proved our paradigm–and considering how different it is from “established” software business practices, it truly is a paradigm–successful, and like any success story, there are many in the world who are desperate to understand why, so that they too may follow our road to success. They still have their doubts, and contrary to what some of us might think, they have good reason to doubt. Many in the industry we would drastically change have honest, good, and valid reasons for being doubtful of the seemingly siren call of Open Source. It is not enough for them to understand the empirical reality that OSS is blisteringly, frighteningly, and tremendously successful in its pursuit of speed, stability, quality, and soon, ubiquity. They must know why, in terms they understand–and, since it is always valuable to understand the perspective of users, so must we.

---

The Ghost Of Source Code Past

Analogous Key Arrays, or AKA´s, are command arrays with keyboard hotkeys that correspond to the physical location of the onscreen commands. For example, given a 3×3 array(the standard size for an AKA) of commands on screen, a 3×3 array on the keyboard should be chosen. Consider the following illustration, created from the DoxSTAR modification to the popular game Starcraft by Blizzard Entertainment.

3×3 Command Box from Starcraft with corresponding hotkeys overlaid for this illustration.

It should be noted quickly that AKA´s do not document the effects of a given command, nor do they require any graphical modification to command icons. (The graphical modifications here are for illustration purposes only.) Position alone specifies a command array as an AKA. This, of course, means the user is dependant upon some other means of determining that an AKA is being used–tooltips are the ideal means here, and were chosen for this function in DoxSTAR. AKA´s are quite powerful, but have a number of caveats to their usage. They have, I think, popped up in small numbers throughout the history of GUI´s. This document is an attempt to codify their properties. I shall attempt to explain their advantages and disadvantages with candor; this page shall be updated if additional observations come with time. Peer review is a good thing.

Much of the science that supports Pie Menus(see also this Mactech article) applies to Analogous Key Arrays. Pie Menus, unlike linear menus, place commands in a circle around a central point, thus making all commands equidistant from eachother. There´s some decent research showing this is a significantly quicker method of making commands accessible. The link applies in that since most multidimensional command arrays form pie-menu like structures anyway, a positional keyboard equivalent of those structures should gain the positive effects noted in much of the research for Pie Menus.

In fact, AKA´s should hold a substantial speed advantage over Pie Menus due to the intrinsically faster nature of key entry versus coordinate based mouse clicks. Provided a modicum of famliarity with the keyboard´s layout, it is faster to move one´s fingers to the appropriate key and press it than to move the mouse to a specific onscreen location and click. This does not, however, mean it is faster to execute a command via keyboard than via mouse–that´s just not true. Mouse access is much easier in general, because “Open File” is significantly more descriptive than “Ctrl+O”. In other words, mouse-driven actions use the screen to self-document themselves, whereas keyboard driven actions usually require shortcuts to be memorized.

AKA´s are therefore designed to address this state of affairs. The first thing they do is maximize the intrinisic advantage of the keyboard. With all keys in the AKA being in close proximity, travel time to execute a given command is significantly reduced–Fitts´ Law at work(side note: This link extends interestingly on Fitts´ Law, as well as explains it). The tight proximity also aids non-touch typists–AKA´s take the form of either positionally relative arrays(key is two left from other command) or, especially in the 3×3 case, positionally cardinal arrays(key is in the upper right hand corner). Even the touch typer benefits from the tight array, since the hand position doesn´t have to shift to contact the heavily used keys in the AKA. The intrinsic speediness of the keyboard is extended to its maximum capacity.

Where Cluehunting Comes From

(Editor´s Note: I have received an impressive and incredible amount of email regarding cluehunting, and I thank everybody who mailed me. Much of the text here needs to be rewritten to accommodate the lucid and honestly surprising quantity and quality of research people put into advancing this proposal. Some stuff regarding the true history of cluehunting does need to be modified.. Bear with me.) Cluehunting is an advanced Expansion Agent, defined as a system that allows the computer to search possible “expansions” throughout given contexts given a “clue” by the user. Clues are defined as segments of data(type irrelevant) that the computer would be able to utilize to predict the final contents of the user’s intention. Expansions are the presumed intentions of the user. Finally, contexts are the “search space” that is being scanned–the file system context, the launcher context, or even a thesaurus/spell check context are all valid options.

It would be completely unfair to describe cluehunting as a totally original concept–it stands, if you will, on the shoulders of giants. Tab-completion is the oldie, and as far as I know originated with the Unix shell tcsh, though it’s also a hidden option in the NT command shell. This technology is quite file-system specific: Enter as much as you know about a path, starting with the root, and tab complete will expand what you type to fit. For example–enter /usr/home/eff and hit tab, and you will be given the first entry in /usr/home/ that begins with “eff”. Some limited regular expressions are allowed–for example, if I’m in the directory /usr/home/effugas and type unzip *.zip, I will be able to tab through each zip file in my home directory. Very slick.

Tab Completion is nice, but it has it’s flaws. First of all, tab has become the de facto standard for “advance to next field” in GUIs, and there’s no way I want to get rid of one of the best keyboard timesavers in existence. Secondly, it searches files and only files. There are other search contexts that should be hit. Finally, tab-complete provides no way to expand into anything but a single entry–what if the didn’t want just one of the group, what if the user wanted to expand into all entries that fit the given form? In other words, instead of just one zip, all of *.zip was inserted? Would be logical in a number of situations.

Tab Complete’s newborn sibling, Autocomplete, was a web browser innovation that began at the much maligned UI shop known as Microsoft and was later adopted by Netscape for its Communicator browser. (To be as fair as possible, the emacs editor includes substantial autocomplete facilities. I am referring here to the fact that this was the first implementation of autocomplete for ordinary users, and as far as I know was the first implementation among the thousands of Windows apps over the last few years.) As Microsoft integrated Internet Explorer and Windows Explorer, both the Run Dialog and the Web Open Dialog possess Autocomplete functionality. (Actually, Microsoft Word will also Autocomplete anything you type that is related to a few known categories, i.e. date, author name, etc., but I’ll deal with this later.) So what does this bring to the table? Well, we see the beginnings of clue contexts showing up here, since at first glance it appears that the run menu will autocomplete files and the web browser will autocomplete web sites. But these are both searches of the same clue context–the history context, in which things that have been typed before are called back to be expanded back into reality. And how does Autocomplete expand entries? In the middle of typing, inverted text will appear containing the contents of what the computer is guessing the user is trying to get at. This text will only appear to the next valid level–http:// will expand to http://www.best.com, but it will not expand to http://www.best.com/~effugas nor http://www.best.com/~effugas/Personal/SILC/silc.html. There’s no way to really scroll through possible entries in this history-based autocomplete–the first thing that matches will be matched to its first level, and that’s all you get.(Ed Note: Holding shift and arrow down lets you scroll through possible autocompletes on Netscape.) Worse, sometimes a delay in typing is required to simply trigger an autocomplete. Still, this functionality is total joy, even with all of its warts.

---

What´s New In Cluehunting

Mandatory Registration: My Experience

I recently had the single worst experience in computer software–and I would never have expected it: I am referring, of course, to my experiences with Caere’s Omnipage Pro 8 software, which I had purchased from an unsuspecting CompUSA for a price of $80. (I also purchased Pagekeeper Standard, since it had finally been updated to 32 bit.)

The time was 4AM, and after installing the app, an expected dialog box had popped up with an unexpected twist. The box stated that if I did not provide my personal information to Caere within 25 uses of the application, my access to the software I had purchased would be terminated. The personal information, as demanded by the web site as a condition of using their product–honestly, I must admit I had believed that paying Caere would be sufficient–consisted of:

• Name
• Address
• Phone Number
• Email

You can view the registration page at http://www.caere.com/live/content/support/registration/prodreg.htm?. A previous edition of this page mentioned that even Microsoft feared this tactic; now I hear that they´re planning to use it in Office 2000. DIVX on the Desktop doesn´t sound fun.

(These are for Pagekeeper, since I finally broke down and installed it. What a pity–wonderful programs crippled by idiots in Marketing slavering over demographics.)

Executive Summary

A common aspect of most client-server network designs is the login script. A set of commands executed upon provision of correct username and password, the login script provides the means for corporate system administrators to centrally manage their flock of clients. Unfortunately what´s seemingly good for the business turns out to be a disastrous security hole in the University environment, where students logging into the network from their dorm rooms now find the network logging into them. This hole provides a single, uniform point of access to any number of previously uncompromised clients, and is a severe liability that must be dealt with with the highest urgency. Even those in the corporate environment should take note of their uncomfortable exposure and demand a number of security procedures described herein to protect their networks. One possible solution for some may be the DoxPrint system designed by this author; it allows users to print to Novell Print Queues over the Network Neighborhood without requiring any Novell code on the client. Affected universities should consider switching to systems that do not require full logins, until more stable and secure systems are available.

---

Access Rights

What if I told you that every time you turned on your computer, the government could control exactly what would load? What if, every time you entered your username and password, your ISP gained the ability to specify exactly what software should load, what files to send, maybe even what data to erase? What if, merely by accessing a web page, your system came under the full control of the page’s author, or more accurately any possible author of that page, authorized or not?

In each case, the security violation is quite obvious. Merely drawing electricity, connecting to the Internet, or accessing a web page does not constitute an open license to fully control a computer. In legal terms, each action by the user is an ongoing communication under contractural obligations–for example, the user agrees to pay a fee and provide authentication material in the form of username and password, and in return the ISP agrees to provide Internet access. Never does the user agree to a “remote root access contract”! Whether this access is, in fact, used or abused is irrelevant. None of the user’s actions constitutes acceptance of “handing over the keys of the computer” to an external agent.

---

Power To Control

Abstract

Microsoft Windows 95 and 98 clients have the capability to bind multiple TCP/IP stacks to the same MAC address, simply by having the protocol added more than once in the Network control panel. This is actually quite useful, except for the fact that these stacks can run concurrently on the same IP, even if they receive their IP through BOOTP/DHCP. The effect of the bug is to cause the number of ACKnowledgement packets sent to be equal to that of the number of loaded and bound TCP/IP stacks, creating excessive and significant network noise and collisions. At least one Samba 2.0.0beta1 server on an affected subnet can become completely inaccessible when one of these machines start misbehaving.

Redundant ACKing can be referred to as TCP Chorusing, due to the minor time delays introduced between multiple copies of identical data. The problem is undetectable using the Ping command built into Windows 95 or 98–this is a significant bug in and of itself. Linux´s ping is not similarly crippled. NT does not detect TCP Chorusing with its Ping command.

---

Introduction: Discovery

A word of warning: This is the first edition of this document and there are bound to be errors. My ego isn’t so fragile as to be bothered if I made a misstatement of fact when writing this. Just tell me.

My university possesses a generally excellent network, but on occasion certain dorms would grind to a halt for no apparent reason. Seeking answers, I used a windows platform pinger to see if there were correlations between network downtimes and the presence of specific IP´s on a specific subnet. We use essentially static IP´s distributed from a DHCP server–a cookie seems to be assigned to a given MAC address on first request for an IP, and all future IP´s are given on that IP. Nothing out of the ordinary was found using the Windows pingers, so I decided I´d automate the testing process over time using an excellent Linux tool entitled fping. (In another environment, I might have merely shoved up a sniffer, but the secure hubs and my lack of permission to modify them in any way prevented that possibility.

Preface

This was my Systems Programming Term Project a while back. I wrote all the conceptual stuff, and came up with the original idea, while my group and I collaborated on the sample implementation. The implementation is…who knows where. This is what I have. Comments to me at [4]effugas@best.com. Oh, most of this text was written in December of 1997, but some minor updates have been done as of January 1998.

---

Introduction: Why Do We Network?

Think of a network, any network, in fact, every network. Ignore the how, the when, the who, the whatever, just think of the why. Why do we seek to connect one computer to another? It is not an inexpensive investment for any institution, so how do we justify the substantial TCO (Total Cost of Ownership) that computer networking incurs? We can derive our justification by examining computers as an extension of those who utilize them: The purpose of civilization is to uncouple existence with the means necessary to sustain existence, i.e. I do not need to know how to farm in order to eat, nor do I need to know how to fabricate a CPU in order to take advantage of one. Computers are networked according to the same logic: Since computers cannot self-fabricate all that is necessary for them to be most effective, they must be able to go elsewhere for what they require. Thus, computer networks are quite simply about getting information from point A to point B.

---

Unicast: The Old Standby

The major model utilized for file distribution on the global Internet, as well as on most smaller networks, is the unicast model: One server sends a requested file to the client who requested it. This process is repeated for each additional client that requests the file. The more client requests, the more files the original server must send. If an error is detected in the transmission, such as packets received out of order, the server automatically retransmits packets to compensate for that computer’s error. To use a more human analogy, it’s like being a teacher who uses one on one sessions to educate students. This has the advantage in that the teacher can quickly deduce what concepts the student failed to grasp and teach accordingly. While this model works for teaching a few students, it fails if the teacher becomes responsible for hundreds of students simultaneously.