Return to Bad Software: What To Do When Software Fails.


 

Article 2B and Reverse Engineering

Published in Uniform Commerical Code Bulletin, November, 1998

Copyright (c) 1998, Cem Kaner. All rights reserved.

 


 

At its annual meeting, NCCUSL passed the Perlman amendment to Article 2B, which included the following language:

NCCUSL (Perlman) 2B-110 (a) If a court as a matter of law finds the contract or any term of the contract to have been unconscionable or contrary to public policies relating to innovation, competition, and free expression at the time it was made, the court may refuse to enforce the contract, or it may enforce the contract without the impermissible term, or it may so limit the application of any impermissible term as to avoid any unconscionable or otherwise impermissible result.

Commissioner Perlman's motion reflected several different concerns. One of them is a belief, held by many intellectual property specialists, that Article 2B will be used as an end run around the Copyright Act's balancing of the sometimes conflicting rights and interests of the public and of creators. Within this is a particular issue that has caused grave concern within the engineering community--the possibility that Article 2B will be used to eliminate the public's right to reverse engineer computer software.

It is common to see clauses in software licenses that bar the customer from reverse engineering the software. To the best of my knowledge, no such clause in a software product that was sold (or "licensed") in the mass-market has been enforced (s!ee Sega Enterprises Ltd. v. Accolade, Inc., 977 F.2d 1510, 9th Cir. 1992). Such clauses have been enforced in negotiated, non-mass-market licenses. Under the Copyright Act, such clauses would not be enforceable in a contract for the first sale of a software product, but Article 2B characterizes what appear to be sales in the mass market as licensing transactions, and the protections of the Copyright Act might not extend to licenses. Clauses that bar reverse engineering would become presumptively enforceable in an Article 2B mass-market software license, because they are contractual use restrictions (see 2B-102(b)(13)) and limitations on the scope of the license (see 2B-102(b)(42)).

On the floor at the NCCUSL meeting, Commissioner Perlman stated that he was asking for a sense of the house motion and that the precise wording was negotiable. The corresponding wording that appeared in the August 1 draft of Article 2B is this:

August 1 Draft 2B-105 (b) A contract term that violates a fundamental public policy is unenforceable to the extent that the term is invalid under that policy.

The modification differs from the original in two key ways:

  1. It no longer refers to innovation, competition, or free expression but instead refers to "fundamental public policy." I think that this provides courts with less guidance as to the types of terms that are impermissible.
  2. It requires that the contract term be "invalid under that policy." This additional requirement might be completely innocuous or it might not. I am mindful of Reporter Nimmer's recent statement that:

There have been no cases in which [Copyright Act] Section 301 preemption was used successfully to challenge and invalidate a term of a contract that was enforceable as a matter of general state contract law.

(Ray Nimmer, "Breaking Barriers: The Relation Between Contract And Intellectual Property Law" Conference on the Impact of Article 2B, Berkeley, April 23-25, 1998. www.SoftwareIndustry.org/issues/guide/docs/rncontract-new.html.)

If Professor Nimmer's statement is a fair and accurate summary of Copyright Act preemption case law, then the proposed modification to Section 2B-105 will provide no safeguard against mass-market license terms that unfairly limit fair use of copyrighted materials.

As a particular concern, if Professor Nimmer's statement is a fair and accurate summary of Copyright case law, then restrictions on reverse engineering, in mass-market software licenses, will become enforceable as a matter of general state contract law (Article 2B) and will therefore almost certainly not be invalidated by the Copyright Act.

If Professor Nimmer's statement is a fair and accurate summary of Copyright case law, then the battle to preserve a right to reverse engineer software will have to be fought within Article 2B, because Article 2B will settle the matter by making enforceable a ban that many of us think is not enforceable under the law today and should not be enforceable under the law tomorrow.

I am not writing this paper to debate or clarify the legal issues surrounding reverse engineering. Rather, I am writing to clear up a misunderstanding about the reasons that people use reverse engineering and the types of consequences that are likely in the face of a ban on the practice. If we are to decide the fate of this practice as part of the 2B process, we should understand what we are deciding.

What Is Reverse Engineering

As I understand the term, "reverse engineering" encompasses any activity that is done to determine how a product works, to learn the ideas and technology that were used in developing that product.

Reverse engineering can be done at many levels.

PRINT 5

A translation of this simple command into Assembly Language might require several hundred (or more) lines of Assembler code.

When you disassemble a program, you get thousands of lines of code, in a language that was not used by the original programmer, that lacks the comments, variable names, formatting and other signals used by the programmer to explain the meaning of the program. To translate this back into a high level language is painstaking work, and very time consuming. Andrew Johnson-Laird described this process in detail in his excellent paper, "Software Reverse Engineering in the Real World" 19 U. Dayton L.R. 843, 1994.

You can learn a lot by reverse engineering. But in terms of competitive use, there are limits on what you can do with what you learn. You can't just copy your competitor's code into your product-the copyright laws bar that. And many of the great ideas that you can find in code have been patented. You can't use those either. As Johnson-Laird pointed out, reverse engineering is rarely a cost effective way of developing a competitive product.

Why Do People Reverse Engineer?

I heard remarks on the floor at the NCCUSL annual meeting and in private discussions that programmers reverse engineer programs in order to discover trade secrets. That the programmer who learns these secrets will then be able to use the discovered technology to develop products that compete with the original program. The idea is that reverse engineering is a form of unfair competition, and therefore software developers should be allowed to protect their technology by banning the practice.

Andrew Johnson-Laird dealt with this misperception in his paper (19 U. Dayton L.R. 843). He made the point that every programmer uses reverse engineering techniques for a variety of reasons. He provided hypothetical examples to illustrate his points.

My comments are intended to draw attention to, reinforce and extend Johnson-Laird's analysis. Unlike Johnson-Laird, I am not a specialist in reverse engineering. I don't offer reverse engineering services to clients. My examples are based on my personal experience as a software developer, software development manager, and software quality consultant. I chose examples out of my personal experience with the hope that by adding personal details, I could make them feel a bit more real to a reader who has probably never done this type of work. As you'll see, these reflect the normal course of normal practice, rather than the work of a reverse engineering hotshot (which I am not).

So why have my colleagues and I reverse engineered?

Here are the examples:

Personal Education

Professionals learn much of their craft by studying the work of other professionals. Lawyers read other lawyers' briefs and depositions. Programmers read other programmers' code.

I first studied how to write operating systems and interpreted programming languages by disassembling Applesoft, which was both a variant of the BASIC programming language and a disk operating system for the Apple II computer. As was to be expected from deep study of someone else's code, I also recognized (and took advantage of) opportunities to improve the product.

John Vokey (now the Chairman of the Psychology Department at the University of Lethbridge) and I used Apples to control experiments in our laboratories. We added various devices (clocks, analog-to-digital converters, tone generators, etc.) to these computers. To control them, we extended Apple's operating system. We also extended Apple's BASIC in order to analyze our data at a high level of precision. We published several papers detailing our analyses and extensions of Applesoft (For example, Cem Kaner & John Vokey, "Disassembling Machine Language Programs Without Leaving BASIC", 4 Compute! Issue #5 146, 1982; "Modifying Apple's Floating Point BASIC", 4 Compute! 68, February, 1982; "Subroutine Master", 6 Nibble 46, November, 1985).

We didn't do this work in order to compete with Apple. We extended the capabilities of this machine's software, making it much more useful to a research community.

The typical no-reverse-engineering clause doesn't contain exceptions for personal education or for development of extensions that will make the original product more salable. It just bans reverse engineering. Banning reverse engineering (via license restrictions approved by Article 2B) will mean a ban of these activities. Such a ban would have stunted my professional development and the development of many other of the best programmers that I know.

Limitations and Defects

One of the oddities of the Apple operating system was that, at unpredictable times, it would insert noticeable delays between events. Our experiments presented stimuli for specific times and measured the times it took for people to respond. The Apple delays didn't matter to most users, but they created timing errors in our work. On reading the code, I discovered that these delays were caused by a clean-up operation--as needed, the operating system would reorganize its use of memory in order to make more memory available on demand for programs that needed it. The need to reorganize memory was an inherent limitation of the operating system. We were stuck with it. But we weren't stuck with the unpredictable timing. Once I knew the cause of the delays, it was easy to add a function to BASIC that let the programmer force a memory cleanup at a convenient time. Delays could now occur between trials (individual test cases within an experiment that might run a session of 100 trials) rather than, randomly, within trials.

One of the defects of this dialect of the BASIC language involved its generation of random numbers. A high quality random number generator (RNG) is essential for certain types of mathematical research. For example, simulations are sometimes the only way to compare the behavior (and thus the power) of different statistical functions under a wide range of alternative situations. (Cem Kaner & John Lyons, "Tables and Power Comparisons for Different Versions of the Kolmogorov-Smirnov and Schuster Statistics," Technical Report No. 67, Dept. of Psychology, McMaster University, 1979; Cem Kaner, Serge Mohanty & John Lyons, "Critical Valus of the Kolmogorov-Smirnov One-Sample Tests" 88 Psychological Bulletin 498, 1980). Errors in simulation could lead to misunderstanding of the characteristics of a function resulting in erroneous interpretation of experimental data. (For an example of this type of discussion, see Cem Kaner, "Results Concerning Gatlin's Tests for Bias in Finite Sequences" 77 J. Amer. Soc. For Psychical Research 31, 39-40, 1983.)

I studied (sometimes by disassembly) the random number generating capabilities of several programming languages, including Applesoft BASIC. One common type of random number generator (the linear congruential generator) does a series of multiplications, additions and divisions to create each random number. The algorithm uses Integer arithmetic--fractions are discarded. Applesoft BASIC used what looked like the linear congruential algorithm but with floating point arithmetic (which kept the fractions) instead. The result was a generator that was less effective at producing a statistically random sequence of numbers. This is a subtle mistake. If you don't understand RNG's, your normal instinct as a programmer would be to use floating point arithmetic for these calculations. This error wasn't unique to Apple: Lyons, Vokey and I discovered it in two other programming languages as well. We developed a simulation-quality replacement generator for the Apple and eventually published it (Cem Kaner & John Vokey, "A Better Random Number Generator" 72 Micro 26, 1984).

There's some sublety in determining that a random number generator (RNG) isn't working correctly, but simulations based on a defective RNG are usually invalid and usually have to be redone, often at substantial expense. Colleagues of ours, including some mathematicians, used Applesoft's RNG without realizing its defects. We initially recognized a problem in the Applesoft RNG by running some simple tests of it, but we didn't understand the extent of the problem until we read the (reverse engineered) code. A more subtle and more serious example of an RNG problem arose in early simulation work on 32-bit mainframes (machines like the IBM 360). Many mathematicians did serious work using a popular RNG called RANDU (Donald Knuth, 2 The Art of Computer Programming 2d, 1981). Simple tests did not and could not reveal RANDU's problems. The critical test that revealed problems in RANDU (the Spectral Test described by Knuth) depended on knowledge of RANDU's parameters--specific numbers used in calculations. RANDU's parameters were published, but to find the values of these parameters for other generators, we had to reverse engineer code.

So here we have a relatively common class of problems that aren't easily discovered except by reverse engineering. The consequences of incorrectly relying on the integrity of this function can be enormous--people model the behavior of many types of complex systems using random-number driven simulators. An Article 2B-enforceable ban on reverse engineering would prevent you (or your staff) from effectively checking the integrity of an RNG in a programming language you were using, killing your opportunity to prevent losses caused by the use of a defective RNG. Additionally, you might never discover that the RNG you are using is defective because an Article 2B-based ban on reverse engineering would prevent researchers who study such things from reverse engineering code, discovering a bad generator, and publishing a warning to the rest of the programming community.

Here's another example of a defect. Year 2000-related defects are on the mind of many attorneys today. Article 2B would render enforceable a ban on reverse engineering or modification (repair) of the code by the customer. This is, after all, just a use and scope restriction. Result: a licensor who creates a defective program (such as Y2K-bugged) can prevent your company from determining the extent of defectiveness of the code and from fixing it. Instead, you have to pay the licensor top dollar for the privilege of patiently waiting until the licensor gets around to fixing your system. If you can find the licensor. And if the licensor is still willing and able to work with you. Hopefully the fix will be done, debugged, and working before January, 2000. It's bad enough to have an industry that routinely disclaims all warranties for its products. But to set up a legal structure that lets vendors routinely profit (and enforces their right to profit) from defects designed into their products? This goes a bit far, don't you think?

Defects in Third Party Products

Over the years, I managed the development of several mass-market programs for contact management, desktop publishing, forms design, and project scheduling. To make these products profitable, I had to find ways to minimize the number of complaints we received from customers. At an industry average cost to the software publisher of $23 per call, complaint calls are expensive (for supporting data, see Cem Kaner & David Pels, Bad Software: What To Do When Software Fails, 1998, Chapter 2).

Both of these are examples of reverse engineering done in the service of interoperability--making a product able to work with some other product, in this case by working around defects in that other product's software. Sometimes, when we recognized a defect in another company's product, we decided not to modify our code to be compatible with the other product. If the product operated strangely enough, if its market share was small enough, or if its technical support organization was arrogant and unhelpful enough, we would consider a decision to refuse to support the product. If we knew exactly how the product was malfunctioning, or how its performance deviated from a recognized standard, we could make an informed decision. After making that informed decision, we could explain the problem (the underlying defect) to technically sophisticated complaining customers. And we could privately explain it to magazine reviewers, who wouldn't then take us to task for choosing not to attempt to be compatible with the offending product.

Compatibility With (Ability To Work With) Another Product

The previous examples involved reverse engineering to determine the cause and nature of a defect in a product. But sometimes we did reverse engineering simply to figure out how to work best with a product that worked properly. For example, when a printer or modem or other device would come out, some colleagues of mine would study the machine and its driver (a program that helps other programs control the device) to determine whether they should upgrade their programs to take advantage of new capabilities.

The documentation that comes with these devices is often quite modest. Many products come with documentation that was imperfectly translated into English from some other language. And the drivers often come with bans on reverse engineering. For example, I have a "Microsoft Software License" for the "Microsoft Windows 95 Driver Library." It says that "You may not reverse engineer, decompile or disassemble the Software." (According to the license, the Software is the Windows 95 Driver Library.)

So if the documentation is incomplete, inaccurate, and/or incomprehensible, how do you learn how to write code for a given device? The ban on reverse engineering might be intended to protect Microsoft (or the device makers who licensed Microsoft to publish their drivers in Win 95) from competition in the driver market or in the device market, but this ban would also block companies who merely want to make their product work with a device, companies that have absolutely no wish to compete in device/driver markets.

You can buy books on how to write device drivers. How do you think the authors of those books learned what to write?

Compatibility With (Ability To Share Data With) Another Product

I assisted in the development of a desktop publishing (DTP) product and eventually became the project manager for several releases of it.

A DTP program takes text (from editors or word processing programs) as input and makes it easy for the user to design a page that will present that text effectively. The DTP program makes it easy to lay text down in columns, to wrap it around pictures, to apply various special effects to the text, and so on. Markets for desktop publishing and word processing programs have gradually converged. These days, I think the best word processors help the user with page layout about as well as a mediocre desktop publishing program. But back in 1990-1993, I would have defined the word "masochist" as someone who did fancy page layout with a word processor instead of a desktop publisher. Word processors and desktop publishers were complementary, not competitive, products.

Most word processors and editors save documents in proprietary file formats. To succeed in the DTP market, we had to figure out how to read, interpret, and preserve the formatting information in these files. For example, if you wrote a document with Word Perfect and applied boldface and italics to some text, you would expect that text to be italicized and boldfaced when you first viewed it with our desktop publisher.

It is possible to reverse engineer a file format. Imagine creating two copies of the same document, differing only in that some text is boldfaced in one file but not in the other. Look at the differences between the two files and you learn a lot about its coding of boldfacing. Repeat tests like this for every other imaginable treatment of text, for added pictures, etc., and eventually you can read the file. It is, of course, simpler and cheaper to just get a specification from the company that designed the file format.

I called many of these companies, asking them to send us their file format specs. Some agreed. Some didn't. For reasons that I'll never understand, one company not only refused; they said that I was trying to develop a competing product and then insisted to me that their license agreement barred us from reverse engineering anything of theirs, including their file format, and that we had better not try it.

I didn't ask any of these companies for permission to reverse engineer their file formats. After all, I was going to be analyzing files that contained my documents, that I had created, that I held copyright to, that I could distribute freely to as many people as I wanted (none of whom had to sign any license agreements). But under Article 2B, this publisher could create a suitable restriction in a mass market product. All the license would have to say is that the user cannot use the product in a way that is calculated to reveal the file format. (Maybe the language needs some polishing, but you get the point.) This is just another restriction on the use of the product, and we could also define the licensed scope of use in a way that excludes this type of reverse engineering. That's all that appears to be needed for Article 2B.

This might sound as though I'm stretching the point, but I'm not. I've met lawyers who think that such a restriction would be proper. One attorney who sometimes attends 2B meetings privately explained to me that this type of restriction should be proper and that companies like mine could license the file format information from his company. Of course, he admitted, this would leave him free to decide which companies to license this information to and which not. Every company would like to be able to pick its competitors. But that doesn't mean that the Uniform Commercial Code should bless the practice.

We came into that market with no DTP or word processing history, facing established competition. A company much larger than us had about a 75% market share in our intended niche. We worked like mad to build a better product and eventually we overtook our largest competitor. This is the kind of success that Silicon Valley is made of. But if we hadn't been able to read other companies' data files, we'd have gotten nowhere. No one would buy a DTP program that can't accurately import text.

Silicon Valley became remarkably successful as an innovation-driven community because we all (engineers, not lawyers) were able to stand on each others' shoulders. Changing those rules, as Article 2B will do, should be done with extreme caution.

Learn Principles That Guided a Competitor

I worked as a human factors analyst / programmer for a company that made PBX's (private telephone systems that you install in your offices). One of my tasks was to design an attendant's console. This is the oversized telephone with way too many buttons that you can find on the desk of the receptionist in a typical 100-200 person company.

As part of my research, I spent a day or two at each of several companies, watching their receptionists work with other companies' consoles (and sometimes playing with the consoles myself). My goals were to see the extent of variation of designs on the market, to figure out the principles that the other designers had considered important in designing their consoles, to see the ways that the consoles gave feedback (information) to the receptionists, and to see what made the receptionists angry or confused or impatient.

I learned far more from these observations that I could have ever learned from studying code.

Imagine yourself as corporate counsel for a PBX manufacturer who doesn't want the likes of me studying its designs. Could you write a use restriction into the license for the PBX (or the software that drives the PBX) that bars a customer company from cooperating with a non-employee who wants to study the behavior of the PBX or any of its components? Of course. Would this be enforceable under Article 2B?

There's a complexity in that a sale of a PBX could be interpreted as being primarily a sale of goods, a transaction under Article 2, rather than 2B. However, a PBX is essentially a computer that drives a lot of peripheral devices (mainly, telephones). Under Section 2B-104(3) of the August 1 draft of Article 2B, the software should probably remain within the scope of 2B.

Assuming that this is an Article 2B transaction, I think the use restriction would be enforceable. Should it be?

Next, consider doing competitive research on mass market software products. For example, imagine designing a desktop publishing program. Buy a dozen copies of each of your main competitors' products. Sketch out some document designs (pen and ink drawings) and create some word processor files that can be used to provide text for the documents that people will create with the competitors' products. Hire a few dozen temporary workers (specify a pattern of desired employment backgrounds to your local temporary labour agency). Assign a dozen people to each of your competitors' programs and watch how they use those programs to create the documents that you have sketched out. For each design, measure how long it takes people to create the same document with the different products. And write down the errors that people make when trying to create these documents (patterns will show up and they'll be different for the different products).

Should a publisher of a product in the mass market be able to ban this kind of research? Surely, it could do so under Article 2B with an appropriately written use restriction.

Detect Theft

At one company where I worked, a senior member of our staff joined a company that then published a product that competed with a key product of ours. Some people suspected that this new product had been built with some of our code, so our executives assigned some of our senior programmers to reverse engineering the competitor's product, looking for evidence that our most prized routines had been copied. We didn't find anything.

Other companies have found evidence of copying by doing reverse engineering. For example, in the case of Apple Computer, Inc. v. Franklin Computer Corp. (714 F.2d 1240, 3d Cir. 1983), but a key piece of evidence was that James Huston's (one of Apple's programmers) name appeared in part of Franklin's code as did the word "Applesoft." Presumably, Apple found this by reverse engineering Franklin's code.

I don't see an exception built into Article 2B that allows a company to reverse engineer a competitor's code for evidence of theft. And if there is none in Article 2B, then under what other law do we find a rule that in a licensing transaction, one company has the right to reverse engineer a competitor's code?

Demonstrate Falseness of Claims

Syncronys Softcorp published SoftRAM95. (I have no personal knowledge of the facts of this case.) SoftRAM95 was allegedly promoted as a software alternative to expanding your computer's memory. For example, according to the Federal Trade Commission's Complaint, Syncronys had claimed that its product would have the effect of doubling your computer's memory, from 4 MB to 8 MB. In the Matter of Syncronys Software. (1996) Docket C-3688. www.ftc.gov/os/9610/c3688cmp.htm.

According to the FTC, Syncronys sold approximately 600,000 copies of SoftRAM95. Andrew Schulman, a Senior Editor of O'Reilly & Associates, maintains a web page called SoftRAM95: "False and Misleading" at http://ftp.uni-mannheim.de/info/Oreilly/windows/win95.update/softram.html. This page is an archive of documents related to the SoftRAM95 case. One of the documents at that page is said to be a press release issued by Syncronys, that reported the results of a customer survey allegedly done by Dataquest, an internationally recognized technology market research and consulting firm. http://ftp.uni-mannheim.de/info/Oreilly/windows/win95.update/dataquest.txt. The results reported included these:

The only problem is that, according to the FTC's complaint, SoftRAM95 did little or nothing to increase the amount of available memory on a Windows 95-based computer. This case illustrates a big problem with high technology products: most people can't tell if they are being cheated.

Looking back to reverse engineering, one of the ways that the public learned that SoftRAM95 did (allegedly) almost nothing to increase available RAM was through reports of the results of reverse engineering. See, for example, Mark Russinovitch "Reverse-Engineered Disassembly of SoftRAM 95 - Windows 95 Version" http://ftp.uni-mannheim.de/info/Oreilly/windows/win95.update/softdiff.asm, and Andrew Schulman, "Analysis of SoftRAM 95 SOFTRAM1.386" http://ftp.uni-mannheim.de/info/Oreilly/windows/win95.update/softram1.txt. To the best of my knowledge, Russinovich and Schulman did this work as private individuals and not as part of any government investigation. Their work probably helped trigger a government investigation.

According to Charles Cooper ("Syncronys Warns Dr. Dobbs Over Upcoming Review", PC Week Online, June 21, 1996, www.zdnet.com/pcweek/news/0617/21esync.html), Dr. Dobbs Journal (which is a leading programmers' magazine) intended to publish a review of SoftRAM95 and was warned by Syncronys: "They [Syncronys] wanted to put us [Dr. Dobbs] on notice that they would protect their rights for defamatory or misleading statements as well as protect their copyrights and trade secrets." According to Cooper, Dr. Dobbs Journal chose to go ahead with the piece despite the warning.

How would this have played out if Article 2B was in force? Assume that the SoftRAM95 license would have contained a broad ban of all forms of reverse engineering. Could Russinovich and Schulman have done their reverse engineering research? Could a magazine like Dr. Dobbs Journal have published the results of reverse engineering studies without fear of liability? Would private citizens have been able to lawfully collect enough data on this product to interest the Federal Trade Commission in pursuing a case when most of the customers reported that they were satisfied with the product?

Now, The Obvious Generalization

None of the issues that I've raised in this paper are unique to software. Article 2 is being revised. Perhaps in Article 2, we should allow manufacturers to license hard goods in the mass market. Television sets, for example, differ tremendously in their capabilities and qualities, depending on the technology built into them. If a software manufacturer can ban reverse engineering, why shouldn't a television maker or a refrigerator maker also be able to ban it?

Reverse engineering is widely used in the design of new products. Anything that changes that will change the character of research and development (software or traditional goods) in the United States. We're pretty good at R&D in the United States. As lawyers, we should take care to avoid breaking something that doesn't need fixing.



Return to Bad Software: What To Do When Software Fails.

 
The articles at this web site are not legal advice. They do not establish a lawyer/client relationship between me and you. I took care to ensure that they were well researched at the time that I wrote them, but the law changes quickly. By the time you read this material, it may be out of date. Also, the laws of the different States are not the same. These discussions might not apply to your circumstances. Please do not take legal action on the basis of what you read here, without consulting your own attorney.
Questions or problems regarding this web site should be directed to Cem Kaner, kaner@kaner.com.
Last modified:January 29, 1999. Copyright © 1998, Cem Kaner. All rights reserved.