Defects; UCC Article 2B; Uniform Commercial Code; Software Quality; Software Law; Computer Law; Bug;

Bad Software: What To Do When Software Fails.

What is a Serious Bug? Defining a "Material Breach" of a Software License Agreement

Copyright © Cem Kaner, 1996. All rights reserved.

(This memo was written for the January 10-12, 1997 meeting of the Article 2B Drafting Committee.)

To: Article 2B Distribution
From: Cem Kaner
Re: Definition of a Material Breach of a Software License Agreement
Date: December 10, 1996

A software defect is a "material breach" of the contract for sale or license of the software if it is so serious that the customer can justifiably demand a fix or can cancel the contract, return the software, and demand a refund. The ABA's, Section on Patent, Trademark, & Copyright Law, Committee on Computer Programs (Model Software Licensing Agreement, 1992) calls these Material Bugs. I'll use the same phrase.

If a defect is not "material" then the customer is probably stuck with the program and entitled to, at most, a partial refund.

How should we decide whether a bug is serious enough to be called "material"?

The July (and previous drafts) of Article 2B

Here's the proposed definition of a material breach of the software contract in July, 1996 (and in several previous drafts).

2B-109 (July) (b) A breach is material if the contract so provides or, in the absence of express contractual terms, if the circumstances, intent of the parties, language of the contract, and character of the breach indicate that the breach caused or may cause substantial harm to the interests of the aggrieved party, or if it meets the conditions of subsection (c) or (d).

(c)A breach is material if it involves:

(1)knowing or grossly negligent disclosure or use of confidential information of the aggrieved party not justified by the license;

(2)knowing infringement of the aggrieved party's intellectual property rights not authorized by the terms of the license and occurring over more than a brief period; or

(3)an uncured, substantial failure to pay a license fee when due which is not justified by an existing, colorable dispute about whether payment is due.

(d)A material breach occurs if the aggregate effect of the nonmaterial breaches by the same party satisfy the standards for materiality.

Look closely at (c), which defines a material breach by listing examples. All of these protect the publisher from breaches of the contract by the customer. For example, the customer is in trouble if s/he (1) discloses the publisher's confidential information, (2) pirates the software, or (3) doesn't pay for the program. None of these helps the customer argue that a given bug is material.

The September and November Drafts

The September, 1996 draft added a further clause to the list. A breach is also material if it involves:

a failure to perform in a manner consistent with express performance standards;

In September, I asked Ray Nimmer what this means. What is an "express performance standard?" Is it any precise, verifiable statement about the program that the seller makes to the customer? For example, can the customer get a refund if the program fails to match the written documentation?

He said, no, that's not what he meant. He was referring to the specific promises that were contained in a negotiated contract, that said that the program must do specified things or have specified characteristics.

So I said that this is unfair. What about customers who don't have negotiated contracts that specify all the right things? What about mass-market customers? If the program doesn't work, don't these customers have any recourse?

Ray said that this is a genuine problem. But he doesn't know how to solve it. How should we define material defect when the bug is not a direct nonconformity with an important part of the specification?

Then he said to me that I was the person who kept raising these issues, so maybe I could take a crack at drafting the definition.

And I promised that I would. Not that I claim any particular experience or skill in statutory drafting. I hope and trust that you will read this proposal for the idea it contains and recognize that its awkwardnesses will be cleaned up during the Article 2B review process.

How Should We Define a Serious Defect?

Failure to conform to specifications is a common theme in legal books, but many of the software development contracts provide vague, incomplete specifications that will change over time without being updated in the contract itself. Discussions within the software development community consistently recognize that most failures in commercial software products are due to errors in the specifications or requirements. A widely used number is that 80% of the money spent fixing or dealing with software problems can be traced back to requirements errors.

As a result, texts that focus on software errors don't limit themselves to failure to meet a specification (this type of failure is called nonconformance). Here are some examples from well respected texts in the field:

IEEE ( 1989), IEEE Standard Dictionary of Measures to Produce Reliable Software, ANSI/IEEE Standard 982.1-1988, p. 13:

Defect: A product anomaly. Examples include such things as (1) omissions and imperfections found during early life cycle phases and (2) faults contained in software sufficiently mature for test or operation. See also fault.

IEEE (1994), IEEE Standard Classification for Software Anomalies, IEEE Standard 1044-1993, p. 3.

Anomaly: Any condition that deviates from expectations based on requirements specifications, design documents, user documents, standards, etc., or from someone's perceptions or experiences. Anomalies may be found during, but not limited to, the review, test, analysis, compilation, or use of software products or applicable documentation.

Grady, Robert B. & Caswell, Deborah, L. (1987) Software Metrics: Establishing a Company-Wide Program. PTR Prentice-Hall, p. 78

A defect is any flaw in the specification, design, or implementation of a product. . . . If a flaw could not possibly have been detected, or if it could have been detected and would not have been corrected then it is an enhancement. Defects do not include typographical or grammatical errors in engineering documentation.

Ishikawa, Kaoru (translated by David J. Lu) (1985) What is Total Quality Control? The Japanese Way, Prentice-Hall. (Ishikawa is the leading Japanese quality control theoriest):

On page 46 ff. he explains why he doesn't trust quality as measured in terms of compliance with standards and specifications. The problem is that these are not true measures of the quality of the product. He works through an excellent example dealing with a role of newsprint. The true measure of quality is whether the paper rips while on the rotary press. Published standards, in terms of such things as tensile strength, provide only secondary measures of how the product will perform in the field. Continuing, on page 56, "There are no standards-whether they be national, international, or company-wide-that are perfect. Usually standards contain some inherent defects. Consumer requirements also change continuously, demanding higher quality year after year. Standards that were adequate when they were first established, quickly become obsolete. [¶]We engage in QC to satisfy customer requirements"

Jones, Capers (1991), Applied Software Measurement, McGraw-Hill, page 273.

A software defect is simply a bug which if not removed would cause a program or system to fail or to produce incorrect results. Note: the very common idea that a defect is a failure to adhere to some user requirements is unsatisfactory because it offers no way to measure requirements defects themselves, which constitute one of the larger categories of software error.

Mundel, August, B. (1991) Ethics in Quality, ASQC Quality Press, p. 164.

Any variation from the specifications is a nonconformity . . . . There is a group of nonconformities which represent serious threats to the welfare of users and bystanders. These nonconformities are called defects, and they not only can cause injury but may also result in the manufacturers, designers, or sellers being sued under the product liability laws. There are also a class of defects called design defects which can be responsible for customer dissatisfaction, loss, injury or death. Despite the fact that all of the product conforms to the design, the product is faulty and is not properly designed.

Myers, Glenford J. (1976), Software Reliability: Principles & Practices, John Wiley & Sons, pp. 4-6. This is one of the seminal books in the software testing / quality control literature.

One common definition is that a software occurs when the software does not perform according to its specifications. This definition has one fundamental flaw: it tacitly assumes that the specifications are correct. This is rarely, if ever, a valid assumption: one of the major sources of errors is the writing of specifications. . . . [¶] A second common definition is that an error occurs when the software does not perform according to its specifications providing that it is used within its design limits. This definition is actually poorer than the first one. . . . . [¶] A third possible definition is that an error occurs when the software does not behave according to the official documentation or publications supplied to the user. Unfortunately, this definition also has several flaws. There exists the possibility that the software does behave according to the official publications but errors are present because both the software and the publications are in error. A second problem occurs because of the tendency of user publications to describe only the manual for a time-sharing system that states, "To enter a new command press the attention key once and type the command." Suppose that a user presses the attention key twice by accident and the software system fails because its designers did not plan for this condition. The system obviously contains an error, but we cannot really state that the system is not behaving according to its publications. [¶] The last definition that is sometimes used defines an error as a failure of the software to perform according to the original contract or documentation. Although this definition is an improvement over the previous three, it still has several flaws . . . written user requirements are rarely detailed enough to describe the desired behavior of the software under all possible circumstances. [¶] There is, however, a reasonable definition of a software error that solves the aforementioned problems: A software error is present when the software does not do what the user reasonably expects it to do."

Roetzheim, William H. (1991) Developing Software to Government Standards, Prentice-Hall, p. 146

"Software defects can be divided into four broad categories: (1) requirements defects, (2) design defects, (3) code defects, and (4) documentation defects." See also Dunn, Robert (1984) Software Defect Removal, McGraw-Hill, p. 6-7 for the same distinctions.

Whether or not the specification or user documentation addresses the following issues, all of them should be considered material bugs, shouldn't they?

the product doesn't work at all
the disks are blank
the product caused substantial harm to the property or the business of the licensee
the licensor supplied a product that lack promised or advertised features or capabilities
the software deprives the customer of a key benefit that she reasonably expected (for example, anyone would expect a word processor to be able to print and it's unreasonable if a particular one can't provide that benefit)
the customer spent so much time, effort, and money dealing with a bug that the customer's costs exceeded the cost of the product. (If you think that this isn't a serious enough loss to the customer, what if the customer's losses run at ten times the cost of the software, or one hundred times the cost? At some point, the amount of the losses caused by the software becomes excessive, doesn't it?)

Reflecting the Relationships Between Licensor and Licensee

One of the crazy-making issues in product development is the question of who should bear the risk for an error in the understanding of the customer's requirements.I don't think that there is a simple answer to this question that is fair and reasonable. Instead, I propose that we think about four common classes of software transaction. I propose specific rules for each case. Perhaps I am proposing the wrong rules, but my fundamental point here is the contention that these four classes deserve their own rules.

1. The customer writes the specifications and requirements and asks the developer to write a program as specified.
In my view, the software developer meets its obligations if it writes a program that meets the specifications. If the specs say "2+2=5" then the program does not breach the software contract if it generates the wrong answer (5) whenever it adds 2+2. Let the specifier beware.
2. The developer writes the specifications and requirements, in preparation for custom development, but the customer is sophisticated.
If the customer is a computer expert, then it is able to review the requirements and specifications just as well as the staff of the developer. The customer is also probably in a better position to understand its own requirements than the developer. Therefore it is reasonable to hold this customer accountable for reviewing the specifications.
Article 2 of the current Uniform Commercial Code defines a "merchant" as follows:
2-104 (1) "Merchant" means a person who deals in goods of the kind or otherwise by his occupation holds himself out as having knowledge or skill peculiar to the practices or goods involved in the transaction or to whom such knowledge or skill may be attributed by his employment of an agent or broker or other intermediary who by his occupation holds himself out as having such knowledge or skill.
2-104(3) "Between merchants" means in any transaction with respect to which both parties are chargeable with the knowledge or skill of merchants.
Article 2B uses essentially the same definition:
2B-102 (26) "Merchant" means a person that deals in information of the kind, a person that by occupation purports to have knowledge or skill peculiar to the practices or information involved in the transaction, or a person to which knowledge or skill may be attributed by the person's employment of an agent or broker or other intermediary that purports to have the knowledge or skill.
When Microsoft buys Apple computers, it is a merchant. When a large, local hospital buys a bunch of Apples, it might be a big business, but it is not a merchant. In these situations, I would treat a hospital as a merchant if it retained an independent consultant to review the specs, etc.
3. The developer writes the specifications and requirements, in preparation for custom development, but the customer is not sophisticated.
When doctors, dentists, insurance brokers, small grocery store owners, and other small business people buy software, they have no clue how to specify the software, no clue how to evaluate a requirements document, no clue how to test the software, and no clue how to cost-effectively find a consultant who has these skills. In common computer parlance, these customers are called Clueless.
In UCC parlance, these customers are non-merchants.
These customers rely on the knowledge and experience of the developer. If the developer makes errors in defining the requirements or the specifications, which result in serious errors in the operation of the program, this is the developer's bug, not the customer's.
4. The developer writes a mass-market product. The customer has no input into the design or development of the product.
In the mass-market case, design errors belong to the developer, and never to the customer. Internal specifications that were used during development are largely irrelevant to the customer. The end product works in a reasonable way, and as advertised and as documented, or it does not.

The Proposed Statutory Language

This language expresses my sense of the fundamental differences between these transactions.

SECTION 2B-108. BREACH OF CONTRACT.

(a) Whether a party is in breach of contract is determined by the terms of the agreement and by this article. Breach occurs if a party fails to perform an obligation timely or exceeds a contractual limitation.

(b) A breach of contract is material if the contract so provides. In the absence of express contractual terms, a breach is material if the circumstances, including the language of the agreement, expectations of the parties, and character of the breach, indicate that the breach caused or may cause substantial harm to the interests of the aggrieved party, that the injured party will be substantially deprived of the benefit it reasonably expected under the contract, or that the breach meets the conditions of subsection (c), (d), (e), (f) or (g).

(c) If the licensee provides the specification documents that are incorporated in the contract, then a breach is material if:

(i) the software fails to perform in conformance with and in the time required by express performance standards or specifications;
(ii) the software fails to perform in conformance with the specifications and this failure either deprives the licensee of a significant benefit of the product or results in costs to the licensee that exceed the price paid for the software;
(iii) where the specifications are silent, the software's performance is unreasonable and it results in costs to the licensee that exceed the price paid for the software. The licensee has the burden of demonstrating that a reasonable licensor would consider the software's performance to be unreasonable.

(d) If the contract is between merchants, and it contains specification documents, then a breach is material if:

(i) the software fails to perform in conformance with and in the time required by express performance standards or specifications;
(ii) the software fails to perform in conformance with the specifications and this failure either deprives the licensee of a significant benefit of the product or results in costs to the licensee that exceed the price paid for the software;
(iii) where the specifications are silent, the software's performance is unreasonable and it results in costs to the licensee that exceed the price paid forthe software. The licensee has the burden of demonstrating that a reasonable licensor would consider the software's performance to be unreasonable.

(e) If the contract is not between merchants, and the licensor provides the specification documents that are incorporated in the contract, then a breach is material if:

(i) the software fails to perform in conformance with and in the time required by express performance standards or specifications;
(ii) the software fails to perform in conformance with the specifications and this failure either deprives the licensee of a significant benefit of the product or results in costs to the licensee that exceed the price paid for the software;
(iii) the software fails to perform in conformance with the end user documentation or other documentation delivered to the licensee and this failure either deprives the licensee of a significant benefit of the product or results in costs to the licensee that exceed the price paid for the software;
(iv) where the specifications and other documentation are silent, the software's performance is unreasonable and as a result, it either deprives the licensee of a significant benefit of the product or it results in costs to the customer that exceed the price paid for the software. The licensee has the burden of demonstrating that a reasonable person would consider the software's performance to be unreasonable.

(f) If the contract is for a mass-market license, then a breach is material if:

(i) the software fails to perform in conformance with the end user documentation or other documentation delivered to the licensee and this failure either deprives the licensee of a significant benefit of the product or results in costs to the customer that exceed the price paid for the software;
(ii) where the documentation is silent, the software's performance is unreasonable and as a result, it either deprives the licensee of a significant benefit of the product or it results in costs to the licensee that exceed the price paid for the software. The licensee has the burden of demonstrating that a reasonable person would consider the software's performance to be unreasonable.

(g) A material breach of contract occurs if the cumulative effect of nonmaterial breaches by the same party satisfies the standards for materiality.

(h) If there is a breach of contract, whether or not material, the aggrieved party is entitled to the remedies provided for in this article and the agreement.

Cem Kaner attends Article 2B meetings as an observer. He consults on technical and management issues, practices law, and teaches within the software development community. His book, Testing Computer Software, received the Award of Excellence in the Society for Technical Communication's 1993 Northern California Technical Publications Competition. He has managed every aspect of software development, including software development projects, software testing groups and user documentation groups. He has also worked as a programmer, a human factors analyst / UI designer, a salesperson, a technical writer, an associate in an organization development consulting firm, and as an attorney (typically representing customers and software development service providers). He has also served pro bono as a Deputy District Attorney, as an investigator/mediator for Santa Clara County's Consumer Affairs Department, as an Examiner for the California Quality Awards. He holds a B.A. (Math, Philosophy, 1974), a J.D. (1993), and a Ph.D. (Experimental Psychology, 1984) and is Certified in Quality Engineering by the American Society for Quality Control. He teaches at UC Berkeley Extension, and by private arrangement, on software testing and on the law of software quality.

[ Main | List of Articles | Bad Software | UCC 2B | Law of Software Quality | Digital Signatures | Bookstore | Court Cases | Links | Press Releases | About Us | What's New ]

The articles at this web site are not legal advice. They do not establish a lawyer/client relationship between me and you. I took care to ensure that they were well researched at the time that I wrote them, but the law changes quickly. By the time you read this material, it may be out of date. Also, the laws of the different States are not the same. These discussions might not apply to your circumstances. Please do not take legal action on the basis of what you read here, without consulting your own attorney.