Thursday, October 27, 2005

Getting Buggy ... Some gyan about Bugs ...

This is post is in response to few typical interview questions that are posted in various user groups. I typically don’t jump in and answer to them - But in some case I just can not remain silent - some questions and answers make me to speak up. As if they beg for answers. Here is one such occasion and here is how I responded....

1. What is difference between Bug and Defect: There are many definitions that float around there are no simple and universally acceptable definitions for these things. When used in an informal environment, both defect and bug mean same thing. It is some unwanted unexpected behavior that bugs somebody who matters. This definition of bug does not change depending upon the phase of SDLC. A bug is a bug is a bug is a bug. Same holds good for defect. I quote Cem Kaner, James Bach and Michael Bolton in this connection. Believe me they say same thing - no one can dare to question them in the knowledge in testing field. As per Michael Bolton - “I say that you may define "defect" in any way that you like, as long as the person that you're speaking with or writing to understands your definition."

Michael Bolton in a Google group post --

A bug is something that threatens the value of the product, or, if you like, a bug is something that bugs someone who matters. Both of these definitions come from James Bach. Your definition may differ. "We" depends on the context of the project. On a typical project, someone (the project manager) has the authority to determine whether something (a bug, failure, fault, defect, and symptom) is serious enough to merit attention. In my context, an intermittent problem is a bug if the project manager says it's a bug. James also wrote an article on intermittence in his blog; try http://blackbox.cs.fit.edu/blog/james


Following is an excerpt from Cem Kaner’s blog – Note that according to him the use of word defect in more formal context means "Legal Implications". If there is a defect in software, an end user can sue the producer of the software. He recommends that word "Bug" is more informal.

Quote: Cem Kaner --

I have two objections to the use of the word defect.

(a) First, in use, the word "defect" is ambiguous. For example, as a matter of law, a product is dangerously defective if it behaves in a way that would be unexpected by a reasonable user and that behavior results in injury. This is a failure-level definition of "defect." Rather than trying to impose precision on a term that is going to remain ambiguous despite IEEE's best efforts, our technical language should allow for the ambiguity.

(b) Second, the use of the word "defect" has legal implications. While some people advocate that we should use the word "defect" to refer to "bugs", a bug-tracking database that contains frequent assertions of the form "X is a defect" may severely and unnecessarily damage the defendant software developer/publisher in court. In a suit based on an allegation that a product is defective (such as a breach of warranty suit, or a personal injury suit), the plaintiff must prove that the product is defective. If a problem with the program is labeled "defect" in the bug tracking system, that label is likely to convince the jury that the bug is a defect, even if a more thorough legal analysis would not result in classification of that particular problem as "defect" in the meaning of the legal system.

We should be cautious in the use of the word "defect", recognize that this word will be interpreted in multiple ways by technical and no technical people, and recognize that a company's use of the word in its engineering documents might unreasonably expose that company to legal liability.

Unquote Cem Kaner.


2. Bug severity v/s Priority - Who assigns them: When the developers have plenty of time, bug arrival rate is lagging behind the fixing rate - nobody really cares about "Priority" and to some extent "Severity". Both of these are filtering mechanisms to select few bugs from the whole lot so that only important ones are fixed first. Severity is one way of grading the bugs from "bug impact" point of view - so that tester can influence the fix - say "This is more serious needs to be fixed first". After all, as very few people know - real value of tester is in getting a bug fixed than simply logging it. Severity can be/is assigned by the tester; can be modified by test lead if there is real need. There after it is in developer’s court. Developers use rating called "Priority" to pick top bugs to fix. So priority is set by Dev lead in consultation Program/Project manager some times even client will get involved. This can not happen without buy-in from test lead. What I am describing is IDEAL situation. In a mature Test organization this happens. I have see this (been a party to it) happening in companies like Microsoft. In Microsoft (also in many other organizations) – they use a process (ceremony in Agile world) called “Bug Triage” where dev lead, test lead and PM sit across the table with the list of bugs and deliberate on severity and priority. More often than not, the discussion is more oriented towards “Priority” than “Severity”. Bug Triage meeting is a formal platform to change the “Severity” and “Priority” levels.

In most of the test groups I have seen, Severity rating unfortunately is a means of performance of developer or tester. Like “This tester has logged 10 Severity 1 bugs” OR “The module developed by you had maximum number of Severity 1 bugs”. But then, that is another big topic of debate.

Last but not the least - All those who wanted to know about bugs but did not know whom to ask - read "Bug advocacy” by Cem Kaner - the bible on bug management. You will never have any doubts about bugs in rest of your life.

www.kaner.com/pdfs/bugadvoc.pdf

Buggy - all the way is'n it? I love to talk about bugs and the way they are managed...

Ideas and views are welcome...

Shrini

2 comments:

Anonymous said...

Another good post. Given me lots to think about.

Cem Kaner said...

Shrini:

You say that if James Bach, Mike Bolton and I agree on something, no one can challenge us.

I just don't see the world that way. We're thoughtful people, but serious, thoughtful people disagree with us on many things. I've changed my mind many times on many issues over the years as I gained better perspective or mature in the field. Look at this, for example, http://www.satisfice.com/kaner/?p=11

===================

In terms of "defect" versus "bug", my point is not that people can sue you for "defects." Instead, it is that sometimes, in some circumstances, people can sue you for defects. And if they do sue you because your product is allegedly defective, then you will have a very, very hard time in court if your own records say that you have 100 unfixed "defects" in your software.

Let me give an analogy. Suppose that you really, really like to drink coffee. Coffee has caffeine. Caffeine is addictive. It is also a stimulant. In the United States, we often call stimulants "speed" -- though we usually mean amphetamines, which are much stronger than caffeine. Suppose you describe your affection for coffee by writing on your blog that you are addicted to speed. Do you think that this might be misunderstood, or make it harder for you to defend yourself if you are ever (erroneously) arrested for possession or sale of drugs?

Saying your product has a "defect" is like saying you are addicted to speed (when you mean coffee). Your friends and coworkers might know exactly what you mean, but other people might form the wrong conclusion.

-- Cem