Facts and Fallacies of Software Engineering

Text in black are quotes; text in green are my notes. I sometimes write in Spanish.

• Fact 1 The most important factor in software work is not the tools and techniques used by the programmers, but rather the quality of the programmers themselves. #

• Fact 2 The best programmers are up to 28 times better than the worst programmers, according to “individual differences” research. Given that their pay is never commensurate, they are the biggest bargains in the software field. #

• the correlation between test scores and on-the-job performance is nil. #

• computer science class grades and on-the-job performance correlated abysmally also [Sackman 1968].) #

• Fact 3 Adding people to a late project makes it later. #

• Fact 4 The working environment has a profound impact on productivity and product quality. #

• Fact 5 Hype is the plague on the house of software. Most software tool and technique improvements account for about a 5 to 35 percent increase in productivity and quality. But at one time or another, most of those same improvements have been claimed by someone to have “order of magnitude” benefits. #

• Fact 6 Learning a new tool or technique actually lowers programmer productivity and product quality initially. The eventual benefit is achieved only after this learning curve is overcome. Therefore, it is worth adopting new tools and techniques, but only (a) if their value is seen realistically and (b) if patience is used in measuring benefits. #

• how long it took them (they are quite likely to know the answer to that question) and how much benefit they achieved #

• Fact 7 Software developers talk a lot about tools. They evaluate quite a few, buy a fair number, and use practically none. #

• Once the thrill of trying out a new tool has worn off, the poor schedule-driven software developer must build real software to real schedules. And, all too often, the developer reverts to what he or she knows best, the same tools always used. #

• Fact 8 One of the two most common causes of runaway projects is poor estimation. (For the other, see Fact 23, page 67.) #

• The two causes of runaways that stand head and shoulders above all others are poor (usually optimistic) estimation and unstable requirements. #

• Fact 9 Most software estimates are performed at the beginning of the life cycle. This makes sense until we realize that estimates are obtained before the requirements are defined and thus before the problem is understood. Estimation, therefore, usually occurs at the wrong time. #

• Fact 10 Most software estimates are made either by upper management or by marketing, not by the people who will build the software or their managers. Estimation is, therefore, done by the wrong people. #

• Fact 11 Software estimates are rarely adjusted as the project proceeds. Thus those estimates done at the wrong time by the wrong people are usually not corrected. #

• Fact 12 Since estimates are so faulty, there is little reason to be concerned when software projects do not meet estimated targets. But everyone is concerned anyway. #

• in our culture today, people are trying so hard to achieve (impossible) schedules that they are willing to sacrifice completeness and quality in getting there. #

• Fact 13 There is a disconnect between management and their programmers. In one research study of a project that failed to meet its estimates and was seen by its management as a failure, the technical participants saw it as the most successful project they had ever worked on. #

• Fact 14 The answer to a feasibility study is almost always “yes.” #

• Fact 15 Reuse-in-the-small (libraries of subroutines) began nearly 50 years ago and is a well-solved problem. #

• Fact 16 Reuse-in-the-large (components) remains a mostly unsolved problem, even though everyone agrees it is important and desirable. #

• Fact 17 Reuse-in-the-large works best in families of related systems and thus is domain-dependent. This narrows the potential applicability of reuse-in-the-large. #

• Fact 18 There are two “rules of three” in reuse: (a) It is three times as difficult to build reusable components as single use components, and (b) a reusable component should be tried out in three different applications before it will be sufficiently general to accept into a reuse library. #

• Fact 19 Modification of reused code is particularly error-prone. If more than 20 to 25 percent of a component is to be revised, it is more efficient and effective to rewrite it from scratch. #

• Fact 20 Design pattern reuse is one solution to the problems inherent in code reuse. #

• Fact 21 For every 25 percent increase in problem complexity, there is a 100 percent increase in complexity of the software solution. That’s not a condition to try to change (even though reducing complexity is always a desirable thing to do); that’s just the way it is. #

• Fact 22 Eighty percent of software work is intellectual. A fair amount of it is creative. Little of it is clerical. #

• Fact 26 When moving from requirements to design, there is an explosion of “derived requirements” (the requirements for a particular design solution) caused by the complexity of the solution process. The list of these design requirements is often 50 times longer than the list of original requirements. #

• Fact 27 There is seldom one best design solution to a software problem. #

• Fact 28 Design is a complex, iterative process. The initial design solution will likely be wrong and certainly not optimal. #

• Fact 29 Programmers shift from design to coding when the problem is decomposed to a level of “primitives” that the designer has mastered. If the coder is not the same person as the designer, the designer’s primitives are unlikely to match the coder’s primitives, and trouble will result. #

• Fact 30 COBOL is a very bad language, but all the others (for business data processing) are so much worse. #

• Fact 31 Error removal is the most time-consuming phase of the life cycle. #

• Fact 32 Software that a typical programmer believes to be thoroughly tested has often had only about 55 to 60 percent of its logic paths executed. Using automated support, such as coverage analyzers, can raise that roughly to 85 to 90 percent. It is nearly impossible to test software at the level of 100 percent of its logic paths. #

• Fact 33 Even if 100 percent test coverage were possible, that is not a sufficient criterion for testing. Roughly 35 percent of software defects emerge from missing logic paths, and another 40 percent from the execution of a unique combination of logic paths. They will not be caught by 100 percent coverage. #

• Fact 34 It is nearly impossible to do a good job of error removal without tools. Debuggers are commonly used, but others, such as coverage analyzers, are not. #

• Fact 35 Test automation rarely is. That is, certain testing processes can and should be automated. But there is a lot of the testing activity that cannot be automated. #

• Fact 36 Programmer-created built-in debug code, preferably optionally included in the object code based on compiler parameters, is an important supplement to testing tools. #

• Fact 37 Rigorous inspections can remove up to 90 percent of errors from a software product before the first test case is run. #

• Fact 38 In spite of the benefits of rigorous inspections, they cannot and should not replace testing. #

• Fact 39 Postdelivery reviews (some call them “retrospectives”) are generally acknowledged to be important, both from the point of view of determining customer satisfaction and from the point of view of process improvement. But most organizations do not do postdelivery reviews. #

• Fact 41 Maintenance typically consumes 40 to 80 percent (average, 60 percent) of software costs. Therefore, it is probably the most important life cycle phase of software. #

• Fact 42 Enhancement is responsible for roughly 60 percent of software maintenance costs. Error correction is roughly 17 percent. Therefore, software maintenance is largely about adding new capability to old software, not fixing it. #

• Fact 43 Maintenance is a solution, not a problem. #

• Fact 44 In examining the tasks of software development versus software maintenance, most of the tasks are the same—except for the additional maintenance task of “understanding the existing product.” This task consumes roughly 30 percent of the total maintenance time and is the dominant maintenance activity. Thus it is possible to claim that maintenance is a more difficult task than development. #

• Remember that the development life cycle was a 20-20-20-40 affair—20 percent requirements, 20 percent design, 20 percent coding, and 40 percent error removal. #

• Fact 45 Better software engineering development leads to more maintenance, not less. #

• Fact 46 Quality is a collection of attributes. #

• Quality in the software field is about a collection of seven attributes that a quality software product should have: portability, reliability, efficiency, usability (human engineering), testability, understandability, and modifiability. #

• Fact 47 Quality is not user satisfaction, meeting requirements, meeting cost and schedule targets, or reliability. #

• Fact 48 There are errors that most programmers tend to make. #

• Fact 49 Errors tend to cluster. #

• Fact 50 There is no single best approach to software error removal. #

• Fact 51 Residual errors will always persist. The goal should be to minimize or eliminate severe errors. #

• Fact 52 Efficiency stems more from good design than from good coding. #

• Fact 53 High-order language (HOL) code, with appropriate compiler optimizations, can be about 90 percent as efficient as the comparable assembler code. Or even higher, for some complex modern architectures. #

• Fact 54 There are tradeoffs between size and time optimization. Often, improving one degrades the other. #

• Fact 55 Many software researchers advocate rather than investigate. As a result, (a) some advocated concepts are worth far less than their advocates believe, and (b) there is a shortage of evaluative research to help determine what the value of such concepts really is. #

• Fallacy 1 You can’t manage what you can’t measure. #

• GQM approach #

• Fallacy 2 You can manage quality into a software product. #

• Fallacy 3 Programming can and should be egoless. #

• Fallacy 4 Tools and techniques: one size fits all. #

• Fallacy 5 Software needs more methodologies #

• Fallacy 6 To estimate cost and schedule, first estimate lines of code. #

• Fallacy 7 Random test input is a good way to optimize testing. #

• Fallacy 8 “Given enough eyeballs, all bugs are shallow.” #

• Fallacy 9 The way to predict future maintenance costs and to make product replacement decisions is to look at past cost data. #

• Fallacy 10 You teach people how to program by showing them how to write programs. #

• Reality is the murder of a beautiful theory by a gang of ugly facts. #