|
|
 Number 6 April 2003 |
|
|
|
|
Comparing Security: Commercial Software vs. Open Source |
 |
|
| How do you measure security? There’s no widely accepted and objective answer to this question. There is, however, a widely accepted and objective organization that serves as a clearinghouse for security information. It’s the CERT Coordination Center, part of the Software Engineering Institute located at Carnegie Mellon University. This group periodically sends out security advisories, each of which describes a specific security problem that CERT believes is serious enough to warrant alerting the world. (For the complete list, see www.cert.org/advisories.) CERT advisories don’t cover every possible issue, but they’re a reasonable proxy for comparing the relative security of different categories of software. Examining the CERT advisories produced in the last couple of years yields some interesting results. Based on the kind of software they apply to, I’ve grouped the advisories into four categories:
In some cases, it’s challenging to decide whether an advisory should be categorized as impacting commercial Unix or open source software, since some affect both. In cases like this, I’ve generally chosen to assign the advisory to the commercial Unix category. Also, a few of the advisories aren’t about software at all (they detail security problems in a protocol such as TCP, for example), and so I haven’t included them here. With these caveats, here’s how the CERT advisories issued in 2001 break down:
The column on the left shows all CERT advisories for commercial software, including 14 issued on Microsoft products alone. 2001 was the year of Nimda and Code Red (which received two advisories all by itself) and the Anna Kournikova email virus—it was a bad security year for Redmond. In fact, I’d argue that the problems of 2001 and the widespread publicity they received were what established Microsoft’s reputation for weak security. The commercial Unix world was the subject of only ten advisories that year, while all other commercial products garnered six. As the column on the right shows, a mere four advisories were issued in 2001 for open source software. The picture for 2002 isn’t quite the same, however. Here’s a breakdown of the CERT advisories issued for that year:
Taken as a group, there were still more advisories produced for commercial software than for open source software. Both Microsoft and the commercial Unix world did better in 2002 than 2001, although the count for other commercial software rose. Yet the most noticeable thing about 2002 is the greatly increased number of CERT advisories issued for open source software. At 15, the open source world had more advisories in 2002 than Microsoft received in 2001. The kinds of advisories issued for open source were also interesting. Some of them describe Trojan horses inserted in open source software distributions, for instance, a problem that never shows up in the commercial software advisories. Finding Trojan horses in open source software is probably easier, since you can examine the source code. Still, this is the sort of problem that one might not expect, since the visibility of the source should theoretically make it difficult to modify in this way without being caught. Open source fans argue that given enough eyeballs, all bugs are shallow. Yet what really matters isn’t the number of eyeballs—it’s the brains that drive those eyeballs. Are they in the heads of competent, disciplined software developers? Is the process they’re following likely to lead to secure code? In software development teams, quality matters more than quantity. As these numbers show, eyeballs alone apparently aren’t sufficient to guarantee secure code. What about the relative security of Windows vs. Unix? Even in 2001, Microsoft’s annum horribilis, the total number of CERT advisories issued for Unix software, (that is, the sum of those for commercial Unix software and the overwhelmingly Unix-based open source world) was equal to those issued for Windows software. In 2002, the sum of Unix-related advisories was much larger than the Windows total: 24 vs. 6. Comparing security by counting advisories doesn’t tell the whole story, of course. The Code Red and Nimda attacks in 2001 affected far more people and cost far more money than all of the more numerous problems experienced by the open source world in 2002 put together. At least in part, this reflects the more widespread use of the targeted Microsoft software. If open source continues to grow in popularity, I’d expect that the level of pain its security problems cause will grow, too. It’s also fair to say that Microsoft software has tended to attract more attention from the kind of people who create viruses and worms than has open source software. As the use of open source spreads, this too will likely change. After all, isn’t it easier to find and exploit security holes in software if you can look at the source code? By the end of the first quarter of 2003, Microsoft and the open source world had received four CERT advisories each. Unfortunately for Microsoft, one of these was the Slammer SQL Server worm, which indicates that despite its progress in 2002, Microsoft still has plenty of room for improvement. Yet the assertion that open source software is inherently more secure than commercial software looks hard to justify. Making the argument on philosophical grounds, as many have done, can lead to good debates. Examining the data, however, is much more likely to lead to good decisions. David Chappell’s May Speaking Schedule
May 9: May 14: May 19: May 20: May 21: May 22: May 29: Coming up in June: June 1:
|
|
||
 |
David Chappell is Principal of Chappell & Associates (www.davidchappell.com) in San Francisco, California. Through his speaking, writing, and consulting, David helps information technology professionals around the world understand, use, market, and make better decisions about enterprise software technologies. David has given keynotes at conferences and in-house events in the U.S., Europe, Latin America, and the Middle East, and his seminars have been attended by tens of thousands of developers and decision makers in thirty-five countries. David’s books have been translated into ten languages and used in courses at MIT, ETH Zurich, and other universities. His consulting clients have included HP, Microsoft, Stanford University, and other technology vendors and users.
|
©Copyright2007
David Chappell and Associates
|
|
||||
|
