There is an article “Perfect software” by Jack Ganssle on EETimes dated 3/1/2009 12:00 AM EST. A few points from there:
- “Theoretically, Software is the only component that can be perfect, and this should always be our starting point.” – That is said by Jesse Poore. Following his web page you’ll find out how perfect software can be made at no much more cost than normal software.
- There are industrial providers of highly reliable OS’s. The article mentioned Micrium and Green Hills. Following Micrium’s web page, there are list of standards for certifications. Those must be the processes and rules the industry follow. That’s based on formal validation, but diverted from the theory a lot.
Interestingly, regarding the commercial software, data collected by Capers Jones suggests that “software in general has a defect removal efficiency (the percentage of bugs removed prior to shipping) of 87%. Firmware scores a hugely better 94%.” To me, none of these are near perfect. The article an earlier answerer mentioned indicates that NASA space shuttle team achieved a 99% bug removal rate, but the cost is at 35 million per year for about 400k lines of code.
A more interesting article “Software for dependable systems” by the same author on 11/1/2009, seems to be more relevant. It can be summarized like this:
- As to development process, the industry has followed DO-178B or IEC61508. It emphasize testing with greatest coverage. But little evidence can be found about effectiveness of this.
- A certification authority, the Committe on Certificably Dependable Software Systems, has published a book titled “Software for Dependable Systems – Sufficient Evidence”. It’s a good reference.
- The book, basically asks for three things:  Make explicit rules for dependability tests.  Tests against the rules, but also make sure the tests are understandable by normal customers or regulators with a magnitude less time than spent by the developers.  Be sure it is the experts in software engineering and problem domain are doing the development and test.
- The supplier of software must commit to a warranty or other remedies for any software failure.
- Use a safe language, specifically not C. SPARK is suggested.
- Design for contract approach is one of the strength of SPARK. Design-for-contract is an important technology to embed tests into every function/method call, and always execute it at runtime. More than a simple assert() in old days, the contract may also embeds tests against structural intention and make sure no violation can be slipped in at later times in maintenance cycle when the original developers mostly have moved on to other projects. There are evidences that design for contract has produced very reliable software products. SPARK and its tools can be used to assist generating certification test cases to simplify the certification process.
According to my memory HP practiced design-for-contract almost a decade ago. With a small team, 500k lines of code, only 2 bugs reported after the delivery. Very impressive.
In my view, dependable or perfect software can only be achieved if the cost is not prohibitively high. Frameworks or automations is a must-have.
(Originally published on programmers.stackexchange.com)
This work by minghuasweblog.wordpress.com is licensed under a Creative Commons Attribution 3.0 Unported License.
|This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0/ or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View, California, 94041, USA.|
|Published on minghuasweblog.wordpress.com on Sep 28, 2011 @ 5:26 GMT|