Saturday, February 21, 2009

Non-Functional Requirements | System Reliability

Among the many nonfunctional requirements that can be included in a specification are those related to safety and reliability. How can we demonstrate that these requirements are testable? How can we demonstrate the reliability of a system that is required never to fail?

It is a common practice to test the functional requirements of a system. However, testing the nonfunctional requirements like safety, reliability and security are usually put off until the end of the design process. This usually involves a lot of rework since the nonfunctional requirements have significant effects to the workflow itself.

In order to test software reliability, I found this “solution” over the internet called “Software Reliability”. Yes, it has the same name. This method of testing software reliability considers the probability of a failure-free software operation for a period of time in a specified environment. While this may be a possible test of software reliability, there is still no perfect solution or test that we can perform to test software because of its complexity. People have long since falsely believed that software does not break down or rust like machinery. However, I believe that it is impossible to demonstrate the reliability of a system that is required never to fail. Never is such a timeless and infinite word and it is unquantifiable. The most we can do is to make sure that the system undergoes functional and nonfunctional testing since early during the design phase. This way, risks and breaches will be identified and addressed earlier.

I also found over the internet an interesting article about testing software security, where security is one of the non-functional requirements. It was stated there that software security is about making software behave in the presence of a malicious attack. Testers must use a risk-based approach, grounded in both the system’s architectural reality and the attacker’s mindset, to gauge software security adequately. The former is called the white-box approach and the latter the black-box approach. The white-box approach entails analyzing the source code and design, which is effective in identifying bugs in the code and flaws in the design. On the other hand, the black-box approach, which I found really interesting and quite sensible, involves testing the software security by probing it with various inputs. During this test, malicious inputs can be supplied to the system in an effort to break it. If it breaks, then that is an identified problem that must be addressed. The thing that I found most interesting in this approach is the fact that for testers to be able to really test the system, bias must be eliminated and therefore they must “think like an attacker” and “burn” with the desire to foil the system.

However, despite of all the available methods to test nonfunctional requirements, there is no sure-fire way to guarantee that the system will never fail. I think that from all the research and methods that I have read, the most practical and probably most effective is to test the system at regular intervals even during the design phase. Though it may not totally eliminate risks and prove a software’s reliability and functionality, it can definitely lessen the negative effects if the system is to undergo functional and nonfunctional glitches.

No comments:

Post a Comment