The State of Non-Functional Testing

Introduction

While in the previous article, we delved into some functional testing types(Unit, Integration, and Regression Testing), in this one, we will talk about Non Functional testing and some of the tools we use to achieve its implementation.

You may ask yourself what is the scope of this type of testing? As the name implies, Non-Functional testing is performed to assess and verify the non-functional requirements of the system under test. The application's security, performance, portability, and usability, alongside other non-functional characteristics, should be scrutinized to meet the business expectations associated with the software product.

One conclusion you'll probably reach before the end of this article, and what I'm also trying to get through, is that Non-functional testing sadly gets much less attention than it deserves. Most times the costs of including Non-functional testing procedures in your software development lifecycle are much lower than costs provoked by a serious security breach, data leak, or even just the extra server time, extra storage space or lower battery life caused by low optimisation or overall poor performance.

Security Testing

I've chosen to start with Security testing because more often than not, it's overlooked during the development process. There are a lot of resources allocated towards a software project, front-end and back-end developers, QA, Dev Ops, Managers, Scrum Masters, etc. Rarely would one see the presence of a penetration tester or a security engineer. Only when a malicious entity breaks into the system, then and only then resources are put in place for this very important process.

According to OWASP®, security testing can be split according to the types of vulnerability being tested:

• Vulnerability Assessment - During this assessment, the system under test is scanned and analyzed for security issues.
• Penetration Testing - The system under test undergoes attacks from a white-hat hacker.
• Runtime Testing - The system undergoes analysis and security testing from an end-user.
• Code Review - The system's code base goes under review.

One of the tools used in the Security Industry is OWASP® Zed Attack Proxy. ZAP is a security tool, designed for use against web applications and its scope is detecting vulnerabilities and security risks.

Top 10 OWASP security risks to be taken into account by the Pentesters or developers

• Broken Access Controls
• Cryptographic Failures
• Code injection/Unsanitized data
• Insecure Design
• Security Misconfiguration
• Obsolete/Vulnerable Components
• Identification and Authentication Failures
• Software and Data Integrity Failures
• Security Logging and Monitoring Failures
• Server-Side Request Forgery (SSRF)

Performance testing in the context of Modern Web Applications

Nowadays, when designing a modern web app, development teams can expect its system to be under heavy loads, perhaps even hundreds of thousands of concurrent users. Performance Testing is the process that verifies that an app can support an adequate number of client-server transactions, promptly, and without impeding the user experience. 53% of mobile site visits are abandoned if pages take longer than 3 seconds to load. A further 2-second delay resulted in abandonment rates of up to 87%.

We can break down this umbrella term - Performance Testing, into several subtypes.

• Load testing determines the baseline number of users that the system supports in normal conditions. The test results can give insight into what to expect when the systems go live.
• Stress testing verifies how the system performs under higher-than-expected traffic load. The test can be used to determine bottlenecks and possible breaking points where the back end fails due to the burden on the hardware resources or cloud scalability.
• Spike testing simulates a sudden influx of users, well above the normal baseline arriving in a short period time. This type of test is done when expecting higher than normal traffic volumes.
• Endurance testing, or Soak testing simulates heavy user load, spread over extended periods. The goal is to determine that the system can withstand higher-than-usual traffic without gradual deterioration of response times and other performance KPIs.

Performance KPI's

During performance testing, we follow some Key Performance indicators. Using the information gathered from these quantifiable metrics we can determine if the test is successfull or not. In the case of test failure, the development team can identify areas of poor performance and fix or improve on them. Some of these KPIs are:

• Average Response times, the most commonly used metric. The median of all response times observed for all users accessing a certain resource.
• Response Error Rates, this indicates the number of times the transaction fails per 100 transactions.
• The Number of Concurrent users, the number of active users in the system.
• Requests per second - the number of requests per second the server is subjected to during performance testing.

Conclusion

As most things in engineering, the rabbit hole goes much much deeper and we barely scratched the surface - but what I wanted to point out through this article is that while often overlooked, Non-Functional testing is and should be a core part of the software development lifecycle. Human nature makes us not see the value in preventive measures until we've bumped our heads a few times. People building software, whether they're entrepreneurs or engineers, don't always love to invest in things which have an unquantifiable value. That's the case with NFT as well, more often than not people regret not taking it seriously after it's too late. It is a great preventive measure to apply and avoid pitfalls and potential disasters. As an interesting read, take a look at the list of Biggest Data Breaches in US History to see some really disastrous consequences that having a security vulnerability may produce. Performing Non-functional testing and taking it seriously, in the long run, saves money, time and frustration. I hope this brings some light onto a somewhat overlooked subject in smaller or mid-sized software projects.