Non-functional behaviors are “How” the system should perform its functions. In today’s digital age, it’s not just about if an application works, but more importantly, how it works. Testing an application to validate its performance, security and accessibility is commonly known as Non-functional Testing (NFT).
Shift left with NFT, allows developers / testers to leverage scaled-down environments for non-functional testing at a component level as and when, they are developed. The re-use of open source frameworks and functional automation scripts to perform early NFT, reduces the overall vulnerabilities in later stages. Additionally, Application Performance Management (APM) for end-to-end monitoring, alert mechanism and virtualization techniques, ensure continuous Non-Functional Testing.
Non-Functional Testing in DevOps is integrated across the lifecycle, here are few typical use cases:
In this module, you will explore how experience is assured with non-functional testing. In digital, customer experience is critical for success and non-functional aspects play a significant role in an application’s functionality. You will see how an application’s performance, security and accessibility are validated in continuous pipeline with an open source tool stack.
The Experience Assurance pipeline consists of two stages viz. source and build.
With every code change committed to the repository, all three tests will be triggered in a sequence. Various dependencies and commands required for these tests are defined in the buildspec.yml file in the source code repository.
Performance Tests are executed with the open-source tool Jmeter (version 5.1.1). The tests will run with five concurrent users and results will be saved in the directory “performance_test_results” in the target s3 bucket.
Performance Tests source files:
Security Tests are triggered via the open-source tool owasp ZAP. These tests will perform a passive baseline security scan with predefined rules against the given URL and save the results under “security_test_results” in the target s3 bucket.
Security Tests source files:
Accessibility Tests are run using Selenium and AXE tool to analyze the application’s response against accessibility guidelines. Here, accessibility checks are done on steps executed by Selenium scripts (for each page that is rendered) and results are displayed under “accessibility_test_results” in the target s3 bucket.
Accessibility Tests source files: