Testing has always been an important step to ensure the quality of the software R&D process, but with traditional R&D model, there has been a lack of linkage between testing and the software life cycle. With the promotion and implementation of the DevOps model in the software industry, more frequent delivery further aggravates the industry's concerns about testing. Inefficient testing is often the primary reason for delays in delivery, and the testing process has become the biggest bottleneck for enterprises to transform into DevOps.

In order to solve such challenges, the concept of "continuous testing" was proposed and gradually became the inevitable pursuit of the industry. In DevOps, continuous testing provides a continuous feedback mechanism that acts as a catalyst throughout the product delivery pipeline, with automatic feedback at each stage ensuring that defects are addressed early in the development process.

This article will focus on sharing how Sunline's R&D collaborative management platform helps companies get feedback on software release business risks as soon as possible in the "continuous testing" stage of DevOps.

Building an integrated continuous testing platform with technology accumulation

As a carrier of DevOps practices in the testing field, continuous testing is a basic and continuous activity that should run through the entire software delivery cycle. Compared to the traditional testing mode that is prone to many issues in the DevOps era, the first thing that continuous testing needs to change is the "post-testing" situation. Emphasizing on pre-testing, continuous testing define tests as early as possible with parallel testing and development, maintaining close collaboration in the process to achieve quick feedback on business risk.

Focusing on the ultimate goal of "improving efficiency through continuous testing", MOne Test, an integrated continuous testing platform created by Sunline’s MOne product team, helps the R&D team take testing as a basic activity throughout the entire software delivery process with powerful test management functions. In the process, the software delivery cycle is greatly shortened, testing and R&D are synchronized and iterated, continuous testing is realized, and high-quality delivery is achieved through:

·       Full life cycle covering different stages from test planning, test execution to test report analysis.

·      Rich team collaboration supporting testing teams of different sizes, and provides all the functions to meet the needs of team collaboration and process automation.

·       Continuous testing with the ability to integrate the testing process into continuous delivery and DevOps system, seamlessly connecting to defect management tools (JIRA, MOne Project) and continuous integration tools (Jenkins, MOne Pipeline), etc.

·      Integration covering test case management\review\execution, interface\WEB automation, fault drill, performance test (compatible with JMeter native jmx script) and other functions.

The practice of continuous testing within iterations in MOne Test

The testing process generally includes use case design, use case review, test execution, and test reporting. MOne Test integrates continuous testing platform to increase the collaboration and feedback between testing and other roles in the process, and opens up data interaction with the project management platform. The purpose is to help the team solidify practical experience through product capabilities, so as to achieve high efficiency within iterations test with:

Efficient organization and management of test cases to easily organize use cases in the tree-structured use case library, testers can flexibly reuse test cases to form use case reviews and test plans, greatly improving work efficiency.

Docking with mainstream project management platforms such as JIRA, MOne Project, etc., test cases can quickly associate requirements and defects, and in the process of test case execution, defects can be quickly created and synchronized to a third-party defect platform.

Rich online collaboration with multiple roles in the team allow multiple participations in the use case review and execution at the same time, the work status is synchronized real-time, cross-test redundancy is avoided and information is synchronized efficiently.

Efficient testing requires automation as the cornerstone

In the high-frequency delivery scenario of DevOps, how can the team improve the efficiency of test execution? The first thing that comes to our mind is automated testing as it is the basis of continuous testing. Only with a high degree of automation can it meet the high-frequency release requirements of continuous delivery.

Currently, most testing tools use open-source interface automation frameworks or open-source interface testing tools to automate interfaces, such as Postman and JMeter. However, this method has high maintenance costs and lacks the function of batch execution, which cannot meet the needs of rapid verification.

MOne Test is an integrated continuous testing platform, which fully refers to the page layout of open-source tools in design. It is easy to use, and supports batch execution, which can realize "zero code" interface automation.

Rich parameter transfers to meet the needs of data construction with the main types of parameters are as follows:

·       Parameter constructor to quickly generate data for testing, such as random numbers, timestamps, ID numbers, etc.

·       Public parameters put public data in the script into the public parameters for unified management, which is easy to maintain.

·      Step parameters allow quick reference of data returned by the previous step, and support referring to the data returned by different types of steps.

·      Transaction data show data reference across scripts to meet complex banking scenarios.

·      Project environment variables allow quick change to the script URL and public data by switching the project environment (test, SIT, UAT) when the script is executed so that the same script can cover different running environments.

Flexible scenario arrangement supports the combination of steps such as API, API template, database query (MySQL, Oracle, PostgreSQL), JavaScript script, etc., to jointly complete the interface test of complex business scenarios.

Support batch execution visualizing the entire execution process of each API request i.e., request header, request body, response header, response body, assertion in the script execution step will be recorded. If the execution fails, the failure step will be prominently marked and the cause of the failure will be displayed.

Integration of automated testing capabilities into the DevOps system provide Jenkins plug-ins, which can directly schedule batch tasks on the test platform after API key authentication. Self-developed MOne Pipeline provides tool-specific operation steps to easily realize one-click automatic deployment and testing.

MOne Test helps the interface automation construction of a banking system

·       The automated scripts cover a total of 2,000 key transactions and develop nearly 3,400 automated test cases.

·     Batch tasks can be fully returned to the system every week and each time before going online, and the system can be fully expanded and executed concurrently in multiple environments.

·       Generate a fixed system start date through transactions, and restore data through scripts after each round of testing performed.

Through the construction of interface automation, the efficiency of the bank's testing can be greatly improved. Multiple scripts can be executed concurrently during task execution, thereby greatly shortening the regression cycle and reducing the risk of digitalization.

In the process of practicing continuous testing, enterprises not only need technical support, such as the basic capabilities of continuous development, continuous integration, and continuous deployment but they also need to pay attention to the precipitation of data, and continuously optimize behaviors based on data indicators, so as to realize the benefits of DevOps.

In the next article, we will share more on the practices in the "continuous deployment" phase, so stay tuned!