Performance Testing Guideline
Version: 2, effective date: 28-Nov-2022
Contents
1 Standard description
This document aims to provide a clear guideline about T&T Performance Testing, defining its standards and types that should be applied to all JTI testing projects which use the T&T environments.
2 Document objectives and benefits
2.1 Objectives
Objectives of the document are: - Description standards and types that should be applied to all JTI testing projects in T&T environments - Description of performance testing and its types - Defining the performance testing process - Suggesting test metrics and tools
3 Definitions
| Abbreviation / Term | Explanation |
|---|---|
| T&T | Track and Trace |
| ROI | Return on Investment |
| KPI | Key Performance Indicators |
Please refer to IT Glossary in the IT P&P portal for further definitions.
4 Roles & Responsibilities
Accountable: Approves the activity or recommendations from a person or team.
Responsible: Responsible for doing the work associated with an activity, either by completing personally or through sole accountability for a team’s output.
Consulted: Reviews the output of an activity and provides input but has no approval authority. Provides support to activity and may be consulted by the team during the course of the activity.
Informed: Is informed about activities to aid in planning own work.
| # | Activity | GDC TTDC | BTS T&T | GSC CP |
|---|---|---|---|---|
| 1 | Define and maintain standard | A/R | C | I |
| 2 | Apply the standard in the T&T Implementation projects | A | R | C |
5 Performance Testing
Performance testing is a software testing method for evaluating a software application’s:
-
speed
-
reaction time
-
stability
-
dependability
-
scalability
-
resource utilization
under a certain workload.
The fundamental goal of performance testing is to find and remove performance bottlenecks in software applications. It is also known as “Perf Testing”, and is a subset of performance engineering.
5.1 Types of Performance Testing
-
Load Testing: Verifies the application’s capacity to execute under expected user loads. The goal is to discover performance bottlenecks before launching the software application.
-
Stress Testing: Entails subjecting an application to significant workloads in order to determine how it manages heavy traffic or data processing. The goal is to determine an application’s breaking point.
-
Endurance testing: Ensures that the software can withstand the projected load for an extended length of time.
-
Spike testing: Evaluates the software’s response to sudden huge increases in user load.
-
Volume testing: Data is entered into a database, and the general behavior of the software system is monitored. The goal is to test the performance of a software application with small to substantial database sizes.
-
Scalability testing: Verifies the efficiency of a software program in “scaling up” to handle an increase in user load. It aids in the planning of capacity additions to your software system.
6 Performance Testing Process
6.1 Appropriate Testing Environment
Examine the software, hardware, and network setup that will be used for the performance test. Understanding the host system’s strengths and limitations is critical later in spotting possible difficulties.
6.2 Determine the KPI
In this phase, the business side should calculate how many concurrent requests the system must handle and how rapidly it must respond. Resource usage and platform should be considered carefully, and KPI should be defined.
6.3 Plan and Design
Define how usage is expected to differ across end users throughout the planning and design process, as well as scenarios to test that demonstrates a common workload.
For example: Over a ten-minute period, the test may check the system’s ability to handle one thousand simultaneous real-life aggregation requests.
6.4 Execute the Test
Monitor and record platform performance data for analysis while the test is running. This data not only determines if the infrastructure is capable of handling the traffic load effectively, but it also aids in determining where improvement should be made in order to meet the business needs.
6.5 Examine, Fine-tune, and Re-test
Examine the outcomes of the exam, both during and after the test. If the testing platform encounters a difficulty, such as a CPU or a network bottleneck, halt the test and resolve the issue before continuing the operation.
Analyze, consolidate, and disseminate the information with the team after the test findings are complete. Use this data to fine-tune the test to assess different levels of platform performance and to suggest strategies to increase platform performance. Repeat the test and track performance gains to determine if the suggested changes enhance performance.
7 Suggested Test Metrics
- General
- Response Time
- Sum of requests that were serviced by shared connections from several clients
- Indicator showing the maximum number of requests
- Average waiting times
- Maximum Load of the response
- Hardware/OS
- The percentage of time that the CPU is actively processing data.
- Processing that makes use of a computer’s actual memory
- The amount of time that read/write operations take up on the disk
- The amount of memory that is exclusively occupied by a single process (used to measure memory leaks)
- Memory utilized in a virtual machine
- Amount of disk I/O used to fix page faults caused by hardware
- Total number of hardware interruptions experienced by the CPU per second Average number of bad pages dealt with by the processor
- Length of the output packet queue Number of requests to read or write to the specified disk on average during the sample period
- Active thread count is the total number of threads that are presently being processed.
- In the system, the pace at which unused memory is returned to the (garbage collector)
- Network
- The rate in bits per second at which the network interface operates (bandwidth)
- Amount of data sent across the interface in one second in bytes
- How many times a file is requested from a web server per second
- Rate of requests per second received via network
- Database
- The percentage of SQL requests where cache, time, and resource saved by avoiding disk read/writes.
- Availability of data restoration capacity
- Database and table-locking quality
8 Suggested Performance Testing Tools
Platform performance.
-
Apache JMeter
-
CPU Expert
-
MSSQL Monitoring Tool
-
Locust
9 Document control
9.1 Document Owner
The owner of this document is Berhan Cem Ozelbicer.
9.2 Contact person
Questions and feedback regarding this standard should be submitted to Berhan Cem Ozelbicer.
9.3 Revision History
| Version | Effective date | Purpose of change | Author |
|---|---|---|---|
| 2 | 28-Nov-2022 | First Publish-Ready version | Berhan Cem Ozelbicer |
| 1 | 23-Nov-2022 | First version of the document | Berhan Cem Ozelbicer |
10 References
References will be added here when necessary.
ANY QUESTIONS?