Testing Standard
Version: 6, effective date: 10-Oct-2025
Contents
1 Standard Description
This document defines the details of Testing Standards for JTI’s Track and Trace procedures.
2 Document Objectives and Benefits
2.1 Objectives
This document aims to provide information about T&T Testing Standards and types that should be applied to all JTI testing CHG(s) which involves T&T.
- Define General Test Approaches
- Define Rules and Standards for Test Scope Description of T&T Applications
3 Definitions
| Abbreviation / Term | Explanation |
|---|---|
| T&T | Track and Trace |
| ROI | Return on Investment |
| FRS | Functional Requirement Specification |
| TS | Technical Specification |
| UAT | User Acceptance Testing |
| CHG | ITSP Change Request |
| BA | Business Analyst |
| RM | Release Manager |
| QA | Quality Assurance |
Please refer to IT Glossary in the IT P&P portal for further definitions.
4 Roles & Responsibilities
| # | Activity | GDC T&T | BTS T&T | GSC CP |
|---|---|---|---|---|
| 1 | Define and maintain Standard | A/R | C | I |
| 2 | Apply the Standard in the T&T Implementation CHG(s) | A/R | R | C |
5 Test Processes
Each T&T application has a scope of possible business scenarios to be tested. The scope provides a matching test coverage based on the business criteria and a test type applicable to each specific T&T application: if an acceptance criterion or type is not applicable to a particular application, the related test scope can be ignored. It is mandatory to have an acceptance criteria, TS(technical specification) document and FRS(Functional Requirement Specification) document to start the testing procedure.
All testing methods should be carried out following the testing team management’s action plans and decisions. In testing process-related activities, the following roles are involved:
-
Release Aanager(TTDC): Responsible for Release organization, planning, monitoring the reports, and decision making.
-
Business Analyst(s)(BTS): Duties include maintaining the product backlog, documentation(for example : FRS that will be provided to App Owner for approval and QA team for Test Design), writing acceptance criteria, assisting with initial test case creations, and providing test data for the QA team.
-
QA Manager(TTDC): Responsible for planning, coordinating, and controlling the testing activities, making decisions on suitable test methods and tools, documenting the test activities, and creating test reports to be presented to the Project Manager.Responsible to make desicion if a tested item(s) are meeting expectations for PRD deployment.
-
QA Engineer(s)(TTDC): Responsible for analysis and validation of system requirements, creating test cases for different types of testing, executing all types of testing, design and develop automation scripts when needed, detect and track software defects.
-
System Owner(s)(TTDC/BTS): Responsible for development/defect fixes, assisting QA teams during the deployment, providing documentation (User Manual, Product Specification Document, TS, FRS etc.) and internal training when needed.
-
Business Owner(TTDC/BTS): Test manager and QA engineers together, form a QA Team.
5.1 Testing Landscape
From testing perspective application contains the following environments(naming may differ depending on the software provider):
-
DEV - Development environment used for development and internal testing by Vendor/Dev teams
-
QA - Quality assurance environment used for auto and manual functional and non-functional testing by QA team
-
PrePROD - Pre-production environment used for final user acceptance testing (optional). This environment is similar in terms of specification and version to PROD environment
-
PROD - Production environment
Notes:
- The system may contain several QA environments for parallel testing.
- The system may not always contain Pre-Production Environment therefore all testing should be run on QA Environment(s).
5.2 Prerequisites
At this stage, requirements for QA team, if there are any, to start test planning must be met. The Dev team and Business analyst cooperate closely to construct an understanding of testing to be done, detailed analysis is left to QA team. User stories and acceptance criteria must be written, specific access rights, documentation(User Manual, Product Specification Document, FRS etc.), test data(if required by QA team) must be provided by either Dev team or BA. Meetings between Dev team and QA team with involvement of Business analyst can take place in this stage to improve the understanding of testing required. For all CHG(s) - Product Materials(SKU) should be defined and delivered to QA Team/Environment(BTS) - Pack Codes to be used should be registered and delivered to QA Team/Environment(BTS) - Business Entities(BE) should be defined and delivered to QA Team/Environment(BTS) - FRS with detailed Acceptance criteria should be delivered for Test Planning/Design - Reference documents should be shared with QA team (BTS/GCSS)
5.3 Planning Phase
Before each CHG stage all stakeholders should be aligned on planning phase, documented board meetings should sign off the documented agreement of each member. A draft plan can be constructed in this phase with the contribution of all the leads of each team involved. This plan will be developed further with feedback from each team and eventually finalised in a later stage also by contribution of all teams involved. FRS should be ready to finalize this stage by approval from QA team.
5.3.1 Test Design
This step entails establishing roles and allocating responsibility for testing-related tasks in QA team. If other teams are required to be involved in the testing process, each team is responsible for resource allocation in their respective teams. The qualifications and expertise of those participating are also established; nonetheless, everyone is responsible for the testing process’s quality and traceability. The test design process starts with the Business Analyst (BA) providing and clarifying the high-level requirements and acceptance criteria with the final version of the FRS, which serve as the foundation for the entire testing effort. This critical information allows the QA team to establish clear test objectives and identify comprehensive test scenarios that map out how a user interacts with the system. From this clarified base, the QA team can then apply systematic techniques, such as Equivalence Partitioning and Boundary Value Analysis, to generate a set of detailed, step-by-step test cases that ensure maximum coverage and effective defect detection.
5.3.2 Test Duration Estimation
This phase is where the QA team quantifies the effort needed, estimating the total time to execute all test cases. The calculation considers feature complexity, the mix of manual/automated testing, and resource availability, including time for setup, data, and defect re-testing. Estimates rely on methods like historical data to provide reliable input for the schedule.
5.3.3 Test Planning
Test plan is a document that outlines the scope, strategy, resources, and timeline of planned testing operations. Test suites and cases will be created by QA team in accordance with acceptance criterion, documentation and user stories provided to them in earlier stages.
5.3.4 Risk Analysis
The first step BTS and TTDC should conduct a risk analysis about possible impact on integrated systems or related functions.FRS should be updated immediately as soon as a scope increase is detected.
5.3.4.1 Test Flow for New Software/Deployment
The list below sets up a general flow for software testing to a completely unfamiliar environment, market, and/or customer:
-
The test environment(s) should be prepared, installed, deployed, configured, and tested in such a way that no issues can arise as a result of the environment setup. Prior documentation should clearly and thoroughly outline the requirements, including hardware requirements, operating system specifications, and software/tools with compatible versions. Responsible: QA Team & Business Analyst & Dev Team
-
Smoke Testing is the first step of the test life cycle ensuring the high-level whether main functions of the software appear to work correctly. This testing is rudimental, and any defects found here means that the testing cannot continue. Dev teams must fix the issues found here for testing to continue. Responsible: QA Team
-
Integration Testing is the step of the test life cycle ensuring each system and component is in place, accessible and communicating with each other. Responsible: QA Team
-
System Tests is the collection of all user stories and related test cases, QA Team will be running all the test cases to verify the deployed software running as expected. Responsible: QA Team
-
User Acceptance Tests should be run by the related stakeholder(s), necessary documentation and training should be provided to stakeholders. UAT team must be formed by the Requestor of UAT testing and should include all the parties that will use the product after it goes-live. While the QA team may lend assistance to UAT team, the testing must be done by UAT team (if the CHG is minor in scope UAT can be handled by the QA team). It is best to conduct this stage of testing in an environment as close to real-life version of the product developed. As this stage will include stakeholders and common users, defects identified by the QA team should be fixed by Dev team before this UAT takes place. Responsible: UAT Team & Business Analyst
5.3.4.2 Test flow for Software Maintenance/Updates
If a testing activity is going to be conducted for a software system that is already deployed, a system test has been done and running, testing types will be changing to save time and budget. For QA team to understand the scope of the changes made by the Dev team, release notes must be provided to QA team as well as additional documentation written by either Dev team or the BA. If the scope of the new deployment is substantial(changes the base of the working system(s)), training meetings between Dev and QA teams may take place with attendance of BA.
-
If the Development team fixed the incidents/defects previously recorded or a new feature has been added to the software after the deployment of the updated version, first step is to run Smoke tests. If issues are found while Smoke testing, the process cannot continue until the issues are fixed by the Dev team. Responsible: QA Team
-
When the smoke tests are successfully passed and issues found during smoke tests are fixed by the Dev team, the Test Manager should analyse and decide the test type that should be applied according to the scope of the update. Severe defects identified in this process can and must block the deployment of the update to the production environment. For example: Usually, smoke tests are followed by sanity testing of a fixed functionality. But if the scope of update consists of several new functionalities or bug fixes, the impact on the system can be creating a risk of affecting other areas. In this case, the test manager may create a large test suit for Regression Test or even a wide scope for complete System Testing. Responsible: QA Team
-
Incidents/defects which are detected after the deployment of the release in production environment, depending on the severity of it, may result in a roll-back of the system version or fixed in the next deployment. If the fix can be isolated and will not have any impact on other functions, can be tested by 24/7 support team as a sanity test and applied on the system without testing by QA team. Responsible: GDC TTDC
5.4 Test Execution Techniques and Tools
In an ideal testing process, there should be two types of test execution: manual and automated. Test manager should choose the applicable type of execution for each type of test, it is recommended that manual testing should be used for System testing of the new functionality and automated testing should be used for regression testing.
In some situations, new functionality will expand or change the old functionality which can create complex test cases. In these situations, it is best to manually test these functionalities before deployment in production environment and then integrate them in automated testing where possible.
It’s advised to use testing methodologies and standards that are up to date and well-established.
5.4.1 Manual Testing
A type of software testing in which test cases are manually executed by QA engineers without the use of automated technologies is defined as manual testing. Manual testing seeks to identify bugs, defects, and weaknesses in a software product. Manual software testing is the most fundamental of all testing methods and evaluates if the business needs are covered by the application as expected.
5.4.2 Automated Testing
Test automation involves the execution of test suites using software automation tools. There is no need for human interaction needed for test execution after the test automation script is developed.
Return on investment(ROI) expected to be improved because of this. Purpose of having test automation is to reduce the number of manual test case execution.
5.4.3 Tools
5.4.3.1 Test Plan Management Tool
Software testing is tracked, organised, and controlled using test plan management technologies. They are typically used to support and monitor manual testing. This tool makes the reporting and traceability of the testing process possible.
5.4.3.2 Bug Tracking Tool
The process of recognising issues, reporting on them, and compiling issue lists is known as issue tracking, or issue management. It is advised to have a robust system here to effortlessly follow up on issues identified.
5.4.3.3 Software Testing Tools
Software testing tools are used to ensure that the tested item fulfils its requirements and objectives.
6 Test Methods and Types
This document does not go into full details of each testing method or type. Software testing is a wide and complex field which one can spend years on a specific area of the field it to master. What is included in this section is to provide a general understanding of testing methods and types.
6.1 Testing Methods
6.1.1 White Box Testing
White Box testing examines the fundamental structure, architecture, and source code of an application as well as the design documents.
Because source code is accessible to QA Team in white-box testing, it is only appropriate for CHG in which the source code is available, and the Development team is involved.
White box testing aims to address the following issues: - Internal security defects - Paths in the coding procedures that are broken or poorly organized - The flow of certain inputs through the code - Expected outcome optimisation - Conditional loop functionality - Individualized testing of each statement, object, and function
Two examples to white box testing is as below;
- Code Coverage Testing: Determines the number of lines of code that are successfully validated under a test method, which aids in determining how extensively a source code is verified and is being executed. By inserting statements that monitor code execution in critical parts of the source code testers can gather information about the inner workings of the system as well as the performance.
- Path Coverage Testing: Aims to uncover any defects in a piece of code that is anticipated to correspond to a business criteria operating in parallel by analysing each line/part of code. This method is meant to run all or a subset of paths through a computer program. This method is very labour intensive, and it is best to carry out this method in critical parts of the source code.
6.1.2 Black Box Testing
Black Box testing is examining the functioning of software applications without access to source code, implementation details, internal procedures, or design documents.
This type of testing can be carried out to evaluate the functionality, security, performance, and other aspects of a software. This testing is carried out by simply emulating the business actions(inputs) in the system and comparing the outcomes(outputs) provided by the system against acceptance criteria and FRS documents.
6.2 Types of Testing
6.2.1 Functional Testing
-
Unit Testing: Aims to execute tests in a single program function or component, such as a method, process, API, module, or function. Objective is to ensure that all components of the source code work as expected. It is being created by application developers and is defined in “SYS003 Track and Trace DevOps Standard” (References: 1.4).
-
Integration Testing: Aims to test all integrated software elements or systems as a group. The goal of this level of testing is to identify interoperability issues that may arise during the integration of certain software modules or systems.
-
Smoke Testing: Verifies that software builds are deployed and stable. Smoke testing confirms that QA Team can continue to further test stages. It consists of a small number of tests that are run on every build to verify the functionality of the program.
-
Sanity Testing: Aims to conduct testing after receiving a software assembly with small modifications in code or functionality to ensure that defects have been resolved and that the changes are applied.
-
Regression Testing: Aims to ensure a recent program or code update has not negatively impacted current functionalities. A complete or partial collection of previously performed test cases that are re-ran to check that existing functionality works as intended after an update.
-
System Testing: Aims to conduct testing of wholly integrated software application. Objective is to examine the end-to-end system and business requirements or health check of the whole system after a major deployment.
6.2.2 Non-Functional Testing
This is a type of black box testing is not related to testing of specific functionality, but non-functional requirements such as performance, scalability, usability.
- Performance Testing: Aims to evaluate the speed, reaction time, dependability, scalability, accessibility, usability and resource consumption of an application in accordance with the business acceptance criteria defined by CHG(S) specification.
- Security Testing: Aims is to identify all potential loopholes and weaknesses in a software application that could lead to information leakage, data loss for. Objective is to protect the business and prevent mentioned hazards.
7 General Approach and Rules of Testing
Below a summary of the steps which underlies the general approach and rules of the testing activities of CHG(s).
-
Analysis: Before testing, the initial step must be to identify and properly obtain knowledge about the product. Product documentation should be reviewed, and if feasible, important user training should be provided to ensure that users understand all the software’s capabilities and how to use it. Analysis documentation and FRS documents should be given for future iterations as a foundation for User Stories and Test Cases. Business analysts should be able to provide user stories and test data to the QA team.
-
Developing a Test Strategy: Developing a test strategy is a vital step in creating a test plan. Test Strategy should be included in Test Plan document by a test manager. It outlines the different testing methods that will be used in testing process.
-
Defining Test Objective: The overall purpose and achievement of the test execution is referred to as the test objective. The goal of the testing should clearly be defined for all parties involved.
-
Defining Test Criteria: A test criterion is a standard or norm that may be used to underpin a test method or test judgment based on business requirements. This is essential to be able to determine the outcome of a test that is conducted.
-
Resource Planning: A resource plan is a comprehensive list of all the resources needed to fulfil a given assignment. A resource might be a person, an equipment, a tool, or a material required to perform a job.
-
Test Environment Planning: Test plan should include the configuration of software and hardware on which the Testing Team will run test cases. These may include user settings, access privileges needed for Testing Team.
-
Schedule & Estimation: The Test Plan should include an estimated effort analysis for how long test activities will take to complete. Aims to create of a timetable based on given estimation.
-
Test Deliverables: Test Deliverables is a list of all the documentation, tools, and other components that must be created and maintained to support the testing process.
8 Document Control
8.1 Document Owner
This document has been written by Berhan Cem Ozelbicer.
8.2 Contact Person
Questions and feedback regarding this standard should be submitted to Olga Ovchinnikova
8.3 Sign Off
| Name | Title | Function | Signoff Date |
|---|---|---|---|
| Avni Cengel | GDC T&T Delivery Center Director | IT T&T Delivery Center | |
| Ioana Petcu | BTS T&T Service Director | IT BTS T&T |
8.4 Revision History
| Version | Effective date | Purpose of change | Author |
|---|---|---|---|
| 6 | 22-Oct-2025 | Revision of document | Berhan Cem Ozelbicer |
| 5 | 05-Oct-2023 | Revision of document | Atakan Izdas |
| 4 | 08-Apr-2022 | Revision of theoretical parts, naming | Berhan Cem Ozelbicer |
| 3 | 07-Sep-2021 | Final revision by GDC team | Berhan Cem Ozelbicer |
| 2 | 05-Aug-2021 | Second version of the document/ Appendix A added | Berhan Cem Ozelbicer |
| 1 | 06-July-2021 | First version of the document | Berhan Cem Ozelbicer |
9 References
- Internal JTI documents for testing
-
SAP Testing Standards
-
Erik van Veenendaal, “Standards: to be used with common sense!”, from Testing Experience, December 2009
-
Jens Fricke & Thomas Niess, “Test 2.0 - Advanced Testing Techniques in Life Insurance - A New Quality in Mathematical Testing of Portfolio Management Systems in Life Insurance”, from Testing Experience, March 2010
-
2007 The Development of a Thorough Test Plan in the Analysis Phase leading to more Successful Software Development Projects Jim Nindel-Edwards Seattle Pacific University Gerhard Steinke Seattle Pacific University
-
Garousi, V., Felderer, M., & Hacaloğlu, T. (2017). Software test maturity assessment and test process
-
Improvement: A multivocal literature review. Information and Software Technology, 85, 16-42.
ANY QUESTIONS?