Software Testing Strategies and Software Metrics

What is the primary objective of System Testing

System testing evaluates an integrated, complete software application to verify that it complies fully with specified business and technical requirements, testing end-to-end user workflows, performance, and security.

Acceptance Testing and User Acceptance Testing (UAT)

Formal testing conducted by the client or end-users to determine whether the system satisfies business acceptance criteria and is approved for production deployment.

Alpha Testing and Beta Testing

• Alpha Testing: Conducted at the developer's site by internal employees in a controlled staging environment.
• Beta Testing: Conducted at end-users' external sites in uncontrolled, real-world operating environments without developer supervision.

What is Regression Testing and why is it essential

Selective re-testing of an existing, previously operational system to confirm that recent code modifications, bug fixes, or enhancements have not introduced new unintended defects or broken existing functionality.

What is Smoke Testing (Build Verification Testing)

A shallow, wide set of automated preliminary tests run on every new build to verify that vital core features function before deploying the build to the QA team for comprehensive testing.

What is Sanity Testing

A very narrow, focused test performed after receiving a quick bug fix build to verify that the specific reported defect has been resolved without performing a full regression cycle.

Performance Testing, Load Testing, and Stress Testing

• Performance Testing: Evaluates speed, responsiveness, and stability under specified workloads.
• Load Testing: Evaluates system behavior under anticipated normal and peak user traffic volumes.
• Stress Testing: Pushes the system beyond maximum operational capacity to evaluate breaking points and recovery behavior.

What is Agile Testing and the 'Shift-Left' approach

Agile testing integrates testing continuously throughout the sprint lifecycle rather than as a separate post-coding phase. Shift-Left moves testing activities earlier in the development cycle (requirements, architecture) to prevent defects early.

Software Metrics and their purpose in QA management

Quantitative measures of the degree to which a software system, process, or component possesses a given attribute (e.g., defect density, code coverage, test execution velocity), used for estimation, tracking, and process improvement.

Defect Density and state its mathematical formula

$Defect Density = \frac{Total Number of Confirmed Defects}{Size of Software (KLOC or Function Points)}$

The Software Testing Strategy Pyramid (Mike Cohn)

The Ideal Testing Pyramid

• Unit Tests (Base): Thousands of tests executing in seconds; pinpoints root-cause bugs directly at function/class level.
• Service/API Layer (Middle): Verifies business logic and contract interactions between components without UI latency.
• UI / E2E Layer (Apex): Validates critical business journeys from a user perspective. Kept lean to minimize maintenance fragility and execution time.

Compare Smoke Testing, Sanity Testing, and Regression Testing

The Halstead Software Science Metrics

Maurice Halstead's Algorithmic Metrics (1977)

Based on four primitive counts:

• \eta_1: Number of distinct operators (keywords, arithmetic, logic operators).
• \eta_2: Number of distinct operands (variables, constants, identifiers).
• N_1: Total occurrences of operators.
• N_2: Total occurrences of operands.

Derived Metrics:

1. Program Vocabulary: \eta = \eta_1 + \eta_2
2. Program Length: N = N_1 + N_2
3. Volume: V = N × \log_2(\eta) (Information units to represent algorithm)
4. Difficulty: D = ≤ft(\frac{\eta_1}{2}\right) × ≤ft(\frac{N_2}{\eta_2}\right)
5. Effort: E = D × V (Mental effort required to develop or understand)

Detail Regression Test Selection and Test Case Prioritization Strategies

Regression Suite Optimization Techniques

Running 10,000 regression tests on every commit is computationally infeasible. Strategies include:

1. Test Suite Minimization: Eliminating redundant test cases using integer linear programming or greedy set-covering algorithms while retaining 100% requirements coverage.
2. Regression Test Selection (RTS): Dynamically identifying and executing only the subset of tests whose code coverage intersects with git diff modifications.
3. Test Case Prioritization (TCP): Ordering test cases to maximize the Average Percentage of Faults Detected (APFD) metric, running high-risk, recently failed, or critical business path tests first.

Agile Testing Practices: Test-Driven Development (TDD) and Behavior-Driven Development (BDD)

Modern Agile Quality Engineering

• Test-Driven Development (TDD) - Red-Green-Refactor:

1. Red: Write an automated unit test that specifies a desired feature. Run test; observe it fail.
2. Green: Write the minimal production code necessary to pass the test.
3. Refactor: Clean code, eliminate duplication, optimize structure while keeping tests green.

• Behavior-Driven Development (BDD): Bridges business stakeholders and engineers using natural language domain specifications (Gherkin syntax: Given, When, Then), executed via tools like Cucumber.

Provide an In-Depth Analysis of Software Reliability Models (SRGM), Calculating Mean Time Between Failures (MTBF) and Availability Metrics

Software Reliability Engineering

1. Reliability Metrics Defined

• Mean Time to Failure (MTTF): Expected operating time before a system experiences a failure.
• Mean Time to Repair (MTTR): Average time required to diagnose, patch, and restore a failed system to operation.
• Mean Time Between Failures (MTBF): Total average time between consecutive system failures:

$MTBF = MTTF + MTTR$

2. Steady-State Availability Formulation

$Availability A = \frac{MTTF}{MTTF + MTTR} = \frac{MTTF}{MTBF}$

• To achieve high availability (Five Nines): Either drastically increase MTTF (via rigorous testing) or minimize MTTR to seconds (via automated Kubernetes pod restarts and automated failovers).

3. Jelinski-Moranda Software Reliability Growth Model (SRGM)

Assumes N initial faults in software. The failure rate after i-1 faults have been discovered and removed is proportional to remaining faults: $\lambda(t) = \phi (N - i + 1) where \phi$ is the fault exposure rate. Enables statistical forecasting of the testing time required to achieve target reliability before product release.

Design and Architect an Enterprise Continuous Testing Framework Embedded in a GitOps CI/CD Pipeline

Enterprise Continuous Testing Blueprint

Code Complexity Metrics: Comprehensive Treatment of Halstead Metrics, McCabe Cyclomatic Complexity, and Object-Oriented Metrics (Chidamber & Kemerer)

Comprehensive Software Complexity Framework

1. Chidamber & Kemerer (C&K) Object-Oriented Metrics Suite

1. WMC (Weighted Methods per Class): Sum of cyclomatic complexity of all methods in a class. High WMC indicates a 'God Object' violating Single Responsibility.
2. DIT (Depth of Inheritance Tree): Maximum length from node to root. Deep trees (> 5) increase inheritance complexity and defect proneness.
3. NOC (Number of Children): Number of direct subclasses inheriting from a class.
4. CBO (Coupling Between Object Classes): Count of other classes to which a class is coupled. High CBO degrades modularity and test isolation.
5. LCOM (Lack of Cohesion in Methods): Measures how closely related methods of a class are to its instance variables. High LCOM indicates classes that should be split.

2. Synthesizing Metrics for Defect Prediction

Machine learning defect prediction models correlate high V(G), high CBO, and low LCOM with historical defect density, automatically directing QA test coverage towards high-risk modules.

Examiners frequently evaluate: Testing Pyramid (5 marks), Smoke vs Sanity vs Regression (5/10 marks), MTBF/MTTF formulas (5 marks), and Halstead metrics (10 marks).