Testing Strategy

Testing Strategy - Steps 3-7/9

Overview

This document provides a comprehensive overview of the testing strategy for the Burglary Support Network application, covering all testing types, workflows, best practices, and guidelines.

Date: 2025-10-30 Status: ✅ Complete Total Automated Tests: 650+ Test Types: 5 (Unit, E2E, Visual, Performance, Accessibility)

Testing Philosophy

Core Principles

Testing Pyramid - More unit tests, fewer E2E tests
Test What Matters - Focus on user value and critical paths
Fast Feedback - Quick test execution in development
Prevent Regressions - Catch bugs before production
Continuous Quality - Automated testing in CI/CD
Maintainable Tests - Clear, readable, reusable test code

Quality Goals

Functionality - Features work as expected
Performance - Fast, responsive application
Accessibility - WCAG 2.1 AA compliance
Visual Consistency - UI matches designs
Security - No vulnerabilities in dependencies

Testing Types Overview

1. Unit Tests ⚡

Purpose: Test individual components, functions, and utilities in isolation

Framework: Vitest + React Testing Library

Coverage:

Components: 90% statements, 80% branches
Hooks: 85% statements, 75% branches
Utilities: 95% statements, 85% branches
Contexts: 90% statements, 85% branches

When to Use:

Testing component logic
Testing utility functions
Testing custom hooks
Testing state management

Example:

test('calculateTotal should sum item prices', () => {
  const items = [
    { price: 10, quantity: 2 },
    { price: 5, quantity: 1 },
  ];
  expect(calculateTotal(items)).toBe(25);
});

Run Commands:

make test              # Run all unit tests
make test-watch        # Watch mode (recommended for development)
npm run test           # Direct command

2. E2E Tests 🔄

Purpose: Test complete user flows and interactions from end to end

Framework: Playwright

Coverage: 391 tests across 26 files

Authentication flows
Incident reporting
Community features (Q&A, mentorship, stories)
Security education
Insurance claims
Recovery tracking
Shopping cart
Navigation

When to Use:

Testing user workflows
Testing page interactions
Testing form submissions
Testing navigation
Testing authentication

Example:

test('user should be able to report incident', async ({ page }) => {
  await registerAndLoginUser(page, generateTestUser());
  await page.goto('/report-incident');
  await fillIncidentForm(page, generateIncidentData());
  await page.getByRole('button', { name: /submit/i }).click();
  await expect(page.getByText(/success/i)).toBeVisible();
});

Run Commands:

make test-e2e          # All E2E tests
make test-e2e-chrome   # Chrome only
make test-e2e-ui       # With Playwright UI
make test-e2e-debug    # Debug mode

3. Visual Regression Tests 🎨

Purpose: Detect unintended UI changes by comparing screenshots

Framework: Playwright native screenshot comparison

Coverage: 25+ screenshots

Public pages (7)
Auth pages (2)
Education pages (4)
Components (3)
Responsive designs (4)
Error states (1)
Dark mode (1)

When to Use:

After CSS changes
After layout modifications
After design updates
Before major releases

Example:

test('homepage should match baseline', async ({ page }) => {
  await page.goto('/');
  await waitForPageStable(page);
  await expect(page).toHaveScreenshot('homepage.png', {
    fullPage: true,
  });
});

Run Commands:

make test-visual        # Run visual tests
make test-visual-update # Update baselines

4. Performance Tests ⚡

Purpose: Ensure application meets performance standards and budgets

Tools: Lighthouse CI + Playwright Performance API

Coverage: 20+ tests + 7 pages

Page load times
Core Web Vitals (FCP, LCP, CLS, TBT, TTI)
Resource budgets (JS, CSS, images)
Network optimization
Rendering performance

When to Use:

Before major releases
After adding large dependencies
After bundling changes
For performance optimization

Example:

test('homepage should load within budget', async ({ page }) => {
  await page.goto('/');
  const metrics = await collectPerformanceMetrics(page);
  expect(metrics.navigation.loadComplete).toBeLessThan(3000);
  expect(metrics.resources.jsSize).toBeLessThan(500 * 1024);
});

Run Commands:

make test-perf              # Playwright performance tests
make test-perf-lighthouse   # Lighthouse CI audit
make test-perf-all          # Both

5. Accessibility Tests ♿

Purpose: Ensure WCAG 2.1 AA compliance and screen reader compatibility

Framework: Playwright + axe-core

Coverage: 22 tests

Color contrast
Keyboard navigation
ARIA labels
Focus management
Screen reader support
Semantic HTML

When to Use:

After UI changes
Before major releases
For new features
Regulatory compliance

Example:

test('homepage should have no accessibility violations', async ({ page }) => {
  await page.goto('/');
  const results = await new AxeBuilder({ page }).analyze();
  expect(results.violations).toEqual([]);
});

Run Commands:

# Accessibility tests run as part of main CI
npm run test:e2e accessibility

Test Pyramid

          /\
         /  \        E2E Tests (391)
        /    \       Visual Tests (25+)
       /      \      Performance Tests (20+)
      /        \     Accessibility Tests (22)
     /          \
    /____________\   Unit Tests (~200+)

Distribution:

Unit Tests: ~30% of total (but fastest)
E2E Tests: ~60% of total (comprehensive coverage)
Visual Tests: ~4% of total (UI validation)
Performance Tests: ~3% of total (budgets)
Accessibility Tests: ~3% of total (compliance)

CI/CD Integration

Workflow Triggers

Workflow	Push (main)	Push (develop)	Pull Request	Scheduled	Manual
Main CI	✅	❌	✅	❌	❌
E2E Tests	✅	✅	✅	❌	✅
Visual Regression	✅	✅	✅	❌	✅
Performance	✅	❌	✅	Daily 2AM	✅
Accessibility	✅	✅	✅	❌	❌

Main CI Pipeline

File: .github/workflows/ci.yml

Steps:

Install dependencies
Security audit
Format check
Lint
Build
Unit tests (with coverage)
E2E tests (Chromium)
Upload artifacts

Duration: ~20-25 minutes

Test Execution Order

Parallel Jobs:

┌─────────────┐  ┌─────────────┐  ┌─────────────┐
│   Main CI   │  │  E2E Tests  │  │   Visual    │
│  (Chromium) │  │ (Multi-Br)  │  │ Regression  │
└─────────────┘  └─────────────┘  └─────────────┘
        │                │                │
        └────────────────┴────────────────┘
                         │
                   ┌─────────────┐
                   │ Performance │
                   │   (Daily)   │
                   └─────────────┘

Artifact Management

Artifact Type	Retention	Size	Purpose
Test Results	30 days	~10MB	Debugging, analysis
HTML Reports	30 days	~50MB	Visual inspection
Allure Reports	30 days	~20MB	Advanced analytics
Failure Videos	7 days	~100MB	Debugging failures
Visual Diffs	14 days	~30MB	UI change review
Lighthouse Reports	30 days	~5MB	Performance analysis

Testing Workflow

Development Workflow

Developer Flow:
1. Write code
2. Run unit tests (make test-watch)
3. Run affected E2E tests locally
4. Commit changes
5. Push to branch
6. CI runs all tests
7. Review test results
8. Fix any failures
9. Merge when all tests pass

Feature Development

New Feature:
1. Write unit tests (TDD approach)
2. Implement feature
3. Add E2E tests for user flows
4. Add visual tests if UI changes
5. Update baselines if needed
6. Run performance tests
7. Check accessibility
8. Submit PR

Bug Fix Workflow

Bug Fix:
1. Write failing test that reproduces bug
2. Fix the bug
3. Verify test passes
4. Run related E2E tests
5. Check for visual regressions
6. Submit PR

Testing Best Practices

General Principles

Test Behavior, Not Implementation
- Focus on what the user experiences
- Avoid testing internal state directly
- Test public APIs
Keep Tests Independent
- Each test should run in isolation
- No shared state between tests
- Use unique test data

Use Descriptive Test Names

// Good
test('should show error when email is invalid');

// Bad
test('test1');

Follow AAA Pattern
- Arrange: Set up test data
- Act: Execute the action
- Assert: Verify the result
Use Test Helpers
- Create reusable helper functions
- Use test data generators
- Share common setup/teardown

Unit Testing Best Practices

✅ Do:

Test edge cases
Mock external dependencies
Test error handling
Keep tests fast (<100ms)
Use meaningful assertions

❌ Don’t:

Test third-party libraries
Test implementation details
Create brittle tests
Share mutable state
Test multiple things in one test

E2E Testing Best Practices

✅ Do:

Test critical user paths
Use proper waits (networkidle, element visibility)
Generate unique test data
Clean up test data
Use page object models

❌ Don’t:

Test every edge case (unit tests cover those)
Use hard-coded timeouts
Rely on test execution order
Share user accounts
Skip error handling

Visual Testing Best Practices

✅ Do:

Mask dynamic content (dates, IDs)
Wait for page stability
Use consistent viewport sizes
Review baselines before committing
Update baselines intentionally

❌ Don’t:

Screenshot during loading states
Include live data feeds
Commit baselines without review
Test animations (disable them)
Update baselines to “fix” tests

Performance Testing Best Practices

✅ Do:

Set realistic budgets
Test production builds
Use throttling
Monitor trends over time
Fix regressions early

❌ Don’t:

Test development builds
Ignore warnings
Set unrealistic budgets
Skip performance tests
Accumulate performance debt

Test Data Management

Test Data Strategies

Generated Data - Use randomBytes() for unique IDs
Factory Functions - generateTestUser(), generateIncidentData()
Test Constants - TEST_CREDENTIALS, TEST_LOCATIONS
Fixtures - Reusable test data files

Example Generators

// User generator
export function generateTestUser(): TestUser {
  return {
    email: `test-${randomBytes(8).toString('hex')}@example.com`,
    password: 'Test1234!@#$',
    name: `Test User ${Date.now()}`,
  };
}

// Incident generator
export function generateIncidentData(): IncidentData {
  return {
    date: new Date().toISOString().split('T')[0],
    postcode: 'SW1A 1AA',
    description: `Test incident ${randomBytes(4).toString('hex')}`,
  };
}

Data Cleanup

Tests should clean up after themselves
Use beforeEach and afterEach hooks
Delete test users/data in teardown
Use unique prefixes for test data

Debugging Failed Tests

Local Debugging

Unit Tests:

# Run specific test
npm run test -- -t "test name"

# Debug mode
npm run test -- --inspect-brk

# Coverage report
npm run test -- --coverage

E2E Tests:

# Run with UI (best for debugging)
make test-e2e-ui

# Run with browser visible
make test-e2e-headed

# Debug specific test
npx playwright test --debug --grep="test name"

# View trace
npx playwright show-trace trace.zip

Visual Tests:

# Compare screenshots
npx playwright test visual-regression --ui

# Update baselines
make test-visual-update

CI Debugging

Download Artifacts
- Test results (JSON)
- HTML reports
- Videos (for failures)
- Traces (for detailed inspection)
Review Logs
- GitHub Actions logs
- Console output
- Error messages
Reproduce Locally
- Check out the PR branch
- Run tests locally
- Use CI container if needed

Common Issues

Flaky Tests:

Add proper waits
Check for race conditions
Increase timeout if needed
Fix test isolation issues

Timeout Errors:

Check server startup
Verify network requests
Increase timeout temporarily
Fix slow operations

Assertion Failures:

Review actual vs expected
Check test data
Verify page state
Update expectations if valid

Continuous Improvement

Metrics to Track

Test Coverage - Aim for 85%+
Test Execution Time - Keep under 30 minutes
Flaky Test Rate - Keep under 5%
Test Maintenance Time - Minimize
Bug Escape Rate - Track production bugs

Regular Reviews

Weekly:

Review flaky tests
Update failing tests
Check test duration

Monthly:

Review test coverage
Update test strategy
Optimize slow tests

Quarterly:

Audit test suite
Remove obsolete tests
Refactor test helpers
Update documentation

Command Reference

Quick Reference

# Development
make test-watch           # Unit tests (watch mode)
make test-e2e-ui         # E2E tests (UI mode)

# Local testing
make test                # All unit tests
make test-e2e            # All E2E tests
make test-visual         # Visual regression
make test-perf           # Performance tests

# CI simulation
CI=true make test        # Unit tests in CI mode
CI=true make test-e2e    # E2E tests in CI mode

# Maintenance
make test-visual-update  # Update visual baselines
make test-e2e-report     # View E2E report
make lint                # Run linter
make type-check          # TypeScript check

Full Command List

See make help for complete list of available commands.

Documentation Index

E2E Test Baseline - e2e-test-baseline.md
E2E CI Integration - e2e-test-ci-integration.md
E2E Verification - e2e-test-verification.md
Visual Regression - visual-regression-testing.md
Performance Testing - performance-testing.md
Testing Strategy - This document

Conclusion

The Burglary Support Network has a comprehensive, multi-layered testing strategy:

✅ 650+ Automated Tests - Extensive coverage ✅ 5 Test Types - Unit, E2E, Visual, Performance, Accessibility ✅ 5 CI/CD Workflows - Automated testing on every PR ✅ Fast Feedback - Tests run in parallel ✅ Clear Guidelines - Best practices documented ✅ Maintainable - Reusable helpers and patterns ✅ Quality Assurance - Multiple quality dimensions

This ensures high-quality, reliable, performant, and accessible software.

Status: ✅ Step 7/9 (Documentation) Complete