headplane/TESTING_PLAN.md

OIDC Improvements Testing Plan

Testing Philosophy

Our improvements follow "convention over configuration" - we need tests that verify:

1. Smart defaults work automatically (zero config scenarios)
2. Explicit configuration overrides defaults (progressive enhancement)
3. Error cases provide helpful guidance (user-friendly failures)
4. Edge cases are handled gracefully (production resilience)

Test Priority Matrix

🔴 HIGH PRIORITY (Must Have)

These tests cover the core functionality users will depend on daily.

1. Group Discovery (app/utils/oidc.ts)

describe('Smart Group Discovery', () => {
  test('finds groups in userInfo.groups (priority 1)')
  test('falls back to claims.groups when userInfo missing')
  test('handles Keycloak resource_access.headplane.roles')
  test('handles Azure cognito:groups')
  test('handles nested paths correctly')
  test('filters out non-string values')
  test('returns empty array when no groups found')
  test('handles malformed/null data gracefully')
})

2. Role Mapping (app/server/web/roles.ts)

describe('Environment-First Role Mapping', () => {
  test('uses environment variables when configured')
  test('falls back to convention-based mapping')
  test('respects role hierarchy (owner > admin > etc.)')
  test('handles case-insensitive group matching')
  test('returns member for unrecognized groups')
  test('handles empty/null groups array')
})

describe('Convention-Based Role Assignment', () => {
  test('ceo/cto/founder → owner')
  test('admin/administrator → admin') 
  test('devops/sre/network → network_admin')
  test('helpdesk/support → it_admin')
  test('audit/compliance → auditor')
  test('random-group → member')
  test('multiple groups picks highest privilege')
})

3. Configuration Enhancement (app/server/config/oidc-enhancer.ts)

describe('Configuration Self-Healing', () => {
  test('auto-adds groups scope when missing')
  test('detects optimal scope for known providers')
  test('generates redirect_uri from environment')
  test('provides provider-specific insights')
  test('validates configuration correctly')
  test('handles unknown providers gracefully')
})

🟡 MEDIUM PRIORITY (Should Have)

These tests cover important edge cases and integration scenarios.

4. Environment Variable Parsing

describe('Environment Variable Role Mapping', () => {
  test('parses comma-separated group lists')
  test('handles whitespace in group names')
  test('ignores empty environment variables')
  test('overrides work correctly')
})

5. Error Handling Enhancement

describe('Enhanced Error Messages', () => {
  test('provides helpful suggestions for invalid_grant')
  test('guides users on scope configuration issues')
  test('logs authentication success with details')
  test('suggests environment variables for missing groups')
})

🟢 LOW PRIORITY (Nice to Have)

These tests cover advanced scenarios and comprehensive edge cases.

6. Provider-Specific Optimizations

describe('Provider Detection', () => {
  test('detects Google Workspace correctly')
  test('detects Azure AD correctly')
  test('detects Keycloak correctly')
  test('provides appropriate scope for each')
  test('gives relevant setup tips')
})

7. Integration Scenarios

describe('Full Authentication Flow', () => {
  test('complete authentication with role assignment')
  test('database storage of groups and capabilities')
  test('first user becomes owner regardless of groups')
  test('subsequent users follow role mapping')
})

Test Implementation Strategy

Phase 1: Core Unit Tests (Required for PR)

Focus on the business logic that users depend on:

1. Group discovery with all fallback paths
2. Role mapping with environment variables and conventions
3. Configuration enhancement and validation

Phase 2: Integration Tests (Follow-up PR)

Focus on the full user experience:

1. End-to-end authentication flow
2. Database integration with role assignment
3. Real provider configuration testing

Phase 3: Comprehensive Coverage (Future Enhancement)

Focus on edge cases and advanced scenarios:

1. Provider-specific testing with real examples
2. Performance testing with large group lists
3. Security testing for malicious input

Mock Data Strategy

Realistic Test Data

Use actual examples from popular identity providers:

// Google Workspace
const googleClaims = {
  iss: 'https://accounts.google.com',
  email: 'alice@company.com',
  // Google doesn't provide groups in standard claims
}

// Azure AD
const azureClaims = {
  groups: ['IT Administrators', 'Company Employees'],
  roles: ['Application.Admin']
}

// Keycloak
const keycloakClaims = {
  resource_access: {
    headplane: { roles: ['admin', 'developer'] }
  },
  realm_access: { roles: ['user'] }
}

// AWS Cognito
const cognitoClaims = {
  'cognito:groups': ['admins', 'users']
}

Edge Case Data

Test boundary conditions and error scenarios:

const edgeCases = {
  emptyGroups: { groups: [] },
  nullGroups: { groups: null },
  stringGroups: { groups: "not-an-array" },
  mixedTypes: { groups: ['admin', 123, null, 'user'] },
  malformedNested: { resource_access: 'not-an-object' },
  deeplyNested: { a: { b: { c: { groups: ['deep-admin'] } } } }
}

Success Criteria

For Production Readiness

• ✅ 90%+ test coverage of new OIDC functionality
• ✅ All happy path scenarios work correctly
• ✅ Common error cases provide helpful guidance
• ✅ Edge cases don't crash the application

For Upstream PR

• ✅ Core business logic thoroughly tested
• ✅ Environment variable integration verified
• ✅ Configuration enhancement validated
• ✅ Backward compatibility confirmed

For User Confidence

• ✅ Real-world scenarios covered with actual provider data
• ✅ Migration scenarios tested (existing → enhanced)
• ✅ Deployment scenarios verified (env vars, containers)
• ✅ Documentation includes testing examples

Testing Tools & Patterns

Test Structure

// Arrange - Set up test data
const mockClaims = { groups: ['admin', 'users'] }
const mockUserInfo = { email: 'test@example.com' }

// Act - Execute the function
const result = extractGroups(mockClaims, mockUserInfo)

// Assert - Verify expected behavior
expect(result).toEqual(['admin', 'users'])

Environment Variable Testing

beforeEach(() => {
  // Clear environment variables
  delete process.env.HEADPLANE_ADMIN_GROUPS
})

afterEach(() => {
  // Restore original environment
  Object.assign(process.env, originalEnv)
})

Provider-Specific Testing

describe.each([
  ['Google', googleTestData],
  ['Azure', azureTestData], 
  ['Keycloak', keycloakTestData]
])('%s Provider', (providerName, testData) => {
  test('detects groups correctly', () => {
    // Provider-specific test logic
  })
})

This testing plan ensures our improvements are production-ready while maintaining the simplicity and reliability that makes them valuable.