headscale/DEPLOYMENT_GUIDE.md

# OIDC Role Mapping Deployment Guide

This guide provides step-by-step instructions for deploying the OIDC role mapping functionality to production environments.

## Pre-Deployment Checklist

### Prerequisites
- [ ] Headscale 0.23.0+ installation
- [ ] Headplane 0.6.1+ installation  
- [ ] OIDC provider with group claims support
- [ ] Database backup procedures in place
- [ ] Monitoring and alerting configured

### Compatibility Matrix
| Component | Minimum Version | Recommended Version |
|-----------|----------------|-------------------|
| Headscale | 0.23.0 | Latest stable |
| Headplane | 0.6.1 | Latest stable |
| Go | 1.21+ | 1.22+ |
| Node.js | 18+ | 22+ |

## Phase 1: OIDC Provider Configuration

### Keycloak Setup
```bash
# 1. Create new client for Headscale
Client ID: headscale-client
Client Protocol: openid-connect
Access Type: confidential
Valid Redirect URIs: https://your-headscale.com/oidc/callback

# 2. Configure group mapper
Name: groups
Mapper Type: Group Membership
Token Claim Name: groups
Add to ID token: ON
Add to access token: ON
Add to userinfo: ON
```

### Azure AD Setup
```bash
# 1. App Registration
Name: Headscale OIDC
Redirect URI: https://your-headscale.com/oidc/callback
ID tokens: Enabled

# 2. API Permissions
Microsoft Graph > GroupMember.Read.All
Microsoft Graph > User.Read

# 3. Token Configuration
Add groups claim to ID tokens and access tokens
```

### Okta Setup
```bash
# 1. Application Creation
Application Type: Web Application
Grant Types: Authorization Code
Redirect URIs: https://your-headscale.com/oidc/callback

# 2. Claims Configuration
Add "groups" claim to ID token and access token
Filter: Regex .*
```

## Phase 2: Headscale Deployment

### 1. Database Backup
```bash
# SQLite backup
cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup.$(date +%Y%m%d)

# PostgreSQL backup
pg_dump headscale > headscale_backup_$(date +%Y%m%d).sql
```

### 2. Build with Groups Support
```bash
# Clone latest code with OIDC role mapping
git clone https://github.com/juanfont/headscale.git
cd headscale

# Apply the OIDC role mapping patches
# (These would be your committed changes)

# Build binary
go build -o headscale ./cmd/headscale
```

### 3. Configuration Update
```yaml
# /etc/headscale/config.yaml
oidc:
  issuer: "https://your-provider.com/realm"
  client_id: "headscale-client"
  client_secret: "your-secret"
  scope: ["openid", "profile", "email", "groups"]  # Add groups scope
  extra_params: {}
  allowed_domains: []
  allowed_groups: []  # Leave empty to allow all groups
  allowed_users: []
  expiry: 180d
  use_expiry_from_token: false
```

### 4. Service Deployment
```bash
# Stop service
sudo systemctl stop headscale

# Replace binary
sudo cp headscale /usr/local/bin/headscale
sudo chmod +x /usr/local/bin/headscale

# Run migration (automatic on startup)
sudo systemctl start headscale

# Verify migration completed
sudo journalctl -u headscale -f | grep "migration"
```

### 5. Verification
```bash
# Check database schema
sqlite3 /var/lib/headscale/db.sqlite ".schema users"
# Should show 'groups' column

# Test OIDC login
headscale users list
# Test with OIDC user login and verify groups are stored
```

## Phase 3: Headplane Deployment

### 1. Database Migration
```bash
# For development/testing
pnpm drizzle-kit push

# For production - apply migration manually
sqlite3 /path/to/headplane.db < drizzle/0003_add_groups_column.sql
```

### 2. Configuration Update
```yaml
# config.yaml
oidc:
  enabled: true
  issuer_url: "https://your-provider.com/realm"
  client_id: "headplane-client"
  client_secret: "headplane-secret"
  scope: "openid profile email groups"  # Add groups scope
  redirect_uri: "https://your-headplane.com/admin/oidc/callback"

# Optional: Custom role mappings
role_mapping:
  owner: ["company-owners", "cto-group"]
  admin: ["it-admins", "platform-team"]
  network_admin: ["network-team", "devops"]
  it_admin: ["helpdesk", "support-team"]
  auditor: ["compliance", "security-team"]
```

### 3. Build and Deploy
```bash
# Build with role mapping support
pnpm build

# Deploy (method depends on your setup)
# Docker example:
docker build -t headplane:latest .
docker stop headplane
docker run -d --name headplane headplane:latest
```

### 4. Verification
```bash
# Test role mapping
curl -s "http://localhost:3000/admin" | grep -i "login"

# Check logs for role assignments
docker logs headplane | grep "role.*mapping"
```

## Phase 4: Testing & Validation

### Automated Tests
```bash
# Run comprehensive test suite
cd docker-dev
docker compose -f docker-compose-oidc-test.yml up -d
./test-oidc-roles.sh
```

### Manual Testing Checklist
- [ ] Owner login assigns owner role with full capabilities
- [ ] Admin login assigns admin role with administrative access
- [ ] Network admin gets network-specific permissions
- [ ] Auditor gets read-only access
- [ ] Regular member gets zero capabilities (no UI access)
- [ ] Group changes in OIDC provider reflect in next login
- [ ] Existing CLI users continue to work normally

### Monitoring Setup
```bash
# Add alerts for authentication failures
# Monitor user login patterns
# Track role assignment distributions
# Alert on unexpected privilege escalations
```

## Rollback Procedures

### Emergency Rollback
```bash
# 1. Stop services
sudo systemctl stop headscale headplane

# 2. Restore Headscale
sudo cp /usr/local/bin/headscale.backup /usr/local/bin/headscale
cp /var/lib/headscale/db.sqlite.backup.* /var/lib/headscale/db.sqlite

# 3. Revert Headplane
docker run -d --name headplane headplane:previous-version

# 4. Update configs to remove groups scope
# Edit config files to remove "groups" from OIDC scope

# 5. Restart services
sudo systemctl start headscale
```

### Staged Rollback (Recommended)
```bash
# 1. Disable OIDC role mapping in config
role_mapping:
  enabled: false  # Add this feature flag

# 2. All users get default 'member' role
# 3. Manually assign roles as needed
# 4. Plan proper rollback during maintenance window
```

## Production Considerations

### Security
- Use strong client secrets for OIDC clients
- Implement proper certificate management for HTTPS
- Regular security updates for all components
- Monitor for suspicious role assignments

### Performance
- Groups are cached in database to minimize OIDC calls
- Consider connection pooling for high-traffic environments
- Monitor database performance with new queries

### Backup Strategy
```bash
# Automated backup script
#!/bin/bash
DATE=$(date +%Y%m%d_%H%M%S)

# Headscale backup
cp /var/lib/headscale/db.sqlite /backups/headscale_${DATE}.sqlite

# Headplane backup  
sqlite3 /var/lib/headplane/db.sqlite ".backup /backups/headplane_${DATE}.sqlite"

# Retain 30 days of backups
find /backups -name "*.sqlite" -mtime +30 -delete
```

### Monitoring Metrics
- OIDC authentication success/failure rates
- Role assignment distribution
- Group membership changes
- API call patterns by role
- Database query performance

## Troubleshooting

### Common Issues

**Groups not appearing in tokens**
```bash
# Check OIDC provider group mapping configuration
# Verify groups scope is included in request
# Test with OIDC debugging tools
```

**Wrong role assignments**
```bash
# Check group name mapping in Headplane config
# Verify case sensitivity and exact matches
# Review role hierarchy logic
```

**Database migration failures**
```bash
# Check disk space and permissions
# Verify database isn't locked by other processes
# Review migration logs for specific errors
```

**Performance issues**
```bash
# Monitor database query performance
# Check for proper indexing on groups column
# Review OIDC response times
```

This deployment guide ensures a smooth transition to OIDC role mapping while maintaining system stability and security.