headscale/DEPLOYMENT_GUIDE.md

OIDC Role Mapping Deployment Guide

This guide provides step-by-step instructions for deploying the OIDC role mapping functionality to production environments.

Pre-Deployment Checklist

Prerequisites

• Headscale 0.23.0+ installation
• Headplane 0.6.1+ installation
• OIDC provider with group claims support
• Database backup procedures in place
• Monitoring and alerting configured

Compatibility Matrix

Component	Minimum Version	Recommended Version
Headscale	0.23.0	Latest stable
Headplane	0.6.1	Latest stable
Go	1.21+	1.22+
Node.js	18+	22+

Phase 1: OIDC Provider Configuration

Keycloak Setup

# 1. Create new client for Headscale
Client ID: headscale-client
Client Protocol: openid-connect
Access Type: confidential
Valid Redirect URIs: https://your-headscale.com/oidc/callback

# 2. Configure group mapper
Name: groups
Mapper Type: Group Membership
Token Claim Name: groups
Add to ID token: ON
Add to access token: ON
Add to userinfo: ON

Azure AD Setup

# 1. App Registration
Name: Headscale OIDC
Redirect URI: https://your-headscale.com/oidc/callback
ID tokens: Enabled

# 2. API Permissions
Microsoft Graph > GroupMember.Read.All
Microsoft Graph > User.Read

# 3. Token Configuration
Add groups claim to ID tokens and access tokens

Okta Setup

# 1. Application Creation
Application Type: Web Application
Grant Types: Authorization Code
Redirect URIs: https://your-headscale.com/oidc/callback

# 2. Claims Configuration
Add "groups" claim to ID token and access token
Filter: Regex .*

Phase 2: Headscale Deployment

1. Database Backup

# SQLite backup
cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup.$(date +%Y%m%d)

# PostgreSQL backup
pg_dump headscale > headscale_backup_$(date +%Y%m%d).sql

2. Build with Groups Support

# Clone latest code with OIDC role mapping
git clone https://github.com/juanfont/headscale.git
cd headscale

# Apply the OIDC role mapping patches
# (These would be your committed changes)

# Build binary
go build -o headscale ./cmd/headscale

3. Configuration Update

# /etc/headscale/config.yaml
oidc:
  issuer: "https://your-provider.com/realm"
  client_id: "headscale-client"
  client_secret: "your-secret"
  scope: ["openid", "profile", "email", "groups"]  # Add groups scope
  extra_params: {}
  allowed_domains: []
  allowed_groups: []  # Leave empty to allow all groups
  allowed_users: []
  expiry: 180d
  use_expiry_from_token: false

4. Service Deployment

# Stop service
sudo systemctl stop headscale

# Replace binary
sudo cp headscale /usr/local/bin/headscale
sudo chmod +x /usr/local/bin/headscale

# Run migration (automatic on startup)
sudo systemctl start headscale

# Verify migration completed
sudo journalctl -u headscale -f | grep "migration"

5. Verification

# Check database schema
sqlite3 /var/lib/headscale/db.sqlite ".schema users"
# Should show 'groups' column

# Test OIDC login
headscale users list
# Test with OIDC user login and verify groups are stored

Phase 3: Headplane Deployment

1. Database Migration

# For development/testing
pnpm drizzle-kit push

# For production - apply migration manually
sqlite3 /path/to/headplane.db < drizzle/0003_add_groups_column.sql

2. Configuration Update

# config.yaml
oidc:
  enabled: true
  issuer_url: "https://your-provider.com/realm"
  client_id: "headplane-client"
  client_secret: "headplane-secret"
  scope: "openid profile email groups"  # Add groups scope
  redirect_uri: "https://your-headplane.com/admin/oidc/callback"

# Optional: Custom role mappings
role_mapping:
  owner: ["company-owners", "cto-group"]
  admin: ["it-admins", "platform-team"]
  network_admin: ["network-team", "devops"]
  it_admin: ["helpdesk", "support-team"]
  auditor: ["compliance", "security-team"]

3. Build and Deploy

# Build with role mapping support
pnpm build

# Deploy (method depends on your setup)
# Docker example:
docker build -t headplane:latest .
docker stop headplane
docker run -d --name headplane headplane:latest

4. Verification

# Test role mapping
curl -s "http://localhost:3000/admin" | grep -i "login"

# Check logs for role assignments
docker logs headplane | grep "role.*mapping"

Phase 4: Testing & Validation

Automated Tests

# Run comprehensive test suite
cd docker-dev
docker compose -f docker-compose-oidc-test.yml up -d
./test-oidc-roles.sh

Manual Testing Checklist

• Owner login assigns owner role with full capabilities
• Admin login assigns admin role with administrative access
• Network admin gets network-specific permissions
• Auditor gets read-only access
• Regular member gets zero capabilities (no UI access)
• Group changes in OIDC provider reflect in next login
• Existing CLI users continue to work normally

Monitoring Setup

# Add alerts for authentication failures
# Monitor user login patterns
# Track role assignment distributions
# Alert on unexpected privilege escalations

Rollback Procedures

Emergency Rollback

# 1. Stop services
sudo systemctl stop headscale headplane

# 2. Restore Headscale
sudo cp /usr/local/bin/headscale.backup /usr/local/bin/headscale
cp /var/lib/headscale/db.sqlite.backup.* /var/lib/headscale/db.sqlite

# 3. Revert Headplane
docker run -d --name headplane headplane:previous-version

# 4. Update configs to remove groups scope
# Edit config files to remove "groups" from OIDC scope

# 5. Restart services
sudo systemctl start headscale

Staged Rollback (Recommended)

# 1. Disable OIDC role mapping in config
role_mapping:
  enabled: false  # Add this feature flag

# 2. All users get default 'member' role
# 3. Manually assign roles as needed
# 4. Plan proper rollback during maintenance window

Production Considerations

Security

• Use strong client secrets for OIDC clients
• Implement proper certificate management for HTTPS
• Regular security updates for all components
• Monitor for suspicious role assignments

Performance

• Groups are cached in database to minimize OIDC calls
• Consider connection pooling for high-traffic environments
• Monitor database performance with new queries

Backup Strategy

# Automated backup script
#!/bin/bash
DATE=$(date +%Y%m%d_%H%M%S)

# Headscale backup
cp /var/lib/headscale/db.sqlite /backups/headscale_${DATE}.sqlite

# Headplane backup  
sqlite3 /var/lib/headplane/db.sqlite ".backup /backups/headplane_${DATE}.sqlite"

# Retain 30 days of backups
find /backups -name "*.sqlite" -mtime +30 -delete

Monitoring Metrics

• OIDC authentication success/failure rates
• Role assignment distribution
• Group membership changes
• API call patterns by role
• Database query performance

Troubleshooting

Common Issues

Groups not appearing in tokens

# Check OIDC provider group mapping configuration
# Verify groups scope is included in request
# Test with OIDC debugging tools

Wrong role assignments

# Check group name mapping in Headplane config
# Verify case sensitivity and exact matches
# Review role hierarchy logic

Database migration failures

# Check disk space and permissions
# Verify database isn't locked by other processes
# Review migration logs for specific errors

Performance issues

# Monitor database query performance
# Check for proper indexing on groups column
# Review OIDC response times

This deployment guide ensures a smooth transition to OIDC role mapping while maintaining system stability and security.