Skip to main content

Web based SSH Instant Access

Webshell (webshell.drheck.dev) Setup & Rollout Guide

This idocument aprovides the authoritative, end-to-end setup guide for rolling out webshell.drheck.dev, a browser-based SSH terminal service authenticated via Authentik Single Sign-On (SSO) and short-lived OpenSSH certificates signed by Smallstep (step-ca).


1. Architectural Overview

[ Browser Client ]
        │
        ▼ (HTTPS / WebSocket)
[ Nginx Reverse Proxy (/etc/nginx/sites-available/webshell) ]
        │
        ├── auth_request ──► [ Authentik Embedded Outpost ] (SSO Auth & Claims)
        │
        ▼ (HTTP / WebSocket + X-Authentik-Username Header)
[ Webshell Container (ttyd + login-wrapper.sh) on eth77 fd77:...:771d:7681 ]
        │
        ├── Step SSH Certificate Sign ──► [ Step-CA Container on fd77:...:ca:9000 ]
        │
        ▼ (SSH with Ephemeral Ed25519 Certificate over IPv6)
[ Host ul OpenSSH Daemon on [fd77:7777:5010:0:1337:f1ac:d00d:337]:22 ]

Key Components

  1. step-ca: Smallstep Certificate Authority issuing 5-minute ephemeral Ed25519 user certificates, running as unprivileged system user akadminstepca (1101:1101945:945).
  2. webshell: Debian Bookworm container running static ttyd 1.7.4, step CLI 0.25.0, and login-wrapper.sh.
  3. Authentik: Provides OIDC integration for step-ca and ForwardAuth single sign-on (webshell.drheck.dev).
  4. Nginx: Terminates TLS and proxies HTTP/WebSocket connections while passing authenticated identity headers (X-Authentik-Username).
  5. OpenSSH (sshd): Configured to trust step-ca signed user certificates for passwordless OS login on host ul.

2. Network & ULA IPv6 Allocation

Every container on host ul binds to a unique ULA IPv6 address on interface eth77 (fd77:7777:5010:0:1337:f1ac:d00d:.../64).

  • step-ca IPv6: fd77:7777:5010:0:1337:f1ac:d00d:ca (Port 9000)
  • webshell IPv6: fd77:7777:5010:0:1337:f1ac:d00d:771d (Port 7681)
  • Host ul ULA IPv6 SSH Target: fd77:7777:5010:0:1337:f1ac:d00d:337 (Port 22)

3. Step 1: Step-CA Configuration & Provisioners

A. Docker Compose & Environment

In /data/docker/webshell/docker-compose.yml, define step-ca (running as unprivileged system user 1101:1101945:945) and webshell with dual-network connectivity:connectivity, explicit ssh_known_hosts mounting, and /etc/ul_passwd validation mounting:

services:
  step-ca:
    image: smallstep/step-ca:latest
    container_name: step-ca
    restart: unless-stopped
    user: "1101:1101"945:945"
    env_file: .env
    volumes:
      - ./step-ca:/home/step:rw
    ports:
      - "[fd77:7777:5010:0:1337:f1ac:d00d:ca]:9000:9000"
    networks:
      - infra_net

  webshell:
    build: .
    container_name: webshell
    restart: unless-stopped
    environment:
      - STEP_CA_URL=https://step-ca:9000
    volumes:
      - ./login-wrapper.sh:/usr/local/bin/login-wrapper.sh:ro
      - ./known_hosts:/etc/ssh/ssh_known_hosts:ro
      - /etc/passwd:/etc/ul_passwd:ro
      - ./step-ca/certs/root_ca.crt:/etc/step-ca/root_ca.crt:ro
      - ./step-ca/secrets/webshell_svc_pwd:/etc/step-ca/webshell_svc_pwd:ro
    ports:
      - "[fd77:7777:5010:0:1337:f1ac:d00d:771d]:7681:7681"
    networks:
      - infra_net
      - webshell_net
    depends_on:
      - step-ca

networks:
  infra_net:
    external: true
  webshell_net:
    enable_ipv6: true
    ipam:
      config:
        - subnet: fd77:7777:5010:771d::/64

B. CA Directory Permissions

On host ul, restrict access to all CA keys and provisioner passwords to unprivileged system user akadmin:akadminstepca:stepca (1101:1101945:945, /usr/bin/nologin) with mode 700:

sudo chown -R 1101:1101945:945 /data/docker/webshell/step-ca
sudo chmod -R o-rwx /data/docker/webshell/step-ca
sudo chmod 700 /data/docker/webshell/step-ca/secrets /data/docker/webshell/step-ca/secrets/*

4. Step 2: Host ul OpenSSH Daemon Trust & Web Session Banner Setup

To allow sshd on host ul ([fd77:7777:5010:0:1337:f1ac:d00d:337]:22) to accept certificates signed by step-ca:, enforce strict known_hosts checking, and serve a dedicated login banner exclusively for web sessions:

  1. Extract the CA user public key:
    docker exec step-ca step certificate inspect --format json /home/step/certs/user_ca_key.pub
    
  2. Append the public key (ecdsa-sha2-nistp256 ... or ssh-ed25519 ...) to /etc/ssh/drheck_ca_user_key.pub.
  3. Verify that /etc/ssh/sshd_config.d/10-drheck-certs.conf contains:
    TrustedUserCAKeys /etc/ssh/drheck_ca_user_key.pub
    HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub
    
    Match Address fd77:7777:5010:771d::/64
        Banner /etc/ssh/webshell_banner.txt
    
Create /etc/ssh/webshell_banner.txt on host ul:
sudo touch /etc/ssh/webshell_banner.txt
sudo chmod 644 /etc/ssh/webshell_banner.txt
Generate /data/docker/webshell/known_hosts from host ul's public host key:
echo "fd77:7777:5010:0:1337:f1ac:d00d:337 $(cat /etc/ssh/ssh_host_ed25519_key.pub)" > /data/docker/webshell/known_hosts
chmod 644 /data/docker/webshell/known_hosts
Reload OpenSSH daemon:
sudo systemctl reload sshsshd

5. Step 3: Web Terminal Backend (ttyd + login-wrapper.sh) Setup

A. Dockerfile (/data/docker/webshell/Dockerfile)

Mandatory: Include -6 and -i :: in CMD so ttyd binds to IPv6 inside the container when connected to an IPv6 Docker bridge network.

FROM debian:bookworm-slim

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends     ca-certificates curl openssh-client bash procps     && rm -rf /var/lib/apt/lists/*

# Install static ttyd 1.7.4 & step CLI 0.25.0
RUN curl -sSL https://github.com/tsl0922/ttyd/releases/download/1.7.4/ttyd.x86_64 -o /usr/local/bin/ttyd     && chmod +x /usr/local/bin/ttyd     && curl -sSL https://dl.step.sm/gh-release/cli/docs-step-cli/v0.25.0/step_linux_0.25.0_amd64.tar.gz | tar -xz -C /tmp     && mv /tmp/step_0.25.0/bin/step /usr/local/bin/step     && chmod +x /usr/local/bin/step     && rm -rf /tmp/step_0.25.0

COPY login-wrapper.sh /usr/local/bin/login-wrapper.sh
RUN chmod +x /usr/local/bin/login-wrapper.sh

EXPOSE 7681
ENTRYPOINT ["/usr/local/bin/ttyd"]
CMD ["-6", "-i", "::", "-W", "-H", "X-Authentik-Username", "/usr/local/bin/login-wrapper.sh"]

B. Backend Login Wrapper (/data/docker/webshell/login-wrapper.sh)

#!/usr/bin/env bash
set -euo pipefail

# Resolve username across CGI-style environment variables exported by ttyd/proxy
TARGET_USER="${X_AUTHENTIK_USERNAME:-${TTYD_USER:-${HTTP_X_AUTHENTIK_USERNAME:-${AUTHENTIK_USERNAME:-}}}}"

if [[ -z "${TARGET_USER}" ]]; then
  echo "Authentication Error: Missing Authentik identity." >&2
  exit 1
fi

# Strict alphanumeric regex check
if [[ ! "${TARGET_USER}" =~ ^[a-z_][a-z0-9_-]{0,31}$ ]]; then
  echo "Authentication Error: Invalid username format." >&2
  exit 1
fi

# Retrieve numeric UID from host passwd database
TARGET_UID=$(awk -F: -v u="${TARGET_USER}" '$1 == u {print $3}' /etc/ul_passwd 2>/dev/null || true)
if [[ -z "${TARGET_UID}" ]]; then
  echo "Authentication Error: User does not exist on target host." >&2
  exit 1
fi

# Reject privilegedsystem accounts (UID < 1000), admin, and system OSdocker* accounts
caseif (( TARGET_UID < 1000 )) || [[ "${TARGET_USER}" in== root|admin|postgres|www-data|daemon|nobody|bin|sys|sync|games|man|lp|mail|news|uucp|proxy|backup|list|irc|gnats|systemd*"admin" ]] || [[ "${TARGET_USER}" == docker*) ]]; then
  echo "Authentication Error: Access to privileged or system account is forbidden." >&2
  exit 1
;;
esacfi

KEY_DIR="/tmp/ssh_${RANDOM}_$$"
mkdir(mktemp -md 700-t "${KEY_DIR}ssh_XXXXXX)"
KEY_PATH="${KEY_DIR}/id_ed25519"

cleanup() {
  rm -rf "${KEY_DIR}"
}
trap cleanup EXIT INT TERM

# Generate ephemeral Ed25519 keypair silently
ssh-keygen -q -t ed25519 -f "${KEY_PATH}" -N "" -q>/dev/null 2>&1

# Sign user certificate via JWK provisioner (--sign flag is mandatory for non-interactive execution)
if command -v step >/dev/null 2>&1 && [[ -n "${STEP_CA_URL:-}" ]] && [[ -f "/etc/step-ca/webshell_svc_pwd" ]]; then
  step ssh certificate "${TARGET_USER}" "${KEY_PATH}.pub"     --sign     --provisioner="webshell-svc"     --provisioner-password-file="/etc/step-ca/webshell_svc_pwd"     --ca-url="${STEP_CA_URL}"     --root="/etc/step-ca/root_ca.crt"     --not-after=5m     --force >/dev/null 2>&1 || true
fi

# Login passwordless to host ul SSH daemon over local ULA IPv6 with strict known_hosts checking
exec ssh -o StrictHostKeyChecking=no          -o UserKnownHostsFile=/dev/nullyes          -i "${KEY_PATH}"          "${TARGET_USER}@fd77:7777:5010:0:1337:f1ac:d00d:337" "$@"

6. Step 4: Authentik Single Sign-On Provisioning (ak shell)

To configure Authentik ForwardAuth without encountering 500 Internal Server Error, Redirect URI Error, or ERR_TOO_MANY_REDIRECTS, execute the following Python provisioning script in authentik-server:

docker exec -i authentik-server ak shell << 'EOF'
from authentik.core.models import Application
from authentik.providers.proxy.models import ProxyProvider, ProxyMode
from authentik.providers.oauth2.models import RedirectURI, RedirectURIMatchingMode, RedirectURIType
from authentik.outposts.models import Outpost

SERVICE_NAME = "webshell"
EXTERNAL_HOST = f"https://{SERVICE_NAME}.drheck.dev"

# 1. Inherit default scope property mappings from a working provider template
template_provider = ProxyProvider.objects.get(name="cockpit")

# 2. Create ProxyProvider with complete OAuth, redirect URI, and cookie settings
provider, _ = ProxyProvider.objects.get_or_create(
    name=SERVICE_NAME,
    defaults={
        "external_host": EXTERNAL_HOST,
        "mode": ProxyMode.FORWARD_SINGLE,
        "authorization_flow_id": "f9ad5326-f668-4d1f-861d-43b47f0a002e",
        "cookie_domain": "drheck.dev",
        "grant_types": ["authorization_code", "client_credentials", "password"],
        "redirect_uris": [
            RedirectURI(
                matching_mode=RedirectURIMatchingMode.STRICT,
                url=f"{EXTERNAL_HOST}/outpost.goauthentik.io/callback?X-authentik-auth-callback=true",
                redirect_uri_type=RedirectURIType.AUTHORIZATION
            ),
            RedirectURI(
                matching_mode=RedirectURIMatchingMode.STRICT,
                url=f"{EXTERNAL_HOST}?X-authentik-auth-callback=true",
                redirect_uri_type=RedirectURIType.AUTHORIZATION
            ),
        ]
    }
)

provider.property_mappings.set(template_provider.property_mappings.all())
provider.save()

# 3. Create Application
app, _ = Application.objects.get_or_create(
    slug=SERVICE_NAME,
    defaults={
        "name": SERVICE_NAME.capitalize(),
        "meta_launch_url": f"{EXTERNAL_HOST}/",
        "open_in_new_tab": True,
        "provider": provider,
    }
)

# 4. Mandatory: Assign provider to authentik Embedded Outpost
outpost = Outpost.objects.get(name="authentik Embedded Outpost")
if provider not in outpost.providers.all():
    outpost.providers.add(provider)
    outpost.save()

print(f"Successfully provisioned {SERVICE_NAME} and attached to Embedded Outpost.")
EOF

7. Step 5: Nginx Reverse Proxy Configuration (/etc/nginx/sites-available/webshell)

map $http_upgrade $proxy_target_connection_webshell {
    default upgrade;
    ""      "";
}

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    http2 on;
    server_name webshell.drheck.dev;

    ssl_certificate     /etc/nginx/certs/fullchain.pem;
    ssl_certificate_key /etc/nginx/certs/privkey.pem;

    client_max_body_size 64M;

    # Authentik ForwardAuth Subrequest Configuration
    auth_request     /outpost.goauthentik.io/auth/nginx;
    error_page       401 = @goauthentik_proxy_signin;
    auth_request_set $auth_cookie $upstream_http_set_cookie;
    add_header       Set-Cookie $auth_cookie;
    auth_request_set $authentik_username $upstream_http_x_authentik_username;
    auth_request_set $authentik_email $upstream_http_x_authentik_email;

    location / {
        proxy_pass http://[fd77:7777:5010:0:1337:f1ac:d00d:771d]:7681;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $proxy_target_connection_webshell;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Authentik-Username $authentik_username;
        proxy_set_header X-Authentik-Email $authentik_email;
    }

    location /outpost.goauthentik.io {
        proxy_pass              https://auth.drheck.dev/outpost.goauthentik.io;
        proxy_set_header        Host auth.drheck.dev;
        proxy_set_header        X-Original-URL $scheme://$http_host$request_uri;
        proxy_set_header        X-Forwarded-Host $http_host;
        proxy_set_header        X-Forwarded-Proto $scheme;
        add_header              Set-Cookie $auth_cookie;
        auth_request_set        $auth_cookie $upstream_http_set_cookie;
        proxy_pass_request_body off;
        proxy_set_header        Content-Length "";
    }

    location @goauthentik_proxy_signin {
        internal;
        add_header Set-Cookie $auth_cookie;
        return 302 /outpost.goauthentik.io/start?rd=$scheme://$http_host$request_uri;
    }
}

8. Step 6: Backup & Restore Scripts

Every service on host ul requires shellcheck-verified <service>_backup and <service>_restore scripts located in /home/luke/git/backups/ul/scripts/ with 10% par2 redundancy.

  • /home/luke/git/backups/ul/scripts/webshell_backup: Archives /data/docker/webshell/ (.env, docker-compose.yml, step-ca/) to /data/backups/services/webshell/YYYY-MM-DD_HHMM.SS/webshell_config.tar.xz and generates .par2 files via par2 create -r10.
  • /home/luke/git/backups/ul/scripts/webshell_restore: Verifies archives with par2 verify and extracts configurations back into /data/docker/webshell/.

9. Initial Assumptions & Attempts vs. Actual Working Configuration

Technical Area Initial Assumption / Attempt Actual Working Configuration & Root Cause Prevention Standard Outpost Assignment (500 Internal Server Error) Creating ProxyProvider in Authentik automatically activates ForwardAuth routing. Creating a provider does not assign it to an Outpost. Unassigned hostnames return 404 Not Found from /outpost.goauthentik.io/auth/nginx, which Nginx converts to 500 Internal Server Error. Explicitly attach providers to authentik Embedded Outpost (outpost.providers.add(provider)) before enabling Nginx traffic. Redirect URIs (Redirect URI Error) Instantiating ProxyProvider sets default authorization callback URLs. The Django ORM defaults redirect_uris to []. Authentik rejects callback URIs against an empty database list. Explicitly define RedirectURI objects for both /outpost.goauthentik.io/callback and root / in ProxyProvider.redirect_uris. Cookie Domain & Scopes (ERR_TOO_MANY_REDIRECTS) Providers operate without defining cookie_domain, grant_types, or property_mappings. Without property_mappings, the OAuth request omits critical scopes (ak_proxy). Without explicit cookie_domain='drheck.dev', Authentik refuses to grant ForwardAuth proxy tokens, causing an endless redirect loop. Clone property_mappings from cockpit and explicitly set cookie_domain='drheck.dev' and grant_types. CGI Env Mapping (Missing Authentik identity) ttyd -H X-Authentik-Username exports X_AUTHENTIK_USERNAME directly to child scripts. ttyd transforms HTTP headers into standard CGI environment variables (HTTP_X_AUTHENTIK_USERNAME). Always check CGI-style fallback variations: TARGET_USER="${X_AUTHENTIK_USERNAME:-${TTYD_USER:-${HTTP_X_AUTHENTIK_USERNAME:-${AUTHENTIK_USERNAME:-}}}}". IPv6 Dual-Stack Listening (502 Bad Gateway) Default ttyd invocation listens on all interfaces when connected to an IPv6 Docker network. Without -6 -i ::, ttyd binds only to IPv4 0.0.0.0. When Docker Engine forwards published IPv6 ports to the container's IPv6 address, Nginx connections are rejected (Connection refused). Always pass "-6", "-i", "::" to ttyd in Dockerfile so it listens in dual-stack mode. Container Script Updates (old script execution) Modifying login-wrapper.sh on host ul and running docker compose restart runs the updated script. docker compose restart only restarts containers without re-reading COPYed image files. Always bind-mount operational scripts (./login-wrapper.sh:/usr/local/bin/login-wrapper.sh:ro) in docker-compose.yml. CA Secret Privilege Restrictions Administrative or personal user accounts own the CA keys and JWK provisioner secret. Standard unprivileged or administrative (wheel) user accounts should not have read access to CA private keys or provisioner passwords. Assign ownership of /data/docker/webshell/step-ca to unprivileged system user stepca:stepca (945:945, /usr/bin/nologin) and run step-ca container under user 945:945. SSH Client Host Warnings (Warning: Permanently added...) Invoking ssh with /dev/null as known_hosts prints key addition warnings on every login. Connecting without a populated known_hosts file causes OpenSSH to log a warning to stderr whenever an ephemeral host entry is added. Mount a static /etc/ssh/ssh_known_hosts file generated from /etc/ssh/ssh_host_ed25519_key.pub and enforce StrictHostKeyChecking=yes. Cockpit Login MOTD Banner (Web console: https://ul...:9090) Cockpit's MOTD issue updater (update-issue) injects :9090 URLs into /etc/motd.d/. Cockpit's cockpit-issue.service inspects local interfaces and writes direct port URLs to /run/cockpit/issue, which /etc/motd.d/cockpit symlinks into login greetings. Remove /etc/motd.d/cockpit and /etc/issue.d/cockpit.issue symlinks and mask cockpit-issue.service (systemctl mask cockpit-issue.service). Dynamic Host UID Verification Hardcoding system username lists (`root admin ...`) in wrapper scripts to block system accounts. Web-Session-Only Login Banner Printing greetings via script echo or editing server-wide /etc/motd. Script echo output is lost before terminal attachment, while /etc/motd changes affect standard SSH logins. Configure Match Address fd77:7777:5010:771d::/64 in /etc/ssh/sshd_config.d/10-drheck-certs.conf with Banner /etc/ssh/webshell_banner.txt.