SSH CA setup

OVERVIEW

• Authentication Flow: Incoming SSH clients authenticate using an OpenSSH user certificate signed by the trusted DrHeckserver's User Certificate Authority (CA).
• LDAP & Authentik Integration: Host ul derives user accounts from Authentik via LDAP (managed via SSSD). When a client presents an SSH certificate, sshd validates the certificate signature against the trusted DrHeck CA public key and checks that one of the certificate's valid principals matches the requested LDAP login username. No per-user ~/.ssh/authorized_keys file is required.
• Key Storage & Backup: *
  • The User CA private key (drheck_ca_user_key) and Host CA private key (drheck_ca_host_key) must never reside directly on client machines or internet-facing servers (such as ul).
  *
• Store the offline CA private keys and passphrases securely in Vaultwarden.

* Deploy only the CA public keys (drheck_ca_user_key.pub and drheck_ca_host_key.pub) to host ul and client machines. Key Inventory & Revocation:

Every signed public key and certificate is automatically archived in /etc/ssh/ca_inventory/ with a descriptive label and fingerprint. Any compromised key can be revoked immediately using sudo revoke-key <label>.

---

PHASE 1: CA GENERATION & VAULTWARDEN BACKUP

Generate CA Keypairs on a secure offline or admin workstation:

When prompted by ssh-keygen, enter a strong, unique passphrase to encrypt each CA private key.

# Generate the User CA keypair
ssh-keygen -t ed25519 -f drheck_ca_user_key -C "drheck-user-ca"

# Generate the Host CA keypair
ssh-keygen -t ed25519 -f drheck_ca_host_key -C "drheck-host-ca"

Secure Storage in Vaultwarden:

1. Create a secure note or attachment in Vaultwarden containing the private keys (drheck_ca_user_key, drheck_ca_host_key) and their passphrases.
2. Remove the private CA keys from the unencrypted local storage of your workstation after verifying the backup.

---

PHASE 2: MAIN SERVER CONFIGURATION (ul)

1. Deploy Trusted User CA Public Key

• Copy the public CA file drheck_ca_user_key.pub to host ul at /etc/ssh/drheck_ca_user_key.pub.
• Set secure permissions:

  bash   sudo chmod 644 /etc/ssh/drheck_ca_user_key.pub
  sudo chown root:root /etc/ssh/drheck_ca_user_key.pub
  

2. Configure Host Certificate (Signed by Host CA)

To keep the private drheck_ca_host_key offline, the host key must be signed on your secure workstation:workstation or via the signing script:

1. Copy ul's public host key /etc/ssh/ssh_host_ed25519_key.pub to your secure workstation.
2. On your workstation, sign the host key:

ssh-keygen -s drheck_ca_host_key -I "drheck-ul-host-cert" -h -n ul,ul.drheck.dev -V +52w ssh_host_ed25519_key.pub

3. Copy the resulting certificate ssh_host_ed25519_key-cert.pub back to ul at /etc/ssh/ssh_host_ed25519_key-cert.pub.

4. Set secure permissions on ul:

sudo chmod 644 /etc/ssh/ssh_host_ed25519_key-cert.pub
sudo chown root:root /etc/ssh/ssh_host_ed25519_key-cert.pub

3. Configure OpenSSH Daemon on ul

• Create the config file /etc/ssh/sshd_config.d/10-drheck-certs.conf

:

sudo vim /etc/ssh/sshd_config.d/10-drheck-certs.conf

• Add the following directives to trust incoming User CA certificates and deploy the host certificate:

TrustedUserCAKeys /etc/ssh/drheck_ca_user_key.pub
HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub

4. Validate and Reload SSHD on ul

• Validate the configuration syntax before reloading:

sudo sshd -t

• If no errors are returned, reload the daemon:

sudo systemctl reload sshd

---

PHASE 3: CLIENTSTREAMLINED MACHINE ONBOARDING &REMOTE SIGNING (One-Liner Pipelines)

To eliminate manual scp, sudo mv, and context switching, helper utilities on host ul (/usr/local/bin/sign-host-key and /usr/local/bin/sign-user-key) accept public keys on stdin and stream the signed certificate directly to stdout.

1. ClientUnlock KeyVaultwarden Generation(Run Once per Onboarding Session on ul)

Log

Oninto ul and unlock the Linuxakadmin clientVaultwarden machine,vault generate(keeps acredentials standardin SSHRAM keypairfor if1 one doesn't exist: hour):

ssh-keygensudo -tiu ed25519akadmin -fvw ~/.ssh/id_ed25519 -C "client key label"unlock

2. CertificateStream Signing& RequestSign New Host Key (Run on New Host)

Transfer only

From the machine being onboarded, pipe the host's public key (~/.ssh/id_ed25519.pub)directly andover theSSH target LDAP username to the DrHeck CA administrator.

3. Sign the Client Public Key

On the secure workstation, sign the client public key using the vaultedinto drheck_ca_user_keysign-host-key (defining the exact LDAP username as the principal):

ssh-keygencat /etc/ssh/ssh_host_ed25519_key.pub | ssh -st drheck_ca_user_key -Iul "client-hostname-YYYYMMDD"sudo -nsign-host-key <ldap_usernamehostname> -V[principals]" +52w| id_ed25519.sudo tee /etc/ssh/ssh_host_ed25519_key-cert.pub

• Interactive Label Prompt: If run with ssh -It: Certificate identity string printed in, the server'sscript authenticationprompts logs.on your terminal: Enter inventory label for this host [<hostname>]:
• Automated Label Flag: You can also pass -n-label <name>: Validor principals;-l must<name> exactlyto matchskip the targetprompt: LDAP

  cat username/etc/ssh/ssh_host_ed25519_key.pub | ssh ul "sudo sign-host-key bmo95 -l studio-pc" | sudo tee /etc/ssh/ssh_host_ed25519_key-cert.pub
  

Set secure permissions on ul.the new host:

-Vsudo +52wchmod 644 /etc/ssh/ssh_host_ed25519_key-cert.pub
:

Certificate validity duration (e.g., 52 weeks or shorter).

4.3. DeployStream the& Sign User Key (Run on Client CertificateWorkstation)

From

Transfer the generated certificate id_ed25519-cert.pub back to theyour client machineworkstation, andpipe placeyour ituser inpublic key directly into sign-user-key:

cat ~/.ssh/id_ed25519.pub | ssh -t ul "sudo sign-user-key <ldap_username>" > ~/.ssh/id_ed25519-cert.pub
(alongside the private key).

Set file permissions on the client:

chmod 644 ~/.ssh/id_ed25519-cert.pub

5.

Security Client& SSHLifecycle Configuration (~/.ssh/config)

Configure the host entry on the client machine to use the cert automatically:

Host ul
    HostName <ul_fqdn_or_ipv6>
    User <ldap_username>
    IdentityFile ~/.ssh/id_ed25519

  (Note: OpenSSH automatically detects and presents id_ed25519-cert.pub when utilizing id_ed25519 if they are in the same folder).Details:

• The public key is piped directly into a temporary /dev/shm RAM filesystem on Trustingul.

The CA private key and passphrase are decrypted from Vaultwarden into /dev/shm, piped via SSH_ASKPASS, and securely shredded upon command exit (trap ... shred -u). The private keys never touch physical disk storage on ul. A copy of the Hostpublic CA:key Addand thissigned linecertificate toare automatically preserved in /etc/ssh/ssh_known_hostsca_inventory/ (system-wide)for orauditing ~/.ssh/known_hostsand on your client machines. This tells the clients to trust ul's host certificate automatically without prompting about finger-prints:revocation.

@cert-authority * ssh-ed25519 <drheck_ca_host_key.pub_content>

---

PHASE 4: AUDITING & KEY REVOCATION (KRL)

1. View Inventory of Active & Revoked Keys

To see all signed hosts and user keys:

revoke-key --list

Example Output:

==========================================================================================
LABEL                TYPE     FINGERPRINT               STATUS      
------------------------------------------------------------------------------------------
danknasty            host     SHA256:oj796ZOeYmi6hAm+pG [ACTIVE]     
ul                   host     SHA256:1a4QDcX1p/QT4qbFdc [ACTIVE]     
borrowbook           user     SHA256:HPu6XQUlARHnLWgMWo [ACTIVE]     
metropolis           user     SHA256:22OChYHL9QxAe57Noe [ACTIVE]     
==========================================================================================

2. Revoke a Compromised Key

To immediately invalidate any key or certificate:

sudo revoke-key <label>

What this does:

Locates the archived public key in /etc/ssh/ca_inventory/. Adds the key to OpenSSH's Key Revocation List (KRL) at /etc/ssh/revoked_keys. Ensures RevokedKeys /etc/ssh/revoked_keys is configured in /etc/ssh/sshd_config.d/10-drheck-certs.conf. Automatically reloads sshd. Any future connection attempts with that key or certificate are instantly rejected.

---

PHASE 5: VERIFICATION & MIGRATION

1. Test Certificate Authentication

• Connect from the client machine with verbose debug output:

ssh -v ul

• Confirm that the client authenticates using the certificate signature:

Server accepts key: pkalg ssh-ed25519-cert-v01@openssh.com

2. Client Host Verification (~/.ssh/known_hosts)

Add the Host CA public key to /etc/ssh/ssh_known_hosts (system-wide) or ~/.ssh/known_hosts on all client machines:

@cert-authority * ssh-ed25519 <drheck_ca_host_key.pub_content>

Clients will now connect to any host bearing a valid DrHeck host certificate without TOFU ("Trust On First Use") prompts or host key verification warnings.

3. Migration Strategy (Disabling Raw Keys)

• Keep PubkeyAuthentication yes active on ul to allow standard .ssh/authorized_keys logins during client onboarding.
• OnceTo allenforce clientscertificates arefrom verifiedthe andexternal migrated,internet disablewhile legacykeeping .ssh/authorized_keys filebootstrapping lookupsavailable system-wideon byyour addingtrusted theLAN: following

  # line to /etc/ssh/sshd_config.d/10-drheck-certs.conf:

AuthorizedKeysFile /dev/null

Match Address 10.0.0.0/24,fd77:7777:5010::/64
    AuthorizedKeysFile .ssh/authorized_keys

(This keeps standard certificate authentication fully active, but prevents users from logging in via traditional uncertified public keys).