One Login for a Two-Box Cloud

My entire cloud is two computers. A small VPS holding the front door, and a workstation in my office running k3s that does the actual work. You could carry the whole thing down a flight of stairs in one trip. And yet, at some point last year, this two-box empire had accumulated five separate credential systems — and every one of them was a different flavor of "I'll fix this properly later."

That is the embarrassing part of self-hosting nobody puts in their homelab screenshots. The Grafana dashboards are gorgeous. The ~/.kube directory is a crime scene.

The inventory of shame

Before fixing it, I forced myself to write down what "logging in" actually meant across the stack. The list was worse than I expected:

System	Credential	Lifetime	Where it lived
Kubernetes	client cert inside a copied kubeconfig	~forever	three laptops, one of them retired
MinIO (S3)	static access key + secret	forever	mc alias, two .env files
Vault	a token from vault login	renewed by hand	shell history, twice
Harbor	robot account password	forever	~/.docker/config.json, plaintext
Postgres, misc	passwords	forever	.env files of every project

Five systems, five revocation stories, and — if I'm honest — zero revocation practice. If a laptop had walked away, my realistic recovery plan was "rotate everything and lose a weekend." For a platform whose whole reason to exist is that I control it, that's not control. That's just proximity.

The deeper problem isn't any single credential. It's that every static secret is a small mortgage: it costs nothing the day you copy it, and then it charges interest forever — every machine it touches, every backup it slips into, every year it silently doesn't expire.

Steal the idea, skip the invoice

The thing is, this problem is already solved. Anyone who has worked on AWS knows the shape of the solution: you type aws sso login, a browser tab opens, you prove who you are once, and then every service credential you touch for the rest of the day is derived from that session — short-lived, scoped, and revocable at the identity provider. There is no long-lived secret sitting on your disk that's worth stealing.

AWS didn't invent magic. They composed three boring, open standards: an identity provider, OIDC tokens, and per-service token exchange. All of those run happily on a workstation in Dak Lak.

So the design goal became a single sentence:

One human login per day; everything else is derived, short-lived, and revocable in one place.

The pieces:

• Keycloak as the identity provider — the only place a password exists.
• One CLI — I called it anh, which is either my name or Vietnamese for "big brother," depending on how the day is going — that performs the login and brokers credentials for everything else.
• The rule that makes it work: the CLI never mints anything. It only exchanges. Every downstream system already speaks OIDC or has a token-exchange endpoint; the CLI's whole job is to walk up to each of them with a fresh identity token and ask politely.

The broker

anh login runs a standard authorization-code flow with PKCE: it opens the browser, Keycloak does the actual authentication, and the CLI catches the redirect on localhost. What lands on disk is a session file with a refresh token — 0600, short-lived, and useless to anyone who can't also reach my identity provider as me.

From there, each subsystem is one subcommand that trades the OIDC token for whatever that system natively understands.

Kubernetes, without copying kubeconfigs

The old ritual — SSH to the server, copy /etc/rancher/k3s/k3s.yaml, hand-edit the server URL, paste the blob into another laptop — is the exact anti-pattern EKS killed with DescribeCluster. You don't copy an EKS kubeconfig from anywhere; you ask the API "describe this cluster," it hands you the endpoint and the CA, and your kubeconfig is generated.

So I built the ten-line version of DescribeCluster: a tiny endpoint that vends exactly two facts — the apiserver URL and the cluster CA. Neither is secret (the CA lives in the kube-root-ca.crt ConfigMap that every pod already sees). What was missing was a place to ask.

$ anh kube write-config
✓ session valid (expires in 9h)
✓ cluster-info: apiserver + CA fetched
✓ wrote ~/.kube/config (context: anhhoang)

The generated kubeconfig contains no credential at all — just an exec plugin:

users:
  - name: anhhoang
    user:
      exec:
        apiVersion: client.authentication.k8s.io/v1
        command: anh
        args: ["kube", "token"]

Every kubectl call shells out to anh kube token, which returns the current OIDC token from the cached session. The k3s apiserver is started with the --oidc-* flags, so it validates the token against Keycloak directly and maps groups to RBAC. Onboarding a new machine stopped being an afternoon and became: install the CLI, run two commands, done. Revoking a machine is one click in Keycloak — the exec plugin just starts failing.

Object storage, without access keys

MinIO has quietly shipped one of the most useful features in self-hosted infrastructure: an STS endpoint. AssumeRoleWithWebIdentity accepts an OIDC token and returns temporary S3 credentials, with the policy chosen by a claim in the token:

$ anh mc login
✓ exchanged OIDC token via STS (AssumeRoleWithWebIdentity)
✓ temporary credentials valid 1h, policy: dev-readwrite
✓ mc alias 'anh' updated

The static access keys that used to live in .env files are gone. What remains expires within the hour, and the policy — which buckets, which prefixes — is decided by group membership in Keycloak, not by which key happened to get pasted where.

Vault and the registry

The same move, twice more. Vault's JWT auth method logs in with the OIDC token and returns a Vault token whose TTL and policies come from the token's claims — anh vault login wraps it. Harbor federates to Keycloak for the humans, and the CI robot accounts that remain are scoped to a single project and rotated, because "CI can push one image repo" is a very different blast radius from "this password is root everywhere."

None of these integrations is longer than a screen of code. The work was never the code. The work was deciding that one system owns identity, and then refusing every shortcut that would quietly create a second one.

A personal cloud earns the right to stay small only if logging into it is even smaller.

The sharp edges

Two problems bit me hard enough to be worth writing down.

Concurrent refresh. Keycloak rotates refresh tokens: use one, get a new one, the old one dies. Now imagine kubectl firing five exec-plugin calls in parallel while mc wakes up in another terminal — every process sees an expired access token, every process races to refresh, one wins, and the winners' new refresh token gets clobbered by a loser writing its stale result back to the session file. Congratulations: you've been logged out by your own success. The fix is embarrassingly old-fashioned: an flock around the session file, so exactly one process refreshes and everyone else reads the result. Distributed systems humility, delivered at the scale of one laptop.

Logout must revoke. The first version of anh logout deleted the session file and called it a day. That's not logout, that's littering — the refresh token was still perfectly alive server-side, just no longer on my disk, which is strictly worse than knowing where it is. Logout now calls Keycloak's revocation endpoint first and deletes the file second. If the revocation fails, it says so loudly instead of pretending.

Both bugs share a lesson I keep re-learning: in identity systems, the unhappy paths are the product. Anyone can build the flow that works when one process logs in once. The system earns its keep at 2 AM, when three terminals and a cron job all disagree about whose token is freshest.

What it feels like now

The morning ritual is one line:

$ anh login
✓ authenticated as anh (session: 10h)

After that, kubectl, mc, vault, and docker push simply work — each one silently exchanging the session for its own short-lived credential, none of them holding anything worth stealing. The inventory of shame now has one row, and that row expires by dinner.

Was it over-engineering for two machines? I've thought about that a lot, and I've landed on no — for a reason that has nothing to do with security. Friction decides what you build. When touching the cluster required archaeology on old kubeconfigs, I batched my ideas and lost most of them in the queue. Now the distance between "I wonder if—" and a running pod is one login that was already done at breakfast. The platform got smaller the day its front door did.

That, and the quieter reason: this is my infrastructure. Nobody audits it, nobody makes me rotate anything, nobody would ever know about the plaintext robot password. Building it right anyway — at 5 AM, for an audience of exactly one — is the whole discipline. You don't rise to the security posture of your ambitions; you fall to the standard of your defaults.

So I fixed the defaults.