ProjectsSelf-hosted production
The sites and applications I ship, and the infrastructure I design to serve them: the overall diagram, the detailed decisions, the patterns, the operations log and what comes next.
Work: sites & applications in production
The diagram, as a clickable logical view
Blocks marked with this arrow are clickable: each opens its decision record, with the need, the options, the choice and the tradeoff.
Case studies, by domain
Patterns & technical write-ups
Operations log
Real incidents from the platform, in short form: the symptom, the diagnosis, the fix, and what each one changed.
June 19, 2026 · Provider outage: server unreachable
Symptom. Friday June 19th, midday: no domain name resolves on the tailnet devices anymore, even though the Internet connection is fine. Immediate cause: those devices use the platform’s internal DNS as their resolver, and the server hosting it no longer responds.
Diagnosis. Connection fine but resolution silent: the problem is on the server side. Quick confirmation from the provider: a cooling incident in the Roubaix datacenter, announced on OVHcloud’s status page (public source). About two and a half hours of downtime in practice for this server; full resolution of the incident took about twelve hours for the last affected servers.
Fix. No lever during the incident: the failure is physical, at the provider. The useful work happens before and after.
What it changed. Two lessons. The DNS dependency first: making the server the resolver for all tailnet devices turns a provider outage into a general resolution outage on every device. Fixed since: a fallback resolver is declared in the tailnet DNS configuration, so devices keep resolving if the server stops responding. The single server next: the risk is known and accepted at this scale, and a multi-server infrastructure remains a deliberate learning goal (the beyond the single server project). Changing providers would protect from nothing: this kind of incident can happen anywhere.
June 1, 2026 · An automatic update cuts access to all private services
Symptom. Every private service answers 403: password vault, internal tools, dashboards. Public sites work normally. Discovered two days after the incident began, by noticing that the password vault no longer synced: tailnet services are not used every day, so the outage stayed silent over a weekend.
Diagnosis (1.5 hours). The reverse proxy logs show a source address in
172.x, the Docker network gateway, instead of the client’s tailnet address.
The tailscale-only filter, which only allows the private network,
therefore rejects everyone. Tracing back: Tailscale had auto-updated a few
days earlier, and the new version introduces connection-marking rules that,
combined with the existing masquerading, rewrite the source of packets
coming in through the tailnet before they reach the containers.
Fix. Persistently disable SNAT for local routes on the Tailscale side, with no side effect here since this node routes no subnet.
What it changed. Two days of silent incident on half the platform, detected by a lucky usage check: this is precisely what the observability and alerting roadmap project must make impossible. A complementary lesson: an automatic update can change low-level network behavior; for network components, read the changelog before updating, and pin the version if needed.