Internal tools tend to rot the moment they leave the founding team. We rebuilt our admin from scratch around three principles: every action should be auditable, nothing should require a tribal-knowledge runbook, and there should be exactly one button that resolves the on-call page.
Five years ago this would have been a controversial post. Today we have battle-tested operators, decent storage classes and a community that has finally agreed on backup tooling. Here is the stack we landed on for a fleet of around forty clusters.
After two years of running a custom message broker we finally pulled the plug and migrated everything to NATS JetStream. The migration took six weeks of careful planning, three weekends of late-night cutovers and one production incident that we are still apologising for.
Sometimes the right answer is not Hugo or Astro but a file you can read in one sitting. We walked through every line in a guild lunch and somehow shipped a new docs site by the end of the day.
Most database migrations fail not because the SQL is wrong but because nobody tested them against a realistic dataset. We share the rollout checklist we use for every schema change, including the awkward question of when to lock a table and when to live with the inconsistency.