rollback

And a rollback is just a git revert and another push away -- no need to keep build artifacts lying around if you believe your build is deterministic.

Then you're stuck trying to rollback, and then spend days fixing the other service.

The docker container tagging would also avoid the rollback limitations mentioned in the article.

No to mention the ones with serious bugs that enter an install-rollback loop that takes hours each time (like the awful KB 2919355).

Upgrades are very close to being fully atomic, and you can rollback arbitrarily through the boot menu in case a system change—an upgrade or configuration change—ruins something.

To rollback, I can just revert to the right commit and re-run ansible-playbook.

If I had the time/energy to try out new distros at home, I would probably try Nix or something similar, that emphasizes rollback capability.

Or write changed structures to disk in a secondary thread in the background, force structure syncs on write (if there are changes), and pull them from disk on rollback (as necessary).

A simple offline cryptographic signature (with proper verification) would have prevented this incident, but it would not allow key rotation and revocation or prevent rollback, freeze, mix-and-match, etc.

I've used apt pinning before though, and I'm quite sure rollback of a package update that has a dependency conflict with the old version is not as easy.

It's Chrome fault to use push for a new standardized header without properly checking its impact: the issue has been raised during the beta phase, but they underestimated it and did not rollback.

Create a transaction, Run expected counts etc then Run the delete, do counts or other queries to insure what happened was what I intended and then commit or rollback.