Lessons learned writing highly available code

原文地址

• Put limits on everything.
• Retry, but with exponential back-off.
• Use supervisors and watchdog processes.
• Add health checks, and use them to re-route requests or automate rollbacks.
  • As a developer, think about how to boil all the variables around your system into a single boolean “is this thing healthy? Is it working?”
• Redundancy is more than just nice-to-have, it’s a requirement.
• Prefer battle-tested tools over the “new hotness”.
  • It can be tempting to reach for the newest tools that promise big new features.
  • New technologies frequently have unknown-to-you and unknown-to-them modes of failure.
  • Newer tools also tend to have immature facilities for running in production.