Learning from failure why a total site outage can be a good thing

"Although an outage is a terrifying prospect, you should embrace it as an opportunity. Failure can expand and improve your understanding of your systems. Three years ago, Indeed suffered one of the worst outages in its history. No single fault or failure caused this outage. Rather, it was a com...

Full description

Bibliographic Details
Main Author: Elman, Alex
Format: eBook
Language:English
Published: [Place of publication not identified] O'Reilly Media 2019
Subjects:
Online Access:
Collection: O'Reilly - Collection details see MPG.ReNa
LEADER 02996nmm a2200313 u 4500
001 EB001930523
003 EBX01000000000000001093425
005 00000000000000.0
007 cr|||||||||||||||||||||
008 210123 ||| eng
050 4 |a TK5105.888 
100 1 |a Elman, Alex 
245 0 0 |a Learning from failure  |b why a total site outage can be a good thing  |c Alex Elman 
260 |a [Place of publication not identified]  |b O'Reilly Media  |c 2019 
300 |a 1 streaming video file (34 min., 16 sec.) 
653 |a Computer system failures / fast / (OCoLC)fst00872650 
653 |a Pannes système (Informatique) 
653 |a Computer software / Development / http://id.loc.gov/authorities/subjects/sh85029535 
653 |a Computer system failures / http://id.loc.gov/authorities/subjects/sh98002500 
653 |a Web site development / http://id.loc.gov/authorities/subjects/sh98004795 
653 |a Sites Web / Développement 
653 |a Computer software / Development / fast / (OCoLC)fst00872537 
653 |a Web site development / fast / (OCoLC)fst01173243 
041 0 7 |a eng  |2 ISO 639-2 
989 |b OREILLY  |a O'Reilly 
500 |a Title from title screen (viewed December 12, 2019) 
856 4 0 |u https://learning.oreilly.com/videos/~/0636920338451/?ar  |x Verlag  |3 Volltext 
082 0 |a 000 
520 |a "Although an outage is a terrifying prospect, you should embrace it as an opportunity. Failure can expand and improve your understanding of your systems. Three years ago, Indeed suffered one of the worst outages in its history. No single fault or failure caused this outage. Rather, it was a complex interaction of bugs, design decisions, capacity loss, and poor situational awareness during incident response. Indeed learned valuable lessons from this event. It identified ways to make the systems more resilient and improved the approach to the incident lifecycle within the engineering culture. Alex Elman uses the narrative of this incident to demonstrate how a site-wide outage can inform increased resilience and reduced operational complexity. Learning from failure is a feedback loop rather than a one-off process. He applies Indeed's outage as a practical example of what an iteration of this loop can look like. He shares with other SREs the success that has risen from this failure. Indeed hasn't had a global site outage in the three years since this event. Alex begins with a discussion of failure to set the stage for delivering the incident background, then discusses incident response and situational awareness. He explains conducting incident postmortems and learning from failure and designing for reliability, including resilience patterns such as circuit breaking and graceful degradation. Finally, he gets into resilience testing, running chaos tests, and closing the feedback loop, leaving some time for a question and answer session. This session was recorded at the 2019 O'Reilly Velocity Conference in San Jose."--Resource description page