Production Outage

Adding sanitized customer data helped the team catch more defects before production. A shipped app update caused a production outage after QA concerns were overridden. The outage occurred because the bug appeared only with real customer da…

1 sources - 6 claims

Adding sanitized customer data helped the team catch more defects before production. A shipped app update caused a production outage after QA concerns were overridden. The outage occurred because the bug appeared only with real customer data, not in the test environment. The incident led to sanitized customer data being added to final testing. The outage damaged credibility in the short term and required emergency remediation. The incident resulted in QA receiving unconditional veto authority over releases.