Disruptions on BHN AWS EKS Clusters impacted Multiple BHN Platforms

Incident Report for Tango Card

Resolved

Resolution: BHN has successfully resolved the issue. Our team collaborated closely with AWS to address the impact caused by AWS EKS Clusters. We are pleased to confirm that the system has fully recovered as of 12:00 PM PT. We will now initiate the Root Cause Analysis (RCA) process to prevent future occurrences.

This issue impacted Tango Platforms where Tango Orders were failing (Tango Portal and Tango API). This is resolved now.

Posted Mar 17, 2025 - 09:07 PDT

Investigating

Business Impact: BHN was experiencing EKS issues in the AWS service that impacted the following systems:
1. Japan Region:
• Lawson -> LoadIt flow (Google)
o Pre-Authorizations (1032)
o Shipments (182)
o PSLOADIT (571)
o GMGTOP (242)

2. US Region:
• Giftcards.com - D2C
• Tango - B2B
• Rakuten - Affiliate orders

Posted Mar 17, 2025 - 09:07 PDT

This incident affected: RaaS API v2 - Production, Rewards Genius - Production, and Reward Link - Production.