Post-mortem: Legacy PlayReady DRM Service Outage
Date: 2/24/2024
Impact: This incident caused an outage for both OneView and the Legacy PlayReady DRM service.
Root Cause: A typographical error (typo) was identified in a connection string value that was deployed in preparation to migrating a supported database to SQL 2019. This typo prevented the affected services from establishing necessary connections to the database before and after the migration.
Resolution: The typo was corrected, and connectivity was restored for both services.
Timeline:
- [8:00 am]: SQL Migration work begins. Swank Technical team reviewing logs of affected services.
- [8:15 am]: Errors related to Legacy PlayReady DRM found (prior and after migration).
- [8:30 am]: Error traced to typographical error in connection string and repaired. Service affected confirmed.
- [8:35 am]: Legacy PlayReady DRM service issues reported to Swank Digital Support.
- [9:00 am]: Affected customer reports back that service is working at their location.
Lessons Learned:
- Highlighted a gap in our Legacy monitoring for this service.
Next Steps:
- Add additional monitoring to the legacy service that includes database calls
Comments
0 comments
Please sign in to leave a comment.