Resolved -
Root cause: Redis ran out of memory when a huge base64-encoded PDF got stuck in the document-delivery queue, pushing usage over 90%. The team temporarily upgraded the Redis node to a bigger instance, which cleared the alert, and then deleted the oversized queue key to free up about 1.2 GB.
Jun 18, 10:56 UTC
Monitoring -
A fix has been implemented and we are monitoring the results.
Jun 18, 10:51 UTC
Update -
We are continuing to work on a fix for this issue.
Jun 18, 10:42 UTC
Identified -
We're experiencing an elevated level of API errors due to cache issues. The team is working on solving the issue.
Jun 18, 10:30 UTC